As Hadoop Developer in Risk & Finance Core Services, the ideal candidate for this developer role should be able to:
1.Build high performing data models on big-data architecture as data services.
2.Build high performing and scalable data pipeline platform using Hadoop, Apache Spark and object storage architecture.
3.Partner with Enterprise data teams such as Data Management & Insights and Enterprise Data Environment (Data Lake) and identify the best place to source the data
1.Experience in Hadoop ecosystem tools for real-time batch data ingestion, processing and provisioning such as Apache Flume, Apache Kafka, Apache Sqoop, Apache Flink, Apache Spark or Apache Storm
2.Java or Python experience
4.Design and development experience with columnar databases using Parquet or ORC file formats on Hadoop
5.-Apache Spark design and development experience using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs)
6.Deliver the data services on container-based architecture such as Kubernetes and Docker
7.ETL (Extract, Transform, Load) Programming experience
Hadoop Spark Apache and ETL