SPARK/SCALA Developer
Plymouth, PA
Long-term Contract
· Good knowledge and Hands on Scala Programming
· Good skills on modular program development in Scala
· Proficiency in Spark Scala for technical development and implementation
· Data pipeline using Spark Scala
· Load disparate data sets by leveraging Kafka consumers
· Ability to utilize Hive, Spark, Cassandra, Mesos and Kafka
· Experience with AWS components and services, particularly EMR, S3, and Lambda
· Good understanding of file formats including JSON, Parquet, Avro, and others
· Strong development skills around Hadoop, Spark, MapReduce, and Hive
· Strong understanding of Hadoop internals
· Experience with open source technologies such as Cassandra
· Experience with messaging and complex event-processing systems such as Kafka and Storm
· Facilitate in job scheduling/monitoring
· Good knowledge on Spark concepts and Implementation
· Extensive hands on Data frame and Dataset operations of Spark
· Good Understanding of Bigdata Technologies and Hadoop
· Good knowledge on Hive and HiveQL
· Knowledge on Cassandra Architecture and CQL is desirable
· Good Hands on in Hadoop stack (MapReduce, Hive, Sqoop, Oozie)
· Excellent in writing shell scripts
· Experience with application architecture in a big data environment.
· Expert with Big Data technologies like Hadoop, Spark, Scala, Hive, HBase, Pig, Cascading
· Experience with AWS technologies or equivalent cloud stack As a hands-on engineer, influence all architecture decisions
· Build reusable code, with the ability to scale with very large data volumes
· Knowledge on Java Programming is desirable
· Data Migration/ETL knowledge is desirable