Big Data Engineer + SCALA

Please call me at 408 462 5912 to discuss If you are qualified for this position & submit your resume in Word doc at prasad@cognateinc.com along with a brief cover letter and full contact info

Position: Big Data Engineer + SCALA
Location: San Francisco, CA
Duration: 12 Months

Job Description:
  • Build scalable data pipelines on top of Hive and Spark using the Airflow scheduler/executor framework
  • Collaborate with Payments Engineering, Data Science, FP&A, Accounting, and Product Management to produce accurate and timely financial data
  • Apply Machine Learning models to forecast and visualize key revenue metrics and trends
  • Anomaly detection and analysis for payments integrations, processing, and processor reconciliation
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
  • Craft data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
  • Work with data and analytics experts to strive for greater functionality in our data systems.

Required Experience:
  •  Proficient understanding of distributed computing principles. 
  • Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming.
  • ETL or Pipeline design/implementation with Large Distributed Databases (Hive/Spark/Scala preferred)
  • Experience in data validation, sampling & regression tests.
  • Pipeline stability and performance tuning.
  • Working knowledge of relational databases and query authoring (SQL)
  • Experience with Scala is Must.
  • Love to use and develop open source technologies like Kafka, Hadoop, Hive, Presto, and Spark.
  • Rigor in high code quality, automated testing, and other engineering standard methodologies.
  • Payment data processing or system design knowledge is a plus (not required)
  • Experience with NoSQL databases, such as HBase or Cassandra is a plus (not required)
  • Outstanding interpersonal skills.
  • BS/MS/PhD in Computer Science or a related field (ideal)

Prasad G | Sr. Recruiter | Cognate, Inc.
Digital & Cloud Apps Integration
www.cognateinc.com

How to Apply:
If you are qualified for this position, please submit your resume in Word format along with a brief cover letter to APPLY NOW through this site.

About Us:
Headquartered in the heart of Silicon Valley, Cognate Inc. was established in 2005 and is a leading IT staffing and IT Consulting Company. Our focus areas includes Staffing and Consulting for Cloud Computing, Data Analytics, Big Data, Enterprise Security & Risk Management, Mobile Applications.


Want to apply later?

Type your email address below to receive a reminder

Apply to Job

ErrorRequired field
ErrorRequired field
ErrorRequired field
Error
Error
insert_drive_file
insert_drive_file