Please call me at 408 462 5912 to discuss If you are qualified for this position & submit your resume in Word doc at prasad@cognateinc.com along with a brief cover letter and full contact info
Position: Big Data Engineer + SCALA
Location: San Francisco, CA
Duration: 12 Months
Job Description:
- Build scalable data pipelines on top of Hive and Spark using the Airflow scheduler/executor framework
- Collaborate with Payments Engineering, Data Science, FP&A, Accounting, and Product Management to produce accurate and timely financial data
- Apply Machine Learning models to forecast and visualize key revenue metrics and trends
- Anomaly detection and analysis for payments integrations, processing, and processor reconciliation
- Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
- Craft data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
- Work with data and analytics experts to strive for greater functionality in our data systems.
Required Experience:
- Proficient understanding of distributed computing principles.
- Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming.
- ETL or Pipeline design/implementation with Large Distributed Databases (Hive/Spark/Scala preferred)
- Experience in data validation, sampling & regression tests.
- Pipeline stability and performance tuning.
- Working knowledge of relational databases and query authoring (SQL)
- Experience with Scala is Must.
- Love to use and develop open source technologies like Kafka, Hadoop, Hive, Presto, and Spark.
- Rigor in high code quality, automated testing, and other engineering standard methodologies.
- Payment data processing or system design knowledge is a plus (not required)
- Experience with NoSQL databases, such as HBase or Cassandra is a plus (not required)
- Outstanding interpersonal skills.
- BS/MS/PhD in Computer Science or a related field (ideal)
Prasad G | Sr. Recruiter | Cognate, Inc.
Digital & Cloud Apps Integration
www.cognateinc.com
How to Apply:
If you are qualified for this position, please submit your resume in Word format along with a brief cover letter to APPLY NOW through this site.
About Us:
Headquartered in the heart of Silicon Valley, Cognate Inc. was established in 2005 and is a leading IT staffing and IT Consulting Company. Our focus areas includes Staffing and Consulting for Cloud Computing, Data Analytics, Big Data, Enterprise Security & Risk Management, Mobile Applications.