Vydia’s Data Engineering & Data Science efforts are an integral part of our success. Through our data, we nurture our artists, creators, partners, and internal users with insights to help them engage their audience.
Our Extract-Load-Transform workflows promote quick analysis and richer, complex investigations all at once, and our data warehouse supports both data science and analytics. Of all the terabytes of data we gather, one-third is well-structured, with the remaining being mostly semi-structured and some wholly unstructured, and it currently doubles every 5.3 months!
As a Sr. Data Engineer, you will own Vydia’s multitude of data pipelines. You will design and implement our ELT workflows, which originate at partner APIs created from scratch in Python, and conclude in the Redshift Warehouse. You will take the lead on our Airflow pipeline and Kubernetes deployment scheme. You will work closely with our data science, BI, and product teams in figuring out current and future needs.
As a leader on the Data team, your responsibilities will include:
- Ensuring the availability and timely delivery of data, company-wide
- Modeling new data sets and crafting all new ELT workflows and Airflow pipelines
- Lead the orchestration of the workflows and contribute strongly to infrastructure decisions
- Improving on and monitoring of existing pipelines and oversight of our ELT workflow
- Maintaining a single version of truth for our data and working with others to implement continuous integration (CI) data quality tests
- Mentoring and guiding your junior colleagues and leading with vision, and with respect to the company’s data strategy; implementing best practices with respect to Python coding and Kubernetes deployments
While we are not married to any tool or technology we also look for those intimately familiar with Python and have previous experience using Airflow, Docker and Kubernetes.
We use AWS S3 & EC2 extensively, our current DW is on Redshift and our app relies mostly on Postgres. We use Looker in-house for BI and Product Engineers work mostly in Ruby and Python, while our data scientists work in R.
We want to learn more about you. If you feel like you are the right fit for this role, let us know.
In our mind, being a perfect fit means that you have the necessary hard skills and expertise, and the complimentary soft skills.
- You are a Python pro and have regularly used AWS or Google Cloud Platform to manage data and move it between applications
- Do you love APIs? When you encounter a new one, do you study it inside and out and learn every corner of it, as though you designed it yourself?
- Working with deeply-nested, complex JSON is a fun day at the office for you.
- You can articulate the merits and pitfalls of the different approaches in designing a pipeline.
- You are passionate about data quality control and know how and where to anticipate potential errors.
- Working “in the cloud” is not a point of distinction for you, it is a given.
- You understand what it means to work at a tech startup. Hopefully, this is what excites you more than anything else about working here.
- You intuitively know how to extract value and insights from data.
- You love the idea of building the data scene in NJ and being a leader in this community.
- You have orchestrated workflows using Airflow and are familiar with the challenges and how to overcome them.
Critically, you're a person who thinks in data; you relate the real world to data, and vice-versa. You understand that data is not the end goal but a vehicle to help get us where we are going, and you see your role as the person most critical in making it happen.
Reasons to work with us:
- As an Inc 500 Fastest Growing Company in America Vydia offers huge opportunities to grow with the company.
- Vydia was named a Best Place to Work in NJ by NJBIZ in 2017 due to its collaborative, fast paced, and fun, thriving environment
- Full medical/dental/vision package, 401k retirement and financial wellness plans
- Generous vacation policy; work hard and take time when you need it
- Team lunch every Monday and a well-stocked kitchen full of snacks
- A free on-site gym facility
- Open, creative workspace located inside the historic site of NJ innovation, Bell Works
- Leadership identified as the 2017 Tech Innovator of the year and highlighted by Entrepreneur Magazine as builder of one of the most Entrepreneurial companies in America.
- We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status.
Vydia (vydia.com) is a fast growing, Inc. 500 technology company that has built a premiere platform for the exploding video content industry. Our solutions empower music creators to easily distribute, manage, and optimize their video and audio content through one centralized platform. The company is viewed as one of the leaders in the space as evidenced by strategic partnerships and integrations with Vevo, YouTube, Facebook, Spotify, Apple, Dailymotion and several more. The vision of the company is to disrupt the entertainment industry through automation, data, and artificial intelligence that helps both creators and the next generation of music distribution. Vydia is constantly evolving and has attracted more than 200,000 musicians, influencers, and brands worldwide. The company has recently completed Series A financing, poising it to continue the exponential growth it has achieved since its inception.