Data Engineering Manager
Skills
Computer ScienceDevopsAmazon Web ServicesApache SparkBusinessData IntegrityDownstream Oil And Gas
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- 8+ years in data engineering or ETL development
- 4-5 years in a lead or management role
- Expert-level ETL knowledge
- Strong SQL Server and Postgres skills
- Experience with Databricks ecosystem including Delta Lake and Unity Catalog
- Proficiency in Spark SQL and Python/PySpark
- Strong SQL and RDBMS troubleshooting skills
- Ability to perform root-cause analysis
- Bachelor's or Master's degree in Computer Science Engineering or related field
Nice to have
- Curious leader
- Hands-on mindset
- Analytical rigor
- Continuous improvement
- Agile approach
Day to day
- Own the end-to-end delivery of complex data engineering projects across a modern cloud stack.
- Lead and mentor a distributed team of developers in North America and India while providing clear technical direction and removing blockers.
- Deepen understanding of upstream and downstream systems to ensure seamless integration and high-quality data delivery.
- Build testing frameworks and QA plans to protect data integrity and improve reliability across ETL pipelines.