RRemoteville
Aorton Inc logoAorton Inc

Lead Big Data Engineer

Skills
Azure Data FactoryData EngineeringJavaSQLApache SparkBig DataCode Review
Role

What the job involves

The main requirements, responsibilities and hiring steps.

Requirements

  • Experience with Apache Airflow
  • Experience with Apache Spark
  • Big Data processing proficiency
  • Proficiency in Java or Python
  • Experience with Kubernetes
  • Knowledge of SQL
  • Experience with ETL processes
  • Use of Github and CICD tools
  • 5+ years of development experience
  • Experience with Spark Notebooks and Synapse
  • Experience managing or leading a team of 10+
  • High level architecture experience

Day to day

  • This is a high-level technical/leadership role involving meetings with product partners and business stakeholders, tracking project timelines and scope.
  • The job also requires reviewing and writing code with a roughly 50/50 split between leadership and technical responsibilities.
  • The role manages all developer and QA workstreams, ensuring tasks are on track, managing blockers, and escalating delays if needed.