Senior Data Engineer
Skills
Amazon Web ServicesAnalyticsApache SparkBusinessModelingPipelinesPython
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Strong experience with Databricks or Snowflake on AWS
- Proven Python skills including Spark and Airflow expertise
- Advanced SQL skills and end-to-end data modelling experience
- Experience building batch and real-time data integrations
- Hands-on CI/CD skills with Git Jenkins or similar
- Ability to ingest cleanse and structure large diverse datasets
- Knowledge of Terraform or similar IaC tools
- Experience with Docker and Kubernetes is beneficial
- Exposure to production Generative AI is an advantage
- Strong collaboration skills and a proactive attitude
Nice to have
- Collaborative
- Proactive
- Innovative
- Detail-oriented
- Adaptable
Day to day
- Build and support data products within a modern data platform
- Design and deliver robust data solutions with engineers scientists and product teams
- Develop optimise and maintain batch and real-time data pipelines across analytics and AI workflows
- Integrate diverse data sources with strong quality standards while improving orchestration monitoring and performance
Hiring process
- Apply via the company website
- Recruitment team may provide support and adjustments
- Interview guaranteed for applicants with a disability who meet minimum criteria