Data Engineer
Skills
Data EngineeringApache SparkData ArchitectureData ModelingData ModelsData QualityData Warehousing
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Strong proficiency in Python
- Experience with Generative AI model deployment and orchestration
- Understanding of database technologies and modelling techniques
- Experience with managing Databricks environments
- Competency in working with Spark
- Understanding of data warehousing modelling techniques
- Experience in setting up CI/CD DevOps pipelines
- Experience with Azure and AWS data technologies
- Understanding of graph technologies and modelling techniques desirable
- Experience with GCP and Scala desirable
- Excellent communication skills
Nice to have
- Experience with graph technologies
- Experience with GCP
- Experience with Scala
Day to day
- Build scalable and efficient data pipelines
- Design Data Architecture for processing complex data
- Setup and maintain AI and data models and technologies
- Collaborate with stakeholders and optimize data systems
- Monitor and optimize performance, troubleshoot issues, and implement improvements