Software/Data Engineer
Skills
DevopsWarehouse OperationsApache AirflowApache SparkBig Data AnalyticsData AnalyticsData Warehousing
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Expert in batch distributed data processing and near real-time streaming pipelines with Kafka Flink Spark or similar
- Experience in data lake or big data analytics platform implementation with cloud based solutions preferably AWS
- Proficient in Python programming and software engineering best practices
- Expertise with RDBMS data warehousing data modelling and relational SQL such as Redshift and PostgreSQL plus NoSQL databases
- Proficiency in DevOps CI/CD pipeline management and infrastructure as code deployment best practices
Nice to have
- Curious
- Collaborative
- Experimental
- Learning mindset
- Community-minded
Day to day
- Shape the architecture of data products for analytics and data science use cases such as forecasting feature engineering customer behaviour and new data source integration
- Lead data transformation best practices across data modelling performance optimisation data governance and reliability to keep data consistent available and trustworthy
- Build reusable data platforms that help teams ingest store transform and serve their own data products while partnering with data scientists and machine learning engineers
Hiring process
- Apply now