Data Software Engineer
Skills
DevopsWarehouse OperationsApache SparkBig Data AnalyticsData AnalyticsData WarehousingDatabases
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Expert in batch distributed data processing and near real-time streaming pipelines with Kafka Spark or similar
- Experience in Databricks is a plus
- Experience in big data analytics platform implementation with cloud-based solutions AWS preferred
- Proficient in Python programming and software engineering best practices
- Expertise with RDBMS data warehousing data modelling and relational SQL such as Redshift PostgreSQL and NoSQL databases
- Proficiency in DevOps CI/CD pipeline management and infrastructure as code deployment best practices
Nice to have
- Curious
- Collaborative
- Experimental
- Quality-focused
- Growth-minded
Day to day
- Shape the architecture of data products for analytics and data science use cases like forecasting and feature engineering.
- Lead data transformation by defining best practices in data modelling performance optimisation and data governance.
- Build reusable technology to ingest store transform and serve data products for internal teams.
- Collaborate with data scientists and machine learning engineers to refine data requirements and strengthen data infrastructure.
- Raise data quality standards through continuous assessment with stakeholders and ongoing experimentation with emerging technologies.
Hiring process
- Apply now