Data Engineer
Skills
Agile MethodologiesContinuous Integration And Continuous DeliveryMicrosoft AzureAirflowAmazon Web ServicesCloud ApplicationsData Engineering
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- 5+ years of data engineering or backend data infrastructure experience
- 2+ years hands-on experience with Databricks Spark or distributed data processing in production
- Strong proficiency with PySpark and Python
- Experience with data modeling
- Experience with Airflow dbt or similar orchestration and transformation tools
- Experience with data governance lineage or security best practices
- Experience with Azure AWS or Google Cloud
- Strong knowledge of CI/CD tools and version control workflows
- Experience with relational and NoSQL databases
- Experience monitoring debugging and scaling cloud-based data systems
- Strong communication skills and ability to work independently in a remote team
Nice to have
- Customer centric
- Collaborative
- Curious
- Self-motivated
- Detail-oriented
Day to day
- Design and optimize scalable data pipelines in Databricks to process large data sets.
- Collaborate with data scientists engineers and cloud architects to build secure reliable data systems.
- Automate infrastructure provisioning and support production data workflows in secure cloud environments.