Data Engineer
Skills
API ManagementApache SparkData AnalyticsData IntegrityDimensional ModelingModelingPermissions
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Hands-on experience as a Data Engineer or similar role
- Strong experience with Databricks and the broader Databricks ecosystem
- Experience building and maintaining ETL/ELT pipelines
- Strong knowledge of Apache Spark PySpark SQL and Delta Lake
- Experience working with structured semi-structured and unstructured data
- Understanding of data quality data validation monitoring and control processes
- Experience with Unity Catalog including permissions access control metadata and governance concepts
- Ability to support and work within architected Databricks environments
- Familiarity with cloud data platforms such as AWS Azure or Google Cloud Platform
- Strong SQL skills and experience optimizing queries and data transformations
- Ability to troubleshoot pipeline failures performance issues and data integrity problems
- Strong communication skills and ability to work directly with customers and stakeholders
- Ability to translate business and technical requirements into working data solutions
Nice to have
- Hands-on
- Detail-oriented
- Customer-facing
- Collaborative
- Analytical
- Problem-solving
Day to day
- Design and build scalable data pipelines using Databricks and Spark-based processing
- Implement ETL and ELT workflows to ingest transform validate and deliver data
- Support Databricks environments data governance controls and reliable cloud-based data solutions
- Troubleshoot pipeline failures performance issues and data integrity defects while improving monitoring and accuracy