Senior Data Engineer
Skills
AirflowAthenaBig DataData ArchitectureData ModelingDatabasesDatasets
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Software Engineering background with Python and SQL experience
- Knowledge of data processing frameworks like Pandas, dbt, or Spark
- Experience with AWS, Azure, or Google Cloud
- Familiarity with data orchestration solutions like Airflow, Oozie, or Luigi
- Understanding of relational and non-relational database design
- Experience with software engineering best practices and automation
Nice to have
- Production experience with large datasets
- Knowledge of BI and analytics tools
- Understanding of data security and PII issues
- Experience with big data cloud technologies
- Interest in adopting new technologies
Day to day
- Contribute to the design, build, and operational management of data lake and analytics solution on AWS technologies
- Use state-of-the-art technologies like Airflow, dbt, and Spark for data processing
- Support Data Analysts and Data Scientists with automation, tooling, and data pipelines