Senior Data Engineer
Skills
Data EngineeringSQLAirflowData PipelinesData QualityExtractTransform
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Bachelor's degree or equivalent Master's is a plus or equivalent experience
- 3+ year of experience contributing to and providing technical leadership in a data engineering or software development team
- Hands on experience in design and development of data integration/ETL data pipelines from various data sources and data formats
- Knowledge of best practices for Airflow DAG management scheduling and monitoring
- Experience with implementing data quality checks error handling and retries in Airflow workflows
- Hands-on experience designing developing and maintaining data pipelines using Apache Airflow including creating custom operators sensors and plugins
- Familiarity with Airflow Executors for task parallelism and resource management
- Experience driving technical ideas and communicating clearly to both technical and non-technical audiences at all levels of the organization
- Strong development testing debugging skills at all levels unit system integration and performance testing along with detail-oriented documentation skills
- Effective communication and problem-solving skills
- Strong database engineering skills with emphasis Postgres SQL DynamoDB Snowflake
Nice to have
- AWS RDS experience
- Data governance knowledge
- Object-oriented programming
- Experience with Git
- Experience with Jira
- AWS ECS Fargate knowledge
- Big data technology
Day to day
- Design develop and maintain scalable data pipelines with Snowflake Apache Airflow dbt SQL and Python with AWS services
- Support and contribute to the continuous improvement tuning applications infrastructure developments process controls and upgrades of the data platform
- Provide guidance hands-on development and operational support for the deployment of database scripts and changes across multiple environments
- Collaborate with Moody's technical teams and business owners as needed during the design and implementation
- Manage individual time and tasks effectively
- Collaborate with cross-functional teams to understand data requirements and implement effective solutions