Data Engineer
Skills
AWS GlueAmazon CloudwatchDocumentationGithubGlueIdentity And Access ManagementModeling
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Strong hands-on experience with Pentaho Data Integration PDI
- Proven expertise in AWS Glue PySpark and AWS data ecosystem including S3 Lambda Athena and IAM
- Solid understanding of ETL ELT patterns data modeling and pipeline orchestration
- Ability to quickly analyze legacy systems and translate them into cloud-native architectures
- Excellent communication and documentation skills
Day to day
- Assess existing Pentaho ETL jobs transformations and schedules
- Re-architect and migrate workflows into AWS Glue with PySpark or Scala for a scalable serverless pipeline
- Build and optimize Glue Jobs Crawlers Workflows and Catalog integrations while ensuring performance reliability and maintainability
- Collaborate with internal data engineering teams to validate data flows dependencies and business logic
- Implement CI/CD best practices for Glue deployments and provide logging monitoring alerting documentation and knowledge transfer