Senior Data Engineer
Skills
Amazon Simple Notification ServiceApache SparkAzure DatabricksData ProcessingData ServicesEngineeringEvolution
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Senior-level experience shipping production backend or data services in Python
- Production AWS architecture experience with event-driven services such as SQS SNS EventBridge Step Functions and Lambda
- Infrastructure-as-code with Terraform
- Experience standing up and managing cloud infrastructure end-to-end
- Experience operating systems in production including incident response performance capacity and cost
- Ability to be accountable when production issues occur
Nice to have
- Reliable
- Maintainable
- Cost-efficient
- Production-minded
- Independent
- Accountable
Day to day
- Implement Python data pipelines and backend services on AWS for a customer-facing production platform
- Design and manage AWS infrastructure with Terraform and Terragrunt, then deploy the system into the customer's environment
- Own the platform after launch by handling incident response, performance, capacity, cost, and ongoing evolution