Senior Data Engineer
Skills
Computer ScienceDevopsWarehouse OperationsAdvanced Cardiac Life SupportDatabasesDownstream Oil And GasEvolution
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- 7+ years of data engineering experience with significant streaming specialization
- Bachelor's degree in Computer Science Engineering or related STEM field
- Hands-on experience with Apache Kafka including cluster management performance tuning and ecosystem tools
- Production experience with AWS MSK and Amazon Kinesis
- Strong background in real-time data processing and stream analytics
- Proficiency in Python Java and Scala for streaming applications
- Experience with Apache Spark Streaming Apache Flink or AWS Lambda for stream processing
- Knowledge of Avro Protocol Buffers JSON and schema registry management
- Experience with AWS streaming and analytics services
- Understanding of Infrastructure as Code using Terraform or CloudFormation
- Experience with CI/CD pipelines and cloud observability tools
- Knowledge of SQL and NoSQL databases and data warehousing solutions
Nice to have
- Technical leadership
- Mentorship
- Strategic planning
- Risk management
- Conflict resolution
- Transparent communication
Day to day
- Design and maintain scalable real-time streaming architectures using Kafka MSK and Kinesis to move high-volume data efficiently
- Build fault-tolerant event-driven pipelines that process transform and analyze streaming data for immediate operational insight
- Optimize secure high-performance stream processing systems and support integration with data lakes warehouses and downstream applications