How Remoteville checks and expires listings
Senior Data Engineer
Skills
Computer ScienceDevopsWarehouse OperationsAdvanced Cardiac Life SupportDatabasesDownstream Oil And GasEvolution
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- 7+ years of data engineering experience with significant streaming specialization
- Bachelor's degree in Computer Science Engineering or related STEM field
- Hands-on experience with Apache Kafka including cluster management performance tuning and ecosystem tools
- Production experience with AWS MSK and Amazon Kinesis
- Strong background in real-time data processing and stream analytics
- Proficiency in Python Java and Scala for streaming applications
- Experience with Apache Spark Streaming Apache Flink or AWS Lambda for stream processing
- Knowledge of Avro Protocol Buffers JSON and schema registry management
- Experience with AWS streaming and analytics services
- Understanding of Infrastructure as Code using Terraform or CloudFormation
- Experience with CI/CD pipelines and cloud observability tools
- Knowledge of SQL and NoSQL databases and data warehousing solutions
Nice to have
- Technical leadership
- Mentorship
- Strategic planning
- Risk management
- Conflict resolution
- Transparent communication
Day to day
- Design and maintain scalable real-time streaming architectures using Kafka MSK and Kinesis to move high-volume data efficiently
- Build fault-tolerant event-driven pipelines that process transform and analyze streaming data for immediate operational insight
- Optimize secure high-performance stream processing systems and support integration with data lakes warehouses and downstream applications