Data Engineer
Skills
Computer ScienceApache SparkComplex SystemsDownstream Oil And GasDriftEnhanced Data Rates For GSM EvolutionHbase
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- 2+ years building and operating production software systems
- Fluency in at least one programming language
- Experience debugging moderately complex systems and improving reliability or performance
- Strong fundamentals in data structures testing version control and Linux basics
- Spark or PySpark experience desirable
- Hadoop ecosystem exposure desirable
- Workflow orchestration experience desirable
- Search or indexing experience desirable
- Kubernetes and infrastructure as code desirable
- Degree in Computer Science or numerical degree
Nice to have
- Pragmatic
- Detail-oriented
- Collaborative
- Analytical
Day to day
- Engineer distributed ingestion services that pull data from diverse sources and handle messy edge cases while delivering clean outputs to downstream systems.
- Build high-throughput batch and near-real-time processing components with strong focus on performance scalability and predictable cost.
- Design and evolve data contracts and improve platform reliability through testing observability and resilient pipelines.