RRemoteville

How Remoteville checks and expires listings

Hack The Box logoHack The Box

Senior Data Engineer

Skills
Software As A ServiceApache BeamApache SparkData ServicesDriftModelingMusic Composition
Role

What the job involves

The main requirements, responsibilities and hiring steps.

Requirements

  • Strong data modelling and warehouse architecture skills including dimensional event-driven and lakehouse patterns
  • Hands-on experience with GCP data services especially BigQuery with Pub/Sub Dataflow Bigtable and Cloud Composer as strong pluses
  • Production experience with streaming pipelines on Dataflow Beam Flink or Spark Structured Streaming ingesting from Kafka and or Pub/Sub
  • Solid SQL and strong Python with production-quality code experience
  • Experience with ClickHouse or another columnar OLAP engine in production
  • Workflow orchestration experience with Airflow or Prefect or Dagster
  • Comfortable with dbt or equivalent transformation frameworks
  • Experience migrating off legacy warehouses onto cloud-native stacks is a plus
  • Working knowledge of ML in production including feature engineering feature stores model deployment drift monitoring and retraining
  • Docker and Kubernetes experience
  • CI CD mindset infrastructure as code sensibility and a bias for simple observable systems
  • Bonus CDC tooling Datastream Debezium Vertex AI Feature Store

Nice to have

  • Opinionated about clean architecture
  • Allergic to over-engineering
  • End-to-end ownership mindset
  • Pragmatic problem solver
  • Comfortable with ambiguity
  • Bias for simple systems

Day to day

  • Own and evolve data pipelines on GCP, improving reliability data quality and trust across the organisation.
  • Design and run end-to-end streaming and batch workflows from ingestion and CDC through transformation serving and feature layers.
  • Partner with ML engineers to build feature pipelines monitor drift support retraining and power analytics ML training and online inference.
  • Translate stakeholder needs into pragmatic data products integrate external sources and APIs and continuously improve observability and cost efficiency.

Hiring process

  • Submit application
  • Meet Talent Acquisition
  • Meet hiring team
  • Complete 2 assignments and debrief
  • Final conversation with senior leadership
  • Receive offer and accept