RRemoteville

How Remoteville checks and expires listings

Hack The Box logoHack The Box

Senior Data Engineer

Skills
Software As A ServiceApache BeamApache SparkData ServicesDriftModelingMusic Composition
Role

What the job involves

The main requirements, responsibilities and hiring steps.

Requirements

  • Strong data modelling and warehouse architecture skills including dimensional modelling event-driven and lakehouse patterns
  • Hands-on experience with GCP data services especially BigQuery with Pub/Sub Dataflow Bigtable and Cloud Composer as strong pluses
  • Production experience with streaming pipelines on Dataflow Beam Flink or Spark Structured Streaming ingesting from Kafka and or Pub/Sub
  • Solid SQL and strong Python for production-quality code
  • Experience with ClickHouse or another columnar OLAP engine in production
  • Workflow orchestration experience with Airflow or Prefect or Dagster
  • Comfortable with dbt or equivalent transformation frameworks
  • Experience migrating off legacy warehouses onto cloud-native stacks is a plus
  • Working knowledge of ML in production including feature engineering feature stores model deployment drift monitoring and retraining
  • Docker and Kubernetes experience
  • CI CD mindset infrastructure as code sensibility and a bias for simple observable systems
  • Bonus: CDC tooling such as Datastream or Debezium and Vertex AI Feature Store

Nice to have

  • Opinionated about clean architecture
  • Allergic to over-engineering
  • End-to-end ownership
  • Pragmatic
  • Collaborative
  • Process-minded

Day to day

  • Own and evolve data pipelines on GCP from ingestion through transformation and serving
  • Design and build batch and streaming pipelines using Dataflow Pub/Sub Kafka BigQuery Bigtable and ClickHouse
  • Partner with ML engineers to build feature pipelines monitor data drift and support retraining for analytics ML training and online inference
  • Improve data quality reliability observability and cost efficiency while helping migrate away from Snowflake to a GCP-native stack

Hiring process

  • Submit your application
  • Meet the Talent Acquisition team
  • Meet the hiring team
  • Complete 2 assignments and debrief with the hiring team
  • Final conversation with senior leadership
  • Receive and accept the offer