RemotevilleRemote Sr./Staff Data Engineer Job in Palo Alto, CA Machinify, Inc.
Sr./Staff Data Engineer Machinify, Inc.
$200000 - $250000
SQLAirflowApache SparkCustomer DataData PipelinesData ScienceDistributed ComputingExtract Transform LoadQuery WritingWorkflow Management
Palo AltoCA
Bending the healthcare cost curve with AI.
100+ employees
Artificial IntelligenceHealthcareData AnalysisCloud Computing
Role
Who you are
- Deep experience as a hands-on Data Engineer building production data pipelines
- Experience managing the delivery of complex data
- Experience in ETL orchestration and workflow management tools preferably Apache Airflow
- Experience in Spark or other distributed computing frameworks
- SQL and Python experience
- Advanced SQL performance tuning
- Knowledge of Kubernetes and building Docker images
- Experience in AWS & GCP
- Experience working with APIs to collect or ingest data
- Manage SLA for all pipelines in allocated areas of ownership
- Experience with streaming technologies like Kafka, Spark streaming
- Experience with ELK stack, Grafana
What the job involves
- Understand all aspects of a business problem including those unrelated to their area of expertise, weigh pros and cons of different approaches and suggest ones likely to succeed
- Work with cross-functional organization including engineering, delivering, subject-matter experts, product managers, as well as platform engineers to deliver a scalable framework
- Map customer data into Machinify canonical form, identify and ingest non-canonical fields and generalize the process to a minimal level of customization
- Proactively design and adapt the canonical form to suit changing query patterns and needs
- Ultimately own data availability and quality for the Data Science organization
Company
Company mission
We develop software that helps people get the right medical care, at the right time, at the right price. The $4 trillion healthcare industry is the largest and most complex sector of the U.S. economy. It involves maze-like processes, arcane rules, a multi-party payment system... and the yearly processing of 7 billion health claims. Healthcare data is fragmented across industry players, stored in legacy systems, and is often unstructured and not machine-readable (think faxes). We started Machinify to leverage healthcare data at scale to drive down costs and improve outcomes. Our software platform leverages the latest advances in machine learning, large language models, data analytics, and cloud processing to solve previously intractable problems.Company values
- Machinify is committed to hiring talented and qualified individuals with diverse backgrounds for all of its positions. Machinify believes that the gathering and celebration of unique backgrounds, qualities, and cultures enriches the workplace.