What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Experience with distributed processing frameworks like Databricks/Spark.
- Proficiency in Python and PySpark.
- Knowledge of distributed storage and retrieval databases like SQL, MongoDB, Elasticsearch.
- Comfortable with Kubernetes operations and debugging.
- Strong skills in writing unit and integration tests.
Nice to have
- Python
- Distributed Frameworks
- SQL
- NoSQL
- ML Experience
- Monitored Services
Day to day
- Design and build data pipelines for crawling and indexing infrastructure.
- Develop and maintain production databases both SQL and NoSQL.
- Collaborate with machine learning teams to optimize infrastructure and data pipelines.
- Debug live production issues and proactively write tests to prevent future incidents.
- Ship user-facing production services with high performance and monitoring.