How Remoteville checks and expires listings
Senior Data Engineer
Skills
Data EngineeringSQLAirflowAmazon RedshiftApache SparkDatabasesExtract
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Expert SQL skills.
- 4+ years experience with:
- Scaling and optimizing schemas.
- Performance tuning ETL pipelines.
- Building pipelines for processing large amounts of data.
- Proficiency with Python, Scala and other scripting languages.
- Experience with:
- MySQL and Redshift.
- NoSQL data stores, methods and approaches.
- Kinesis or other data streaming services.
- Airflow or other pipeline workflow management tools.
- EMR, Spark and ElasticSearch.
- Docker or other container management tools.
- Developing infrastructure as code (IAC).
- Ability to effectively work and communicate with cross-departmental partners and non-technical teams.
Nice to have
- Experience with Segment customer data platform with integration to Braze.
- Terraform.
- Tableau.
- Django.
- Flask.
Day to day
- Handle data engineering tasks in a team focused on improving search functionality and customer search experience.
- Design, develop, and own ETL pipelines that deliver data with measurable quality.
- Scope, architect, build, release, and maintain data oriented projects, considering performance, stability, and an error-free operation.
- Identify and resolve pipeline issues while discovering opportunities for improvement.
- Architect scalable and reliable solutions to move data across systems from multiple products in nearly real-time.
- Continuously improve our data platform and keep the technology stack current.
- Solve critical issues in complex designs or coding schemes.
- Monitor metrics, analyze data, and partner with other internal teams to solve difficult problems creating a better customer experience.