Principal Data Engineer
Skills
Artificial IntelligenceAzure DatabricksData WarehousingDatabasesExtractTransformLoad
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- 9+ years experience in information technology or computer science
- BS or MS degree in Computer Science or related technical field
- 4+ years of Python or Java development experience
- 4+ years of SQL experience
- 4+ years of schema design and dimensional data modeling experience
- Experience designing building and maintaining data processing systems
- Experience with cloud platforms such as Snowflake Azure or Databricks
- Hands on experience with programming languages such as Python SQL or GoLang
- Knowledge of SQL database design concepts and data models
- Experience with big data tools such as Hadoop Spark and Kafka
- Experience with relational and NoSQL databases including MongoDB Postgres and Cassandra
- Experience with data pipeline and workflow management tools such as Airflow Dataflow and Composer
- Good knowledge of software engineering principles methodologies and best practices
- Ability to multitask and shift priorities as needed
- Ability to use BI tools for data analysis and data QA
Nice to have
- Master’s degree
- Data engineering certification
- Cloud experience
Day to day
- Expands and optimizes data architecture and data pipelines for cross-functional teams
- Builds and maintains robust data systems from the ground up with consistent delivery architecture
- Supports developers analysts architects and scientists by resolving data delivery and infrastructure needs