Lead Data Engineer
Skills
Data EngineeringAzure DatabricksData GovernanceData QualityInfrastructure ManagementLeadershipPyspark
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Demonstrate expertise in Python
- Demonstrate expertise in PySpark
- Demonstrate expertise in Databricks would be highly desirable and advantageous
- Ability to design and implement scalable data architectures for both batch and streaming data processing
- Proficiency in using cloud platforms such as AWS, Azure, or Google Cloud for data infrastructure management
- Knowledge of data governance practices including data quality management
- Proven experience in leading and mentoring technical data engineering teams
Day to day
- As the technical lead for the team building the strategic Data Platform at LexisNexis IP, you will be instrumental in executing our data strategy for the Data Platform.
- Your role will be pivotal in developing and implementing advanced solutions for data integration, quality control, and continuous delivery, driving our data operations to new heights.
- Your expertise will be crucial in embedding best practices and state-of-the-art data engineering tools, ensuring that our workflows are both efficient and scalable.