Remote Data Engineer Job in UK Magnit

Who you are

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• At least 5 years' experience in a similar role.
• Excellent communication skills (verbal and written) in English.
• Expert-level proficiency in SQL Server, including on-premises environments.
• Strong experience with Azure Stack, including ADF, Azure Portal, Blob Storage, and Data Lake.
• Proficient in Databricks, PySpark, and Python for data processing and analysis.
• Experience with Power BI for data visualization.
• Skilled in data modeling, including creating ER diagrams and handling data normalization/denormalization.
• Proven experience with delta loading and change data capture (CDC) techniques.
• Strong understanding of data orchestration principles.
• Hands-on experience with CI/CD pipelines in Azure DevOps.
• Proficient in writing ARM and Bicep templates as well as Yaml templates.
• Familiarity with AWS is a plus.
• Excellent problem-solving skills and attention to detail.
• Strong communication skills, with the ability to translate business requirements

Desirables

• AWS
• CI/CD pipelines
• ER diagrams

What the job involves

• Design, develop, and maintain data pipelines using Azure Data Factory (ADF), Databricks, PySpark, and SQL.
• Manage and optimize Azure resources including Azure Portal, Blob Storage, and Data Lake.
• Implement and manage data workflows, ensuring seamless data orchestration and integration.
• Develop and maintain ETL processes, including delta loading and change data capture (CDC) mechanisms.
• Perform data modeling, including normalizing and denormalizing data, and creating ER diagrams.
• Utilize Power BI (PBI) for data visualization and reporting.
• Manage on-premises SQL Server databases and develop SSRS reports.
• Implement CI/CD pipelines in Azure DevOps, including writing ARM and Bicep templates.
• Develop and maintain Yaml templates for infrastructure as code.
• Ensure data security, quality, and governance across all platforms.
• Collaborate with stakeholders to understand business requirements and translate them into technical specifications.