Remote Data Engineer Job in San Francisco Bay EY

Who you are

• Proficiency in NiFi with experience in creating and optimizing processors and process groups
• Strong knowledge of Azure Datalake with experience in handling Delta Lake file formats (Parquet)
• Expertise in Python and PySpark for data manipulation and analysis
• Advanced SQL skills with the ability to work across multiple databases including Oracle, SQL Server, and proprietary systems
• Experience with Databricks for data engineering tasks, including building and managing data pipelines
• Experience in building pipeline strategies for data loading and ETL processes, including setting up and managing workflows.

What the job involves

• Develop and enhance data processing systems to ensure seamless data integration across multiple databases including Oracle, SQL Server, and proprietary databases
• Implement and optimize NiFi processors and process groups to streamline data workflows and enhance performance
• Design, develop, and maintain ETL pipelines using Azure Datalake with Delta Lake file format (Parquet) to support data analytics and reporting
• Perform SQL queries to extract, transform, and load data from various sources into target systems ensuring data integrity and consistency
• Collaborate with cross-functional teams to identify and resolve system issues, ensuring data quality and reliability.

Application process

• Submit application online
• Requests for accommodation if needed to be emailed