Data Engineer
Skills
Data EngineeringAnalyticsComputationData LakesData ModelingData ScienceData Warehousing
What the job involves
The main requirements, responsibilities and hiring steps.
Requirements
- Strong programming skills in cloud data processing.
- Experience in large-scale data lakes and proficient data modeling for analytics.
- Familiarity with Spark or similar parallel processing frameworks; Databricks knowledge beneficial.
- Interest in cybersecurity; preferably in identity security.
- Capable of working autonomously on distributed systems and large-scale data processing.
- Must be skilled in Python or similar object-oriented programming languages.
Nice to have
- Spark
- Realtime processing
- Graph data stores
Day to day
- Build and enhance state-of-the-art datalakehouse consuming billions of events daily.
- Focus on security and computational efficiency to filter data and produce actionable insights.
- Engage in software optimization to improve data processing efficiency.
- Develop and maintain data workflows and use observability practices to monitor pipeline performance.
- Create innovative ML/Engineering solutions within the cybersecurity domain.