RRemoteville
LDSS Corp logoLDSS Corp

Data Engineer

Skills
Computer ScienceApache SparkData AnalyticsData ExchangeDownstream Oil And GasMicrosoft 365Modeling
Role

What the job involves

The main requirements, responsibilities and hiring steps.

Requirements

  • Bachelor’s degree in computer science engineering or a related field
  • 2-3 years experience with Databricks Apache Spark and cloud-based data engineering and modeling preferred
  • 4 years preferred in SQL for querying and managing relational data
  • 3 years preferred in AWS cloud services including S3
  • 4 years preferred in Python for data analysis cleansing and visualization
  • Experience with SharePoint lists document libraries and Microsoft Graph
  • Experience with Microsoft 365 apps and integration workflows
  • Experience developing and consuming APIs for data exchange
  • Experience with Jira or other agile workflow management systems
  • Ability to build trust with customers and stakeholders
  • Ability to work collaboratively with contractors government personnel and stakeholders
  • Ability to engage SMEs to derive business logic for data transformation methods

Nice to have

  • Customer-focused
  • Detail-oriented
  • Self-starter
  • Collaborative
  • Adaptable
  • Meticulous

Day to day

  • Build and optimize ETL data pipelines with Databricks and Apache Spark for mission-critical datasets
  • Develop scalable data models using Spark DataFrames and Spark SQL to query aggregate and integrate diverse sources
  • Use Python Regular Expressions APIs SharePoint Microsoft Graph AWS and Microsoft 365 to cleanse automate share and maintain data workflows
  • Collaborate with SMEs technical stakeholders and cross-functional teams to translate legacy data into modern schemas and business logic