RRemoteville
Illumination Works logoIllumination Works

Data Engineer

Skills
Computer ScienceMicrosoft Azure Machine LearningSQLAmazon Web ServicesApache SparkAzure DatabricksData Modeling
Role

What the job involves

The main requirements, responsibilities and hiring steps.

Requirements

  • Must have or be willing to obtain Secret Clearance (this requires US Citizenship)
  • Strong SQL skills
  • Experience with Snowflake, Databricks, Spark SQL, PySpark, and Python
  • Cloud experience: Azure, AWS, or GCP
  • Develop and maintain ETL pipelines
  • Database design and principles
  • Data modeling, schema development, and data-centric documentation
  • Experience integrating data from a variety of data source types
  • Recommend and advise on optimal data models for data ingestion, integration, and visualization
  • Experience improving code performance and query optimization
  • Use Continuous Integration/Continuous Delivery (CI/CD) concepts to engineer a standardized data environment
  • Outstanding problem-solving skills
  • Excellent verbal and written communication skills and the ability to interact professionally with a diverse group, including executives, managers, and subject matter experts

Nice to have

  • Experience with both structured and unstructured data
  • Experience with ML software, like SciKit Learn, SparkML, TensorFlow, Horovod, and Petastorm
  • Experience with Hadoop, Hive, and R
  • Understand best practices for effective predictive models, including cluster management and algorithms
  • Bachelor’s or Master’s degree in Computer Science, Mathematics, or comparable academic discipline

Day to day

  • As a Data Engineer, you will work hands-on with challenging data engineering, data management, and analytics projects.
  • You will collaborate with data scientists, analysts, business users, and IT teams to design, implement, and deploy data services and analytics.
  • We value passion, curiosity, and perseverance with an ability to communicate ideas and results to diverse audiences.