Data Engineer/ETL

Data Engineer/ETL

Location: Pittsburgh, PA

Contract for 12+ months

Description

Data Engineer - ETL with Python, Hadoop & Snowflake

Responsibilities

• Organize business needs into ETL/ELT logical models and ensure data structures are designed for flexibility to support scalability of business solutions
• Craft and implement data pipelines utilizing Glue, Lambda, Spark, and Python
• Define and deliver reusable components for ETL/ELT framework.
• Define optimal data flow for system integration and data migration
• Integrate new data management technologies and software engineering tools into existing structures
  
  

Supplier Call Notes

They are looking for a true data engineer/ETL

Top 5 Skills are Python, Snowflake, Airflow, Spark, SQL

Primary skills are spark and Python - Must be strong here

Must be good with Airflow (scheduling tool)

Need good skills with Hadoop and Snowflake - MUST have very good experience in Snowflake

They will be using some SQL and Kubernetes on the job as well

Will be building out ETL pipelines, data pipelines, regulatory requirements - 100% ETL work, no data modeling

Looking for 6-7+ years experience minimum - 100% engineering role