Data Engineer - ETL \/ Python \/ Hadoop \/ Snowflake
Pittsburgh, Pennsylvania
Employer: Compunnel
Industry:
Salary: Competitive
Job type: Part-Time
Job Description:
Responsibilities
Organize business needs into ETL/ELT logical models and ensure data structures are designed for flexibility to support scalability of business solutions
Craft and implement data pipelines utilizing Glue, Lambda, Spark, and Python
Define and deliver reusable components for ETL/ELT framework.
Define optimal data flow for system integration and data migration
Integrate new data management technologies and software engineering tools into existing structures
Qualifications
Experienced in Design, Development, and Implementation of large - scale projects in financial industries using Data Warehousing ETL tools (Pentaho)
Experience in creating ETL transformations and jobs using Pentaho Kettle Spoon designer and Pentaho Data Integration Designer and scheduling
Strong knowledge and experience of SQL, Snowflake, Python and Spark
Experience with Big Data/distributed frameworks such as Spark, Kubernetes, Hadoop, and Hive
Ability to design ETL/ELT solutions based on user reporting and archival requirements
Strong sense of customer service to consistently and effectively address client needs
Self-motivated; comfortable working independently under general direction
Primary Skills:
Spark
Python
Snowflake
Airflow
Hadoop
Education: Bachelors Degree
Additional client information:
Responsibilities
Organize business needs into ETL/ELT logical models and ensure data structures are designed for flexibility to support scalability of business solutions
Craft and implement data pipelines utilizing Glue, Lambda, Spark, and Python
Define and deliver reusable components for ETL/ELT framework.
Define optimal data flow for system integration and data migration
Integrate new data management technologies and software engineering tools into existing structures
Qualifications
Experienced in Design, Development, and Implementation of large - scale projects in financial industries using Data Warehousing ETL tools (Pentaho)
Experience in creating ETL transformations and jobs using Pentaho Kettle Spoon designer and Pentaho Data Integration Designer and scheduling
Strong knowledge and experience of SQL, Snowflake, Python and Spark
Experience with Big Data/distributed frameworks such as Spark, Kubernetes, Hadoop, and Hive
Ability to design ETL/ELT solutions based on user reporting and archival requirements
Strong sense of customer service to consistently and effectively address client needs
Self-motivated; comfortable working independently under general direction
Primary Skills:
Spark
Python
Snowflake
Airflow
Hadoop
Education: Bachelors Degree
Additional client information:
Created: 2024-06-26
Reference: PATDC4959162
Country: United States
State: Pennsylvania
City: Pittsburgh
ZIP: 15216
Similar jobs:
-
Data Engineer
Insight Global in Ridley Park, Pennsylvania -
Senior Staff Software Engineer - Autonomy Data: Motion Planning Data Engine
Aurora Innovation in Pittsburgh, Pennsylvania💸 $170000 - $272000 per year -
Sr. Software Engineer (PoP Bulk Data Publishing Team)
Experis in Radnor, Pennsylvania -
Staff Software Engineer - Data Platforms
Aurora Innovation in Pittsburgh, Pennsylvania -
Lead Big Data Engineer
Clarivate in Philadelphia, Pennsylvania -
Senior Manager Data Engineering
Pennsylvania Medicine in Philadelphia, Pennsylvania -
Data Infrastructure Engineer
MongoDB in United States, Pennsylvania -
R&D Engineer II (Simulation Data Processing)
Ansys in Canonsburg, Pennsylvania -
Senior Big Data Engineer
Clarivate in Philadelphia, Pennsylvania -
Data Platform Support Engineer (Kafka Engineer)
System One Holdings, LLC in Philadelphia, Pennsylvania -
Data Protection Platform Engineering Manager
Peapod Digital Labs in Carlisle, Pennsylvania -
Senior Application Engineer - Data Management (Remote)
Ansys in Canonsburg, Pennsylvania -
Regional Safety Engineer - Berwick, PA, Data Center Health and Safety
Amazon in Berwick, Pennsylvania -
Data Engineer - Horsham, PA
UnitedHealth Group in Horsham, Pennsylvania -
Senior Manager Data Engineering
Pennsylvania Medicine in Philadelphia, Pennsylvania -
Senior Data Engineer PennDnA
Pennsylvania Medicine in Philadelphia, Pennsylvania -
Data Center Regional Mechanical Engineer (Field Engineering), Field Engineering
Amazon in Berwick, Pennsylvania -
Senior Software Engineer - Data Platform
Veeva Systems in Philadelphia, Pennsylvania -
Senior Software Engineer - Autonomy Data: Labels Platform
Aurora Innovation in Pittsburgh, Pennsylvania💸 $130000 - $195000 per year -
Data Platform Support Engineer (Kafka Engineer)
System One Holdings, LLC in Pittsburgh, Pennsylvania