Data Engineer, Python, AWS, Data Pipelines
Job Description:
• Design and maintain scalable ETL/ELT pipelines in Python, integrating structured and semi-structured data (JSON, CSV, XML) across Snowflake, MongoDB, Postgres, and AWS services (S3, Glue, Lambda, EC2, EMR, Redshift, RDS).
• Build and optimize transformation and orchestration workflows using DBT, Airflow, Prefect, or Dagster.
• Implement data governance, quality checks, and security best practices throughout data pipelines.
• Extend and optimize the Elastic Hierarchy framework to harmonize financial data from various systems.
• Collaborate with analysts, ML engineers, and product teams to deliver business-ready datasets and data solutions.
Requirements:
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
• Strong Python programming skills with experience in data processing and automation.
• Proven experience with the AWS data ecosystem, including S3, Glue, Lambda, EMR, EC2, Redshift, and RDS.
• Hands-on experience with Snowflake, MongoDB, and Postgres databases.
• Proficiency with DBT or similar data transformation tools, and with orchestration frameworks such as Airflow, Prefect, or Dagster.
• Knowledge of data mapping, attribution, or reconciliation (experience in financial services is a strong plus).
• Understanding of hybrid/on-premise deployment models within enterprise environments.
• Excellent English communication skills and ability to collaborate effectively within distributed teams.
Benefits:
Apply tot his job
Apply To this Job
• Design and maintain scalable ETL/ELT pipelines in Python, integrating structured and semi-structured data (JSON, CSV, XML) across Snowflake, MongoDB, Postgres, and AWS services (S3, Glue, Lambda, EC2, EMR, Redshift, RDS).
• Build and optimize transformation and orchestration workflows using DBT, Airflow, Prefect, or Dagster.
• Implement data governance, quality checks, and security best practices throughout data pipelines.
• Extend and optimize the Elastic Hierarchy framework to harmonize financial data from various systems.
• Collaborate with analysts, ML engineers, and product teams to deliver business-ready datasets and data solutions.
Requirements:
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
• Strong Python programming skills with experience in data processing and automation.
• Proven experience with the AWS data ecosystem, including S3, Glue, Lambda, EMR, EC2, Redshift, and RDS.
• Hands-on experience with Snowflake, MongoDB, and Postgres databases.
• Proficiency with DBT or similar data transformation tools, and with orchestration frameworks such as Airflow, Prefect, or Dagster.
• Knowledge of data mapping, attribution, or reconciliation (experience in financial services is a strong plus).
• Understanding of hybrid/on-premise deployment models within enterprise environments.
• Excellent English communication skills and ability to collaborate effectively within distributed teams.
Benefits:
Apply tot his job
Apply To this Job