Lead Data Engineer (Remote)
About the position
This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Data Engineer. In this role, you will play a crucial part in constructing robust data pipelines that serve as the backbone for internal business analytics. Collaborating closely with senior engineers, you will ensure the seamless ingestion and preparation of data from various sources, elevating data quality for analysis. Your contributions will directly influence critical decision-making processes, providing leaders with necessary insights efficiently. This role demands a commitment to excellence, allowing you to make a real impact in a fast-paced, results-oriented environment.
Responsibilities
• Build and maintain Azure Data Factory pipelines to ingest data from multiple sources.
• Write Python code in Databricks to clean raw data and move it into the silver layer, handling deduplication, type casting, and validation.
• Monitor daily jobs and troubleshoot any failures to ensure pipeline stability.
• Implement automated checks to verify data integrity between source systems and the data lake.
Requirements
• 5+ years of experience in Data Engineering, with a focus on ETL/ELT pipelines in cloud environments.
• Proficiency in Azure Data Factory and Databricks for pipeline development.
• Strong proficiency in SQL for data analysis and Python for scripting.
• Experience implementing automated data quality checks and troubleshooting pipeline failures.
• Familiarity with data schemas and APIs from platforms like Microsoft Dynamics 365, Salesforce, and ServiceNow.
• Demonstrated experience using LLMs to enhance data engineering workflows.
• Bachelor’s degree in Computer Science, Software Engineering, Data Engineering, or a related field.
Benefits
• Comprehensive benefits package including medical, dental, and vision coverage.
• Paid time off and a Registered Retirement Savings Plan (RRSP).
• Tuition reimbursement and paid parental leave.
• Access to a library of personal and professional training resources.
• Employee discounts and additional insurance coverage.
• Starting salary anticipated between $185,000 and $200,000, commensurate with experience.
• Bonus eligibility.
Apply tot his job
Apply To this Job
This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Data Engineer. In this role, you will play a crucial part in constructing robust data pipelines that serve as the backbone for internal business analytics. Collaborating closely with senior engineers, you will ensure the seamless ingestion and preparation of data from various sources, elevating data quality for analysis. Your contributions will directly influence critical decision-making processes, providing leaders with necessary insights efficiently. This role demands a commitment to excellence, allowing you to make a real impact in a fast-paced, results-oriented environment.
Responsibilities
• Build and maintain Azure Data Factory pipelines to ingest data from multiple sources.
• Write Python code in Databricks to clean raw data and move it into the silver layer, handling deduplication, type casting, and validation.
• Monitor daily jobs and troubleshoot any failures to ensure pipeline stability.
• Implement automated checks to verify data integrity between source systems and the data lake.
Requirements
• 5+ years of experience in Data Engineering, with a focus on ETL/ELT pipelines in cloud environments.
• Proficiency in Azure Data Factory and Databricks for pipeline development.
• Strong proficiency in SQL for data analysis and Python for scripting.
• Experience implementing automated data quality checks and troubleshooting pipeline failures.
• Familiarity with data schemas and APIs from platforms like Microsoft Dynamics 365, Salesforce, and ServiceNow.
• Demonstrated experience using LLMs to enhance data engineering workflows.
• Bachelor’s degree in Computer Science, Software Engineering, Data Engineering, or a related field.
Benefits
• Comprehensive benefits package including medical, dental, and vision coverage.
• Paid time off and a Registered Retirement Savings Plan (RRSP).
• Tuition reimbursement and paid parental leave.
• Access to a library of personal and professional training resources.
• Employee discounts and additional insurance coverage.
• Starting salary anticipated between $185,000 and $200,000, commensurate with experience.
• Bonus eligibility.
Apply tot his job
Apply To this Job