Senior Data Engineer
This is a remote position.
Job Title: Senior Data EngineerExperience: 7–9 YearsLocation: Remote Notice Period: Immediate Joiners Only
About the EngagementWe are looking for experienced Senior Data Engineers to join a large-scale, multi-year Data Mapping and DataOps Platform modernisation programme for a major government ministry.This strategic initiative focuses on transforming how data is managed, governed, and leveraged as a core enterprise asset.Key Focus Areas
Data Engineering: Designing, building, and optimising scalable data pipelines to enhance stability, traceability, and refresh frequency
Data Mapping & Metadata Capture: Onboarding critical financial systems into a centralised enterprise data catalogue and supporting governance initiatives
Work will be delivered in an Agile/Scrum model, collaborating closely with data product managers and cross-functional teams.Platform OverviewThe platform is built on Microsoft Azure and Microsoft Fabric, with an existing in-house Finance Data Catalogue (FDC).This role will contribute to scaling the platform into a production-grade DataOps ecosystem.Key Responsibilities
Design, build, optimise, and maintain scalable data pipelines
Develop and manage ELT pipelines, orchestration, and automation using Python
Capture and onboard metadata into enterprise data catalogues
Perform data analysis to identify patterns, anomalies, and quality improvements
Contribute to data product design, including data models and structures
Design and implement data models for analytics and reporting
Build and maintain CI/CD pipelines for automated deployment
Develop automation scripts using Python, Bash, and PowerShell
Develop and maintain RESTful APIs for integration and interoperability
Implement data quality checks and validation frameworks
Set up monitoring, alerting, and performance tracking
Develop dashboards and reporting solutions using Microsoft Fabric & Power BI
Collaborate with stakeholders to translate business needs into technical solutions
Ensure compliance with data governance, privacy, and security standards
Maintain technical documentation and communicate with both technical and business teams
Mandatory RequirementsCandidates must have recent (last 48 months) hands-on production experience in:
Python (data pipelines, scripting, automation)
SQL (data transformation, modelling)
Microsoft Azure & Microsoft Fabric
ELT Pipeline Development
Git (version control & collaboration)
Metadata Management & Data Catalogue onboarding
Data Product Design
Agile/Scrum methodologies
Data Analysis & Data Quality Assessment
Technical Skills RequiredCloud & Platform
Microsoft Azure
Microsoft Fabric
Data Engineering
Python, SQL
Bash, PowerShell
Pipelines & DevOps
ELT pipelines
CI/CD pipelines
Data Modelling
Schema design
Metadata management
Data product design
API Development
RESTful APIs
Analytics & Reporting
Power BI
Microsoft Fabric Reporting
Version Control
Git, GitHub
Methodology
Agile / Scrum
Nice to Have
Advanced Power BI (DAX, dashboarding, modelling)
Data visualization beyond standard reporting
Experience with REST API integrations
Exposure to:
Azure Synapse Analytics
Azure Data Lake
Azure DevOps
Experience in government/public sector or regulated environments
Familiarity with data governance frameworks
Experience in financial systems modernisation programs
Apply Now