Data Engineer (Azure Databricks)
Company Description
Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989.
Miratech has coverage across 5 continents and operates in over 25 countries around the world. Miratech retains nearly 1000 full-time professionals, and our annual growth rate exceeds 25%.
Job Description
We are looking for a Middle Data Engineer specialized in Azure Databricks to join our data platform team.
The candidate will design and develop modern data pipelines and Lakehouse architectures, leveraging Azure Databricks, Spark, and Azure Data Factory, while integrating with existing SQL Server-based data warehouse environments, also evolving our data platform towards scalable, cloud-based data architectures, enabling advanced analytics and business intelligence.
Key responsibilities:
Design, develop, and maintain data pipelines using Azure Databricks
Build and optimize data transformations using PySpark and SQL in Databricks
Implement and maintain Lakehouse architectures using Delta Lake
Develop ETL/ELT pipelines orchestrated through Azure Data Factory
Integrate data from multiple sources into the data platform and analytical layers
Design and maintain data models and data warehouse structures for analytics
Ensure data quality, scalability, and performance of large-scale data processing pipelines
Collaborate with BI teams to support Power BI and reporting platforms
Support and evolve existing SQL Server data platforms and ETL solutions (SSIS) when required
Contribute to the design of modern cloud-based data architectures
Qualifications
3+ years of experience in Data Engineering or Data Warehouse development
Strong experience with Azure Databricks
Experience developing data pipelines using PySpark and Spark SQL
Solid understanding of distributed data processing and big data concepts
Experience working with Delta Lake and Lakehouse architectures
Strong SQL skills and experience with SQL Server relational databases
Experience building data pipelines using Azure Data Factory
Experience handling large datasets and performance optimization
Nice to have
Experience with Spark optimization techniques (partitioning, caching, cluster tuning)
Experience with structured streaming in Databricks
Knowledge of CI/CD pipelines for data platforms (Azure Devops)
Familiarity with Power BI
Experience in migrating from traditional ETL process to cloud architectures
Additional Information
We offer:
Culture of Relentless Performance: join an unstoppable technology development team with a 99% project success rate and more than 30% year-over-year revenue growth.
Competitive Pay and Benefits: enjoy a comprehensive compensation and benefits package, including health insurance, and a relocation program.
Work From Anywhere Culture: make the most of the flexibility that comes with remote work.
Growth Mindset: reap the benefits of a range of professional development opportunities, including certification programs, mentorship and talent investment programs, internal mobility and internship opportunities.
Global Impact: collaborate on impactful projects for top global clients and shape the future of industries.
Welcoming Multicultural Environment: be a part of a dynamic, global team and thrive in an inclusive and supportive work environment with open communication and regular team-building company social events.
Social Sustainability Values: join our sustainable business practices focused on five pillars, including IT education, community empowerment, fair operating practices, environmental sustainability, and gender equality.
* Miratech is an equal opportunity employer and does not discriminate against any employee or applicant for employment on the basis of race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity, or any other protected status under applicable law.
Miratech helps visionaries change the world. We are a global IT services and consulting company that brings together enterprise and start-up innovation. Today, we support digital transformation for some of the world's largest enterprises. By partnering with both large and small players, we stay at the leading edge of technology, remain nimble even as a global leader, and create technology that helps our clients further enhance their business. We are a values-driven organization and our culture of Relentless Performance has enabled over 99% of Miratech's engagements to succeed by meeting or exceeding our scope, schedule, and/or budget objectives since our inception in 1989.
Miratech has coverage across 5 continents and operates in over 25 countries around the world. Miratech retains nearly 1000 full-time professionals, and our annual growth rate exceeds 25%.
Job Description
We are looking for a Middle Data Engineer specialized in Azure Databricks to join our data platform team.
The candidate will design and develop modern data pipelines and Lakehouse architectures, leveraging Azure Databricks, Spark, and Azure Data Factory, while integrating with existing SQL Server-based data warehouse environments, also evolving our data platform towards scalable, cloud-based data architectures, enabling advanced analytics and business intelligence.
Key responsibilities:
Design, develop, and maintain data pipelines using Azure Databricks
Build and optimize data transformations using PySpark and SQL in Databricks
Implement and maintain Lakehouse architectures using Delta Lake
Develop ETL/ELT pipelines orchestrated through Azure Data Factory
Integrate data from multiple sources into the data platform and analytical layers
Design and maintain data models and data warehouse structures for analytics
Ensure data quality, scalability, and performance of large-scale data processing pipelines
Collaborate with BI teams to support Power BI and reporting platforms
Support and evolve existing SQL Server data platforms and ETL solutions (SSIS) when required
Contribute to the design of modern cloud-based data architectures
Qualifications
3+ years of experience in Data Engineering or Data Warehouse development
Strong experience with Azure Databricks
Experience developing data pipelines using PySpark and Spark SQL
Solid understanding of distributed data processing and big data concepts
Experience working with Delta Lake and Lakehouse architectures
Strong SQL skills and experience with SQL Server relational databases
Experience building data pipelines using Azure Data Factory
Experience handling large datasets and performance optimization
Nice to have
Experience with Spark optimization techniques (partitioning, caching, cluster tuning)
Experience with structured streaming in Databricks
Knowledge of CI/CD pipelines for data platforms (Azure Devops)
Familiarity with Power BI
Experience in migrating from traditional ETL process to cloud architectures
Additional Information
We offer:
Culture of Relentless Performance: join an unstoppable technology development team with a 99% project success rate and more than 30% year-over-year revenue growth.
Competitive Pay and Benefits: enjoy a comprehensive compensation and benefits package, including health insurance, and a relocation program.
Work From Anywhere Culture: make the most of the flexibility that comes with remote work.
Growth Mindset: reap the benefits of a range of professional development opportunities, including certification programs, mentorship and talent investment programs, internal mobility and internship opportunities.
Global Impact: collaborate on impactful projects for top global clients and shape the future of industries.
Welcoming Multicultural Environment: be a part of a dynamic, global team and thrive in an inclusive and supportive work environment with open communication and regular team-building company social events.
Social Sustainability Values: join our sustainable business practices focused on five pillars, including IT education, community empowerment, fair operating practices, environmental sustainability, and gender equality.
* Miratech is an equal opportunity employer and does not discriminate against any employee or applicant for employment on the basis of race, color, religion, sex, national origin, age, disability, veteran status, sexual orientation, gender identity, or any other protected status under applicable law.