[Remote] Senior Azure Databricks Engineer

Note: The job is a remote job and is open to candidates in USA. Fractal Analytics is a strategic AI partner to Fortune 500 companies, aiming to empower human decisions through intelligence. They are seeking a Senior Azure Databricks Engineer to lead the migration of on-premise SQL Server databases to the Azure Databricks Lakehouse Platform, focusing on designing ETL frameworks and data governance strategies. Responsibilities • Design and implement data migration pipelines from on-prem SQL Server to Azure Databricks Lakehouse. • Build parameter-driven/config-based ETL mappings for reusable and dynamic pipelines. • Migrate SQL stored procedures, views, and business logic from SQL Server to Databricks Unity Catalog stored procedures within the Databricks Lakehouse using Databricks SQL or PySpark. • Configure and manage Databricks Unity Catalog for governance, lineage, access control, and security. • Develop and optimize Delta Lake tables following the Medallion architecture (Bronze, Silver, Gold). • Orchestrate ETL workflows via Azure Data Factory (ADF) and Databricks Workflows. • Implement data access controls, account-level permissions, and object ownership via Unity Catalog. • Collaborate with architects, developers, and business stakeholders to ensure data quality, lineage, and governance. • Troubleshoot and optimize ETL pipelines for performance, scalability, and cost efficiency. • Support CI/CD integration using Databricks Repos, Git, and Azure DevOps. Skills • 10+ years of experience in data engineering, including cloud migration projects. • Hands-on expertise in Azure Databricks, ADF, ADLS Gen2, and Databricks Unity Catalog. • Very good knowledge of SQL DB Stored procs, SSIS, SSRS. • Proven experience in migrating SQL stored procedures, views, and business logic to Databricks Unity Catalog stored procedures. • Strong skills in Databricks SQL, PySpark, and Python for data transformation. • Experience designing parameter-driven/config-based ETL frameworks. • Deep understanding of Delta Lake, Medallion architecture, and data modeling. • Knowledge of data governance, lineage tracking, and security policies. • Experience with CI/CD automation, Git, and Azure DevOps. Benefits • Health, dental, vision, life insurance, and disability plans • 401(k) Plan after 30 days of employment • 11 paid holidays • 12 weeks of Parental Leave • Free time PTO policy Company Overview • Fractal is an AI firm with the aspiration to power every human decision in theenterprise. It was founded in 2000, and is headquartered in New York, New York, USA, with a workforce of 1001-5000 employees. Its website is Company H1B Sponsorship • Fractal has a track record of offering H1B sponsorships, with 54 in 2025, 51 in 2024, 55 in 2023, 77 in 2022, 51 in 2021, 49 in 2020. Please note that this does not guarantee sponsorship for this specific role. Apply tot his job