Lead Data Engineer/Databricks
Job Summary: We are seeking a Lead Data Engineer to spearhead our cloud data initiatives, combining hands-on technical expertise with strong leadership. The ideal candidate will drive the design, development, and deployment of robust data solutions in Microsoft Azure, leveraging tools like Databricks, Unity Catalog, Asset Bundles, and Azure Data Factory. This is a key role responsible for delivering scalable, secure, and high-performance data infrastructure. Key Responsibilities: • Execute the migration of legacy Databricks environments to Unity Catalog. • Lead and mentor a team of data engineers, providing technical direction and support. • Architect, develop, and manage scalable data pipelines and integrations on the Azure cloud platform. • Design and implement solutions using Azure Data Factory, Databricks, and Unity Catalog. • Enforce data governance, security, and compliance standards within cloud data environments. • Collaborate cross-functionally with data scientists, analysts, and business stakeholders to understand data needs. • Manage and deploy Asset Bundles to promote re-usability and governance in data workflows. • Monitor, optimize, and troubleshoot large-scale ETL/ELT pipelines for performance and reliability. Required Qualifications: • Proven experience in a Lead Data Engineer or Senior Data Engineer role. • Strong hands-on experience with Azure cloud services, especially in data architecture and data engineering. • Expert-level knowledge in Databricks, including Unity Catalog, Asset Bundles, and legacy migration strategies. • Proficiency with Azure Data Factory for building and orchestrating data workflows. • Deep understanding of data modeling, ETL/ELT processes, and data governance. • Excellent communication skills and experience leading or mentoring technical teams. • Bachelor's or Master's degree in Computer Science, Engineering, or a related field. Preferred Qualifications: • Azure certifications (e.g., Azure Data Engineer Associate). • Experience with CI/CD pipelines and Infrastructure as Code (IaC). • Familiarity with Delta Lake, Spark optimization techniques, and real-time data processing. Job Type: Contract Work Location: Remote Apply tot his job