Cloud Infrastructure Architect (Databricks)
This is a remote position.Role ObjectiveLead the architectural design and automation of a Databricks 2.0 ecosystem, focusing on scalable Infrastructure-as-Code (IaaC), multi-environment governance, and decentralized CI/CD orchestration. Core Responsibilities
Infrastructure Design: Architect Databricks 2.0 environments (Dev/UAT/Prod) using Terraform, managing networking, compute quotas, and cluster policies.
Security & Governance: Define RBAC frameworks and Unity Catalog strategies. Manage Microsoft Entra ID integrations, Service Principals, and automated key rotations.
CI/CD Orchestration: Build automated pipelines for promoting jobs, notebooks, and workflows, enabling independent team deployments via approval-gate processes.
Environment Parity: Design automated data refresh workflows to sync production data to lower environments for consistent testing.
Technical Stack
Platform: Databricks 2.0 (Unity Catalog, Workflows, DLT).
IaC/DevOps: Terraform, GitHub Actions/Azure DevOps, Git.
Identity: Microsoft Entra ID (Service Principals/RBAC).
Data: Delta Lake, SQL, Python.
Key Requirements
Architectural Expertise: Proven track record in designing secure, scalable Data Lakehouses.
Automation-First Mindset: Experience replacing manual setups with robust, version-controlled infrastructure.
Leadership: Ability to bridge the gap between Infrastructure (Infra) and Data Engineering (DE) teams.
Apply Now