Databricks Engineer
Overview:Our client is implementing Databricks from the ground up and seeking mid‑to senior‑level engineers who can guide best practices, platform adoption, and governance. This is a greenfield environment, ideal for consultants who want to shape architecture and standards from day one. You will collaborate with Palantir teams and help establish a modern data ecosystem that includes Unity Catalog and emerging AI capabilities. What you’ll do:Lead Databricks platform build‑out. Define and implement foundational structures for catalogs, schemas, and data assets; set up permissions and governance in Unity Catalog.Establish best practices. Advise on ingestion, modeling, and orchestration patterns for practical applications in a new Databricks environment.Enable AI features. Explore and prototype Databricks AI capabilities (Mosaic AI, AI/BI Genie, DBRX, foundation models) to inform future adoption.Collaborate cross‑platform. Work with Palantir engineers and Snowflake teams to ensure seamless integration and governance alignment.Document and coach. Create playbooks, standards, and knowledge artifacts to accelerate internal adoption. Required qualifications:Databricks expertise: Hands‑on experience with Databricks platform components (Workspaces, Clusters, Jobs, SQL Warehouses) and Unity Catalog governance.Data Engineering: Strong SQL/PySpark skills; experience designing ingestion and transformation pipelines; familiarity with CI/CD and orchestration tools.Governance: Practical knowledge of catalog structures, asset registration, and permission models; prior involvement in a Unity Catalog implementation is highly preferred.Adaptability: Comfort working in a greenfield environment with evolving requirements; ability to set direction amid uncertainty.Bachelor’s degree in Data Science, Statistics, Mathematics, Business Analytics, or a related field, or equivalent professional experience. CertificationsDatabricks certifications (Lakehouse Fundamentals, Data Engineer Associate/Professional) are a strong plus; willingness to obtain certification post‑hire is expected. Apply tot his job