[Remote] Senior Databricks Engineer

Note: The job is a remote job and is open to candidates in USA. Bounteous is a premier end-to-end digital transformation consultancy dedicated to partnering with ambitious brands to create digital solutions for today’s complex challenges and tomorrow’s opportunities. They are seeking a Senior Databricks Engineer to design, build, and optimize large-scale data and analytics platforms on the Databricks Lakehouse, owning the architecture and delivery of production-grade data pipelines while collaborating with analytics and data science teams.ResponsibilitiesArchitect, build, and maintain scalable ETL/ELT pipelines on the Databricks Lakehouse Platform using PySpark, Spark SQL, and Delta LakeDesign and implement medallion (bronze/silver/gold) data architectures and enforce data quality, governance, and lineage standardsOptimize Spark jobs and cluster configurations for performance and cost, including partitioning, caching, and autoscaling strategiesImplement and manage Unity Catalog for access control, data governance, and cross-workspace asset sharingBuild and orchestrate workflows using Databricks Workflows, Delta Live Tables, and CI/CD pipelinesCollaborate with data scientists, analysts, and business stakeholders to translate requirements into reliable data productsEstablish engineering best practices, conduct code reviews, and mentor junior data engineersMonitor production pipelines, troubleshoot failures, and drive root-cause analysis and continuous improvementSkills5+ years of data engineering experience, with 3+ years building production solutions on Databricks and Apache SparkExpert proficiency in Python (PySpark) and advanced SQLDeep hands-on experience with Delta Lake, Unity Catalog, and the medallion architecture patternStrong experience with at least one major cloud platform (AWS, Azure, or GCP) and its core data servicesProven track record optimizing Spark performance and managing cluster costExperience with data modeling, warehousing concepts, and building dimensional/analytics-ready datasetsProficiency with Git-based version control, CI/CD, and infrastructure-as-codeBachelor's degree in Computer Science, Engineering, or equivalent practical experienceDatabricks certification (Data Engineer Associate/Professional)Experience with Delta Live Tables, structured streaming, and real-time data processingFamiliarity with MLflow and supporting machine learning workflows in productionExperience with orchestration tools (Airflow, dbt) and data observability platformsExposure to data governance, security, and compliance frameworks (e.g., GDPR, HIPAA, SOC 2)Hands-on experience using AI coding assistants (e.g., Claude Code, GitHub Copilot, Cursor) to accelerate development, refactoring, and code reviewFamiliarity with large language model APIs and SDKs (e.g., Anthropic Claude, OpenAI) and prompt engineering for data and analytics use casesExperience integrating GenAI capabilities into data pipelines or applications, including retrieval-augmented generation (RAG) and vector searchAwareness of responsible AI practices, including evaluation, guardrails, and cost/latency trade-offs when deploying LLM-based solutionsCompany OverviewBounteous is a digital experience consultancy that helps companies deliver transformational brand experiences. It was founded in 2003, and is headquartered in Chicago, Illinois, USA, with a workforce of 1001-5000 employees. Its website is http://www.bounteous.com.Company H1B SponsorshipBounteous has a track record of offering H1B sponsorships, with 21 in 2025, 9 in 2024, 5 in 2023, 14 in 2022, 11 in 2021, 18 in 2020. Please note that this does not guarantee sponsorship for this specific role.