Databricks Unity Catalog engineer (Remote)
About the position We are seeking an experienced engineer with 15 to 19 years of experience to join our team. The ideal candidate will have expertise in Databricks Unity Catalog, Python, Databricks SQL, Databricks Workflows, and PySpark. Experience in Mobility-T&H Loyalty partner integration and Loyalty cards & membership management is a plus. This is a hybrid role with day shifts and no travel required. Responsibilities • Lead the design and implementation of data solutions using Databricks Unity Catalog ensuring data governance and security. • Oversee the development and optimization of data pipelines using Python and PySpark to ensure efficient data processing. • Provide expertise in Databricks SQL to create and manage complex queries and data models. • Develop and maintain Databricks Workflows to automate data integration and processing tasks. • Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. • Ensure the scalability and reliability of data solutions to meet the growing needs of the organization. • Conduct code reviews and provide guidance to junior team members to maintain high coding standards. • Monitor and troubleshoot data workflows to ensure smooth and uninterrupted operations. • Stay updated with the latest industry trends and technologies to continuously improve data solutions. • Work closely with stakeholders to ensure data solutions align with business objectives and deliver value. • Provide technical leadership and mentorship to the team fostering a culture of continuous learning and improvement. • Ensure compliance with data privacy and security regulations in all data solutions. • Contribute to the development of best practices and standards for data engineering within the organization. Requirements • Must have extensive experience with Databricks Unity Catalog, Python, Databricks SQL, Databricks Workflows, and PySpark. • Should have a strong understanding of data governance and security principles. • Must be proficient in developing and optimizing data pipelines. • Should have experience in creating and managing complex queries and data models. • Must have excellent problem-solving and troubleshooting skills. • Should have strong communication and collaboration skills. • Must be able to work independently and as part of a team. • Should have a proactive approach to learning and staying updated with industry trends. • Must have a strong attention to detail and a commitment to delivering high-quality solutions. • Should have experience in mentoring and guiding junior team members. • Must have a strong understanding of data privacy and security regulations. • Should have a proven track record of delivering successful data solutions. Nice-to-haves • Experience in Mobility-T&H Loyalty partner integration and Loyalty cards & membership management. Benefits • Medical/Dental/Vision/Life Insurance • Paid holidays plus Paid Time Off • 401(k) plan and contributions • Long-term/Short-term Disability • Paid Parental Leave • Employee Stock Purchase Plan Apply tot his job