Azure Databricks Architect (Remote)
About the position Responsibilities • Own the design and implementation of scalable data solutions using Azure Synapse SQL Pool and Azure Synapse Spark Pool. • Supervise the development and optimization of SQL queries to ensure high performance and reliability. • Guide in PySpark to develop and maintain data processing pipelines. • Collaborate with multi-functional teams to understand business requirements and translate them into technical solutions. • Ensure data integrity and security by implementing best practices and standards. • Monitor and fix data systems to identify and resolve issues promptly. • Develop and maintain documentation for data architecture processes and workflows. • Mentor and guide junior team members to improve their technical skills and knowledge. • Stay updated with the latest industry trends and technologies to continuously improve data solutions. • Participate in code reviews to ensure code quality and alignment to standards. • Work closely with collaborators to deliver high-quality data solutions that meet business needs. • Contribute to the overall data strategy and roadmap of the company. • Drive innovation and efficiency in data management practices. Requirements • 12+ years of experience in data engineering and cloud platform management • Solid grasp of Databricks, Python/Pyspark, Azure Synapse Analytics, Azure Data Factory, Databricks Notebooks, Delta Lake and Azure services • Expertise in developing ETL pipelines and optimizing workflows using Databricks • Sophisticated skills in programming languages such as Python or SQL • Hands-on experience with distributed systems and data orchestration tools • In-depth knowledge of cloud architecture, including data storage, security, and scalability • Ability to troubleshoot sophisticated data migration challenges and performance bottlenecks • Experience with CI/CD pipelines, version control, and automation • Strong leadership and interpersonal skills, with the ability to handle multi-functional teams and communicate technical concepts to non-technical collaborators. Nice-to-haves • Experience in SC Intelligence and Planning within the Consumer Goods domain Benefits • Paid parental leave • Employee stock purchase plan • Paid holidays • Disability insurance • Health insurance • Dental insurance • 401(k) • Paid time off • Vision insurance • Life insurance Apply tot his job