[Remote] Lead Data Engineer (Databricks | Python | PySpark)

Note: The job is a remote job and is open to candidates in USA. Staffing Technologies is seeking a Lead Data Engineer to design, build, and lead modern cloud-based data platforms with a strong focus on Databricks, Python, and PySpark. This role combines hands-on engineering with technical leadership, owning architecture decisions, delivery standards, and scalable data solutions.ResponsibilitiesLead the design and delivery of cloud-native data platforms using DatabricksArchitect and implement Lakehouse and Data Warehouse patternsBuild and optimize ETL/ELT pipelines using Python and PySparkEstablish engineering standards , reusable frameworks, and metadata-driven orchestrationReview designs, vet solutions with the team, and lead demos and retros prior to deploymentEnforce data quality, lineage, monitoring, and alerting across pipelinesMentor engineers and provide hands-on technical leadershipPartner with analytics and business teams to align solutions with data and reporting needsSkillsMUST be EST time zoneLead the design and delivery of cloud-native data platforms using DatabricksArchitect and implement Lakehouse and Data Warehouse patternsBuild and optimize ETL/ELT pipelines using Python and PySparkEstablish engineering standards, reusable frameworks, and metadata-driven orchestrationReview designs, vet solutions with the team, and lead demos and retros prior to deploymentEnforce data quality, lineage, monitoring, and alerting across pipelinesMentor engineers and provide hands-on technical leadershipPartner with analytics and business teams to align solutions with data and reporting needs~15 years of total experience in data or software engineering3+ years in a technical lead role5+ years building cloud-based data platformsProven delivery of production-grade, scalable data systemsExcellent Communication Skills are critical hereHands-on experience with Databricks Notebooks, Jobs, and workload optimizationBuilding pipelines using Lakeflow / Declarative PipelinesData ingestion via Databricks connectorsImplementing data lineage, quality checks, monitoring, and alertingTable, compute, and performance optimization within DatabricksAdvanced Python with strong packaging and dependency managementExpert PySpark for distributed data processingClear understanding of Spark vs single-node executionSpark performance tuning and troubleshootingStrong SQL for mid-to-complex transformationsQuery and data model optimization to reduce compute and improve performanceStrong adherence to SOLID and DRY principlesExperience building parameterized, reusable frameworksAgile/SCRUM delivery experienceGit-based development workflows and code reviewsTesting strategies: unit, integration, and end-to-endCI/CD pipelines and Infrastructure as Code (Terraform)Company OverviewStaffing Technologies is an IT staffing firm. It was founded in 1994, and is headquartered in Alpharetta, Georgia, USA, with a workforce of 201-500 employees. Its website is http://www.staffingtechnologies.com.