[Remote] 100% Remote-Data Engineer

Note: The job is a remote job and is open to candidates in USA. Beacon Hill is a premier National Information Technology Staffing Group, and they are seeking a Senior Data Engineer to build and support production-grade data pipelines. The role involves migrating existing data assets to Databricks Unity Catalog, optimizing pipelines, and ensuring data quality and reliability.ResponsibilitiesMigrate existing data assets and pipelines to Databricks Unity CatalogRefactor and optimize existing pipelines using modern engineering standards and best practicesBuild and maintain scalable PySpark-based data pipelinesDevelop new source-of-truth datasets and modernize legacy data workflowsDesign and implement workflow orchestration using Databricks JobsDeploy and manage pipelines using Databricks Asset BundlesEnsure data pipelines are reliable, observable, and production-readyImplement data quality controls, monitoring, and operational alertingSupport platform migrations, testing, validation, and production cutoversPartner with engineering teams to improve platform stability, scalability, and data reliabilitySkillsStrong hands-on experience building and supporting production-grade data pipelines using PySpark and Apache SparkExtensive experience working within Databricks environmentsExperience with Databricks Unity Catalog migrations and modernization initiativesExperience with Databricks Unity Catalog governance and security modelsStrong understanding of ETL/ELT design, data modeling, and incremental data processingExperience modernizing legacy data platforms and data migration projectsExperience working with large-scale datasets in cloud-based data platformsExperience with workflow orchestration, scheduling, and pipeline automationKnowledge of cloud data platforms such as Snowflake and/or Google CloudExperience implementing data quality checks, monitoring, logging, and alertingStrong troubleshooting, collaboration, and problem-solving skillsFamiliarity with advertising, analytics, measurement, or large-scale data ecosystemsExperience with CI/CD and DataOps best practices for data engineering environmentsBenefitsMedical, dental, vision, and federal and state leave programs as required by applicable agency regulations to those that meet eligibilityUpon successfully being hired, details will be provided related to our benefit offeringsCompany OverviewBeacon Hill offers business consulting services. It was founded in 2000, and is headquartered in Boston, Massachusetts, USA, with a workforce of 1001-5000 employees. Its website is https://bhsg.com.