[Remote] Senior Data Engineer- Databricks SME

Note: The job is a remote job and is open to candidates in USA. Cloud7Works, Inc. is a small business federal IT contractor specializing in cloud modernization, enterprise architecture, cybersecurity, and advanced analytics for federal civilian agencies. They are seeking two Senior Data Engineers with deep Databricks expertise to support a mission-critical, FedRAMP High-authorized data platform on AWS, focusing on workspace architecture, data governance, and secure AWS integration.ResponsibilitiesAdminister and optimize Databricks workspaces, clusters, and SQL Warehouses via Terraform and GitHub Actions CI/CD pipelinesDesign and maintain medallion pipeline architecture using Delta Live Tables and Auto Loader for large-scale data filtering and transformation use casesGovern data access end-to-end through Unity Catalog — enforce RBAC/ABAC, PII tagging, audit logging, retention policies, and data lineageEnable self-service analytics through Databricks Genie, SQL semantic layers, and materialized views for business consumersIntegrate Databricks with mainframe sources (JCL/COBOL/DB2), streaming platforms (Kafka/Kinesis), CDC tools (Debezium/DMS), and SaaS systemsDrive performance optimization using Photon, Z-ordering, liquid clustering, and rightsizing with continuous cost monitoringMaintain platform observability — data quality expectations, metrics, alerting, and resilient error handlingSupport 24x7 platform operations including proactive monitoring, P1 incident response (initial response within 4 hours), and continuous health checksCollaborate with cross-functional technical teams and government technical leads during normal business hoursMaintain requirements traceability, architecture review artifacts, and records-compliant documentation repositoriesSkills8+ years in cloud technology3–5 years hands-on Databricks (non-negotiable)Unity Catalog — RBAC/ABAC, dynamic row-level security, column masking, tagging, lineage, Delta SharingDelta Live Tables and Auto Loader — medallion architecture design and operations (bronze/silver/gold)Databricks SQL Warehouses and Genie — semantic layer and self-service analytics enablementPerformance optimization — Photon engine, Z-ordering, liquid clusteringWorkspace, cluster, and SQL Warehouse provisioning and administration via Infrastructure as CodeS3, IAM, VPC, PrivateLink, CloudWatch — secure connectivity and observabilityStreaming integrations — Kafka, Kinesis, EventBridge, SQS, SNSCDC and batch ingestion — Debezium, AWS DMS, AWS DataSyncAmazon EKS — Kubernetes workload integration with DatabricksTerraform and GitHub Actions — automated Databricks and AWS provisioningDatabricks Asset Bundles — environment promotion (dev/test/prod)SAST/DAST security scanning, policy-as-code, approval gatesBlue/green and canary deployment strategiesActive Databricks certifications — Databricks Certified Data Engineer ProfessionalActive AWS certifications — Solutions Architect or Data Analytics SpecialtyExperience in federal, regulated, or large-scale tax/financial data environmentsFedRAMP High or FISMA-governed environment experienceClean Rooms and cross-account Delta Sharing configurationInformatica Enterprise Data Catalog (EDC) / Axon metadata and lineage integrationPower BI and Tableau dashboard development on top of Databricks SQLBachelor's degree in Computer Science, Information Technology, or related fieldCompany OverviewCloud7Works is an SBA 8(a)–certified small business that helps federal agencies modernize mission-critical systems through secure cloud, data, AI, and enterprise architecture solutions. It was founded in 2009, and is headquartered in Leesburg, Virginia, US, with a workforce of 11-50 employees. Its website is https://www.cloud7works.com/.