[Remote] Senior Data Engineer (Streaming & Databricks)

Note: The job is a remote job and is open to candidates in USA. Lumenalta is a technology solutions company that partners with forward-thinking organizations to drive business growth. They are seeking a Senior Data Engineer with expertise in streaming architectures and Databricks to architect and deliver a Lakehouse platform for high-frequency sensor telemetry and complex data pipelines.ResponsibilitiesArchitect and implement data landing zones for streaming telemetry (Striim) and historical relational data (Oracle), enforcing strict asset and handle-level segregation across ingestion layersDesign and deploy end-to-end batch and streaming data pipelines within a Databricks Lakehouse environment, leveraging Delta Live Tables (DLT) for reliable state management and incremental processingDevelop transformation logic to calculate descriptive statistics (min, max, avg, counts) at ingestion time and build advanced Silver and Gold layer pipelines for complex multi-sensor time-alignmentConfigure ETL clusters and SQL Warehouses; establish environment-wide monitoring, spend alerts, and cost projections to manage cloud resource consumption and FinOps accountabilityLead production cutover activities, including environment hardening, catch-up batch processing for historical backfill, and phased loading of historical parametersDefine and enforce data quality standards and pipeline observability practices across all layers of the LakehouseCollaborate with cross-functional stakeholders to translate manufacturing and IoT domain requirements into robust, scalable data architecture decisionsSkills5+ years in data engineering with a proven track record of delivering production-grade pipelines in complex, high-volume enterprise environmentsAdvanced hands-on experience with Databricks—Delta Lake, Delta Live Tables (DLT), SQL Warehouses, and cluster management at scaleProven experience with Striim or equivalent Change Data Capture tools in high-volume manufacturing, IoT, or telemetry contextsStrong proficiency in Python/PySpark and advanced SQL for both pipeline development and complex transformation logicExperience managing high-frequency time-series datasets and performing multi-sensor synchronization across misaligned temporal streamsFamiliarity with cloud cost management practices, including cluster sizing, spend monitoring, and resource optimization within Databricks environmentsComfortable owning go-live activities—environment hardening, cutover planning, and post-deployment stabilization in critical production settingsBenefitsEmployment_type: FULL_TIMEJob_location_type: TELECOMMUTEBe 100% dedicated to one project at a time so that you can innovate and grow.Be a part of a team of talented and friendly senior-level developers.Work on projects that allow you to use leading tech.This is a fully remote position; however, candidates must be based in regions that align with the Pacific, Central, or Eastern U.S. time zones to ensure effective collaboration with client and team schedules.Occasional travel to Georgia will be required.Company OverviewLumenalta is a software and app development company that creates unique digital transformation solutions using innovative technology. It was founded in 2000, and is headquartered in New York, New York, USA, with a workforce of 501-1000 employees. Its website is https://lumenalta.com/.