Senior Data Engineer/US Citizen/Minnesota- Hybrid
At Enable Data Incorporated, we are excited to welcome a talented Senior Data Engineer to join our dynamic team! With our extensive knowledge in application, data, and cloud engineering services, we strive to create groundbreaking solutions that provide real value to our clients.
Key Responsibilities
ā¢ Design, develop, and maintain scalable data pipelines using Apache Spark (PySpark and/or Scala)
ā¢ Build and optimize data workflows on Databricks, including Delta Lake, notebooks, and scheduled jobs
ā¢ Ingest, transform, and curate large-scale structured and semi-structured datasets
ā¢ Perform performance tuning and cost optimization of Spark workloads and Databricks clusters
ā¢ Implement data quality checks, monitoring, and error handling
ā¢ Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
ā¢ Support batch processing and, where applicable, streaming data pipelines
ā¢ Follow best practices for testing, documentation, security, and version control
Required Qualifications
ā¢ 5+ years of experience in Data Engineering or related roles
ā¢ Strong hands-on experience with Apache Spark (PySpark or Scala)
ā¢ Proven experience working in Databricks environments
ā¢ Strong SQL skills and experience with relational and analytical databases
ā¢ Experience building and maintaining ETL/ELT pipelines at scale
ā¢ Familiarity with modern data lake architectures (Delta Lake preferred)
ā¢ Experience with Git-based version control
Additional Requirements
ā¢ Must be a U.S. Citizen
ā¢ Must be currently located in Minnesota (MN), or willing to relocate
Apply tot his job
Apply To this Job
Key Responsibilities
ā¢ Design, develop, and maintain scalable data pipelines using Apache Spark (PySpark and/or Scala)
ā¢ Build and optimize data workflows on Databricks, including Delta Lake, notebooks, and scheduled jobs
ā¢ Ingest, transform, and curate large-scale structured and semi-structured datasets
ā¢ Perform performance tuning and cost optimization of Spark workloads and Databricks clusters
ā¢ Implement data quality checks, monitoring, and error handling
ā¢ Collaborate with analytics and business stakeholders to deliver well-modeled, analytics-ready data
ā¢ Support batch processing and, where applicable, streaming data pipelines
ā¢ Follow best practices for testing, documentation, security, and version control
Required Qualifications
ā¢ 5+ years of experience in Data Engineering or related roles
ā¢ Strong hands-on experience with Apache Spark (PySpark or Scala)
ā¢ Proven experience working in Databricks environments
ā¢ Strong SQL skills and experience with relational and analytical databases
ā¢ Experience building and maintaining ETL/ELT pipelines at scale
ā¢ Familiarity with modern data lake architectures (Delta Lake preferred)
ā¢ Experience with Git-based version control
Additional Requirements
ā¢ Must be a U.S. Citizen
ā¢ Must be currently located in Minnesota (MN), or willing to relocate
Apply tot his job
Apply To this Job