Big Data Engineer || Remote || W2 & C2C
Job Description :
We are seeking an experienced Big Data Engineer (EL3 Level) with strong expertise in Apache Spark and Scala to design, develop, and optimize large-scale data processing solutions in the Healthcare domain. The ideal candidate will work on building scalable data pipelines, integrating diverse healthcare datasets (claims, EMR/EHR, provider, member data), and enabling analytics and reporting solutions while ensuring HIPAA compliance and data security.
This is a fully remote opportunity supporting enterprise healthcare data platforms.
Key Responsibilities
Big Data Engineering
• Design and develop distributed data processing pipelines using Apache Spark with Scala.
• Build batch and real-time data pipelines using Spark Core, Spark SQL, and Spark Streaming.
• Optimize Spark jobs for performance tuning (partitioning, caching, broadcast joins, memory management).
Healthcare Data Integration
• Process and transform healthcare datasets including:
• Claims data (837/835)
• EHR/EMR data
• Member & Provider data
• HL7/FHIR formats
• Ensure data quality, validation, and compliance with healthcare regulations (HIPAA).
Cloud & Data Platform
• Work on cloud-based big data platforms (AWS/Azure/Google Cloud Platform).
• Develop data pipelines using:
• Data lakes (S3/ADLSS)
• Hive/Delta Lake/Iceberg
• Kafka for streaming
• Implement CI/CD for data pipelines.
Data Modeling & Warehousing
• Design scalable data models (star/snowflake schema).
• Implement ETL/ELT frameworks.
• Support analytics and reporting teams with optimized datasets.
Governance & Security
• Implement data masking, encryption, and PHI protection strategies.
• Collaborate with compliance teams to ensure regulatory standards.
Required Qualifications
• 12+ years of IT experience in Big Data Engineering.
• Strong hands-on experience in:
• Apache Spark
• Scala
• Experience with distributed data processing and cluster tuning.
• Strong SQL knowledge and data modeling experience.
• Experience working with healthcare datasets.
• Experience with:
• Hadoop ecosystem (Hive, HDFS)
• Kafka
• Airflow
• Cloud experience (AWS/Azure/Google Cloud Platform).
Preferred Qualifications
• Experience with:
• Databricks
• Delta Lake
• Experience implementing Data Lakehouse architecture.
• Knowledge of FHIR/HL7 standards.
• Experience in real-time healthcare analytics.
• Certification in AWS/Azure Data Engineering.
Apply tot his job
Apply To this Job
We are seeking an experienced Big Data Engineer (EL3 Level) with strong expertise in Apache Spark and Scala to design, develop, and optimize large-scale data processing solutions in the Healthcare domain. The ideal candidate will work on building scalable data pipelines, integrating diverse healthcare datasets (claims, EMR/EHR, provider, member data), and enabling analytics and reporting solutions while ensuring HIPAA compliance and data security.
This is a fully remote opportunity supporting enterprise healthcare data platforms.
Key Responsibilities
Big Data Engineering
• Design and develop distributed data processing pipelines using Apache Spark with Scala.
• Build batch and real-time data pipelines using Spark Core, Spark SQL, and Spark Streaming.
• Optimize Spark jobs for performance tuning (partitioning, caching, broadcast joins, memory management).
Healthcare Data Integration
• Process and transform healthcare datasets including:
• Claims data (837/835)
• EHR/EMR data
• Member & Provider data
• HL7/FHIR formats
• Ensure data quality, validation, and compliance with healthcare regulations (HIPAA).
Cloud & Data Platform
• Work on cloud-based big data platforms (AWS/Azure/Google Cloud Platform).
• Develop data pipelines using:
• Data lakes (S3/ADLSS)
• Hive/Delta Lake/Iceberg
• Kafka for streaming
• Implement CI/CD for data pipelines.
Data Modeling & Warehousing
• Design scalable data models (star/snowflake schema).
• Implement ETL/ELT frameworks.
• Support analytics and reporting teams with optimized datasets.
Governance & Security
• Implement data masking, encryption, and PHI protection strategies.
• Collaborate with compliance teams to ensure regulatory standards.
Required Qualifications
• 12+ years of IT experience in Big Data Engineering.
• Strong hands-on experience in:
• Apache Spark
• Scala
• Experience with distributed data processing and cluster tuning.
• Strong SQL knowledge and data modeling experience.
• Experience working with healthcare datasets.
• Experience with:
• Hadoop ecosystem (Hive, HDFS)
• Kafka
• Airflow
• Cloud experience (AWS/Azure/Google Cloud Platform).
Preferred Qualifications
• Experience with:
• Databricks
• Delta Lake
• Experience implementing Data Lakehouse architecture.
• Knowledge of FHIR/HL7 standards.
• Experience in real-time healthcare analytics.
• Certification in AWS/Azure Data Engineering.
Apply tot his job
Apply To this Job