Senior Data Engineer with Big Data and Azure Cloud Expertise
We are seeking a highly skilled Data Engineer with strong expertise in Data Engineering, Big Data Technologies, Cloud Infrastructure, and Business Intelligence to design, develop, and maintain scalable data platforms.
The ideal candidate will be responsible for building end-to-end data systems, including data ingestion, transformation, storage, processing, and analytics-ready data warehouses. You will work closely with analytics, product, and engineering teams to ensure reliable and high-performance data infrastructure that powers reporting, dashboards, and advanced analytics.
This role requires experience in modern data architectures, distributed computing systems, real-time streaming pipelines, and cloud-native infrastructure.
Key Responsibilities
Data engineering
Build scalable ETL pipelines and automated workflows.
Use Apache Airflow for orchestration and automation.
Process data using Apache Spark and Hadoop ecosystem.
Implement batch and real-time data processing pipelines.
Perform data cleaning, transformation, and quality validation.
Big data and streaming
Develop distributed processing systems using Apache Spark.
Build real-time pipelines using Kafka, Flink, and Spark Streaming.
Work with HDFS, Data Lakes, Hive, Pig, and HBase.
Data warehousing and databases
Build warehouses using Snowflake, BigQuery, and Amazon Redshift.
Design Star Schema and Snowflake Schema models for analytics.
Optimize queries and indexing.
Work with PostgreSQL and MongoDB while ensuring ACID-compliant transactions.
Cloud and devOps
Deploy and manage infrastructure on Azure (Data Factory, Storage, Data Lakes)
Use Docker, Kubernetes, and Microservices architecture.
Implement Terraform (Infrastructure as Code) and CI/CD pipelines.
Build scalable cloud-native architectures.
Data extraction and automation
Develop data extraction pipelines from APIs, databases, and external sources.
Implement automated data ingestion and web scraping solutions.
Build automated reporting and data synchronization workflows.
Apply tot his job
Apply To this Job
The ideal candidate will be responsible for building end-to-end data systems, including data ingestion, transformation, storage, processing, and analytics-ready data warehouses. You will work closely with analytics, product, and engineering teams to ensure reliable and high-performance data infrastructure that powers reporting, dashboards, and advanced analytics.
This role requires experience in modern data architectures, distributed computing systems, real-time streaming pipelines, and cloud-native infrastructure.
Key Responsibilities
Data engineering
Build scalable ETL pipelines and automated workflows.
Use Apache Airflow for orchestration and automation.
Process data using Apache Spark and Hadoop ecosystem.
Implement batch and real-time data processing pipelines.
Perform data cleaning, transformation, and quality validation.
Big data and streaming
Develop distributed processing systems using Apache Spark.
Build real-time pipelines using Kafka, Flink, and Spark Streaming.
Work with HDFS, Data Lakes, Hive, Pig, and HBase.
Data warehousing and databases
Build warehouses using Snowflake, BigQuery, and Amazon Redshift.
Design Star Schema and Snowflake Schema models for analytics.
Optimize queries and indexing.
Work with PostgreSQL and MongoDB while ensuring ACID-compliant transactions.
Cloud and devOps
Deploy and manage infrastructure on Azure (Data Factory, Storage, Data Lakes)
Use Docker, Kubernetes, and Microservices architecture.
Implement Terraform (Infrastructure as Code) and CI/CD pipelines.
Build scalable cloud-native architectures.
Data extraction and automation
Develop data extraction pipelines from APIs, databases, and external sources.
Implement automated data ingestion and web scraping solutions.
Build automated reporting and data synchronization workflows.
Apply tot his job
Apply To this Job