Data Engineer (20260002)
Responsibility
Develop and maintain ETL processes to extract, transform, and load data from various sources to target data repositories.Work closely with the Data Architect to understand data models and ensure that ETL processes align with the overall data architecture.Monitor and maintain data infrastructure to ensure data quality and accuracy, including implementing data security and privacy policies.Provide technical support to end-users to help them access and utilize data effectively.Work collaboratively with other members of the data team to ensure the effective and efficient use of data within the organization.Develop and maintain documentation of ETL processes and data infrastructure
Qualification
Strong SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working with a variety of databases.Experience with dbt (data build tool) for data transformation, testing, and documentation.Experience building and optimizing 'big data' data pipelines, architectures and data sets on Databricks platform.Build processes supporting data transformation, data structures, metadata, dependency and workload management.A successful history of manipulating, processing and extracting value from large disconnected datasets.Experience supporting and working with cross-functional teams in a dynamic environment.Experience with object-oriented/object function scripting languages: Python, Java, Scala, C#, etc.Experience with Apache Airflow for workflow orchestration and pipeline monitoring.Experience in using Databricks, PySpark for big data processing.Previous work designing/managing data structures within a data warehouse.Experience in data stream processing and its processor is a plus.Experience developing application eg. web application, mobile application, REST web services is a plus.
Apply Now
Develop and maintain ETL processes to extract, transform, and load data from various sources to target data repositories.Work closely with the Data Architect to understand data models and ensure that ETL processes align with the overall data architecture.Monitor and maintain data infrastructure to ensure data quality and accuracy, including implementing data security and privacy policies.Provide technical support to end-users to help them access and utilize data effectively.Work collaboratively with other members of the data team to ensure the effective and efficient use of data within the organization.Develop and maintain documentation of ETL processes and data infrastructure
Qualification
Strong SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working with a variety of databases.Experience with dbt (data build tool) for data transformation, testing, and documentation.Experience building and optimizing 'big data' data pipelines, architectures and data sets on Databricks platform.Build processes supporting data transformation, data structures, metadata, dependency and workload management.A successful history of manipulating, processing and extracting value from large disconnected datasets.Experience supporting and working with cross-functional teams in a dynamic environment.Experience with object-oriented/object function scripting languages: Python, Java, Scala, C#, etc.Experience with Apache Airflow for workflow orchestration and pipeline monitoring.Experience in using Databricks, PySpark for big data processing.Previous work designing/managing data structures within a data warehouse.Experience in data stream processing and its processor is a plus.Experience developing application eg. web application, mobile application, REST web services is a plus.
Apply Now