Senior Data Engineer - Full Stack (US, Remote)
Job Title: Senior Data Engineer - Full Stack
Experience: 8+Years
Role Summary
We are seeking a highly skilled Senior Data Engineer – Full Stack to build and maintain internal tools, automation frameworks, and workflows that enhance the efficiency, reliability, and scalability of our data and machine learning platforms. This role will work closely with Data Engineers, Data Scientists, and ML Engineers to streamline operations across the data lifecycle.
Key Responsibilities
Design and develop CLI tools, scripts, and internal utilities to automate repetitive tasks across the data platform, including:Pipeline execution and orchestration
Data governance workflows
Metadata synchronization
Environment setup and configuration
Test harness development
Automate workflows on Databricks, including:Job deployment and scheduling
Environment provisioning
MLOps processes using APIs, Terraform, or Databricks SDK
Build and implement robust testing frameworks:Integration testing for pipelines
End-to-end validation of ETL/ELT workflows
Testing and validation for ML inference workflows
Improve overall productivity, scalability, and reliability of the data and ML engineering ecosystem
Develop lightweight internal tools and dashboards using frameworks such as React, Streamlit, or similar technologies to:Visualize data pipelines and workflows
Demonstrate model inference capabilities
Provide configuration and operational controls
Enable internal productivity monitoring and dashboards
Collaborate with cross-functional teams to identify automation opportunities and implement best practices
Required Skills & Qualifications
Strong experience in Python and scripting for automation and backend development
Hands-on experience with Databricks platform and ecosystem
Experience with APIs, Terraform, and/or Databricks SDK for automation
Solid understanding of ETL/ELT pipelines and data platform architecture
Experience building testing frameworks for data pipelines and ML workflows
Familiarity with CLI tool development and system automation
Knowledge of MLOps principles and practices
Experience with modern development practices, including:Spec-driven development
Use of coding agents or automation-assisted development tools
Version control and CI/CD pipelines
Nice to Have
Experience building dashboards or internal tools using React, Streamlit, or similar frameworks
Familiarity with Databricks AI/BI or other data visualization tools
Exposure to data governance and metadata management frameworks
Experience working with cloud platforms (AWS preferred)
Preferred Experience
8+ years of experience in Data Engineering, Platform Engineering, or related roles
Experience working in data-driven or ML-focused environments
What You’ll Bring
Strong problem-solving mindset with a focus on automation and efficiency
Ability to work in a fast-paced, collaborative environment
Passion for building scalable internal tools and improving developer productivity
Experience: 8+Years
Role Summary
We are seeking a highly skilled Senior Data Engineer – Full Stack to build and maintain internal tools, automation frameworks, and workflows that enhance the efficiency, reliability, and scalability of our data and machine learning platforms. This role will work closely with Data Engineers, Data Scientists, and ML Engineers to streamline operations across the data lifecycle.
Key Responsibilities
Design and develop CLI tools, scripts, and internal utilities to automate repetitive tasks across the data platform, including:Pipeline execution and orchestration
Data governance workflows
Metadata synchronization
Environment setup and configuration
Test harness development
Automate workflows on Databricks, including:Job deployment and scheduling
Environment provisioning
MLOps processes using APIs, Terraform, or Databricks SDK
Build and implement robust testing frameworks:Integration testing for pipelines
End-to-end validation of ETL/ELT workflows
Testing and validation for ML inference workflows
Improve overall productivity, scalability, and reliability of the data and ML engineering ecosystem
Develop lightweight internal tools and dashboards using frameworks such as React, Streamlit, or similar technologies to:Visualize data pipelines and workflows
Demonstrate model inference capabilities
Provide configuration and operational controls
Enable internal productivity monitoring and dashboards
Collaborate with cross-functional teams to identify automation opportunities and implement best practices
Required Skills & Qualifications
Strong experience in Python and scripting for automation and backend development
Hands-on experience with Databricks platform and ecosystem
Experience with APIs, Terraform, and/or Databricks SDK for automation
Solid understanding of ETL/ELT pipelines and data platform architecture
Experience building testing frameworks for data pipelines and ML workflows
Familiarity with CLI tool development and system automation
Knowledge of MLOps principles and practices
Experience with modern development practices, including:Spec-driven development
Use of coding agents or automation-assisted development tools
Version control and CI/CD pipelines
Nice to Have
Experience building dashboards or internal tools using React, Streamlit, or similar frameworks
Familiarity with Databricks AI/BI or other data visualization tools
Exposure to data governance and metadata management frameworks
Experience working with cloud platforms (AWS preferred)
Preferred Experience
8+ years of experience in Data Engineering, Platform Engineering, or related roles
Experience working in data-driven or ML-focused environments
What You’ll Bring
Strong problem-solving mindset with a focus on automation and efficiency
Ability to work in a fast-paced, collaborative environment
Passion for building scalable internal tools and improving developer productivity