Data Pipeline Engineer – SQL Expert
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Software Engineer II
Type: Full-Time
Compensation: $70–$95/hour
Location: Remote
Role Responsibilities
• Drive technical implementation and testing of complex data pipeline features within the SPECTRA platform. Ensure robust end-to-end functionality and meet project release timelines.
• Design and develop data processing pipelines using SQL and Python. Handle large-scale data collection, transformation, and quality monitoring workflows.
• Build and maintain dashboards and visualization tools. Provide visibility into data quality, rater performance, and platform metrics for stakeholders across multiple teams.
• Create comprehensive end-to-end tests and monitoring alerts. Ensure platform reliability, data integrity, and proactive identification of issues in the data collection workflow.
• Debug and troubleshoot complex data flow issues across the SPECTRA platform. Identify root causes and implement solutions to maintain high system availability.
• Collaborate closely with Technical Leads and engineering teams. Implement backend data processing logic and ensure adherence to development standards and best practices.
Qualifications
Must-Have
• Experience in SQL development writing complex queries, data transformations, and working with large datasets in production environments.
• Proficient in Python with experience in data processing, automation, and pipeline development using object-oriented programming principles.
• Experience with data engineering pipelines including ETL processes, data modeling, and building scalable data processing workflows.
• Hands-on experience with data visualization and dashboard tools such as Tableau or similar platforms.
• Strong analytical and problem-solving skills with demonstrated ability to debug complex data issues and implement effective solutions.
• Experience with data quality monitoring and automated testing frameworks.
Preferred
• Degree in Computer Science or related field.
• Machine Learning experience with understanding of ML model training data requirements, feature engineering, and model evaluation metrics.
• Experience building and scaling large products or systems.
• Experience with building privacy and ads-related products.
• Experience working with human labeling and annotation systems including workflow design, quality assurance, and vendor management in data collection operations.
• Prior experience working with the company or its internal tools.
Compensation & Legal
• Pursuant to the California Fair Chance Act, Los Angeles County Fair Chance Ordinance for Employers, Los Angeles Fair Chance Initiative for Hiring Ordinance, and San Francisco Fair Chance Ordinance, qualified applicants will be considered for assignment with arrest and conviction records.
Application Process (Takes 20–30 mins to complete)
• Upload resume
• AI interview based on your resume
• Submit form
Resources & Support
• For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
• For any help or support, reach out to: [email protected]
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Apply Now
Apply Now
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Software Engineer II
Type: Full-Time
Compensation: $70–$95/hour
Location: Remote
Role Responsibilities
• Drive technical implementation and testing of complex data pipeline features within the SPECTRA platform. Ensure robust end-to-end functionality and meet project release timelines.
• Design and develop data processing pipelines using SQL and Python. Handle large-scale data collection, transformation, and quality monitoring workflows.
• Build and maintain dashboards and visualization tools. Provide visibility into data quality, rater performance, and platform metrics for stakeholders across multiple teams.
• Create comprehensive end-to-end tests and monitoring alerts. Ensure platform reliability, data integrity, and proactive identification of issues in the data collection workflow.
• Debug and troubleshoot complex data flow issues across the SPECTRA platform. Identify root causes and implement solutions to maintain high system availability.
• Collaborate closely with Technical Leads and engineering teams. Implement backend data processing logic and ensure adherence to development standards and best practices.
Qualifications
Must-Have
• Experience in SQL development writing complex queries, data transformations, and working with large datasets in production environments.
• Proficient in Python with experience in data processing, automation, and pipeline development using object-oriented programming principles.
• Experience with data engineering pipelines including ETL processes, data modeling, and building scalable data processing workflows.
• Hands-on experience with data visualization and dashboard tools such as Tableau or similar platforms.
• Strong analytical and problem-solving skills with demonstrated ability to debug complex data issues and implement effective solutions.
• Experience with data quality monitoring and automated testing frameworks.
Preferred
• Degree in Computer Science or related field.
• Machine Learning experience with understanding of ML model training data requirements, feature engineering, and model evaluation metrics.
• Experience building and scaling large products or systems.
• Experience with building privacy and ads-related products.
• Experience working with human labeling and annotation systems including workflow design, quality assurance, and vendor management in data collection operations.
• Prior experience working with the company or its internal tools.
Compensation & Legal
• Pursuant to the California Fair Chance Act, Los Angeles County Fair Chance Ordinance for Employers, Los Angeles Fair Chance Initiative for Hiring Ordinance, and San Francisco Fair Chance Ordinance, qualified applicants will be considered for assignment with arrest and conviction records.
Application Process (Takes 20–30 mins to complete)
• Upload resume
• AI interview based on your resume
• Submit form
Resources & Support
• For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
• For any help or support, reach out to: [email protected]
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Apply Now
Apply Now