[Remote] Senior Prompt Engineer: LLM Migration & Optimization
Note: The job is a remote job and is open to candidates in USA. Welo Data is seeking a Senior Prompt Engineer to lead the technical migration of template workflows into high-performance LLM autoraters. This role focuses on optimizing model performance using advanced tools and manual refinement to ensure high accuracy and nuance in automated systems.
Responsibilities
ā¢ Take full ownership of the end-to-end technical migration of templates to LLM autoraters
ā¢ Utilize Automatic Prompt Generation (APG) and supervise Automated Prompt Optimization (APO) tools to push model performance past plateaus and deadlocks
ā¢ Continuously measure quality against "gold data" baselines, tracking precision, recall, and $F_1$ scores to justify launch readiness
ā¢ Manually draft and refine complex prompts to overcome anti-patterns and architecture gaps that automated tools can't solve
Skills
ā¢ Bachelor's, Master's, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field
ā¢ 4+ years of experience tuning LLMs for strict, structured outputs, complex classification, and few-shot learning
ā¢ High proficiency in identifying error patterns and using SQL or data analytics tools to monitor performance
ā¢ Fast learner capable of mastering proprietary internal tools and 'Goose API' style interfaces with minimal oversight
ā¢ Familiarity with shadowbot monitoring and disagreement tracking
ā¢ Experience in AI model evaluation and software engineering
ā¢ Deep understanding of semantics, logic, and Chain-of-Thought (CoT) prompting
ā¢ Proven ability to draft high-level Launch Certification Documentation
Company Overview
ā¢ With 27+ years of experience, Welo Data is the human-centered infrastructure for globally effective AI. It was founded in undefined, and is headquartered in , with a workforce of 1001-5000 employees. Its website is https://t.mtrbio.com/WeloData.
Apply tot his job
Apply To this Job
Responsibilities
ā¢ Take full ownership of the end-to-end technical migration of templates to LLM autoraters
ā¢ Utilize Automatic Prompt Generation (APG) and supervise Automated Prompt Optimization (APO) tools to push model performance past plateaus and deadlocks
ā¢ Continuously measure quality against "gold data" baselines, tracking precision, recall, and $F_1$ scores to justify launch readiness
ā¢ Manually draft and refine complex prompts to overcome anti-patterns and architecture gaps that automated tools can't solve
Skills
ā¢ Bachelor's, Master's, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field
ā¢ 4+ years of experience tuning LLMs for strict, structured outputs, complex classification, and few-shot learning
ā¢ High proficiency in identifying error patterns and using SQL or data analytics tools to monitor performance
ā¢ Fast learner capable of mastering proprietary internal tools and 'Goose API' style interfaces with minimal oversight
ā¢ Familiarity with shadowbot monitoring and disagreement tracking
ā¢ Experience in AI model evaluation and software engineering
ā¢ Deep understanding of semantics, logic, and Chain-of-Thought (CoT) prompting
ā¢ Proven ability to draft high-level Launch Certification Documentation
Company Overview
ā¢ With 27+ years of experience, Welo Data is the human-centered infrastructure for globally effective AI. It was founded in undefined, and is headquartered in , with a workforce of 1001-5000 employees. Its website is https://t.mtrbio.com/WeloData.
Apply tot his job
Apply To this Job