Operations Research Model Prompt Evaluator
**Role Overview**
We are seeking expert operations research professionals to author and verify high-quality open-ended prompts for AI model evaluation. You will craft and review challenging, unambiguous optimization and decision-science problems across core subdomains, assessing AI reasoning quality and helping establish rigorous evaluation standards for frontier language models.
You will be assigned one of two task types:
- **Authoring Task** — Create 5 original, open-ended prompts from your assigned subdomain at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional). Prompts should require human judgment to evaluate the quality of the AI's response, such as optimization modeling, algorithmic analysis, or stochastic reasoning.
- **Verification Task** — Review 5 authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness. Edit prompts and difficulty ratings where needed.
• *Operations Research Subdomains Covered**
Linear & Integer Programming, Network Optimization & Graph Theory, Stochastic Models & Queuing Theory, Game Theory & Decision Analysis, Supply Chain & Logistics Optimization, Simulation & Metaheuristics.
• *Key Responsibilities**
- Author clear, unambiguous, open-ended operations research prompts that elicit evaluable AI responses
- Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty
- Ensure all 5 prompts in a task are sufficiently distinct from one another with varying difficulty levels
- Apply expert judgment to assess the depth and quality of quantitative reasoning required
- Edit prompts and difficulty assignments where standards are not met
• *Ideal Qualifications**
- Master's degree or higher in Operations Research, Industrial Engineering, Applied Mathematics, or a closely related field
- 2–6 years of professional or research experience in optimization, logistics, or decision science
- Strong command of mathematical programming, probabilistic modeling, and algorithmic methods
- Experience with solvers (Gurobi, CPLEX) or simulation tools is a strong plus
- Excellent written English and ability to craft precise, well-scoped technical questions
• *More About the Opportunity**
- Expected commitment: 10+ hours/week
- Asynchronous, fully remote work
Apply tot his job
Apply To this Job
We are seeking expert operations research professionals to author and verify high-quality open-ended prompts for AI model evaluation. You will craft and review challenging, unambiguous optimization and decision-science problems across core subdomains, assessing AI reasoning quality and helping establish rigorous evaluation standards for frontier language models.
You will be assigned one of two task types:
- **Authoring Task** — Create 5 original, open-ended prompts from your assigned subdomain at varying difficulty levels (undergraduate, advanced undergraduate, or graduate/professional). Prompts should require human judgment to evaluate the quality of the AI's response, such as optimization modeling, algorithmic analysis, or stochastic reasoning.
- **Verification Task** — Review 5 authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness. Edit prompts and difficulty ratings where needed.
• *Operations Research Subdomains Covered**
Linear & Integer Programming, Network Optimization & Graph Theory, Stochastic Models & Queuing Theory, Game Theory & Decision Analysis, Supply Chain & Logistics Optimization, Simulation & Metaheuristics.
• *Key Responsibilities**
- Author clear, unambiguous, open-ended operations research prompts that elicit evaluable AI responses
- Verify prompts are within the scope of the assigned subdomain and correctly rated for difficulty
- Ensure all 5 prompts in a task are sufficiently distinct from one another with varying difficulty levels
- Apply expert judgment to assess the depth and quality of quantitative reasoning required
- Edit prompts and difficulty assignments where standards are not met
• *Ideal Qualifications**
- Master's degree or higher in Operations Research, Industrial Engineering, Applied Mathematics, or a closely related field
- 2–6 years of professional or research experience in optimization, logistics, or decision science
- Strong command of mathematical programming, probabilistic modeling, and algorithmic methods
- Experience with solvers (Gurobi, CPLEX) or simulation tools is a strong plus
- Excellent written English and ability to craft precise, well-scoped technical questions
• *More About the Opportunity**
- Expected commitment: 10+ hours/week
- Asynchronous, fully remote work
Apply tot his job
Apply To this Job