AI Agent Evaluation Analyst for Autonomous Agents (No coding required)
We’re hiring detail-oriented, analytical contributors to help test and improve autonomous AI agent evaluations. This is part-time, fully remote work with flexible hours, ideal for people who enjoy finding edge cases, questioning assumptions, and strengthening complex systems. What you’ll do • Review and refine agent evaluation tasks and scenarios for logic, completeness, and realism • Identify inconsistencies, ambiguities, and missing assumptions • Define gold-standard expected behaviors for agents • Annotate reasoning paths, cause-effect relationships, and plausible alternatives • Collaborate with QA, writers, and developers to suggest refinements and expand edge case coverage • Ensure autonomous agents are tested thoroughly and realistically What we’re looking for • Strong analytical thinking and excellent attention to detail • Fluent written English with clear documentation skills • Comfort reading structured formats such as JSON or YAML (no need to write code) • Ability to reason about complex systems and spot what could break or be misinterpreted Nice to have Prior exposure to QA/test-case thinking, logic puzzles, or evaluation frameworks Apply tot his job