Model Evaluator @ Austin, TX/Sunnyvale, CA- Hybrid -1+yr
REQUIREMENT
Model Evaluator
Project Duration: 1 year, with possible extension based on performance
Location - Austin, TX/Sunnyvale, CA
Work Type - Hybrid ( 3 days office must)
Type of Visa - GC/Citizen - Independent Candidates only
Technical Skills
ā¢ Strong understanding of LLMs, generative AI, and transformer-based architectures.
ā¢ Experience with Python, data analysis, and model evaluation frameworks.
ā¢ Familiarity with prompt engineering, embeddings, RLHF/RLAIF, and LLM-based scoring methods.
ā¢ Experience building evaluation datasets and working with annotation platforms.
ā¢ Understanding of safety alignment, bias detection, and adversarial testing.
ā¢ Tools & Platforms
ā¢ ML/AI frameworks: PyTorch, TensorFlow, HuggingFace, LangChain.
ā¢ Evaluation/annotation tools: Scale AI, GroundTruth, Labelbox, Prodigy.
ā¢ Prompt testing tools: Weights & Biases, MLflow, OpenAI evals, LLM-as-a-judge pipelines.
Thanks & Regards,
John Stanley- Sr. BDM / Delivery Manager
Maintec Technologies Inc
8801 Fast Park Drive, Ste. 301, Raleigh, NC 27617
Mobile: +1 (919) 267-1887 / +91- 98411-45549
Email: [email protected]; www.maintec.in | www.maintec.com
LinkedIn :www.linkedin.com/in/johnstanley1/
Bangalore | Chennai | Hyderabad | Pune | Noida | USA
Apply tot his job
Apply To this Job
Model Evaluator
Project Duration: 1 year, with possible extension based on performance
Location - Austin, TX/Sunnyvale, CA
Work Type - Hybrid ( 3 days office must)
Type of Visa - GC/Citizen - Independent Candidates only
Technical Skills
ā¢ Strong understanding of LLMs, generative AI, and transformer-based architectures.
ā¢ Experience with Python, data analysis, and model evaluation frameworks.
ā¢ Familiarity with prompt engineering, embeddings, RLHF/RLAIF, and LLM-based scoring methods.
ā¢ Experience building evaluation datasets and working with annotation platforms.
ā¢ Understanding of safety alignment, bias detection, and adversarial testing.
ā¢ Tools & Platforms
ā¢ ML/AI frameworks: PyTorch, TensorFlow, HuggingFace, LangChain.
ā¢ Evaluation/annotation tools: Scale AI, GroundTruth, Labelbox, Prodigy.
ā¢ Prompt testing tools: Weights & Biases, MLflow, OpenAI evals, LLM-as-a-judge pipelines.
Thanks & Regards,
John Stanley- Sr. BDM / Delivery Manager
Maintec Technologies Inc
8801 Fast Park Drive, Ste. 301, Raleigh, NC 27617
Mobile: +1 (919) 267-1887 / +91- 98411-45549
Email: [email protected]; www.maintec.in | www.maintec.com
LinkedIn :www.linkedin.com/in/johnstanley1/
Bangalore | Chennai | Hyderabad | Pune | Noida | USA
Apply tot his job
Apply To this Job