Amazon AGI
Applied Scientist
- Build benchmarks and RL gym environments that test AI agents on realistic tasks.
- Evaluate reasoning models and responsible AI (RAI) behavior.
- Develop LLM-as-a-judge systems: judge design, evaluation rubrics, and automated quality assessment.
- Design quality frameworks for training and evaluation data: auditing strategies, sampling methods, and quality metrics.