Portrait of Zahir Aghayev

Zahir Aghayev

Applied Scientist, Amazon AGI

I help AI earn trust: careful tests of what models and agents get right, and what they still get wrong.

For the full experience, use a laptop.

Move your mouse. Click anywhere to send a ripple.Tap anywhere to send a ripple.

About

At Amazon AGI, I work on a question that sounds simple and is hard to answer well: how good is this model, really? I build benchmarks and agent evaluations, design LLM judges and scoring rubrics, and make sure the data behind them can be trusted.

I came to AI through engineering. My PhD combined optimization, reinforcement learning, and physics-informed models to make complex energy and environmental systems work better.

Now
Measuring what AI can doApplied Scientist, Amazon AGI
Before
Teaching AI to optimize real systemsPhD, University of Connecticut

Research areas

PhD research philosophy

How do we make optimal decisions when problems are too constrained to optimize, too expensive to solve, and too urgent to delay?

  • Learn what's possible

    A reinforcement learning algorithm that learns to identify feasible regions on the fly and uses that knowledge to guide the optimization search.

  • Leverage what's known

    Physics-informed machine learning that pairs the speed of data-driven models with the reliability of physics.

  • Predict what matters

    Pipelines that turn high-dimensional, noisy data into clear, actionable predictions.

Five areas, one point cloud. Hover an area to find it in 3D.Tap an area to find it in 3D.

Experience

Where I am now, and the path that led here.

  1. Amazon AGI

    Applied Scientist

    May 2026 to presentBoston, MA

    • Build benchmarks and RL gym environments that test AI agents on realistic tasks.
    • Evaluate reasoning models and responsible AI (RAI) behavior.
    • Develop LLM-as-a-judge systems: judge design, evaluation rubrics, and automated quality assessment.
    • Design quality frameworks for training and evaluation data: auditing strategies, sampling methods, and quality metrics.
  2. University of Connecticut

    Graduate Research Assistant

    May 2022 to May 2026Storrs, CT

    • Developed ORACLE, a reinforcement learning framework that learns feasible regions with classifier-based constraint handling while optimizing.
    • Applied PINNs and neural operators (PI-DeepONet) inside optimization frameworks.
    • Curated and analyzed biological datasets, training predictive models to 98.4% accuracy.
    • Mentored 4 students; co-authored 9 peer-reviewed publications.
  3. Wärtsilä

    Data Science Intern

    June 2024 to August 2024Herndon, VA

    • Built weak-cell detection with time-series analysis for battery systems inside Wärtsilä's GEMS Digital Energy Platform.
    • Quantified energy gains from weak-cell replacement, up to 0.5% unlockable capacity.
    • Designed and deployed module-level anomaly detection; contributed to the GEMS Explorer platform.
    • Led end-to-end model deployment from prototype to production with R&D, product, and customer teams.
  4. The Dow Chemical Company

    Data Science Intern

    May 2023 to August 2023Freeport, TX

    • Curated and standardized 300+ historical formulations into an analytics-ready dataset.
    • Built and validated proof-of-concept ML models for formulation stability, over 90% accuracy.
    • Ran SHAP analysis for interpretability; built Power BI dashboards and presented to stakeholders.
  5. University of Connecticut

    Graduate Teaching Assistant

    August 2021 to May 2022Storrs, CT

    • Helped teach classes of up to 70 students in Applied Machine Learning, Unit Operations, and Process Dynamics and Control.
    • Received the Outstanding Teaching Assistant Award.

Skills

  • LLM evaluation
  • Machine learning
  • Deep learning
  • Reinforcement learning
  • Optimization
  • Physics-informed machine learning
  • Surrogate modeling
  • Statistics
  • Time-series analysis
  • Design of experiments

  • Benchmarking
  • Agent evaluation
  • LLM-as-a-judge
  • Responsible AI
  • Reasoning-model evaluation
  • Predictive modeling
  • Process optimization
  • Anomaly detection
  • Algorithm development
  • Environmental modeling

  • Python
  • PyTorch
  • TensorFlow
  • SQL
  • Amazon Bedrock
  • Pyomo
  • scikit-learn
  • Pandas
  • PySpark
  • GAMS
  • MATLAB

Languages

  • English fluent
  • Azerbaijani native
  • Turkish native
  • Russian basic

Interests

  • Puzzles and logic problems
  • Documentaries
  • Strategy games

Say hi.

Email is the fastest way to reach me.