Tool-Enabled Accuracy Evaluation

Score a CalculatorTools-equipped agent on computing 10! against the expected 3628800.

Demonstrates accuracy evaluation for an agent using calculator tools.

accuracy_with_tools.py
"""
Tool-Enabled Accuracy Evaluation
================================

Demonstrates accuracy evaluation for an agent using calculator tools.
"""

from typing import Optional

from agno.agent import Agent
from agno.eval.accuracy import AccuracyEval, AccuracyResult
from agno.models.openai import OpenAIChat
from agno.tools.calculator import CalculatorTools

# ---------------------------------------------------------------------------
# Create Evaluation
# ---------------------------------------------------------------------------
evaluation = AccuracyEval(
    name="Tools Evaluation",
    model=OpenAIChat(id="o4-mini"),
    agent=Agent(
        model=OpenAIChat(id="gpt-5.2"),
        tools=[CalculatorTools()],
    ),
    input="What is 10!?",
    expected_output="3628800",
)

# ---------------------------------------------------------------------------
# Run Evaluation
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    result: Optional[AccuracyResult] = evaluation.run(print_results=True)
    assert result is not None and result.avg_score >= 8

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno openai

Export your OpenAI API key

export OPENAI_API_KEY="your_openai_api_key_here"

Run the example

Save the code above as accuracy_with_tools.py, then run:

python accuracy_with_tools.py

Full source: cookbook/09_evals/accuracy/accuracy_with_tools.py