Storage-Backed Response Performance Evaluation

PerformanceEval invokes the benchmark function separately for runtime and memory measurement, producing four model requests total.

run_agent() sends two model requests. PerformanceEval invokes it once for runtime and once for memory, so the evaluation sends four requests total.

PerformanceEval runs the callable separately for each enabled metric: warm-ups first, then runtime measurements, then memory measurements. num_iterations applies to each metric, and the default is 10 additional warm-up calls. Model retries, tools, delegation, and memory extraction can add provider requests beyond the callable count.

Memory measurements use Python’s tracemalloc; they do not measure process RSS, GPU memory, database-server memory, or remote model memory. Record dependency versions, database state, and model settings when comparing results.

response_with_storage.py
"""
Storage-Backed Response Performance Evaluation
==============================================

Demonstrates measuring performance when storage-backed history is enabled.
"""

from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.eval.performance import PerformanceEval
from agno.models.openai import OpenAIChat

# ---------------------------------------------------------------------------
# Create Database
# ---------------------------------------------------------------------------
db = SqliteDb(db_file="tmp/storage.db")


# ---------------------------------------------------------------------------
# Create Benchmark Function
# ---------------------------------------------------------------------------
def run_agent():
    agent = Agent(
        model=OpenAIChat(id="gpt-5.2"),
        system_message="Be concise, reply with one sentence.",
        db=db,
        add_history_to_context=True,
    )
    response_1 = agent.run("What is the capital of France?")
    print(response_1.content)

    response_2 = agent.run("How many people live there?")
    print(response_2.content)

    return response_2.content


# ---------------------------------------------------------------------------
# Create Evaluation
# ---------------------------------------------------------------------------
response_with_storage_perf = PerformanceEval(
    name="Storage Performance",
    func=run_agent,
    num_iterations=1,
    warmup_runs=0,
)

# ---------------------------------------------------------------------------
# Run Evaluation
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    response_with_storage_perf.run(print_results=True, print_summary=True)

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno openai sqlalchemy

Export your OpenAI API key

export OPENAI_API_KEY="your_openai_api_key_here"

Run the example

Save the code above as response_with_storage.py, then run:

python response_with_storage.py

Full source: cookbook/09_evals/performance/response_with_storage.py