Memory Update Performance Evaluation

Benchmark agent run latency with update_memory_on_run enabled against a SqliteDb over 5 iterations per metric.

Demonstrates measuring performance when memory updates are enabled.

The evaluator calls run_agent() 10 times: five runtime and five memory measurements, with no warm-ups. Each call also performs memory extraction; timing includes the function’s console output.

PerformanceEval runs the callable separately for each enabled metric: warm-ups first, then runtime measurements, then memory measurements. num_iterations applies to each metric, and the default is 10 additional warm-up calls. Model retries, tools, delegation, and memory extraction can add provider requests beyond the callable count.

Memory measurements use Python’s tracemalloc; they do not measure process RSS, GPU memory, database-server memory, or remote model memory. Record dependency versions, database state, and model settings when comparing results.

response_with_memory_updates.py
"""
Memory Update Performance Evaluation
====================================

Demonstrates measuring performance when memory updates are enabled.
"""

from agno.agent import Agent
from agno.db.sqlite import SqliteDb
from agno.eval.performance import PerformanceEval
from agno.models.openai import OpenAIChat

# ---------------------------------------------------------------------------
# Create Database
# ---------------------------------------------------------------------------
db = SqliteDb(db_file="tmp/memory.db")


# ---------------------------------------------------------------------------
# Create Benchmark Function
# ---------------------------------------------------------------------------
def run_agent():
    agent = Agent(
        model=OpenAIChat(id="gpt-5.2"),
        system_message="Be concise, reply with one sentence.",
        db=db,
        update_memory_on_run=True,
    )

    response = agent.run("My name is Tom! I'm 25 years old and I live in New York.")
    print(f"Agent response: {response.content}")

    return response


# ---------------------------------------------------------------------------
# Create Evaluation
# ---------------------------------------------------------------------------
response_with_memory_updates_perf = PerformanceEval(
    name="Memory Updates Performance",
    func=run_agent,
    num_iterations=5,
    warmup_runs=0,
)

# ---------------------------------------------------------------------------
# Run Evaluation
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    response_with_memory_updates_perf.run(print_results=True, print_summary=True)

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno openai sqlalchemy

Export your OpenAI API key

export OPENAI_API_KEY="your_openai_api_key_here"

Run the example

Save the code above as response_with_memory_updates.py, then run:

python response_with_memory_updates.py

Full source: cookbook/09_evals/performance/response_with_memory_updates.py