Async Function Performance Evaluation

Benchmark an async agent.arun call over 10 iterations per metric with PerformanceEval.arun, printing runtime and memory results.

Demonstrates performance evaluation for an asynchronous function.

This setup calls arun_agent() 30 times: 10 warm-ups, 10 runtime measurements, and 10 memory measurements. Each call runs the agent.

PerformanceEval runs the callable separately for each enabled metric: warm-ups first, then runtime measurements, then memory measurements. num_iterations applies to each metric, and the default is 10 additional warm-up calls. Model retries, tools, delegation, and memory extraction can add provider requests beyond the callable count.

Memory measurements use Python’s tracemalloc; they do not measure process RSS, GPU memory, database-server memory, or remote model memory. Record dependency versions, database state, and model settings when comparing results.

async_function.py
"""
Async Function Performance Evaluation
=====================================

Demonstrates performance evaluation for an asynchronous function.
"""

import asyncio

from agno.agent import Agent
from agno.eval.performance import PerformanceEval
from agno.models.openai import OpenAIChat


# ---------------------------------------------------------------------------
# Create Benchmark Function
# ---------------------------------------------------------------------------
async def arun_agent():
    agent = Agent(
        model=OpenAIChat(id="gpt-5.2"),
        system_message="Be concise, reply with one sentence.",
    )
    response = await agent.arun("What is the capital of France?")
    return response


# ---------------------------------------------------------------------------
# Create Evaluation
# ---------------------------------------------------------------------------
performance_eval = PerformanceEval(func=arun_agent, num_iterations=10)

# ---------------------------------------------------------------------------
# Run Evaluation
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    asyncio.run(performance_eval.arun(print_summary=True, print_results=True))

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno openai

Export your OpenAI API key

export OPENAI_API_KEY="your_openai_api_key_here"

Run the example

Save the code above as async_function.py, then run:

python async_function.py

Full source: cookbook/09_evals/performance/async_function.py