Llama OpenAI Metrics

Stream a Llama 4 Maverick run with YFinance tools and inspect message, run, and session metrics.

The source below uses the broken compatible adapter and removed agent.run_response. Its last-run lookup also needs a database. Run the complete Current Example to capture streamed output and inspect its stored metrics.

metrics.py
"""
Meta Metrics
============

Cookbook example for `meta/llama_openai/metrics.py`.
"""

from typing import Iterator

from agno.agent import Agent, RunOutputEvent
from agno.models.meta import LlamaOpenAI
from agno.tools.yfinance import YFinanceTools
from agno.utils.pprint import pprint_run_response
from rich.pretty import pprint

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------

agent = Agent(
    model=LlamaOpenAI(id="Llama-4-Maverick-17B-128E-Instruct-FP8"),
    tools=[YFinanceTools()],
    markdown=True,
)

run_stream: Iterator[RunOutputEvent] = agent.run(
    "What is the stock price of NVDA", stream=True
)
pprint_run_response(run_stream, markdown=True)

run_response = agent.get_last_run_output()

# Print metrics per message
if run_response.messages:
    for message in agent.run_response.messages:
        if message.role == "assistant":
            if message.content:
                print(f"Message: {message.content}")
            elif message.tool_calls:
                print(f"Tool calls: {message.tool_calls}")
            print("---" * 5, "Metrics", "---" * 5)
            pprint(message.metrics)
            print("---" * 20)

# Print the metrics
print("---" * 5, "Collected Metrics", "---" * 5)
pprint(run_response.metrics)
# Print the session metrics
print("---" * 5, "Session Metrics", "---" * 5)
pprint(agent.get_session_metrics())

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    pass

Current Example

The in-memory database lasts for this Python process. Consuming the stream finishes the run before reading it back. Confirm Meta account/model access before running.

metrics.py
from agno.agent import Agent
from agno.db.in_memory import InMemoryDb
from agno.tools.yfinance import YFinanceTools
from agno.utils.pprint import pprint_run_response
from rich.pretty import pprint

from os import getenv

from agno.models.openai.like import OpenAILike


def llama_model(**kwargs):
    return OpenAILike(
        api_key=getenv("LLAMA_API_KEY"),
        base_url="https://api.llama.com/compat/v1/",
        supports_native_structured_outputs=False,
        supports_json_schema_outputs=True,
        **kwargs,
    )

agent = Agent(
    model=llama_model(id="Llama-4-Maverick-17B-128E-Instruct-FP8"),
    db=InMemoryDb(),
    session_id="llama-metrics-demo",
    tools=[YFinanceTools()],
    markdown=True,
)
pprint_run_response(agent.run("What is the stock price of NVDA?", stream=True), markdown=True)
run_response = agent.get_last_run_output()
if run_response is None:
    raise RuntimeError("No completed run was stored")
for message in run_response.messages or []:
    if message.role == "assistant":
        pprint(message.content or message.tool_calls)
        pprint(message.metrics)
pprint(run_response.metrics)
pprint(agent.get_session_metrics())

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install dependencies

uv pip install -U agno llama-api-client openai yfinance

Export your Meta Llama API key

export LLAMA_API_KEY="your_llama_api_key_here"

Run the example

Save the complete Current Example above as metrics.py, then run:

python metrics.py

Full source: cookbook/90_models/meta/llama_openai/metrics.py