DB

Store a vLLM agent's session history in Postgres and reuse it across turns.

db.py
"""Run `uv pip install sqlalchemy` and ensure Postgres is running (`./cookbook/scripts/run_pgvector.sh`)."""

from agno.agent import Agent
from agno.db.postgres import PostgresDb
from agno.models.vllm import VLLM
from agno.tools.websearch import WebSearchTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------

# Setup the database
db_url = "postgresql+psycopg://ai:ai@localhost:5532/ai"
db = PostgresDb(db_url=db_url)

agent = Agent(
    model=VLLM(id="Qwen/Qwen2.5-7B-Instruct"),
    db=db,
    tools=[WebSearchTools()],
    add_history_to_context=True,
)

agent.print_response("How many people live in Canada?")
agent.print_response("What is their national anthem called?")

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    pass

Run the Example

The server needs a supported serving environment with hardware and memory suitable for the selected model. For the standard GPU setup, use supported Linux hardware; Windows users can use a supported WSL environment or a separate serving host. The Python client can run separately.

Start the server in the first terminal

In the serving environment, install vLLM and start the model. Leave this foreground process running:

uv venv .venv-vllm
source .venv-vllm/bin/activate
uv pip install -U vllm
vllm serve Qwen/Qwen2.5-7B-Instruct --host 127.0.0.1 --port 8000 --enable-auto-tool-choice --tool-call-parser hermes

Open a second terminal in your example directory for the client steps below.

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install client dependencies in the second terminal

uv pip install -U agno "psycopg[binary]" ddgs openai sqlalchemy

Configure the local vLLM client

In this second terminal, select the server started above. These examples use an unauthenticated loopback server, so the SDK key is a nonempty placeholder. If you enable server authentication, use its configured key instead.

export VLLM_BASE_URL="http://127.0.0.1:8000/v1"
export VLLM_API_KEY="vllm-local"

The requested model ID must match the model served above. For a server on another supported host, configure its reachable URL and authentication instead of the loopback URL.

Run PgVector

docker run -d \
  -e POSTGRES_DB=ai \
  -e POSTGRES_USER=ai \
  -e POSTGRES_PASSWORD=ai \
  -e PGDATA=/var/lib/postgresql \
  -v pgvolume:/var/lib/postgresql \
  -p 5532:5432 \
  --name pgvector \
  agnohq/pgvector:18

Run the example

Save the code above as db.py, apply the listed edits, then run in the second terminal:

python db.py

Full source: cookbook/90_models/vllm/db.py