Code Generation

Generate Python code with DeepSeek Coder served by vLLM.

code_generation.py

"""Code generation example with DeepSeek-Coder.
Run vLLM model: vllm serve deepseek-ai/deepseek-coder-6.7b-instruct \
        --dtype float32 \
        --tool-call-parser pythonic
"""

from agno.agent import Agent
from agno.models.vllm import VLLM

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------

agent = Agent(
    model=VLLM(id="deepseek-ai/deepseek-coder-6.7b-instruct"),
    description="You are an expert Python developer.",
    markdown=True,
)

agent.print_response(
    "Write a Python function that returns the nth Fibonacci number using dynamic programming."
)

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------

if __name__ == "__main__":
    pass

Run the Example

The server needs a supported serving environment with hardware and memory suitable for the selected model. For the standard GPU setup, use supported Linux hardware; Windows users can use a supported WSL environment or a separate serving host. The Python client can run separately.

Start the server in the first terminal

In the serving environment, install vLLM and start the model. Leave this foreground process running:

uv venv .venv-vllm
source .venv-vllm/bin/activate
uv pip install -U vllm
vllm serve deepseek-ai/deepseek-coder-6.7b-instruct --host 127.0.0.1 --port 8000

Open a second terminal in your example directory for the client steps below.

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

Install client dependencies in the second terminal

uv pip install -U agno openai

Configure the local vLLM client

In this second terminal, select the server started above. These examples use an unauthenticated loopback server, so the SDK key is a nonempty placeholder. If you enable server authentication, use its configured key instead.

export VLLM_BASE_URL="http://127.0.0.1:8000/v1"
export VLLM_API_KEY="vllm-local"

The requested model ID must match the model served above. For a server on another supported host, configure its reachable URL and authentication instead of the loopback URL.

Run the example

Save the code above as code_generation.py, apply the listed edits, then run in the second terminal:

python code_generation.py

Full source: cookbook/90_models/vllm/code_generation.py