vLLM
vLLM is a fast and easy-to-use library for running LLM models locally.
| Example | Description |
|---|---|
| vLLM Basic | Run a Qwen2.5-7B agent on a local vLLM server in sync, async, and streaming modes. |
| Code Generation | Code generation example with DeepSeek-Coder. |
| DB | Store a vLLM agent's session history in Postgres and reuse it across turns. |
| Memory | Personalized memory and session summaries with vLLM. |
| Retry | Review retry settings and why invalid model IDs cannot reliably exercise the retry path. |
| vLLM Structured Output | Request JSON and validate a MovieScript with Pydantic through vLLM. |
| vLLM Tool Use | Add web search tools to a vLLM agent and stream responses sync and async. |