Knowledge

Build knowledge bases from basic RAG through production patterns, vector databases, and cloud storage integrations.

Examples for giving agents searchable knowledge, ordered from a first RAG agent to production deployment.

Getting Started

ExampleDescription
Basic RAGContext injection, the simplest way to give an agent access to documents.
Agentic RAGThe agent gets a search_knowledge_base tool and decides when to query.
Website Per PageIngest sitemap pages with per-page content records.
Loading ContentLoad from local files, URLs, raw text, topics, and batch operations.

Building Blocks

ExampleDescription
Chunking StrategiesSide-by-side comparison of document splitting strategies.
Search TypesVector, keyword, and hybrid search compared.
RerankingScore and reorder retrieved candidates with Cohere.
FilteringNarrow search results with document metadata.
Agentic FilteringThe agent builds metadata filters from the user query.
EmbeddersChoose and configure embedding models.

Production

ExampleDescription
Multi-Source RAGCombine PDFs, web pages, and text in one knowledge base.
Knowledge LifecycleInsert, skip existing content, and remove vectors; track records separately.
Multi-TenantIsolate per-tenant data with isolate_vector_search.
AgentOSServe agents and knowledge as API endpoints.
Error HandlingIngestion patterns that survive bad files and failed loads.
SSRF HardeningRestrict URL-fetching readers with allowed_hosts.

Advanced

ExampleDescription
Custom RetrieverFull control over retrieval logic, bypassing the Knowledge class.
Custom ChunkingImplement your own chunking strategy.
Graph RAGBuild a knowledge graph from documents with LightRAG.
Knowledge ToolsThink, search, and analyze tools for reasoning over knowledge.
Knowledge ProtocolCustom knowledge sources via the KnowledgeProtocol interface.
Prefix SearchSearch-as-you-type matching on partial words.

Integrations

Cloud

ExampleDescription
AWSLoad files and folders from S3 buckets.
AzureLoad from Azure Blob Storage containers.
Azure (SAS Token)Blob Storage access with SAS token authentication.
GCPLoad from Google Cloud Storage buckets.
SharePointLoad from SharePoint document libraries.
GitHub Dynamic RepoOverride the source repository per request.
Multi-SourceCombine multiple remote sources in one Knowledge instance.

RAG

ExampleDescription
Agentic RAG with Infinity RerankerAgentic RAG with an Infinity reranker backend.
Agentic RAG with LightRAGAgentic RAG backed by LightRAG.
Local RAG with LangChain and QdrantLocal inference and storage after source and model downloads.

Readers

ExampleDescription
DocumentsPDF auto-detection and explicit Excel reading.
DataCSV and JSON readers.
WebWebsite crawling and a PDF URL.
Docling: PDFPDF processing with different output formats.
Docling: Office DocumentsMicrosoft Office document processing.
Docling: ImagesOCR for image files.
Docling: AudioSpeech-to-text transcription for audio files.
Docling: MarkupMarkup and structured document formats.
Docling: Data FilesSpreadsheet and data file processing.
Docling: Input TypesPaths, URL strings, and file-like input.

Vector Databases

ExampleDescription
QdrantThe recommended vector database for production.
LocalEmbedded ChromaDB and LanceDB for prototyping.
ManagedServerless Pinecone with zero infrastructure management.
PgVectorVector search inside PostgreSQL.