AgentQL

Scrape pages with AgentQLTools, using the default extractor or a custom AgentQL query over a live Playwright browser.

Enable Agno agents to scrape website content using AgentQL tools.

Prerequisites

  • Install dependencies: uv pip install -U agno openai agentql.
  • Export your AgentQL API key: export AGENTQL_API_KEY=your_api_key.
  • Export your OpenAI API key: export OPENAI_API_KEY=your_openai_key.
  • Run python -m playwright install chromium in the activated environment. The toolkit opens a visible Chromium window, so a desktop display is required.
"""
AgentQL will open up a browser instance (don't close it) and do scraping on the site.
"""

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.agentql import AgentQLTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------

# Example 1: Enable specific AgentQL functions
agent = Agent(
    model=OpenAIChat(id="gpt-5.4-mini"),
    tools=[
        AgentQLTools(
            enable_scrape_website=True,
            enable_custom_scrape_website=False,
            agentql_query="your_query_here",
        )
    ],
)

# Example 2: Enable all AgentQL functions
agent_all = Agent(
    model=OpenAIChat(id="gpt-5.4-mini"),
    tools=[AgentQLTools(all=True, agentql_query="your_query_here")],
)

# Example 3: Custom query with specific function enabled
custom_query = """
{
    title
    text_content[]
}
"""

custom_agent = Agent(
    model=OpenAIChat(id="gpt-5.4-mini"),
    tools=[
        AgentQLTools(
            enable_scrape_website=True,
            enable_custom_scrape_website=True,
            agentql_query=custom_query,
        )
    ],
)

# Test the agents

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    agent.print_response(
        "Scrape the main content from https://docs.agno.com/introduction", markdown=True
    )
    custom_agent.print_response(
        "Extract title and content from https://docs.agno.com/introduction",
        markdown=True,
    )

Use the default extractor

Remove agentql_query="your_query_here" from Example 1 before running it: a nonempty query registers the custom tool even when enable_custom_scrape_website=False. The placeholder in Example 2 is not a valid query.

The current custom method iterates the result dictionary's keys, so Example 3 returns field names instead of the extracted values. For this content-extraction example, replace each toolkit configuration with AgentQLTools() and use the default scrape_website tool. The custom-query program is retained above for correspondence with the cookbook.

Run the Example

# Clone and setup repo
git clone https://github.com/agno-agi/agno.git
cd agno
git checkout d703c34f3abf3c41275d3fb2da6e0518a8881f24

# Create and activate virtual environment
./scripts/demo_setup.sh
source .venvs/demo/bin/activate
uv pip install -U agentql
python -m playwright install chromium

python cookbook/91_tools/agentql_tools.py

For details, see AgentQL cookbook.