Website Tools - Web Scraping and Content Analysis

Use WebsiteTools to let an agent fetch and summarize a page. With no knowledge base attached, the toolkit registers one read_url function.

website_tools.py

"""
Website Tools - Web Scraping and Content Analysis

This example demonstrates how to use WebsiteTools for web scraping and analysis.
Shows enable_ flag patterns for selective function access.
WebsiteTools is a small tool (<6 functions) so it uses enable_ flags.
"""

from agno.agent import Agent
from agno.tools.website import WebsiteTools

# ---------------------------------------------------------------------------
# Create Agent
# ---------------------------------------------------------------------------


agent = Agent(
    tools=[WebsiteTools()],  # All functions enabled by default
    description="You are a comprehensive web scraping specialist with all website analysis capabilities.",
    instructions=[
        "Help users scrape and analyze website content",
        "Provide detailed summaries and insights from web pages",
        "Handle various website formats and structures",
        "Ensure respectful scraping practices",
    ],
    markdown=True,
)

# Example usage

# ---------------------------------------------------------------------------
# Run Agent
# ---------------------------------------------------------------------------
if __name__ == "__main__":
    print("=== Basic Web Content Search Example ===")
    agent.print_response(
        "Search web page: 'https://docs.agno.com/introduction' and summarize the key concepts",
        markdown=True,
    )

Run the Example

Set up your virtual environment

uv venv --python 3.12
source .venv/bin/activate

uv venv --python 3.12
.venv\Scripts\activate

Install dependencies

uv pip install -U agno beautifulsoup4 openai

Export your OpenAI API key

export OPENAI_API_KEY="your_openai_api_key_here"

$Env:OPENAI_API_KEY="your_openai_api_key_here"

Run the example

Save the code above as website_tools.py, then run:

python website_tools.py

Full source: cookbook/91_tools/website_tools.py

Examples

Primitives

Context

Models

Tools

More

Website Tools - Web Scraping and Content Analysis

Run the Example

​Run the Example

Run the Example