← All integrations
Import.io + CrewAI

Web scraping tools for CrewAI

MCPServerAdapter connects a crew to the Import.io Web Scraper MCP over Streamable HTTP. Every agent that needs the web gets the same tools.

Connects via
MCP
Setup
10 min
Available on
10,000 free successful calls, then $0.0002 per call
Get an MCP keyRead the MCP docs
Python
import os
from crewai import Agent, Task, Crew
from crewai_tools import MCPServerAdapter

server = {
    "url": "https://mcp.import.io/mcp",
    "transport": "streamable-http",
    "headers": {"Authorization": f"Bearer {os.environ['IMPORTIO_MCP_KEY']}"},
}

with MCPServerAdapter(server) as tools:
    analyst = Agent(role="Market analyst", goal="Report live competitor pricing",
                    backstory="You check the web before you answer.", tools=tools)
    task = Task(description="Collect prices for the top 5 espresso machines at williams-sonoma.com.",
                expected_output="A table with product, price and URL.", agent=analyst)
    print(Crew(agents=[analyst], tasks=[task]).kickoff())

How it connects.

Source

The web

Any public page, rendered, through managed proxies

Engine

Import.io

Rendering, proxy routing, captcha handling, extraction

Path

Import.io MCP

https://mcp.import.io/mcp

Destination

CrewAI

Give every agent in a crew live web data tools.

What teams do with it.

Research crews

One agent gathers pages, another extracts the numbers, a third writes the brief.

Collect prices for the top 5 espresso machines at three retailers.

Competitive briefings

Weekly crews that read competitor sites and summarize what moved.

Summarize new products and price changes on these competitor pages.

Lead and listing research

Extract company or listing details from public pages into a table.

Extract name, location and category for every listing on this directory page.

Set it up.

  1. pip install crewai "crewai-tools[mcp]".
  2. Describe the Import.io server with its URL, streamable-http transport and a bearer header.
  3. Open MCPServerAdapter in a with block and pass its tools to your agents.

Questions.

Should every agent get every tool?

No. Give read-only tools to research agents and keep browser actions with the one agent that needs them.

What does the Import.io MCP cost?

Signup includes 10,000 successful calls. After that it is $0.0002 per successful call. Setup requests, tool listings, malformed requests, blocked requests and failed upstream calls are never billed, and you can set a monthly spend limit that stops paid usage when reached.

OAuth or API key?

Use OAuth in interactive clients that support it: the client opens a sign-in, stores the credentials and reuses them. Use an API key for scripts, servers and clients without OAuth, sent as Authorization: Bearer mcp_live_…. Keep keys out of prompts and shared config files.

Which tools does the server expose?

importio_web_fetch, importio_render, importio_extract_data and importio_get_html, page helpers such as importio_page_text, importio_page_links and importio_search_cards, and browser actions for clicks, form input, scrolling, pagination, waits, screenshots and captcha handling. Requests can pick a proxy country and type, with bounded retries when a page blocks.

Related integrations.

All 50 integrations

MCP

LangChain & LangGraph

Load Import.io tools with langchain-mcp-adapters into any agent.

Setup 10 minMCP

OpenAI Agents SDK

Import.io as a Streamable HTTP MCP server for your agents.

Setup 10 minMCP

LlamaIndex

Import.io tools for LlamaIndex agents and workflows.

Setup 10 minMCP

Google ADK

Import.io as an MCPToolset for Agent Development Kit agents.

Setup 10 min

Give CrewAI
the live web.

Get an MCP keyRead the MCP docs