← All integrations
Import.io + Pinecone

Web data into Pinecone for RAG

Schedule an extractor, embed the fields that matter and upsert them with their source URLs. Your web-data retrieval index stays as current as the web it came from.

Connects via
EndpointMCP
Setup
15 min
Available on
Endpoints: every planMCP: 10,000 free calls, then $0.0002 each
Start free trialTalk to sales
Python
import json, os, requests
from openai import OpenAI
from pinecone import Pinecone

rows = [json.loads(l) for l in requests.get(os.environ["IMPORTIO_JSON_URL"]).text.splitlines() if l.strip()]
texts = [f"{r.get('title', '')}\n{r.get('description', '')}" for r in rows]  # your extractor's fields

vectors = OpenAI().embeddings.create(model="text-embedding-3-small", input=texts).data
Pinecone().Index("web-data").upsert([
    (r.get("url", str(i)), v.embedding, {"url": r.get("url", ""), "title": r.get("title", "")})
    for i, (v, r) in enumerate(zip(vectors, rows))
])

How it connects.

Source

The web

Any public page, rendered, through managed proxies

Engine

Import.io

Rendering, proxy routing, captcha handling, extraction

Path

Data endpoint

Latest successful run, CSV or JSON

Destination

Pinecone

Extracted pages embedded into a vector index for RAG.

Also connects via hosted mcp server.

What teams do with it.

Answers grounded in current pages

Product, policy and pricing pages indexed on a schedule, not once.

Citations that resolve

Every vector carries the URL it came from.

Refresh without rebuilding

Upsert by ID on each run and only changed content moves.

Set it up.

  1. pip install requests openai pinecone.
  2. Set IMPORTIO_JSON_URL and your model and Pinecone keys.
  3. Embed the text fields and upsert with URL metadata. Use your extractor’s field names.

Questions.

Scheduled index or live fetch?

Both. Keep a scheduled index for breadth, and give the agent the Import.io MCP for pages the index doesn’t cover yet.

Do endpoint downloads count as queries?

No. The latest-run CSV, JSON and Google Sheets endpoints return data your extractor already collected, so they don’t count toward your plan. Only the Live Query API runs an extraction per request.

How fresh is the data?

As fresh as your schedule. Run the extractor hourly, daily or weekly and the endpoint always serves the latest successful run.

What does the Import.io MCP cost?

Signup includes 10,000 successful calls. After that it is $0.0002 per successful call. Setup requests, tool listings, malformed requests, blocked requests and failed upstream calls are never billed, and you can set a monthly spend limit that stops paid usage when reached.

OAuth or API key?

Use OAuth in interactive clients that support it: the client opens a sign-in, stores the credentials and reuses them. Use an API key for scripts, servers and clients without OAuth, sent as Authorization: Bearer mcp_live_…. Keep keys out of prompts and shared config files.

Related integrations.

All 50 integrations

EndpointManaged

Hugging Face

Extracted records pushed as a versioned dataset to the Hub.

Setup 10 minMCP

LangChain & LangGraph

Load Import.io tools with langchain-mcp-adapters into any agent.

Setup 10 minMCP

LlamaIndex

Import.io tools for LlamaIndex agents and workflows.

Setup 10 minEndpointREST API

Python & pandas

Latest-run data into a DataFrame in three lines.

Setup 5 min

Fresh web data
in Pinecone.

Start free trialTalk to sales