← All integrations
Import.io + MySQL

Web data in MySQL

Import.io’s documented database integration: read the Import.io extractor’s latest run with pandas and write it to MySQL through SQLAlchemy.

Connects via
EndpointREST API
Setup
10 min
Available on
Endpoints: every planAPI: every plan
Start free trialTalk to sales
Python
import io, os, requests
import pandas as pd

# "CSV from latest successful run" on the extractor's Integrate tab
csv = requests.get(os.environ["IMPORTIO_CSV_URL"]).content.decode("utf-8-sig")
df = pd.read_csv(io.StringIO(csv))
from sqlalchemy import create_engine

engine = create_engine(os.environ["DATABASE_URL"])  # mysql+pymysql://user:pass@host/db
df.to_sql("competitor_prices", engine, if_exists="append", index=False)

How it connects.

Source

The web

Any public page, rendered, through managed proxies

Engine

Import.io

Rendering, proxy routing, captcha handling, extraction

Path

Data endpoint

Latest successful run, CSV or JSON

Destination

MySQL

The documented pandas and SQLAlchemy loader.

Also connects via the REST API.

What teams do with it.

Catalog sync

Keep a products table current from supplier or marketplace pages.

Price history for your store

Store competitor prices beside your own catalog.

Swap the database later

Change the SQLAlchemy URI and the same code writes to another engine.

Set it up.

  1. pip install requests pandas sqlalchemy pymysql.
  2. Set IMPORTIO_CSV_URL and a mysql+pymysql:// connection string.
  3. Write the DataFrame with to_sql.

Questions.

Is this in the Import.io docs?

Yes. “Writing Import.io Data to a Database using Python” covers MySQL and shows how to swap the URI for other databases.

Do endpoint downloads count as queries?

No. The latest-run CSV, JSON and Google Sheets endpoints return data your extractor already collected, so they don’t count toward your plan. Only the Live Query API runs an extraction per request.

How fresh is the data?

As fresh as your schedule. Run the extractor hourly, daily or weekly and the endpoint always serves the latest successful run.

How do I authenticate?

Pass your API key as the _apikey query parameter. Copy it from account settings at app.import.io.

Is the API on every plan?

Yes. The trial and Standard include rate-limited API access. Professional and Advanced include full API access and webhooks.

Related integrations.

All 50 integrations

EndpointREST API

PostgreSQL

Append each run into Postgres with pandas and SQLAlchemy.

Setup 10 minEndpointREST API

SQL Server

Load extractor results into SQL Server or Azure SQL.

Setup 10 minEndpointREST API

MongoDB

NDJSON rows straight into a collection.

Setup 5 minEndpointREST API

Python & pandas

Latest-run data into a DataFrame in three lines.

Setup 5 min

Fresh web data
in MySQL.

Start free trialTalk to sales