Market data beside GA4 and ads
Join competitor prices with your own traffic and spend.
← All integrations
Import.io + BigQuery
Web data in BigQuery
Import.io’s JSON output is newline-delimited JSON, BigQuery’s native load format. Load it yourself in a few lines, or have Managed Services deliver into your dataset.
Destination your-project.web_data dataset
Built by your named Import.io team
Schedule hourly to monthly, your choice
Schema agreed up front, typed columns
QA checked before every delivery
Terms SLA with fees at riskimport io, os, requests
from google.cloud import bigquery
client = bigquery.Client()
data = requests.get(os.environ["IMPORTIO_JSON_URL"]).content # NDJSON
job = client.load_table_from_file(
io.BytesIO(data), "my-project.web_data.competitor_prices",
job_config=bigquery.LoadJobConfig(
source_format=bigquery.SourceFormat.NEWLINE_DELIMITED_JSON,
autodetect=True, write_disposition="WRITE_APPEND"))
job.result()How it connects.
Source
The web
Any public page, rendered, through managed proxies
Engine
Import.io
Rendering, proxy routing, captcha handling, extraction
Path
Managed delivery
Built, run and QA’d under an SLA
Destination
BigQuery
NDJSON loads natively; managed delivery lands it for you.
Also connects via data endpoints.
What teams do with it.
Looker Studio at scale
Dashboards over every run, not just the latest.
ML-ready tables
Feature tables for pricing and demand models in BigQuery ML or Vertex AI.
Set it up.
- Managed: share the project and dataset with your Import.io team.
- Self-service:
pip install google-cloud-bigquery requests. - Load the latest-run JSON with
source_formatNEWLINE_DELIMITED_JSON.
Questions.
Do I need to convert the JSON?
No. Import.io JSON is NDJSON, which BigQuery loads as is.
Who sets up managed delivery?
Your Managed Services team. They agree destination, format, schedule and schema with you, then build, run, monitor and QA the feed under an SLA.
Which destinations can Managed Services deliver into?
Amazon S3, Google Cloud Storage, Azure Blob Storage, SFTP, Snowflake, BigQuery, API, email and Slack alerts. Aperture delivers into the same destinations.
Do endpoint downloads count as queries?
No. The latest-run CSV, JSON and Google Sheets endpoints return data your extractor already collected, so they don’t count toward your plan. Only the Live Query API runs an extraction per request.
How fresh is the data?
As fresh as your schedule. Run the extractor hourly, daily or weekly and the endpoint always serves the latest successful run.
Related integrations.
ManagedEndpoint
Snowflake
Delivered into Snowflake by our team, or loaded with write_pandas.
Setup 15 minS3 / SFTPManagedDatabricks
Auto Loader picks up every delivery into a Delta table.
Setup 15 minS3 / SFTPManagedAmazon Redshift
Files land in S3; COPY loads them into Redshift.
Setup 15 minManagedWebhookGoogle Cloud Storage
Managed delivery into your GCS bucket, ready for BigQuery.
Setup 10 min