Let’s mix it up

I’m back again! This time I had our wonderful Data Scientist Ignacio on hand to help me team you all about making a Mix.

First off, what is a Mix?

A Mix is where you make a number of Connectors, with the same schema, which you can then combine and query with one input.

The best way to understand this is to try an example. Let’s say we work for one of the major UK supermarkets and we want to keep track of what our competitors are charging week to week. Instead of visiting each site individually, you can build a Connector to each, then combine them into a Mix and search all of them at once!

For this example, I’m only going to use the big 3 (Tesco, M&S, Waitrose), but when you’re building your own you can add up to 100 different sources.

First the data

The first thing we need to do is build our Connectors. I showed you how to build a simple Connector to Tesco’s website by recording a query for my favorite thing: beer! It’s very important when creating your Connector that you use the same schema. A schema is just a fancy name for the names and values of your columns. For example, if on one Connector you say price is a currency and on another Connector that price is a number, the app won’t know how to combine them.

It’s also helpful to write your schema down somewhere so you can refer to it when you’re building your Connectors. In this case I’m going to use: Name (text) and Price (currency).

Note: Connectors let you train multiple pages (pagination) of results so you can get all that juicy data!

At this point we had a bit of an issue with the internet being too slow for pagination (typical), so I took the opportunity to show you how to export a log file. By pressing ALT + E (or CMD + ALT + E for you Mac users out there) you can export the training you have done up to that point. This allows you to save what you’ve done so that you never lose your hard work. To import your settings back into import.io, simply open the app and press ALT + I.

Right, Let’s Mix

Now for the good part! I showed you how to create a Mix by simply combining the 3 supermarket Connectors I built earlier and showed you how to query them. And voila! Here it is.

Bonus Features

  1. See which data source each entry comes from by clicking on the little pink plus in the upper left hand corner of your table.
  2. Sort your data by clicking on the column header you want to sort (I generally sort by price).
  3. Add multiple queries – search for both pizza AND beer.

For those of you who are technically inclined, you can use our integrate page to generate a sample query code in a bunch of different languages.

Questions

How can we get the URL from the site I’m extracting data from?

The great thing about import.io is we do this for you automatically! To see it click the little pink plus in the upper left hand corner of your table.

Can I add another page that has different data (ie. description) on it?

Sure you can! You can combine any combination of Extractors, Connectors and Crawlers to any Dataset. Just a word of caution though, the closer you can keep the schemas the better – otherwise it starts to add a lot of columns making it a little difficult to keep track of them.

Next up….

I’m bowing to the superior combined knowledge of the two Chrises (Alexander and Bamford) who will be taking you through some of our more advanced developer features. That webinar will take place on 13 May at 4pm GMT. Sign up today!

Turn the web into data for free

Create your own datasets in minutes, no coding required

Powerful data extraction platform

Point and click interface

Export your data in any format

Unlimited queries and APIs

Sign me up!