Attention: There is now a NEW version of this post. Please click here.
Today’s webinar was brought to you by a very special guest, none other than Co-Founder and Product Evangelist, Andrew Fogg! It’s rare that I willingly relinquish the webinar spotlight, but when I heard what he had come up with, I just knew I had to let him tell you all about it.
To give you just a little bit of background, one of the things users request is the ability to bulk upload a bunch of URLs or queries to an Extractor or a Connector. At the moment, unless you know how to use them over the API, you have to add them one at a time from the data set page. And, if you have say 50 or 100 URLs, that can be a bit of a slow process. I’ve of course put the dev team hot on the trail of adding this feature, but in the meantime Andrew has come up with a handy work-around using Google Sheets.
For this webinar, in the interest of time, we will need to assume that you already know how to build import.io Extractors and Connectors. If you don’t, no need to worry, you can check out my latest Getting Started webinar which will teach you everything you need to know!
The Batch Search Sheet
Andrew has built this very handy Google sheet, which allows you to put the ID of any import.io Connector or Extractor and then put in as many queries or URLs as you want at once. In order to use his sheet for yourself, you will first need to make a copy of it to your own Google Drive.
The first two things you will need is your user ID and API key (you will need your password for this) which you can get from your My Account page. Simply copy and paste these into their corresponding boxes in spreadsheet (you’ll only have to do this once). These help us identify you and allow you access to your import.io sources.
The next thing you’ll need is the GUID for the Extractor or Connector that you want to query. You can get this from the My Data page or from the data set page itself.
Next, click on the “Get input names button” – this will go to import.io and look up the GUID you have provided and bring in the input you trained it on.
The final step is to map your inputs in sheet 2. This is where you put in all the queries you want to do (one per cell). Then you will need to go back to sheet one and specify which column you have put your query list in.
Now you’re ready to go. All you have to do is go to sheet 3 and hit the “Get results” button. And just like that, all that data will be pulled back into the sheet live from the site.
One Step Further
Let’s take this up a notch. One of the things people like to do is to use the output of one API as the input for another. In this case, Andrew used a Connector to get product URLs from the search results in REI and then an Extractor to get data from the individual product pages.
Using his batch search spreadsheet he used his Connector to generate the product URLs and then upload all of them into his Extractor. The benefit to using this method as opposed to building a Crawler, is that you can refresh the data whenever you want so that the data is always up-to-date.
And One More (just for fun)
You can also use this sheet with Connectors that need multiple inputs, such as the one Andrew built to Mandarin Oriental, which need both a “check in” and “check out” date to show the prices of hotels. Now, to find out when the cheapest time to go on holiday is, you can plot the rate change for the entire month.
How does he do it?
Lastly, Andrew gave a everyone quick peek behind the curtain at the Google Apps Script that he wrote to make his spreadsheet.
Everything Andrew showed you is available for you to go and play with and we want to hear from you! If you have any questions or improvements, please feel free to have a play around with it and let us know what you come up with (firstname.lastname@example.org)!
Join Us Next Time
Next time I’ll once again be joined by the wonderful Kristaps from infogr.am. This time we’ll be showing you how to integrate live data into your infrographic. It’s going to be a cracking show so sign up now!
Turn the web into data for free
Create your own datasets in minutes, no coding required
Powerful data extraction platform
Point and click interface
Export your data in any format
Unlimited queries and APIs