The boys in the algorithms department have been at it again! Today, we are proud to launch Import.io Magic – the absolute quickest way to turn a webpage into a table of data.
Simply paste the URL to the page you want data from into the box and hit the “Get Data” button. Our algorithms will do the rest. No training, no plugin, no downloading. You can do everything from the comfort of your own browser, it even works on your tablet and mobile device!
How does it work?
The Magic algorithm looks for the biggest list (with the most data) on the page. It then uses that list to auto-generate the rows and columns and bring them into a table. In some cases it can even get the subsequent pages.
At the moment Magic can only extract data from pages with multiple results – no product pages guys. Here are some example pages that work nicely.
What can I do with my table of data?
We’ll return everything that is in the list we’ve found on that page into a table. You can then edit the table column headers, delete columns, and drag and drop them into a different order. Once you’re satisfied with your table, you can Copy the visible table or Download all the pages into a CSV. This will be static data and will not be refreshable.
What about an API?
We can also create you an API to the source data. This means that you will be able to access the live data from the site whenever you want. The API is only available to the initial source data, which means that any of the changes you make to the table will not be reflected in the API. If you want customize your API, you can download the free desktop app.
What should I do if it doesn’t work?
As I mentioned earlier, Magic is only compatible with sites that have multiple results on them. It works best for sites that clearly have a primary list, and may struggle on sites – like the BBC homepage – which have many lists. It will also have trouble with sites that are heavy on the javascript. We are always working to improve the algorithms and will continue to build on and improve the tooling. If you want data from a site that doesn’t work on Magic, you can download our free desktop product and use our intuitive point-and-click training to map the data manually.
Why did we do it?
Pretty soon, everyone is going to have to start competing on data. At import.io, we believe that web data should be available to everyone who needs it. We think that Magic will help lower the barrier to entry for anyone who is interested in using data.
You can read more about this data revolution and how it will affect us in my upcoming book The Rise of Data: How Access to Data Will Change Absolutely Everything.
2 Comments. Leave new