It’s been two weeks since our last webinar – due to traveling for the Data Summit – and there’s loads to catch up on. For this week’s edition, I decided to give you guys an in-depth look at our newest tool, Magic, which we launched at the Summit on oct 30th. Magic lets you extract data direct from your browser (or tablet/mobile) automatically – with no training or downloading or any of those pesky rows and columns. It’s a really great showcase of what our algorithms team has been working on, not to mention it’s really cool.
Normally when you’re using import to get data you need to open our Desktop app and train it on which bits of data you want. With Magic, all you need to do is get the URL for the page you want data from and paste it into the bar on the homepage or at the top of: magic.import.io.
The algorithms which power Magic will then turn all the information on that page into a nice table of data, it can even get the rest of the pages! Then you can delete or rename your columns as you see fit. Once you’re happy with your table, you can download the data as a CSV (however many pages you want), copy the visible table, share the link on Twitter or Facebook and even create an API to the source data.
Edit your table Download as CSV Create API
To give you guys an insight into how Magic actually works, I wrangled the wondrous Chris Bamford (one of the geniuses behind Magic) away from his monitor to give you the low-down:
“It navigates itself to the page and analyzes the HTML looking for common patterns. Specifically, the Magic algorithm is looking for text and images to try and work out which list on the page is the primary one.”
– Chris Bamford, certified genius
The rules of magic
Because we like to be honest with you guys, we’ll be the first to admit that Magic isn’t perfect. It works best on sites where there is an obvious primary list – and of course it won’t work on single row pages.
As you can see it also doesn’t quite work when I use it on my Twitter (it’s @alex_gimson in case you feel like following me!) feed – it’s decided to separate the @ sign from the twitter handles – which is not what I wanted. In cases like this, you can still use our desktop app for more manual and specific training. It’s not quite as fast, but it is much more customizable – and it’s free!
We’ve also pushed the new automatic data finding algorithm into our “Suggest Data” button in the new workflow. This means that suggest data should be even better at getting all the data relative to your first training click.
Take the magic with you
Did you know that Magic also works on your phone and tablet? Well it does! Because there is nothing to train and no plugin, you can use magic.import.io on your phone or on your tablet. As far as we know, we’re the first company to allow for data extraction on the go.
We get so many wonderful questions from you guys, both in the webinars and on support. To make sure that I’m giving them enough importance, I’ve decided to start a new column on our blog called Agony Alex, where I’ll answer the most popular questions from that week. So keep your eye out for that coming very soon!
Join us next time
Our next webinar will be on Tuesday Nov 18, and will cover…well…I don’t know yet. I couldn’t decide what to do it on, so I thought I’d ask you, the viewer. Pop your suggestions in the comments box and I’ll pick the best one and do the webinar all about that topic!
Turn the web into data for free
Create your own datasets in minutes, no coding required
Powerful data extraction platform
Point and click interface
Export your data in any format
Unlimited queries and APIs