These days there are so many better alternatives to wasting hours (sometimes days) coding PHP web crawlers. With today’s technology you can build a web crawler in minutes using a simple visual interface and a tool that learns what you want. If you can see it on the page, you can have it!
Rich Data Formats
The web crawler in import.io is highly targeted so you only get the data you need from the pages you are interested in and allows you to extract data in a variety of formats.
Here are some of the data types our features let you extract without using any PHP at all:
- Time / Date
- Whole Numbers
- Raw HTML
Using the import.io app is quick too, you can build a Crawler in about 5 minutes with practice, which is faster than creating a custom crawler in any programming language. Here is a video showing you how to replace your PHP web crawler with an import.io in less than 20 mins:
If you’re more of the reading type, you can check out this written tutorial as well!
If you are a bit more technical, import.io offers several advanced options which you can use to harness the true power of the targeted crawler. Reading this Advanced Crawling tutorial, will show you all the awesome features available in our Advanced mode.
Using your crawled data
Because import.io Crawlers are really Extractors with legs, you can refresh parts of your data instead of re-crawling the whole site. With the help of this awesome web app one of our developers built, you can use the Crawler over the API to update only the rows you need changed.
Another great feature of import.io is that getting your data back into your preferred programming language over an API with our client libraries is easy. All your sources will be automatically federated into one place, which means connecting and accessing data is simple.
If you’re unsure how to use some of our advanced features or can’t quite get your Crawler to work, just contact our friendly support team on firstname.lastname@example.org.
Turn the web into data for free
Create your own datasets in minutes, no coding required
Powerful data extraction platform
Point and click interface
Export your data in any format
Unlimited queries and APIs