Every Friday at import.io we have a company-wide meeting where we get together and listen to one of the team talk about something data or company related that they’ve been working on. If we’re not convinced, we fire them. After that, we have cake.
Just kidding… there’s no cake.
Last week’s Friday talk revealed that a lot of our users really want a better way to put data into databases like SQL.
Occasionally we developers get a little time to ourselves to hack together some side projects or proof of concepts that could later get turned into features. Inspired by what I learned on Friday, I thought it would be a fun idea to come up with an easier way to send data from a website straight into an SQL database with zero hassle.
importSQL – as I have so cleverly named it – is a python tool that automates the process of grabbing your data from any of your import.io sources and put it directly into an SQL database of your choosing. All you have to do is build your data source and then tell the script how to map the data from import to your SQL database fields.
Essentially, there is just a JSON file that you put your configuration into once and then you can re-use that script again whenever you want to add the data from another url.
For example, to import your data into a database you have defined, you only need to run the following command:
Full instructions and an example about owls (which are great) can be found here.
If you guys like it, this project could potentially find it’s way into our desktop app. I’d really like to get your feedback and any related feature requests on the github page.
Turn the web into data for free
Create your own datasets in minutes, no coding required
Powerful data extraction platform
Point and click interface
Export your data in any format
Unlimited queries and APIs