Technical

Insight and technical features for the code savvy

Artificial Intelligence Regulation

Artificial Intelligence Regulation: Let’s not regulate mathematics!

Written by:

On Wednesday, ahead of today’s White House Frontiers Conference, the White House Office of Science and Technology Policy released its report on Preparing for the Read more

5 New Advanced Data Extraction Features To Try Out

Written by:

You asked for more powerful data extraction features. We took your feedback and set our Engineers loose on the challenge. We are excited to announce Read more

Query API – Service Availability

Written by:

Last updated: 2pm PST, Wednesday 9th March 2016 The new Query API has been in production for 24 hours now at 99.9% availability.   We will Read more

Neural nets: How Regular Expressions brought about Deep Learning

Written by:

2016 is the year of Deep Learning, whose popularity has been on a steep incline ever since Google bought DeepMind at the end of 2014. Last Read more

Using AWS Lambda and API Gateway to create a serverless schedule

Written by:

At import.io, we believe that the best DevOps is NoOps.

That doesn’t mean that we don’t like DevOps people. On the contrary, we think they rock!

But ideally, we want everyone to be able to automate their work wherever possible. Instead of spending time on repetitive and mundane tasks, we push those jobs onto computers – leaving ourselves more time to work on great features.

With that in mind, we are starting to adopt microservices patterns as we scale our engineering efforts, so that new features are now delivered as separate components from the main platform. This allows us to iterate on new functionalities quickly, test different technology stacks and involve people who do not have the inside knowledge about the platform in the development process.

Our latest project, Scheduled APIs (which will let you run your Import.io APIs on a schedule), gave us a chance to revise our technological stack, and have a look around at new paths for building long lasting components. Specifically, we used a set of new AWS solutions: Amazon Lambda and API Gateway.

Instantly mirror an S3 bucket

Written by:

We show you have to instantly mirror an S3 bucket, downloading files as soon as they are created or updated for local processing.

Using Kinesis and Kibana to get insights from your data

Written by:

Matthew Painter, CTO at import.io, walks you through how we use Amazon Kinesis for managing our routing of event data, such as queries being made on the platform, and how to allow your product and user teams to analyze the events in the fantastic Kibana 4, a “flexible analytics and visualization platform” that is powered by Elasticsearch.

Create a live data viz with SAP Lumira

Written by:

One of the greatest thing about the import.io community is that they inspire each other to try new things and share fun projects. Recently, Shankar Narayanan SGS became inspired by Ronald Konijenburg’s post about getting data from the World Series of Poker and visualizing it in SAP Lumira. In it, Ron used import.io to get data from the WSOP website and visulaiz it using SAP Lumira. But he did it with a CSV, which meant that the data was static. 

XPath 101: 3 common XPaths for extracting data

Written by:

XPaths are what make the import.io app go ‘round. Without them, we wouldn’t be able to extract data, so needless to say, they’re pretty important. The reason XPaths are so important, is because when you click on data to train the tool, behind the scenes our algorithms are trying to work out the corresponding XPath for that data. These are the three most common uses of Xpaths in import.io; but first, what is an XPath anyway? 

importSQL: The no hassle way to send your data direct to SQL

Written by:

Every Friday at import.io we have a company-wide meeting where we get together and listen to one of the team talk about something data or company related that they’ve been working on. If we’re not convinced, we fire them. After that, we have cake.

Just kidding… there’s no cake.

Last week’s Friday talk revealed that a lot of our users really want a better way to put data into databases like SQL.

Occasionally we developers get a little time to ourselves to hack together some side projects or proof of concepts that could later get turned into features. Inspired by what I learned on Friday, I thought it would be a fun idea to come up with an easier way to send data from a website straight into an SQL database with zero hassle.