On Wednesday, ahead of today’s White House Frontiers Conference, the White House Office of Science and Technology Policy released its report on Preparing for the Future of Artificial Intelligence. The report is optimistic, comprehensive and well-balanced. In summary: full-speed ahead. But let’s be smart when it comes to Artificial Intelligence…
You asked for more powerful data extraction features. We took your feedback and set our Engineers loose on the challenge. We are excited to announce 5 brand new advanced data extraction features that will help you get data out of more websites: Disable CSS Default Column Values Advanced Regex Support…
Last updated: 2pm PST, Wednesday 9th March 2016 The new Query API has been in production for 24 hours now at 99.9% availability. We will continue to monitor this situation but you can assume that normal service has been resumed. Any questions please reach out to firstname.lastname@example.org First published: 9am PST,…
2016 is the year of Deep Learning, whose popularity has been on a steep incline ever since Google bought DeepMind at the end of 2014. Last year’s technical breakthroughs, acquisitions, funding deals and, open source releases have all helped to cement Deep Learning as the hip artificial intelligence. Our CTO, Matt Painter, explains…
At import.io, we believe that the best DevOps is NoOps.
That doesn’t mean that we don’t like DevOps people. On the contrary, we think they rock!
But ideally, we want everyone to be able to automate their work wherever possible. Instead of spending time on repetitive and mundane tasks, we push those jobs onto computers – leaving ourselves more time to work on great features.
With that in mind, we are starting to adopt microservices patterns as we scale our engineering efforts, so that new features are now delivered as separate components from the main platform. This allows us to iterate on new functionalities quickly, test different technology stacks and involve people who do not have the inside knowledge about the platform in the development process.
Our latest project, Scheduled APIs (which will let you run your Import.io APIs on a schedule), gave us a chance to revise our technological stack, and have a look around at new paths for building long lasting components. Specifically, we used a set of new AWS solutions: Amazon Lambda and API Gateway.
Matthew Painter, CTO at import.io, walks you through how we use Amazon Kinesis for managing our routing of event data, such as queries being made on the platform, and how to allow your product and user teams to analyze the events in the fantastic Kibana 4, a “flexible analytics and visualization platform” that is powered by Elasticsearch.
One of the greatest thing about the import.io community is that they inspire each other to try new things and share fun projects. Recently, Shankar Narayanan SGS became inspired by Ronald Konijenburg’s post about getting data from the World Series of Poker and visualizing it in SAP Lumira. In it, Ron used import.io to get data from the WSOP website and visulaiz it using SAP Lumira. But he did it with a CSV, which meant that the data was static.
XPaths are what make the import.io app go ‘round. Without them, we wouldn’t be able to extract data, so needless to say, they’re pretty important. The reason XPaths are so important, is because when you click on data to train the tool, behind the scenes our algorithms are trying to work out the corresponding XPath for that data. These are the three most common uses of Xpaths in import.io; but first, what is an XPath anyway?
Every Friday at import.io we have a company-wide meeting where we get together and listen to one of the team talk about something data or company related that they’ve been working on. If we’re not convinced, we fire them. After that, we have cake.
Just kidding… there’s no cake.
Last week’s Friday talk revealed that a lot of our users really want a better way to put data into databases like SQL.
Occasionally we developers get a little time to ourselves to hack together some side projects or proof of concepts that could later get turned into features. Inspired by what I learned on Friday, I thought it would be a fun idea to come up with an easier way to send data from a website straight into an SQL database with zero hassle.