Location: Anywhere
Import.io is on a mission to enable the world’s data.
Effective use of data can provide incredible efficiency and value to almost any business, from the smallest independent company to the largest multinational. But, it can also advance education, academic research, charitable work, investigative journalism and many other efforts that help to make our world a better place.
Import.io delivers both SaaS product and Managed Services solutions that enables users to convert the mass of data on web sites into structured, machine-readable data with no coding required. Our highly efficient and scalable platform allows customers to process 1,000s of URLs concurrently or on a schedule and gain access to millions of rows of data that they use for hundreds of different use cases. Some common uses include building training sets for machine learning models, doing retail price monitoring and analysis, web data integration into business applications, market research, academic research, etc.
The Role:
We are looking to build world class capabilities to extract data from any website, using a combination of anti-bot techniques and technologies third party proxy providers as well as build out our own. At some point, it’s also conceivable that we will even offer our proxy network as a separate product in addition to our Web Data Integration capabilities.
And that’s where you come in. We want you to build a robust platform that is comprised of both residential and data center proxies, and other anti-bot (anti-scraping) techniques. To accomplish this, you will have the opportunity to define the team you need, and work with our server architect and the CTO.
Types of challenges you’ll tackle on at Import.io
We are a tight knit group of high calibre engineers. So while we can write down some examples of types of things you will be doing, expect to have a much wider range of tasks. After all, that’s part of the reason why you’d want to join a small company, right? Don’t get stuck only looking at a narrow slice of the pie at a big company!
At Import.io, Architects are looked to as thought leaders and trusted advisors by our entire company including executive staff, engineers, sales, and ultimately our customers. Architects are expected to be subject matter experts in the area/categories that you own – architecture, technology to use, integration, security, business strategy, etc.
The primary responsibilities of this role include:
- Use your experience and knowledge to build a proxy network using both residential and data center IP addresses
- Make sure that network is fault fault tolerant so you don’t have to get paged in middle of family dinner with a problem. Also make sure it scales up/down on its own depending on load.
- Figure out the best ways for Import.io’s web data integration platform get around anti-bot, avoid our traffic going to Google Analytics, and work around all other anti-crawling technologies.
- Put in the monitoring and define escalation policies/alerts for abnormalities. Build out monitoring dashboards.
- Build the right metering so we know who is using how much of the network.
- Tie usage to cost, and find ways to reduce it.
- Work with Product and Sales determine optimal pricing strategy when the time to expose the proxy network as a product of it’s own.
- Leads root cause analysis and troubleshooting effort for production incidents.
- Write “runbooks” and train the network operators.
We’re not here to tell you what skills are required. We can, but better would be for you to tell us what skills you have, and why you think you can accomplish the above. But to help give you an idea what we have in mind (and honestly, this section helps with SEO):
- Must be a self-starter with the ability and drive to own and deliver.
- You have a professional attitude. You have the ability to analyse complex technical issues and provide options to overcome challenges. That also means you can debate merits and pitfalls with peers then commit and align to the decision.
- You don’t need babysitting. You can execute with a sense of urgency and with minimum oversight. After all, others are going to be looking to you for guidance, not the other way around.
- You hate it when the code you are responsible for doesn’t work. You take it personal. You do what it takes for it to never happen again.
- That means you know that YOU own the quality of the product. You don’t need to rely on QA (although there will be QA that will test your code to keep you honest).
- To do that, you know how to develop testing plans, validate against design goals, and measure quality objectively via metrics.
- You want to, and know how to, automate everything. You never want (or need) to manually do the same thing twice.
- You are your own biggest critic. You get pissed off at yourself when you can’t finish something you started and deploy it all the way to production.
- In depth knowledge of networking protocols, especially HTTP and other relevant networking technologies such as Firewalls, proxy servers, routers, switches, etc.
- Expertise in wide area networks.
- Exceptional business acumen.
- Research latest relevant technologies and solutions. Determine suitability and provide guidance on the best solution for the future.
- Extensive experience in the software industry, with extensive knowledge of software development and enterprise software technologies.
- B.S. Computer Science, Software Engineering, Management Information Systems or equivalent degree. Or equivalent industry experience.