Import.io is on a mission to enable the world’s data.
Effective use of data can provide incredible efficiency and value to almost any business, from the smallest independent company to the largest multinational. But, it can also advance education, academic research, charitable work, investigative journalism and many other efforts that help to make our world a better place.
Import.io delivers both SaaS product and Managed Services solutions that enables users to convert the mass of data on web sites into structured, machine-readable data with no coding required. Our highly efficient and scalable platform allows customers to process 1,000s of URLs concurrently or on a schedule and gain access to millions of rows of data that they use for hundreds of different use cases. Some common uses include building training sets for machine learning models, doing retail price monitoring and analysis, web data integration into business applications, market research, academic research, etc.
We are looking to build world class capabilities to extract web data, in very large mass. That’s where you come in.
We’re looking for you – the cloud platform expert that knows how to build the cloud based backend that is so efficient in performance, yet scalable and is reliable with >99.999% uptime. The platform must be able to handle thousands of both headless browser and browserless clients processing billions of web pages per day, non-stop. To accomplish this, you will have the opportunity to define the team you need, and work with other area architects and the CTO.
Types of challenges you’ll tackle on at Import.io
We are a tight knit group of high calibre engineers. So while we can write down some examples of types of things you will be doing, expect to have a much wider range of tasks. After all, that’s part of the reason why you’d want to join a small company, right? Don’t get stuck only looking at a narrow slice of the pie at a big company!
At Import.io, Architects are looked to as thought leaders and trusted advisors by our entire company including executive staff, engineers, sales, and ultimately our customers. Architects are expected to be subject matter experts in the area/categories that you own – architecture, technology to use, integration, security, business strategy, etc.
The primary responsibilities of this role include:
- Use your experience and knowledge to build a cloud backend described above.
- That means you’re a cloud wizard. You probably worked extensively with cloud providers like AWS, Azure and GCP.
- You know when to use open source vs hosted services are appropriate. But what about that commercial version? Today, there are so many choices that many find it hard to choose (or they choose randomly). For example, mesos or k8s? Host it yourself or go with EKS? What about ECS? Some might say “oh boy” but you know exactly how to make that choice.
- You know how to instrument the platform, so at any given moment, you can tell what’s going on. You also know now to rely on “looking” to find out something is wrong because you have a keen sense for effective use of monitoring and alerts. Tools like PagerDuty and OpsGenie come to mind.
- And by the way, you HATE it when you get a PagerDuty calls you. Each time it does, you take a personal and do whatever it takes to make sure you never get called again.
- Establish a very efficient sysops team/process.
We’re not here to tell you what skills are required. We can, but better would be for you to tell us what skills you have, and why you think you can accomplish the above. But to help give you an idea what we have in mind (and honestly, this section helps with SEO):
- Must be a self-starter with the ability and drive to own and deliver.
- We are an AWS shop. You should have extensive experience with it. GCP or Azure is ok, but you’d need to prove to us that you can make the switch.
- While you obsess over knowing the latest technologies, you have the business acumen to know when it should be used versus when to stick to older, tried and true technologies.
- While we are not married to any specific programming language or stack, you should know that large portions of our current server is written in java and node.
- You need to be very good with databases – postgres, aurora, redis, redshift and the like.
- Data streaming technologies like kafka and kinesis should be second nature to you.
- You know when to use APMs and other monitoring tools like DataDog & grafana. But you gotta keep costs reasonable so how else would you monitor & visualize it?
- You hate it when the code you are responsible for doesn’t work. You take it personal. You do what it takes for it to never happen again.
- That means you know that YOU own the quality of the product. You don’t need to rely on QA (although there will be QA that will test your code to keep you honest).
- To do that, you know how to develop testing plans, validate against design goals, and measure quality objectively via metrics.
- You want to, and know how to, automate everything. You never want (or need) to manually do the same thing twice.
- You are your own biggest critic. You get pissed off at yourself when you can’t finish something you started and deploy it all the way to production.
- You have a professional attitude. You have the ability to analyse complex technical issues and provide options to overcome challenges. That also means you can debate merits and pitfalls with peers then commit and align to the decision.
- You don’t need babysitting. You can execute with a sense of urgency and with minimum oversight. After all, others are going to be looking to you for guidance, not the other way around.
- B.S. Computer Science, Software Engineering, Management Information Systems or equivalent degree. Or equivalent industry experience.