Creating a good user experience is challenging. Especially when your product is technically complex, you have to strike that all important balance between functionality and simplicity. For those of you who have been with us for a while, you’ll know that our product (and website) have been through quite a few iterations over the last two years. It’s been an interesting journey, and the other day I had the opportunity to examine it in some detail when I was asked to give a talk on our UX experiences at Wellington Partners CEO Summit, for those who don’t know Wellington they are one of our fabulous investors and have been working with us since more or less day one.
Goal Setting
When we first set out to create import, we were trying to fill what we saw as a clear need in the market. People needed data from the web, but getting to it was challenging. It required you to write a web scraper, a process for which you either had to be a developer with a lot of time on your hands or have enough money that you could employ one who did. Our goal was to make this process simpler by creating a tool that allowed you to access data from the web programmatically without having to be a developer or write any code.
The nature of our product presented an interesting UX challenge. In essence we were creating a tool that (we hoped) could replace the need to write web scrapers manually – a process that is very complicated and time consuming. And this, is what we came up with:
Now I’m sure you’ll agree with me, that this clearly suffers from the fact that none of our 5-man team were designers. It’s pretty hideous, but that’s not really the point – because UX isn’t really about how something looks (although that’s certainly important) it’s about how something works.
An Ugly Duckling
This first tool, which we called Connector Builder, was not only pretty ugly; it was also very complicated to use. It was at minimum a 15 minute process that required you to record a zero results action and define a schema before you could even get to the data you wanted to extract. If you were lucky – and managed to avoid all the error messages – you ended up with something like this:
Make sense? Probably not. Don’t worry, not a lot of people understood what was going on either.
Despite the complicated nature of our product we were still quite proud of it. We even won best startup at Strata London! Although I have to imagine that had more to do with the vision than the actual product – considering our CTO was still coding the backend at the time of the demo.
We were proud of it because we had done what we set out to achieve. Even though it was complex, it was still less complex and much quicker than writing an actual web scraper by hand. Mission accomplished!
Reality Check
What we quickly realised, when we tried to start actually getting people to use Connector Builder, is that we had made a fatal flaw in our assumption. We assumed, because we had created a product that was simpler than the traditional way that that is what our users would measure us against. We were wrong.
Most people who wanted data didn’t think: How do I write a web scraper?; they thought: How can I get data easily?. We knew that without our tool they’d have to write a scraper, but that was because we had industry knowledge.
What users were really comparing us to then, was other apps. Apps are simple. Apps are sleek. Apps do not require 15 minutes to get to the end point. Thanks in part to Apple; the app market has made radical moves towards simplicity – I mean, even cats can use iPad’s these days!
What we had created was a mismatch between what we thought was a good UX and what users thought was a terrible UX.
Back to the drawing board
It took longer than I’d like to admit for us to realise exactly what had gone wrong. We were always planning on making the tool simpler, we just didn’t realise how simple people expected it to be.
Over the next two years we began chipping away at the product, moving from one iteration to the next. Always with the drive to make it simpler and take away the places where people got stuck.
Part of it was the technology. As our algorithms got better at recognizing data, the app required less and less user input to extract the data.
We also started listening more to what our users actually wanted instead of just deciding what we thought they wanted. It turned out that a fair few of them just wanted to build Crawlers and that they weren’t interested in our live APIs.
With each new version, we gained more traction and more users. And always, always we tried to slim it down and make it simple. Until eventually, we came out with this:
This newest version is great, but what we also realised from doing the user testing that got us there; was that even this – simple as it is – was actually more than some people wanted.
Rethink Everything
This realization required us to make a fundamental change in the way we thought about our product. It’s a classic case of just because you can doesn’t always mean you should. The reason our product is as complicated as it is, is because it lets you do a lot of stuff – handle js, define xpaths, view source code, etc. Which is great, but it only appeals to one segment of the market. In order to appeal to a wider market we had to do something radically different.
Right now we’re working hard on a new product that will radically simplify the process of getting data and make it far more accessible to a wider market. This is not another iteration of our current tooling, it’s a radical new approach to our market.
Listening isn’t enough.
So, what did we learn? At every stage we listened to users. However it took us a while to learn how to ask the right question. The question should have been: How much effort are you willing to put in to get data? Because, at a certain point, even though we were easier than the alternative, at a certain point users just gave up trying. In our case, for mass appeal, 3 minutes is all we get in most cases. After that there is diminishing returns.
It’s been a long and winding road to where we are today. We’ve gone from a 15 min process to a 2 min one. Part of that was the technology, but much of it was the UX. In the end you need both: good technology and asking the right questions.
