Quick solutions to two common data extraction problems

In this week’s webinar, I give you an insight into how to solve two of the most common issues from our support channel: un-clickable data and disappearing Connector actions. These tips may seem advanced, but they’re actually quite easy to do once you know what’s going on.

Un-clickable Data

import.io is able to extract data by generating an XPath to the data based on your clicks. However, sometimes you can’t click on the data point because you simply can’t see it – such as when it’s in a drop-down menu or a mini tab. To get around this issue, you needed to manually insert an XPath into the XPath override box. You can do this quite easily by using the inspect element feature in a normal browser and just pasting in the XPath it gives you.

Disappearing Connector Actions

Connectors were originally designed for you to record a search (like “jeans” on Asos), but you can also use them to record other actions such as button clicks. But, sometimes simply recording the button click isn’t enough and the actions will disappear when you stop recording.

To stop this from happening you need to create what’s called a URL Connector. To do that, you hit the record button and then click inside the URL before you do the rest of your interactions on the site. That way, all of your page clicks will be saved.

Next time…

Join me next week, when I’ll be showing you how to troubleshoot a Crawler.

Turn the web into data for free

Create your own datasets in minutes, no coding required

Powerful data extraction platform

Point and click interface

Export your data in any format

Unlimited queries and APIs

Sign me up!

Comments

Hi not sure if this comment section is being monitored:

I need to extract price from an authenticated extractor – all good until I run the Data Set and I lose all the prices. SO…
Following the xpath tutorial I manage to get the data for each column but all the prices appear together in one cell (i.e 50 prices for row one, 50 prices duplicated in row two etc)

Original xpath to price:
//*[@id="batchBids"]/div/div/table/tbody/tr[2]/td[3]/div/div[1]/b

One of my multiple attempts at different failed variations:
//*[@id="batchBids"]/div/div/table/tbody/tr/td/following-sibling::td

How do I get each price to appear in its own column?

Barry

Hi Barry

Thanks for your message and I hope you are well!

It looks as though the following sibling command here will get everything in the 1st div (hence why it all gets grouped together) Without the website it’s quite hard to diagnose so would it be possible to send me over the URL to alex.gimson@import.io for me to take a look at it.

If it’s an authenticated source, we might have to have a quick screen sharing session!

Thanks Barry, looking forward to hearing from you soon!

Alex

Comments are closed.