Automating web scraping rental listings for competitive research

One of the beautiful things about is the ability to automate and schedule your data extractions (commonly known as web scraping). For instance, several of our customers need to extract data daily from different rental listing sites or retail websites to find who and what to target in their own marketing efforts. Command line scripts can automate this task, making it easy to get the data. For those who need the data, but may not know how to build a command line script – to the rescue!

We built a script for this use case and we want to share it with you. Our script allows a customer to pull URLs from a Google Sheet and feed them into an extractor. The script is implemented using Python, which is a common tool for building and sharing scripts. Customers only need to follow the instructions here.

The script automates the following steps:

  1. Copies URLs from Google Sheets to a designated extractor
  2. Initiates an Extractor run
  3. Provides status of the Extractor run
  4. Displays the list of URLs in the Extractor
  5. Performs the extraction of specific data from each URL
  6. Displays the extracted data in a Google Sheet

This is a common use cases that solves for customers. For a complete technical description, check out the README. You don’t need an IT pro to automate web scraping. can help with automation.

Extract data from almost any website