Designer News Statistics by Artiom Dashinsky
image courtesy of Genady Okrain
Every field has it’s go-to source for all the new and hot information you should know from around the industry. Tech-people have Hacker News created by Y-Combinator. But for designers, that site has to be Designer News.
Designer News is a community for sharing design-related stuff. You can easily find designers, product managers and design-oriented developers working in companies like Google, Facebook, Yahoo and Airbnb discussing the latest trends in the design world.
Every day more and more users join Designer News and since the community is all about showing you the hottest things right now, there is no good way to catch up on what’s been or see what the all-time best things ever posted to the website were. Or let’s say you have a free hour and you want to watch a good lecture about design and you go to DN – there is no way to search for a specific kind of content. So, I figured I’d build one! I was also curious to analyse the people on Designer News and to get a better understanding of the community.
I decided to collect all the data available on Designer News, analyse it and publish the results. First of all I prepared a list of information I want to figure out, like the most popular posts in every specific category, most active users, companies with the most amount of users participating on DN, top users titles, etc.
Putting It Into Practice
Designer News has a basic API which allows you to do things like build a mobile app. But the API didn’t give enough data for my needs. So apparently I had to build a scraper which could scan all the pages on the website, extract the data I need and analyse it.
I consulted with my code pals who convinced me that I need to find someone on oDesk or Elance who would develop the scraper for me. I didn’t really fancy spending hundreds of thousands of dollars employing a freelance developer for a personal project, so I decided to do a bit of research first to see if I could find a tool that would help me do it myself. A quick Google search lead me to import.io, which looked like an ideal solution.
Using import.io, I created two Crawlers – one for users and one for posts. Both of them had a very basic URL-pattern, which made them easy to crawl. After building my crawler configuration, I created a list of several thousand of URLs I wanted to scan, using a tool called Numbers, and copy- pasted it to import.io’s where to crawl box.
After several hours I had two spreadsheets with 30,000 rows. At this stage I started work on the design and infographics for presenting this data to Designer News community. I tried to use graphic, language and design patterns which were familiar to DN users.
Then, since my data analysis needs were pretty basic I just used Numbers and Wizard to sum up and organize the results.
After I’d finished designing the HTML of the site, Aleks was kind enough to write me a basic PHP-script which takes the relevant data from the CSV I exported using import.io and sends it directly to my website – instead of me having to enter in the new data manually.
Publishing the results
￼￼￼￼I was really excited by the results of my research and was impatient to show it to DN community. Just 5 minutes after posting a link to my new site – Designer News Statistics – with the visualisation of my results on the DN forum, it jumped to the first place. And after a day become second most popular thing ever posted on Designer News!
The feedback from the design community was great as well. I received tweets from the CEO of LayerVault and the creator of Product Hunt who wondered if someone would create a similar visualisation for the Product Hunt community. The page was also featured on Sidebar.io and within the first two days of being live, the website had received more than 10k visitors – which is huge considering DN has 7k registered users. And of course people on DN were crazy about it!
Now that I have such a powerful tool in import.io, I have some ideas for future projects! If you’re interested in helping me make some awesome stuff, drop me a line on firstname.lastname@example.org or reach me out on Twitter @hvost.