By Import.io CEO, Gary Read, Forbes Councils Member
Forbes Technology Council
Original post: Forbes Technology Council
While the alternative data market is thriving—and projected to be worth $350 million in 2020, up from $183 million in 2016—we have witnessed organizations suffer from alternative data fatigue in 2019. The same groups of alternative datasets are being packaged and sold to the same groups of hedge funds, who then typically spend more time preparing the data than analyzing it, and the market is becoming saturated with new companies all offering the same promise of competitive insights.
Secure Custom Information
In order to avoid alternative data fatigue, make sure you are securing custom information that will drive your business forward. If you’re reading this article, you’ve more than likely come here to learn how your business can harness the advantages provided by alternative data and are also likely aware of the immense hype that the business news media has proliferated surrounding the potential of alternative data to provide business-critical insights. Rest assured that if you’ve noticed the potential of alternative data, your competitors have also made this same realization or won’t be far behind.
Thanks to the marked increase of interest in alternative data, research firms offering the best alternative data insights have begun selling alternative datasets to investors, retailers, travel agents and other various trade industry professionals. Since everyone and their neighbors are currently hoping to gain a leg up on the competition by leveraging alternative data, a given dataset might be sold to two, three, four or any number of companies—each of which compete with one another and each of which are completely unaware they’re using the same information against each other.
Remember, the definition of “alternative data” can vary depending on what is considered a traditional source of data. Over time, data sources that were once considered alternative, nontraditional data sources become widely adopted, while new alternative data sources constantly emerge. In this case, if everyone is using the same alternative dataset, then the data is no longer “alternative” and becomes standard, traditional information accessible by anyone.
The Primary Alternative Data Fatigue Culprit
A large part of the reason so many organizations experience alternative data fatigue is the excessive amount of time that data scientists and teams tasked with handling alternative data initiatives spend harvesting and organizing datasets.
Most alternative datasets are composed of information pulled from multitudes of sources and websites. Useful web data may not be difficult to find, but gathering that information from multiple sources across the web detracts from the amount of time that could be used analyzing and interpreting findings. It also increases the chance for human error and creating duplicate and unnecessary entries.
Once the necessary alternative data has been compiled, it must now be standardized and verified as accurate. To be considered “accurate,” data must meet two criteria: form and content. The form criteria means that it must adhere to a standard format. Using a standard format prevents confusion and ensures that there won’t be ambiguity regarding the meaning of the data when it is analyzed by a computer.
The meaning of the data is the content criteria. Content is the information contained in the data or the message the data is communicating. For example, there are many ways that a single date can be written. January 10, 2020 can be written as 1/10/2020 or 10/1/2020, and these different forms can portray entirely different meanings depending on who is reading the data.
When this sort of alternative data collection and standardization is performed manually, the data science experts who have been tasked with extracting insights from the alternative dataset are unable to focus their efforts where they would be most effective.
Fighting Off Fatigue
So, how can your organization stymie alternative data fatigue? Here are a few suggestions:
• Automation: Typically, once an alternative dataset has been extracted and prepared, the data must be integrated into business processes to be analyzed and inform strategic decisions. Usually, the collected data is left as stand-alone files that must be manually integrated. Similar to the automated standardization of data, automating the integration process significantly reduces the risk of data science teams experiencing data fatigue, and allows them to focus their efforts on data analysis. Integration can be automated by preparing data with APIs to support frictionless integration with internal business systems and develop robust datasets for analytics purposes.
• Outside help: Companies hoping to best capitalize on the benefits of alternative data would be wise to seek an alternative data provider that offers a noncompete exclusion clause in any business agreements. This exclusion clause would essentially consist of a mutual agreement between the data provider and the organization using said data, in which the data provider promises to only share that specific dataset with the organization in question and will not resell that same dataset to any other organizations, regardless of industry.
• Customization: If your company decides to seek outside help, it’s also important to partner with a company that employs web data integration capabilities and can create custom datasets that will give you the knowledge your company needs to succeed. Web data integration treats the entire web data life cycle as a single, integrated process, with a focus on data quality and control.
• Other potential solutions: There are other ways to avoid data fatigue by collecting custom data. For example, sentiment analysis of social media feeds, news reports or corporate announcements creates datasets specific to your company. Using credit card data to gain insights into consumer spending behavior or using satellite or surveillance images to count cars in parking lots are also options for collecting unique data. Web scraping can provide custom datasets as well by programming your web scraper to collect the exact data that you need.
Organizations that don’t take advantage of the solutions suggested above will spend countless hours manually preparing their data for analysis and will undoubtedly continue to experience alternative data fatigue.