With 90 percent of the world’s data having been created in the last two years, demand for data-driven insights leveraging web data continues to escalate. Total spend on Web Data Integration (WDI) is estimated to hit $5 billion in 2019 according to Opimas Research.
Web Data Integration has been getting a lot of press recently with industry analysts like Opimas Research and Ovum both reporting on the market. DMN and TechTarget have also recently reported on Web Data Integration use cases. To scale and meet the fast growing enterprise demand for its solutions, Import.io recently announced new capabilities to its Web Data Integration platform.
But what is Web Data Integration?
“Web Data Integration is a new philosophy that empowers businesses to take full advantage of web data for mission critical business purposes. It is defined by high-quality data that is rapidly delivered and integrated directly into business processes with low resource requirements and little to no business risk for the user,” said Gary Read, CEO of Import.io. “In the same way that SaaS platforms for business intelligence, CRM and hundreds of other disciplines have taken away the need for engineers to write their own solution, Import.io is doing the same for the web data market. No more coding, no more bad data, no more worrying about whether the data will arrive on time.”
Why should organizations care about Web Data Integration?
The web is the world’s biggest repository of data and is a source of tremendous potential for data-driven insights. The problem is that the data on the web is not structured nor is it organized to be read by machines; it is formatted and presented to be read by humans. To fuel insights and innovation, this data needs to be made machine readable so that it can be consumed at scale.
In the past, teams have tried to use legacy web scraping tools to extract the web data they needed. However, these basic tools left organizations with data that’s incomplete, inaccurate, unreliable, and out of date—while introducing high costs and business risk.
Tony Baer of Ovum recognizes this but added that “when treated with the same level of data validation discipline that is normally accorded to conventional BI data or big data, web data can yield valuable insights. With Web Data Integration, web data now becomes high-value data that can be used to generate insights, complementing or augmenting the enterprise data that sits in data warehouses.”
Why Web Data Integration Now?
The cost of legacy approaches to accessing and leveraging web data is enormous. In 2019, nearly $4.7B will be spent on the internal teams and infrastructure employed to get web data. For that $4.7B investment they are often getting:
- Poor quality, unreliable, dirty
- A process that is resource intensive & specialized
- Legal risk and uncertainty
- Stand-alone data sets
- Long lead times to getting new data
Import.io’s Web Data Integration
Today, approximately 80 percent of spend on web data in 2018 was internal reflecting that up until now, and there has not been an enterprise-class SaaS solution available. Clearly Import.io saw this as an opportunity as it acquired Connotate to accelerate growth and further solidify its market leadership position in Web Data Integration.
Import.io extracts data from alternative sources such as industry blogs, social media sites, social news aggregation sites, ecommerce and travel aggregators, retail sites, and just about any website. Extracted data is then prepared by cleansing, normalizing and enriching the data using Import.io’s comprehensive transformation capabilities. Finally, prepared data can be integrated to internal business systems, delivered to any data repository or visualized with Import.io Insights. This automation can be scheduled hourly, daily, weekly, or monthly to ensure freshness and analysis on demand.
With its Web Data Integration solution, Import.io enables users to quickly and cost effectively deliver high-quality web data at enterprise scale, without requiring expensive engineering teams to constantly be writing code, monitoring quality and maintaining logic. By doing this, Import.io allows enterprises to drive positive business outcomes by gaining deeper insights from robust alternative data sets.
In our next blog, we’ll take a look at some of the differences between Web Data Integration and web scraping.
Ready to try a Web Data Integration platform? Sign up for a 7-day free trial.