Businesses are going big with their big data strategy.
According to a 2017 study by New Vantage partners, 37.2% of surveyed organizations have invested more than $100M in a big data strategy within the past five years, with 6.5% investing over $1B.
And it’s paying off. Nearly 50% of companies report they’ve successfully reduced costs using big data, and another 44% report that big data has created new avenues for innovation. Clearly, businesses are seeing the rewards in pursuing a big data strategy.
But the big data ecosystem can be daunting to contemplate with its ever-growing list of vendors and buzzwords. And there is significant risk involved with pursuing a big data transformation: Gartner Research estimated that 60% of big data projects in 2017 would fail or be abandoned. Gartner analyst Nick Heudecker subsequently tweeted that the failure rate was actually closer to 85%.
With so much at risk, companies need to think carefully about the strategy they will pursue to gain a business advantage from big data. Here’s what you need to know to shape the ultimate big data strategy.
Have a Bigger “Why”: Know Your Business Case
Big data isn’t easy. It demands forethought, technical skill, data wrangling and political subtlety. Articulating a clear and compelling vision for how your business will be more competitive, profitable, efficient and customer-focused – and having a working roadmap – will go a long way towards keeping your big data transformation focused and agile. As the saying goes, “The key to saying ‘no’ is having a bigger ‘yes.’”
Forrester Research suggests the following guidelines for developing a business case for a big data strategy:
- Define today’s use cases. Organizations should first catalog their current, operational processes that use data, including a description of where the data comes from and where it goes. This list should encompass business processes, applications and data elements used by each division of the business (HR, Finance, Operations, Sales, etc) as well as a summary of the user journey, transactions and analytics for later analysis.
- Define tomorrow’s use cases. One future use case includes current processes that could be enhanced with improved data and insights, such as:
- Marketing campaign analytics to paint the picture of the customer journey toward conversion and attributing sales to marketing channels.
- A hospital analyzing discharges to predict high readmission risks and drive proactive, preventative outreach.
Another future use case includes entirely new analytic insights, such as predicting demand or even individual customer behavior to drive targeted promotions.
- Find the data. The needs assessments described above should help map out your “point A” data management culture. This map should include the internal and external data sources, data platforms and applications that support current processes, and will need to be evolved and enhanced to support your future processes.
- Align with corporate strategy. No big data strategy can live in a vacuum. Even with real-world input of use cases and data management architecture as described above, a big data strategy needs to account for, align with, and be prioritized by the organization’s overall strategy.
Prepare to Govern
Before you get lost in the ever-growing constellation of big data buzzwords and vendors, start building a culture of data governance. The first step towards effective data governance includes assembling the right resources with the necessary authority and skill sets to define an agile data governance regime. This team should include:
- A C-level executive with oversight of data governance: Data governance requires a C-suite advocate to engage other business executives and influencers on shaping the big data strategy and defining resource and procurement needs.
- Data governance leader(s): Designated by business leaders, not IT, these resources flesh out the policies, procedures and standards for the effective use and management of data. It is their job to ensure that data governance initiatives align with business strategy.
- Data stewards: These are front-line resources with a deep understanding of how their respective lines of business use data today and will use it tomorrow. They’re tasked with enforcing data governance policies and standards as well as monitoring data needs and reporting new requirements back to the business.
- Enterprise architects: These resources build and manage the technical infrastructure that manages an organization’s data. Their work includes provisioning data using established policies and protocols to any data consumer.
- Data analysts: There’s no such thing as a perfect data source. There will always be a need to test for data quality and fix data quality errors. Data analysts fulfill this need by temporarily fixing data issues and engaging data source owners to permanently remediate issues.
As implied above, it’s imperative that the business define data governance policies and IT implement those policies. Informatica puts it this way:
“The ultimate objective of data governance is to generate the greatest possible return on data assets. If business wants to be sure to capture critical opportunities to leverage data to support operations, strategy, and customer experience, it needs to govern data assets as it does other enterprise assets such as financial securities, cash, and human resources (HR).”
Businesses need to see data as an asset that can be mined and refined for business value. With this attitude in mind, a big data strategy is in position to find data that can benefit the business.
Find the Data
With the business case in mind and data governance controls in place, you can begin discovering, evaluating, sourcing and conforming data. The needed data could reside in siloed legacy systems or even outside the organization in social media conversations. Organizations should think expansively about the data that can feed their big data strategy, which can include:
- Archived data: Scanned versions of forms and statements created by legacy systems.
- Internal documents: This includes the myriad of files that can be created by desktop applications, like word processing files, spreadsheets, HTML pages, PDF files, etc.
- Multimedia files: Think digitized pictures, videos and audio files.
- Operational and analytic databases: Most organizations usually have some data stored in SQL, NoSQL, and/or Hadoop environments.
- Business applications: Think of the traditional, enterprise-class applications used to run businesses, like ERP, HR, CRM, PoS, and content management systems.
- Social media data: Unstructured text from social media platforms that can be used for brand sentiment analysis.
- Sensor data: With Internet of Things (IoT) technologies, sensors attached to smart devices can provide geolocation, temperature, noise, attention, engagement and biometrics data.
- Public web sources: This includes publicly-available data from both private and government sources on a wide array of topics like traffic, finance, stock markets and government health data.
- Machine log data: This is data captured at the machine level, usually involving activity on servers and mobile devices.
The trick here is to make the right investments in new data architectures and rigorous data governance methods that maintain a common definition and source of quality data on an ongoing basis. Maintaining some order around your data will greatly simplify the building of analytic models for business value.
Build The Models
Analytic modeling is the practice of applying data science to a company’s data to meet the needs of the business case. No matter if the model in question is exploratory, optimizing or predictive, all models must have a documented business value, end user(s) and tight management to ensure a “single source of truth” for a given model.
As your big data strategy grows, matures and creates value, you may have opportunity to join analytic models across different functions of the business. For example, a model that optimizes materials procurement could be linked to a model that predicts manufacturing throughput. Of course, this approach depends on an agile system of model management to provide consistent insights to your user base through Big Data applications.
Deploy the Tools
The best data and most elegant analytic models in the world are worthless if their target users don’t use them. The presentation of business insights derived from analytic models must be easily accessible, easy to use, embedded in target users’ daily workflow and focused on effective business decisions.
So, the effective development and maintenance of big data applications depends on a lean and agile approach to managing product roadmaps. Organizations must strive to deliver only the insights needed to target users when they are needed. Iteratively deploying and enhancing big data applications should help prevent application bloat as well as keep end users engaged.
Big data is more than a trendy buzzword. It encompasses data sources, technologies, best practices and a culture that marshals data assets for maximum business advantage. Organizations considering a Big data strategy should think clearly about the business case for their particular industry and corporate culture. Then, with a governance framework in place, companies can organize their data for modeling, analysis and distribution to their target users at the right time and place.
In this way, a big data strategy turns a company’s data into a strategic asset that generates business value.