Data (opens in new tab) is the fuel that powers everything we do and has the potential to be a company’s most valuable asset. That is not a new line of thought in the industry, but it remains firm and supported by calculated forecasts: for example, it is widely estimated that there will be 175 zettabytes of data in the global datasphere by 2025. For the next two years, this equates to every person on the planet producing at least 1.7 MB of data every second.
Organizations are sitting on a mountain of data, which includes humans, machines, and Internet of Things (opens in new tab) (IoT) devices, as well as edge systems and beyond. It is everywhere, and its importance to businesses is undeniable in allowing them to thrive in today’s fast-paced world.
Data has the potential to become a significant competitive differentiator for businesses. However, many people’s current approach to data is suboptimal.
Most businesses continue to view data through the lens of a project. When they come across a problem that they want to solve with data, they launch a new initiative to collect that data, cleanse it, prepare it, and then analyze it for a specific use case.
However, a newer line of industry thought – and unfortunately, this approach – has several drawbacks. It’s time-consuming, results in duplicated work, and the outputs of each project can’t usually be repurposed to solve other use cases.
What exactly do we mean by “data products”?
Instead, organizations should approach data management as if it were a product, shifting focus away from individual challenges and toward solutions that can solve them repeatedly. In other words, they should adopt a product-centric (rather than project-centric) approach to data.
A data product, in simple terms, is a high-quality, ready-to-use set of data that people across an organization can easily access and apply to various business challenges.
Data products are typically built on scalable platforms by teams with expertise across the entire data lifecycle. They provide an individual with everything they need to understand and use data to solve a problem by combining three key components: data, public cloud technology, and custom developed business logic.
They are crucially designed with the entire organization’s data needs in mind. Data products, unlike data projects, can be repurposed to support a wide range of use cases across multiple functions.
It is also worth noting that they can come in a variety of asset types, such as reusable data sets, machine learning models, and dashboard reports.
Pursuing data-driven intelligence
The desire for businesses to make data-driven decisions has been around for decades, whereas data products as a solution may appear to be a relatively new concept. But they were already there; we just weren’t aware of it. Simple examples abound, such as Google Maps, which we can literally hold in our hands. They collect data on various routes from A to B and provide optimized answers for time or cost to reach your destination. Another example would be ChatBots, such as ChatGPT (applications that answer simple questions in online chat) in customer service, which reduces costs and response time. Expedia, for example, has transformed the experience and time required to book travel. Expedia constantly monitors changing flight and hotel prices and availability in order to recommend the best combinations for our needs.
What has changed in the last decade? The evolution of the public cloud, enabling a step function in the creation of largely unstructured data – be it files, video or web data, or IoT data – that is now stored on previously unimaginable scales. Not only that, but thanks to dramatic improvements in scalability and performance, the public cloud has made it much more cost effective to analyze massive amounts of data in near real-time.
As a result, the groundwork has been laid for the data product concept to evolve significantly over the last three to four years as businesses strive to integrate analytics into business processes and become more intelligent. And we’ve seen companies reap the benefits of successful data product development.
When organizations manage data as if it were a consumer product, they can realize the near-term value of their data investments and pave the way for faster future value. It promotes data democratization by eliminating the need for teams to waste time searching for data, processing it into the appropriate format, and creating bespoke data sets.
Time to insight is dramatically reduced when data products are pre-built and, typically, backed by well-defined interfaces with an emphasis on UX, while users can also rely on these models knowing that they have been rigorously tested.
The distinction between insight and instinct
Indeed, organizations that take a product approach to data are better able to improve decision-making, increase efficiency, and achieve lower cost-per-use than data projects, all while unlocking new revenue streams as they become more data-driven.
Importantly, these advantages are not limited to any one company, industry, or region. We’ve seen data products used in the oil and gas industry to facilitate dynamic gas pricing, allowing one firm to raise gas prices without reducing demand, for example. Meanwhile, we’ve seen a utilities company successfully deploy data products to reduce the time it takes to detect and resolve network(opens in new tab) anomalies by 60%.
The true value of data lies in enterprises’ ability to interpret data and drive intelligent decision-making in this way: businesses with a better understanding of their customers and products can improve their services and differentiate themselves from competitors.
This is precisely the advantage that data products provide; they are frequently the difference between companies making changes based on informed insights and misinformed instincts.
Changing a data culture
So, how does an organization transition from a culture of data projects to a culture of data products?
There’s no doubt that many businesses need to improve their capabilities across the board. To manage increasingly demanding processes, they will need to work on improving data quality and eliminating inconsistent data, developing more robust and reliable models, and increasing big data competencies, for example.
Furthermore, firms should work to implement best practices in the form of regulation and commitments to ensure that technologies are used in the most transparent, ethical, and equitable manner possible. This includes creating datasets with good representation, testing for bias by defining metrics across different subgroups, and performing model sensitivity analyses.
However, these things are likely to be achievable only if companies first secure, implement, and leverage the right skillsets, technology strategies, and partnerships that will propel them solidly into a relatively new space for them. For any data product strategy to be successful, companies must invest in training and individuals must make a personal effort to learn and embrace new skills.
Without a doubt, vital efforts like this will help to lay the groundwork for effective data products capable of empowering organizations of all sizes to tap into data and drive intelligent decision making.
