I actually have all the time had a soft spot for words that completely capture the essence of an idea. During one among my trips to Japan, I discovered the word Tsundoku. It refers back to the habit of acquiring books and letting them pile up without reading them. I immediately fell in love with the word because, like many, I actually have a habit of shopping for more books than I can read. Some I ultimately get to, while others simply accumulate.
There’s something amusing and oddly satisfying about these piles of unread books — they symbolise potential knowledge and the enjoyment of collecting. They stand as a testament to our mental aspirations, even when we don’t all the time fulfil them.
As founding father of Kindata.io and a consultant for giant corporations on data value, I often encounter usage of knowledge catalogues that make me wish to scream, “Tsundoku!”. These catalogues, like stacks of unread books, are stuffed with detailed descriptions of knowledge that sits idle, giving a false sense of accomplishment. A book not read is knowledge wasted; similarly, data not exploited for business value is potential wasted.
No person likes waste, and the professionals I work with aren’t any exception. They’re desperate to see their data used to its full potential. Additionally they understand that getting there would require a change in mindset. Working in data, in addition they appreciate the facility of well-named concepts. I actually have heard many terms getting used for this work of alignment to business value, but I have to say that I actually have grow to be keen on one which is picking up: Data Value Lineage. This term resonates deeply with me since it perfectly captures what I’m championing. It highlights the necessity to turn idle data into actionable insights by ensuring that every thing that you just do in the info teams is firmly linked to value creation.
At first glance, selecting to call an idea a method as a substitute of one other may appear inconsequential. Nonetheless, naming concepts is powerful especially if that you must bring with you a complete organisation.
Within the sections that follow, we are going to delve deeper into the specifics of Data Value Lineage. After defining the concept and what it entails, we are going to explore learn how to start with a realistic implementation approach inside your data organisation.
Data value lineage is the aggregation of two terms commonly used amongst data professionals: data lineage and data value. Let’s get back to the roots of those concepts.
Data Lineage
Data Lineage is defined by the IBM Knowledge Center:
“Data lineage is the means of tracking the flow of knowledge over time, providing a transparent understanding of where the info originated, the way it has modified, and its ultimate destination inside the data pipeline.”
Data Value
“Data Value is the economic price that an organisation can derive from its data. This includes each tangible advantages like revenue generation and price savings, in addition to intangible advantages like improved decision-making, enhanced customer experiences, and competitive advantage.”
This definition is synthesised from commonly accepted industry principles, as a direct authoritative source just isn’t currently available.
Reconciling two different worlds
The 2 previous definitions take us in numerous directions. Data lineage is all about understanding dependencies, bias, and potential quality issues. It provides explainability and helps data engineering teams repair broken pipelines. It’s effectively a proper description of your data supply chains. Yet, for most individuals, it stops when the info is effectively utilized by an application.
Data value then again focuses on something completely different: the contribution of knowledge to the organisation business objectives. The underlying assumption is that this data value could be formally measured. We’re firmly on the planet of strategy, finance and business cases, not on the planet of pipelines debugging.
I’m now picturing Jean-Claude Van Damme performing one among his legendary splits on two high piles of unread books and pondering very hard about unifying these two concepts…
A proper definition of knowledge value lineage
Here is the primary formal definition of knowledge value lineage.
“Data value lineage is the means of reconciling in any respect times the enterprise data assets (including their maintenance costs) and the principal delivery tasks performed inside an information organisation with the measured contribution to enterprise value drivers”
There we’re, we’ve a proper definition! I would really like to insist on a few elements of this definition that I believe are fundamental:
- Process: This just isn’t nearly providing after the actual fact documentation, that is about ensuring that the organisation is considering the link to business value as a part of its standard ways of getting things done.
- In any respect times: Things change and contribution to business objectives just isn’t insulated from that. Data providing elements of value at one time limit is just that, no more, no less and there isn’t any guarantee that this value will hold through time.
- Enterprise data assets: I’m purposefully using a large notion. In fact when prioritising data value lineage initiatives, it is advisable to start smaller. My very own pragmatic advice based on our first projects with kindata.io could be to start out with data products and business use cases (more on this later).
- Maintenance costs: keeping data assets available for consumption has a value. A part of it’s cloud resources (where this rejoins Finopps practices) but rather a lot is data engineers’ time allocated to maintaining, repairing and evolving the assets.
- The principal delivery tasks: That is pushing the definition slightly but ideally, any significant task performed inside the data organisation should a method or one other be linked to value drivers. If time is spent improving timeliness of an information set for instance, how does this provide measurable business value?
- Measured contribution: We would like as much as possible to formalise the contribution to business value. It’s critical that these measurements are performed by the business sponsors, not inside the data organisation. We also should be pragmatic in these measurements, it is a means, not an end.
- Enterprise value drivers: We’re aligned with the definition of Gartner of business value. Anything that the business values counts, much more so if it might probably be measured.
As you’ll be able to see, data value lineage, while inherently a straightforward concept, covers a really wide scope. Inevitably, this involves bringing together individuals inside your organisation with very different profiles, mandates and concerns. Let’s explore learn how to get things moving pragmatically in the proper direction.
Some change management considerations
Attending to think and act across silos and mentalities just isn’t trivial. Inertia is a strong force and putting eyes blinders on is usually the one technique to get anything done in large corporations.
Through the various change management projects that I actually have driven and even observed, I actually have found three invariably useful critical success aspects:
- Meaning and worthiness: When changes “just make sense,” they’re more easily adopted. This ultimately results in the sentiment, “Why haven’t we all the time done it like that?” When the brand new approach feels intuitive and clearly higher, resistance diminishes significantly.
- Natural extension of existing practices: Change is more prone to succeed if it requires minimal deviation from current practices. By allowing people to maintain doing most of what they were doing before and changing small things consistently, you integrate the brand new practices easily into their routines.
- Immediate perception of value: For any change to be embraced, individuals must see immediate advantages. If things that were hard or out of reach grow to be easy and natural, persons are more prone to adopt the brand new practices enthusiastically.
The excellent news is that it is comparatively easy to tick all three boxes with data value lineage.
- Meaning and worthiness: It doesn’t take much research to seek out loads of inefficiencies in resource allocation and collaboration between data producers and consumers. Also the concept of knowledge value lineage comes quite naturally to very different profiles.
- Natural extension of existing practices: You don’t fundamentally change what you do, you simply add a skinny extra layer on top to make connections which are otherwise hard to determine. You leverage your existing investments in data governance and financial control and activate them to deal with business value generation.
- Immediate perception of value: By connecting the dots, you’ll be able to very fast discover untapped potential of knowledge, increase the business throughput of your data resources, improve the info discovery process and promote data democratisation.
Getting began
What I actually have found particularly useful in rolling out an information value lineage approach across an organisation is to start out with the three basic questions: what, how, why. This might sound simplistic but in point of fact most organisations are continually mixing these three questions together and find yourself not getting any clear answers.
What refers to the info initiatives which are valued by the business. Within the projects that we run, this covers each traditional analytics projects (dashboards, reports…) in addition to data science / AI initiatives (predictive models, advice engines, chatbots…). The term that we use for a data-driven project that gives tangible business value is a business use case.
How covers many elements but from the angle of knowledge value lineage, an important one is data sourcing. Increasingly more organisations, inspired by the principles of data mesh are adopting the concept of data product or productised data set. Contrary to business use cases, on this terminology, an information product doesn’t contribute on to business value, yet they arrive with maintenance costs.
Why is in regards to the contribution to business value. How do you expect each business use case to contribute to at least one or several value drivers? Once the projects are delivered and enter into maintenance mode, is that contribution sustained?
Once we’ve a transparent understanding of those three questions, we are able to start documenting the essential constructing blocks:
- The worth drivers and their metrics
- The portfolio of business use cases
- The catalogue of knowledge products
The primary pragmatic level of knowledge value lineage is to define the connections between these three levels.
Let’s take a quite simple example:
You wish to optimise your energy consumption through a data-driven approach. The business use case (energy cost reduction) sources data from two data products (Company Energy Consumption and Utility Bills and Tariffs). It contributes to 2 business drivers: Cost Reduction and Sustainability.
The arrows between the info products, business use case and value drivers are the backbone of knowledge value lineage. They make the link between the three fundamental questions. You’ll notice that the arrows are bi-directional:
- Whenever you navigate from data products to value drivers, you get a transparent view of the real usefulness of your data products. Each individual data product is indeed a part of many similar consumption chains and you’ll be able to easily get the massive picture of generated value in any respect times. If the generated value doesn’t live to your expectations, you’ll be able to take corrective actions resembling internal promotion, refactoring or decommissioning.
- Whenever you navigate from business driver to data products, you get an up up to now top-down view of your data-driven contribution. Again, if the image just isn’t aligned to your expectations, you’ll be able to initiate strategic decisions starting from investment in data supply chains, delivery of specific latest business use cases or boosting of the usage of existing ones.
Data value lineage is a strong concept because it offers a structured approach to making sure that each piece of knowledge in your organisation contributes to business value. By reconciling data assets, tasks, and business outcomes, you’ll be able to maximise the impact of your data initiatives.
It’s also an incredible name that may rally people across the info, business and financial control organisations.
Don’t just find contentment in creating dozens of underused data products, fooling yourself with the illusion of value generation. Avoid the zen contemplation of unused piles of knowledge, Tsundoku-style. As an alternative, take motion and harness the total power of your data through data value lineage, ensuring that each effort directly contributes to business value generation.
