What is lean?
Lean is a methodology particularly suited to the development and production of data products. From the customer’s perspective, Lean considers any activity that does not directly contribute to what the customer truly needs as waste (Tempelman & Schildmeijer, 2023). Lean aims to identify and eliminate these wastes, leaving only value-adding steps. This approach leads to data products that better align with customer expectations, thereby achieving higher quality. Lean defines eight types of waste within a process, which can be interpreted in the data world as follows:
- Transportation: unnecessary movement of (master) data between systems, processes, or departments, consuming time and resources without adding value.
- Inventory: storing data that is not needed, such as outdated or redundant information, which offers no added value to current processes or analyses.
- Motion: inefficient actions and processes by people in obtaining and processing data, reducing productivity.
- Waiting: time lost due to users, processes, or systems waiting for data to become available or processed.
- Overproduction: collecting and producing more data than necessary, leading to excess storage and complexity without added customer value.
- Process complexity: unnecessary steps, excessive quality controls, and a lack of strategy or standards, resulting in inefficient and hard-to-manage data processes.
- Defects: incorrect data input, processing, or insufficient validation, leading to unreliable analyses and inefficient processes.
- Unused talent: underutilizing employees by neglecting their technical skills or business knowledge.
By identifying these types of waste and analyzing the operational process through the data management framework described in DAMA DMBoK, you can eliminate waste and create more customer value. Some wastes are easier to address than others; start with those that have a significant impact on value creation and are relatively simple to resolve.
The importance of a robust development process for a new data product
Understanding business processes and required decisions
A common misconception in data-driven work is that data dictates direction, whereas it is people (the business) who drive decisions. Data supports decisions and processes led by the business. Every dataset represents only part of the reality and requires interpretation within a broader context. Therefore, it is essential to first have a clear understanding of business processes and the decisions needed before diving into the data. As discussed in the first article, organizations create value through their processes, and data as a product can help make these processes more effective and efficient.
From push to pull
A well-known challenge is the lack of alignment between business and IT, which hampers the effectiveness of data products. The BI team plays a crucial role as a bridge between business and IT: while IT focuses on technical solutions (like API integrations and performance), BI translates business questions into data models and analyses. This requires a shift from traditional push thinking (produce as many data products as possible) to pull thinking, where the business clearly specifies the insights it genuinely needs.
Since these insights and specifications are not always fully known at the start, a flexible, iterative process is needed where the business, BI specialists, and IT collaborate on the data product. By remaining adaptable and regularly aligning goals, waste is minimized, and the product continues to meet real needs. Depending on the organization, the BI team may be organizationally under IT (a push situation) or positioned closer to the business (a pull situation). In either case, strong collaboration between all parties is essential to maximize value from data.
