Use Case
September 2, 2026
Building a Metrics Stack That Tells the Right Story
# DevOps
# Leadership Insights
# Thought Leadership
# Case Study
How to move beyond standard DORA measures to reflect real flow, enablement, and customer value
Ja'Mesa Dixon
When the Metrics Do Not Tell the Whole Story
This conversation started because of the enablement work we are doing around our operating model. As we have worked through how that operating model shows up in our tooling, it has also initiated broader discussions about enablement and assessing its effectiveness. That led me to the DevOps team.
I initially went into the conversation asking about DORA metrics , but underneath that question I was really trying to understand something broader: how do we balance standard flow and delivery measures with what success actually means for a specific team?
There is value in standard measures. They create a common language and help leaders understand trends. But I also wanted to know whether they told the whole story for this team and the way it operates. That question opened up a much bigger conversation than I expected.
Starting With Familiar Engineering Metrics
We started with deployment frequency because it felt like a familiar place to begin. I asked about it, and the response from the team was essentially, "Okay, sure. We can measure that." It was recognizable, observable, and easy enough to discuss.
Then I brought up lead time, and the questions started immediately. Where does lead time begin for this team? Does it start when demand appears in ServiceNow? When the work is pushed into Jira? When engineering accepts it? Or when a code change enters the delivery pipeline? And where should it end?
Changing the starting and ending points changes the story. Measuring only after work reaches engineering may give us a useful view of engineering flow while leaving out the time spent waiting elsewhere. A broader value-stream question may need a different boundary.
Next came change failure. The team pointed out that they do not typically operate by rolling a change back. Their normal response is often to fix forward. That raised another useful question: what constitutes a failed change when recovery may come through a hot fix, refinement, or another change that moves the solution forward? The issue was whether we had a shared definition that reflected how the team actually works.
Production Is a Milestone, Not the End of the Story
The turning point came when the technical lead made a point about what happens after production. Solutions continue through enhancements, refinements, bugs, and other changes. For this team, a production release is a milestone. It is not necessarily the end of the value lifecycle.
That led me to ask a different question:
Does success mean that we reached production, or does success mean that we made someone’s life easier?
Everyone was essentially on the same page.
Reaching production matters. Delivery speed matters. Quality matters. But if a DevOps or platform team is building templates, pipelines, automation, and other enabling capabilities for technology teams, then one of the most important questions becomes: are teams actually using what we built? If a capability reaches production but does not improve the consumer experience, delivery activity alone does not tell us whether we created the intended value.
Measure the System You Actually Have
That brought us back to the measures we currently use, including committed versus completed work and story points. There is a reason leaders use them. In our environment, story pointing, including the use of points in predictability and cost conversations, has been part of the way technology leadership understands delivery.
Those measures are not inherently bad. The question is whether they are the best fit for every system. This particular team operates as a Kanban team, and the work has a high degree of variability. Story points can still provide information, but they may not give us the clearest picture of how work actually moves through this team’s system.
We were not trying to throw away familiar measures. We were trying to understand what each could tell us and what else was needed to see the full story. If the question is actual time, cycle time may be more useful. If the question is predictability, historical flow and service level expectations provide another lens.
The larger lesson for me was simple:
Build measurements for the system you have, not only for the system you wish you had.
Then, as the system improves, allow the way you measure it to evolve too.
Building a Smaller Metric Stack
Once we moved away from searching for one perfect metric, the list expanded quickly across flow, capability, platform health, business enablement, and enterprise outcomes. We considered everything from SLE attainment and cycle time to adoption, automation, self-service, platform health, and manual effort eliminated.
The problem was obvious: a metric stack that contains everything can create more noise than insight. So we narrowed the list to six measures that, together, could tell a more balanced story from delivery through adoption and outcome:
- SLE attainment (P85) - Predictability: Can consumers reasonably predict when work will be completed? As this team matures its Kanban practices, clearer workflow, managed WIP, and service level expectations can give us a more grounded way to discuss predictability.
- Median cycle time - Speed and flow: Are we getting faster or slower? Instead of converting estimates into time, cycle time lets us look at the actual elapsed time work spends moving through the defined part of the system.
- Aging or blocked WIP - Flow health: Where is flow breaking? Aging and blocked work can surface dependencies, bottlenecks, and constraints. This is especially valuable because the same thinking can move from the team level into value-stream conversations.
- DevOps capability adoption - Adoption: Are teams actually using what we are building? Templates, pipelines, and other capabilities only create value if consumers can discover them, trust them, and use them.
- Automation and self-service coverage - Enablement: Are we removing manual work and unnecessary dependency? This helps show whether enablement is making teams more autonomous rather than simply moving work to a different queue.
- Consumer outcome improvement - Outcome: Did application delivery become easier, faster, or safer? This is where the technical measures connect back to the reason the platform capability exists in the first place.
The Value Is in the Relationships
The value of the stack is not only in the individual measures. It is in what they can tell us together. A team may build a high level of automation, but adoption tells us whether consumers are actually using it. Cycle time may improve while aging or blocked WIP points to a constraint elsewhere in the workflow. Self-service coverage may increase, but consumer outcomes help us ask whether application delivery actually became easier, faster, or safer.
One metric can provide context for another. Instead of using a single number as a judgment of performance, we can use a small set of connected measures to understand how the system is behaving and where the next improvement opportunity may be.
Using AI to Organize the Conversation
After the team conversation, we used ChatGPT to help analyze what we had discussed. The practitioner conversation produced the questions and judgment; AI helped organize themes and consider a hybrid of flow and team-specific success measures. The output became another input to the discussion, not the answer, and we narrowed the ideas into the six measures that best represented the story we wanted to tell.
What Technology Leaders Can Take From This
This started with one DevOps team, but I think it is a microcosm of a broader challenge for technology leaders, especially as operating models evolve and AI changes how work is performed.
Start with the question you are trying to answer, not simply the metric that is already available. If you want to understand time, use measures that reflect actual time. If you want to understand predictability, look at the historical behavior of the system. If you want to understand enablement, look at adoption and self-service. If you want to understand value, eventually you have to ask whether the consumer outcome improved.
Value stream management can also help pinpoint waste, delays, and constraints beyond the boundaries of one team. As those constraints are addressed and the operating model matures, the measurement model should not remain frozen. There should be room to shift what is measured as the system changes.
Tell the Right Story
For me, the larger lesson is this:
The goal is not to find one perfect metric. The goal is to build the right story using the right measurements.
That means understanding how work flows through delivery, whether enabling capabilities are adopted, whether teams become more self-sufficient, and whether outcomes actually improve. The metric stack should help leaders and teams make better decisions, not simply produce a cleaner dashboard.
Most importantly, the measurements should reflect the system we actually have, not simply the system we desire. As that system matures, the metric stack can mature with it. When the measures tell the right story, they become more than reporting. They become a way to understand the system and decide where improvement matters next.
What about you? What metrics are you using and what path led you to that set? Do you think they might change in the future and, if so, why?
Sign in or Join the community
Where conversation, connection, and real-world practices come together.
Create an account
Where conversation, connection, and real-world practices come together.
4
Comments (2)
Popular
Dive in
Related
Blog
DevOps Information Overload Is Real, and It’s Quietly Wrecking Good Engineering
By Brian Teller • Jun 24th, 2026 • Views 90
Blog
From Monitoring to Observability: What Modern DevOps Teams Are Missing
By Sunil Agarwal • Jun 18th, 2026 • Views 42
Blog
DevOps Information Overload Is Real, and It’s Quietly Wrecking Good Engineering
By Brian Teller • Jun 24th, 2026 • Views 90
Blog
From Monitoring to Observability: What Modern DevOps Teams Are Missing
By Sunil Agarwal • Jun 18th, 2026 • Views 42