« All Episodes

The Good and Bad of Choosing Measurements - Traps and Opportunities of Measuring What Matters

Published 10/17/2025

In this episode, I dive into the management mantra that "what is measured is managed" and explain why this simple assertion often leads to a complex trap. We discuss why the act of measuring team productivity is never neutral—it's an intervention that immediately changes behavior, often resulting in unintended consequences like gaming the metrics. We'll explore how to collaborate with your team to find measurements that truly drive desired behaviors.

  • Understand why the phrase "what is measured is managed" means that measuring something triggers management of that thing, in varying degrees.
  • Learn why the act of measuring something is an intervention, especially when done with the intent of turning it into a target (e.g., increasing PRs or decreasing bugs), and how this action is shown to change behaviors around the thing being measured.
  • Discover how measurement can lead to unintended consequences, such as when tracking velocity via story points causes team members to inflate or deflate their estimates, making the measurement itself less meaningful.
  • Explore why giving ownership of metrics to the people acting to improve them makes sense, and how you can collaborate with your team to determine what kinds of measurements should be taken.
  • Recognize that if you want something to survive—such as paying down tech debt—you must feed it with time, resources, and attention, ensuring your actions line up with what you claim to care about.
  • Understand that since your calendar and dashboards will get crowded if you measure everything, the exercise of choosing metrics is as much about accepting that you have to choose what you will actively manage and invest in.

📮 Ask a Question

If you enjoyed this episode and would like me to discuss a question that you have on the show, drop it over at: developertea.com.

📮 Join the Discord

If you want to be a part of a supportive community of engineers (non-engineers welcome!) working to improve their lives and careers, join us on the Developer Tea Discord community by visiting https://developertea.com/discord today!

🧡 Leave a Review

If you're enjoying the show and want to support the content head over to iTunes and leave a review! It helps other developers discover the show and keep us focused on what matters to you.

Transcript (Generated by OpenAI Whisper)

Hello, everyone, and welcome to Development Team. My name is Jonathan Cottrell, and my goal on the show is to help different developers like you find clarity, perspective, and purpose in their careers. In today's episode, I want to talk about this often repeated phrase in management. It's not just in engineering management. It's in any modern organization. Somebody in leadership has probably repeated this phrase because it was written down a long time ago by some management leader. I'm not even sure who. Probably somebody who thought very deeply about the subject. But nevertheless, the phrase is what you measure gets managed, right? What is measured is managed. And this is on its own. Let's face it, it seems true, right? It seems likely that if you want to manage something, the first step to being able to do that is to get a handle on it, to know what it is, right? If you want to improve some particular outcome, then you need to measure the outcome to determine where you even are to begin with. And so we imagine that if we wanted to, let's say, for example, a very naive interpretation of this, of this assertion, is that if we wanted to manage how our people on our teams are spending their time, then we would directly measure how they spend their time. So we might implement some kind of time tracking, self-reported time tracking, or we might implement some kind of more extreme example might be surveillance software. Maybe we'll start counting the number of actions that they take in particular platforms. This is a very common kind of management tooling direction. And there's platforms out there that will do this for you. And I'm not here to say that you shouldn't do this or that you should do this, but rather you should understand the effects. If you're going to do something like this. So the reason it's naive is because we believe that measuring is an innocuous action. That it is somehow fundamentally neutral to observe something. Even at the kind of atomic level, this isn't true, right? We know that that's false. But it has also been proven true that measuring something, especially measuring something with the intent of turning it into a target, as in we want to change whatever we are measuring here. We want to increase the number of PRs. We want to increase the number of tickets delivered. We want to decrease the number of bugs reported. We want to increase the number of tickets delivered. We want to increase the number of tickets delivered. That the measurement action itself has been shown to change behaviors around the thing being measured. Additionally, human behavior changes when the person that you're observing, when they believe, not even when they know, but when they believe they are being, that they're being watched. That there's some surveillance watching them. People will change their behavior, usually in protective ways, right? And so we should read into this assertion that what you measure is managed. To more specifically say, when you measure something, you are triggering management. In other words, measuring something causes management of that thing. And this is, you know, in varying degrees, right? Just measuring something is not what we're talking about here. What we're talking about is measuring something with some kind of visibility, some kind of, you know, discussion or targeting, you know, bringing this in front of the team for your manager and saying, we want to move this number from 20 to 30, right? We want to increase. Our engagement in some way. We want to reduce the number of meetings that we have, whatever the intent is. If that's shared, then doing that sharing is in and of itself going to change behaviors. Now you might say, well, of course, that's exactly what I want it to do. I want to change behaviors here. That's the whole point of setting these goals of measuring, of trying to manage this stuff. We want people to do something different. So we're creating some kind of. Framework of measurement to, to encourage the change. What this ignores though, is that there are many ways that behavior may change. Some of them are ways that you wouldn't predict or even desire. Very simple examples of this. Say you start measuring your velocity by looking at story points, right? And if you previously had a habit of pointing stories. And now you start measuring story points, excuse me, I have a cold. You start measuring story points and you recognize that, you know, your velocity is less than you want it to be, or you want to increase it by 10%. It's very difficult. Uh, when you're measuring from this kind of disconnected perspective, it's difficult to know the difference between inflated estimates, right? Story point estimates. You've probably. All seen this. Um, and actual increase in velocity. And you can also have a rebound effect of that too, where people are afraid of looking like they're gaming the system. And so they will deflate their estimates. So you have a drop in velocity that is actually not meaningful. The measurement itself is causing this kind of, uh, secondary and third order effects. And so. You. Put these measurements in and you think that there is initially a measurement and then an intervention. But the truth is the measurement itself is an intervention. And sometimes that intervention is difficult to control. And so it makes some sense to avoid inserting measurements or making visible only the things you as a manager, you as a tech lead. You as a VP or head of engineering or CTO. Only the things you think should be measured. If you're trying to improve velocity. It might make sense to start by asking your team. What kind of measurements should we take to measure our velocity? And if the team can help determine, okay. You know, if, if we are trying to figure out what our velocity is, then. Maybe number of stories. Or. Uh, number of features shipped. Maybe there's a way to track that. Maybe there's a way. To, uh, to measure our productivity in a different way. That is more accurate. If you give ownership of these metrics to the people that are acting in order to improve them. Uh, and if you, if you allow that to be a more collaborative process. Well, you're not saying, well, I'm just not going to measure them. Right. That, that would be, uh, an error in. In another direction. If you choose not to measure, then you're, you're not managing anything potentially. Right. So it does make sense to, to measure something. But recognize that measurement in and of itself is an intervention. That it will trigger some kind of management. So the trick is to ensure that the things you are measuring are triggering the right kinds of management. Right. That you are triggering the kinds of reaction, the kinds of. Uh, mod. Of. Uh, modification. That you desire in the team. Now it's not always a perfect system. You're not always going to get that right. Sometimes you will have behaviors. You will have reactions. That you don't necessarily, you didn't really want to drive. You didn't want that to be the reaction. Uh, but if you continuously. Uh, inspect these, these kinds of interventions, these measurements, and you collaborate with the. Uh, the people that you are, uh, you know, implementing. Uh, you know, implementing. Uh, the people that you are implementing them on behalf of. You're much more likely to, to come out with something that's a useful direction. Right. It's also important to note that eventually some of these gaming mechanics. Um, you know, these, these inter, these, this kind of responsive intervention, like if the team was to inflate their story points. That eventually ends. If you were to inflate your story points. And then continue to inflate them, then continue to inflate them. That becomes pretty obvious eventually. Right. So it's more likely that they will inflate the story points. Initially. And then, uh, you know, things will settle out. It's also important to note, by the way. That this is, this is not some malicious intent. That we're uncovering in the human psyche. People are going to respond to measurement naturally. Uh, they may not even know. And you may not even know. And this is what makes it so tricky. You may have full trust on your team. And your team may be accidentally gaming the metrics. Right. They may be, you know, in, in totally clear conscience. Uh, you know, gaming some kind of metric by inflating story points. Unknowingly. This is completely normal. Right. And we keep coming, coming back to story points. Cause it's the most obvious example that most of you have probably dealt with. But there are other things that make a ton of sense. Uh, to, to measure. To make visible. You know, the, the original kind of impetus for this discussion is the fact that. A lot of times we say we care about certain things, but. Our, our processes don't react or don't, don't, uh, represent that we care about them. So we say that we care about setting goals, but we have no calendar events. Where we are sitting down to try to actually walk through the process of setting a goal. We say we care about, uh, you know, paying down tech debt. But we have no process. We have no, um, you know, no, no sustainable way to invest in that. So one principle I believe in, and I've talked about this on the show before is that when you want something to survive, you feed it. If you want something to die, you starve it in business. If you want something to survive, you spend time on it. You spend money on it. It's been some kind of resources. You put somebody's attention, somebody's passion towards that thing. If you want it to die. You force it into a corner. You talk about it. Rarely you take the meetings off the board and you take a totally async, for example, right? You don't designate anybody to own it. Um, you're starving that thing. You're saying that you care about it maybe, but your actions don't necessarily line up with that. Well, this is true in the things that we measure as well. If we say we care about our team velocity. But we're doing nothing to ensure that we're doing it. We're doing nothing to improve our team velocity. We're not measuring anything related to velocity. Do we actually care about it? Or do we care about people thinking that we care about it? These are two completely different things, right? So, uh, if you do care about something, you should be able to point to what you're doing to show that you care about it. To actually invest in that thing. Now, hopefully what you'll recognize is that things start to get crowded. Right? Your, your calendar will get crowded if you have a meeting for everything that you care about. Um, your, your dashboards are going to get crowded if you have 30 metrics because you care about all of them. So, this exercise is as much about accepting the fact that even though you wish you could care about a lot of things, you have to choose what you're going to actively manage. What you're going to actively invest in. And so, if you're going to actively invest... Invest in getting this, you know, this particular metric right or improving this particular thing, then you need to make space for it. You need to figure out the right measurements. You need to figure out the right way to make it visible. Right? You need to figure out the right kinds of investment. Um, and, and pay attention to the fact that all of these things, when you put it on the table, the act of measuring, the act of making it visible, the act of, you know, putting it in front of people will in and of itself trigger management. It will cause management. And you have to be ready for what that means. Think through how can you, you know, intentionally shape the kinds of triggers that you're creating, the first, second and third order effects. How do you make sure, you know, that, that you're collaborating on these kinds of interventions with your teammates? Thanks so much for listening to today's episode. Hopefully you enjoyed this episode. If you enjoyed it, uh, leave a review in iTunes. It's actually still one of the best ways to help the show continue succeeding, uh, you know, past 10 years now. So, um, you know, we're, we just now, um, we're, you know, we move very slowly sometimes because this podcast is not my full time job, but, um, we started publishing this YouTube. Um, so you may be watching this, you may be watching my face right now on YouTube, or maybe you're listening to this on iTunes, uh, leaving a review in iTunes, subscribing in YouTube, subscribing and whatever podcasting app you use to make sure you don't, make sure you don't miss out on the show, but also because that helps the show continue to succeed into the future. Thank you so much for listening and until next time, enjoy your tea.