Better Low-Information Estimates - Putting the "Scientific" in SWAG
Published 12/17/2024
How can you create better scientific wild a** guesses when the work is not well defined? We discuss a few strategies in this episode. We also talk a bit about what an estimate is in the first place, and how it differs from a commitment / deadline.
🙏 Today's Episode is Brought To you by: Wix Studio
Devs, if you think website builders mean limited control—think again.
With Wix Studio’s developer-first ecosystem you can spend less time on tedious tasks and
more on the functionalities that matters most:
● Develop online in a VS Code-based IDE or locally via GitHub.
● Extend and replace a suite of powerful business solutions
● And ship faster with Wix Studio’s AI code assistant
All of that, wrapped up in auto-maintained infrastructure for total peace of mind.
Work in a developer-first ecosystem. Go to wixstudio.com
📮 Ask a Question
If you enjoyed this episode and would like me to discuss a question that you have on the show, drop it over at: developertea.com.
📮 Join the Discord
If you want to be a part of a supportive community of engineers (non-engineers welcome!) working to improve their lives and careers, join us on the Developer Tea Discord community by visiting https://developertea.com/discord today!
🧡 Leave a Review
If you're enjoying the show and want to support the content head over to iTunes and leave a review! It helps other developers discover the show and keep us focused on what matters to you.
Transcript (Generated by OpenAI Whisper)
With the end of the year comes a lot of reflection and time looking back. But for most of us, it also comes with a lot of looking forward, forward-looking planning for, for example, Q1. I want to talk a little bit about planning and some of the pitfalls that I've seen in my career and that you've probably seen in your career as well in the planning process, and specifically as it relates to planning ill-defined work. In most startup environments, work is not extremely well-defined before the organization, wants to make a decision about whether they're going to invest in it. And so the organization wants to lean a bit on the engineering teams, implementation teams, design, et cetera, to provide some kind of price tag. And I say this in this particular way, because I want you to shift your frame of thinking into your leader's frame of thinking. The people who are trying to make the decisions about, what kind of work are we going to pick up next? This question is looking for some kind of information in order to make that assessment, in order to make that decision. The leaders who are determining what projects to pick up next probably have some concept of a budget, mostly in terms of resources or FTEs, for example, and they want to evaluate projects based on their, expected cost and their series of benefits. If you're in a highly tuned kind of metrics driven organization, these projects are theoretically spread across your, you know, for example, your OKRs. If you are in an organization that is early in the startup phase, then you're probably looking for quick wins ROI that is realized as soon as possible. There's a lot of different strategies, strategies that these decision making kind of efforts would be going towards. But for the most part, what they're trying to do is identify some point of value. And this works both in agile environments as well as in waterfall environments. So you're looking for some delivery point and the cost to get to that delivery point compared to the benefits of getting to that delivery point. This simple ROI, calculation, of course, turns out to be more complex in reality. But in order to make reasonable decisions with as limited level of information, these simple factors need to be condensed down into the simplest form possible, which is why you are being asked right now, probably for estimates. How long would this take? But if your organization is like many organizations, the investment to define the work down to its clearest, smallest format may not necessarily be done, or perhaps it's done for some work and not done for others. In this episode, I want to give you some tactical advice for how you might estimate when the amount of information is limited. First, let's talk about today's sponsor. I'm going to talk about the! With Wix Studio, you can spend less time on your UI coding and fiddling, hosting, security, and more on the custom logic and functionalities that actually matter to your business. Develop in your preferred coding environment online in a VS code based IDE or locally through a GitHub integration. Either way, with Wix Studio, you're deploying in a click. You can extend and replace hundreds of powerful business solutions and custom built features with APIs and integration, and you can extend and replace hundreds of powerful business solutions and custom built features with APIs and integration, and when you need to speed things up, Wix Studio's AI assistant is on hand to generate tailored code snippets, troubleshoot bugs, and retrieve product answers in just seconds. All of that is neatly wrapped up into an automatically maintained infrastructure for total peace of mind. Work in a developer first ecosystem. Go to wixstudio.com. That's W-I-X studio dot com. Thanks again to Wix for sponsoring today's episode of Developer Tea. How do we estimate the unknown? We may take a few episodes to talk about estimation because it is a fundamentally interesting and deeply psychological process. And most people are used to doing estimates in only one very narrow way. And when I say narrow, I mean, I'm talking about estimates that are very narrow. I'm going to talk about a couple of different ways to approach this problem. Understand that the concept of estimation means something a little bit different to most other people. And so if you were to ask, let's say, any given engineering leader what an estimate means, they might say, oh, it's the number of story points that you put on a story. Someone else might say, what's the date that you expect to get something done by? The truth is, it can be parts of both of those things. But estimates are largely some guess about the future combined with some idea of how likely that guess is. OK, so think about this very, very clearly here. Some guess about what will happen in the future paired with some likelihood or some confidence that that's going to actually happen. In many ways, these are like casting bets. Bets on, for example, a sports event. If you notice, bets are always cast in virtually every sports event in both directions. And the reason for this is because no matter what we do, there is always some level of uncertainty before something happens. There is always some potential that the underdog will win in your estimates for, for your work. There may be some, you know, unlikelihood that you would consider disqualified. In other words, there's no way that we get it done this week. There's a zero percent chance of that happening. But as you extend further out your confidence interval, if you're going with a date based or date range based estimate, you could say, OK, it's going to get done in two to three weeks or it's going to get done in two to three months. Your confidence interval will go up as that time increases, because in virtually every case, the more time you have, the more likely more is getting done. And so as you accumulate the, you know, the completed work in a body of work, you are estimating the likelihood going up. And the further out you go, the higher the chance is that you've accumulated enough to complete that body of work. So that's the basic idea. Kind of. Defining an estimate. Some guess about the future with paired with some confidence interval. So what is not an estimate? Well, a single date that you are saying something will absolutely be done by. Deadlines are not estimates. So that's the key differentiator I want you to walk away with. A deadline is not an estimate. A specific concrete date is not an estimate. Usually, usually companies are using. Those specific deadlines as a commitment device. All right. And a commitment device. Operates on our psychology in a bunch of different ways. All right. So a couple of those ways. One, if you have a clear deadline, then you have created a container. This is following Parkinson's law. The idea is that work will fill the time that you've allotted for its completion. Right? So if you were to set. A shorter deadline than Parkinson's law states that. You would theoretically finish that. That work. In a shorter amount of time. Parkinson's law, I think. Needs to be balanced out with what is actually feasible. That is. If you set a very long deadline, it's very likely that the project is not going to complete. Until that very long deadline. But you can't set an arbitrarily short deadline. And expect it to have the same effect. Now, why is this important? Well, I want you to understand the difference between a commitment. The deadline. And an estimate. All right. But I also want you to be able to set or meet deadlines that are reasonable. Because this is a very difficult to learn skill. But one that is highly valued. Now, in my opinion. Deadlines are far too often abused. And can sometimes lead to. Unnecessary stress. And unnecessary stressful environments. But. If you can learn how to set deadlines in a way that is less stressful. And is more like a Parkinson's law trigger. That is. Let's say that you had a confidence interval of 90%. And you set a deadline that is near. That 90% mark. Then it's possible. It's possible that the deadline. Will help avoid. A scope creep. For example. Right. You'll have to. Have to make some harder decisions. Because you're facing the deadline. Upcoming. Okay. So. Understanding that deadlines are commitment devices. Very often they are intended to mitigate risk. That risk being. You know. Whatever that remainder 10% risk is. In your original estimate. A deadline. Or a commitment. Helps you avoid. A risk. Avoid that 10%. Right. Because the commitment is saying. I'm going to do. Whatever I need to do. To reduce that risk. To zero. On behalf of the company. Or else. My personal reputation. Or the team's reputation. Is put on the line. For missing a commitment. Now understand that again. The psychology difference here. Missing an estimate. Versus missing a commitment. Missing an estimate. Is essentially saying. Well. It seems like our guess was incorrect. It's squaring with the reality. Of the risk. That's inherent. That's a part of your estimate. Missing a commitment. Is saying. Well. I believed that something was doable. I told everyone that I would lay out. In order to make it doable. But unfortunately. It wasn't doable. Now it's important to understand that. Most organizations. For better or for worse. Use deadlines. They use deadlines mostly. Once again. As a risk mitigation tool. But it's also important to understand. Whether the deadline. Is real. In other words. What are the actual consequences. Both for the business. But also for the teams. If they actually miss a deadline. Does this come back. On let's say a performance review. What happens if you miss one deadline. Versus missing multiple deadlines. Without knowing. What the consequences. Of missing the deadline are. The arbitrary pressure. Of the deadline takes over. And our imaginations. Fill in the blank. With the worst possible thing. This can be. Even more stressful. Than just having a deadline. Okay. So enough talk about deadlines. I do want to give you some practical advice. Once again. For estimating work. That is not well defined. Okay. I'm going to give you three. Very quick frameworks for doing this. Or. Or. Or tactics. Ways of managing this kind of. Estimation. The first is called. Affinity estimation. Or relative estimation. You probably have done something. Kind of like this. With t-shirts. But I'm going to teach you. A slightly different path for this. If you have. Let's say. A body of work. That's about an epic. So let's say. Eight to twelve. Items. It could be eight to twelve. It could be twenty. Or it could be five. How do you know. Without breaking the word. Work down. Because. Let's say you're. You're not. You don't have enough time. To actually do the discovery. Before the estimate. Is needed for. That. Planning process. For example. Without breaking that word down. How could you make a pretty. Reasonable guess. As to about how much work this is. You're going from five to twenty. That's four times as much work. So which one. You know. Where should we lean. My recommendation is. You talk to the team. Especially the team members. Who have been on this team. Long enough. And have worked on. Something that is. Similar. Before. Right. So. Something that. They've done before. That they would say. Yeah. This is about the same level. Of complexity. About the same. Amount of work. Right. Even better. If you could find. Two or three of those. And then identify. You can do. Multiple things. If you're using. Something like story points. You could. Total up the story points. In those. What are called. Reference classes. Those. Previous bodies. Of work. That this team has done. That are similar. To this one. You could. Total up the number. Of stories. If you're not using. Story points. You could look at. The cycle time. Assuming that the team. Was totally focused. On those. On those. Efforts before. Could you. Look at the. The amount of time. That those efforts took. And apply a similar. Estimation. For the new one. Right. So. What you're doing. Is you're trying to get. Something that is. A little more. And then a little less. And then something. That's right in the middle. All right. Something that. The team says. Oh yeah. This is. This is right on. This is something. That is. Very close. Something that's a little more. Or something. That's a little less. If you had both of those. You could estimate. Somewhere in the middle. Right. So again. The goal is. Is to shift. The conversation. Away from. Trying to. Do a bunch of. Technical specification. And guessing. About the actual work. To be done. And instead. Use something. That humans are. Very good at. Very. Very. Good at. Which is. Relative comparison. Okay. We would. Have a better. Intuition. A better. Natural approach. To comparing. This new body. Of work. To a previous. Body of work. That we've done. Before. So this is what. Affinity. Estimates are. There is. Some affinity. Or connection. That this is. Similar. Or it is. You know. Tied to. This other thing. They are. Alike. In some way. The second. Is. A kind of. A modified. Version of this. If you were to take. You know. Those. The affinity. Highest. The affinity. Lowest. And the affinity. Middle. You can. Apply something. Called. Pert. And this is. A three point. Estimation. And it involves. Taking. These three. Points. Right. The. The lowest. The medium. Or the most likely. And the highest. And. Running. Them through. A really. Basic. Essentially. A distribution. All right. I'll give you that. Formula. Just a second. Another way. That you can get. The lowest. Medium. And highest. Is to use. The same. Concept. But use. Your team's. Kind of. You know. Swag. Guesses. In other words. Asking your team. Look at this work. And tell me. What is the worst. Case scenario. What is the. Absolute. Best. Case scenario. Or in other words. Optimistic. And pessimistic. And then. What is the most. Likely. Scenario. Once you have. All of those numbers. From your team. You can average. For each of those. Three points. And then. It's a basic. Calculation. You take. The. Worst case. You add it to. Four. Times. The most likely. Case. And then. Add that. To. The best case. Scenario. And then. Divide. All of that. By six. So the idea. Is you have. You know. Six. Possible. Kind of. Inputs. To this thing. One will be the worst case. One will be the best case. And the other four. Will be the likely case. The. Kind of the middle ground. Between all of these. This creates a distribution. That you're pulling. Essentially a mean point. On this distribution. And. If you. You know. Plug in some. Some kind of. Fake numbers for this. You can see that a very high. Worst case. Is going to skew the number. But it's not going to skew it. Completely. Because we're waiting. We're waiting. The total average. Towards the middle. Towards that most likely case. So this is a really useful method. Because. It allows you to take in. Multiple inputs. And it tries to. Approach something like. A. A statistical. Or a statistically backed. Approach to this. So. So you're still using judgment. In this case. Right. You know. It's hard to get away from judgment. When there's not a lot of information. Available. The last. Method I want to talk to you about. Is. Is breaking the rules a little bit. Because you're going to have to ask for. A very small amount. Of discovery time. Okay. Instead of saying. Well I can produce this. In three hours. You might ask for. Let's say three days. Or maybe a week. And the idea here. Is once again. Based on. An interesting statistical. Phenomena. Around uncertainty. Specifically. Before you do. Any kind of investigation. Into the work. That is. Associated with. You know. A particular work item. Let's say. You know. This. This is something that is. On the order of. About a quarter. Worth of work. Before you do. Any kind of investigation. At all. The theoretical. Uncertainty. We'll talk about what that means. In just a minute. But the theoretical uncertainty. Is at its. Absolute maximum. Potentially. An infinite level. Of uncertainty. Now it's very likely. That the. Practical uncertainty. Here. Is. It. Is. Much lower. Than the. Theoretical uncertainty. The theoretical uncertainty. Would say that. There's some amount of work here. That is. Impossible to do. With all of the time. You know. In the universe. Right. That's a theoretical uncertainty. It's extremely unlikely. We can probably rule that out. But even if. We're only talking about. A practical uncertainty. For example. This might be. A year. Of work. Right. It might be. An entire year. Of. Investment. We don't know. We don't know. We don't know. We don't know. We don't know. We don't know. We don't know. Because we've done. Virtually. No investigation. Into what this work is. But. You know. The maximum. We can imagine. Without doing. Any further investigations. A year. Working from that assumption. The. First bit. Of information. You get. Is the most. Valuable. Let me explain. What I mean. The smallest amount. Of investigation. Is likely. To yield. Large. Improvements. The largest. Improvements. In your. Refinement. Of your estimation. Think about it. If you have. A ton. Of uncertainty. Possibly. This is. This could take a year. With that level. Of uncertainty. You are likely. To learn. In the very. First pieces. Of information. About this work. That. It's going to take. Much less. Than a year. Perhaps. You can cut. That estimate. By up to. 50%. A small. Investigation. Might lead. To a major. Increase. In your confidence. In your first. Estimate. You could go. Very easily. From. A 10. Or 20%. Confidence. To a 50%. Or 60%. Confidence. In your estimate. Later refinements. Have. A much. Lower. ROI. In other words. You know. Trying to determine. Exactly. How long. Something will take. Down to the. Let's say. Day. Is. Extremely. Difficult. In fact. Most often. It is not worth. The time. The time. You're spending. Investigating. Or planning. Will exceed. The time. That it would take. To actually. Do the work. And so. It's an inverted. Kind of. Economic situation. This actually. Does happen. In a lot of workplaces. There's a lot of. Reporting. And status. Kind of. Communication. That happens. That exceeds. The cost. Of actually. Doing the work. You've probably. Experienced this. In your workplace. Before. But in the. Earliest phases. That is. Completely flipped. The other direction. And the smallest. Amount of investigation. Is going to yield. Major. Major improvements. So. Here's my recommendation. Okay. In order to. Provide. An estimate. On something. That is not well. Defined. Try to identify. The one or two areas. That could. Drastically. Change. Your estimate. Okay. These are. Kind of. Principal components. Of a longer. Estimate. So. The way you would do this. Is talk to the team. And say. Okay. What exactly. Could happen. That would make. The scope of this. Completely unattainable. Or. Would. Would absolutely. Explode. The scope. On this project. And what they're going to come back with. Is probably. Three or four things. Depending on how big. The project is. They may say. Well. If. As long as we don't have to. Worry about. External vendors. Or. As long as we don't have to. Worry about. Compliance. Or. If we can do this work. After. If we can sequence it. After we do this other work. Then it. Cut down. It cuts down a lot of the. Potential complexity. These are areas that. You can provide. As a part of your. Estimate. Right. When you're coming back. And you say. Okay. Here's some assumptions. We are making. These assumptions. This is really critical. These assumptions. Are buying you. Okay. You're. They're buying you. A lot of. Extra certainty. In your. Estimates. If you put that. Leader hat. Back on. And. You know. Imagine that they're. Purchasing something. They're purchasing a car. And your assumption is. Well if you. Choose a standard color. Right. If you choose it. A car with a standard color. Then it's going to be. Much more affordable. Than if we have to. You know. Wait for the car. To have a. Custom color. Or custom paint job. Applied. Overall. These. Three different approaches. Can. They can all be combined. Right. These three approaches. Can help you. Wrangle. Some of the. Less defined. Work. That may be sitting. In front of you. To estimate. It's very difficult. To estimate work. Already. But. Choosing some approach. That. Doesn't try to dig into. The specific details. Because you probably. Don't have time. To do that. But isn't. Just a wild guess. Without any kind of. You know. Mitigation. For the risks. Of that wild guess. And is also. A true. Estimate. A true. Estimate. A true. Estimate. Is. Some kind of range. A low. And a high range. As well. As a confidence interval. This is a guess. About the future. With a confidence interval. Taking one of these. Three approaches. Is likely. To decrease. The size. Of the range. And it's also. Likely. To improve. Your confidence. Intervals. Thank you so much. For listening. To today's episode. Of developer tea. I hope you enjoyed. This discussion. I hope it's useful. To you. If you're planning. For the next couple. Of months of work. Or if you're an engineer. On a team. And you're trying. To understand. How do we do all of this. In such a short amount. Of time. I encourage you. To share this. Share this episode. With whatever. Engineering leader. You think needs to hear it. This by no means. Is the full picture. Of course. We didn't even talk about. Things like dependencies. Or you know. Ongoing work. Multiple work streams. We didn't talk about. Saturating those work streams. So there's a lot. More complicated. Stuff. That we could get into. But. For the sake of. The simplest projects. These three. Techniques. Are going to be. A very useful approach. To improving. Your. Kind of blind. Estimates. Thanks so much for listening. Thank you again. To today's sponsor. Wix Studio. With Wix Studios. Developer first. Ecosystem. You can spend less time. On tedious tasks. And more time. On the functionality. That actually matters. To your customers. To your business. Develop. Online. In a VS code. Based IDE. Or you can develop. Locally. Using GitHub. You can extend. And replace. A suite of powerful. Business solutions. And ship faster. With Wix Studios. AI code assistant. All that. Is wrapped up. In an automatically. Maintained. Infra. For total. Peace of mind. Working. Developer first. Ecosystem. By heading over to. Wix. Studio. Dot com. Thanks again. To Wix Studio. For sponsoring. Today's episode. Of developer team. As we approach. The end of the year. We are likely to. Have a couple of weeks. Where we don't have. Episodes. I'm going to try. To put out. An episode. Next week. To talk a little bit. About goal setting. And. To discuss. The new year's resolutions. We talk about this. Every year. On the show. To talk a little bit. More in depth. About resolutions. And how to make them. Work better for you. So look out. For that episode. Next week. And until next time. Enjoy your tea.