What if every decision you made had a bet attached to it? You could lose something from it. And, of course, you can gain something from it. In today's episode, we talk about exactly this proposition and the idea that actually every decision you make is a bet. We just don't think about it that way. And the value of changing our thinking so that we do that more often. Today's guest is Annie Duke. Annie is a former professional poker player, but now she is a consultant and she is interested in helping people make better decisions. We talk about where this interest comes from and this idea that if you can shift your thinking away from black and white, decision-making and more towards probabilistic thinking, you'll probably have better outcomes. In the second part of this interview, we talk about practical ways to approach thinking in bets. And there's some ways that you can kind of provide heuristics to your brain, kind of shortcuts, or you can think about them as mental hacks to help you think more probabilistically. Now let's get straight into the interview with Annie Duke. Annie, welcome to the show. Well, thank you for having me. I am ecstatic to have you on the show. I've shared your talks and your book with quite a few people now. And I'm excited to chat with you about a whole list of topics. But first, I would love to ask you a question that maybe you aren't asked very often. I don't want to introduce you. I want you to introduce yourself. How do you want people to remember you? Oh my gosh. Like on my epitaph? Or, you know, just tomorrow. If you're talking to somebody about this episode, how do you want them to remember you? Specifically about this episode, because otherwise they're going to be like, I hope they think I was a good mother. That's great. Actually, that's great. Yeah, mainly just good mom. I would say that what I hope that people remember me as is someone who took concepts of evolution and evolution and evolution evolution and evolution evolution evolution that are pretty mathematical and complex and managed to communicate them in a useful way to people that simplified them and while allowing people to execute on the math that's at the base wasn't overwhelming them with those mathematical concepts. So in other words, let me say that again. I want people to think of me as someone who's taking very deeply mathematical concepts and putting them into words so that people can understand and execute on them. That's a really cool vision to have because so much of your career and experience has kind of required that of you to be able to understand these concepts and then actually use them in practice. It's one thing to have this theoretical background but it's another to be able to apply that. And I think developers actually face this problem. But I'd love for you to talk about ways that you see and you can probably hear my dog in the background. Ways that you see that playing out. And more specifically in your career, what are some times that you have had to take this theory and really see it through? Hmm. Well, I can kind of think about this on both sides. From when I had to be executing the ideas myself and also when I had to be communicating the ideas to other people. So let me start with the second thing first. I think a lot in terms of the way that I communicate about decision-making in general. At the same time, when I'm做ing the evolution of evolution, in general and how to think probabilistically and feedback elicitation and the kinds of things that I write about in the book, which is just how do you deal with the fact that when you get an outcome that it's hard to know what you're supposed to learn from it, right? I mean, this is kind of the base of what I write about in the book, that any particular outcome doesn't necessarily tell you very much about what the quality of your decision was, but we act like it does. And how do we kind of deal with that problem? So as I think about how did I sort of tackle that issue for myself, on the teaching side, I think back a lot to when I first started teaching, poker in particular. And I started doing these seminars. And obviously, there's a lot of math to poker. And poker is a deeply complex game in terms of the way that you're thinking through the probabilities and what the right choice is and how you're figuring out what other players have. And if you figure out what other players have, how you're sort of trying to figure out what the probability of success for different actions are, what the payoffs are. I mean, you can hear from just the way that I'm talking about it. It's complicated. Mm-hmm. And when I first started teaching, I... I think that I was trying to give the people that I was talking to all of that, right? Like... Yeah. I need to tell you what all the math is. I need to tell you how to do these equations. I need to tell you all the things that you should be thinking about, so on and so forth. And I kind of realized that while that might have been really good for me for showing off what I knew, in a sense, it wasn't good for them. Because... Because we learn stepwise. You know, it's like when you take math at school, they start you off, you know, with some simple stuff and they start to build and eventually, you know, you're doing calculus. You know, and obviously that's true with anything that you do for developers, right? It's not like you start off writing incredibly complex code. You start simple. And the thing that I sort of realized was that some of the biggest jumps... That people can make are in these... What I sort of think about is framework jumps. How are you thinking about the world? Then if somebody's curious beyond that to start really digging down into the math, there's lots and lots of ways for people to go find that stuff out. But getting to that first jump of how do you think about the world in a way that's useful for you solving this problem was much more important than some of this other stuff. And particularly... Much more important than me showing them what I happen to know. And in poker, one of the big jumps is that you need to make in terms of framework is don't think so much about what your hand is, but think about the way that the other person might be viewing you. Think about the way that the other person might be reacting to their own hand. So to sort of get outside of your own view and what you saw about your own hand and start thinking about yourself from the perspective of somebody else. Yeah. That's such a big change in the way that you sort of think about yourself. Yeah. Yeah. And I think about the world that that in itself was such a huge jump in the way that they played poker. So then as I think about how do you apply that to decision making, that's this really big jump that I'm trying to get people to make, which is when you make a decision, there is not a single outcome that can occur. There's many, many possible ways that the world can turn out. And so once you have an outcome, that's only one of many possibilities. And there's all sorts of ways in which as human beings, we look at that one possibility, and we think it's the only thing that could have happened. We think it's a really very, very strong signal for what the quality of the decision was. And in this way, it really leads us astray because we lose sight of the fact that there's so much uncertainty. There's such a big intervention of luck between when you make a decision and when you don't. Yeah. Yeah. Yeah. And whatever outcome that you get. And once we sort of lose sight of that, a lot of things can really kind of start to go astray. So obviously, there's other frameworks that I'm offering within Thinking and Bets. But as you know, that's the opening framework. Let's sort of think about this problem. And obviously, there's quite a bit of math to thinking about signal and noise and how large a sample size do you need given the variance. And so I think that's a really big thing. And I think that's a really big thing. And I think that's a really big thing. All of the intricate details of the math. But not all of those things will add the same increment of progress or the same increment of value for you. So shifting the way that you think could change your outcome, whatever the outcome is, whether you're playing poker or developing software or any other decision system that you're in. It could shift it by 100%. Right. But then another small thing that may take a little bit longer is that you're not going to be able to do the same thing. But then another small thing that may take a little bit longer is that you're not going to be able to do the same thing. But then another small thing that may take a similar amount of effort. That might be a very small refinement that changes it by 1%. And what's interesting to me about this, about what you're presenting here is that when you look at world-class athletes, for example, you're seeing the differences in their, let's say, their 40-yard sprint time. You're seeing those differences in very, very small numbers. Right. And so you know that those athletes are focusing on. Right. Those tiny, tiny details. But when you're in the earlier part of kind of your journey of expertise, when you're just starting out playing poker, you're just starting out sprinting, you're really looking for these bigger, almost heuristic style gains. Right. It's think this way, not that way. Rather than, you know, move your ankle, you know, slightly to the right. It's think about running in a totally different light. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. Right. it's those little tiny changes start to make really big differences because now you've closed the gap right so now you've got people who are all at a particular level where now small changes make a really big difference right that those are the deciders so i'm focused on the first part right like how do you teach somebody just basics of form right like yeah let's think about what the mechanics of running are in general right and if you if you think about the fact that the way that you swing your arms really makes a difference to the way that you run that's going to make a really big difference in how fast you run now am i going to make you an olympic sprinter well no like nate silver will make you an olympic sprinter but i'm going to get you ready to confront what he what he has to say yeah right so so that so that i can sort of pass you on having given you these really good building blocks of how do we think about the world how do we think about outcomes how do we think about the way that we're communicating with each other in such a way that either depending on how we do it we might be eliciting feedback that's really really distorted yeah or eliciting feedback that has really high fidelity right and that depends on on how are we thinking about how we communicate with the world what are the systems that we're setting in place for ourselves to make sure that that we're getting the best the best feedback how are we thinking about in general the way that we're uh scenario planning the way that we're forecasting the way that we're thinking about how things might turn out or how things might not turn out how are we thinking about our ability to really embrace and accept uncertainty and that understanding that not swatting uncertainty away but allowing it into your life is what allows you to be a better decision maker like these are kind of the broad questions that i'm trying that that i'm i'm hopefully trying to create like a better decision maker and i'm trying to create a shift in the way that people are viewing viewing the viewing the world and viewing viewing information in that way and that i mean it sounds like i feel a little bit you know like oh i'm biting off this really big thing and somehow i'm um i'm i'm doing this like but i mean it's my goal and and i know that what i'm doing is making it what i'm trying to do is make a dent in it right like if i if i can get a little bit of a shift in the way that somebody thinks i mean i don't i don't think that i can necessarily accomplish all of these big goals that i have but you know dream big right right and then if i if i can if someone someone thinks about you know an outcome slightly differently if someone says oh you know well just one time like maybe i'm resulting or well there are lots of different ways that things turned out but let me try to think about the likelihood of those things or how am i allowing myself to fall into an echo chamber or any of those things if i can get little changes in those i just i do feel like it has a really big impact on on people's lives i mean i hope so that's that's what my goal is absolutely i think you know for me probably the biggest shift for me was and i'm going to kind of walk this out slowly belief for me has always been kind of this mystical thing um a belief is something that you hold that no one should have access to right we don't try to influence each other's beliefs because they're almost sacred and and we almost have like this cultural uh appreciation for avoiding um discussion about belief and i think this this is a confusing thing because beliefs are kind of conflated with values and so the thing that really changed for me when i was reading through thinking bets and watching your talks and um you know was this shift this concept of changing your your definition of belief to something that is a little bit more about you know would you um would you put a bet on it or how much would you bet on it right and so it demystified this concept of belief and now i see belief as what i see to be true today's episode is sponsored by century century is a perfect sponsor for this episode because when you're thinking probably about the way your application code runs you can think differently about how to solve problems for example if i told you to make a bet about how good your test coverage is then hopefully you know that you can't make a perfect bet you can't get a hundred percent test coverage why is that well humans are not very good at writing tests we do okay at it but we're actually not going to be able to cover all of the problems that we're going to be able to cover in this episode all of the use cases because it's really hard to predict for example how people are going to interact with your application so what can we do about it how can we make sure that we are catching errors when they occur and hopefully we can proactively fix them before they impact our customer base well century provides you an avenue to do exactly that the very first moment any error shows up in your code century will alert you in pretty much any channel you can imagine choosing for example slack and it'll let you know where is the error occurring it'll give you a full stack trace it'll even give you a link back to the commit to the code that actually caused this error to happen and this will help you track down how you can fix it go to century.io to get started today thank you again to century for sponsoring today's episode that's century.io century.io century.io century.io century.io century.io which options we think we can choose in the sense that we can't choose all options at once. So we have to think about that. And then our beliefs are also going to determine what we think the possible outcomes are of the choice that we take. And also what the probability of each of those outcomes are. So our beliefs are sitting at the base. They're the foundation of every single decision that we make. And no matter what the type of belief is, you are essentially betting on the quality of that belief because it's driving those decisions. So if you think about it, for example, the fact that you get into an airplane, right? You have beliefs about airplanes and pilots and safety and everything. And you have beliefs about the quality of that belief. And so you have to in this particular case, physics, right? Because we assume that the plane will stay in the air and things like that. And you're willing to bet on your beliefs about airplanes by getting into an airplane. It's what causes you to not do certain things. Like your beliefs make it so that you do not jump off buildings, at least not without like a parachute or a parasail or something. You don't jump off a building naked. Because, we have beliefs about the world and about physics and gravity and things like that, that we feel that we would go splat. So those things are really obvious because those have to do with the physical world. But we also have beliefs, for example, about things that maybe we don't end up splat on the ground. But when we think about the decisions we make in politics, for example, we have beliefs about what particular policies are going to be used to cause outcomes that whatever our values are, and my values might be different than yours, are going to produce the best set of possible outcomes, balancing out what I want for myself versus what I want for society. And I want it to align with what my values are, right? So that's going to be whether I have beliefs about trade or about climate or about other economic issues outside of trade, about immigration. Right. And so I hold these belief systems. And when I go to cast my vote, right, I'm now betting on my beliefs. And we're doing this, for example, like in your world, in software development, you have beliefs about how long things will take, what the payoff for those things might be, how often you're going to have success or failure with whatever it is that you're cutting. And so I hold these belief systems. And when I go to cast my vote, I'm now betting on coding. You have beliefs about what the syntax is, because obviously you can have choices about that and what's going to be best for you. I hear, I don't know, I hear there's lots of debate about semicolons or whatever. I'm not a developer. So people have very strong beliefs about those things though, right? Those are all values, to be honest. Right. So people have beliefs about those. The littlest, tiny detail to the biggest detail of like, what are the possible outcomes of this particular, if I code in this particular way? What features do I think that people will like or not like? And literally your experience is informing all of these beliefs that you have and your beliefs are deriving the decisions that you make. And so given that, at the moment that you go to decide to like, I'm going to develop a particular feature, you're choosing all sorts of things, like what, what's going to be the best for me, what's going to be the worst for me, what's going to be the worst for me, what's going to be the worst for me, what's going to be the worst for me, what language are you developing? And like, how, what, how, how are you writing that code? How long is it going to take? How does that compare in terms of the resources that you're going to have to put into that versus some other feature that you might develop? My beliefs are driving how much I think that people will like that feature, for example. And so every single day in every single way, whether it's what are you ordering in a restaurant? Are you getting on a plane? What, you know, what feature are you choosing to develop over other features? How are you choosing to develop other features? Choosing in particular to code that. Every single decision you make is a bet that's driven by the beliefs that you have. And what that means is that we have to be much more vigilant about our beliefs than we generally are. And we can think about our beliefs kind of, we can broadly divide it and our beliefs into categories. There's stuff we know, and then there's stuff we don't know. Um, and we want to think about two things when we're thinking about beliefs, because we really want that foundation to be strong. The first thing is, how do we get more of the stuff we don't know into our own head? Yeah. Obviously that's incredibly helpful, right? Like I want the stuff I don't know to get into my head. So that has to do with how do you create an attitude toward the world of curiosity and open-mindedness? Because kind of to your point about saying like, you felt like your beliefs were sacred and siloed, that kind of attitude is actually going to reduce your ability to get stuff you don't know into your head because you're sort of unwilling to put things up for discussion, um, or to, to hear, uh, what other people say, particularly if it's in conflict with your own beliefs, right? So we want to think about how do I get stuff I don't know into my head? That's really important because that helps to fill in your knowledge gaps. But then there's another thing that's really important to do that we do actually much less than we do. And that's, that's really important. And that's really important. Yeah. Less of than the first thing, which is how do I make sure that I'm really doing good internal audits of my own beliefs? Because we hold all sorts of beliefs, kind of going back to that, like, what'd you believe when you were 20 that you don't believe anymore? Um, we hold all sorts of beliefs that are not completely correct. I mean, they, they're, they also very often aren't completely incorrect. I mean, sometimes they are, um, and that's sitting in the stuff I know category. Right. Cause I think I know it. Um, but you know, it's usually somewhere in between totally correct and totally incorrect. Um, and very often we are overconfident in the beliefs that we have, and it would be good if we pulled the confidence back. Uh, very often there's calibration that we can do around the beliefs or the belief is biased. Um, and we want to be sort of like having a lot of vigilance around the things that we know, or at least think we know that are in that, you know, stuff I know box. Um, so that we can start to, to clean that up and get that to be better as well. And by doing those two things, like being a really good information extractor, like how am I getting things that Jonathan knows that I don't know into my head. Right. And then also how am I doing these really good sort of cleanups around, around my own knowledge now, because that's all informing the decisions I make that, that at its, at its base is going to make me a better decision maker. Yeah. Yeah. And this is, this is such an important discussion for developers. I think, you know, we, we go through these, these kind of bike shag conversations as developers where we're trying to decide, is this the right technique? Is that the right technique? You know, is this framework the right one? Uh, this language is 20 times better and I'm going to write 17 blog posts to explain why. And the truth is that very often these things are not in absolutes. And I think this is another takeaway that really helped me kind of see things, uh, competing and more kind of circumstantial. And so you have a decision in a given context may be right, or maybe effective is probably a better word. Uh, whereas the decision in another given context may be effective. And so to write those in kind of vacuum, uh, uh, decision-making pods is really difficult to do. Um, you have this, this human element when you're, when you're working with software, you have a human element of this looks good to me and it may not look good to you. And so there's, it's not that it's purely subjective, but rather that, you know, it's, it, there's, there's not a 100% right and a 100% wrong. Um, most often, every once in a while, like you're saying, there's, there's that things we know, right. And we can have a hundred percent confidence or close to a hundred percent confidence. Right. So much of what we deal with is circumstantial. So much of what we deal with, uh, is, is in that middle ground, even when we're talking about things that, that are measurable. And so, you know, I, I think that we end up doing this really bad thing where we judge the behaviors of either ourselves or other people. Let's say you're a manager and you, you judge the behavior of your teammates, uh, or the people that you manage. And I think this can be really damaging. I love to talk with you about this, this idea of being wrong. Um, resulting and judging people based off of those results. And how can we draw that line when, Hey, you know what, it's my job as a manager to kind of judge the, uh, the performance of the people that I'm managing. So how can I do that while also recognizing that sometimes these results, you know, they aren't a factor of the quality of the decision at all. They have nothing to do with it. Yeah. So, so first of all, let me just say like, what you just said, I think is so important on, on kind of two levels. Uh, actually I'd say three levels. Uh, thing number one is that a decision that's good in circumstance A may not be a decision that's great in circumstance B, right? Like just, just because a particular thing you did, uh, was really good in, in one situation doesn't mean it's the right thing in another situation, but also across people that you, that's true, right? So what works for me may not work for you. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Mm-hmm. Just because it worked for me doesn't mean that I should say that you're doing it wrong because, right? That, that doesn't mean that it would necessarily work for you or that it would be the right decision for you. And, and likewise, so, so I learned, I, by the way, I learned that in poker all the time. Like I would watch other people play and I would see that they were executing particular tactics or strategies that were working really, really well for them that I recognized, well, it's important for me to understand that they're doing this and to sort of see what the value of what they're doing is. Mm-hmm. Because I think it's actually a good tactic or strategy, but it's not one that would work for me particularly. Mm-hmm. Mm-hmm. Because I couldn't fit it into sort of the sum total of the way that people viewed me at the table or I, I was lacking a particular skill that you would need in order to make that tactic work or whatever it might be. So you can recognize that it's not like a one-size-fits-all situation. The framework is, right? But not the actual, the actual way that you actually execute. And then the other thing that you touched on, which I think is really important, is that my values might be different. Mm-hmm. So the conclusion that, that the outcome that I'm trying to get to might be different than yours. And that's fine. Right? So, but I think that we're very quick to judge when people have different values than we do. Almost judge them as, as beliefs, right? Right. We do. And, and here's the thing, like if we both go into a restaurant, you know, you're, you might be trying to get something that's the tastiest and I might be trying to get something that's the healthiest. Yeah. And so we could order two totally separate things and both be completely right for us. Yeah. Right. And why would, am I supposed to look at you and judge you for what you're eating? That doesn't make any sense. Sure. Right. So, so I just, I just want to say that, but yeah, so, so I think here, here's, here's the problem is that when we start to judge people based on outcomes, there's some bad things that can come from that. So, so let me just define resulting for everybody. Um, so resulting is saying, if I know the quality of the outcome, that tells me everything I need to know about the quality of the decision. Now that's, that's true for some things you can, you can do that for some things. So for example, if I'm playing chess and I lose to you, um, and all you know, is that I lost to you. Um, and all you know is that I lost to you. We do actually know something about the quality of my decision-making in comparison to yours. It was worse. So in that particular case, resulting happens to get you to the right conclusion, which is just saying, tell me what the quality of the outcome was. And that then, then I can get to the quality of the decision. But for most things that we do in life, that's actually not true. Um, it's actually, it actually kind of leads us to a very bad conclusion. So, I mean, obviously you can think about poker. If I lose a hand of poker to you, that doesn't mean I played it poorly because, I could have had the best hand and just gotten unlucky because of the turn of a card or, and, you know, just something as simple as if I go through a green light, I can get in a car accident. So, uh, obviously it would be absurd to say, well, if I know, if I know that you got in a car accident that I know you, you drove poorly, because we understand that, that that's, that doesn't have a strong enough relationship between the decision and the outcome, um, in order to be able to get there. And even, even if you're thinking about something like, you know, code breaking, um, it doesn't necessarily mean that the decision-making that led to that was poor because there are unknowns, right? Sometimes one of the things that we need to remember is that the thing that we can't know when we're in the process of making a decision, there's one thing that we can never know, which is what, how it's going to turn out. That that's information that only reveals itself after the fact. So if you're in a situation where, where you're coding and there's different choices that you can make, um, about how, how you might want to code that. Sometimes you can't find out that the code will break, that it will break until it actually breaks. So, which, which sounds simple, but people don't act that way. Absolutely. Right. So now when it breaks, they're like, ah, you, you were stupid. You made a mistake. you know, whatever. No, what a horrible developer you are. Right. Right. Exactly. When it's like, no, there were a variety of different choices that I could make. And this seems like the hot seemed like the highest probability for things to work out well. But then there was something that I couldn't be for, you know, that I, that I could only figure out was a problem until after the fact. So what I'm trying to do, so this isn't something as simple as that, that seems so straightforward, right? That at the time that you're, at the time, that you're actually writing the code, there's, there's different choices that you can make, right? There's different branches that you can sort of branch off on and take. There's different ways that you can write it. And what you're doing is trying to choose what the highest probability of success is for things to actually come out. Well, and sometimes it breaks and you can't know that until after that's happened. And then that will sometimes reveal where, where the stress point was. Right. Yeah. You know, what are you, what are you doing to people when you're, when you're resulting on them in these cases, you're not giving them freedom to sort of feel like they can try or they can take risks or they could do new things. So if we want to think about like, how could we write the most, the most elegant or efficient code, right? Well, you're going to have to break stuff along the way in order to figure out like, how can I really streamline this or get this as elegant as possible? You're going to have to break stuff along the way. So you have to give people freedom to be able, to sort of experiment along in there because that's the way that you find new paths. And that's the way that you find like efficiencies that you couldn't find before or ways that things are more elegant than you could have otherwise seen or things where you can speed things up, you know, or places where you can slow things down. A huge thank you to Annie for joining me on developer tea. This is part one, the end of part one. If you don't want to miss out on part two of this interview, then I encourage you to subscribe and whatever podcasting, app you use before the end of the episode, a huge thank you to today's sponsor century. You can find errors in your code before your users leave your application by setting up century head over to century.io to get started today. Pretty much every language is supported by the way. So go and check it out. Century.io. If you found this episode or other episodes of developer tea valuable to your career or to your personal life, or just in general, if you like what we do, the best way that you can help developer T out and help us continue doing this is to leave a review in iTunes. The reason this helps is number one. It helps other developers just like you find the show and decide they want to listen to it. The second reason is it helps iTunes know that there's people out there who like this show. So go and leave a review and iTunes. The best way to do this is just look in the show notes. We've got a link to do exactly that. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Exactly. That today's episode in every episode of developer T can be found on spec. Dot. FM. And there's a super cool feature. We've talked about it before where you can search for different topics across every episode on the spec network. This is not just developer T. It's also shows like design details and react podcast and framework and orthogonal and does not compute. These are all shows that you can find on the spec network. with excellent content that's waiting for you to listen to it. Head over to spec.fm to check that out. And of course, today's episode wouldn't be possible without our awesome producer, Sarah Jackson. Thank you so much for listening to today's episode. And until next time, enjoy your tea.