What was the last good story that you heard, or maybe read it or watched it on a TV show, or maybe a friend told you a story? Perhaps you've told yourself a story. In fact, we tell ourselves stories all the time, and in today's episode, we're talking about how that can be a problem. My name is Jonathan Cutrell, and you're listening to Developer Tea, my goal on this show, is to help driven developers become better at what they do, so they can have a positive impact on the people around them, but also to help them connect to what they care about, connect to their career purpose, connect to the things that they want out of their career, so they can understand how to better move towards that. I say they, but really I'm talking about you. I'm talking about the people who listen to this show. Thank you so much for joining me. Maybe you don't feel like a developer. This is probably one of those narratives that we're talking about. Maybe you don't feel like a developer. You don't feel like you belong in that group of people. Perhaps the narrative that you tell yourself is that you don't belong because you're not formally trained, and so you feel like an imposter. We talked about imposter syndrome before on the show. Certainly not a new topic. Another reason you may not feel like a developer is because you don't fit the mold of a developer. Whatever that mold is in your mind, you don't fit it. Perhaps you're not as interested in some things that other developers you know are interested in, whether that's culturally or even in your job. While we won't go so far as to say that everyone is a developer, anyone that wants to be a developer is a developer. That's not necessarily true, and it's not really even that useful, not really helpful to anyone to say that. What I will say is that it's very possible that as long as you are developing something, that is really the core definition of being a developer. But what are the other narratives that we tell ourselves that can really kind of steer us wrong? Maybe you don't have fear of missing out, but certainly this bias, this narrative bias, is what it's called. It's actually a real thing. You can go and Google it, narrative bias. This is how we see the world. We believe the stories that we use to make sense out of things. So for example, if you were to tell somebody about your childhood home, perhaps you would tell them about features of your home, but maybe in more salient way, in more human connection way, it's to tell them stories. Stories about the home include features of the home as you go through those stories. This helps us understand things. It helps us create some kind of rhythm, some kind of sequence, and it allows us to envision either ourselves or as if we're watching a video, it allows us to see things play out. So why is this so powerful? Beyond the intuitive answer that allows us to see things play out, narrative bias is powerful because our brains are trying to make sense all the time. There's really two major goals that the brain has. Number one, make sense of things so that we can stay alive. And number two, be as lazy as possible. So if a story makes sense, the brain will accept it. Our brains are lazy, but they're also since making it's kind of this weird paradox where they really are trying to wrap around a particular event and trying to make sense of it so that we can predict if that's going to happen again in the future. We can avoid the bad parts and maybe seek after the good parts. So our brain is trying to, in some ways, predict the future. That's what we call learning. But the downside is that our since making brains, our amazing since making brains, aren't really all that accurate. And because they're lazy, anything that is cohesive becomes authoritative. So what this really plays out as is stories become more engaging and perhaps more persuasive than pure raw data. Asking someone to write their own story out of information, oh, the brain is lazy. We've already mentioned this. And so instead, if you tell them a story, if you give a story, then that story's probably going to be adopted faster than it is rewritten. So why are we talking about stories? Why are we talking about narratives on a developer podcast? Well, as it turns out, we tell ourselves stories all the time. We're trying to make sense of events that we didn't necessarily see happening in real time. We weren't really recording those events. We weren't trying to remember them. It's not like we were actually watching a movie. Instead, it's kind of like we're trying to make a movie out of what happened. This happened recently on our development team. We realized that all of a sudden, a lot of our records have been deleted. And this was out of a production database. Of course, we had backups. So it wasn't like it was a big problem. But if somehow we had a data breach, of course, we need to figure out why. And hopefully, engineers who are listening to this, you have the same first inclination that somehow there was a hack. Somebody compromised your database. Someone has figured out how to inject SQL that dropped all your tables. Maybe they figured out a cross site scripting hack or some kind of Ruby on Rails vulnerability that hasn't been discovered yet. And they issued a command to figure out what tables you have and then delete them. And this seems reasonable, doesn't it? It seems that a production database wouldn't suddenly empty itself out. That seems like kind of the alternative story. And it's easy to reject that story. That Postgres is that unreliable. That it would suddenly empty itself out. So what is the other option on the table? The most common story, the story we hear the most often, and the one that makes sense, at least intuitively, is that someone hacked the site. We had some kind of data breach. And once we told ourselves this story, we started looking for the vulnerability. We looked through our logs, we looked through reporting systems. We double checked that we had things up to date. We noticed that we had a few gems that were out of date. So we kind of walked down the path of figuring out maybe one of these gems has a vulnerability. What happened next? I will never forget my whole career. I started asking myself, what event actually occurred here? The logs didn't show any kind of cross site scripting, any kind of major hacking attack where somebody's hammering the server with a bunch of passwords to try to break into an account. None of that had happened. All we had was a few logs from our database, but our application code wasn't logging. Now, you don't need to understand Rails to understand what was happening here. What that led us to believe is that somehow someone had access to our database directly. Some seasoned engineers and perhaps some really clever, younger engineers that are listening to this show, you might actually have an inclination of what happened here. But for us, we thought, wow, somebody must have gotten access to our credentials, our database credentials. So we were telling ourselves new stories that made sense. It still felt like a hack to us. It still seems that that initial story that we accepted was overriding our sensibility of what could have happened. Finally, we started asking the question, what else could have happened? What kind of database vulnerability could happen on a very secure platform we're using Haroku, a platform that cares quite a bit about security. And the way that Haroku manages database credentials, really, we couldn't have easily given those away. And so we started trying to decide what else could have been true. What other story could we tell ourselves? I started thinking of ways that I might go about emptying out a database. Of course, I could probably write some SQL queries to list out the tables and hold those in memory and then try to drop all those tables. I could write some other SQL to drop all of the records. But then it dawned on me that we emptied databases out all the time when we run our tests. Specifically, there's a gem called database cleaner that will delete all of the records in the database for you between tests if you want it to. This allows you to start with a fresh database. But database cleaner is only available in the test environment. It's only available locally. Finally, almost accidentally, we checked our environment variables. It turns out that we had set our environment variables to point to the remote database and to always be in production mode. What this means is we were running our tests against the production database. Hopefully, anyone who's listening to this podcast knows, but not only was this incredibly destructive using database cleaner on the production database did indeed delete all of the records in the database, but it also seems pretty obvious. It also seems pretty clear that, of course, your environment variables are important and you should have checked those. Now, with hindsight, we're able to look back and say, well, yeah, of course, we should have checked those. The story that we told ourselves in the beginning overrode that reasonability and overrode our ability to see things clearly. So I encourage you to understand that there's going to be a moment where even the seemingly obvious bugs that you're facing, even when you are a senior engineer, when you've seen all kinds of complex bugs, you will almost certainly encounter simple bugs, simple problems, simple scenarios where your story prevents you from seeing reality. I encourage you to force your brain out of this lazy mode of accepting stories and instead ask yourself simple questions like, what else could be true? What am I assuming about this situation? What can I do to break those assumptions? What has to be true for those assumptions to hold true? These are all very important questions to ask yourself, not only when you're making big decisions, but also in the smaller things, in the things where you feel like the story is authoritative. I encourage you to always question the narratives that you believe. Please ask how valid is this belief? What am I basing this belief on? Can I break it down into smaller pieces? Can I break it down into smaller variables? Can I trace it down to its fundamental parts? Once you start thinking this way, then a lot of these biases, at least you are competing against them, the bias of narrative, but also other biases that play in amplify the narrative bias. Negativity bias. In the story of the test suite running against the production database, the negativity bias caused us to believe the worst thing had happened. That somebody had attacked us, that we had lost all of our data. If we didn't have the negativity bias, perhaps we would have considered other avenues earlier. Thank you so much for listening to today's episode of Developer Tea. I hope that this has been challenging. I hope it's been inspiring. And I hope it's been interesting. Hopefully you are thinking about the ways that your brain works, that you just have to figure out how to deal with as a human. Encourage you to go ahead and subscribe in whatever podcasting app you are using right now if you don't want to miss out on future episodes, just like this one. Thank you so much for listening. And until next time, enjoy your tea.