Dealing with exceptions is kind of this like core core leadership skill and particularly dealing with exceptions as you design Process and figuring out how do you find it sort of like architecting software? How do you find the common patterns then how do you make them extensible to kind of handle like what reality brings you? It is is a really powerful topic to think about particularly when you think you how do you also make this a fair And kind of just system not just one that kind of Gives the results that people wanted to have all the time You just heard the voice of Will Larson. This is part two of my interview with Will Will is an engineering manager at Stripe and he recently published a book called an elegant puzzle This is for engineering managers But I recommend that engineers of all levels take a look at this book There's lots of practical advice and tons of resources as we discussed in the previous episode My name is Jonathan Cutrell. You're listening to Developer Tea and my goal on this show is to help driven developers like you Find clarity, perspective and purpose in your careers and I think Will is going to help you do some of that in today's episode So let's get straight into the interview with Will Larson And you mentioned a just system. This is a concept that I believe you talk about in your book And it's something that I wanted to to bring up now because I think this is something that Solves a lot of the questions And I guess kind of conundrums that you might find like for example in an engineering manager interview You know, how do you Establish a culture where the people feel the people that are directly reporting to you They feel like you're taking care of them like you care about them as as people but also that you're demanding The the quality level that is necessary for success and I think that's that's a really hard Thing for people to kind of wrap their minds around and I think that this this Terminology of a just environment is exactly That kind of the road to an answer to that. Can you describe what a just environment would look like? So when I think about kind of Justice in process, which is which is kind of a interesting idea I really think we want to make sure that people know how to accomplish things the right way That the company has said like the way we want you to get up level is this the way that we want you to be exposed to New roles and to apply to those new roles is this the way that we want you to kind of have your compensation updated is this We really want to make sure that people who do what we're asking them to Are rewarded at least as well as people who don't do what we're asking them to For example, if we're saying that there is a kind of a certain process to get your compensation Adjusted each year say there's some sort of automatic Compensation adjustment system that happens to kind of bring you up to your current kind of whatever like your title is and the kind of Pay me in there plus like an increase of like cost of living or something We want to make sure that someone who is Just following that process gets the same exact treatment as someone who's been advocating for themselves kind of every Every couple of months to get like a raise or whatnot And but also we want to make sure that we're not putting some people in the position where they're just they got hired with a lot less money So that they have to advocate for themselves consistently to try to get like a consistent result But really we want to make sure that no matter how people interact with the process they get like Approximately the same results and to me that's justice where we don't incentivize people to Defect from the process. We don't incentivize people where they think kind of following the processes for suckers or whatnot We actually want the best results to be for people who do what we ask them to do as we design these systems that they then live in Yeah, I think that's important because it it means that both sides have responsibility You as the company the manager You have a responsibility to know what you're looking for and that's that's hard, but it's also useful and And if you can articulate what you're looking for and somebody actually brings you what you're looking for rather than creating this kind of mysterious environment where The picture in my mind is somebody kind of saying well, you know just in process That's that's a situation where Where it's very stressful to be a developer, but it's also unlikely to succeed right because the way that somebody might perform in that environment is erratic and It certainly doesn't match everyone's personality to go out of their way and find a way to quote and press their boss That's that's a bad a bad situation for everyone One of the kind of like the the open but slightly hidden secrets behind this idea of like consistency and fairness And company is in process is that People really know how to run fair processes. It's just hard. It just takes a lot of time. So consider it two different examples If you have like a really important project that you need someone to run What you could do is you could kind of send out an email to the company ask people to apply over the next week You could make a clear rubric of how you'll select from the people that apply you could go nudge people to kind of encourage them to apply People who might not think they're ready, but you think already and then you could select someone you could Give feedback to everyone not selected explaining why they weren't selected and then you could send an announcement saying who was selected to lead it You could do all of that or you could just pick someone that that you think would do a great job and like move on to the next problem And so one of the challenges is that like people want justice, but they don't necessarily want to like invest the time into doing it and also There are ways to make this cheaper though, and so one of the things I've been thinking about is a lot of the processes that I've designed and seen designed are kind of just and fair, but but they're they're really heavy and so a lot of the work I've been doing in kind of management over the last years trying to understand How do we find these systems that have the same properties of kind of fairness and Consistency, but are also a lot less overhead to implement in that that is when I think we get to this place We're going to actually have kind of everything we want where I think often when people aren't following these kind of more structured processes It's just like a efficiency like we need to get this done Not not kind of a absence of desire to have consistency for the folks they work with and can you share You know kind of what what is an example of Making a more efficient and fair process is there something that you have in mind that That like like a process that you would actually refactor to be more efficient, but also still just So going back to this example like the selecting kind of leads for projects like something we've done is we've moved towards pre materializing the the selections Where one of the challenges is that we'd have like a project like an emergency that comes up We need someone to lead it, but we don't have time to steal like a full process to like take a week and find someone We need someone like tomorrow or today And so something we've done is like how do we like pre-run these selection processes? We're gonna actually find the next person probably a queue right? Yeah, but exactly right How can we do the work ahead of time instead of doing it in the moment? Yeah, that makes total sense and it's it's not so much about you know doing less It's about finding the ways to Kind of have on time delivery in that particular scenario, but I'm sure there are ways that we can We can optimize for doing a little bit less like Taking advantage of people's opt-in behaviors and Allowing people to kind of be on tracks for example is what I was thinking about is a way to say okay You know if you're interested in this particular kind of growth then this you know This is the stream that you want to be in perhaps you're not going to have something in this area immediately But when something comes along you're already in the stream right you're already kind of a part of the group that will You know be opted in to this rather than asking everyone at that moment or every moment something comes along I think that's exactly right I think this kind of like pre-materializing this putting people in stream or you know exactly what they're looking for Or say kind of like set them up for that. I think that's that's definitely kind of the right way to think about it. It's I think you can actually to your point about like being Like not necessarily easier or less work I think there are ways you can make it less work where if you actually batch these things You can actually do a process and you can get like the next 15 people to work on these sorts of projects Or you can do it all kind of the fixed overhead or running the process you just do that once But but I think you're definitely right that there there are kind of There's still a lot of kind of work there Um, and sort of a variation on this idea though is that there are also like types of projects that are like critical for the business that are not necessarily Glamorous or exciting and like a lot of like compliance overhead like Teams that worked on GDPR last year were not necessarily like Desperate to work on kind of this this project and I think that's where really the company culture can be so important It about kind of focusing on business impact as like core to what makes work desirable and rewarding work and avoiding this idea of like Complexity or interestingness as kind of what the company Recognizes and values. Yeah, that's very hard to do right because it can be It can be a slog to work through some of these things and I think you know that brings me to another point that I think is important to to note and that is it you know all of these systems all of these models They are all exactly that they're their models which means you know, they are useful But they're also going to miss something along the way right we're going to have You know a moment where we hope to have All of these systems in place But because of some overlap or some particular scenario that we couldn't have predicted or somebody's You know particular tastes even The system doesn't necessarily cover everything and so I wonder you know When you come up to these kind of messy situations where yes, you you have this kind of idealistic way of dealing with problems But you know when you encounter these messy situations what skill sets do you think are important to fall back on When the model just isn't quite doing You know doing the work that you need it to do and instead you have to kind of do this manual Kind of human work right it's almost HR work in a way What are the what are those skills that you think are important for engineering managers to cultivate? But when the model isn't working. I think there there's kind of two different things to do There's kind of diagnosing the current situation and there's like figuring out how to fix the model But I think you know that when something's going really wrong The the first thing I do is I try to talk individually with everyone involved and Then I try to pull the group together and actually have that like kind of come to truth team meeting Where everyone has to like really kind of get their perspectives out and I found as I've gotten better at running those meetings I found them extremely effective at getting kind of all the problems out on the table Early on I think my comfort level with Discomfort was not high enough for it, but increasingly I think that's a gotten really powerful so you know first Listening asking seconds to synthesizing And then really to me is like trying to understand the data going into the model and coming out It did in some cases like some human cases It's interesting like what is the data mean? There's like just a few people involved but really trying to understand Kind of the the principles that are driving conflict for example they have two teams are really You know quote unquote misaligned and disagree about some problem It's almost always been the case for me that it's it's actually like there's like some sort of structural misalignment Let us set them up for that not that there's some sort of like people who are Fighting with each other or something like that. There's actually something real misaligned that that it really pays to try to understand the forces that are motivating the people to act this way versus just projecting that the people are kind of bad actors or something like that It's almost never the case that people are the bad actors. It's almost always some structural thing driving them into weird behavior Yeah, I agree with that completely it's a Is it Hanlands razor or I believe it's yeah, it's Hanlands razor not awkums, but Hanlands razor essentially says that We should never attribute to malice what What can be explained with pretty much any other explanation? and and most of what humans are reacting to is some environmental factor Typically it's something that they don't necessarily understand or that they're trying to understand and so I think I think what you're saying is is absolutely Critical but it's also something that's hard to I guess see until you fixed one of those situations until you've seen those principles at work and then you said okay You know, what is it exactly is this team Aligned in a way, you know on this particular project that they are responsible for too much for example Maybe the the responsibilities are misaligned or maybe the expectations on one team are higher than the expectations on another those kinds of misalignments can be can drive Down the line that is you know one one two three four steps removed from the original problem to begin with There's a whole kind of genre of kind of like the wrong goals led to the wrong kind of behavior Um, and I think this your your kind of comments fall directly into that I actually haven't seen too many cases in my career where there's like a story where someone had the wrong goal And they just did this totally wrong thing. I've definitely Solved the right problem in the wrong way where I sort of like regretted the approach longer term. I think At Uber we kind of spun up the service provisioning in a system in a way where people could just self-service And we ended up having essentially one service per engineer at one point And I think that was an unintended consequence of that approach But it was still the right problem and the right goal and just the unexpected consequences down down the road from it. I think that's I think what you're saying is is Critical to the kind of the systems thinking mindset, right? The hope is that with some systems thinking and we won't go into too much detail here because the book That you mentioned is is really a better way to learn about it But the hope would be that you can identify You know once you've gone through the process of kind of modeling out a system of various interactions You can identify unexpected interactions unexpected result from you know a particular node and that it's kind of a graph Problem typically when you look at a system a particular node in that sending a signal to another node You may not you know intuitively understand the relationship between two disparate parts of Of that graph until you model it out and what you're saying makes a lot of sense that you can look at these second and third order effects and If you try to solve the problem in one way those first order effects may look right Right, you may have Part of the solution or you may have some of the effects that you want but these other unintended consequences Certainly can happen too So one of the interesting things about systems thinking is that oftentimes when you're thinking about solving a problem It's like here's the goal But systems thinking doesn't let you specify the goal It just lets you specify the change and then see the result And so this is forces you to have like a really clear hypothesis about what you want to do And instead of just thinking about the goal it actually you have to change these levers trying to accomplish the goal And I think that's really powerful. It's not just we will you know deliver five new products next year It's like how many engineers do I need to hire and when to actually deliver five new products next year? And I think that kind of thoughtfulness By not letting you just specify the outcome but actually specifying the action you will take though create the outcome Forces just like clear thinking than than any other approach I've really seen Yeah, and there's there are tools to kind of model these things out You know to to visualize What does that system look like when you when you put all the pieces together you put all the interaction? You know the interacting nodes or whatever you want to call it Where you say okay, you know we want to deliver five things Well, let's let's put a node on here that's Engineers and then let's put You know unexpected absences right and so you would have those two things that feed into the system and you may have you know 10 different things that you think are going to contribute or detract from you achieving that particular Output at that point And I think you know, it's it's really interesting to watch as you know, especially if you have some kind of automated way of running these things It's interesting to see how inner you know an input in one particular area may have a feedback Input to another area right and the feedback loop is the thing that makes systems thinking so interesting Yeah, it's that that's exactly right. I think We often go into these things thinking we understand like how they work thinking we understand where the core constraint is But but so often by actually taking real data of how things have worked and trying to model that We learn just like new things for example One of things I've learned is that like typically our hiring is constrained on engine manager hiring not on our ability to hire kind of engineers And so that has caused me to really rethink where my time goes when when we hire and if I hadn't done that modeling and Then been wrong about it. I would have never kind of realized that was the right place for us like put the majority of our time Yeah, that makes total sense and I do think there are a lot of these kind of unintuitive things about you know the environment that we interact with we think that if we you know directly work on hiring Engineers in that particular example that okay, of course, you know, that's going to have the right return But maybe there's a bigger lever and I think that's kind of the The the the key question is what is the right lever for me to pull to get the things that I want Today's episode is sponsored by Git Prime We've been talking about engineering management in today's episode and the previous episode of Developer Tea and You probably have noticed that the best engineering managers are excellent debuggers They can see problems in code, but they can also see problems in teams in a recent episode We talked about the idea of pattern matching and that is exactly what these great engineering managers are doing They're looking for patterns and what kind of patterns are they looking for? Well, that's what today's sponsor Git Prime helps you figure out Git Prime has published a book outlining 20 patterns for effective engineering teams You can get the book for free by going to Git Prime.com slash 20 patterns. That's Git Prime GIT pr im e dot com slash the numbers two zero patterns 20 patterns Thanks again to Git Prime for sponsoring today's episode of Developer TeaI'd love to ask you a question Kind of switching gears here. What are some of the more Unintuitive things I guess it's similar in some ways But more unintuitive things that you may have learned since Becoming an engineering manager that you think other engineering managers Maybe they also have the the same kind of misconception that you had But now you see things a little bit differently or are there some some things that you wouldn't have expected to be true But you found them to be true Sure, this is a pretty large category one thing that I thought was not true at all Which I now believe is true is that I used to think that there were kind of If I just had authority like if I just had authority over this I could accomplish anything But but really what I found is authority is Pretty difficult tool to wield at scale So if you think about making a top down decision around some sort of oral change or a top down decision around kind of changing the process to do something It it's so hard to do that the right way You need to figure out the stakeholders you to figure out the impacted parties You need to go talk to all of them Build alignment get them engaged roll it out properly Let everyone know through like structure communication um And that's kind of it's actually very hard to wield authority Properly because it requires so much restraint And I've really come to believe that if you to wield authority properly you actually have very few kind of appropriate moves you're able to take in any given point like your flexibility is just like greatly constrained By all the constraints Conversely like I used to think you know operating without authority was very difficult Um, but what I really found is that there's kind of a freedom of operating without authority Which you don't have to follow all the steps Now I'm so a lot of these processes that I've experimented with about kind of how we do Um kind of Interviewing people internally for roles that we're considering project selection Um cold sourcing these are things we just ran as experiments internally to the team And there was like none of the overhead of kind of doing it properly because it's just an experiment locally to the team We're not asking anyone to like buy into it We're not asking people to like do it our way and I really think That to me has been kind of eye-opening where in a lot of cases The authority actually really slows you down because you have to be so thoughtful with it And not having authorities I think sometimes liberating to actually move much much faster and that's something I would have never imagined was true until fairly recently That is a very interesting kind of dichotomy of Freedom versus kind of I guess responsibility is is that the The constrained piece of that and we see this in organizations large organizations tend To make small Teams innovation teams is what they're often called or innovation department That are less responsible for kind of the the growth of the company or You know the success of the for the stockholders and sometimes they even make entire companies That operate inside of the company just to Can shelter them from that higher level authority that they otherwise might have And so I think this works both at the individual level and it works at the kind of the organizational level And you can watch this happening in very young startups young startups have the ability to pivot their entire Business plan right their entire strategy and Though that may not be easy on the kind of the emotional state of the people at that startup financially it may be totally possible to do that and You know, it's kind of the big ship turning um, you know if you're a light and agile You know lowercase a agile Then it's much easier to To make adjustments and to continue experimenting because the stakes are low There's kind of an interesting juxtaposition here of kind of the innovators dilemma which is One of the reason you have innovation teams in large companies is that the company business gets so valuable as you scale That it's irrational to do any investment outside of supporting that core business And it makes it very hard to do new things um conversely um what I really don't like innovation teams because typically they create the social structure Where like why wouldn't you want to be doing the cool new thing with no production responsibilities When you if you have that option like you should just do that that seems like a lot easier And so like there's this interesting kind of friction between This the the business need and kind of the how do you design a process where you don't Reward people for kind of bumping off and not doing the critical production work The most essential work to like run the current business And then the business if you only do that you actually starve the business of like the the future By kind of over focusing on today and figuring out how to balance those sorts of things I think it's is really some of the most interesting parts of a kind of management and kind of Just thinking in this domain overall Yeah, it makes total sense. I think it makes sense to Have your department your engineering department keep a pulse on whatever's keeping the business alive um and you know artificially shielding a group of individuals from that is can be can can so kind of this bad discord Amongst the the people who aren't in that team versus the people who are it's like the you know what made what made you guys so special right Um, then that's that's probably not a healthy culture But it also couldn't could mean that you know the operational Mindset is entirely different. So moving whatever that innovative thing is into the mainline business Maybe a huge task if it wasn't Kind of integrated to begin with That that that is definitely one of the kind of the the most painful anti-paradins Which is the innovation team that doesn't hand off to to the maintenance team to kind of Run the new thing and then moves on to their their new kind of newest shiny thing Yeah, I think you know If you look at strike we've we've started to launch more and more different product lines Oh as we get older and I think one of the things that we're trying to do is we're trying to later the the products on top of each other Not kind of strictly compete and not strictly Um kind of extends the existing functionality, but actually build things that are built on top of our existing platforms So I do think there's ways that you can be kind of thoughtful about the organizational In business design to kind of avoid some of these kind of downsides We're also allowing you to kind of innovate move quickly and kind of not get kind of stagnant in how your company grows Yeah, that makes sense I'd love to switch gears for a moment here again and ask you kind of from a high level as as People are as engineers are growing in their careers They're going to face the moment where they have to kind of make a decision. Are they going to continue Being an engineer for the rest of their career or are they going to kind of um Develop these management skills and I was talking with Matt Klein recently um who created envoys at lift as a long career as an IC and he wants to stay and individual contributor Um, so I'd love to know in a what advice do you have for someone who is facing this conundrum of you know Do I do I go into this uh different kind of path into management or do I continue being an engineer? Do you have advice for people who are who are facing that decision? So one of the kind of perverse things of Silicon Valley kind of engineering in particular But as I think also common across kind of engineering in 2019 at large is I think we're we tend to like have this like default assumption that we're going to be in the industry for like 10 years or 15 years That it's kind of this really short term thing so for example Your your five six seven years into your career as a manager or as an engineer Let's say you're a senior engineer at a company and someone comes to you and's like hey Do you want to do you want a driving a manager? Um, I think people are often like well if I try being a manager You know, it might not go well and it's going to take me six months. No, it's not going well And then it's going to take me three months to read ramp And I'm going to lose nine months on my kind of career track to getting promoted to staff engineer or something like that And I totally understand like why kind of people get into that mentality But the reality is like if you're going to be in this industry for 30 35 40 years Losing nine months to try out a new role isn't isn't that big of a deal like you're you're still getting paid You're just like haven't gotten promoted at the same pace you were expecting to So my my first thought is that I think there's a sense that experimentation is far more expensive than then we think it is And you know, this isn't true for everyone and so if you are Supporting several generations of your family off your income you you have a lot less risk tolerance If you have younger kids and you're supporting them you have a lot less risk tolerance And so it's not this is not universal advice like everyone has to understand kind of their risk profile But in general, I found that people Really ought to try management if they're interested in it They don't like it you pick up a different perspective And I think it's really powerful perspective And then you can just bounce back off if there's almost no permanent decisions you can make I think people frame things permanently sometimes, but but I think far more people Go into management and then kind of step out of it after a couple of years And I think people consider I think that's it's pretty normal to kind of go through that path In those folks, I think in the moment they feel like they've lost something But so much of being an effective staff engineer is the same sorts of skills that you have to build to be an effective manager So I really think there's an accelerant here. It's kind of like slowing down to speed up if you want to go back to the icy path But I really I really don't think there's harm and I think there's a lot of upside if you give it a try Yeah, I totally agree with this idea that We have this kind of mental picture of our careers that You know the next six months are incredibly important to the long-term reality and that's not just to discount the importance of what may happen in six months But rather to say that You know the amount of change that you're going to go through your career Or throughout your career is it's going to continue to grow and It's that you're going to continue facing these new situations um, you know, one of the things that I said early on in the life of this podcast is as Developers are trying to especially very young Engineers if you're trying to decide for example what language should you learn and your head is spinning because there's so many new things They're coming out left and right and you're trying to decide. Okay, you know What should I do and you start learning one language and then you hop up? We hop over to another one and try to learn a language a week almost and the truth is you're probably better off if you just Win ahead and committed to one language for six months And a lot of people who are listening to this right now they already have a little bit of anxiety thinking about committing to To a language for six months, right? But the truth is you know If you if you look at the grand scheme of your career um, very quickly that six months that you invested To develop a much deeper understanding than you would if you were hopping around to every other language and just learning hello world That six months is going to return very quickly and you're much less likely to to fall behind Then you may immediately think you are and I think the same principle applies Uh to and to trying out a position as a manager So much career advice is it's kind of just uh here's this incantation that I use to get into this cool role if you to pursue this incantation You will also get into cool role and that could be you know learning a language every week for ten weeks that could be You know writing an engineering blog that could be being a conference speaker that could be like almost almost anything There there are more ways to kind of be successful in your career than I think any one person can can list There's just so many different paths to follow that can kind of get you there and I think It's easy. I've certainly been very anxious at times in my career about how do I get to the next thing the right thing? um, but But there's so many different paths to follow that will get you there that that honestly I think just As you said focusing I'm doing some some good work building some depth Do that over and over and you'll you'll get someone really really really good Thank you so much to will for joining me on today's episode and the last episode and the next episode of Developer Tea This is a three-part interview with Will Larson And of course, thank you to today's sponsor get prime head over to get prime calm slash Two zero patterns that's get prime calm slash 20 patterns To read the free book about the 20 patterns you should be looking for on your engineering team Thank you so much for listening to today's episode If you're listening to this show that means that you care about your career You are looking to level up as an engineer and spec.fm was built specifically for you Spec is a network of podcasts including this one that is totally focused on helping you as an engineer Level up in your career go and check it out spec.fm. Thank you to today's producer Sarah Jackson my name is Jonathan Cutrell and until next time enjoy your tea