« All Episodes

Proof of Complexity and Seeking Simplicity

Published 12/10/2018

Today we're talking about the reality that the work we're doing as developers is complex and those complexities interact with each other. In today's episode, we're talking about ways to chase simplicity when working on complex problems.

Thanks to today's sponsor: Manifold

Why build your own logging platform, CMS or Authentication service yourself when a managed tool or API can solve the problem for you?

With services covering authentication, messaging, monitoring, CMS, and more, Manifold will keep you on the cutting edge, so you can focus on building your project rather than focusing on problems that have already been solved.

As a Developer Tea listener, you will get a $10 credit to put toward services when you sign up. Get started at https://www.manifold.co/devtea.

Get in touch

If you have questions about today's episode, want to start a conversation about today's topic or just want to let us know if you found this episode valuable I encourage you to join the conversation or start your own on our community platform Spectrum.chat/specfm/developer-tea

🧡 Leave a Review

If you're enjoying the show and want to support the content head over to iTunes and leave a review! It helps other developers discover the show and keep us focused on what matters to you.

Transcript (Generated by OpenAI Whisper)
The systems we work on as developers are often complex. They have multiple parts, for example, clients, users. Some kind of machine that is going to consume whatever code you are writing. That client might be code that uses your code. It might be an application that actually consumes your code in some way or uses the output of your code. We have all kinds of dependency management problems. In the JavaScript community, for example, we've recently seen that dependency management can be not only difficult, but it can also have some problems that are unexpected, things like security issues. We are going to sit here and talk about how to fix all of those problems. Instead, just taking a moment to affirm the reality that the things that we do are complex. That's before we really even get to the actual problem that you're solving. The code that you're inheriting from other developers or from yourself in the past, the concepts in whatever domain that you're working in, that you have to wrap your mind around and then create these virtualized representations of those concepts in your code. The complexity continues to mount because all of these things, they can interact with each other. All of your code dependencies may actually have implications based on what domain you're working in. What you inherit absolutely has a relationship to dependency management. All of the problems that you face, and we've only gone through a short list, and that list is very long. We can't cover every single thing that creates complexity in development, but all of those items can interact. When you face a problem as a developer, very often you're only solving small facets of a larger, more complex problem. Every problem has different context, and those different contexts mean that your solution may look completely unique from the next person because of all of those interplaying factors. But in today's episode, I want to talk about the other end of the spectrum, the simple end of the spectrum. I want to discuss the idea of how you can chase simplicity when working inside of a complex system. My name is Jonathan Cutrell and you're listening to Developer Tea, and my goal on the show is to help driven developers connect to their career purpose and do better work so they can have a positive influence on the people around them. Today's episode is sponsored by Manifold. You're probably using managed cloud services. If you're working on anything of any level of complexity as we've already discussed, then these managed cloud services actually help you reduce that complexity by eliminating the time and effort that you would need to put in to build your own stuff. You wouldn't go and build your own logging platform or try to create your own physical servers for really basic server needs, right? You wouldn't go and create your own authentication protocol and service. You can use a managed tool or an API that can solve those problems for you. But how do you know which tool to integrate? How can you learn to stitch all those tools together? And how do you end up managing the access that you and your teammates have to those services? Managing all these details and finding these tools that integrate well together can, on its own, be a full-time job, but Manifold exists to make your life easier by providing a single workflow to organize your services, connect your integrations, and ultimately share all of that with your team. You can discover the best services for your project in the Manifold marketplace or you can bring your own custom integrations and manage them all in a single dashboard. With services covering authentication, messaging, monitoring, content management, and plenty more, Manifold will keep you on the cutting edge so you can focus on building your project rather than focusing on problems that have already been solved. And by the way, Manifold is 100% free. You pay for the services that you use, but Manifold, the integrator that is working to help you solve these problems in a more streamlined way, that part is free. On top of that, Manifold is providing you with a $10 credit. If you had over to Manifold.co slash DevT, you can get started with that $10 credit today. Think we get to Manifold for sponsoring today's episode. So we're going to talk about simplicity in just a moment, but before we do that, I want to kind of drive home this idea of complexity, specifically with relation to how things change when they're interacting with each other. We mentioned the multiple dimensions that we have to be thinking on as developers, but let's talk for a moment about how complexity changes as you add these kind of interplaying factors. So imagine that you have 100 factors that could cause an issue, a problem, a bug in your code. And we know that we usually have more than 100 factors, but let's just say that you have 100 lines of code, and maybe you're trying to decide which line of code is really the source of your problem. This is kind of a bad example because stack traces help us with these things, and not every line is equal to every other line, but stick with me here, 100 factors that could go wrong. Now if there was only one thing that caused the problem, if there was only one dimension to your problem, then you have 100 possibilities. Okay, this isn't so bad, right? Because we can pretty quickly eliminate half of those just by kind of eyeballing the problem, maybe reading a stack trace and doing a quick Google search, we can probably eliminate maybe even 75% or 90% of those potential issues, which leads us to 10 remaining problems. This is something we can manage because we can typically kind of look through what's happening in the problem and try different solutions for those 10 issues, but when we start changing the number of interacting factors, that number goes up and it goes up quickly. Let's say for example that any two unique factors in that 100 could be causing your problem. What doesn't go to 200? Instead it actually goes up to 4,950. This is a combination calculation and please correct me if I'm wrong, if you know something more about combination calculations, but complexity grows at an astounding rate. Now warning, I'm going to try to explain what this little formula is so you can do it on your own if you'd like. We have in number of objects in this case 100 and then our number of samples and you can find this if you just Google combinations calculator, you'll find this the same calculator I'm using to explain this on calculator soup is the website. But the formula for this combinations, this complexity calculation is that in number 100 factorial divided by, and this is all one number, the sample factorial in this case 2 factorial times the object number that in minus r factorial. So in this case it's 100 factorial divided by 2 factorial times 98 factorial. If the numbers were 20 and 5 then the calculation would be 20 factorial over 5 factorial times 15 factorial. And for those of you who don't remember what a factorial is or you never heard, the simple definition of a factorial is the product of a number, a whole integer and all of its preceding whole integers. So 5 factorial would be 5 times 4 times 3 times 2 times 1. Okay so why does all of this matter? And am I telling you to go and do math? Of course not. But I really want you to grasp just how complex things can get. If we decide to go with 3 rather than 2, remember we have 100 possibilities and any grouping of 3 unique items, that number jumps up to 161,700 different combinations. Now that's order independent by the way. That means that these combinations are not including combinations of the same items reordered. Those are unique combinations that you can make out of 3 different items with 100 sample size with 100 possible items. And you can imagine that any given problem that you face in any code base of some reasonable size with any kind of domain information that is being added, any kind of packages that you're depending on externally, all of these things are added complexity. All of these things could be part, a factor to be considered. And so the complexity of what we do is enormous sometimes. And sometimes it's so enormous that it takes us days to figure what we feel like should be a very simple thing to figure that thing out. And for those of you wondering when you go to 4, that number jumps to 3,921,225 different combinations of 4 items when you have a sample size of 100. So this is a very important thing to grasp because as developers, when we add even one simple thing, one extra dependency, one more thing to think about. When we don't keep things as simple as we possibly can, we create future scenarios where we have to evaluate exponentially more possible pathways. Now I will admit that this may be an exaggerated example, right? Because you're not going to walk every single one of those pathways. You're probably going to be able to eliminate a large portion of the possible, you know, interactions between various code pieces. But there will be times when very unexpected things will happen in your code. So we have a bias towards complexity as developers or maybe a better way to put it. We have a bias against simplicity. Perhaps because we have worked in code bases that end up being complex and the things that we appreciate about software development often are complex. And when we see simple things, we at least see them composed into complex applications. We see simple code that is, for example, abstracted and has multiple levels of indirection. And so our natural instinct maybe has developers is to gravitate towards complex solutions. And we may even push others that we work with to gravitate towards those complex solutions. And we can feel this happening all the time. So for example, imagine that you found the simplest solution to a problem that you were trying to solve. Now that problem or the solution to the problem doesn't have any kind of abstraction. You're not pulling classes out. You're not refactoring that code. You're not adding new dependencies. You're solving the problem all in the same place and in a very straightforward way. Now very often if you were to write that kind of solution and submit it in a PR based on some kind of internal culture or perhaps based on what we're used to as developers, we often will comment on how to make that code more maintainable. How to improve that code and it goes from simple to less simple, perhaps not egregiously complicated. And most likely those comments are reasonable and the trade-offs are often worth it. And you end up with a better product in the end. But in the end we often avoid submitting that original simple solution and we instead try to jump ahead. We move away from that simple solution and we gravitate towards complexity and sometimes that gravitation takes us too far. Sometimes we over optimize, we abstract classes out before we really need to. We refactor and then we refactor again. We try out multiple external dependencies before we settle on one. So my challenge to you is this, as you go throughout your work this week, when you encounter a problem, remember that complexity grows and it doesn't grow linearly. It grows very fast. It grows exponentially so that just enough is not only the most efficient way to solve a problem in the beginning, but it also happens to be the most sustainable in the long run. If you're writing quality code and I want to make this distinction, the code can't be poor quality. You can't choose non-descriptive variable names or difficult to understand method procedures because you don't want to write a little bit of extra code or you're trying to save byte space. That's not what we're talking about here. So I encourage you to write quality code, but write only enough to solve the problems that you are trying to solve. If you're trying to solve future problems or if you're trying to over optimize your code or abstract it too thoroughly and make it reusable before you ever really even need to reuse it, those are the signs, those are kind of the red flags that you're creating a more complex solution to a simple problem. So focus and seek simplicity and once you actually implement something that is simple enough, then you can decide what about it is insufficient. Do this amongst your peers, do this in poll requests, in code reviews, do it in a pairing session, try the simple thing first and if it's not sufficient, then solve the thing that is insufficient. Don't try to refactor all of the code, don't try to abstract all of the code, only solve the insufficiencies. This will maintain the original simplicity and instead of actually trying to go through and clean up all of the code, this is something that we do as developers, we try to clean up a PR or refactor the entire change set. Instead focus on refactoring those small pieces that are insufficient for solving the problem in the way that maybe your company standards require, for example. Thank you so much for listening to today's episode of Developer Tea. Thank you again to Manifold for sponsoring today's episode. You can get $10 worth of credit to use on any service on Manifold's marketplace. Head over to Manifold.co slash devt that's manifold.co slash dvtea. Thank you again for listening. If you've enjoyed today's episode, I encourage you to subscribe and whatever podcasting app you use this episode is coming out on a Monday, but we release on Monday, Wednesday and Friday, virtually every week. We have a break coming up for Christmas. We're going to be doing some re-aering, but usually we're releasing brand new content on a weekly basis more than once a week, usually three times a week. So if you don't want to miss out on future episodes, I encourage you to subscribe and whatever podcasting app you're using to listen to Developer Tea today. Thank you so much for listening and until next time, enjoy your tea.