« All Episodes

Using ChatGPT for Career Experience Simulations - LLMs and Direct Simulations for Faster Career Experience Gains

Published 12/3/2024

Can you use ChatGPT to become a better engineer? I'm not just talking about learning a language or fixing a bug with ChatGPT providing advice... I'm talking about career simulations. In this episode we discuss how the airline industry has fixed the problem of experience, and how we can learn from their insights to create low-stakes learning environments that boost our career and confidence through the roof.

🙏 Today's Episode is Brought To you by: Wix Studio

Devs, if you think website builders mean limited control—think again.
With Wix Studio’s developer-first ecosystem you can spend less time on tedious tasks and
more on the functionalities that matters most:
● Develop online in a VS Code-based IDE or locally via GitHub.
● Extend and replace a suite of powerful business solutions
● And ship faster with Wix Studio’s AI code assistant
All of that, wrapped up in auto-maintained infrastructure for total peace of mind.
Work in a developer-first ecosystem. Go to wixstudio.com

📮 Ask a Question

If you enjoyed this episode and would like me to discuss a question that you have on the show, drop it over at: developertea.com.

📮 Join the Discord

If you want to be a part of a supportive community of engineers (non-engineers welcome!) working to improve their lives and careers, join us on the Developer Tea Discord community by visiting https://developertea.com/discord today!

🧡 Leave a Review

If you're enjoying the show and want to support the content head over to iTunes and leave a review! It helps other developers discover the show and keep us focused on what matters to you.

Transcript (Generated by OpenAI Whisper)

experience is an excellent teacher you know this of course this is one of the reasons why years of experience is a very typical requirement on any job posting and rightfully so experience is hard to replace with any kind of academic learning for example that's not to say that academic learning is not useful it's not to say that you can't gain experience through other means other than time but experience is a critical part of your career one of the reasons that time and experience over time is one of the critical factors for success in your career is because there aren't a ton of opportunities in your kind of day-to-day work to have for example critical conversations you may have a critical conversation once a month for example and so over time you may need to accumulate let's say 20 or 30 of those critical conversations before you can confidently approach any given critical conversation now this kind of waiting around is difficult to do it takes a long time to get to the place where you've had enough of these kinds of discussions to feel confident i want to switch gears and talk for a moment about a completely different industry and you'll see why a few minutes the airline industry or even just the aviation industry is a completely different industry and so i want to switch gears and talk for a moment about a completely different industry and you'll see why flying private airplanes if you've been listening to this podcast for a while you know that i have my private pilot's license and i've been flying for most of my life either flying myself or flying as a passenger with my with my father who also has his pilot's license and has flown many different aircraft so by nature of this kind of exposure i am familiar with this industry and you might know that aviation especially the airline industry is known for its ability to fly in the air and its safety record some might uh you know disagree that this is true for all of aviation certainly it changes as you go through less training but the training is the part that i want to talk about so in order to uh operate an airline safely in order to operate a large you know 200 passenger jet for example you need to know how the jet behaves but not just when the jet is in the air but when the jet is in the air and when the jet is in the air and when the jet is in the air is operating correctly not only when the jet is you know doing what it's expected to do all systems operational uh you know no red lights blinking on the dash so to speak you also need to know what happens when something fails how should i respond to it how do i deal with multiple failures what if i lose an engine or what if my hydraulic system fails now we don't have to enumerate all the possible failures of a you know 737 on this show but the important thing to understand is these pilots know how to deal with these emergencies and as it turns out these pilots know how to deal with these emergencies through a type of experience but very rarely is a pilot experiencing actual real life emergencies multiple times in their career certainly not enough to learn through direct experience so what is it that they do to deal with these emergencies and how do they deal with these emergencies they are doing to get them this experience to get them ready for dealing with an emergency i'm sure this comes as no surprise this happens as a part of their training as a part of their learning process and it happens most importantly in a low stakes environment this is something that very often we entirely ignore in our careers as software engineering leaders today i'm going to teach you how to deal with these emergencies and how to deal with these emergencies and how to teach you how to think about creating a low stakes environment to accelerate your experience gathering right after we talk about today's sports i'm going to ask you to stay put for five seconds longer than you would when i say the words website builder i know what you're thinking it probably has links to a lot of the things that you're going to be talking about today but i'm going to ask you to stay put for five seconds longer than you're going to be talking about today because i'm going control or outputs a bunch of dirty code this is my nightmares from the dream weaver days but what about a nodejs based builder that lets you add full stack javascript code to any site well with wix studio you can spend less time on your ui uh less time on hosting less time more time on the custom logic and the functionality that actually matters that makes a difference to you and your employees because if you're going to be able to add full stack javascript code to any client. You can develop in your preferred coding environment, whether that's online in a browser, in a VS Code-based IDE, or locally through a GitHub integration using whatever tools you're used to. Either way, with Wix Studio, you're deploying in a click. Extend and replace hundreds of powerful business solutions and custom-built features with APIs and integrations. And when you need to speed things up, Wix Studio's AI assistant is on hand to generate tailored code snippets, troubleshoot bugs, and retrieve product answers in just a few seconds. All of that is neatly wrapped up in an automatically maintained infrastructure for total peace of mind. Working a developer-first ecosystem, head over to wixstudio.com. That's W-I-X-studio.com. Thanks again to Wix for sponsoring today's episode of Developer Team. So we're talking about creating a low-stakes environment. And we mentioned the aviation industry as a good model or a kind of a mental model for this. And the way that they do this in their training is they create simulations. They build the situation, whether that's in the air or on the ground. For example, a simulated engine failure. This might pull the power on the engine all the way back to idle. This is essentially the same as losing the power to your engine in real life. But it maintains a level of safety because your engine is not actually turned off. This is important because you can go through the same motions that the airplane is going to respond in the same ways as it would if your engine actually failed. This simulation allows you to... To learn and create the muscle memory that you would need in order to actually respond to a real life engine failure. So I want you to think about this specifically in terms of simulation. Of course, also, especially in the airline industry, it's a lot more cost effective. Even if you could simulate these things in real life up in the air, if you could simulate them flying at 737, this would be... Very cost prohibitive to do. It's very unlikely that you're going to do this kind of simulation with passengers on the airplane, for example. And so the airline industry relies on actual simulators. These are these very complicated devices that are the size of a room that you would sit in and it has all of the high fidelity kind of instruments that you would expect to see in the real airplane. And so it's very, very cost prohibitive to do. And so the airline industry is going to be very, very expensive. So if you could simulate these things in the airplane, right? So the simulation has been created to give you a high level of immersion, right? This concept of immersion is that your brain is going to operate in the simulation in a very similar manner, right? You're going to build some connections in your brain. And here's the important part. Okay. You might think, okay, well, that's all fake. But when it comes to real life, how do we know that that's going to transfer? There's actually very good reasons. Research. Okay. Very good research on simulation. And as it turns out, this kind of overlaps with research on imagination. Our brains are not very good at determining what is real, quote, real, and what is not real, what is either imagined or simulated. Even if you can cognitively recognize, right? Like you can reason, you can rationally think through, okay? This is a simulation or this is my imagination. Your brain will still decode your imagination and simulations as something very real in a very kind of related sense. We, we experience high levels of emotion when hearing stories. Sometimes these stories are not real at all. You can imagine somebody getting emotional, watching a movie or reading a book, right? We can connect to something that is entirely, entirely, you know, built up in our head. And, uh, we, we connect to that. And so, uh, our brains experience that as, as real as anything else. Similarly, we can imagine the future and, uh, get excited about the future. We haven't experienced it yet, uh, but we anticipate and that anticipation gives us the same kinds of feelings that we would have if we were actually experiencing the thing, right? The, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, right the the other side of the coin is we can anticipate and feel anxiety and we can feel you know our hands get clammy uh our heart rate increases when we're just thinking about an uncomfortable situation or you know a meeting that's coming up that we're not really looking forward to maybe it's going to be a little bit of a stressful meeting we imagine that meeting even though it's not happening to us we could be in a totally safe and secure environment and when we start imagining it our body starts to produce the fight or flight response that happens when we're in a threatening situation and so your brain is decoding your imagination as a threat right so uh there's there's plenty of evidence that shows that this is the case that imagination simulation are both uh you know very difficult to um you know to to parse through for your brain at its lowest levels so we can take advantage of this reality right we can take advantage of it and we can simulate these kinds of situations these critical conversations for example right we can simulate these situations that we face in our work uh interviews is another great example right so i'm going to share with you two different tactical ways that you can simulate these these kinds of things in your daily work the first is a kind of direct simulation okay direct simulation what this looks like is let's imagine that you're you are preparing for technical interviews you may give yourself a leak code problem this is something you're probably already doing you may not realize that you are doing a simulation when you do this if you don't feel like it's a simulation one thing you can do is you can constrain yourself on the amount of time that you take right so instead of just giving yourself you know an endless amount of time imagine that there are people watching and in fact you can even have somebody uh watch you you can uh you can watch a video of somebody watching you you can watch a video of somebody watching you you can watch a video of somebody watching you you can watch a contact a close colleague maybe somebody in the developer t community uh on our discord you can contact somebody there and ask them if they would be you know willing to do kind of a mock uh technical interview with you all right so uh people are watching you so you get a little bit of that feeling of somebody is judging the work that you're doing uh rather than you know treating this just as oh i'm learning the material you can also prepare for the other side of this interview which is kind of the pressure the feeling of the interview which is kind of the pressure of the pressure coming from another person watching so try to emulate right notice that we're kind of you know mixing a little bit of terminology here but emulation is trying to make the uh the situation that you're in look a lot like the other you know the real situation they're trying to simulate right so try to emulate some of those properties uh emulate the amount of time that it might take emulate you know the number of people or you know the kinds of questions that those people are going to be asking you and you know the kind of questions that those people are going to ask you try to create uh you know a mock version of the same thing this is a direct simulation another example of a direct simulation might be like a mock presentation walkthrough right you have a press and presentation to give to your colleagues uh you have a colleague that you trust that you want to give this presentation to in advance and have them provide you critiques have them take notes during the presentation identify places that feel comfortable and that you want to give to them and that you want to give to them and that you want to give to them and that you want to give to them and that you want to give to them and that these are ways that you can gather feedback and learn right that's the important part here right the the idea is that you're going to get multiple cycles of experience out of a single you know critical event in this case a presentation or an interview and you're going to get this experience by iterating through lower stakes environments so what happens if you completely fail your code fake interview you may have toension This is the low stakes part of this learning, right? If you fail your leet code interview, the only thing that you really lose from that is a feeling of pride maybe, right? Or impressing, you know, this stranger that is doing the mock interview, right? There's very little that you lose, but you gain the learning, right? This is why this is kind of an asymmetrical benefit to you, okay? The asymmetrical side of this is you have a lot more to gain than you have to lose in a simulated environment. The gaining, you know, the upside of a simulation is much larger than the downside and a failure state in the simulation. So these are direct simulations. These are the ones that are more intuitive. These are the things that you would expect to do in order to simulate something. You know, getting dressed in your interview attire. And actually sitting in front of a Zoom camera. Having somebody interview you in a mock interview. These are ways that you could simulate your, you know, that critical event. Another example, right? And this is one that is a little less intuitive, okay? A little less intuitive. The previous ones were intuitive because basically you're just trying to do the thing before you do the thing, right? You're trying to create an opportunity to practice. These are... You know, the most basic kinds of practicing. This second category is fairly new because you're going to use something like ChatGPT or Clot or whatever LLM of your choice, okay? Preferably one of the newer models. The models have gotten to the place, especially more recently, where this can be done to some good effect, okay? And I hesitate to talk about specific tools on this podcast because... It's very possible that this goes out of date or, you know, I don't want to produce clickbait for this show. But I do think that this tool set, it represents a critical change in the way that we can simulate our environment. And here's the reason, okay? LLMs, by their kind of fundamental nature, we're not going to break down all of the specifics of how LLMs work. You can find that content elsewhere. But they are... Particularly good at understanding language, okay? They are particularly good at understanding how humans behave, how they talk, how language evolves over this course of a conversation. Now, understanding that, you know, an LLM can do this kind of language prediction or it can understand a conversation. It can understand semantic meaning. And as it... Moves through a conversation, it understands how the conversation is changing, right? If we imagine, okay, I want to have a simulated conversation, LLMs provide a pretty amazing opportunity to simulate conversations. In other words, what we have in LLMs is something that is particularly good at understanding how a conversation may evolve, all right? You provide your responses. Your responses. You provide some kind of, you know, written input. And the LLM is going to use a ton of information about how other people have responded to similar words, okay? Similar construction, similar kind of logical construction. It's going to use that to provide a response to you. In other words, in many ways, an LLM is fundamentally a conversation simulator. So here's an exercise. I want you to try, okay? And everybody can benefit from this. Not just people who are looking to, you know, improve their interviewing skills, okay? You're especially going to benefit from this if you have an asynchronous communication environment in your work. And most people who are listening to this episode, you do have this somewhere, okay? Somewhere in your work, you have an asynchronous, you know, work environment. Or you have an asynchronous communication environment as a part of your work. All right, so here's the exercise. I want you to think about a person that you need to have a critical conversation with or that you don't really get along very well with at work. But you have an incentive, you have some kind of incentive to get along with them or to persuade them or to talk to them about some particular topic. Maybe you need to provide them with a status update. Okay. You know, maybe this diamond. Dynamic doesn't have to be your boss, right? You don't have to simulate just conversations with your boss. You can simulate a conversation with a coworker. You can simulate a conversation with somebody in a different department, right? Whatever it is, your situation, try to understand or nail down who that person is. Okay, what is it that you want to get out of this conversation with them? And why is it that it hasn't worked before? What about their personality is it that's difficult? What about their personality is it that they're not going to be able to handle? What about their personality is it that they're not going to be able to handle? I want you to try to describe that person as your prompt for ChatGPT or Clot or whatever LLM you're using. An example prompt might look something like this. You are a hard-nosed, deadline-focused boss. And we're going to have a conversation about a body of work, a project that I'm trying to deliver. That is behind schedule. It's behind schedule by a month or two. And you want hard deadlines. You want to push me for that. You're going to corner me and force me to provide these dates. And I want you to do this as if we are sending each other Slack messages. Okay, this kind of prompt gives the LLM a kind of a starting idea. A seed personality. And you will likely be surprised at how well this simulates a conversation. So you start the conversation and you ask them to kind of ask you for an update, for example. Right? Now, what's really interesting is that the LLM can also, you know, you can run the simulation multiple times. You can carry the conversation to its natural endpoint and then ask the LLM, okay, let's start over. You may also be able to ask the LLM something like, help me understand where I'm going wrong. Or help me understand what part of the conversation I need to change my approach on. Now, this isn't a perfect system, of course. Just like almost every AI product you're using right now. There's a big old disclaimer on this episode. That basically, you know, it might work well or it might not. You can certainly trick this prompt. You know, to tell you something that you want to hear. So I want you to exercise caution with this. But if you try to treat this as a true simulation, in other words, imagine that you are really actually messaging that person. Right? That person that you thought of that you have this kind of conflict with or you need to have a critical conversation with. If you actually truly imagine yourself talking to them in this way, you're likely to gain some kind of benefit. Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? Right? we had the hyper optimizer, you know, coaching voice that allowed you to think through the lens of somebody who is only, who only cares about a specific outcome, right? They only care about, you know, getting more money. For example, you can design in this LLM situation, right? You can design this person to be a more caricaturized or a more extreme version of the things that you're struggling with in real life. Why would you want to do this? Well, as a general rule, if you are able to deal with something more extreme, then you are also going to be able to deal with something less extreme. If you can mentally prepare yourself for somebody who exhibits this particular characteristic in a very extreme manner, then you can also prepare for the more realistic, version that you're more likely to encounter in real life. So in my previous example, a boss that only cares about deadlines, very rarely is this actually true. There are, you know, most of the time somebody who's asking you for a specific date of delivery, they have multiple things that they really care about, but they're asking you for a date because that's what they're being asked for, right? So it makes sense to simulate both. It makes sense to simulate both the extreme caricaturized version of, you know, whatever this difficult personality is that you're dealing with. Or maybe, you know, maybe you want to do some kind of written essay simulations, you know, telling the LLM to ask you certain questions about your background. Another example, if you're preparing for interviews, feed your resume to ChatGPT or to Claude. And ask it to, you know, be a pessimist and to poke holes, to find places that they can ask you questions to make you feel uncomfortable or make you feel, you know, inferior. Now this, this is, this sounds like it would be really difficult, but again, here's the important thing. This is a low stakes environment, right? This is a simulation. It's a low stakes environment. There's nothing that's actually bad that will happen as a result of this chat. Right? Very likely the thing that it will do is it will point out areas that you can improve on. It's going to point out, you know, patterns of, of communication that maybe you didn't realize you had, or it might point out, oh, this is a flaw, or this is a thin part of my resume, or this is a confusing wording that I've put in here, or I'm being vague about, you know, this particular job or, or the outcomes that I produced in, in this role. So there are a lot of things that you can use this. And again, I have this big disclaimer because, you know, these, these things are not really deterministic and you should be using them with a little bit of caution, but with the intent of trying to find ways to simulate your real life situations in a much lower stakes environment. Right? So you might be asking, okay, what, why would I use that versus the direct simulation? One, it's available all the time, right? You can, you can use this. At midnight you know, the night before an important meeting, if you really needed to do that it's available to you. Right. Another thing is that if you ask another person to act like you know, a particular person. So if you're asking a colleague or you know, a coworker, or maybe somebody outside of your company, if you're asking them to act like a boss, they may not be able to, to act very well. Right. So it kind of breaks a little bit of that simulation. So with an LLM, the acting portion is, you know, there's, there's not like a, an association that you have to get past with an LLM. So there's no kind of personal connections that muddy the waters with this. Now you may want to use a direct simulation when you're trying to get more qualitative feedback, right? You know, this, and when I say qualitative, I mean feedback about, for example, your, your body language. So if you're using a direct simulation, you may want to use a direct simulation. So if you're using a direct simulation, you may want to use a direct simulation. If it's, if it's dependent on something more than just your pure language that you're using, then a direct simulation may be a better choice. So as an example, if you're simulating in your interview process, you may be able to use an LLM simulation to find content problems, right? If you want to tell a story or tell about your background, the LLM may be able to help you shape your content, but your actual presentation, the warmth that you present or the confidence that you show up with your posture, all of these other kind of more qualitative things are going to be better assessed by an actual person. Overall, a mix of these tools is going to provide you with a much richer tapestry of simulation opportunity, right? So this idea that you have to do a lot of simulation, you have to do a lot of simulation, you have to do a lot of simulation. If you want to wait for something to happen in your career, you can dispose of that idea. And instead you can seek out an opportunity to learn about it through a simulated situation. This is a much lower stakes, much lower cost and a much higher iteration rate. Thank you so much for listening to today's episode. I hope you enjoyed this discussion about simulation. Thank you again to today's sponsor Wix studio website builders of yesterday are gone with Wix studios developer first ecosystem. You can spend less time on tedious tasks and more on the functionalities that matter the most develop online via a VS code based IDE in the browser, or you can develop locally using your tool set and get hub extended, replace a suite of powerful business solutions and ship faster with Wix studios, AI code assistant. All of that is wrapped up and maintained in an automatic infrastructure for total peace of mind. Work at a developer first ecosystem at Wix studio, dot com. I'm going to make a quick three part ask of you at the end of the year here. And hopefully this episode and other episodes of developer T have made a difference in your life that these three things, which are totally free for you to do are doable, uh, on, on your end. Um, and these are the ways that we keep the show alive. The first thing is help another engineer by sending them the show. If you know of an episode that might help them, right? Not don't just send them the show and say, you should, listen to this, help them out, find an episode that makes sense. Maybe, you know, somebody who's going through interviewing process, this episode might be useful to them, right? So send an episode to a person that you know, that might benefit from it. It doesn't have to be an engineer. Uh, sometimes these concepts easily transfer to other, uh, to other disciplines. Certainly, uh, this episode will, will transfer to other disciplines. That's, uh, that's the first thing. Second thing is leave a review in iTunes. We've stopped asking you to leave a review in every platform on the planet. That's because iTunes is the vast majority, uh, of listeners for this show. And if you leave a review anywhere, iTunes is going to have the, the largest impact. Uh, so leave a review in iTunes. You can, of course, still have a review in those other places, but, uh, asking you to leave a review in iTunes, this is the most important thing, uh, to help other engineers find this show and decide to listen to it. The last thing I want you to do, is to find a day between now and the end of the year to spend 10 to 15 minutes disconnecting and thinking back over the year. And, uh, you can use this time also to think forward, but really reflecting on how your year has aligned with your priorities in your career. Here's the thing. Developer T directly doesn't benefit from you doing this. You benefit from this. And I hope that as you move forward in your career and as, especially if you've listened to the show in the past, that you'll continue listening, that this show has been a part of the positive side of your year, uh, since last December. Thanks so much for listening. And until next time, enjoy your tea. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Bye. by the!