« All Episodes

Research Bias (Part 3)

Published 2/9/2018

In today's episode, we discuss more research biases that can lead you down the road of bad decisions and bad information.

###Today's Episode is Brought To You By: WooCommerce WooCommerce is an open source eCommerce solution, built on WordPress. With WooCommerce you can sell physical products, digital downloads, subscriptions, memberships, services, and tickets - plus offer flexible ways to pay, including Apple Pay and Bitcoin powered by Stripe.

They're giving Developer Tea listeners 20% off purchases when you use promo code DEVELOPERTEA at WooCommerce.com/developertea (offer lasts until end March 2018)

Transcript (Generated by OpenAI Whisper)
Before we get started today, I want to talk about this incredible tea that I've been drinking for the past couple of weeks. It's called Mad Monk Tea. This is loose leaf tea, and if you've never had loose leaf tea, then you are missing out. If you've only had the tea inside of the little tea bag, then you're missing out on so much of what loose leaf tea has to offer. I encourage you to go check it out, madmonktee.com. If you use the code Developer Tea, it's a discount code you can use a checkout. You'll get 15% off of madmonktee. Once again, madmonktee.com. Go and check it out. They have such good tea, but they also have other accessories. If you don't know how to get started, they can give you everything you need to get started with loose leaf tea. Thank you again to madmonk. Do we really know how to explain what we want? And more specifically for today's episode, do users really know how to explain the things that they need in an application, or in a product, or in some kind of software? As we're talking about in today's episode, we're talking about research this week on Developer Tea and different types of biases in research. This is a different kind of bias. We've been talking about the person who is conducting the research, the kind of bias that you are responsible for. But we're going to talk a little bit more about how to understand your users in today's episode and how to understand that research better and craft it for better responses. This is really a complex topic. I encourage you to do more research on this discussion because three episodes of Developer Tea is not going to be enough to cover what it means to have quality research methods. This is something that large universities do and they have to go through very rigorous processes and have their studies that they publish white papers on, for example, have those reviewed, have all of the methods reviewed by research boards, for example. So it's really important that if you're going to conduct research and make decisions based on that research, or publish that research in some way for other people to make decisions on, that you take some time to understand how research can go sideways. That was the goal of this week. You're listening to Developer Tea. My name is Jonathan Cutrell. My goal on this show is to help you connect to your career purpose. And ultimately, so that you have a positive effect on the people around you. You do better work. You have a positive effect on the people around you. And that can happen most effectively when you feel connected to the work that you do. And you find purpose in the work you do. So I'm really excited to talk about research and to talk about bias anytime on this show, because the way that we think and the way we behave are intimately connected, the way that we understand the world and therefore our resulting actions as a response to that understanding, those things are tightly connected. So I believe it's important for us to understand more about how our brains work, more about how they break. So that's what bias is. It's a way that your brain kind of breaks from reality. So today's episode is going to discuss this idea that, you know, users don't really always know what they want. I'm reminded of the time before the iPhone came out. There were some people who designed what they predicted the iPhone to look like. And most of those predictions looked like iterations on existing designs. So for example, the physical buttons of something like a blackberry or a razor phone, if anyone remembers these kinds of phones before the full touch screen phone was available. And that was what we expected Apple to do to take some of the kind of physical components of the Apple computers that were being released around the time and to integrate those into a phone. Of course, these predictions were completely wrong. And the phone that came out was totally different. But this brings up one of the biases that we want to talk about today. It's called the anchoring bias. The idea that you have something set in your mind and anything that you imagine next is going to have some kind of line drawn from that original anchor. In other words, you're going to see things as a progression rather than a totally new reality. This anchoring bias is also present in other fields. It's not just in research. And in fact, it's most well known in the category of pricing. Specifically, if you've ever gone to a car lot and you see the prices that are slashed down and you see a relatively inflated kind of list price for a car, you see a line through that and you see the lower price below it. This is an example of anchoring. If you know that the original price was something, let's say, $25,000 and you see the new price of $18,000, you have a much different perception of that because your anchor was set at that original $25,000. And it's very difficult to escape this. Even though we may see that and think, well, yeah, of course, they're going to mark the price down. This is artificial. It's not ever going to be the list price. They're always going to have a markdown. Even though you can cognitively reason through that, that anchor still affects you and you still see that differently than if you were to see just the $18,000 price on that car. In the case of research and in the case of asking users what they want, they're very likely to be responding out of their anchor bias. Also for example, if you have an application, let's say you have a mobile application and you ask users what they want, they're probably going to ask you to change something about that application rather than building something entirely new. And it may be the reality that you need to build something entirely new rather than iterating on that older piece of technology. So how can we avoid the anchoring bias? Well, some of it comes from understanding that it exists in the first place. We're going to talk about ways that you can avoid the anchoring bias and really avoid quite a few of these other biases that we've discussed in this week's episodes of Developer Tea. Right after we talk about today's awesome sponsor, WooCommerce. Today's episode is sponsored by WooCommerce. If you've ever bought anything online, then it's very possible that you have used a WooCommerce store because they power about 30% of all online stores. So WooCommerce has been supporting Developer Tea for quite a long time now. We're going to talk about a specific thing for today's episode that we haven't really talked about before. That is for people who have never actually set up an online store. WooCommerce is an excellent option for you and I'll tell you why. The WooCommerce plugin includes a guided install. This includes configuring your payment methods, shipping, your taxes. If you're not familiar with WooLoo, this will be a great deal. It's a great help and prompts you as you set up your store. There's also a 12-part email series that you can opt into for tips and ideas towards making the first sale, making sure your site is secure, how to create backups, and much more. So the other important thing to understand about WooCommerce is that it's built on top of open source and you're going to take your data with you if you decide to go elsewhere. So you own your data forever and by the way, they support pretty much every payment method you can imagine. Go and check it out. WooCommerce is going to give you 20% off, 20% off. That's a fifth of the cost. Head over to WooCommerce.com slash Developer Tea. If you use the code Developer Teaat checkout, you'll get 20% off. Again, WooCommerce.com slash Developer Tea. That code is good until the end of March of 2018. Make sure you head over there before then, WooCommerce.com slash Developer Tea. Thank you again to WooCommerce for sponsoring today's episode of Developer Tea. So we want to talk about ways of avoiding bias, avoiding all types of bias when we are doing this user research, right? When we're actually asking people, what is it that you want in this application? What we really want to understand is how we can solve their problems better. How can we make them more appreciative of the application or of the service that we've built, of the code that we've built? What is it that they're experiencing that is negative that we've created? What are ways that we can increase their happiness as a user or increase their productivity? What are ways that we can make this even more useful, more valuable to them? These are the core questions that we want to understand. Unfortunately, when we ask these kinds of questions, the responses that we get are going to be prone to a lot of inaccuracy, a lot of that anchoring bias that we've discussed already. Now to be very clear, it's okay to ask these kinds of questions, assuming that you know that the anchoring bias exists. So you may hear an answer that really if you analyze the answer thoroughly, you may be able to come out with an insight. But it's important that you understand the key criteria here for good research. And that is to craft the correct questions. Craft the correct questions. There's another bias that we haven't discussed, which I'm going to nickname the lazy brain bias. But essentially what this bias is, is when you ask questions that sound similar to other questions, you're going to tend to get a similar answer. This bias is present in so many different areas. You can see this, for example, in performing magicians. You can also see this, if you watch, there's a segment of the Jimmy Kimmel show called Lie Witness News. And some of this happens on Lie Witness News as well. Essentially what you'll see is you'll see one or two or three questions that sound very similar. And then that next question, the fourth question includes some kind of key information that makes that question absurd, right? Or it makes that question completely different. But because it's worded the same, because the tone of the person who is asking that question is very similar, you're very likely to get a very similar answer. And for the sake of heuristics, you can kind of think of this as the brain gathering momentum, right? And the direction that questions are heading, and you're very likely to continue answering them. And the neurological reasons behind why this happens is basically, again, because your brain is lazy, it's much easier to consolidate these questions and consider them very similar to each other. So therefore, a reasonable answer is going to be similar to the previous answer. What you want to do to avoid all kinds of bias, including this very specific kind of bias, the lazy brain bias, is to craft thoughtful questions that require more thoughtful answers. For example, avoiding yes or no questions, if possible, is going to provide you much better qualitative information, then if you were to just have a yes or no without any further information. So it's important to understand that each of these questions has an effect on the person that you're asking them to. It's important to understand that the ordering of your questions matters. And so if you're going to take a survey, for example, it may be useful to randomize the order of the questions and make sure you write the question so that they don't depend on any kind of order. Crafting questions to understand the user's problem rather than the thing that they believe is going to solve their problem, this is going to get you to a much better position of understanding. The user doesn't necessarily know what they want in terms of a product. They do know what they're trying to do most of the time, but usually they don't know how to do it. That's why your product exists. That's why you exist. That's why the designer exists to help them figure out a way to do what they're trying to do. And we have to be very careful as developers. We have to be very careful. If you're a designer listening to this, we don't want to approach this from the position of arrogance that phrase can be used and abused arrogantly if we say, well, the user doesn't know what they want. That's kind of a misguided way of thinking about the user. It's not that they don't know what they want, it's that they don't know what they want in a product. They don't know what form they want to be solving their problems with, but they do intimately understand their own problems. They intimately understand what they value. And it's important for us to get at that, get at that information with thoughtful, well-ordered, well-crafted questions, avoiding bias, avoiding leaning in one direction, avoiding leading questions, crafting those so that we understand the user's problems more thoroughly, we understand their restrictions more thoroughly, and we understand how to create a better experience for them. Thank you so much for listening to today's episode of Developer Tea. This is a complex topic. There's no way we're going to cover it thoroughly in three episodes of a podcast. I encourage you to go into your own research. We're going to continue talking about user experience. We're going to talk about bias, probably for the rest of the time that this show exists. So I encourage you, if you enjoy these kinds of discussions, go ahead and subscribe and weather podcasting up, you use. Thank you again to today's sponsor, WooCommerce, head over to WooCommerce.com slash Developer Tea. Use the code Developer Tea at checkout for 20% off. Thank you again to WooCommerce. Thank you so much for listening. Thank you.