Creating a Satisfying User Experience through Prototype Testing

Root, loot, and boot some recruits

First off, identify the participants that you need to run your study. Does it have to do something with their demographics? Do they need to be in a certain geographic place? It’s important to define your intended participant pool.

For example, you’re running a study to understand how users spend their money on credit cards in Canada. Your demographic should most likely be Canadian who owns or has a credit card. It is also important to clearly define participants that do not qualify for your study (this is done in a screener questionnaire before inviting participants to a study).

I call this the “Be Mean and Screen” participants. You can create a screener survey to “loot” the right participants. You don’t want to give too much information away in your screener survey. 

An example would be: “What type of credit card do you currently have?

• American (AMEX) Express Credit Card
• Mastercard Credit Card
• Discover Credit Card
• Visa Credit Card
• Other
• None

You also want to throw in an open-response question and see what participants are usually like. Avoid “one-word answers”, and hopefully “root” for the participants that are more engaging as you may find more insights from them.

An example would be: “Describe how you feel about dogs.”

If someone just says dogs are “cute”, they may not be able to provide you a lot of insights during the interview. Again, this may not be entirely true, but it’s something to keep in mind. You get the point, I’m not trying to shame people here. Now that we have our participants, let’s focus on the next step.the Now, that we have our participants let’s focus on the next step.

A good way to frame questions is to use the Taxonomy of Cognitive Domain, which maps out how certain verbs target a particular thought process. It is a great way to probe your users for more information on a particular topic:

For example, under the evaluation domain, the questions can start off with:

• How would you rate …?
• Given the scenario, how would you prioritize …?

For more information, check out this PDF of Bloom's taxonomy.

It’s also equally important to note that we should not be priming our users because we do not want to force the user to answer in a particular way. A few pointers to listed below are:

• Leading questions - You don’t want to lead them, you want to understand which path they take.
• Yes/No Questions - Remember, we do not want single worded responses; unless you want too.
• Double-barreled Questions - You should never ask two questions in one line, it gets very confusing for the participant.

Sharpen your weapons: Making the prototype

Once you qualify some participants, you’ll need to build a prototype of whatever you are trying to test—be it a financial management app or a dog video site—you want to let your participants try something out so that they can show you some emotions, and reveal some pain points. For that purpose, you want a low fidelity prototype that is fully digital.

If you don’t already know what fidelity means in UX, it means how close to a real product your prototype is. From some hand sketches to a fully interactable interface, there are low-fidelity, high-fidelity, and everything-in-the-between-fidelity prototypes. According to the authors, the reason you would need a low-fidelity-but-fully-digital prototype is that “according to Christian, this is because paper prototypes force abstraction, which makes participant feedback when interacting with them both muted, as well as not realistic.”[^ Piepmeier, J. (2020, July 24). How to Use Prototype Testing to Create Emotionally Satisfying User Experiences]. In this case, a high-fidelity prototype might actually be less efficient, for it will not only require you to put in a lot more work to design and (likely) change it, but also distract your participants with all the design details so much that they forget to think about giving you feedback. The key here is to create something simple that your participants can complete a target task all the way by themselves, because when your product launches, you won’t be by everyone’s side to tell them what to do.

When you have a prototype made, and participants assembled, you are ready to roll. You sit them down in a chair or couch, all ready and comfortable, and you notice they look troubled—they are afraid that you might feel sad if they are to dislike something in that prototype—but you don’t want that. I mean, for one thing, if it’s a prototype made by you, it probably does suck big time, but what you want is to find out what sucks exactly, so you need them to speak up. Tell them. Tell them your overall goals and that it’s more helpful if they give you negative and detailed feedback, that’s the only way this prototype can improve. 

Make them talk

While the test is undergoing, keep your participants talking and explaining their thoughts (as in a think-aloud type of protocol). Do not provide help unless absolutely necessary (and when you do, there’s a problem you need to fix; take notes). There are a few tips on interviewing participants. You want your participants to be serious about helping you test but feel relaxed enough to tell you everything. Be sure to avoid speaking too formally; it tends to make people feel uneasy and too serious (the two of us certainly don’t have a problem with that—LOL); therefore, downplay negative emotions or avoid speaking up. Also, try not to affect their emotions by actively agreeing to or exaggerating their claims, which tends to make them exaggerate as well. There is a delicate balance between being too professional and too unprofessional (say, too agreeable); your goal is to extract “accurate but emotionally laden insights.”[^ Piepmeier (2020), see above]

Let’s circle back to the open-ended questions. After you ask one question like that, you want to get clarification. For example, the question is “describe how you feel about dogs”, and when your participants say “dogs are cute” (plural because who doesn’t love them doggos, especially Katsu, who is super cute but you don’t know that dog and if you do, then you know what we are talking about), you want to follow up, see if they find every single dog cute or just a select few of a breed. You can also restate their answer back to them, see if they agree or make corrections/additions before moving on. There is no need to rush, ask for more clarification if you need to, until you are confident you have extracted enough contextual information.

Act on the intel

The reason you want as much contextual information as possible is that when you analyze your results and come up with plans to improve your product, such as this blog post, you want some actionable items, not just “we need better titles.” Tell your writers something like, “seven readers commented that they found the titles confusing and unnecessary,” and then they know to make the titles more concise instead of—you know—making more bad jokes.

Also, it never hurts to revisit and restate your problems. Make sure to consider more than one point of view, and you might gain additional insights.

Battle Cruiser Ready

Remember, the most important part of testing is identifying the users’ pain points prior to running the test. This is the power of prototyping testing; we hope you had a great time reading our blog post.

You are now ready for battle. All aboard into outer space.