How to Write Good Usability Testing Questions

Key Takeaways:

• Conducting a usability testing session is crucial for ensuring that your users receive the best experience possible when using your product or service, which will ultimately drive conversion for your business. 

• Usability testing offers insights on how users will interact with your product or services, unlike user testing sessions which offer information or whether your users even need what you are potentially offering. 

• There are different phases of usability testing, each of which has its own type of questions that should be asked to get the most out of the session. 

• A good usability question is one that is specific, relevant, thought-provoking, clear and concise. 

“To ask the right question is already half the solution of a problem.” – Carl Jung

In today’s dynamic and ever-changing world, knowing the right questions to ask and learning how to ask them is crucial for you and your business to attain new opportunities and growth. 

As Carl Jung said, half the battle is about finding the right questions, so in this article, we’ll be going through the importance of mastering the art of questioning and how it can benefit your next usability testing session.

Importance of Usability Testing for UX

The importance of usability testing should never be overlooked as it ensures your users can use your product or services. Usability testing serves many different purposes, but it’s primary goal is to gather information in order to identify potential usability issues of your product or service that you are offering in order to improve it for the best experience for your users. 

Unlike user testing which shows you whether or not users need your product or service, user testing determines whether your users can even use it. In other words, user testing can help you better understand if your users can use your product or service effectively, and if not, how you can help them to do so. To learn more about how usability testing can benefit your business, check out our user testing services today.

There are many benefits that a good usability testing session can provide, for example:

1. Higher development ROI – Gathering valuable information about the usability of your product or service means minimizing the amount of work that would need to be done during the developmental process. By using the observations you have collected, you can avoid wasting time and resources on the wrong things, overall lowering both risks and costs.

2. Increase in conversion rates – Usability testing sessions offer key information on how to effectively drive conversion rates as you can identify and avoid any obstacles that your users may face. This information allows you to adjust your products or services according to what your users want and need.

3. Offers insight on user satisfaction – Knowing how satisfied users are and what aspects of a product they look for in order to fulfill their needs is key for retaining your existing customers and can help you better understand and attract new ones. 

4. Identifies issues within the product or services – Customer insights are always useful – no matter if they are positive or negative – if anything, it’s the negative insights from your users that will be the key driver of a better performing product or service.

5. Saves time for both the company and users – Knowing what your customers want, look for, need and more will ultimately help you save time in the long run. You will save time on development and your users can avoid wasting time on an application that doesn’t work properly.

Now that we’ve gone through some of the benefits conducting a usability testing can offer, let’s go through the different phases of a usability testing session – as the questions during each phase will differ slightly.

The 4 Phases of Usability Testing

Usability testing isn’t just throwing out random questions in hopes of getting something useful. There are four different stages within the process of usability testing, those being:

1. Screening

2. Pre-test

3. Testing

4. Post-test

1 Screening

The screening phase, or also known as the planning phase, is the most important part of the usability testing process. Even though every usability test can be different depending on each business’s individual goals and requirements, this screening phase remains crucial – think of it like this, just an hour of planning has the potential to save you and your business countless hours of mistakes. 

It is during this phase of usability testing where you should determine what it is that you really want to learn from the session, whether that is your user’s concerns, areas of interest, and so on. This phase is an opportunity for you to familiarize yourself with your audience so that you can cultivate the most suitable product or service for them.

Here are a few examples of screening questions:

• What is your current age?

• What is the highest level of education you’ve completed?

• What is your current profession?

• Where are you currently living?

2 Pre-test

Now that you’ve planned a few different questions for the screening process, it is time for the pre-test phase of usability testing. This part of the process gives you a final opportunity to question and interview the test participants – this could be through various forms such as a questionnaire or a moderated interview before the real usability test ensues. 

In contrast to the screening phase, during the pre-test phase your goal is to learn about your participants’ knowledge and experience. This will help give a clearer picture of how your participants may react or interact with your product or services, and whether or not you should alter some of your questions during the actual testing stage. 

Here are some examples: 

• How often do you buy products or services online?

• How confident are you with your ability to complete online tasks? 

• Which device(s) do you find yourself using most often to complete them?

• Have you ever used this website or application before?

3 Testing

It’s now time for you to conduct the actual testing. In the planning stage, you’ve gotten to know the demographics of your users. In the pre-test phase, you’ve learnt about their current expertise and knowledge in regards to your application or services; and now, you get to learn how and why they navigate through them.

During this stage, your participants – or users – will be navigating their way through your website or application in their own unique ways. To reveal the reasons behind their moves, here are some questions you could ask them: 

• What are your thoughts as you are viewing this page or feature?

• If you were trying to look for [information], where would you expect to find it?

• How was your overall experience trying to complete this task?

• What motivated you to click on [information]?

4 Post-test

We’ve now arrived at the final phase of this whole process. This is your final opportunity to ask your users any questions that haven’t yet been answered during all the other phases. Use this time to gather feedback from your users’ experiences, impressions or opinions of your product or services. 

Here are some questions you can ask:

• What was your overall impression of the [information]?

• What was your favourite part about it? What was your least favourite part?

• If given the opportunity, how would you change [information]?

• How would you describe your experience overall?

What Makes a Good Usability Testing Question?

When writing usability testing questions, you want them to be able to answer to your goals and objectives to avoid time and resources being wasted. To prevent issues during any phase of usability testing, always make sure your questions follow the following checklist:

1. Specific 

2. Relevant 

3. Thought-provoking 

4. Clear

5. Concise 

Let’s look at an example of a good usability testing question:

“Tell me about the last time you bought something online.” 

Instead of asking the users what was the last thing they bought online, which can be unproductive because users are likely to answer with one word, you give users room to discuss their whole experience.

This question also makes sure users will stick to the topic in the question as it is specific and relevant – yet remains pretty concise so they do not get confused and end up rambling about something that will not answer the question or your goals.

Let’s now take a look at some questions you should avoid.

What Usability Testing Questions Should You Avoid Asking?

Now that you know what makes a good usability testing question, let’s go through some usability testing questions you should avoid and why.

Make sure that your usability testing questions are able to guide the conversation or discussion instead of leading it. The questions you ask should not be pointing your users toward answering in a certain way or in a particular direction. 

Here are some examples of guiding questions:

• Which features of the website would you use the most?

• How much time do you typically spend on a website?

• What features would you expect to find on this type of website?

Notice how these questions are clear, concise and remain open-ended so users can elaborate on their answers. Your questions should be open-ended to allow for longer responses, therefore, potentially allowing more creativity and information to flow easily from your users.  

This will provide you with the best data to improve your products and services during the development process. Read more about our digital product development process here.

Here are some examples of leading questions:

• Did you like this feature?

• Do you think the page loaded too slowly?

• Was it easy to use the functions of the website?

• Will you use our product or services again in the future?

Now, notice how some of these questions aren’t open-ended and only require a yes or no answer? When conducting a usability testing session, you want to gather the most valuable and relevant information within the time frame given and these types of answers may not provide you with as much value as what you could get from more open-ended questions.

This is because a yes or no answer will not provide you with elaborated answers and you will need to waste time asking follow-up questions to get the insights you need. 

These questions do not tell us why the user dislikes or likes the product or services and it also doesn’t give users the opportunity to elaborate on their experiences as well, therefore, wasting everyone’s time and your business’s resources.

Invest In Usability Testing

Usability testing is a process worth investing in as it offers immeasurable insights about your users. Not only will you be able to learn how your users really feel about your product or service, but also how they will interact with it – thus giving you information on how to make your product or services work more efficiently and effectively. 

While conducting a usability testing session may sound easy and simple to do, it can be more difficult or complicated than it seems, it involves a lot of coordination in aspects such as planning, observing, analyzing and more. Which is why it is crucial that you invest in a company with both the experience and expertise in the UX field.

Get in touch with our team

At Morphosis, we have been improving user experiences for well over two decades. We are not only well equipped to investigate user-related issues at scale but we are also able to effectively solve them with our innovative UX strategies. 

Through working with some of the biggest brands in Asia, we’ve grown to become a leading digital consultancy in Thailand, specializing in UX research and strategy, UX / UI design services and digital product development. 

To learn more about how we can help you grow your business, send us an email or send us a message today.

Nunteema (Punn) Supapon Nunteema is the Associate Design Director at Morphosis. She plays a key role in leading the design team and ensures projects meet the highest standards of excellence.