The UX testing environment can be complex and overwhelming for newbies and seasoned veterans alike. To help us navigate this landscape we sat down with Justin Evans (Ozow's Head of UX and Product) who gave us valuable insight into why rapid feedback loops are so powerful, and why testing is a vital link in a product's chain as it can mitigate risk, save time and money.
What is the difference between moderated (formal) and unmoderated (informal) testing?
Moderated testing (formal testing) is a real-time interview with someone who has come to test your solution (this can be in the form of showcasing your actual product/service, or it can be a prototype/concept). This type of test can be in person or virtual. Either way, the idea of the moderated test is that it is a “live conversation” with the participant.
Unmoderated testing (informal testing) is when participants complete a predetermined set of tasks without a moderator / facilitator. The participant is given the flexibility to decide when and where they complete these tasks and provide feedback as their session is recorded.
The idea is to encourage the participant to “think out loud” and speak their thoughts. Since these tests are recorded, the team can review the recording at a later stage. Unlike the moderated tests, there is no moderator / facilitator that can ask follow-up questions. However, after viewing the session, the team can reach out to the participant (if they agree to it) and follow up.
How do you go about moderated testing?
Moderated usability tests are great when you would like to;
- Have a personal experience with your participants and want to ask follow-up questions in real time
- Participants could get stuck with a particular task, so the moderator/facilitator can help them get past that challenge and move the test forward. The challenge can also be technical, such as the participant struggling with the actual prototype
- Moderators/facilitators can also help encourage participants to speak their thoughts out loud and prompt them to share what they are thinking
- Moderators / Facilitators can also observe body language and the tone in their voice to pick up any subtleties of their behaviour
Ozow’s steps to set up a moderated test;
- Define what needs to be tested
- Build out the prototype / simulation of the solution
- Put together a testing plan
- Testing method
- Schedule and Venue
- Participant incentives
- Expected Outcomes
- Recruit approx. 5 participants
- Initiate the test
- Document and play back the results
How do you go about unmoderated testing?
Unmoderated usability tests are great when you would like to:
- Gather feedback quickly (Especially if you are in a Lean/Agile environment).
- Removes the nerves from the participant as a moderator/facilitator can impact how the participant answers.
- Empower the participant to be able to complete the tasks in their own time and at their own pace.
- In some circumstances, the participant will be in the ideal environment when completing the task. Being in a testing environment doesn’t always simulate real world scenarios.
- You and the team can complete other tasks while gathering feedback from participants
Ozow Steps to set up an unmoderated test;
- Define what needs to be tested
- Build out the prototype/simulation of the solution
- The Ozow UX team uses a tool called Useberry to conduct various unmoderated tests
- Initiate the test by publishing it to a testing group (Ozow uses a moderated WhatsApp group which includes a user base who wants to help Ozow succeed)
- After a certain timeframe, gather the result and play back
Why is it important to test?
Testing your solution early and often guarantees that you will get closer to the right solution for your users. When ideating a solution, you are basing it on your own hypothesis, and you need to validate whether the user will not only able to use it but also adopt it, so it forms part of their ecosystem.
Usability testing also helps us to validate the product's usability, i.e., can the user achieve their goals in your solution? This will help reduce the relative costs of rework.
As the table shows, if you test early, you can identify usability issues early, and therefore fixing those issues in the design phase comes at a lower cost. Whereas fixing an issue in production could cost 30x more to fix and rework.
What are the best forms of testing, if any?
The best kind of testing is any testing really, as long as it is fit for purpose, and you get the required results. So, short answer, it depends on what you want to get out of the test.
The different usability testing methods
In-person testing (or studies)
This is where you conduct the test in-person and collect qualitative data.
RITE method
A personal favourite of mine. This follows the same plan as an in-person moderated test, but you iterate the prototype after 1 or 2 sessions. By the end of the round of testing, you would have done 2 –3 iterations of the solution.
Guerrilla testing
This is a great lean and agile method where you take your solution and you go out into the wild, like a coffee shop, and approach potential participants to help run a quick 15 – 30-minute test. You usually incentivise participants with a free cup of coffee for their time.
Card Sorting & Reverse Card Sorting
A useful technique when you want to validate information architecture and navigational structures. The main outcome of this test is to identify terms and labels that users can easily identify with so if they are looking for a piece of information, they know where to find it. You can do this method moderated or unmoderated.
Session recordings
As mentioned before, this method is where you set up a test and allow the participant to complete the tasks at their pace and time.
5 second test
A simple test where you present the user with a single screen solution and give them 5 seconds to absorb the content. After the 5 seconds are over, you ask if they can play back what they saw. This is greatly beneficial to see if the core message is being absorbed by users.
First-click test
A method where we monitor users and test what the first thing, they interact with is. This allows us to see what their first action will be when presented with the solution.
Observational studies
This is a unique in-person and unmoderated method where all you do is observe users in their environment. You are watching their behaviour and how they carry out their day-to-day tasks.
There are several other methods, but these are a few key ones every organisation should have in their testing tool kit.
What does effective testing allow you to do?
The main aim of testing is to help release a solution that is market fit and the user loves. Effective testing allows us to get closer to that. It also reduces the cost of a rework or fixing the issues post-deployment.
So, in short, it creates solutions that people find desirable and usable as well as saving the organisation money. Which is win, win.
Can you define what a Rapid Feedback Loop is?
Users now know that there are people behind the solutions making active decisions for their experience. So, in order for us to be market fit, we must tighten that feedback loop cycle. Talking to our customers often benefits us because every round we are one step closer to releasing a solution that is fit for purpose.
How do you find people to test with?
This is a pain point for a lot of UX professionals. Where to find those participants? There are several companies that have a service contacting participant that follow the target market you have in mind.
If you have a marketing database that participants consent to, then that is a good starting point. However, always ensure that your user base has agreed to be contacted for research and testing purposes.
How important is testing with the correct target market in mind?
Always ensure that your participants are as close to the target market as possible. This ensures that the data you are receiving is more accurate.
At Ozow, we use personas to help map out the target audience we want to test with.
Any closing thoughts? Anything else you’d like to add?
In short, Usability Testing (moderated/unmoderated) of all kinds is important and doing enough benefits the user and the organisation. This phase is often skipped but there are fast methods out there to ensure you rapidly get feedback to help inform your decisions.
