A Leader’s Guide to the Three Essential Phases of Usability Testing

We speak a lot about user-centric design and while most people understand that it’s important, few know how to do it in practice. Many innovation leaders may not know that successful software creation requires three parts to its usability testing alone. You need to act on the right ideas with effective solutions that are made in a user-friendly way. Usability testing is the best (if not only) way to consistently design user-friendly products.

The key mistake that stakeholders and innovation leads make is that they assume a single usability test is enough. Rather than running these tests at the end of the development cycle, you should be using them to inform your actions throughout. Scaling usability tests in three phases will allow you to make changes while it is still cheap and affordable. As shown in the graphic below, don’t risk your ROI by running one test at the end.

Here you’ll learn all about the three phases of usability testing and how they can save you time and money.

Three is Better than One

Adoption and engagement are important goals for any software development project. They are the ultimate arbiters of success since they weigh heavily on the forecasted return on investment from software. Even the best ideas are futile if they are not user-friendly, so you need to focus on usability.

The default approach we see businesses take to usability testing is to apply just one round of testing on a coded product, often in the UAT phase. It might seem cheaper and faster, but it doesn’t allow you to identify usability issues and cost-effectively deal with changing requirements. A risk is that you uncover real usability issues, but they are too expensive to fix at this stage. It’s too expensive to make structural changes to things like features and navigation in the late stages of the software journey. You have to catch and iterate on these kinds of issues early in the process to protect your budget and ROI, and to deliver the good experiences users expect.

You can do this by running usability tests at three points throughout the software design lifecycle – using test observations under real-world scenarios to inform your work.

The Usability Testing Phases

The three phases of usability testing should start before you ever sit down with a user and ask them to review and work with your new applications.

It starts with mapping the journey of how users will interact with your design. From this, you’ll be able to understand the contexts, use cases, and real-world scenarios of how the software will be used in practice. These scenarios and use cases will become the tasks in your usability test and will help you identify the business metrics to measure success.

Reuse these scenarios across all of your usability tests. The only things that change are the fidelity of the application and the test moderator’s level of involvement. The usability tests should evolve across the three phases from the moderator being hands-on, to being hands-off.

Phase One – Low Fidelity Prototype, High Interaction from Test Moderator

Phase one of usability testing should be conducted with a low-fidelity prototype. It may look and feel undeveloped – but that’s the point. You’re reviewing the features, information architecture, navigation, language, and other foundational, and expensive-to-change, aspects of your app design. You want to make sure you get the foundation locked in before continuing, and while it is still cheap to make these changes. Changing the foundation during the engineering phases will be expensive and can kill the project.

This stage should be an open discussion between end-users and the tester(s). The aim is to catch any additional requirements that are needed to make the software user-friendly and achieve objectives while it is still cheap to make these types of changes.

At this point, you’re looking to address whether the software does what it’s meant to do under real-world circumstances. This can be measured through dialogue with the people who will ultimately use it.

The majority of the data will be qualitative. Expertise is needed not just in collecting data and designing and moderating tests, but also in analyzing the data so that you get the right conclusions. Qualitative data can be abundant and you’ll want experts who can handle it. Nielsen Norman Group gives an intro to one type of qualitative data analysis, Thematic Analysis.

Phase Two – Medium Fidelity Prototype, Lower Interaction from Test Moderator

After applying the learnings from phase one, In phase two, you’ll use a medium-fidelity prototype that incorporates the design language that will stylize the software. The model should look like the finished product. If your company has a good design system, using it will accelerate production speed for this phase.

Now, with more certainty that you have the right features, language, and navigation, you can start to gauge whether the design of the app hinders usability.

Test subjects should be asked to complete the same scenarios as in phase one, but with minimal input from the tester. Now is the time to evaluate user performance. Pay attention to how long it takes users to complete tasks and discreetly document error rates. These findings will help you to assess your design against predefined metrics, such as time on task, error rates, learnability, and the System Usability Scale (SUS).

Metrics are important during this stage. If there are conditions in the business case that need to be met to unlock value and ROI, now is the time to evaluate whether the conditions will be achieved. For example, if the time on task needs to be reduced by 10% to unlock value, now is the time to evaluate whether the design achieves the outcome.

During this phase, many business outcomes can be evaluated and predicted, to name a few: time on task reduction, error rate reduction increased pull-through rates, increased cross-selling, cognitive and physical effort reduction, increased collaboration and teamwork, improved decision making, objective and subjective usability, satisfaction, and more. What is measured and predicted at this stage depends on what outcomes the business needs from the innovation.

To get good numbers and estimates, the test moderator should be less involved than in the first phase. They should be ready to hand participants tasks under real-world scenarios and expect the user to use the prototype to work through it while taking an impartial approach to moderating. Now is the time to give participants hints rather than rescuing them if they struggle. Striking the balance between learning from learning users’ challenges, and teaching or demoing the app takes skill and practice. You will want someone with the proper experience to run these tests.

Phase Three – High Fidelity Prototype, Operational Pilot Scheme

Phase three of usability testing involves putting your software design into a real environment to see how it works in practice. By this point, the code has been built and the design has been polished, so you’re strictly checking to see that the product works at scale in an operational context.

This final stage of usability testing could take the form of a pilot scheme. Typically, it is only this last phase that innovations leads and stakeholders run a test, which is very risky. If feature sets or navigation issues are found to be wrong at this late phase, there are typically only a few choices: lose a big chunk of ROI to fix the issues, deliver a product that has a subpar user experience and try to fix it later, or kill the project. These negative outcomes can be avoided by scaling usability tests across three phases to catch issues while they are cheap to fix.

The Benefits of Using All Three Phases

The advantage of taking this three-phased approach to usability testing is that you shouldn’t be faced with late-stage issues. These cost more to fix and can limit the potential return that a software project can bring.

Instead, it’s better to conduct these tests as the product evolves and matures. You’ll be better placed to hit the right progress milestones and fix key issues as you go. It will end up uncovering major problems before sign-off and deployment.

These tests can also help when trying to predict the performance impact that new software will have on an organization. The insights generated to make it easier to adjust the way people are trained, spot long-term problems, and measure the real return on investment.

Putting User-Centered Design into Practice

Through these three stages of usability testing, you can ensure that the software you design works for the people who need it. Their opinions, concerns, and requirements should shape your work and inform the direction of development projects.

Even when following this process, it’s important to remember that checking usability forms is just part of the wider user-centered design lifecycle. Journey mapping and information architecture are also essential components that guard against the risk of inflated costs and extended timelines.

Daito helps clients to put those pieces together and they’re left with user-friendly software that delivers. Through an unwavering focus on end-users (guided by our support), you can achieve the same great results.