When it comes mobile app development when of the most overlooked aspects of the process is user testing, in fact this is often an afterthought and is only considered after the design and development of an app.
by Robbie Allen, xDesign
Fear not, xDesign a leading mobile app
development agency located in Edinburgh, UK are on hand
with our definitive guide to App user testing.
App User testing allows us to iterate on what we learn and
make sure each app appeals to users and meets their expectations before its
launch. Without user testing, there’s no way to know if you’re on the right
track until it’s too late…
Much like app development there is no one size fits all
approach to user testing.
This Definitive Guide to App User Testing has been created using fundamentals and strategies that will help you create your own user testing plan specific to your business case.
Regardless of whether you use this user resource to create
your own in-depth user testing or you reach out to an app development company
like us by the time you have finished reading this article we hope you
understand each of the core user testing steps, the best tools for the job, and
the special considerations you need to make from time to time.
Creating user testing goals
The first step when it comes to app user testing is setting
clearly defined goals. The goals you set will define everything from the
testing methods you implement to the environment you hold the testing through
to the tasks and questions you ask participants.
Before you go any further take some time now to set out what
your goals are.
Depending on what stage of development your app user testing
goals will change. Let’s look at a couple of examples;
Pre Development User Testing Goals
If you plan on conducting user testing before your app’s
development phase, your goals will be more focused on functional requirements and discovery.
That is, finding out what your target users need, what similar apps/products
they already use, and what pain points they need help with
During Development User Testing Goals
User testing goals during development tend to lean more
towards validation and evaluating. That is, finding out if users understand and
enjoy your app, and whether the features meet their needs. For this, leverage
early versions of your app, prototypes, and mockups.
Post Launch User Testing Goals
our goals in the post-launch phase will focus on measurement
and further validation. That is, how your app is performing, opportunities for
optimization, and whether your user experience appeals to users. For this, give
your participants your actual app and try A/B testing to narrow in on what
resonates best with your users
Setting Objectives for each User Test
Once you establish your user testing goals, you need to go a
step further and determine clear objectives for individual tests.
Try to hone in on the most important insights you want to
gather rather than trying to test everything at once. Remember, you’re not
looking for generic feedback, you’re looking to uncover specific problems.
Don’t Do it All
When you try to discover too much at once, you run the risk
of conducting a long and unwieldy test that fatigues your testers. It’s also
easier to confuse your results and next steps when you try to tackle too much.
In Person vs Remote User Testing for your App
User testing can occur either in person or remotely. For
both types of testing, you will want to observe participants using your product
and uncover any usability issues. Where in-person and remote testing differ is
in their pros and cons.
In Person Pros
- With in-person testing, you get the opportunity to
establish a real connection with your participants and see their reactions from
- It’s much easier to read someone’s body language in
person than it is over a video conference.
- In-person testing also makes it easier to use a
participant’s body language cues to gauge when to ask questions for
clarification or dig for more feedback.
In Person Cons
- These in-person interactions come with a cost, however.
- They often take more time because they involve
traveling to a location, setting up a testing environment, and being present
for each participant session.
- You may also incur a cost if you need to rent physical
space for the testing.
Remote Testing Pros
- Remote testing works well for unmoderated sessions as
well as when budgets are more limited.
- This type of testing is also ideal for tighter
timeframes as there’s no need for travel or testing facilities, and because
there is less pressure on test moderators.
Remote Testing Cons
- The major downsides, however, are that you can’t
interpret body language.
- it is more difficult to follow up or do a deeper dive
into your participants’ feedback.
Selecting the RIGHT Recruits
If you recruit the wrong participants, or if you don’t get
enough participants, your user testing will not generate the results you need.
To find the right participants for your user test, you first need to identify
your app’s target audience. We often dive into recruiting participants during
the discovery and user research process here at xDesign.
Behaviour over Demographics
It’s more important to recruit participants based on their
behavior rather than demographics. In choosing your participants, select for
behavior more than age, sex, income, location, education, or marital status.
The best way to do so is by finding people who are already active in similar
This doesn’t mean you should dismiss demographics
How many User Testing Recruits do you need?
The number of participants you need for your test also
depends on your testing goals. You should recruit 5-7 participants to test
usability problems. While the Nielsen Norman Group asserts that you only need five participants to uncover 85% of a product’s
issues, we prefer to schedule seven participants per
test. The two extra participants account for no-shows, which do happen!
Test your User Test
Make sure you do a dry run before your first real session.
Doing so will give you the opportunity to fix any red flags or issues that come
up. We find running a pilot test with a co-worker helps iron out any kinks in
the test script and tasks.
It gives you a chance to evaluate the flow of the test, add
or remove any tasks, and predict any potential roadblocks. You’ll feel much
more prepared and confident for your sessions if you’ve already remedied
Think of it like usability testing your usability test!
Testing Time… Finally
- Start your testing sessions by welcoming the
participants and explaining their role (using your test script!).
- Make sure you note that you’re testing the app and not
them. This takes the pressure off your participants and may help them relax.
- You’ll also need to remind them to think out loud
throughout the session. Participants often forget to verbally express their
thoughts and may resort to sighing or looking confused. Encouraging them to
think out loud will help you gain those valuable insights you’re after.
- Only one person should be moderating each session, but
you can alternate moderators between sessions if necessary.
- After each session, ask the participant follow-up
questions and for any more insights or feedback they might want to offer up.
Now it’s time to watch any recordings and take note of what
you see and hear. Look at all of your observations together to try to determine
why particular things happened and develop theories about the causes of any
friction, frustration, and problems.
That’s a Wrap
Done well, user testing makes sure your app aligns with what
your users expect, need, and want. Without it, there’s no way of gauging
whether you’re on the right track throughout the design and development
We hope this definitive guide to app user testing gives you a good foundation from which you
can build your own successful user testing sessions.