In the previous article (in german) of the series we talked about how to get to a user-friendly information architecture for native apps. In this article, we now want to focus on how you can ensure the quality of the wireframes in further steps. For a product that is continually developed over a longer period, iteratively executed usability tests help to ensure a high quality user experience. Nothing is more revealing than observing a real user struggling with the application. The problems identified can then be eliminated in the next iteration. In the following article, I will discuss the ways in which usability testing can be performed, under what conditions you should choose each method, and some advantages and disadvantages of each method. 


1. Usability Tests in the Lab

Usability testing in the laboratory must be planned a few weeks in advance. According to the chosen method, you have to find participants matching the respective target group. You have to create a schedule and make appointments. It must be clearly defined what should be tested and for what reason. Out of that the so-called test design is developed. Included in the test design are not only the test tasks which the respondent needs to perform with the app, but also materials such as a welcome letter or the consent explanation for the recording of interactions with the app. The recordings are used to document the test and can also be used very well for presentation of results. To allow such recordings the laboratory must be equipped with appropriate software and hardware. A laboratory should consist of at least two rooms. A test room where the tests are performed and an observation room where the observers watch the test either by a one-way mirror between the test room or by viewing recordings of the room transmitted to the computer (see Figure 1)

Figure 1 – Usability Laboratory Observation- (left) and Testing room (right); 1) four observers, 2) the test participant, 3) the moderator.
Tests in the laboratory should be considered if there is enough time and budget available for the project and the ease of use of the product is essential. 

External Usability Tests

To use an external company to conduct usability tests is usually very expensive. However, in this case you don’t need to worry about the test design or the acquiring of subjects. The results are professional and also the evaluation can be carried out by these companies. If you want, for example, only a specific function or an element in the interface to be tested you could also consider conducting remote usability tests. They are a bit cheaper and require less preparation time. For example, a good service for such tests is 

In-House Usability Tests

If you decide to conduct in-house usability tests this can become very costly. A fully equipped traditional laboratory must be equipped with professional software, such as Morae, cameras and much more equipment (see Figure 1). Nowadays you can get cheaper software like “Magitest” or “Lookback“, with which you get good results. This software allows you to record the display, the touch gestures, the participants face and sound with the smartphone’s internal hardware. With such a recorded user interaction, problems can be identified and documented. To acquire suitable participants for testing and to create test designs still remains time consuming. 

Advantages and Disadvantages

A disadvantage of usability tests in the lab is that the environment for the volunteers is always artificial. They know that they are under observation and every step and every statement is under the microscope. Accordingly, the behavior of people can differ somewhat from the behavior in real situations during the test. This sometimes leads to inaccurate results. The advantage of user tests in the laboratory against guerrilla user testing in the field are firstly the planning of tests and secondly, the control of the procedure. The subjects can be chosen precisely to the target group criteria. They have registered for the test, take their time to participate and are prepared for the situation. You usually provide the participants a financial allowance, so they are motivated accordingly. In general, the participants perform the required tasks conscientiously. Another advantage of the laboratory tests is that they can be a bit more extensive and can acceptably take more of the testers’ time. 


For the testing of desktop software, the software “Silverback” is very good. This software also allows you to record the screen, the mouse movements, the participants face and the sound. It is also possible to mark individual tasks or highlights in the test and play them later separated from each other. One limitation of this software is that in only runs on Mac OS operating system. 

2. Usability Tests in the field

Native apps are often developed primarily for mobile applications. To ensure a realistic test scenario, the decision of conducting the test in the field, outside the artificial environment of a laboratory, can be made. Therefore the documentation software previously mentioned can be used here too (see Figure 2).

Figure 2 – Participant uses App in the field with running documentation software in the background (left); person analyses recorded video from session (right).
The tests in the field are planned like laboratory tests. Participants are invited for a certain period. The test procedure and, if necessary, the geographical route here are carefully planned. This method is particularly suitable for apps which are mainly used in mobile environments. 

Advantages and Disadvantages

A clear advantage of this method is the realistic environment in which the tests are conducted. In this environment external events, such as light, noise, construction zones etc, influence the test results, as this would also be given situations in the everyday use of the app. The disadvantage of this method is the difficulty to control the situation. With external circumstances, each test can be different even if the main conditions, for example, using the same route, remain the same. This makes the evaluation and comparison of results more difficult. It is also difficult for the moderator to observe the usage of the app in the field. You must be able to rely on the software with which the tests are recorded so that you can repeat any unclear situations again afterwards. 

3. Guerilla Usability Tests in the field

If the project budget is tight and time is limited, guerrilla usability tests in the field are very good to quickly get a rough idea of how people deal with the interface of the app. This method is also useful for comparing variations.  Guerilla user tests in the field must also be planned thoroughly like usability tests in the lab. You must decide where to test. There are particular public places where people have to wait or have some time to relax. The tests should not take too long in order to not consume too much of their time. Usually the longer the test, the smaller their motivation.  Before testing in the lab, the participants get a consent form to sign, so that they and their interactions with the interface can be recorded. This is usually not a problem because the people are deliberately engaging in the test situation and are prepared. In the field however, where people decide to participate  to a test request spontaneously, this is more difficult, because for many it’s a reason to reject the request. Therefore, it is an advantage if two persons are present in the test. The moderator interacts with the subject, instructs them what to do and asks questions. The other person observes and makes notes of the events, but stays in the background.

Advantages and Disadvantages

The advantage of this approach is that you get results very quickly and cheaply. A disadvantage is that the test has to be short and therefore your app cannot be tested extensively. One must usually focus on critical features and interactions. Another disadvantage is that the test subjects that you find “on the street” do not necessarily fit your target group. Often, people are stressed and disinterested but nevertheless agree to participate to avoid being perceived as rude, by refusing your request. To create an incentive, and to increase motivation, you can (depending on the situation), offer people a small incentive – for example a coffee or something sweet. 

4. Conclusion

Because of the perceived additional time, effort and cost, small teams often shy away from performing usability tests. As I outlined above, there are some methods that don’t need much time or investment, which can lead to big improvements of usability of software applications. One should consider usability engineering not just as a cost factor but as long term investment. A user-friendly application will pay off through customer satisfaction and a measurable increase of success.  What do you think? I’m interested to hear your thoughts and experiences!

(This blog post is also available in German)

0 Kommentare

Einen Kommentar abschicken

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.