To think out loud, i.e. expressing thoughts verbally, is the fundamental concept of the usability test method “Thinking Aloud”. The method consists of requesting testers to voice their thought process verbally while they solve a task on a website. The spoken thoughts are then analyzed. In the field of usability, websites are commonly evaluated with this approach. But Think-Aloud can also be used to collect feedback on videos or ads. The testers are usually accompanied by an instructor who, if necessary, reminds them to speak out loud and may ask questions if necessary. It is also possible to conduct remote studies.
Thinking-Aloud provides a good understanding of the thoughts and values of potential users. Uncertainties can be eliminated by asking questions. The analysis is easy, as only the spoken word needs to be understood and summarized. Without the need for interpretation, user feedback can be collected quickly with this methodology.
The main disadvantage of this method is that the natural behavior of testers is strongly affected. Speaking slows down the interaction. The focus is often no longer on the most efficient execution of the task, but on the feedback that is given. The distraction caused by any questions from the study director is high. Frequently, the focus remains on individual topics, such as specific areas of a website or navigational elements. In a natural situation, the tester would quickly leave a problematic area. With Thinking Aloud, arguments are often repeated, and problems are reinforced, especially if the tester receives positive feedback from the study director (“social desirability”). At the same time, introverted or non-native speakers find it difficult to express themselves and voice criticism in a test situation. This limits the target group.
With the EYEVIDO Lab software, Thinking-Aloud studies are carried out by recording speech during the user session. The study instructor does not need a dictation machine and does not have to carry a transcript of the conversation. An automatic transcription of verbal comments is currently being developed and will further simplify study evaluation in the future.
Testers who visit the same website are collected on a screenshot. Any desired tester can be picked individually, and their feedback played back. Either from selecting the tester in the screenshot or alternatively through a screen casted single user session. This makes the analysis for specific pages very efficient.
In both cases, the tested website is stored as a screen recording, so that it is always clear to which part of the website the comments refer to.
Thinking Aloud influences the behavior of testers. Therefore, eye tracking data recorded in combination with Thinking Aloud cannot be used to analyze user behavior. However, eye-tracking data does provide interesting conclusions as to which sections verbal remarks refer to. The entirety of the eye-tracking data provides information about which parts were particularly in focus during Thinking Aloud and, for example, which parts were particularly heavily criticized.
A variant of Thinking Aloud is the Retrospective Thinking Aloud (RTA). Here, the tester initially interacts with a website as uninfluenced as possible without speaking. For example, they solve an everyday task under conditions that are as real as possible. In the meantime, the user data (screen, mouse, possibly eye-tracking, …) is recorded. Afterward, the tester, together with the study instructor, watch the video of the session and the tester is asked to comment retrospectively on their actions and thoughts. The procedure is time-consuming but very meaningful. Most of the disadvantages of Thinking Aloud are eliminated with this procedure.
Eye-tracking data recorded in the RTA helps the tester to remember the previous experience and to comment on specific areas.
