“Body language says so much!” Says every user researcher ever. Sure, non-verbal cues can say a lot. But I’d argue that this information is not as important to your research as so many people say it is—particularly during remote studies, which are more popular than ever now during the pandemic.
First, let me define some scope for this argument. I’m talking about remote, moderated, usability studies and contextual interviews focused on the use of websites and desktop-based software. And, while they may not be sharing their webcam, they are sharing their screen while performing any tasks with the website or software program. So the context is the participant, in the comfort of their own home, on their own computer, doing the stuff they would normally do at home on their own computer.
There are plenty of other research activities where being on-site and in-person with participants is preferred and even necessary, but that’s another blog post for another time. I’m also not going to cover best practices or tips and tricks for remote research, since these are widely available. (Here’s just one I found.)
The research team at Factor has been conducting remote studies for several years now, and we developed and refined our practice long before the pandemic. And, for the vast majority of projects we undertake, the benefits of remote research outweigh the cons. As many others point out, these benefits include:
- More diverse participant pool, due to the ability to recruit participants from a variety of geographical locations.
- No travel costs.
- Flexible interview schedules (as opposed to requiring all interviews to take place between 9-5 in one time zone, at a certain location).
- Participants can be in the comfort of their own homes (or other spaces), on their own devices.
- The ability to see how different people configure their own devices, which varies considerably. This helps designers better understand how their products are being used (as opposed to bringing them to a lab where you might have a single test device that all participants use).
The cons include:
- Missing some contextual information—the sights, sounds, smells, etc. of the environment.
- Missing body language and other non-verbal cues.
- Awkward interruptions and other communication breakdowns that result from remote conferencing technology.
When I’m moderating a research session, one of my primary goals is to make the participant feel comfortable. I don’t know about you, but when someone is looking over my shoulder when I’m trying to do a task on the computer, I feel anxious and self-conscious and can’t really function the way I would otherwise. Study participants often feel that way, too. While expecting them to behave as if they aren’t being watched at all is unrealistic, I think we can make the experience less stressful.
To that end, we NEVER require participants to use their webcam. We always say that they are welcome to. Even with the option, hardly anyone chooses to make themselves visible on their webcam. So, one argument is, quite simply, if they don’t want to share why make them?
But why don’t participants want to share their webcam? Here are some thoughts:
- Why share more than necessary? Participants signed up for the study because they felt that the incentive or honorarium being offered was worth their time and effort. Sure, they agreed to share their screen and their thoughts and behaviors for a little bit, but why would they want to share even more?
- We’re not friends, colleagues, or family. We’re not longing to see each other’s faces and reconnect. In fact, participants might see research as more of a business transaction. The point of the research is to examine a website or software tool, and learn about how people interact with it. So it makes sense that the focus should be on that.
- Communicating through a screen just isn’t the same as being in person. It can actually feel like a barrier, on top of an already contrived scenario meant to be as close to “natural” as possible. Video/voice delays interrupt the flow of the conversation. Participants may be concerned with their lighting, the angle of their camera, their background mess, or other issues not related to the study.
- Then there’s that pesky reflection of yourself in the corner. As one author points out in this article in Psychology Today,
Another dynamic that plagues video-based connection is the constant presence of one’s own image as they interact with others. Spontaneous and authentic communication is benefitted by the ability to be fully in the moment without the kind of acute self-awareness that comes with watching oneself during a conversation. For anyone with even a mild version of an inner critic, this can have a massive impact on how one is, or is not, in the present moment of a conversation in an authentic and available way. There’s a certain kind of cognitive dissonance here. We are on the call to connect with another but our ambient awareness of ourself redirects our attention.
Dodgen-Magee, D. “Why Video Chats Are Wearing Us Out.” Deviced, April 17, 2020. https://www.psychologytoday.com/blog/deviced/202004/why-video-chats-are-wearing-us-out?em.
This feels really important, and relates to that same sense of anxiety about being judged when someone is looking over your shoulder. As researchers, we should do whatever we can to reduce the participant’s feelings of self-consciousness. I can’t think of anything worse than to literally put a mirror in front of someone while they are being observed.
- Divided attention. I, personally, cannot watch what someone else is doing on screen while also watching their facial expressions via webcam, while actively listening to what they’re saying, and also keeping tabs on my script and the time. (And this is even with a second person taking notes for me!) In these remote sessions, I try to be as engaged as possible with what our participants are saying, and watching what they are doing on the screen. Similarly, when I’m collaborating with a colleague on a shared Google doc or on a call where someone is sharing their screen, a video feed on top of that just seems distracting and unnecessary. Why would we allow additional distraction in our research sessions?
- No major insights or differences in findings. I haven’t gleaned any more insight from sessions where people shared their webcams than from sessions where people didn’t. Even during in-person studies, I haven’t ever come away with any compelling insights based on people’s facial expressions or body language.
People expect that non-verbal cues will help them more accurately judge a participant’s sincerity. (i.e., they said they enjoyed using the app, but their facial expressions indicated otherwise.) Maybe I overestimate my ability to hear nuances in people’s voices that let on whether or not they are being sincere, but I think between listening really well and watching what people are doing, you can usually tell. You can usually hear the frustration, the indifference, and the delight. And there’s the classic, “Oh, this was really easy to use!” after watching someone struggle to complete a task. I didn’t need to see their face to capture that inconsistency, because their behavior told me everything I needed to know. There’s still plenty of information to work with.
- Lastly, bias. Try as we might, it’s just unavoidable. We can be honest with ourselves, and we can call each other out (one of the many reasons we always have at least two researchers in every session and debrief). But while it might seem like a con of remote studies, not being able to know what your participants look like can actually be a positive thing.
So, next time you consider pushing a participant toward turning on their webcam, (or worse, disallowing their participation without it), first ask yourself:
- Will sharing their video create a comfortable environment for the participant to share their thoughts freely?
- Will the webcam improve our understanding of what we’re studying?
