Everything I’ve learnt conducting user research sessions
There’s lots of different ways you can do it – focus groups, task based testing, interviews to name just a few. But. The main, and most important thing with user research is that you get your prototype/designs/products in front of people as often and as early as you can.
“Great design doesn’t live inside designers. It lives inside your users’ heads. You get inside your user’s heads by doing good user research: research that provides actionable and testable insights into users’ needs.”
David Travis, The 7 deadly sins of User Research
So heres my tips for running successful qualitative user research sessions, with a few minor disasters thrown in…
Give clear, specific directions to your office
If you’re conducting research in your office or a research lab provide crystal clear directions to your location. We’ve had problems with people finding our office before and have had to go and retrieve them. If you need to take a photo of the front of your office, or write a small essay with intricate details about how people can find you, do it. No detail is too small and it will mean you don’t lose valuable time. Heck, if you can provide a list of all the nearest car parks and how much they cost, do it. Because if a participant has spent the last half an hour trying to find a car park near your office, they’re not going to be best pleased with you.
Respect the working constraints of your socio-economic groups. Different working patterns, transport options and childcare availability can mean weekday research sessions just aren’t convenient. Work around the type of participant you’re looking to recruit and if that means an after 6pm research session then that’s what you need to do. Also, trying to recruit for the Friday before a bank holiday is pretty much impossible, so don’t bother. Check your calendar and avoid all the big holidays and sporting events.
Participants will be late sometimes
We’ve all been there, you arrive at your destination in plenty of time, go to park up and have to download an app to pay for your parking. But you’ve got no space left on your phone, so you have to delete a load of photos to make room, but then iCloud isn’t playing ball. When you actually get into the parking app, you have to create a password, but apparently you’ve already registered with that email address before… and now you’re late.
Participants will be late sometimes, and that’s ok. Do as much as you can in the time you have and if you’re lucky your participant may stay a little longer to make up the time, just make sure you politely ask permission to do this. If not, getting half way through your prototype or concept cards is still valuable. It’s not a disaster and your participant will already feel bad for being late, so make sure you’re extra friendly to make them comfortable. If you absolutely cannot conduct your session in the remaining time then speak to your recruiter and see what they can do to rectify the situation.
Occasionally, participants won’t turn up
Again, this happens. Sometimes people get called into work last minute and have to cancel. Sometime’s they don’t let you know they’re not coming and turn their phone off. It’s not ideal but just make sure when you’re recruiting you account for this. We’re lucky in that the recruitment agencies we work with try to back fill for us if this happens. We normally test with around 5 participants (Why you only need to test with 5 people by Nielson Norman Group) so we tend to recruit for 6 or 7. We’ve also found different socio-economic groups have a varying drop out rates, so do your research, talk to your recruiter and plan in contingency.
Recruiters will let you down sometimes
We worked on a project where we specifically needed to recruit females for a group research session. Myself and Ryan were all ready to go, 3 out of the 6 participants had arrived, and then 2 males turned up… We had 2 choices, run the session with them or ask them to leave. After a little (quiet) debate in the kitchen we decided to ask them to leave. However as myself and Ryan were running the session, doing this could be awkward and at worst inhibit the other participants. So Charlie our project director took care of it. He politely explained the situation to them, apologised profusely for the mistake and we were able to run our group session to great success. Mix ups happen, but make sure you consider how you handle them carefully. Facilitators need to play the good cop to make sure everyone is comfortable enough to participate in the session.
Don’t be scared of what you might find
Sounds a little dramatic, but your job as a facilitator isn’t to convince participants of the product you’re working on, it’s to get to the truth. If that truth is that they wouldn’t use this product then great, you’ve discovered that now as apposed to when it’s been built.
Don’t use the word ‘test’
‘Test’ implies being right or wrong and with user research there aren’t any right or wrong answers, just honest opinions. So don’t invite your participants to a ‘User Testing Session’, it’s a ‘User Research session’. Also, make sure your recruiter communicates the nature of the session clearly to all participants. Turning up for a group session only to find it’s a one-to-one interview could leave people feeling a little put out.
You didn’t design the prototype, even if you did
If a user think’s you’ve produced the prototype you’re showing them, they’ll be worried about causing offence and you won’t get their honest opinion. So do your house keeping at the start of the session. Explain it’s not a test and there are no right or wrong answers. Honest opinions are greatly appreciated and you didn’t work on the prototype so you’re not going to be offended by anything they say.
NDA’s can take a while to read
If you’re running a 60 minute session and 10 minutes is taken up by a participant reading and signing your forms, that’s valuable time you’ve now lost. Do your prep and get these documents emailed over well before the session and save yourself one sixth of your valuable research time.
Wear a watch
Striking a balance between cultivating valuable discussion and getting through your whole prototype in 60 minutes can be difficult. If you’ve got a participant talking about one of your features that’s great. You absolutely don’t want to cut them off because that may deter them from opening up again. But equally, you’ve got lot’s of other features to get through and you have just spent 15 minutes of a 60 minute session discussing one. So keep track of time and if you need to, prioritise. If it’s looking like you may not get through everything — no problem. Know what you absolutely must cover and make sure you get that done. If necessary, thank them for their contribution on that idea and explain you’re conscious you need to get their opinion on a few more things.
Get someone else to take notes
If you’re busy writing down what someone is saying, you’re not looking at them and if you’re not looking at them you can’t gage how they’re feeling about your prototype. Get someone else to take notes for you so you can focus on having a valuable conversation with your participant. We’ve found post-it notes and sharpies work best for note taking. Marking notes with happy, indifferent and sad faces can speed up going back through findings later on.
Ground your questioning in previous behaviour
Time after time we’ve had participant’s tell us ‘no I don’t let apps use my location, absolutely not, it’s stalkerish’. In some cases this is absolutely true but after we’ve named a couple of apps that routinely use location — Facebook, Twitter, Instagram, we get ‘Oh yeah I use those, that’s ok…’. Sometimes asking an abstract question will prompt users to tell you what they think they would do, not what they actually do. So start your questions with ‘last time you used…’ and ‘have you ever…’ to validate some responses. And if this does happen, don’t call a participant out for contradicting themselves.
Follow Louis Theroux’s lead and embrace the silence
It might feel uncomfortable at first, but that’s exactly the point. If a participant is feeling a little awkward with silence, they’re going to talk more to fill it and that’s great. As a facilitator your aim isn’t to fill every bit of silence with a question, it’s to get a participant to open up.
Sometimes technology will fail you, but don’t panic
Again, expect this to happen. Wi-fi can drop out, your tv may insist on turning itself off in the middle of the session (this has happened and we couldn’t find the remote). But don’t panic, getting flustered isn’t going to help and participants are nice people. It might feel like 5 hours has passed but in reality it’s 5 seconds and they’ll be happy to sit and wait for you to sort out the hiccup. Stay calm and get yourself back on track.
Some participants will be louder than others
In group situations you’ll have a mix of personalities, and sometimes the louder ones can dominate the conversation. Recognise when this is happening and correct it with your body language by turning away slightly and actively asking other participants in the session what they think. Subtlety is the key though, no wild chair swinging necessary.
You will get agreeable participants
It’s only natural. Telling a complete stranger that their prototype needs improving can be a daunting prospect, it’s much easier to nod and tell them it’s great. If you sense this is happening, then work a little harder, ask why it’s great. Ask them to think out loud and talk through what they’re doing whilst interacting with your prototype. You need more than nods and yes’ so keep asking why.
You will get disagreeable participants
Occasionally you may get participants that just want to moan about your prototype and things outside of your control. If you sense this happening don’t struggle through, conclude the session as politely as possible and get in touch with your recruiter.
Look at what participants are doing, not just what they’re saying
A participant might be struggling with a feature in your prototype but at the same time tell you it’s great because they think it’s their fault they haven’t understood it. Body language and facial cues will tell you how they’re really doing so observe as well as question.
Humour is good
Forget the mad men style focus groups and the white scientist jackets, what you want to aim for is something a lot more informal. Humour is a great way to get your participants feeling relaxed and at ease so you can have a conversation. Ask about their lives, their jobs, and build up a rapport with your participant. Remember, this isn’t an interrogation.
Try not to make participants cry
Whilst all research sessions require absolute sensitively to the subject matter being discussed, some such as personal finance, bereavement or health require even more. If you sense your participant is becoming upset or distressed, appreciate you’re dealing with a person and stop the session. Make it clear there is no need to continue, make a brew, step out of the room for 5 minutes. Whatever you need to do to give them some space.
Google hangouts are great
You don’t need a big fancy lab to run a user research session. Get yourself set up in a meeting room or quiet corner and fire up a google hangout so others can observe your session from another room. It works really well and all you have to do is share a link. Just remember to position your computer to best capture the audio and screen of the device you’re using or choose the share your screen option and combine with my next tip…
Quicktime is also great
This is great for recording how a participant interacts with your prototype. Plug your iPhone or iPad into your mac, fire up quicktime and select ‘New screen recording’. This will capture the audio and video from your prototype and display your device screen on your computer (this is when you share your screen on google hangouts). Using quicktime means you can see exactly what a participant tapped etc, as well as what they said. It also provides you with a great asset to present to your client when discussing your findings.
At most we’ve ran 8 sessions in a day. This can be pretty intense and leave you wanting to sit in a dark silent room, so we wouldn’t recommend it. Schedule your sessions with at least half an hour in between to decompress and to account for participants being late.
And finally, enjoy it
We’ve met some great people running user research and have yet to conduct a session where we haven’t learnt something. Like I said before, you don’t need a big fancy lab to produce some valuable, useful insights so there’s no excuse not to do it.
This was originally published on medium here