We who work in the usability industry constantly talk about the importance of speaking to the very people that will use the products we are creating. UX legends like Jakob Nielsen and Steve Krug, among others stress this again and again. However when looking more closely at the research available online, the advice on how to structure an interview is vague and helpful examples are far and few between.
When i started doing user interviews, I thought it was simple enough at first. Just get in a room with a user and start asking lots of questions! But too often I wan’t t happy with the information I had gathered. I often felt there was more to learn and chided myself for forgetting about questions or simply decided against asking them during the interview, just to realize that it would have been helpful information after. That’s when I started to realize how hard it was to conduct a really good interview. In addition to being prepared, you need to be a little tenacious and you need to be humble. There is a multitude of literature written on conducting good interviews (see below), mostly focusing on asking open ended questions, establishing rapport, making interviewees comfortable and being an active listener. These are all good advice, but I was looking for more hands-on, practical tips for people who needed a place to start. i would therefore like to share some of the things I’ve learned along the way.
The purpose of my interviews was to get to know the people I was designing for. I needed to find out:
- When did they use the product?
- What were their goals with it?
- What were their habits surrounding the product?
- What motivated them?
These questions were relevant for me because I became a part of a development team when the product was already in use, but under continuous development. In the absolute start of a project I would have focused more on how the product would be beneficial in the users’ lives. Learning about these goals, habits and motivations helps a great deal when making decisions during further development. In the case our product, an ordering system for the distributions of surveys on the web, we found that the users needed to easily save drafts of orders they were working on. Knowing if they would for instance share the drafts or just pass them on to the next person in line, is a habit that is useful to know to get the architecture right.
Before I got started on calling potential interviewees, I went through a couple of steps that would help me get the most of my interviews. Defining the purpose comes first, since that will make it easier to structure and plan the session. You need to figure out what kind of information you are after. For a further development as i was doing, getting feedback on how they were currently using the product. Were they using it as intended? If not, why? Can we improve that somehow? My purpose was to figure out what these peoples roles were and what they wanted to achieve during the day. Were they using out product extensively, or only once a week? Maybe even just once a month. This is important because if I know how often the product is used, i know if they would be novice or expert users. If research shows that the product is only being used once a month, a steep learning curve is really not a god idea. (Now, you can off course use tools like Google Analytics to find out frequency, but it won’t tell you how to improve your product.) The reason for the interviews is to find the “what”. What are their needs right now? What are they trying to accomplish? The “how” comes later, as in how they make sense of the product, and that is when usability testing on prototypes comes in.
One of the things i learned pretty quickly was that talking to different people (obviously) results in different kinds of answers, which means that it is hard to plan an interview in detail. In Mike B. Fishers article you get a glimpse of some of the kinds of users you will encounter. I therefore prefer to go with the flow, I let the interviewee lead the conversation as much as possible. I let them talk about what they think is important about how they do their work and how they use the product. You as an interviewer need to decide when relevancy turns to ranting and carefully ask them to focus on the subject. In the Reports Handbook, Nicole Kraft recommends writing down questions in preparation for an intverview. Going one step further I always print my questions out and bring them to my interviews. Even though every interview is unique and some questions will be irrelevant and you will make up new ones, keeping a sheet of guiding questions is a comfort for me. I can quickly see if I’ve covered everything I want to cover on a topic before moving on. It also makes it easier for me to allow the interview to stray a bit when an interviewee touches on something interesting, I have my printed questions to help me get back on track. I also use my script as an effective way to put me in the right mindset before an interview. I might have been working on something else earlier and reading through the questions is a good way to help the context switch.
During the interview
I let my interviewees wander a bit when they speak, you want to keep them talking as it is their brain you are picking. I also do this on the expectation that they will talk about what is on their mind e.g. most important to them. On rare occasions they go horribly off topic, but that what your sheet of questions is for. As long as they are saying things that you can use, let them talk. Ask follow up questions when they touch on something interesting. Better to ask too many than too few.
When doing usability testing I am usually following my script more closely because I am testing specific things. During an interview I am trying to learn a little about what motivates them and what their goals are during a day. The best way to learn is to encourage them to tell stories about their day. This might take a while so make sure that you have plenty of time. It can be very frustrating to end a session prematurely.
Sometimes during an interview you might notice that they did not answer the question you asked. There might be several reasons for this;
- They misunderstood your question. If this happens try to rephrase your question. It is likely that they have different reference points than you and simply enterperated what you just said in their own way.
- Their mind is somewhere else. Maybe you were too quick to change subject. I do this all the time when I’m carried away. When they fall quiet after answering a question that requires them to ponder a bit, wait and see if they just need a couple of seconds to collect their thoughts. If you are holding a long session, remember that people can only hold their concentration for so long. Maybe a short break is in place.
- Their meaning of a word is different from yours. One of our target groups are publishers and in discussion I learned that what our development team refer to as campaigns is called something completely different by the target group. Try to avoid confusion by using the same words as your interviewee.
Sometimes they do answer your question, but the answer can be open to interpretation. A classic example could be that an interviewee is referring to something as “it” or is just being general in speech. Mostly there is a common agreement about what “it” is, as in “can you pass me the book, it is on the table”. Sometimes though, you might have a different idea of what “it” is than the interviewee. Teach yourself to recognize when this is happening. I try to eliminate uncertainty by repeating the answer just given to me. In doing so, the interviewee can correct me if I have misunderstood anything and I won’t be drawing any hasty conclusions.
Be sure to tell them and point out when they give good feedback – even if it is things that you and your team already know about. This will make your interviewee more comfortable, but more importantly, they will build confidence that they are actually helping. A user interview is an unusual situation to be in, not many believe that they are of any help. This also applies to usability testing.
Ending the interview
At the end of the interview it is off course appropriate to thank the interviewee for their time and let them know that if they have questions later, they are welcome to contact you. I was however very reluctant to ask if I could contact them if I had any questions. And I usually had questions after almost every interview. After an interview I often kicked myself for not following up an obvious lead or asking a specific question when I had the chance. As I get more and more comfortable in the role of an interviewer I kick myself less often. But I have also begun to ask my interviewees if it is alright to get back to them if I find I have more questions. Actually asking them instead of just assuming that they are ok with it is not only polite, but by saying yes they are making a commitment and are more likely to actually answer your email if you do have follow up questions. I would however like to point out that my interviewees are for the moment users who expect to be called in for further usability tests later in the process, and they are not concerned with anonymity. If you assure your interviewees anonymity, ensure that they will keep that even if you get back to them later.
When I’m doing my interviews I usually record the whole thing. That frees up my mind so I can focus on what they are saying instead of scribbling notes in a horrible font, worrying if I will understand how it all fits together later. As the interview starts I ask them if it’s ok if I record it. There are two reasons for this, the first being that it’s polite, signalling that they have some control over the situation. The second reason is that not informing them could have legal implications should they find out. In About Face 3, Alan Cooper et. al. presents pros and cons for multiple types of recording. Pen and paper being flexible, but time consuming. Video being superior at catching everything going on during the interview, but takes a long time to edit and some people are reluctant to participate. Audio falls somewhere in between and I always use that. It’s easy and flexible -many times I have simply used my mobile phone for record keeping. I do however also always have pen and paper with me. This allows me to take notes of things that are not caught on tape, like facial expressions and pointing to a specific area of an application. Taking notes are also good for when there are two or more things I want to pursue. Naturally, I can only ask about one thing at the time, so I ask about one and make a note of the other for later. Don’t worry too much about facial expressions that won’t get caught on tape. In a way, they actually are. You are capable to hear on someone’s voice when they are distressed, happy or confused. And if you are worried that the signs are too subtle, you always have your pen.
Another benefit with recordings is that you can play them for developers. I have found that most of them like to see how people actually make sense of the things they are creating. It is a good reality check for anyone on a development team.
Conducting interviews is hard and they take time to learn. Just like anything else, you need practice to be good at it. Fortunatley though, some lessons can be be learned from others. So in short:
- Start out with the purpose. Know why you are doing the interviews in the first place. Do you need to get to know your users? Or do you need to identify who your users are?
- When you have your purpose, define the set of questions that will help you identify the problems you are facing and how you might be able to solve them. Make a list to carry with you to the interview. Even if you won’t follow the list completely, you will have a sure way to get back on track if you get side tracked.
- Recognize that people are different and you might need a slightly different approach depending on who you are talking to. Be prepared for interviews that might take longer than planned.
- Try to recognize situations where you might be interpreting the answers and have a counter strategy, like repeating the anwser in different words to see if you got it right.
- Acknowledge good feedback, remind your interviewees that they are indeed being helpful.
By being prepared, being persistent and being humble, you will come a long way.