BLOGCTA_TC_20130726_1_Testing_Ninja_936x110

How to Interview Users to Find Out What They Really Want

art of interviewing

It’s all well-and-good to suggest that developers, especially on Agile teams, work with end-users to find out what they want the software to do. It’s another thing to get useful information without the aid of mind-reading. Here’s how to interview a non-responsive informant and come away with what you need.

The art of interviewing is not just for television personalities. Interviewing stakeholders to find out what they want is an important part of requirements analysis. However, if you don’t have good interviewing skills, the answers you get will be less than satisfactory to the development team.

Timey-wimey, wibbly-wobbly answers lead to wibbly-wobbly software that doesn’t work correctly or doesn’t meet the user’s needs. And isn’t it amazing how clear those requirements suddenly are in retrospect, after the carcass of the newly developed software is spread out on the table?

Snark aside, nailing down software requirements is often a trial both for the interviewer and interviewee.

“It is very difficult to get the requirements out of users,” says Phillip Avelar, managing principal for Advanced Solutions, a Chicago, IL consultancy. “A new analyst has to be taught how to do it.”

“We’ve made advances over the years, but there are still a lot of projects we manage where the people are not well trained in interviewing,” says Avelar.

Then there are the problems with the interviewees, God love them. There are many reasons why people are difficult to interview and the analyst, Agile developer, or software consultant quickly runs into most of them.

The first problem is that users don’t think like developers. Users see a problem, sometimes nebulously, and they don’t automatically translate that into faster database access, or even “complex input procedures.” They just know what they’ve got isn’t working right and they want someone to fix it.

That’s a problem when the definition of “it” is so opaque. Often the problem isn’t even intelligibly associated with a computer problem. “Our order entry is too slow” may mean anything from “There’s a problem with the database updating” to “Suzy gets fumble-fingered after her three-margarita lunches.”

Checking This First

describe the imageOne standard tool for conducting interviews is a checklist. At best this is a list of what the interviewer wants to know. At worst, it’s a list of questions that the interviewer runs through as fast as she can without doing more than recording the answers. Question not on the checklist? Tough, it doesn’t get answered.

Used in this way, a checklist is like a matador’s cape; it conceals more than it reveals.

What a check list is supposed to be is a guide to conducting the interview, not a hard and fast list of all the questions you can answer. Used intelligently, checklists are a useful tool. Used the other way, they’re nothing but trouble. Keep in mind:

  • Your checklist is not a script. You can deviate from what’s on the list as circumstances warrant.
  • Check lists don’t remove the need for active listening. While you’re asking questions, be alert to the answers. Pay careful attention to the way the question is answered, just as you would in an interview without a check list.
  • Not only are you allowed to deviate from the checklist, you should do so at the drop of a hat – or an unexplained pause. Remember: It is entirely possible that the checklist is based on incorrect assumptions and is utterly wrong.

Your job as an interviewer is not to run through a check list; it is to acquire useful information that helps your team build software that the customer really needs.

Whether or not you’re using a checklist, follow-up questions are your friend. Ask them. Ask a lot of them.

Learn How to Really, Really Listen

The essence of successful interviewing is active, engaged, listening. Keep your antennae out as far as they will go and soak up every nuance and implication of what the interviewee is saying.

“Learn to listen very carefully,” advises Steven A. Lowe, CEO and founder of Innovation LLC, a software development consultancy in Chattanooga, TN. “The clue to what they really need is sometimes between the lines, sometimes in an aside. If you’re interviewing more than one person, pay close attention to how they talk to each other. That interaction can be very telling.”

Pay attention not only to what is being said, but to how it is said. Is the user enthusiastic? Frustrated? Uninvolved?

Listen for differences in emphasis. Pay attention to the parts of what is being said really move the listener and which parts leave her seemingly uninvolved.

Ask follow-up questions, lots of them. Make sure you understand everything the interviewee is telling you.

Get Specific

This is no place for wishy-washy, vague answers. Insist on specifics. Try to get the interviewee to nail down as tightly as possible what he is talking about. This can be difficult because the interviewee may start with only a general idea and count on the interviewer to help refine it. This means asking a lot of questions, a lot of specific questions.

Unlike the “You know what I mean” interviewee (see below) these people actually have a pretty good idea of what they want. They just need help refining and articulating it.

While you probe for information, avoid leading questions. That includes questions that present the interviewee with a short list of alternatives: “Do you want ice cream or hot dogs?” or “Do you want that information to pop up or be embedded in the screen?”

Instead, ask open-ended questions, questions that can’t be answered with a “Yes” or “No.” “How do you want this information to appear on the screen? On which screen in the process, or on more than one screen?”

Pay special attention when the interviewee seemingly wanders off in the weeds. If you’re talking about database changes to improve data input and the interviewee starts talking about Suzy’s three-margarita lunches, don’t automatically assume it’s an irrelevancy.

As much as possible, at this stage of the process avoid leading questions or questions that contained unexamined assumptions. The user may unthinkingly agree with you – after all you’re the expert – whether or not it is close to what is wanted.

Later interviews can be more directed as the requirements become clearer.

There are two questions you should always ask. The next-to-the-last question in every interview should be, “If there were no constraints on this project – time, money, approval, or anything else – how would you like this program to work and what would you like to see in it?”

The second question is, “Have you got anything you’d like to add or emphasize? Is there anything I missed?” If you’re lucky, at that point the interviewee will give the game away.

Mirror, Mirror

describe the imageAs you probably guessed, these interviews can get pretty discursive and cover a lot of ground in a not-too-well organized (read “scatterbrain”) manner.

You need to make sure you understand what you’re being told, or at least that you and your interviewee are on the same page.

Reflection is one of the most powerful tools in factual interviewing. This process consists of putting the interviewee’s statements into your own words and repeating them back to him. These statements are usually preceded by statements like, “So in other words…” or “I hear you saying….” Such as:

  • “Let me see if I got this right: You want the video to play on the website even if the user hasn’t clicked on ‘Play’?”
  • “I hear you saying that the problem with the current system is that you have to enter the data twice, in two different places? Is that right?”
  • “So you’re saying that you need that extra hit of speed in the bathroom at coffee break or else you’re likely to go postal on your co-workers?”
  • “You think what’s ruining your golf game is the alligators in the water hazard on the 9th hole?”

Never be judgmental or argumentative in reflecting information back. Your job now is to find out what the interviewee thinks, not to evaluate the answers.

The Sounds of Silence

One of the most useful techniques in interviewing is to give the interviewee time – a lot of time – to answer the question. Ask the question, then shut up and let the other person respond.

This can lead to a lot of uncomfortable silences in the conversation; that’s fine. The silence is as uncomfortable for the person you’re interviewing as it is for you. Eventually, he will be compelled to say something to break the silence.

Clamzo, Boys, Clamzo

Another problem is interviewees who clam up and say little or nothing. This may be a reflection of natural reticence or it may reflect other difficulties. In any case, it’s hard to get requirements out of people who won’t say anything.

It’s a help if you can understand why the person isn’t talking. It may be she’s just naturally uncommunicative. She might be terrified that you’re going to automate her job and she’ll be laid off. It may be she’s been threatened with death by the other people in accounting if she spills the beans about the massive fraud that’s draining money out of the company. Try to get a sense of what’s going on, even if you can’t get her to talk.

One way to handle this is to end the interview gracefully and move on. “Your best bet is to spend time with other people who can articulate,” says Damon Poole, chief Agilist at Eliassen Group, a Wakefield, MA software development consultant.

Another approach is to interview people in groups. “The typical interview is not something we use as much as a workshop,” says Avelar. “If you get the stakeholders all together it eliminates a lot of yes and no answers.”

The problem here is that it’s hard to build trust with the interviewees in a group interview the way you can in a one-on-one interview. Ideally you need a mix of both kinds of interview with everyone on your interview list.

Breaking the Jargon Barrier

Every profession, from software development to drug dealers (don’t confuse the two…), has its own language. Your interviewee undoubtedly has her own terminology specific to the industry, and takes for granted that professionals she works with understand the jargon.

The interviewee is not expected to know IT, but you are expected to know what the terms mean in the problem domain your software intends to solve. This usually means a little research on the part of the interviewer before the process starts. If you encounter an unfamiliar term or concept, ask the interviewee for an explanation. “So a ‘a revenue enhancement program’ is the reason you put the alligators in the water hazard on the ninth hole? How does alligators in the water hazard enhance the club’s revenue?” At which point the interviewee explains about ball hawks and you get to ask, “What’s a ball hawk?”

How Important is This Feature?

Everyone has a long list of “nice to haves” and “like to haves,” but those aren’t the same as the “must haves.” It’s important to identify the requirements the interviewee feels the system must meet to be satisfactory.

Of course some of those “must haves” on each interviewee’s list are likely to be difficult to implement, if not flat impossible. These shouldn’t make it into the final specification, but for now record them anyway. Winnowing down the requirements is a separate step in the process.

“You know what I mean?”

The only right answer to this question is, “No, I don’t know what you mean. Please explain it to me.” Unfortunately, this is a little too blunt for most interview situations. You need to back into the situation by asking specific, clarifying questions.

The real problem is that a lot of the time the interviewee doesn’t know what he means. He has only a general idea of what is wanted and expects the interviewer to read minds and translate that into what he really means. Since all mind reader translators work for the CIA, talk about a no-win situation!

The best solution is to ask more specific questions. But as much as possible avoid leading the interviewee.  The interviewee easily can interpret your questions as suggestions and end up with something she doesn’t really want.

Reduce it To Writing

Finally, you should reduce every interview to writing and show a summary to the interviewee. Often this is the last chance to make sure you and your subject understood each other before you start work on the project.

The written summary serves another purpose as well. In case of a disagreement later about what the subject wanted, you have a record of what was agreed to. That doesn’t mean all of the details make it into the final specification, of course, but it does greatly cut down on the finger pointing.

The Agile Approach

Agile software developers have another technique available to help them clarify user requirements: Build a prototype, call it a “JELL-O man” because it’s not even as firmly fixed as a strawman, and show it to the stakeholders to get their feedback.

“People may not realize that  shops that are really Agile can produce something in a week,” says Poole. What comes out of this process is a long way from a finished product, he points out, but the prototype provides a sketch of the intended result with enough detail, especially in the user interface and general performance, to give the stakeholders a chance to say yes, no, or “What I really want here is….”

Then, in typical Agile fashion, the developers build a more refined model and show it to the stakeholders again. Often these show-and-tell sessions are done for groups, so participants can bounce ideas off each other (and you don’t have to play mediator between competing user demands, but that’s another story).

Refining what the user wants can be a time consuming process, but it you stick with it you’ll have a much better idea of the true requirements.

Will better interviewing eliminate miscommunication in requirements development? No. But it can reduce the number of problems you encounter, and give you a better shot at getting requirements that actually meet the user’s real needs.

See also:


  • Barry Doctor

    I don’t understand why you would propose that developers interact with users. Typical developers lack that appropriate skill to ferret out the product requirements ask you so eloquently describe above. A better solution would be to have your Product Marketing Manager interview users to gather market problems, from both buyers and non-buyers. Then, have your product manager “solve” the problem by writing requirements for the product. Your developers become the “factory” of vet focused coders. They should not be interacting with the buyers.

  • Damon Poole

    Good question Barry. Experience has shown that spending lots of time on up-front requirements usually results in lots of unwanted or unused features. Agile says, rather than attempting to get “better requirements”, let’s face the fact that users just can’t predict what they will need by the time it is delivered to them. But they can provide feedback on something real. The point is not developers vs product managers. The idea is use product management to get the top priorities, implement that, then show it to the actual users and let the developers hear the feedback directly. Rinse, lather, repeat. It is one thing to try to document what users want, it is another thing entirely to hear it directly. These days, there is less and less ability to have long feedback loops. More and more organizations are delivering software on a weekly basis, and some are even delivering production quality releases 100 times per day! (Check out HubSpot) . Changing user expectations requires a change to how we discover and validate market demand.

  • Chris Miles

    Good article. This question is somewhat outside of the scope of your article, but I wondering how do you feel about tempering users’ expectations when showing them prototypes. Often a user/customer, if they are not technical, can think that the prototype is the final product or that completing the final product is very simple. This then causes frustration when there shouldn’t be.  
     
    I think prototyping to get sooner than later feedback should happen but it can also back fire with some people, even when it is explained that this is just to get feedback. What are your thoughts?

  • http://dpoole@eliassen.com Damon Poole

    Chris, good question. Just to clarify a bit, prototypes are one way to go, but pretty soon Agile teams show real working product every iteration (whether that is one week, two, or four), not just a prototype. In any case, there’s no way to avoid the potential backfiring issue. It is a bit of chicken and egg. That said, do what you can to set expectation, but the solution is to do it and keep doing it. After a few times, the users will get used to it and start seeing the value in having their input actually incorporated. I’d also like to point out that everything I’m saying is from the point of view of expecting a non-technical audience.

  • http://www.coderslexicon.com Martyr2

    These are some great points and I have found myself having to learn many of these the hard way. I want to say that listening is probably the hardest and most valuable skill out of the bunch. Our job should be to cut through the clutter of their words and get to the heart of what they need when it comes to the system we are creating. 
     
    Nice blog post. :)

  • Pingback: Interview: How to interview users to find out that they really want | Phil on Enterprise IT

  • Vijay

    Can anybody please tell who is interviewer and who is interviewee in the photo posted at the beginning of this article ???