Interviewing for Evaluations: 3 Tips for Success

The Forward-looking Performance Review of the Green Climate Fund (GCF) by the Independent Evaluation Unit has a list of interview respondents across 20 pages. 

What is so different about evaluation interviews? They are not academic interviews. The interviews for my doctoral research on tiger conservation looked for patterns in what respondents knew, felt, or how they behaved. Evaluation interviews are far less personal and will typically cover the experience and insight related to a specific policy, project or program. But evaluation interviews are not casual conversations either. So, what are some of the things we can do as evaluators to get more out of our interviews? There is a lot of guidance out there. Here are my three tips.
 

1. Informed consent is everything.

As you and the respondent settle down, it is essential to use the opportunity to take control of the conversation and introduce the evaluation very briefly. Some respondents are familiar with the evaluation and its process, yet other respondents will have no idea what an evaluation does! This formal introduction assures the respondent that (a) interviewing is only one among many means of data collection, (b) their interview is one among many interviews, (c) you will not take every word at face value and will instead triangulate the data. During the introduction, it is useful to state briefly: (a) who the evaluators are and who they represent, (b) the purpose of the evaluation, (c) what this evaluation does not cover.
 
Next, it is critical to get informed consent. Our interviews can broach subjects that can compromise the respondent or someone else professionally (or even politically or physically). Therefore, it is crucial to get the respondent to become aware of the parameters of the interview, and then choose to provide consent to being interviewed.

I have found it very useful to take a minute at the beginning of every evaluation interview to a) identify the purpose/scope of the evaluation, b) clarify the measures of confidentiality (are you attributing to respondents? Are you using their names in a list?), and c) explicitly ask each respondent present for consent.

2. On asking questions

Go beyond the generic: Very often a ‘what’ or ‘how’ question will elicit a ‘why’ response. For instance, if you ask someone what should be done to make the readiness programme efficient, they might tell you why the GCF must do so. Ask someone about what the GCF should do to report on impact, and they will first tell you about how important it is to have an impact. This is how we often talk in real life, but it is not helpful. Specific insights are needed for evaluations.

Share the evaluation hypothesis. If I have developed a hypothesis for the evaluation, I find it helpful to ask a well-informed respondent to validate it (depending on the context of the interview, of course). However, it is essential not to use language that is judgmental, emotional or otherwise loaded.

Validate. Triangulate. Clarify. It is important to remember that interviews are typically only one method of data collection. You will need to validate the data. Many interviews will point you to additional data, reports, parts of reports, etc. It is essential to make a note of the report titles or other references.

Ask naïve questions. It is useful to ask questions even if you know the answer or if the answer may be obvious for the following reasons: a) the answer or some nuance may be surprising to you, b) although the answer may be obvious to you, it will be of help to other team members when they do their analysis later, and c) a bonus reason – often the answer seems obvious to us at the time of the interview, but at the time of writing a report you may need more specificity. For instance, during an interview, it might feel foolish to ask someone who lives on an island why climate change is a matter of concern for them. But when you return and start the analysis and writing, you need the voice of the respondent and how they articulate their concerns.

Ask indirect questions. If you are broaching a difficult topic, make the question less personal and more general. If I want to find out about your attitude towards traffic rules or politics, it might not be useful to ask, “do you ever jaywalk?”. Instead, I could ask you whether “people here in general” follow traffic rules or what they think of the government policy on climate change. When the focus is dispersed away from the self, respondents often provide more useful information.

3. Get their number!

Please make sure to get the names and designations of ALL respondents. Doing so minimizes risks of making egregious errors in names and designations. Email addresses and phone numbers are also useful to collect.

It is also useful to remember:
Don’t quote other interviewees. Even if the participants seem to agree with each other, or you expect that they know each other’s opinions, you cannot quote one to the other.
Don’t get too attached. It is important to state your questions in a dispassionate way, even if you feel very strongly about the subject. 
Remain curious. If we as evaluators put ourselves in the shoes of the respondents, we are often able to ask questions in a way that allows respondents to open up.
Read before the interview. With experience, you will spend less time on background reading, but it is essential for any interview. 
Report soon after the interview. Don’t rely on memory to transcribe. 

Interviewing for evaluations is more art than science. It is a skill that comes with experience. Embrace the professional journey and keep on learning! Stay tuned for my next post on conducting evaluation interviews during a pandemic!

Have questions or comments about this blog? Join the conversation on Twitter with the hashtag #IEUBlog

Disclaimer: The views expressed in blogs are the author's own and do not necessarily reflect the views of the Independent Evaluation Unit of the Green Climate Fund.