The Dos and Don’ts of Effective Discussion Guide Writing, Part 1: Question Order

Writing and designing a good discussion guide is crucial for any research study. From question order to language used, there are several factors to consider in order for the study to yield useful and accurate results. Neglecting the importance of a well-designed discussion guide can be the difference between rich consumer feedback and confused, illogical, empty responses. Here are some tips to keep in mind throughout the research design process.

Let’s start with a mathematical activity.

What is the pattern or reasoning behind the following group of numbers: 7 2 3 9 8 8?

Actually, there isn’t a pattern. I chose whichever numbers I felt like including at the time. When you were trying to figure out the reasoning, did you get frustrated? When you realized there was no theory or thought behind the numbers, were you even more frustrated? I bet some of you wanted to stop reading when you couldn’t solve the puzzle. That’s exactly how a respondent feels when the ordering of questions or the flow of a discussion guide is illogical and unplanned. They can feel tricked by the moderator when it starts to seem like there isn’t a point to the study, as if their time spent taking the survey isn’t appreciated and valued.

Here Are Some Guidelines for Arranging Questions in a Logical and Unbiased Way:

Start broad and go narrow.

When structuring questions, start with category knowledge or more broad usage questions first. Move your way to the specific product or copy questions as the guide progresses. Not only is this a great way to introduce respondents to the topic, but it also ensures that the broad category knowledge is not influenced by the product-specific information. 

✔ Start by asking about toothpaste usage in general, including which brands they purchase, what factors are important in the purchasing decision, before asking specifics around a new coffee-flavored toothpaste. With this structure, you can determine if the taste or flavor of toothpaste was important to them before you actually mentioned the coffee flavor.

✖ Introduce the coffee-flavored toothpaste first and then, later, ask what factors are important in the purchasing decision. With this structure, the taste or flavor of toothpaste will most likely be mentioned much more frequently since respondents were primed with that information already.

Don’t skip around.

Arrange questions in an order that allows respondents to provide all of the information on one topic or objective before moving on to another. Your guide is the research story you are telling the respondent: there should be a beginning, middle, and end. You wouldn’t tell the beginning of one story, skip to the end of another, and then return to the middle of the original story. The same should apply to discussion guide writing.

✔ Ask respondents to remember the last time they purchased a car. In particular… What did they research? Where did they go to do research? Which cars were they considering? Did they test drive the car? And finally… Which car did they end up selecting and why? With this structure, respondents talk through the purchase funnel in a logical progression, thinking about each phase at a time.

✖ Ask respondents to first tell you what research they did prior to purchasing a car, then ask which car they purchased, followed by which cars they were originally considering, and then ending with where they went to conduct the research. In this case, respondents now have to think about the research portion of the car-buying process twice—at the very beginning and then again at the end.

Consider the relationship between questions.

If I first ask you to explain why you hate chocolate and then show you a new chocolate concept, you would be more likely to react to that negatively than if I asked you to tell me what you like and/or dislike about chocolate first. Paying attention to the mindset a respondent will be in due to the previous question he or she just answered is highly important when creating an unbiased guide.

✔ First, ask respondents what their initial impressions of Obama’s presidency are to capture unaided and unbiased reactions. Then, dive into what they specifically like and/or dislike about Obama’s presidency. With this question arrangement, you provide space for respondents to give their honest, top-of-mind feedback before diving into specifics.

✖ Start out by asking respondents to explain why Obama is the best president in U.S. history. Then, ask respondents for overall impressions of Obama’s presidency. In doing this, you are signaling to respondents that you want them to discuss why Obama is great; their answers to the question that asks about overall impressions would skew more positively as a result.

Something like question order in a discussion guide can seem simple, and at times, it certainly can be. But that doesn’t mean you should neglect structure design. By using a bit of strategic thinking upfront before execution, you greatly increase your chances of launching a study that will enable respondents to naturally glide through it in a logical way, which can also help you get one step closer to helpful feedback and actionable insights.

If you’d like more research tips, check out a previous blog post that will walk you through projective techniques for keeping online respondents engaged here.

Be sure to check out part two in our series about effective discussion guide writing by clicking here.