Without effective questions and appropriate answer choices you won’t get accurate or valid data from your survey. Below is a list of common pitfalls when writing survey questions. Avoiding these 7 pitfalls will increase the likelihood that your survey provides the data you desire.
Avoid Pitfalls when Writing Questions
1. Avoid Leading Questions
Description:
Example of a leading question
Leading questions unknowingly guide the participant toward a certain stance- either positive or negative. Not only may this limit the data you obtain, but in some cases it may invalidate the results. If questions such as this are needed to glean specific information, try to balance out the survey by including a different stance in another question.
Resolution:
This question has been redesigned by replacing “enjoy” with “satisfied”. Note that the question is also framed to offer a bit more detail about the context and why the question is being asked. In addition, the answer choices are bipolar- both positive and negative options are available.
2. Avoid Double-Barrel Questions
Description:
Double- barrel questions ask two or more questions within the same prompt. This is a very common pitfall as surveyors often include details that are related in context within the same question or add a similar prompt in an attempt to clarify a question. But if multiple dimensions exist in one question (such as “enjoy” and “learn”), the data won’t provide the right information, because it isn’t clear which question the respondent 
is answering.
Resolution:
To resolve double-barrel questions, divide prompts or several dimensions into different questions or use a matrix. With a matrix, several related questions can be asked that have the same rating scale but participants answer each prompt independently.
3. Avoid Questions that Assume
Description:
Asking questions using acronyms, jargon, or technical terms assumes that the participants understand the questions enough to answer accurately. And in some cases, assumes the participants have knowledge about something they may not.
Resolution:
Remove any acronyms, jargons or technical terms. If you wish to include questions that some participants may not be able to answer, provide an opt-out option such as “I do not know”.
Avoid Pitfalls when Writing Answer Choices
4. Match the Question with Appropriate Answer Choices
Description:
If you are asking about satisfaction, be sure your answer choices relate to satisfaction and not to another measure such as quality.
Resolution:
If there is not congruence between the question and the answers, modify the answers so they correspond to the most accurate measure. Baseline offers a list of recommended scales for survey questions at this link: http://baselinesupport.campuslabs.com/entries/21990098
5. Use Mutually Exclusive Answer Choices
Description:
It is very common to overlap choices, especially with a number range. In the example to the right, respondents would need to select 2 different answers if they attend the theatre 3 times per year. Be sure each answer choice is mutually exclusive of the others.
Resolution:
To avoid overlapping answer choices, review each answer separately and compare it to the answer before and after it. If there is overlap, modify the ranges as needed.
6. Use Collectively Exhaustive Answer Choices
Description:
To aid filtering of final data, it is helpful to list possible options instead of using an open textbox. However, be sure that all possible choices are available in the list of answer choices.
Resolution
Try to be as exhaustive as possible. List as many options as are feasible and then include an open textbox to elicit less common answers.
7. Use Opt-outs Sparingly
Opt-outs are often options such as “I do not know”, “I’m not sure”, “I can’t recall”, or NA. At times, these options can offer more exhaustive data and are helpful for the respondents. However, too many of these options may limit the breadth and value of the data collected.
References:
The Research Bunker Blog. (April 27, 2010) Mutually Exclusive & Collectively Exhaustive Survey Tips by George K. https://rmsbunkerblog.wordpress.com/2010/04/27/mutually-exclusive-collectively-exhaustive-survey-tips-market-research-syracuse-survey/
Survey Monkey. Help Center Design Tutorials.
Campus Labs. Survey Design Best Practices Webinar.