Craft Effective Survey Questions

printer Printer Friendly Version

Author John Towler, Ph.D.

How to create a better survey by writing effective questionnaires.

Everyone thinks it is easy to design a survey. After all, surveying is just a matter of asking questions, isn't it? Yes, but the questions have to be worded very carefully. They must be clear, precise, and not open to several interpretations. A well-crafted survey asks carefully chosen, insightful questions.

Choose the Right Question Format
The questions you choose will depend on what you want to find out and how complex your survey needs to be. You can use questions with rating scales, checklists, multiple choices, and weighted scoring, or questions that require a written-in answer. You will have to decide which type is most appropriate for your purposes.

Questions that can be answered with a yes or no may be suitable, but they are not as powerful as questions that must be answered using a scale. For example, compare these two questions you might ask about your Web site. The second question provides more information than the first.

1. Is this site useful? Choose either Yes or No.
2. I feel that this site is useful. Answer by selecting Strongly Agree, Agree, Undecided, Disagree, or Strongly Disagree.

Some questions best lend themselves to drop-down boxes or pull-down lists. For example, if you want to ask people which state they are from, you can ask them to type in their response, but it is much easier if you provide them with a drop-down list of all the states to select from.

If you want to ask a question that probes opinions about a limited number of things, you can use a check box. For example if you want opinions about three different products, you can ask a question such as "Which product do you like best? Choose one of the following: A. scented shaving cream B. unscented shaving cream C. shaving cream for sensitive skin."

The open-ended question will provide the most information. For example, a question such as "What suggestions do you have for improving this site?" will generate ideas that might never have occurred to you. Open-ended questions will make statistical analysis more challenging but may provide more insight into customer needs.

Make Your Survey as Short as Possible
A survey with too many questions can chase away people who might otherwise be inclined to participate. Long surveys encourage people to skip questions, read only part of the question, or opt out mid-survey. A well-worded question can often replace two or three poorly worded ones. Your audience will become restless and possibly resentful if questions appear redundant.

Pose Your Questions With Data Analysis in Mind
If you need to statistically analyze the responses, you must write questions that lend themselves to numerical treatment. That's why most surveys use multiple-choice questions or a scale of, say, one to 10 on which you can rate your answer.

In order to get responses that can be analyzed numerically, you must pose questions that don't allow a neutral response, because a neutral response isn't effective for initiating change or gauging opinion. If your survey includes a scale, select an even number to encourage respondents to choose an answer on one side of the divide. For example, on a scale of one to four, there is no middle number, so your participants must decide which side of the middle to choose instead of giving a neutral response.


Make Sure Your Questions Are Unambiguous
Make sure that your questions are written in plain, easy-to-understand English. A question such as "Does your boss engage you in interactive dialogue?" is useless because the meaning of "interactive dialogue" is subject to interpretation.

Don't use words such as "often," "usually," or "normally." These terms mean different things to different people. What is "normal" for one person may not be normal at all for another. A question such as "Do your customers normally complain very much?" is useless because it is ambiguous.

Be careful to avoid using double negatives, such as "Isn't it true that you dislike junk mail?" since the wording is confusing and may create distortions in the responses as people misread or misunderstand what the question is asking.

Avoid asking two questions in one, as this may elicit two different answers. This puts the respondent in a quandary about how to answer the questions truthfully. For example, a question such as "Do you like the colors red and blue?" presents a conflict for a person who likes one but not the other.

Test Your Questions on a Sample Audience
Testing your proposed survey on a sample audience will help you identify ambiguous questions and potential pitfalls for your data collection, so that you can make a course correction while you still have time. Ask your sample audience if they thought questions seemed redundant or if the survey seemed too long or confusing.

John Towler is a Psychologist and the founder of Creative Organizational Design. Please send comments about this article to jtowler@creativeorgdesign.com. For more information, please contact us.

Re-printable with permission.