Often, companies, nonprofit groups, or politicians want to know what their customers or constituents think of the products/services/programs they offer. One of the methods commonly used for this purpose is a questionnaire. The results obtained can have an impact on changes in corporate image, decision making, and policy changes if the responses given are considered logical. Creating a questionnaire may seem easy and simple, but if it is not designed properly, the results can be distorted and unreliable.
Step
Part 1 of 3: Making Questions
Step 1. Decide what you want to know by distributing a questionnaire
Rethink what data you need and how that data will be processed later. That way, you can predict what questions are right on target, and how you will structure them. A good questionnaire should not be too long. So decide which goals are important and which are not.
Step 2. Plan questions that will help you get the information you need
Start with a wide range of questions, then narrow it down until each question is relevant to a goal. Keep the questions and answers simple, using as few words as possible. You can rely on open-ended questions, closed-ended questions, or a combination of the two.
Step 3. Use closed-ended questions to collect specific answers
Closed questions provide a certain range of choices for respondents. This question can be a yes or no question, true or false, or a question that asks the respondent to agree or deny a statement. Closed questions may look like open questions, but respondents have only a limited number of answers. Examples of closed questions can be seen below:
- "Have you shopped here before?"
- “If so, how often do you shop here?” (This question will provide several explicit answers from which the respondent can choose, for example “once a week” to “once a month”)
- “How satisfied are you with today's shopping experience?” (This question also has limited answers, for example “very satisfied” to “very dissatisfied”)
- “Would you recommend this shop to a friend?”
Step 4. Use open-ended questions to ask for feedback
Open-ended questions produce answers you might not expect, and don't provide a specific range of answers from which to choose. Open-ended questions provide an opportunity for respondents to share certain experiences or expectations. An open question might look like this:
- "What do you shop for?"
- "Where do you usually shop?"
- “Who recommended this shop to you?”
- Open-ended questions are perfect for clarifying previous answers, such as “Why do you feel that way?”
Step 5. Ask questions in a way that won't create confusion and bias
Avoid questions that lead the respondent because leading questions indicate that the questioner is looking for a specific answer and will limit the answers the respondent can comfortably give. You can change the question by providing possible answers or changing the wording so that it doesn't lead the respondent to answer in a certain way.
- You might consider asking the same question in a different way, reducing biased answers and giving you a better chance of getting to know the person's true opinion on the topic.
- The words used in the questions must be chosen in such a way that the respondent can understand them well. Confused respondents will produce data that is not right on target, therefore make sure the questions can be understood as well as possible. Avoid using double negative words, unnecessary clauses, or unclear subject-object relationships.
Part 2 of 3: Implementing the Questionnaire
Step 1. Think about how you will distribute the questionnaire
There are several options to consider. You can use online services to design a questionnaire. After that, send the questionnaire link via email. You can also use telephone or mail campaigns to contact respondents spontaneously. Or you can run the campaign in person, using a professional or volunteer to lead the survey.
Step 2. Design the questionnaire according to the method that will be used to distribute the questionnaire
There are advantages and disadvantages to each method, and each method has its limitations on what it can do. Rethink the distribution method that best fits the topic raised in the questionnaire and the data you want to collect. For example:
- Surveys distributed by computer, telephone, and mail can reach more respondents, while in-person surveys take a long time to run and limit who can participate (this may be useful).
- Surveys distributed by computer, in-person interviews, and by mail may use images, while those conducted by telephone may not.
- Respondents may be too shy to answer certain questions asked in person or over the phone. Decide whether you want to provide clarification for the question if the respondent does not understand something. Keep in mind that clarification can only be given at an in-person interview.
- To answer a survey distributed through a computer, the respondent must have access to a computer. If the questionnaire addresses a topic related to a personal issue, a computer survey may be best suited.
Step 3. Pay attention to the order of the questions you make
The form of the questionnaire is as important as the content of the questionnaire itself. You should try to structure the questions in a logical order or provide clear markings to indicate transitions from one section to another. Different types of questions can affect how respondents fill out the questionnaire.
- You may have to structure your questions so that if someone answers yes or no to a particular question, they can skip questions that don't apply to them. This can help keep the questionnaire focused and less time-consuming to complete.
- “Qualifiers” are questions that screen certain respondents, preventing them from answering questions that were not intended for them. Place the qualifier at the beginning of the questionnaire.
- If demographics is a major concern, ask demographic-related questions up front.
- Place questions that are personal or complex at the end of the questionnaire. Respondents will not feel burdened by this question and may be more open and honest.
Step 4. Decide whether you will offer incentives in exchange for completing the questionnaire
It's often easier to attract respondents if you offer something in return for their time. Online, mail, or telephone questionnaires may offer coupons after respondents have completed filling them out. Questionnaires conducted in person can offer souvenirs as an expression of gratitude for their participation. Questionnaires are also a good way to draw respondents' attention to mailing lists or membership offers that they might otherwise have missed without a questionnaire.
Step 5. Test the questionnaire before you start surveying others
Friends, employees and family members make good test responders. You can test the questionnaire while it is still in development, or test it after the draft is complete.
- Ask for feedback from trial respondents. They may point out any parts that are confusing or odd. The impression that respondents feel on the questionnaire is as important as the content of the questionnaire itself.
- After testing it, perform a statistical analysis to make sure you're gathering the data you need. If you do not get the information you want, make the necessary changes to the questionnaire. You may have to change the wording of some questions, add an introduction, or rearrange the order of the questions, increase or decrease the number of questions so that the questionnaire can get you to your goals.
Part 3 of 3: Revising the Questionnaire
Step 1. Examine the collected data to understand what the questionnaire is actually asking
Keep in mind that questionnaires are often part of a larger campaign. Questionnaires can be modified and used over and over again to target different demographics, ask different questions, or questions that are more aligned with the objectives. After reviewing the results, you may find that while the questionnaire you have created is reasonable, it is not appropriate for achieving your goals.
- For example, you may find that questions like “How often do you shop here?” limiting respondents only to those who shop directly at the store. If you want to find out how people buy a particular product, you may want to broaden your question to include those who shop online.
- Execution methods can also limit data. For example, respondents who answered surveys conducted via the internet were most likely people who had greater computer knowledge than the average person.
Step 2. Revise the question further
Some questions may be fine in testing, but in the field it turns out not to work as expected. Questions should sound reasonable for the particular demographic you are targeting. Think again about whether the respondent can really understand what is being asked, or if your survey is so standard that the respondent doesn't really answer.
For example, questions like, “Why are you shopping here?” may have answers that are too broad so as to mislead respondents. If you want to find out if store decorations affect shopping habits, you can ask respondents to explain what they think about store decorations, brands, etc
Step 3. Review open-ended questions
Check whether the open-ended questions match the objectives to be achieved. It could be that the question is too open so that the respondent will wander everywhere. Or, the question may not be open enough so that the data obtained is not very useful. Rethink what the role of open-ended questions is in the questionnaire and modify it as needed.
As above, broad questions such as, “How do you feel when you shop here?” may not provide sufficient direction to the respondent. Instead, you could ask, “Would you recommend this shop to a friend? Why and why not?"
Step 4. Decide what to do with the lost data
Not all respondents will answer all questions. This could be a problem for you, but it might not be. Rethink which questions were skipped or answered incompletely, if any. This may be due to the order of the questions, the choice of words used in the question, or the subject of the question. If missing data is important, consider using different words to make the question more or less specific.
Step 5. Review what kind of feedback you received
Notice if you get any unusual data trends and decide if this reflects the real situation or is due to a weakness in the questionnaire. For example, closed-ended questions will limit the type of information respondents can provide. Your answers may be so limited that they make a strong opinion look the same as a weak one, or they may not provide a wide enough range of reasonable answer choices.
For example, if you ask a respondent to rate an experience, you should provide response options such as “very dissatisfied” and “very satisfied”, and a variety of options in between
Tips
- You can add a “don't know” answer for respondents who may not have an honest opinion on the question asked. This step will help avoid inaccurate answers.
- Consider carefully when selecting respondents. Even if you have designed the questionnaire well, the results will not be very useful if the data obtained are biased. For example, conducting an internet survey on how respondents use computers could yield very different results if the same survey was conducted over the telephone because respondents may be more familiar with computers.
- If possible, give something in return for the respondent to fill out the questionnaire, or tell the respondent how their answer will be used. Incentives like this can provide motivation for respondents.
