56 lines
3.6 KiB
Markdown
56 lines
3.6 KiB
Markdown
|
# Pareto Principle Checklist for Question Design
|
|||
|
|
|
|||
|
|
### Problems to watch out for before testing the questionnaire.
|
|||
|
|
|
|||
|
|
1. PROBLEMS WITH READING: Determine whether it is difficult for the interviewers to read the question uniformly to all respondents.
|
|||
|
|
1a – WHAT TO READ: Interviewers may have difficulty determining what parts of the question are to be read.
|
|||
|
|
1b – MISSING INFORMATION: Information the interviewer needs to administer the question is not contained in the question.
|
|||
|
|
1c – HOW TO READ: Question is not fully scripted and therefore difficult to read.
|
|||
|
|
|
|||
|
|
2. PROBLEMS WITH INSTRUCTIONS: Look for problems with any introductions, instructions, or explanations from the respondent’s point of view.
|
|||
|
|
2a – CONFLICTING OR INACCURATE INSTRUCTIONS, introductions, or explanations.
|
|||
|
|
2b – COMPLICATED INSTRUCTIONS, introductions, or explanations.
|
|||
|
|
|
|||
|
|
3. PROBLEMS WITH ITEM CLARITY: Identify problems related to communicating the intent or meaning of the question to the respondent.
|
|||
|
|
3a – WORDING: The question is lengthy, awkward, ungrammatical, or contains complicated syntax.
|
|||
|
|
3b – TECHNICAL TERMS are undefined, unclear, or complex.
|
|||
|
|
3c – VAGUE: The question is vague because there are multiple ways in which to interpret it or to determine what is to be included and excluded.
|
|||
|
|
3d – REFERENCE PERIODS are missing, not well specified, or are in conflict.
|
|||
|
|
|
|||
|
|
4. PROBLEMS WITH ASSUMPTIONS: Determine whether there are problems with assumptions made or the underlying logic.
|
|||
|
|
4a – INAPPROPRIATE ASSUMPTIONS are made about the respondent or his/her living situation.
|
|||
|
|
4b – ASSUMES CONSTANT behavior: The question inappropriately assumes a constant pattern of behavior or experience for situations that in fact vary.
|
|||
|
|
4c – DOUBLE-BARRELED question that contains multiple implicit questions.
|
|||
|
|
|
|||
|
|
5. PROBLEMS WITH KNOWLEDGE/MEMORY: Check whether respondents are likely to not know
|
|||
|
|
or have trouble remembering information.
|
|||
|
|
5a – KNOWLEDGE: The respondent is unlikely to know the answer.
|
|||
|
|
5b – An ATTITUDE that is asked about may not exist.
|
|||
|
|
5c – RECALL failure.
|
|||
|
|
5d – COMPUTATION or calculation problem.
|
|||
|
|
|
|||
|
|
6. PROBLEMS WITH SENSITIVITY/BIAS: Assess questions for sensitive nature or wording, and for bias.
|
|||
|
|
6a – SENSITIVE CONTENT: The question is on a topic that people will generally be uncomfortable talking about.
|
|||
|
|
6b – A SOCIALLY ACCEPTABLE response is implied.
|
|||
|
|
|
|||
|
|
7. PROBLEMS WITH RESPONSE CATEGORIES: Assess the adequacy of the range of responses to be recorded.
|
|||
|
|
7a – OPEN-ENDED QUESTIONS that are inappropriate or difficult.
|
|||
|
|
7b – MISMATCH between question and answer categories.
|
|||
|
|
7c – TECHNICAL TERMS are undefined, unclear, or complex
|
|||
|
|
7d – VAGUE response categories.
|
|||
|
|
7e – OVERLAPPING response categories.
|
|||
|
|
7f – MISSING response categories.
|
|||
|
|
7g – ILLOGICAL ORDER of response categories.
|
|||
|
|
|
|||
|
|
### Things to pay attention to when testing the questionnaire.
|
|||
|
|
- Which items require respondents to think hard before they answer?
|
|||
|
|
- What cognitive processes do they adopt to answer difficult questions?
|
|||
|
|
- Which items seem to produce irritation, embarrassment, or confusion?
|
|||
|
|
- Are there any items that respondents consider comical?
|
|||
|
|
- Does the style of the question generate bias?
|
|||
|
|
- Are the answers we get what we really want for the purpose of the study?
|
|||
|
|
- Is there enough variability in the answers received?
|
|||
|
|
- Are there local expressions that should be incorporated into the items to avoid ambiguity?
|
|||
|
|
- Is the questionnaire too long?
|
|||
|
|
- In the eye of the respondent, have any other important issues been overlooked in the questionnaire?
|