Annie Pettir from Peanut Labs gave a brief but insightful presentation on data quality from the perspective of online survey design . As researchers we are constantly involved in administering surveys and we want to ensure that the responses we collect are as accurate as possible. This requires a keen eye for recognizing data quality .
Let's look at some quality control questions that we can easily use in our surveys.
Red herrings: These can be single choice, multiple choice, or rating scale questions. The key to applying this question is to insert some false data, for example putting false names on a list of brands. Annie stresses that when you are going to eliminate respondents you should only consider those who selected two or more false names. This technique also requires the researcher to confirm through a search engine that the false names are really false or have an extremely low incidence.
The high-incidence multiple choice is another type of question you might consider: a multiple choice with a long list of common or high-incidence behaviors. You should flag respondents who do not select several behaviors. You can actually calculate the average of the selected responses and then flag those who for some reason selected less than the average.
Low incidence multiple choice is the other way around. Categories should be made up of behaviors, such as rare diseases, and in these questions respondents who select more than one of the behaviors will be flagged. This will capture respondents who are trying not to be ruled out for incentive eligibility. The following example takes advantage of rare medical conditions.
The following are behaviors to watch out for:
Clicking on too many answers in multiple choice questions morocco phone number a big concern. For example, if the average number of items selected is 7 you should flag respondents who selected all or almost all of the answers.
People who do not follow instructions should be flagged. A common example in multiple choice is asking respondents to select only two or three answers. Inform your programming team that respondents should be able to select as many answers as they want, as we want to track actual respondent behavior.
Overusing the NA (not applicable) or I don’t know options is also a concern. We can flag respondents who overuse the “I don’t know” option. Although a good survey design includes the “I don’t know” option, respondents who overuse this option do so because they do not understand the nature of the survey topic.
Answering in a straight line is a common tactic of tired respondents . We should include a few negative statements in the grid, randomly distributed, to stop respondents and give them a reason to think about their answers. We should also limit the number of items in the grid, so that respondents answer the general research question as best as possible.