To be of real value, market research has to be cleaned of invalid responses, such as non-genuine answers and submissions from outside a required geographic region. While you can’t (yet!) put responses through a lie detector test, there are still many ways to identify rogue submissions.
Following the Quality Control tips below will help you catch responses that are deliberately or accidentally invalid. And, thanks to recent improvements to Nfield’s Quality Control feature, taking action is easier than ever.
Here are measures you can take to safeguard survey response quality, followed by a guide to using Nfield’s improved Quality Control feature.
Best practices for identifying invalid survey responses
Set clear data quality benchmarks
Establish predefined criteria for acceptable completion durations, location accuracy, and logical consistency within sets of answers. See below for examples.
Check completion durations
Check for questionnaires that have been completed within an unreasonably short time. For example, if a survey is estimated to take 10 minutes, any that are completed in under two minutes should be flagged for review.
Leverage location data
Verify that respondents are within the target geographical area. For example, if a CAPI survey delivers responses from Birmingham when the target area was London, those responses should be flagged for further review.
Monitor response patterns
Identify anomalies such as straight-lining (choosing the same answer for all questions), contradictory responses, or unusual answer distributions. For instance, if a respondent selects both “Strongly Agree” and “Strongly Disagree” for similar questions, their responses should be reviewed.
Use attention checks and red herring questions
Embed control questions to detect (unintentionally or deliberately) inattentive respondents. For example, a question could instruct, “Select ‘Strongly Agree’ as your answer for this question.” If a respondent fails this check, their survey submission can be flagged as potentially low quality.
Review open-ended responses
Ensure meaningful engagement by analyzing open-text answers. Responses like “asdasd” or “12345” indicate respondents are not making an effort to answer properly.
Combine automated and manual reviews
For optimum results and efficiency, use a mix of manual and automated Quality Control techniques. For example, automated tools can be developed to detect questionnaires that were completed too quickly, while researchers can manually verify the status of flagged responses.
Act fast on low-quality responses
Ensure unreliable responses are filtered out before finalizing data analysis. Removing inconsistent data promptly prevents derivation of misleading insights and improves overall research quality.
Discover Nfield’s enhanced Quality Control feature
With Nfield’s updated Quality Control feature, market researchers can drill down into each interview to see details such as responses, location, duration and other key insights. Any interviews that appear suspicious can easily be flagged for further quality control checks and possible removal.
- Detailed question responses – show how each respondent answered every question, making it possible to identify inconsistencies or suspicious patterns.
- Survey location data – makes it possible to verify whether survey responses satisfy geographical criteria. See Academy #57 for GPS Location Fix.
- Interview duration – reveals completion times, making it possible to identify responses that are likely to be of poor quality due to surveys being completed too quickly.
- Other key metadata – provides access to timestamps, respondent details and more in one centralized location.
- Flag for review – Researchers can flag specific interviews inside the questionnaire script for additional quality checks. Setting the system variable “TTInterviewQuality” to 5 assigns an interview with a “Needs to be checked” status. This is useful for flagging straight-liners (respondents answering too predictably) or speedsters (respondents completing the survey too quickly). Flagged interviews can then be reviewed directly on the Quality Control page without downloading data files.
How to use Nfield’s Quality Control feature
- Navigate to the Quality Control page in Nfield.
- Click on any interview to expand the details.
- Review responses, location (CAPI only), duration, and key metrics.
- Use the flagging feature inside the questionnaire script to mark responses that require further review.
- Apply the best practices (described earlier in this article) to refine data collection strategies or optimize survey design.
NOTE: Nfield doesn’t currently include automated Quality Control tools. However, custom scripts for these can be inserted into Nfield.
Elevating Quality Control in Market Research
This latest enhancement reflects NIPO’s commitment to helping market researchers collect and manage the highest-quality data. With greater transparency into individual interviews, you can improve data integrity, identify potential issues faster, and enhance the reliability of your research.
Try the new Quality Control feature in Nfield today and experience the difference! If you have any questions or feedback, feel free to reach out to our support team!
