CAPI Online

CAPI quality control: Audit Trail helps you eliminate falsified fieldwork

10 June 2020

Good quality CAPI research relies on trustworthy interviews. You need to be sure fieldwork integrity isn’t compromised by interviewers taking shortcuts, submitting false responses. Nfield CAPI now builds on standard verification checks with an Audit Trail / Logging feature that highlights suspicious input patterns. By analyzing statistical details, this enables you to identify when interviewers are likely to be cheating the system, and take appropriate action.

Building on the basics

Traditional quality control components such as field validation, logic checking, ordering and randomization are, of course, all supported in Nfield. For face-to-face CAPI interviews, additional features such as GPS locations, photographs and audio recordings can also be activated to provide evidence of interviews taking place. (More information about these can be found at Nfield’s quality control options for rock-solid CAPI data.)

All this gets you off to a great start, but still leaves room for cheating. To further close off the possibility of falsified responses, you need to look more closely at response input patterns. Nfield CAPI’s Audit Trail function enables you to spot the tell-tale signs of dishonest interviewer behavior, such as:

Within an interview:
Within an interviewer’s individual behavior:
Within an interviewer’s comparable performance:

The data for annotated items 1, 2 and 3 can be found in the Nfield downloads:

Comparing performance via Audit Trail

Nfield CAPI’s new Audit Trail feature records how long an interviewer stayed on a page and which button they clicked to move on. The results are provided in a csv file. The example below shows how this might look for a single interview.

You can compare each result line against your reasonable expectations and set benchmarks to flag up interviews that fall outside these parameters.

The best method for reviewing this is to build a report (in Microsoft PowerBI or other reporting system) to show suspicious interviews. Incorporating a drill-down function will give you an easy way to view more information.

The two illustrations below show how your reports might look.

Here, the results are sorted to show interview durations in descending order.

Intelligence at your fingertips

Armed with these insights, you can better fortify the quality of your CAPI surveys by eliminating falsified responses. Not only can you challenge interviewers whose input appears suspicious, and remove fake responses from your results, you can also use this feature to deter dishonest interviewer behavior from taking place at all.

And in case you are wondering, this Audit Trail feature will also be introduced for Nfield Online in the near future.

Get a first impression, scheduled soon.

Get a first impression, scheduled soon.

Request a demo to see how NIPO can help you meet your requirements with our smart survey solutions.