Have you ever concluded a survey to discover it’s generated many more completes than specified in your quota targets? This is called Quota Overshoot, and it happens when a lot of respondents fill in a survey at the same time. Quota Overshoot doesn’t reduce a survey’s quality, but it does result in higher-than-necessary expenditure on reward, sample and interview costs.
The good news is Nfield’s Max Overshoot feature enables you to control the number of excess completes, and so avoid unnecessary costs.
Why Quota Overshoot happens
To understand how Max Overshoot works, you first need to understand why Quota Overshoot happens. The one-word technical answer to this is ‘concurrency’.
Concurrency is when two or more respondents are filling in a survey at the same time. For the most part, this is exactly what you need to get your survey completed in the shortest possible time. But if multiple responders remain active when just one more complete is needed, you’ll end up with more completes than the quota. This is because all the interviews already underway at the moment of the last required completion will also continue to completion
How Nfield solves this
Nfield’s Max Overshoot feature gives you control over excess completes by limiting concurrency as your survey nears completion. You can set it to reflect your preferred balance between speed and excess respondent costs
The Plane Game
To explain how this balance shifts, let’s play the Plane Game. Or in these COVID-19 times, imagine playing it!
THE GOAL:
Get 20 paper planes made, each by a different person. Of these, 10 have to be made by males and the other 10 by females. (Want to know how to make a “world record paper plane”? This video shows you!)
THE RULES:
Each plane-maker has to sit at a separate desk while performing the task. When finished, they vacate the desk for a new plane-maker to take their place. This all happens inside a room that players can only enter when a free desk is available. Players who’ve completed their planes stay in the room. Players who decide to give up have to leave the room. When you’ve reached the target number of planes, no more people can come in.
Everyone in the room at the moment the last required plane is completed gets a voucher worth $8 as a reward, even if they’re still finishing their plane.
SCENARIO 1:
You invite a lot of people (let’s say 100) to participate and provide the same number of desks. This way, everyone is making planes at the same time, with nobody having to wait their turn.
This will produce a really fast result. But because you had 100 people in the room, and none of them had given up at the point the 10+10 target was reached, you have to pay them all. That’s a bill of $800.
SCENARIO 2:
You have two rooms, each with just 3 desks. One room is only for the males, the other only for the females. Both rooms operate at the same time. Once 10 male planes have been completed, the male game ends. The same applies for the female game.
This limits each category’s concurrency and, as a result, the number of participants in both rooms combined when the targets are reached will only be 24. An excess of two males and two females. The total reward payout will be just $192. But the game will have taken much longer to complete.
Finding the right balance
Of course this Plane Game is a metaphor for respondents completing surveys, with the two scenarios illustrating extreme ends of the cost vs speed spectrum. As a market researcher, you’ll probably be looking for a happy medium. Nfield achieves this via a formula which changes the number of “desks” as each quota target comes closer to completion. All you have to do is set your desired Max Overshoot number per quota target.
At the start
|
If we illustrate this according to the Plane Game, per quota it looks like:
|
This means we start with 12 desks per quota target.
One new participant is allowed to start after each new complete ONLY IF Successful Completes < the quota target. (Nfield doesn’t allow new interviews to be started once quota target for successful completes has reached its maximum.)
As completes start to accumulate, the number of desks gets reduced. This limits the number of active participants and, in turn, the number of excess participants at the time each quota target is reached.
Quota target = 10
Max Overshoot = 2
| Number of successful completes | Number of desks available | Maximum number of active participants |
| 0 | 12 | 12 |
| 1 | 11 | 11 |
| 2 | 10 | 10 |
| | | | | | |
| 8 | 4 | 4 |
| 9 | 3 | 3 |
| 10 | 2 | 2 |
What about setting the Max Overshoot to zero?
You can set the Max Overshoot to zero to keep costs to an absolute minimum. However, it will take longer to achieve the final complete as, by the end stage, only one respondent can be active at a time, which will amplify the effects of any drop outs.
You can make a judgement call on how seriously this might impact things by looking at your survey’s dropout data in Nfield.
- Drop-out rate: The higher the drop-out rate, the higher the chance of a longer wait for your final complete.
- Drop-out point: If drop-outs typically occur later in your survey, after a higher number of questions have been answered, the amount of time wasted due to drop-out is likely to be higher. Note that Nfield sessions automatically time out if there’s no activity for 15 minutes.
Ensuring value
Two important points to note when using Max Overshoot:
- Although Max Overshoot will limit the costs associated with going over quota, it still results in panellists being wasted when samples are flooded to fill quotas. Best practice is to use minimum and maximum targets in a quota structure that’s designed as simply as possible, in order to reduce or eliminate the need to flood the sample.
- Max Overshoot works in alignment with the maximum quota target. Its use is only beneficial if the target is reached.
Available on demand
Max Overshoot can be a very beneficial feature for controlling costs and provides flexibility to balance cost against speed. However, it does make quota evaluation more complex and may put a big load on Nfield processors. We are therefore rolling it out gradually and are currently only enabling it upon request. To enable Max Overshoot, please contact your account manager. At one point, we will enable this feature for all domains.
If you have any questions about Nfield’s Max Overshoot feature, don’t hesitate to Contact Us.
