a.k.a. “Do I really care how much the participants hate me?”
We get asked this one a lot … and it absolutely, definitely sort of depends.
Let’s start by saying do not (do NOT) create a survey that cannot be saved until all questions are answered. It’ll kill your response rate, you’ll lose some good data, and I as the respondent will feel less inclined to send you Christmas cards.
Always give the respondents a reminder of any questions missed, but give them the option to submit their answers anyway. We tend to offer two buttons/choices on surveys where questions are missed; “Return to survey” and “Submit anyway”.
The only exception to this is where the entirety of the data (or a certain group of questions) is completely worthless without all questions answered – such as in a “score”.
So now you say, “Hey then you … how DO I get around cases where the respondent might really, honestly have no clue on how to respond? Particularly as I’ve gone and taken your advice and we’re using even-numbered scales so don’t they have the response option of neutral?” (More on that in a future post “What scale should I use – Odd or even? Opinion or Fact?”.)
Depending on the situation you’ve basically got two options; using “N/A” as a response option, or using a filter question.
Case1: Allowing for “N/A” as a response.
There are perfectly valid reasons to allow “N/A” as a response, but please use them sparingly, as this will be a very tempting click target for those who just want to get the questions done and don’t want to think about it all too much (much like allowing for a response of “neutral”).
Examples of the legitimate use of “N/A” would be:
- “The on-site PPE is adequate” (they only work from home)
- “The benefits plan works well for me” (they are part-time and aren’t eligible)
Case 2: Using a filter question.
This can be particularly useful if you can use their answer to show or hide the relevant questions. If that’s not possible though, you can then use textual instructions = in the way of “If ‘no’ please skip to question X.”
Examples of good use of a filter question would be:
- “I work from home” (yes/no)
- “I have access to the benefits plan” (yes/no)
… all followed by the relevant questions.
So in all don’t make all questions required, but DO let the respondent “have an out” where appropriate – as any response is far better than no response at all.
Next tip: “How are people going to hear about and access the survey?”