In the two previous posts we talked about why you should use an even number of items in your response scales, and when to allow “N/A” and “I Don’t Know” as options.
Now let’s talk about the optimal number of response options to use in your opinion-related, scaled questions to get the most bang for the buck.
(Questions about fact don’t apply here as there isn’t a judgement call required – the answer is the answer.)
The choice of the best number of options (on a scale) is a balance between two opposing forces which I’ll call:
So then on opinions you’re looking for a response scale with the widest range possible (which helps with the stats), while keeping the range narrow enough to maintain a noticeable difference between each choice.
The more options you have, the more “pin-point” and accurate the response might be.
This is why an “Employee Net promotor Score” (ENPS) “I would recommend working here to a friend” is on a scale from “1 to 10” rather than “Yes/No“.
There is an entire area of psychology and statistics regarding the Just Noticeable Difference (JDN), but in essence it comes down to this: if I were to ask you to grade how white your shirts are when they come out of the wash on a 0-100 scale, can you REALLY tell the difference between an 87 and an 88? If not, then the scale is two wide and you’re getting a false sense of a range of opinion that just isn’t there. (In statistics this is called “False Variance“.)
In the above example there is no way for a normal human to tell the difference between 87 and 88, but if I ask the same question on a 2-pont scale (“white” and “not white”) then the results become useless, and on top of that the respondent would have no idea how to answer.
So start here (using an even number of responses of course):
We ourselves generally choose a 6 response scale for questions on opinion, as we believe it gives the highest range of responses, while still maintaining a JND.
Hope this was of help, but as always give us a shout if you would like to discuss!
Next up – Avoiding unactionable questions
Thanks for reading, and if you’re interested in discussing a survey for your organization call us at 1-604-219-7876, email us at [email protected], or just book a discovery call for a one-on-one chat.
Let's have a chat
Maureen Simons is a senior human resources and communication consultant with over 25 years of experience helping clients achieve their business and organizational objectives through their people. (Linkedin)
Adam Hunter has a Bachelors degree in Mechanical Engineering, an MBA, and 35+ years of technical and programming experience, resulting in a broad mix of analytical, statistical, project-related and business skills. (Linkedin)