XM Knowledge Base
Academic Experience
Survey Straightlining: What is it? How can it hurt you? And how to protect against it.
Straightlining. It’s the bane of survey writers. It happens when your respondents rush through your survey clicking on the same response every time and it’s a serious threat to data quality.
More technically called non-differentiation in ratings, straightlining happens when respondents lose their motivation to engage with a survey because they’re bored, don’t have the mental energy, or they find the survey too complex and demanding.
Instant Download: 7 Tips for Writing Great Survey Questions eBook
Different Types of Straightlining
Straightlining at its most basic is where respondents will provide their answers to a series of questions in the same place on a rating scale:
More sophisticated respondents will provide these sorts of responses by alternating between two or more response options in a predictable manner across a series of questions like in the examples below.
These pattern-based responses are much more difficult to detect than perfect straightlining. As a result, the effects on your data can be even more negative because it is more difficult to identify and clean out the bad responses.
To complicate matters, online survey respondents are often aware that they may get caught and penalized for straightlining, so some of them will come up with response patterns that are completely random or mostly straightlining with just 1 or 2 deviations, such as in the example below:
What can you do about it?
There’s no failsafe, but you can take some steps to protect against straightlining. The best approach is a fairly straightforward three-part strategy:
- Avoid grid or matrix questions – these invite negative respondent behavior and provide a false sense of economy in questionnaire design.
- Present one question per page – as a general rule, this will reduce the likelihood of straightlining. That said, placing a few questions that are highly related to each other on the same page may be acceptable.
- Use response options that map directly to the subject of the question - placing many questions on the same rating scale actually makes the response task harder for respondents, and makes them more likely to straightline and otherwise engage in behaviors that produce low-quality data.
Note: In this blog post we provide a handful of examples of different forms of negative response behavior in surveys. We deliberately decided to present these examples using a BAD question type (Agree-Disagree statements) a BAD response option format (grid or matrix). Please do not use either of these in your own research if you value the quality of your data.