Survey design your respondents will love

What is survey design?

Businesses use surveys for a whole lot of reasons, such as gauging customer satisfaction, employee engagement or the reach of a new marketing campaign.

A survey needs to be carefully tailored to:

• engage respondents
• motivate them to finish it
• deliver valuable feedback data

Good survey design is key to these three things. It’s the process of deciding what a survey wants to achieve, devising suitable questions, and formatting an attractive style and branding, before it goes out to respondents.

Why is good survey design important?

Creating surveys is as much an art as it is a science. It involves attention to detail in its design and flow. And a survey is only as good as the responses it gets. If it looks confusing, or long and boring to your target audience, they probably won’t complete it. Or, worse, won’t even bother to start it in the first place.

So, taking the time to design your survey well is an investment that will give you a decent completion rate and good quality data.

What does good survey design look like?

A good survey will have a clear purpose set out right at the beginning. Questions will be easy to read, understand and respond to, encouraging a variety of answers – closed-ended (tick boxes, selection) and open-ended (boxes for open text comments), arranged in a logical order. You can use images to enhance or clarify questions, and company branding gives a sense of identity and trust. And a survey mustn’t take too long – no more than 15 minutes.

Let’s look at these in more detail:

1. Set your goal

Right from the very start, ask yourself: “What is the purpose of my survey? What are my main objectives?” and set an attainable goal. For example, one department has a high staff turnover after two years’ service, and you want to know why. Rather than a generic employee satisfaction survey, make your inquiries more specific: Is there career progression beyond two years? Is there sufficient industry training? Can people stand a culture or individual for only so long? Make the goal clear at the top of the survey, perhaps: The purpose of this survey is to understand why people leave our company.

Tip: Unless respondents know what their survey information is going to be used for, they won’t be able to give meaningful responses.

2. Choose your survey question types

There are more than 100 different ways to ask a question, and the type of question has a direct impact on the survey results.

Closed-ended questions have pre-populated answers to choose from, such as multiple choice or tick boxes. They are simple to answer and provide quantitative data.

Open-ended questions, also known as open text, ask for feedback in the respondent’s own words. They provide qualitative data. They are the most reliable, but they also lead to survey fatigue faster, so you should limit those types.

These are the most common question types:

Closed-ended question types

• Multiple choice questions form the basis of most research. They can be displayed as a traditional list of choices or as a dropdown menu, tick box, radio buttons, etc.
• Multi-select is used when you want participants to select more than one answer from a list.
• Ranking order is used to determine a respondent’s order of preference for a list of items. These questions are best used when you want to measure your respondents’ attitude toward something. These can be time-consuming for respondents.
• Rating order questions are asked to indicate their personal levels on things such as agreement, satisfaction, or frequency.
• Scale questions. Rather than asking respondents a basic yes or no question, use scales that measure both the direction and the intensity of opinions.

This is critical for survey research. Someone who “strongly supports” a decision is very different from someone who “slightly supports” it. Scales extend the power of analysis from basic percentages to high-level analyses based on means and variance estimates (like t-test, ANOVA, regression, and the like).

Scales make it easier for respondents to answer and for you to conduct your analysis. If scales have the same scale of points, you can quickly compare responses to different questions.

• Matrix Tables are used to collect multiple pieces of information in one question. They provide an effective way to condense your survey or to group similar items into one question. An example is the Likert scale that is useful for measuring more nuanced attitudes and opinions than a simple yes/no. It’s a reliable way to measure perceptions, opinions and behaviors. For example:

How satisfied or dissatisfied are you with our new office?

• Sliders let respondents indicate their level of preference with a draggable bar rather than a traditional button or checkbox.
• Side-by-side questions let you ask multiple questions in one condensed table and provides an effective way of shortening your survey while collecting the same amount of data.
• Personal/demographic questions – leave these to the end of your survey, putting the more friendly, engaging questions first.

Open-ended question types

Text entry is used to gather open-ended feedback from respondents. These responses can be lengthy essays, standard form information such as name and email address, or anything in between.

Tip: Open text questions take longer to answer, so limit them to one or two, at or near the end of your survey.

3. Write the survey questions

There’s a simple approach to this – write the questions that you yourself would want to answer. Chances are, these questions will follow these guidelines:

• Keep them short and easy to understand. Use plain English for your question wording so that the respondent understands them on first reading. Avoid unnecessary adjectives and vague adverbs such as ‘frequently’ and ‘usually’ so there’s no chance that the question will mean different things to different people.

Tip: Write so that school students between 9th and 11th grade would understand the questions.

• Speak your respondents’ language. Asking about caloric content, bits, bytes, and other industry-specific jargon and acronyms is confusing. Make sure your audience understands your language level and terminology and above all, that they understand what you are asking. The best move is to write to your least-informed respondent. If a respondent won’t understand an acronym, either define it or don’t use it.
• Explain unexpected questions. For instance, if it’s important for you to ask toy store customers what time of the day they usually visit the store, briefly explain why that’s relevant.
• Don’t use leading questions. These introduce bias into your survey that influences respondents’ selections. Bias skews useful data. Compare:

Did you enjoy our delicious new sandwich range? (biased)

with

What did you think of our new sandwich range? (unbiased)

• Avoid double-barrelled questions. These ask two questions at once, and are impossible to answer usefully, e.g.:

How satisfied or dissatisfied were you with your hotel suite and breakfast on your recent stay with us?

Better to ask:
a. How satisfied or dissatisfied were you with your hotel suite…?
b. How satisfied or dissatisfied were you with your breakfast…?

• Avoid prestige bias. This is when you associate your question with a person or group that enjoys prestige, and it nudges the respondent to agree, e.g.:

Most doctors say we should include more whole grains in our diet. Do you agree?

It’s hard to disagree with a doctor.

• Don’t use absolutes such as ‘always’, ‘every’ and ‘all’. These force your respondent to either agree or disagree, with no room for nuanced, informative answers. For example:

Do you always eat breakfast cereal?
a. Yes
b. No

Someone who eats breakfast cereal six mornings out of seven would be forced to answer ‘no’. Better data comes from:

How many times a week do you eat breakfast cereal?
None
1-2 times
3-4 times
5-6 times
Every day

• Don’t mix your question types. Look at this example:

During your most recent visit to our store, how satisfied were you with the cleanliness of our restrooms?

Awkward, isn’t it? If it says ‘satisfied’ in the question, you’ll need to have ‘satisfied’ in the answer.

• Justify requests for sensitive information. This is particularly true with any information about ethnicity, family status, income, education, etc. You can diffuse respondents’ concerns by telling them how it is going to be used. For instance, you can explain that purchasing habits will only be analysed in aggregate for benchmarking purposes or that survey results will not be shared outside your organisation.

Tip: Don’t forget to include a PNA – ‘Prefer Not To Answer’ option.

4. Decide your question sequence

Make your survey easier for respondents by keeping questions in their logical order and avoiding changing topics unnecessarily. The funnel approach is a survey flow technique that makes the respondent’s job easier. This approach suggests that you:

• Follow a logical order
• Start with broad and general questions that qualify the respondent and introduce the topic
• Move into more specific questions
• Finish with general, easy-to-answer questions (like demographics)

This approach allows respondents to warm up with broad and general questions, work into more specific and in-depth questions, and cool down at the end. This turns the survey into a smooth road for respondents, which decreases drop-out rates and may even increase the quality of answers you receive.

5. Design the look of your survey

Use images where appropriate

Humans usually respond more favourably to images than to text, and often understand concepts more quickly with an image or an icon. Breaking up long blocks of text with images, header bars, and icons make reading your survey more pleasant. Your software platform should offer question types that include graphics such as smile/frown faces, graphic sliders, and the capability to upload your own images or video. Images should add meaningful context to a question but should not bias responses by making one option seem more attractive than another.

Include your branding

The more you include your branding in all consumer interactions, the more consumers will trust your brand. This extends to surveys. They are more impactful when they contain your logo, colour palette, custom theme and brand-associative images. Whether customers or employees, your respondents will feel more connection with a branded survey, and more inclined to answer the questions thoughtfully.

Tip: Other ways you can extend your branding with a survey include:

• Adding a custom ‘thank you’ to the end of your survey that pops up after completion
• Adding a custom url for your survey link
• Redirecting the respondents to your website when they’ve completed the survey

6. Test your survey design

At its simplest, ask 5 people who match your sample to take your survey, then ask them:

• How long did it take?
• Were any questions confusing?
• Were there any other problems while taking the survey?

This allows you to quickly correct lingering problems before distribution.

We recommend running your surveys through a series of tests to check for potential problems and to make sure you get the data you want. Different strategies include: respondent debriefing, cognitive interviewing, expert evaluation, focus groups, experiments and pilot studies.