Survey Errors: Common Mistakes and How to Avoid Them

When it comes to writing your survey, at first thought it may seem a relatively simple task. However, without proper forethought and planning you could easily be left with a poorly written survey.

Worse still, given that businesses typically undertake a survey to improve aspects of their operations, whether for their customers, their employees or both, survey errors could lead to poor insight that you’re unable to do anything positive with. Subsequently, you’ll want to do everything you can to avoid this happening to you.

So, to help you we’ve outlined below some common survey mistakes that you’ll want to avoid.

1. Avoid asking leading questions

When you’re writing your survey questions, you need to ensure you don’t confuse your respondents or influence how they answer your questions.

Subsequently, you’ll want to avoid creating leading questions, which consciously or unconsciously encourage respondents to answer in a way that reinforces a biased opinion that you may have on a topic. This can especially be a problem in feedback surveys when a survey creator is trying to better understand a respondent’s perception of a product, service or process, but unwittingly pushes their own bias into a question.

An example of a leading question in this context could be:

"How much did you love talking to our customer service team?"

The word ‘love’ is extremely biased, leading the respondent to assume that this customer service team usually offers a quality service. Of course, if a respondent didn’t agree with this, they could avoid answering the question altogether. However, the lack of a response and the insight that goes with it, is as bad as a biased response to the survey creator.

A more neutral way to answer this could be as follows:

"How would you rate your latest experience with our customer service team?"

2. Avoid using loaded questions

Similarly to leading questions, you’ll want to avoid asking loaded questions.

Although they also introduce bias, rather than leading questions that try to push respondents in a particular direction, loaded questions make automatic assumptions about respondents.

A good example of a loaded question, can be seen with the following:

"Where do you usually like to go out to eat?"

This question wrongly assumes everyone can afford and wants to go out to eat.

Alternatively, in order to remove the bias from this question, you could get respondents to answer a qualifying question first. Or, if you didn’t fancy doing that, you could employ skip logic to skip them past any questions that weren’t relevant to them.

Although, the differences between them are small, both leading and loaded questions can confuse, mislead, or influence respondents into answering in a particular way. So, you must try to avoid them as much as possible.

3. Avoid double-barrelled questions

One of the most common errors in surveys is to ask respondents to ask two questions at once, which is otherwise known as a double-barrelled question.

Not only do these types of questions cause confusion for your respondents, but they can also leave you with poor and unhelpful data that you’re unable to do anything useful with.

An example of a double-barrelled question can be seen below:

"How well do you get on with your managers and co-workers?"

While this question is focused on the same thing in terms of your relationship with other people, it’s likely you’ll have different relationships with each group due to their position in the workplace hierarchy. Therefore, you would be better off asking this as two separate questions, one targeted at your manager and the other your co-workers.

4. Avoid using complex language

Irrespective of who may be taking your survey, try to avoid jargon, acronyms or technical terms that may confuse your respondents, and try to use clear, concise and uncomplicated language instead.

If you need to include any complicated terms or concepts, make sure you provide clear definitions and examples to make these easier to understand. Essentially you need to be talking in a language that you know the vast majority of your audience will easily understand.

For example, compare the following questions. The second question provides extra clarification for anyone slightly unsure about the meaning of the first question, to maximise the volume of people who can answer this.

Bad question: "Do you use streaming services?"

Better question: "Do you use streaming services?" (e.g., Netflix, Amazon Prime, Disney+, Now TV, BritBox, etc)

If you can ensure simple survey questions, you’ll ensure almost everyone is able to answer them and go on to complete your survey.

5. Include a survey introduction

Whether it’s forgetting or choosing to skip the inclusion of an introduction for your survey, this is something you must try to avoid.

There are a number of reasons for this. Firstly, having an appropriate introduction, which offers a flavour of what you are doing and why, is one of the best ways to engage your audience.

Secondly, if you don’t include a survey introduction, your respondents may feel uncomfortable or even suspicious about the reasons for your survey and choose to avoid it altogether.

Therefore, including an introduction which explains your survey’s purpose in simple and clear language can have a beneficial impact on its engagement and encouraging respondents to read on.

6. Avoid rushing your survey out

Having worked hard on selecting and crafting your questions, the last thing you want to do is rush to get your survey out and risk not getting the results you hoped to achieve, simply because it wasn’t quite ready.

Even if you’re an expert on the topic of your questions, without further checking it will be difficult to know with complete confidence how recipients might respond to your questions. However, there’s a few things you can do to help with this.

Firstly, re-examine the wording of your questions and consider getting a few colleagues to check them and relay how they would answer them. If you’re going to use email or SMS invitations, you might also choose to send your colleagues a test invitation first to see if they noticed it, or whether they had any issues with it ending up in their SPAM folder.

You could also consider running a pilot test study of your survey with a sample of your target audience. This could involve anything from analysing your question flow and the average time it takes respondents to complete each question to checking your mix of questions or any instances of confusion among recipients. Pilot testing your survey in this way can help you to fine tune it, to help maximise its engagement with respondents and the value of insight you can obtain from it when you send it out for real.

Always leave sufficient time for planning and checking your survey

Having read through the advice we’ve just outlined we hope you appreciate just how important it is to get your language and questions right, include an engaging survey introduction and not rush your survey project.

Ultimately, if you’ve provided sufficient time for planning and testing your survey, you should be able to spot anything that needs tweaking, so you can maximise your survey’s effectiveness when it comes to sending it out.