Screening Questions For Surveys

Screening questions (or screeners) should be placed at the beginning of your online survey to determine which respondents are a good fit to complete your survey, and those which should be immediately disqualified.

There are a number of things to consider before implementing screening questions such as how it will affect your required recipient size to placement and best practices. To help you get started on using screening we’ve outlined some guidance below…

How to effectively use screening questions

The two main types of screening questions:

Behavioural screeners

Just as this sounds, behavioural screeners are used to capture the answers of respondents who behave in a certain way, which could include anything from how they use their leisure time, to what they spend their money on.

For example, let’s say you asked the following question:

"How many times a week do you exercise?"

• Every day
• Two to three times a week
• Once a week
• A couple of times a month
• Never

In this scenario, if you wanted to explore the exercise habits of your more committed respondents, you might choose to only let those who exercise at least once a week or more to continue your survey and disqualify the others.

Industry specific screeners

These can be effective in helping to filter out respondents whose answers could bias your research. This could be as a result of respondents working in the same industry as yours or being close to someone else who does.

For example, if you worked in leisure and wanted to compare your sports shop with your competitors, your industry-specific screener could be:

"Do you or anyone within your family work in the following industries?"

• Food
• Fashion
• Sports
• Household goods
• None of these

To prevent any bias, you might choose to disqualify respondents who select the sports option.

Best practices when using screening questions

Once you’ve selected the type of screening questions you want to ask, you need to think about how and when you will ask them. Here are some pointers:

Ask a screening question at the beginning of your survey – if respondents are not right for your study, you don’t want to waste their time or damage the quality of your data. So, putting screeners at the beginning helps both of you.

Add page breaks in the right place – skip logic is the essential advanced feature used to route your ideal respondents through your survey and disqualify irrelevant survey takers from participating. However, you need to place a page break immediately after your last screening question if you’re to effectively qualify and disqualify respondents at this crucial early stage.

If you’re new to skip logic and would like to find out more about it, you might like to visit our help guides on basic and advanced skip logic.

Keep screeners as simple as you can – avoid using any more than half a dozen answer options for any screener and keep everything clear and concise, as it’s important to remember that your screener although important, is just the gateway to rest of your survey.

Keep ‘Yes’ or ‘No’ answer options to a minimum – as they could trigger a reluctant bias into your survey from respondents who just answer yes to everything, in order to seem more likeable.

You can use more than one screener – sometimes it can take more than one question to determine whether a respondent is right for your survey, so don’t shy away from using a couple of screening questions, if it will help give you more reliable results.

Choosing how many people to survey

Once you’ve finalised how you are going to use screening questions in your survey, you’ll want to consider the distribution and how many people you need to send it to to get the number of response you need.

When you add screening questions, your response rate will be typically lower, but in this instance, this is fine as the feedback you collect will be more valuable. However, this also means that you may want to increase the number of people you survey in total, in order to get closer to the number of responses you originally had in mind.

If you want to get a certain number of responses from those that pass your screening questions, you need to think about your survey’s incidence rate.

The incidence rate is the percentage of individuals who qualify to take your survey, based on the following formula:

Let’s say you decided to run an ‘incidence rate test’ you could create a survey, using just your screening questions. You could then send this to a fraction of your audience, say 100 people and see how they answer your questions.

If 75 of these people pass your screening questions and 25 are disqualified. This gives you a 75/100 x 100 = 75% incidence rate.

Calculating your incidence rate is important, as it will serve as the foundation step to estimating how many more people you need to survey in total, in order to obtain the number of qualified respondents you need.

How to analyse your survey responses

Having collected your survey results you’ll want to analyse the difference between complete and total responses.

Your complete responses represent the number of individuals that answered your survey from beginning to end, as opposed to your total responses, that can also include individuals who answered any question from your survey and those that were disqualified.

To do this you will need to filter your survey, so you’re only viewing complete responses, namely those that passed your screeners. For more information about how to use filters, why not take a look at our ‘how to use filtering tools’ support page.

Using screening questions to improve the quality of your data

Having to deal with irrelevant or biased feedback amongst your survey responses can be annoying at the best of times, adding unnecessary extra time and cost to your analysis and reporting process.

Fortunately, with a few screening questions at the beginning of your survey, you can eliminate a large proportion of these data issues early on and drive up the overall quality of data.