How many survey answers can you trust? What we found screening 2,176 people.
- 22% to 25% stopped by pre-screening, on all three surveys
- 1 in 4 who passed and finished still had to be removed
- Client rejections fell from 7.9% to 2.1% with pre-screening
First 10 people free. No card to start.
In short
When we pre-screened 2,176 people entering three surveys in 2026 for attention, answer quality and bots, 22% to 25% were stopped, about the same share on every survey. Pre-screening alone isn't enough: on one survey, 1 in 4 people who passed it and finished were still removed, nearly all for failing an attention check. Where pre-screening was on, the client rejected 2.1% of the responses delivered, against 7.9% without it, and independent studies put careless answering alone at 8% to 12% of respondents.
Fieldwork August to November 2026. Outside studies checked on 22 November 2026. See the sources.
What we found. Three surveys, 2026.
Everyone who entered went through pre-screening for attention, answer quality and bots before the survey. People who were stopped never reached it. Then we followed the rest through to the data the client kept.
- Stopped by pre-screening for attention, quality and bots
- Result: 22% to 25% of people who entered
- Base: 2,176 people entering three surveys, August to November 2026
- Passed pre-screening and finished, then removed
- Result: 1 in 4 (36 of 144), 34 of them for failing an attention check
- Base: German university students, October to November 2026
- Responses the client rejected, with pre-screening
- Result: 2.1% (4 of 190)
- Base: UK graduates, August 2026
- Responses the client rejected, without pre-screening
- Result: 7.9% (109 of 1,379)
- Base: The same survey, over the same three weeks
The gap in client rejections is unlikely to be chance (Fisher's exact test, p = .002). It held when we only counted the five sample suppliers that sent people both ways: 2.6% against 7.2% (p = .03).
With only 4 rejected responses, the true rate with pre-screening is somewhere between about 0.5% and 5%.
About the same on every survey. Whoever was asked.
The share stopped by pre-screening barely moved between surveys, whatever the topic, audience or country.
- German university students
- Kind of research: Market research
- People who entered: 733
- Stopped by pre-screening: 25%
- UK graduates
- Kind of research: Market research
- People who entered: 1,165
- Stopped by pre-screening: 22%
- UK adults
- Kind of research: Academic study
- People who entered: 278
- Stopped by pre-screening: 23%
Bad answers come in more than one kind. Here's what independent research found.
Bots get the headlines, and they're a real problem. Careless answering is a big one too, and AI can now pass the checks built to catch it. Every number here comes from a university, a journal or Pew Research Center, never from a company that sells sample or checks.
Bots and fake respondents
-
4% to 7%
of people in opt-in online polls were bogus, and 3% to 7% were still there after the usual checks.
-
12%
of adults under 30 in an opt-in poll said they were licensed to operate a nuclear submarine.
-
18%
of 11,114 people in an opt-in poll answered yes to at least one impossible question.
-
61.8%
of the first wave of responses to a survey shared on social media had to be removed, mostly bots.
-
94.5%
of 271 completes to a survey advertised on social media were judged fraudulent.
AI-written answers and AI agents
-
99.8%
of 6,000 attention checks were passed by an AI agent built to take surveys.
-
34%
of people recruited on one online research platform said they had used AI chatbots to answer open questions.
-
33% to 46%
of crowd workers on a summarising task were estimated to have used AI to do it (a preprint, not yet peer reviewed).
Careless and low-effort answers
-
8% to 12%
is the typical share of careless respondents, in a review of the research.
-
11.7%
of participants were careless on average, across 48 crowdsourced studies.
Jones et al., Experimental and Clinical Psychopharmacology, 2022
-
10% to 12%
of students filling in a long survey for course credit answered carelessly.
Attention checks
-
47%
of people in an opt-in online panel passed all four attention checks. 12% failed all four.
Berinsky, Margolis & Sances, American Journal of Political Science, 2014
What this means for your survey.
-
Check people before they reach your survey
In our UK graduate survey, pre-screening cut the responses the client rejected from 7.9% to 2.1%.
-
Let a failed attention check end the survey
In our German student survey, the second check didn't end it, so 32 people finished and had to be removed by hand.
-
Check how fast people answered before you deliver
One complete in the same survey took under half a minute, too fast to have read the questions.
-
Read the open answers, not only the scores
An AI agent can pass attention checks, so look at what people wrote. On surveys built in SampleLabs, Dot reads every open answer for AI-written and pasted text.
How we did this. And what it can't show.
- The surveys
- Three surveys our research team ran for clients from August to November 2026: a market research survey of German university students, one of UK graduates, and an academic study of UK adults.
- Pre-screening
- Before each survey, people went through pre-screening for attention, answer quality and bots. People who were stopped never reached the survey.
- Following each person
- We joined each person's record across three exports: the sample supplier's, the pre-screening results and the survey's own. Every pre-screening record and every complete matched.
- The base
- Rates are shares of everyone who entered a survey, so the three compare on the same base. Client rejections are shares of the responses delivered.
- Its limits
- Three surveys is a small base, so read these as a first measure, not a market average. Only the UK graduate survey had a client list of kept responses, so only it measures client rejections.
Questions people ask
How many survey responses are bots or fake?
It depends on where the people come from. Pew Research Center found 4% to 7% bogus respondents in opt-in online polls, and surveys shared openly on social media have lost most of their responses: 61.8% were removed in one 2021 study, mostly bots. In our 2026 surveys, pre-screening for attention, quality and bots stopped 22% to 25% of the people who entered, a figure that counts bots and careless answering together.
How common are careless or low-effort answers?
Common. A review of the research puts the typical share of careless respondents at 8% to 12%, and a meta-analysis of 48 crowdsourced studies found 11.7%. In our German student survey, 34 of the 144 people who passed pre-screening and finished then failed an attention check inside the survey.
Do attention checks still work?
They catch people who aren't reading, which is still a big group. They don't catch AI: in a 2025 PNAS study, an AI agent passed 99.8% of 6,000 attention checks. Use them next to checks on how people answer. Dot does both in Protect.
How many survey answers are written by AI?
There's no reliable market-wide number yet. In a 2025 study, 34% of people recruited on one online research platform said they had used AI chatbots to answer open questions, and a 2023 preprint estimated that 33% to 46% of crowd workers used AI on a writing task. How we check for AI explains what Dot looks for.
Does pre-screening improve the data a client receives?
In our UK graduate survey, yes. Part of the traffic went through pre-screening and part didn't, over the same three weeks. The client rejected 2.1% of the responses that came through pre-screening (4 of 190) and 7.9% of the rest (109 of 1,379). That gap is unlikely to be chance (Fisher's exact test, p = .002).
Why check inside the survey if people were pre-screened?
Because some people only stop paying attention once they're in. On our German student survey, 36 of the 144 people who passed pre-screening and finished still had to be removed, 34 of them for failing an attention check. Checks before and during the survey catch different people.
Is the share of bad answers the same for every survey?
In our three surveys it barely moved: 22% to 25%, across market research and academic work, students, graduates and adults, in Germany and the UK. So checks are worth running on every survey, not only on the hard ones.
How did you do this research?
We joined each person's record across three exports: the sample supplier's, the pre-screening results and the survey's own. Every pre-screening record and every complete matched. Rates are shares of everyone who entered a survey, so the three compare on the same base.
Can I check my own survey for bots and AI answers?
Yes. Protect puts Dot in front of a survey you already run in Qualtrics, Google Forms or another tool, and checks for bots, rushing, repeat visits and attention. We set it up with you on a short call. AI-written answer checks run on surveys you build in SampleLabs.
Can I quote these numbers?
Yes. Please name SampleLabs and link to this page. The independent figures belong to their authors, so cite the original study, listed under sources at the end of this page.
Where these numbers come from
Our own numbers come from the three surveys above. The others come from these studies, each checked against the study itself.
- Pew Research Center, 2020
- Pew Research Center, 2024
- Pew Research Center, 2026
- Griffin et al., Quality & Quantity, 2021
- Pozzar et al., Journal of Medical Internet Research, 2020
- Westwood, PNAS, 2025
- Zhang, Xu & Alvero, Sociological Methods & Research, 2025
- Veselovsky, Ribeiro & West, arXiv, 2023
- Curran, Journal of Experimental Social Psychology, 2016
- Jones et al., Experimental and Clinical Psychopharmacology, 2022
- Meade & Craig, Psychological Methods, 2012
- Berinsky, Margolis & Sances, American Journal of Political Science, 2014