# How many survey answers can you trust? What we found screening 2,176 people.

- 22% to 25% stopped by pre-screening, on all three surveys
- 1 in 4 who passed and finished still had to be removed
- Client rejections fell from 7.9% to 2.1% with pre-screening

Our research team pre-screened everyone entering three surveys in 2026 for attention, answer quality and bots, then followed each person through to the data the client kept. These are the numbers, next to what independent research has found.

When we pre-screened 2,176 people entering three surveys in 2026 for attention, answer quality and bots, 22% to 25% were stopped, about the same share on every survey. Pre-screening alone isn't enough: on one survey, 1 in 4 people who passed it and finished were still removed, nearly all for failing an attention check. Where pre-screening was on, the client rejected 2.1% of the responses delivered, against 7.9% without it, and independent studies put careless answering alone at 8% to 12% of respondents.

Fieldwork August to November 2026. Outside studies checked on 22 November 2026. Sources at the end.

Protect your survey now: https://samplelabs.io/protect (First 10 people free. No card to start.)

## What we found. Three surveys, 2026.

Everyone who entered went through pre-screening for attention, answer quality and bots before the survey. People who were stopped never reached it. Then we followed the rest through to the data the client kept.

| What we measured | Result | Base |
| --- | --- | --- |
| Stopped by pre-screening for attention, quality and bots | 22% to 25% of people who entered | 2,176 people entering three surveys, August to November 2026 |
| Passed pre-screening and finished, then removed | 1 in 4 (36 of 144), 34 of them for failing an attention check | German university students, October to November 2026 |
| Responses the client rejected, with pre-screening | 2.1% (4 of 190) | UK graduates, August 2026 |
| Responses the client rejected, without pre-screening | 7.9% (109 of 1,379) | The same survey, over the same three weeks |

The gap in client rejections is unlikely to be chance (Fisher's exact test, p = .002). It held when we only counted the five sample suppliers that sent people both ways: 2.6% against 7.2% (p = .03).

With only 4 rejected responses, the true rate with pre-screening is somewhere between about 0.5% and 5%.

## About the same on every survey. Whoever was asked.

The share stopped by pre-screening barely moved between surveys, whatever the topic, audience or country.

| Survey | Kind of research | People who entered | Stopped by pre-screening |
| --- | --- | --- | --- |
| German university students | Market research | 733 | 25% |
| UK graduates | Market research | 1,165 | 22% |
| UK adults | Academic study | 278 | 23% |

## Bad answers come in more than one kind. Here's what independent research found.

Bots get the headlines, and they're a real problem. Careless answering is a big one too, and AI can now pass the checks built to catch it. Every number here comes from a university, a journal or Pew Research Center, never from a company that sells sample or checks.

### Bots and fake respondents

- **4% to 7%** of people in opt-in online polls were bogus, and 3% to 7% were still there after the usual checks. (Pew Research Center, 2020)
- **12%** of adults under 30 in an opt-in poll said they were licensed to operate a nuclear submarine. (Pew Research Center, 2024)
- **18%** of 11,114 people in an opt-in poll answered yes to at least one impossible question. (Pew Research Center, 2026)
- **61.8%** of the first wave of responses to a survey shared on social media had to be removed, mostly bots. (Griffin et al., Quality & Quantity, 2021)
- **94.5%** of 271 completes to a survey advertised on social media were judged fraudulent. (Pozzar et al., Journal of Medical Internet Research, 2020)

### AI-written answers and AI agents

- **99.8%** of 6,000 attention checks were passed by an AI agent built to take surveys. (Westwood, PNAS, 2025)
- **34%** of people recruited on one online research platform said they had used AI chatbots to answer open questions. (Zhang, Xu & Alvero, Sociological Methods & Research, 2025)
- **33% to 46%** of crowd workers on a summarising task were estimated to have used AI to do it (a preprint, not yet peer reviewed). (Veselovsky, Ribeiro & West, arXiv, 2023)

### Careless and low-effort answers

- **8% to 12%** is the typical share of careless respondents, in a review of the research. (Curran, Journal of Experimental Social Psychology, 2016)
- **11.7%** of participants were careless on average, across 48 crowdsourced studies. (Jones et al., Experimental and Clinical Psychopharmacology, 2022)
- **10% to 12%** of students filling in a long survey for course credit answered carelessly. (Meade & Craig, Psychological Methods, 2012)

### Attention checks

- **47%** of people in an opt-in online panel passed all four attention checks. 12% failed all four. (Berinsky, Margolis & Sances, American Journal of Political Science, 2014)

## What this means for your survey.

- **Check people before they reach your survey.** In our UK graduate survey, pre-screening cut the responses the client rejected from 7.9% to 2.1%.
- **Let a failed attention check end the survey.** In our German student survey, the second check didn't end it, so 32 people finished and had to be removed by hand.
- **Check how fast people answered before you deliver.** One complete in the same survey took under half a minute, too fast to have read the questions.
- **Read the open answers, not only the scores.** An AI agent can pass attention checks, so look at what people wrote. On surveys built in SampleLabs, Dot reads every open answer for AI-written and pasted text.

## How we did this. And what it can't show.

- **The surveys.** Three surveys our research team ran for clients from August to November 2026: a market research survey of German university students, one of UK graduates, and an academic study of UK adults.
- **Pre-screening.** Before each survey, people went through pre-screening for attention, answer quality and bots. People who were stopped never reached the survey.
- **Following each person.** We joined each person's record across three exports: the sample supplier's, the pre-screening results and the survey's own. Every pre-screening record and every complete matched.
- **The base.** Rates are shares of everyone who entered a survey, so the three compare on the same base. Client rejections are shares of the responses delivered.
- **Its limits.** Three surveys is a small base, so read these as a first measure, not a market average. Only the UK graduate survey had a client list of kept responses, so only it measures client rejections.

## Questions people ask

### How many survey responses are bots or fake?

It depends on where the people come from. Pew Research Center found 4% to 7% bogus respondents in opt-in online polls, and surveys shared openly on social media have lost most of their responses: 61.8% were removed in one 2021 study, mostly bots. In our 2026 surveys, pre-screening for attention, quality and bots stopped 22% to 25% of the people who entered, a figure that counts bots and careless answering together.

### How common are careless or low-effort answers?

Common. A review of the research puts the typical share of careless respondents at 8% to 12%, and a meta-analysis of 48 crowdsourced studies found 11.7%. In our German student survey, 34 of the 144 people who passed pre-screening and finished then failed an attention check inside the survey.

### Do attention checks still work?

They catch people who aren't reading, which is still a big group. They don't catch AI: in a 2025 PNAS study, an AI agent passed 99.8% of 6,000 attention checks. Use them next to checks on how people answer. Dot does both in Protect. https://samplelabs.io/protect

### How many survey answers are written by AI?

There's no reliable market-wide number yet. In a 2025 study, 34% of people recruited on one online research platform said they had used AI chatbots to answer open questions, and a 2023 preprint estimated that 33% to 46% of crowd workers used AI on a writing task. How we check for AI explains what Dot looks for. https://samplelabs.io/how-we-check-for-ai

### Does pre-screening improve the data a client receives?

In our UK graduate survey, yes. Part of the traffic went through pre-screening and part didn't, over the same three weeks. The client rejected 2.1% of the responses that came through pre-screening (4 of 190) and 7.9% of the rest (109 of 1,379). That gap is unlikely to be chance (Fisher's exact test, p = .002).

### Why check inside the survey if people were pre-screened?

Because some people only stop paying attention once they're in. On our German student survey, 36 of the 144 people who passed pre-screening and finished still had to be removed, 34 of them for failing an attention check. Checks before and during the survey catch different people.

### Is the share of bad answers the same for every survey?

In our three surveys it barely moved: 22% to 25%, across market research and academic work, students, graduates and adults, in Germany and the UK. So checks are worth running on every survey, not only on the hard ones.

### How did you do this research?

We joined each person's record across three exports: the sample supplier's, the pre-screening results and the survey's own. Every pre-screening record and every complete matched. Rates are shares of everyone who entered a survey, so the three compare on the same base.

### Can I check my own survey for bots and AI answers?

Yes. Protect puts Dot in front of a survey you already run in Qualtrics, Google Forms or another tool, and checks for bots, rushing, repeat visits and attention. We set it up with you on a short call. AI-written answer checks run on surveys you build in SampleLabs. https://samplelabs.io/protect

### Can I quote these numbers?

Yes. Please name SampleLabs and link to this page. The independent figures belong to their authors, so cite the original study, listed under sources at the end of this page.

## Where these numbers come from

Our own numbers come from the three surveys above. The others come from these studies, each checked against the study itself.

- Pew Research Center, 2020: Assessing the risks to online polls from bogus respondents. https://www.pewresearch.org/methods/2020/02/18/assessing-the-risks-to-online-polls-from-bogus-respondents/
- Pew Research Center, 2024: Online opt-in polls can produce misleading results, especially for young people and Hispanic adults. https://www.pewresearch.org/short-reads/2024/03/05/online-opt-in-polls-can-produce-misleading-results-especially-for-young-people-and-hispanic-adults/
- Pew Research Center, 2026: No easy fix for bogus respondents in online opt-in polls. https://www.pewresearch.org/methods/2026/08/27/no-easy-fix-for-bogus-respondents-in-online-opt-in-polls/
- Griffin et al., Quality & Quantity, 2021: Ensuring survey research data integrity in the era of internet bots. https://doi.org/10.1007/s11135-021-01252-1
- Pozzar et al., Journal of Medical Internet Research, 2020: Threats of bots and other bad actors to data quality following research participant recruitment through social media. https://doi.org/10.2196/23021
- Westwood, PNAS, 2025: The potential existential threat of large language models to online survey research. https://doi.org/10.1073/pnas.2518075122
- Zhang, Xu & Alvero, Sociological Methods & Research, 2025: Generative AI meets open-ended survey responses: research participant use of AI and homogenization. https://doi.org/10.1177/00491241251327130
- Veselovsky, Ribeiro & West, arXiv, 2023: Artificial artificial artificial intelligence: crowd workers widely use large language models for text production tasks. https://arxiv.org/abs/2306.07899
- Curran, Journal of Experimental Social Psychology, 2016: Methods for the detection of carelessly invalid responses in survey data. https://doi.org/10.1016/j.jesp.2015.07.006
- Jones et al., Experimental and Clinical Psychopharmacology, 2022: Careless responding in crowdsourced alcohol research: a systematic review and meta-analysis of practices and prevalence. https://doi.org/10.1037/pha0000546
- Meade & Craig, Psychological Methods, 2012: Identifying careless responses in survey data. https://doi.org/10.1037/a0028085
- Berinsky, Margolis & Sances, American Journal of Political Science, 2014: Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys. https://doi.org/10.1111/ajps.12081

## Find out which of your answers you can trust.

Dot checks every response for bots, rushing, repeat visits and attention while people answer, in your own survey tool or in SampleLabs.

Protect your survey now: https://samplelabs.io/protect
