Survey Data Quality in India: A Researcher's Checklist to Catch Careless, Duplicate and Fake Responses

AAPOR. The 2010 AAPOR task force report on online panels lists the behaviours linked to satisficing: "response non-differentiation ('straightlining'), random responding, responding more quickly than would be expected" (speeding), response order effects and heavy use of "don't know". It also states that "there is no widely accepted industry standard for editing and cleaning panel data" (AAPOR Report on Online Panels, Public Opinion Quarterly, 2010).

ESOMAR. The 2012 edition of its 28 Questions to Help Buyers of Online Samples (September 2012) asks every provider who is responsible for data quality checks and whether there are procedures against "random responding", "illogical or inconsistent responding", "overuse of item non-response" and "speeding". Question 22 asks about "a confirmation of respondent identity procedure" and procedures "to detect fraudulent respondents" (ESOMAR, 2012 edition). ESOMAR also lists a later 37 Questions to Help Buyers of Online Samples, so check the current edition as well.

ISO 20252. BSI's listing of the 2019 edition says the standard "establishes terms, definitions and service requirements for service providers" and "now also covers access panels, both online and offline" (BSI). Research Live reported on 1 September 2026 that it has been updated as ISO 20252:2026, including "strengthening data quality requirements in sampling and panel management" (Research Live). The full text is sold, so ask a provider which edition it is certified to and for which methods.

You cannot clean what you did not measure. Add these before fieldwork. The numbers are our suggestions, not standards.

1. Attention checks. Put in about one per seven questions, with instructions such as "select the animal" or "select the colour". Rotate the position and keep them short.
2. A reverse-worded item in every grid you plan to check, so a flat line is also a contradiction.
3. One bogus item. For a phone survey: "I have never used a mobile phone".
4. An invented brand in each brand-awareness or purchase list.
5. A neutral screener. Ask the category question among decoys, with an explicit time frame such as "in the last three months". Never hint at the answer that qualifies.
6. A re-check of key eligibility later in the survey, in different words.
7. At least two open-ended questions, so one weak answer is not the only evidence.
8. Exclusion windows. Agree how long a member is kept out of repeat invitations on the same topic.

Attention checks alone are weak, as below. Use them as one flag among several.

Dominik Leiner tested several indicators in four studies and found that "careless responses are most reliably identified by questionnaire completion time" (Leiner, Survey Research Methods, 2019). His relative speed index works page by page. For each page, divide the sample's median time by the respondent's time. A factor of 2 means twice as fast as typical. Cap each page factor at 3, then average across pages.

Leiner suggests "a much more lenient cut-off of 2.0 to identify particularly suspicious records". Respondent A is above it. B, who rushed one page, is not. He adds a limit: if the questionnaire asks for facts or knowledge rather than opinions, completion time is not a valid indicator, because experts answer faster.

AAPOR describes straight-lining as "selecting the same response for all items in a grid" and notes that it "could also indicate a poorly designed questionnaire". Someone who agrees with both "Price matters most to me" and "I never look at the price" has given you a contradiction, not just a flat line.

Trap questions are less reliable than they look. In Pew Research Center's 2019 tests of opt-in samples, "some 84% of bogus respondents pass the trap question and 87% pass a check for responding too quickly" (Pew, February 2020). Leiner makes the same point: a respondent making up coherent answers "will likely pass attention checks". Bogus items worked better in his data, though he notes they were used in only one sub-sample. His examples include "I'm currently filling out a questionnaire" and "I have never ever used a computer".

Open-ends. AAPOR suggests looking for "gibberish or answers that appear to be copied and then repeatedly pasted in question after question". Pew found that 2% to 4% of opt-in respondents "repeatedly gave answers that did not match the question asked", against 0% in address-recruited panels. Read answers in the language they were written in. A short Hinglish answer such as "price thoda zyada hai" is a valid answer, not low effort.

Duplicates. AAPOR says the most common technique is digital fingerprinting, which builds an ID from IP address, browser and device settings. It warns that "false positives are possible (e.g., two persons in the same household)", so review flagged pairs before deleting. Where a household shares one phone, compare the two records' age, gender and answers first.

Screener integrity and fraud. Fakers try to qualify. AAPOR notes researchers "may include false brands, nonexistent products, or low-incidence behaviors" in screening questions. A respondent who claims to buy your invented brand regularly is not a real category buyer.

Write these rules into the analysis plan before fieldwork, so nobody tunes them to get a preferred result. Only rule 1 comes from a published source. The other thresholds are our suggestions, built on the AAPOR and Leiner material above. They are not benchmarks, so adapt them to your questionnaire. Every check removes some honest people and misses some bad ones, which is why AAPOR notes some researchers use "a scoring system in which a case must fail on multiple tests before it is eliminated". Rule 12 does that.

Worked example (made-up file). A study returns 1,000 completes. Rule 1 removes 38. Rules 6, 7 and 9 together remove 7. Review of 6 duplicate pairs confirms 4. Among the rest, 10 respondents carry two or more soft flags. No record is counted twice, so 38 + 7 + 4 + 10 = 59 records out and 941 analysed. Report all four numbers, and compare the key results with and without the removed cases. If the headline finding moves, say so.

Clean data can still produce a wrong chart. Take three or four headline figures and recompute them from the respondent-level file yourself. Check the base for each figure: all respondents, or only those shown the question after skip logic? Check that single-choice answers sum to 100% and that multi-select charts say respondents could pick more than one. Check that any weighting in the method note was applied. Fix anything you find before the report goes out.

These follow ESOMAR's buyer questions and AAPOR's advice that a researcher should confirm a panel has removed multiple identities before using it.

1. How is each member's identity confirmed when they join, and how do you detect fraudulent respondents?
2. Which in-survey checks run automatically, and what are the rules?
3. Are removed responses visible to me, with the reason for each?
4. How often can one person take part in surveys in a given period, and can the exclusion window be set per study?
5. Can I get respondent-level data, including completion times?
6. Do you review questionnaires for quality before they go live?
7. Are you certified to ISO 20252, which edition, and for which methods?

If your job is a consumer survey in India, here is what our platform documents against the checklist above. These are capabilities, not guarantees, and we do not publish the thresholds or removal rates behind them.

• Attention checks inserted automatically, about one per seven questions.
• Flags for speeders and straight-lining.
• Open-end scoring for gibberish, copy-paste and off-topic answers.
• Panel deduplication, with configurable exclusion windows.
• Screener integrity: neutral category screens among decoys, explicit time frames, a red-herring brand, and key criteria re-checked in other words.
• A 9-dimension quality rubric with hard gates, which blocks a weak survey from publishing.

Members of the SuperJ app complete Aadhaar-based KYC once, then take part through a zero-knowledge proof, so researchers get verified answers without identity data. The app has a 25M+ consumer footprint; the survey-eligible panel is a subset and depends on your targeting. Surveys run in as many Indian languages as the study needs (panel methodology). Keep your own duplicate review in place. A 500-complete study costs 500 × ₹120 = ₹60,000 on a Pro bundle (pricing), and the Free plan includes 30 responses for a trial file. The limit: the panel is India only and app-based, so offline households need face-to-face fieldwork.