Survey Methodology Best Practices: Where Survey Error Comes From and How to Control Each Source

Survey methodologists group these problems under "total survey error", which Groves and Lyberg describe as "a conceptual framework describing statistical error properties of sample survey statistics" (Public Opinion Quarterly, 2010). They also note that it does not cover non-statistical dimensions of quality. The table splits the errors into the two families.

AAPOR's best-practice guidance opens with the question "Is a survey the best method for answering your research question?" (AAPOR). Three situations where the answer is no:

• You do not know the answer options yet. Closed questions only offer the options you thought of. Run interviews first and build the list from what people say.
• You need behaviour you can observe or already record. Sales data, app analytics or shelf audits beat asking people to recall.
• The decision needs a number you cannot reach. If the decision depends on households without internet, an online survey will not answer it, however large.

Pew Research Center explains why the second row matters: agree-disagree statements draw more agreement from "less educated and less informed respondents", and it suggests offering "a choice between alternative statements" instead (Pew). Our AI survey generator guide has a fuller question checklist.

Decide the languages from where you will report, not from where it is convenient to field. A national study fielded only in English and Hindi can under-represent the people who answer more comfortably in another language. Translate as a team, with a translator, a reviewer and someone who knows what each question is meant to measure; our Likert guide describes the approach Pew uses for its India surveys.

Mode decides coverage. App and online samples reach connected consumers quickly; phone needs no app or data connection but cannot show images; face-to-face reaches households without internet at a higher cost. Our quantitative research guide sets out official figures on smartphone and internet access and what each mode misses.

AAPOR says that "before fielding a survey, it is important to pretest the questionnaire", and recommends a pilot test to check that all survey procedures work as intended (same AAPOR page). They are two different checks.

Pretest (cognitive interviews). Sit with a few people like your respondents, in each language, and ask them to answer aloud and explain what each question means to them. You are looking for misreadings, not opinions.

Pilot (a small live run). Field the final questionnaire on the real platform and check:

• How long it takes, and where people drop out.
• Which questions are skipped or answered "don't know" most often.
• Whether every route through the skip logic works.
• Whether the open answers show that people understood the question.
• Whether the data file has the variables and labels your analysis plan needs.

AAPOR's guidance on transparency lists items such as "sample size, margin of sampling error, weighting attributes" (same AAPOR page). For an opt-in or panel sample, AAPOR's 2013 task force adds two cautions: a margin of sampling error does not fit such samples, and the term "response rate" should be avoided; it notes that ISO 20252 uses "participation rate" instead (AAPOR task force report, 2013). Fill in:

The ICC/ESOMAR International Code requires researchers to limit personal data to what is "reasonable and directly relevant to the purpose of the research" (Article 3), and to make participation voluntary, based on clear information given at the start (Article 4) (ICC/ESOMAR Code). A short questionnaire with no unnecessary identifiers is the practical form of that rule. Go through the draft and delete every question that no row of your analysis plan uses.

On our platform you write the questionnaire and set the targeting, and questionnaires can be fielded in as many Indian languages as the study needs (panel methodology), so the translation and pretest steps above still apply to each language you choose. Our panel is reached through an app, so it covers smartphone users only; put that in the population line of your methods note.

For a pilot, the Free plan includes 30 responses. A 1,000-complete main study then costs 1,000 × ₹120 = ₹1,20,000 on a Pro bundle (pricing).