# Survey Bias: Types, Indian Examples and How to Fix Each One

The main kinds of survey bias, sampling, non-response, response and question bias, with Indian examples, how to spot each and a fix table.

Published 2026-10-16 · Updated 2026-11-16
Canonical: https://hercules.works/blogs/survey-bias-india

Survey bias is a systematic error that pushes results one way. The four main sources are sampling (who is asked), non-response (who answers), response (how they answer) and question bias (how you ask). Fix each at design time: define the audience, quota on key variables, write neutral questions, randomise options and pilot. The AAPOR report notes that nonprobability online panels have inherent and significant coverage error, so state who your sample represents.

## Bias Is Built In Before Anyone Answers

Most survey bias comes from design, not from respondents lying. Ask the wrong people, ask them in a leading way or lose the people who would have disagreed, and no analysis rescues the result.

This guide lists the main types, gives an Indian example of each (illustrative) and a fix. It also covers how to state limits. On panels, the [AAPOR report](https://aapor.org/wp-content/uploads/2022/11/nfq048.pdf) states that "all nonprobability online panels have inherent and significant coverage error". It dates from 2010 and concerns the United States, so treat it as a caution, not a measurement for India.

## Types of Survey Bias and How to Fix Them

| Bias | What happens | Illustrative Indian example | Fix |
|---|---|---|---|
| Sampling or coverage | Some people cannot be reached | An app survey misses households without smartphones | Name who is excluded; add face-to-face fieldwork where it matters |
| Selection | The sample differs from the target | A fuel survey that is mostly men | Quota on gender, age and city; report the split |
| Non-response | Those who answer differ from those who do not | Only very happy or very unhappy customers reply | Keep it short; follow up; compare responders with the base |
| Acquiescence | People agree with statements | Long agree/disagree grids | Use balanced wording and item-specific scales |
| Social desirability | People give the approved answer | Questions on spending, health or politics | Make it anonymous; use indirect wording |
| Question wording | The question leads | "How much do you love our delivery?" | Neutral wording; see [questionnaire design](/blogs/survey-questionnaire-design-india) |
| Order | Earlier questions colour later ones | Asking about price before asking about the brand | Rotate or order general before specific; pilot |
| Language | Translated scales read differently | A 5-point scale labelled differently in two languages | Native-speaker review; test each language |

## How to Spot Bias in Your Data

- Compare the sample with what you know about the audience: city, age band, gender. Large gaps need weighting or a caveat.
- Look for straight-lining and very fast answers. See [survey data quality](/blogs/survey-data-quality-india).
- Check answers to a question asked two ways. Large gaps mean wording is driving the result.
- Read the open-ended answers. They show how people understood the question.

## State the Limits

Say who the sample represents, the cities and age bands, the dates and the method. Avoid "all Indians" for a panel result. When a figure matters for a decision, check it with a second method. The [omnibus versus panel guide](/blogs/omnibus-vs-online-panel-india) shows when to use a representative sample.

## Worked Example: How Non-Response Moves a Result

Say you email 1,000 customers and 200 reply, a 20% response rate. Among repliers, 70% are satisfied. You do not know the view of the 800 who stayed silent.

| If the silent 800 are... | Overall satisfied | Calculation |
|---|---|---|
| As satisfied as repliers (70%) | 70% | 0.2 × 70 + 0.8 × 70 |
| Less satisfied (40%) | 46% | 0.2 × 70 + 0.8 × 40 = 14 + 32 |
| Not satisfied at all (0%) | 14% | 0.2 × 70 + 0.8 × 0 |
| All satisfied (100%) | 94% | 0.2 × 70 + 0.8 × 100 |

All figures are illustrative. The point: with a 20% response rate, the true figure can lie anywhere from 14% to 94%. The result is only as good as your reason to think the silent group resembles the repliers. A higher response rate narrows that range, and so does a check of repliers against what you know about your customer base.

## Worked Example: Coverage Gaps and Weighting

Weighting repairs a known imbalance. Say your target is 50% men and 50% women, and your sample is 70% men and 30% women. Each man gets a weight of 0.5 ÷ 0.7 = 0.71. Each woman gets 0.5 ÷ 0.3 = 1.67.

Suppose 60% of men and 40% of women say yes. The unweighted result is 0.7 × 60 + 0.3 × 40 = 54%. The weighted result is 0.5 × 60 + 0.5 × 40 = 50%. Illustrative, but it shows a 4-point shift from imbalance alone.

Weighting only works for groups you have in the sample. It also makes the effective sample smaller, so margins widen. See [weighting and effective sample size](/blogs/survey-weighting-effective-sample-size). Weighting cannot repair a leading question, or a group that is not there at all.

Coverage works the same way. If 80% of your target audience can be reached and 20% cannot, and the two groups differ by 20 points on your question, the gap in your overall figure is 0.2 × 20 = 4 points. The size of the error depends on how different the unreached group is, which you often cannot measure. That is why you state who the sample represents.

## Mode and Interviewer Effects on Sensitive Questions

Pew Research Center notes that people may give inaccurate answers to sensitive questions, and that respondents understate alcohol and drug use, tax evasion and racial bias. It also notes that social desirability bias can be greater when an interviewer is present ([Pew, writing survey questions](https://www.pewresearch.org/writing-survey-questions/)).

In practice, a self-completed phone survey may suit a question on spending or health better than a face-to-face interview in a household where family members are listening. Choose the mode for the question, not only for cost. Put sensitive items late, make them optional and use wording that does not suggest a right answer.

## Testing for Order and Wording Effects

Pew also notes that the order in which questions are asked can influence how people respond. You can test for it. Split the sample at random into two halves. Show half the questions in one order and half in the other. Compare the answers.

The same split works for wording. Ask half "How satisfied are you with the delivery?" and half a differently worded version. If the shares differ by more than the margin, wording is driving the answer. Pew also describes acquiescence in agree-disagree questions and suggests offering a choice between alternative statements instead.

## Which Bias to Check First

| Your situation | Most likely problem | First check |
|---|---|---|
| Customer survey with a low reply rate | Non-response | Compare repliers with your customer base on city, age and purchase history |
| Survey on an app or website | Coverage | Write down who cannot be reached |
| Many agree-disagree statements | Acquiescence | Rewrite as choices between statements |
| Questions on spending, health or politics | Social desirability | Switch to self-completion; use indirect wording |
| Same question in two languages | Language | Compare results by language |
| Quota cell filled by only a few people | Selection | Count how many people are in the cell |

## A Bias Statement You Can Put in Your Report

Add a short paragraph to every report. Fill in the blanks.

"This survey was fielded between ___ and ___ among ___ respondents in ___ (cities), aged ___ to ___. It was done on ___ (method) in ___ (languages). People without ___ could not be reached. Quotas were set on ___. Results were / were not weighted by ___. The response rate was ___%. Margins quoted assume a simple random sample and are the best case."

## Running This Study on Hercules

If your job is a consumer survey in India, quotas, screeners and a pilot reduce several of these biases. This is the pilot and main study at list price.

Pricing is per response, and the free plan covers a 30-response pilot: see [pricing](/pricing). How respondents are verified and sampled: [panel methodology](/panel-methodology).

The limit: the app reaches smartphone users only, so offline households need face-to-face fieldwork.

## Frequently asked questions

### What is survey bias?

A systematic error that pushes results in one direction. It can come from who is asked, who answers, how they answer and how the questions are worded.

### What is the difference between sampling bias and non-response bias?

Sampling bias is about who is in the pool you ask. Non-response bias is about who, among those asked, actually answers.

### How do I reduce response bias?

Use neutral wording, balanced scales, anonymous answers for sensitive topics and a pilot. Avoid long agree-disagree grids.

### Does a panel have sampling bias?

Any opt-in panel covers only its members. Ours is app-based, so it reaches smartphone users. Quota on the variables that matter and state the limit.

### Can weighting fix bias?

It can adjust known gaps, such as age or city. It cannot fix a sample that misses a group entirely or a leading question.

### Is a random sample free of bias?

No. Random selection removes selection bias in who is picked, but non-response and question wording still bias the result. A 20% response rate can leave a wide range for the true value.

### What is acquiescence bias?

The tendency to agree with a statement whatever it says. It is common in agree-disagree questions. Offering a choice between two statements reduces it.

### How do I check if my sample is biased?

Compare it with what you know about the audience, such as the city, age and gender mix. Then compare answers across languages and across early and late responders.
