Product Testing Survey in India: Liking, JAR and Penalty Analysis

A product test asks real users to try the actual product and then answer questions. The survey is the measuring instrument. It does not replace the trial itself, so someone must put the product in respondents' hands first.

Two set-ups are common. In an in-home use test (IHUT), people use the product over days in their own kitchen or bathroom, then answer online. In a central-location test (CLT), people taste or try it in one place and answer on the spot. Both can use the same questionnaire. The difference is how long the person has with the product and how much the setting is controlled.

The survey has one job: tell you whether the product is ready, and which attribute to change if it is not. A concept test is different. It measures reactions to a description, not to the product (see product concept testing). Packs are tested separately too, in packaging testing.

Order matters. Ask overall liking first, before any attribute question, so that a later question about sweetness does not shape the overall verdict. Then ask attribute liking, then just-about-right, then purchase intent, then open comments.

Keep to these standard instruments: 9-point hedonic for overall liking, then JAR with penalty analysis. A home-made 1 to 10 scale gives you numbers that nobody can compare with earlier tests.

Add a question on the pack or the price only if the test is about them. Extra topics tire people and mix the signals. Read the open comments yourself, or see open-ended survey analysis for how to code them.

A JAR question does not ask whether people like an attribute. It asks whether the amount is right. The answer tells you the direction of a fix. Penalty analysis then tells you which fix is worth making first.

The method, as MetricGate documents it (June 2026): the penalty weights the drop in liking "by the share of consumers affected, so reach and severity combine into a single prioritizable number." Steps for each attribute:

1. Collapse the JAR scale into three groups: too little, just about right, too much.
2. Compute the mean overall liking of each group.
3. Drop = mean liking of the just-about-right group minus mean liking of the not-right group.
4. Penalty = drop multiplied by the share of all respondents in that group.
5. Rank attributes by penalty, and test the drop for significance if the group is large enough.

Worked example. These are illustrative numbers, not a real study. 200 people rated a snack. Overall liking averaged 6.665 across everyone.

Read it as an editor would. Too-spicy has the largest penalty (0.540), so fix that first. Crunch has the biggest single drop (2.0), but only 10% feel it, so its penalty is 0.200. Count complaints alone and you would chase the wrong attribute. Check the arithmetic: for spiciness, (30 × 6.1 + 110 × 7.4 + 60 × 5.6) ÷ 200 = 6.665. Crunch gives the same 6.665, as it must, because the same 200 people gave one overall rating each.

Two cautions. First, a penalty shows association, not cause. People who dislike a product often also call it too spicy. Second, small groups give unstable drops. Decide in advance the smallest group you will act on. Agree a minimum group size in advance (for example 20% of respondents) and write it in the plan.

This choice changes your sample. Our research design chooser covers it in general. For products:

For a forced two-product choice, 100 respondents need at least 61 choosing the same product before the gap is significant at the 5% level, two-sided (our binomial calculation, shown in the sources log). With 200 respondents the figure is 115. A split of 55 to 45 is not evidence.

Rotate the order. If product A always comes first, you cannot separate A's liking from first-position liking. Our survey builder supports rotation for monadic and sequential monadic designs. Blind the products with neutral codes so that brand names do not score for you.

Two of our case studies are close relatives. The skincare blind test shows what a blind design looks like on packs, and the toothpaste flavour test shows concept reactions. Neither is a use test of a physical product, so read them for design, not for benchmarks.

There is no single number. Published work gives ranges, and they depend on the analysis. A 2018 Sensory Society presentation by Koppel, Koppel and Chambers resampled real consumer data and concluded: for an overall one-way ANOVA, about 110 consumers; for pairwise comparisons between samples the number needed is "MUCH higher and never under 90". They also say the test must include enough people from all the groups you target.

Turn that into a plan. Fix the groups you will read separately first, such as city tier, age band or heavy users, and give each group enough respondents on its own. A plan of 200 total split four ways leaves 50 per group, which is thin. See sampling methods and statistical significance testing for the tests themselves.

Recruit actual category users. A hair oil test among people who do not use hair oil measures politeness, not preference. Use screener questions on category use and on how recently they bought.

Our platform does not ship or hand over products. Your team, or a field partner, must supply the product, brief the respondent and set the trial period. The survey measures the reactions that follow.

Plan these points before launch:

• Delivery and storage. Heat in transit and shelf life change taste. Record the dispatch date and ask respondents on which day they tried it.
• Use instructions. Spell out the dose, time and number of days. Show a photo or short video in the survey if it helps (the builder accepts image, video and audio stimuli).
• Language. Ask JAR terms in the respondent's own language and pretest each version. 'Too salty' and its Hindi or Telugu equivalent may not carry the same weight.
• Scale habits. Yes-saying and high-end scoring are known in Indian surveys, so use fully labelled scales and compare products within one study, not against figures from another.
• Who can take part. The panel is app-based, so respondents are smartphone users. For offline households, use face-to-face fieldwork.
• Safety. For food, cosmetics or anything applied to skin, have allergen and consent text approved by whoever owns product liability.

Ask each respondent to confirm they actually tried the product, and drop those who did not. See survey data quality for checks on speeding and straight-lining.

If your job is the reaction survey for a product test in India, our platform suits that half. You or a field partner supply the product. For a two-product sequential monadic test with 400 respondents who each try both, list pricing looks like this:

Pro is sold in bundles of 400 to 1,000 responses and costs 20% less per response than Plus. Enterprise plans come with a dedicated research team and are priced on request.

The panel has a 25M+ consumer footprint. The survey-eligible panel is a subset, and availability depends on your targeting. Members complete Aadhaar-based KYC once, and each study receives a zero-knowledge proof of verification. Surveys run in 8+ languages, and speeders, straight-lining and inconsistent answers are removed before analysis. See the panel methodology. Poseidon, our analytics engine, lists penalty and JAR analysis among its formulas. Check its output against the worked method above.

The limit: the panel is app-based, so respondents are smartphone users only, and the platform does not supply physical product.