Diary Study in India: A Practical Design for Repeated Self-Reporting
How to run a diary study in India with short repeated surveys: entry and exit interviews, daily prompts, completion incentives and honest analysis of drop-off.
In shortA diary study asks the same people to report the same things over several days or weeks, close to when they happen. It trades breadth for detail: fewer respondents, many observations each, and a clear record of what changed between entry and exit.
Contents
What a Diary Study Captures
Diary Study Method, Plainly
The unit of analysis is the entry, not the person. Fifty respondents over fourteen days can produce seven hundred entries, which is enough to describe patterns within a respondent's week.
Entries can be time-based, event-based or interval-based. A time-based diary prompts at fixed moments, such as the end of each day. An event-based diary asks for an entry whenever a defined thing happens, such as a purchase or a service complaint.
Event-based diaries capture what matters but depend on the respondent noticing and remembering to log it. Time-based diaries are more complete but can miss the moment.
Pick one and say why in the methods section. Switching logic mid-study makes the entries incomparable.
Diary, Survey or Interview
| Method | Best for | Weakness |
|---|---|---|
| One-off survey | A number you can compare across groups | Recall and summarising |
| Interview | Meaning and reasoning | Small n, retrospective |
| Diary study | Sequences, routines and change over time | Drop-off and burden |
Use a diary when the sequence matters: what happens before a skipped dose, a late payment or an abandoned cart. A survey can tell you that it happened. A diary can show what came just before.
Use a survey when you need a stable estimate for a population. Diary samples are small and self-selected, so they are poor at estimating prevalence.
Designing the Diary
Keep each entry under three minutes. A diary that takes ten minutes will be completed for two days and abandoned on the third.
Ask a fixed core every time, plus one rotating question. The fixed core makes trends visible. The rotating question keeps the task from becoming mechanical.
Write the prompt in the language the respondent uses on their phone, and keep the answer formats to taps and short lists. Long free text on a phone produces short, unhelpful entries.
Include a timestamp and a simple context field, such as where they were or who they were with. Context is what turns a log into an explanation.
Test the whole sequence on paper first, then on the phone, with five people who match the sample.
Recruitment and Retention
Recruit for compliance, not only for fit. The best respondent is the one who will still be answering on day ten.
Set expectations at entry: how many days, how long each entry takes, what happens if a day is missed, and how to stop. People drop out when the task turns out to be different from what they agreed to.
Send reminders at a consistent time, and stop reminding after the study ends. A reminder at the wrong hour becomes the reason someone quits.
Plan for attrition in the sample size. If you need forty completed diaries, recruit more than forty and track completion daily.
Keep a study log of who missed which day. Losing one heavy day can bias a week.
Incentives and Ethics
Pay for completion, not for signing up. A split incentive, part on entry and part on finishing, protects both the respondent and the study.
Explain what is collected and who sees it. Diary studies collect routines and locations, which is more sensitive than a one-off survey.
Ask for consent at entry and again if you add a new field mid-study. Being in a study on day one is not consent to a new question on day eight.
Give people a way to withdraw and to ask for their entries to be deleted. Keep the retention period short and tell respondents what it is.
Analysing Diary Data
Start with completion, not content. How many people started, how many finished, which days lost the most entries. Drop-off is a finding about burden as much as about the topic.
Then describe the sequence. Plot the fixed core across days for each respondent, and look for runs, reversals and steady states rather than single points.
Code the rotating question into a small set of categories, and keep one category for entries that do not fit. That category is usually where the insight is.
Compare entry and exit. A change in language or in what people notice is often more informative than the average of the whole period.
Do not report diary findings as population estimates. Report them as patterns within the studied group, with the number of respondents and entries.
Where Diary Studies Fail
The first failure is burden. Too many questions per entry, too many days, and a completion rate that makes the last week meaningless.
The second is reactivity. People change the behaviour they are recording, especially when the diary is about a habit they are trying to change. Say so, and look for the change in the entries.
The third is an entry window that is too wide. Asking people to recall the whole day at midnight produces a summary, not a diary.
The fourth is analysis that ignores time. If the report averages all entries, the study has thrown away the reason it existed.
Running This on Hercules
Build the diary as a short survey and send it on a fixed daily schedule. Keep the core to a handful of taps and put the rotating question at the end.
Pilot with the free 30 responses, then check completion by day before you commit to a longer field period.
Frequently asked questions
What is a diary study?
A diary study asks the same people to report the same things repeatedly over days or weeks, close to when they happen. It is used for sequences, routines and change over time.
How long should a diary study run?
Long enough to capture the routine you care about, and short enough to keep completion high. One to two weeks covers most weekly routines. Add days only if the behaviour is rarer than weekly.
How many respondents does a diary study need?
Fewer than a survey, because each respondent produces many entries. Size for the number of completed diaries you need, then recruit above that to allow for drop-off.
Should the diary be time-based or event-based?
Time-based prompts give more complete coverage. Event-based entries capture the moment but depend on the respondent remembering to log it. Choose based on whether completeness or timing matters more.
How do I keep respondents completing the diary?
Keep each entry under three minutes, set expectations at entry, remind at a consistent hour, and pay part of the incentive on completion.
How much should I pay for a diary study?
Pay for completion, split between entry and finish. The amount depends on the burden and the market. Explain the schedule up front so the payment feels fair for the task.
Can a diary study replace a survey?
No. Diary samples are small and self-selected, so they describe patterns within a group. A survey is still the right tool for a stable estimate of a population.
How do I analyse diary data?
Start with completion and drop-off, then plot the fixed questions across days per respondent, code the rotating question, and compare entry with exit. Report patterns, not population estimates.
Sources and further reading
- ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics — ESOMAR
- SuperJ panel methodology — how the respondents behind Hercules Works studies are verified
Ready to get real consumer insights?
25M+ consumer footprint. Free to start.