Survey Research Brief & Goals India: Turn Raw Prompts into North Star Plans

The brief starts with a hypothesis, not a heading. When your prompt hits the Goal Formulation agent (FORMULATE_GOALS intent), Hercules does not copy your words into a "project description" box. It reformulates them into formulated_goals — a typed array stored in survey_chats.formulated_goals and survey_versions — where each goal is phrased as "Measure X" with an explicit decision it enables, not "Understand X." That subtle shift forces testability: "Measure willingness to pay for 250ml vs 500ml among NCCS B mothers in Tier 2" can be answered with a Van Westendorp module and a Gabor-Granger ladder; "Understand pricing feelings" cannot. The agent applies MECE (mutually exclusive, collectively exhaustive) coverage so goals do not overlap and no stated decision is left without an evidence module.

The brief starts with a hypothesis, not a heading. It also runs blind-spot detection — adding an "Unknown Unknowns" goal the user missed (e.g., a control question on trust when you only asked about price) — and tags each goal with see_why traceability: why it was chosen, which framework influenced it, and which alternatives were rejected. This mirrors the adaptive analysis discipline in services/templates/goal_building_template.py with BRIEF_GENERATION_RULES, and it borrows the So What test that later drafting will enforce: if a goal cannot answer "so what does this mean for the business?" with a concrete threshold, it is rewritten. For the methodology that constrains which goals are legitimate, see strategic research framework India with its study-type taxonomy, and for the design that will serve those goals, see hercules survey creation process India.

audience_payload, chat_state and versioning — the brief is not floating text. Alongside goals, the agent mints audience_payload (INCCS classes A1–C1, city tiers Tier 1 (7 metros), Tier 2 (60+ cities), Tier 3 (30+ towns), age 18-24/24-35/35-45/45-55, behavioural keys like "Quick Commerce User":100) with sample constraints total in [100,400] (or General Population with checkCity:false) so targeting is enforced by SuperJ before the survey starts and demographics are never re-asked in the questionnaire. The payload and brief flip chat_state to GOAL_FORMULATION, unlock CONFIRM_AND_GENERATE, and write a chat_turns row plus a survey_versions snapshot so every edit is versioned. The split-AI-call pattern also applies here: one Gemini call generates structured goals/payload, another writes the conversational summary, fused via asyncio.gather to avoid commentary contaminating JSON.

What "good" looks like before you generate questions. Before question_generation runs, the brief already names per-module contract: construct being measured, method (e.g., MaxDiff for 12 claims, Van Westendorp for price range, semantic differential for imagery), who should answer it (e.g., category users vs brand-aware filter), routing need (screener → core → closing), and question-budget consumed (target 8–12 minutes, 15–25 questions, hard ceiling 20 minutes). It also pre-declares sample implications per cell in the brief only — never in the framework step — so when you later choose Sequential Monadic for 3 packs, you see n per cell and total n before fielding. Alternative designs (2–3 options with one-line rejection reasons) are presented for override, satisfying the no-default principle from the methodology catalog's anti-defaults. If you paste an existing questionnaire instead of a prompt, classification detects PARSE_QUESTIONNAIRE, extracts questions, and still synthesises formulated_goals around it rather than skipping strategy.

The 5-context spine — brief is one of five sources, but the one that leads. Poseidon's report pipeline is built from five context sources stored across PostgreSQL and Parquet: 1) Research Brief (survey_chats.research_brief — the why, decisions and hypotheses), 2) Survey Version / Questionnaire (survey_chats.survey_version — question text, types, options, scales), 3) Response Data (data/{chat_id}.parquet — what people said, DuckDB-on-Parquet columnar queried at 10–20× over Pandas), 4) Audience Payload (survey_chats.audience_payload — who was targeted by NCCS/tier/age/city), and 5) Survey Intelligence (computed at runtime — routing pairs via null-correlation, goal→question mapping, demographic axes). Among these, the brief is injected into five pipeline nodes as the North Star, a design documented in survey_and_analytics/analytics_process.md §8 and traceable to api/graph/report_generator nodes. Without this spine, reports become questionnaire dumps ordered by question number; with it, they become decision-driven narratives ordered by goal importance.

Planner → Insight → Narrative — the brief decides what matters and what it means. First, the Report Planner (survey_intelligence + report_planner + analytics_planner nodes) parses questionnaire, identifies types/demographics/linkages, maps each formulated_goal to specific questions via LLM, and creates an ordered task list that elevates key-metric questions and pushes housekeeping to the appendix. Second, Insight Generation extracts per-question and cross-question findings into structured insight packs with severity ratings, always framing each finding against the brief's decision context rather than reporting "% liked it" in a vacuum. Third, Narrative Synthesis writes three connected outputs — routing-conditional insights (with correct denominators from Survey Intelligence), one goal narrative per stated objective, and demographic threads that trace consistent segment patterns across goals — each tagged by Goal Traceability so coverage can be audited. Because fielding was on the SuperJ app with 60-90%+ completion among verified Indians answering for rewards, insights have the base sizes (n per cell, per city tier, per NCCS) declared in the brief, making "among Tier 2 mothers who are category-aware (n=340)" a verifiable clause, not a guess.

Drafting → Audit — Research Context in every section prompt, checked twice. Fourth, Report Drafting receives research_brief as "Research Context" in every section prompt (alongside survey_version, demographics and survey intelligence) and writes each section chunk-by-chunk with chart tokens {{CHART_task_1_overall}} and structured HTML classes; WeasyPrint later renders PDF, Plotly renders HTML/PNG, and Markdown remains canonical. Fifth, Section Audit and Safety Check both re-inject the brief: Section Audit runs a deterministic claim verification (numbers re-derived via DuckDB) plus an LLM writing-quality audit that checks hallucinations, jargon, causal overreach, and whether the section actually answers the brief; Safety Check validates PII leaks, numeric mismatches and coverage gaps. Both have retry loops (max 2 cycles: NEEDS_REVISION → redraft → re-audit). Report Validation then confirms structural completeness and Report Finalisation assembles cover, chart placeholders, and outputs. If the brief said "decide whether to premium-price by ₹40", the audit will flag a section that merely reports CSAT without linking to price intent — this is the North Star enforcing relevance at machine speed. Explore the engine that executes this spine at poseidon analytics engine and its 5-phase/18-node design at automated research report India.

Implication → evidence, never the reverse. Every paragraph Poseidon drafts must satisfy the So What rule: state BUSINESS IMPLICATION first, then cite evidence (%, number, N, confidence). BAD: "65% prefer digital platforms." GOOD: "Digital platforms are the dominant channel — 65% prefer them (n=420, 95% CI), making mobile-first features the highest-ROI investment for acquisition." This is the single biggest reason Poseidon narratives read like a senior analyst, not a data dump, and it is enforced by prompts and audits, not wishful thinking. The drafting LLM is instructed to structure claims as decision → metric, and the Self-Critique node (score 0–10, threshold-triggered rewrite) reviews narratives against verified numbers, checking unsupported claims, internal contradictions, causal language where only correlation exists, and missing base sizes. Combined with three-layer verification (Numerical Claim Verifier at 0.5 tolerance via fresh DuckDB SQL, Output Validator with sum-to-100 and NPS [-100,100] bounds, Self-Critique at 99.1%/97.4%/94.8% pass rates), the So What rule guarantees that leadership sees recommendations with receipts.

Five context sources → one narrative, with routing-aware denominators. The So What rule only works because all five context sources are present. The Response Data lives as Parquet (columnar, 10–20× over Pandas) so DucthDB reads only the 2–3 columns needed per aggregation (<5% I/O for a 50k×40 survey); the Survey Intelligence node auto-detects routing pairs via DuckDB null-correlation, supplying the correct WHERE clause so "among theatre-goers (n=40), 62% prefer butter popcorn" does not divide by the full n=200. Audience Payload supplies NCCS/tier/age quotas that make segment claims like "NCCS A Tier 1 vs Tier 2 intent gap of 14 pp (p<0.01, chi-square)" testable.

Five context sources → one narrative, with routing-aware denominators. Survey Version supplies scale bounds so NPS uses (Promoters−Detractors)/Total×100 with correct 0–10 cuts, and semantic cache (0.88 embedding threshold, 78% hit, <200ms) ensures repeat So What answers are instant. The result is an 18-node report pipeline that delivers 20–50 pages in Markdown/HTML/PDF with cover, TOC, executive summary, goal-wise analysis, cohort insights, strategic synthesis, and appendix — each chart chosen by a chart-advisor LLM (horizontal bar for NPS, grouped bar for batteries, donut for binary, heatmap for cross-tabs, histogram for continuous, funnel for funnels) in an Okabe-Ito colourblind palette. This is why a brand manager in Mumbai and a founder in Hyderabad hear the same clarity: implication first, evidence attached, decision ready. Pricing is Free ₹0, Starter ₹1,119 (₹895 annual), Pro ₹30,000/quarter (₹24,000 annual) with 20% annual off and 100 free first-month responses.

Survey Knowledge Graph and routing pair detection — denominators always correct. Two context sources deserve special attention because they fix the most common survey analytics errors. Survey Knowledge Graph (Question→Column maps_to, Column→Formula formula, Question→Question routed_from for skip logic, Formula→Demographic slice_by) is built once at ingestion and traversed for every query, with edge confidence EXTRACTED/INFERRED/AMBIGUOUS, so the engine knows which column type is valid for which formula without the LLM inventing math. Routing pair detection complements it: Survey Intelligence auto-discovers parent→conditional edges via null-correlation, so querying a conditional question automatically applies the right filter and reports the correct base size. Poseidon never does narrative arithmetic — the LLM assembles SQL from the graph-resolved columns/formula and DuckDB executes it, then the Numerical Claim Verifier re-derives every number via fresh SQL. This architecture, called schema-first formula-driven analytics, is the core differentiator cited in analytics_process.md and explains why conditional insights are not "computed across the full sample" somewhere offhand. See the engine loop at poseidon analytics engine and the graph lens at survey knowledge graph.

Language, sentiment and theme — 8+ languages, not translated English. The remaining fidelity layer is language. Survey creation in Hercules supports 8+ Indian languages natively (Hindi, Tamil, Telugu, Bengali, Marathi, Gujarati, Kannada, Malayalam, Punjabi) with Indic media acceptance via PyMuPDF/docx/pptx/openpyxl/Pillow extraction and document classifier routing (questionnaire vs discussion guide vs methodology note). Poseidon's analytics therefore operate on structured columns regardless of prompt language, but open-text in any of those languages flows through the qualitative path: theme clustering, sentiment in 8+ languages, and a visual chart advisor that labels directly instead of legend-heavy defaults. Semantic cache respects Parquet schema hash and row-count fingerprint invalidation, so stale language models do not linger after a new wave. Combined with ZK-verified SuperJ respondents (Zero-Knowledge-Proof uniqueness, zero bots) and DPDP-ready storage in Google Cloud Mumbai, the brief→report chain preserves Indian phrasing fidelity rather than flattening Tanglish/Hinglish into English buckets. That is why the same insight reads differently and correctly in Hyderabad versus Haridwar. Pricing is Free ₹0, Starter ₹1,119 (₹895 annual), Pro ₹30,000/quarter (₹24,000 annual) with 20% annual off and 100 free first-month responses.

India targeting lives in the brief, not the questionnaire — enforced on SuperJ before fielding. Audience NCCS A1–C1, Tier 1/2/3 cities, age 18–55 and behavioural traits (e.g., "Quick Commerce User") are declared in audience_payload and enforced by SuperJ before the survey starts, so the SuperJ app at 60-90%+ for rewards never re-asks demographics. Language (Hindi, Tamil, Telugu, Bengali, Marathi, Gujarati, Kannada, Malayalam, Punjabi) and Indic media are declared here too, ensuring Poseidon's qualitative path codes verbatims faithfully, the five contexts stay consistent from brief to report in Hyderabad with DPDP-ready governance, and the Survey Knowledge Graph traverses rather than guesses.

Operational dividend: less re-discovery, more compounding. When every study starts from formulated_goals and ends with goal narratives that tag coverage (Goal Traceability) and cross-section synthesis, brands stop re-briefing the same segmentation each quarter and re-discovering the same insight from FINAL_final_v3.xlsx. Instead, each new brief extends the same evidence spine: Poseidon suggests audiences (e.g., "expand Tier 2 B sample by 40 after last wave's 2.4× variance"), hypotheses ("test COD trust as driver after churn pack flagged it at p=0.02"), and modules ("add Gabor-Granger ladder since Van Westendorp showed narrow acceptable range"). Because SuperJ targeting (NCCS, city tier, age, behavioural) is enforced before the survey starts and routing (business_flow, routing_logic, compiled next/showIf) is deterministic, repeat waves are comparable wave-over-wave without a three-week merge. Clients see median chat answers in ~1.8s (cache <200ms) and comprehensive reports in the 20–50 page family, streamed with SSE progress (connected/agent_enter/heartbeat/final/done) that persists across refresh. The free plan lets teams prove the loop end-to-end before committing to Starter or Pro annual savings.

How to write a brief that earns its keep in 2026. Keep it conversational but include four elements Hercules will formalise for you if you miss them: decision (what will you do differently), population (who decides — NCCS/tier/city/age), stimulus (any concept/pack/ad/price to test), and success threshold (what would make you act). Example: "Should we launch the masala variant at ₹99 in Tier 2? Compare with the current ₹89 pack among NCCS B/C, need top-2-box purchase intent and an acceptable price range." Hercules will classify study type (Concept Screening + Pricing Research), choose Sequential Monadic + Van Westendorp, allocate HOOK→CORE→CLOSE, enforce 8–12 minute burden, and present 2–3 alternatives with rejection lines. You confirm, we compile, SuperJ fields on the SuperJ app for rewards, Poseidon verifies. That is the North Star in action: a business sentence, correctly structured, guiding every downstream artifact from flow_key to final PDF. Built in Hyderabad by Jupiter Meta Labs, it is why teams say the brief feels less like paperwork and more like a strategy partner that remembers everything. Pricing is Free ₹0, Starter ₹1,119 (₹895 annual), Pro ₹30,000/quarter (₹24,000 annual) with 20% annual off and 100 free first-month responses.

The brief as institutional memory for longitudinal compounding. Because formulated_goals and audience_payload are versioned in PostgreSQL, the next wave extends the same spine — Poseidon suggests hypothesis and sample tweaks from prior evidence rather than re-briefing. Built in Hyderabad by Jupiter Meta Labs, Hercules Works therefore compounds learning wave over wave; reports stay decision-ready with the SuperJ app tier/language aware and without a three-week merge. This memory, plus goal traceability and cross-section synthesis, is why repeat clients see faster decisions and fewer re-briefs every quarter. This memory, plus goal traceability and cross-section synthesis, is why repeat clients see faster decisions and fewer re-briefs every quarter on Hercules Works via the SuperJ app at 60-90%+ for rewards.