Hercules Survey Creation Process: End-to-End AI Pipeline from Chat to Live in India

Intent classification and the 4-state chat machine — the front door has to be deterministic. Every message you send hits POST /api/chat, the single stateful orchestrator for Hercules Works. Before any LLM is called, a deterministic-first router in services/ai/router.py runs regex pattern checks — question count, option lists, "show if / skip to" markers — to detect pasted questionnaires versus vague requests, then falls back to Gemini classification only when needed. That protects cost and prevents misrouting. The router decides among six specialist agents and moves the chat through four explicit states tracked in survey_chats.chat_state: INITIAL (new chat, no framework yet), RESEARCH_GUIDE (AI asks clarifying questions to propose a framework), GOAL_FORMULATION (research brief and audience payload exist, user can confirm or refine), and QUESTIONS_GENERATED (questions exist, you can edit, add routing, or deploy). Each state gates allowed intents — so in INITIAL you get GUIDE_RESEARCH_SETUP, PARSE_QUESTIONNAIRE or GENERAL_CHAT, while in QUESTIONS_GENERATED you get EDIT_SURVEY, EDIT_ROUTING or FORMULATE_GOALS. This is why the experience feels guided rather than a free-form chatbot: the state machine knows exactly what it needs next, whether that is audience, decision, or stimulus.

Goal Formulation — turning a raw prompt into formulated_goals that steer everything. Once the research guide has enough context, the FORMULATE_GOALS agent generates the artefacts that become the North Star for the entire study. It writes research_brief as hypothesis-driven goals ("Measure X" not "Understand X") that pass the So What test — every goal must enable a business decision — and checks MECE coverage so no decision is left without evidence. Alongside it, the agent builds audience_payload (NCCS A1–C1, Tier 1/2/3 cities, 18-24/24-35/35-45/45-55 age buckets, behavioural traits like "Quick Commerce User"), a see_why trace that explains why each goal was chosen, and CTA buttons for confirmation. These payloads are persisted to survey_chats (research_brief, audience_payload) and a chat_turns row, with a survey_versions row versioning the snapshot. When you confirm, the split-AI-call architecture fires two parallel Gemini calls via asyncio.gather — question_generation_questions for the JSON array and question_generation_message for the conversational reply — so question content and commentary never contaminate each other, latency drops, and token usage is attributed per call.

Guest sessions, auth merge and tier discipline — start free, no signup friction. A first-time visitor at hercules.works/ai can send a message with no account: the backend creates a guest user, returns a guest_id that the frontend stores and sends as X-Guest-ID on follow-ups, and allows research guide, goal formulation and questionnaire parsing immediately. Final question generation requires login, at which point POST /api/chat/claim-guest-session merges guest chats and token usage into the real user and deletes the guest record. Auth itself is lenient by design: dragonAccessToken + dragonRefreshToken cookies via Hercules, middleware tries local JWT decode → Hercules refresh → session validation, refreshed cookies are forwarded with domain=.hercules.works and secure/httponly/lax flags, but route dependencies decide enforcement. Tier service caches tier data from Hercules on interval, so FREE guests who exhaust free chats receive exhausted:true and require_auth:true rather than a silent failure.

Layer 1 → Layer 2 → Layer 3, with INDIA_SURVEY_DESIGN_RULES as guardrails. Once goals exist, Hercules does not pick a "standard" design. The methodology_catalog.py is the single source of truth: STUDY_TYPE_TAXONOMY (Layer 1) classifies the study type first — U&A, Brand Health/Tracking, Concept/Idea Screening, Product Testing, Packaging/Design, Advertising Pre-test, Campaign Tracking, Pricing Research, Feature/Message Prioritization, CSAT/CX/NPS, Churn, Segmentation, Market Sizing, Shopper Path-to-Purchase, Public Opinion, Employee, or Academic Exploratory — because study type constrains which designs are even legitimate. RESEARCH_DESIGN_CATALOG (Layer 2) then lists equal-citizen designs: Single Monadic, Pure Monadic, Sequential Monadic with Latin Square rotation, Proto-Monadic, Reverse Sequential, Paired Comparison/Forced Choice, Side-by-Side, MaxDiff/BWS, CBC, Kano, TURF, Van Westendorp PSM, Gabor-Granger, Brand-Price Trade-Off, Linear structured, Modular funnel, Tracking wave, and semantic differential batteries.

Layer 1 → Layer 2 → Layer 3, with INDIA_SURVEY_DESIGN_RULES as guardrails. METHODOLOGY_DECISION_FRAMEWORK (Layer 3) reasons in order: Step 1 counts discrete stimuli (0 → no cells allowed, 1 → Single Monadic, 2–4 → Step 2, 5+ → MaxDiff/CBC/screening), Step 2 picks absolute vs relative read plus budget/cell-affordability, Step 3 overlays pricing/trade-off modules, Step 4 runs sanity checks (every decision mapped, burden 8–12 minutes / 15–25 questions, per-cell n stated in brief only, 2–3 alternatives with rejection reasons). INDIA_SURVEY_DESIGN_RULES then corrects for acquiescence and scale-use bias — prefer fully labeled 5-point unipolar over agree/disagree, force behavioral validation via trade-offs, use NCCS A1–C1 not Western income, tier-aware geography, simple English under 15 words for Tier 2/3, 8–12 minute mobile-first instruments on the SuperJ app where surveys are answered for rewards. You can see the downstream routing consequences of these choices on survey business flow routing India and the scale-integrity checks on survey quality rubric linter India.

Why anti-defaults matter — and what never gets bolted on. The catalog enforces hard anti-defaults that every SEO page must respect: never present Sequential Monadic as the default, never propose A/B or split cells when there are no stimuli, never force U&A or ad-testing frames onto public-opinion/civic/academic studies, never propose conjoint/MaxDiff for fewer than 5 items or simple ranking beyond 7. This is why a prompt like "tea vs coffee opinion poll among Tier 2 youth" correctly lands on a Linear structured questionnaire with balanced attitude statements and knowledge checks, while "test 3 pack shots for a new biscuit" lands on Sequential Monadic with randomization rather than a generic grid. The brief the user confirms already names the construct, method, who should answer, routing need, and question-budget per module — and invites override — so the later business_flow stage inherits explicit method justification instead of guessing.

business_flow — the analyst-readable strategy layer. After questions are generated, the business_flow_agent builds the high-level survey flow as JSON with questionnaire_sections (ordered sections with intents and objective mappings), branch_blocks (declarative branching sets), dynamic_lists (piped/carry-forward lists), and rejoin points where branched paths reunite. This is the document an insights lead reads to verify study design: it shows which module serves which formulated_goal, what branching logic is intended, and where demographic quotas (NCCS, city tier) gate entry. It is persisted as survey_chats.business_flow and also embedded in the chat_turns snapshot for versioning. Crucially, business_flow speaks in stable flow_keys — UPPER_SNAKE_CASE like BRAND_RECALL_UNAIDED, BRAND_AWARENESS_AIDED, MAIN_BRAND, NPS_SCORE — not in qIds, so adding or reordering questions does not break the strategy layer. Branch_block_expander then materialises branch_blocks declared once in flow_key terms into concrete per-question edges later, ensuring the business intent stays auditable even after compilation. For the canonical distinction between this layer and the compiled runtime, see survey business flow routing India, and for how India targeting is declared here, see tier 2 tier 3 consumer research India.

routing_logic — the abstract machine plan in flow_key vocabulary. In parallel, the route_builder_agent creates routing_logic, the abstract routing plan also expressed in stable flow_keys rather than concrete IDs. It declares rules, display_rules and dynamic_filters at the flow_key level — which section shows only if a prior condition holds, which option list is filtered from a prior selection, which block loops over a dynamic list. Because it is abstract, it survives question reordering: the compiler will later resolve BRAND_RECALL_UNAIDED → preview_question_7's concrete qId without the LLM rewriting logic strings. This two-layer split (business_flow for humans, routing_logic for machines) is documented in survey_and_analytics/build_process.md and is the reason Hercules can validate routing deterministically instead of hoping an LLM JSON is correct. The audience_payload declared earlier (e.g., NCCS {A1:50, A2:100}, "18-24":{total:75}, total in [100,400], checkCity:false for All India) is not re-asked in the questionnaire — demographic questions are explicitly forbidden for the LLM to generate — so routing here focuses on substantive qualification (category usage screener, brand repertoire filter) rather than quota fields.

Why this split prevents broken surveys. Keeping flow and logic abstract until compilation enables deterministic validation: validate_compiled_routes() checks duplicate/missing qId, unknown references, option selectors that are not valid indexes, branch targets missing showIf protection, loop and terminal hazards, filterFrom/filterOut source existence and option parity, rule coverage against routing_logic, and analysis-tag coverage against business_flow. If validation finds only filter mismatches, strip_filter_mismatch_fields() repairs and re-validates; if legacy routing is still broken, a routing_agent fallback regenerates; if all else fails, strip_invalid_routing() strips ALL routing and saves a linear survey — better linear than broken. This fail-safe ladder means no respondent ever hits a dead-end branch on the SuperJ app. Poseidon's later survey intelligence also benefits: it auto-detects routing pairs via null-correlation and knows conditional questions need denominator correction, a lineage that starts with the clean flow_keys authored here.

Question generation — typing, phasing and identity done right. The question_generation stage normalises everything to 10 canonical types via services/ai/parsers.py _TYPE_ALIASES — multiple_choice (mcq/single/multi/checkbox/select), yes_no (binary/true_false), number_rating (likert/nps/slider/star), ranking, vertical_ranking (grid/matrix), text (open_ended/free_text), dropdown, image_multiple_choice, image_question, video/audio, contact_info/email/phone/address/website, and survey_break — so free-text LLM outputs like "Likert" or "grid" map deterministically. Every generated survey obeys three-phase architecture: HOOK (~20% easy behavioural questions) → CORE (~60% mixed quant + qual) → CLOSE (~20% reflective/open-ended), with no back-to-back number_rating stacks beyond 2, open_ends at inflection points and always as the final question, MECE MCQ options with "Other" (is_explainable:true) and "None of the above" (deselect_all:true) where appropriate. Each question gets two identifiers via _ensure_question_identity_fields(): id = preview_question_1, preview_question_2… for frontend layout/progress, and qId = globally unique UUID for routing stability — so reordering or deleting questions never orphans a next or showIf. Shuffle (shuffle:true) is emitted only for categorical lists (brand lists, store choices) and never for ordered scales (number_rating, ranking, vertical_ranking, yes_no). For surveys >20 questions, 1–3 survey_break cards are auto-inserted at thematic transitions (21–30→1, 31–45→2, 46+→3).

The route compiler — flow_key → qId and the 8 runtime keys. With stable qIds assigned, compile_routing_logic() translates the abstract flow_key vocabulary into the concrete runtime fields the SuperJ execution engine reads. It emits up to 8 top-level keys per question (never nested): next (option-index → qId branch map), terminate (boolean or option-index array ending as disqualified), complete (boolean/array/"ALWAYS" for successful end), showIf (source qId → required option index/array for OR-within-question), showIfElse (skip/terminate/complete when showIf fails), showIfLogic (AND default or OR across multiple sources), requiredOptions (mustHaveAllOf/minCount/mustNotHave for multi-select validation), and filterFrom/filterOut (carry-forward source qId for cascading choices). These are top-level fields on the question object in questions_json — a common SEO error is to nest them under a routing object, which the runtime does not read. The 24 supported scenarios span skip, complete, multi-branch redirect, filter carry-forward, parallel show, First-Match for multi-select conflicts (ascending index wins, no PRIORITY map), conditional display (showIf single/multi AND/OR), termination, requiredOptions validation, cascading 3-level filters, exclusive "None"→terminate, and mixed skip/terminate on the same question. For a full map of this layer versus the abstract layers, see survey business flow routing India and the panel logistics at consumer panel India.

Post-compilation validation and hardening. Immediately after compilation, deterministic validation runs as described earlier; repaired or stripped routing is re-versioned into survey_versions.questions_json and survey_chats. Code sanitisation is implicit via the parsers — keyboardType numeric handling, maxChar caps for numeric text, and normalisation of flow_key → qId references — so what SuperJ receives is a tight, deterministic questionnaire rather than a prompt blob. This is also where the Poseidon lineage is seeded: survey_chats.survey_version stores the full questions_json that Poseidon will later read directly from PostgreSQL alongside business_flow, routing_logic, audience_payload and formulated_goals, without a second export step. That direct DB hand-off is what makes the later 5-context reconstruction paragraphs accurate rather than approximated.

Quality Linter — the 5-point rubric before any Indian sees the survey. Before deployment, the linter and compute_survey_quality_scorecard() run five checks that every Indian instrument must pass: accessibility/ethics/neutrality framing (no caste/religion unless explicitly requested and user-asked, no Aadhaar/PAN/UPI collection, no leading "Don't you think…" phrasing), scale balance (prefer fully labeled 5-point unipolar, balance positive/negative statements, randomise statement order), keyboard correctness (text questions get keyboardType defaulting to numeric where needed, with sane maxChar caps), option-count discipline, and India design rules (sentence length, tier-appropriate language, joint-decision framing). Alongside the rubric, SD-032→035 and SD-044 data-quality designs are armed: attention checks (instructional manipulation with known correct answers), speeder detection at 40–50% of median LOI, straight-lining/pattern detection on grids, open-end gibberish AI scoring, and ZK panel deduplication. Failures are scored, not silently ignored, so authors can fix rather than field a flawed instrument. You can read the full rubric deep dive on survey quality rubric linter India and the data-quality practices at best practices for improving data quality in online surveys.

SuperJ deploy — from Hercules draft to live on the SuperJ app. When you approve, the SuperJ service converts the internal questions_json to SuperJ's MongoDB payload via services/superj_service.py: it maps each canonical type to SuperJ UI types, preserves top-level routing fields (next/showIf/filterFrom…), handles keyboardType numeric mapping for text inputs, enforces shuffle only on eligible types, and registers/updates the SuperJ survey shell via SUPERJ_CREATE_SURVEY_AI_URL. Lifecycle status moves indraft → inreview (chat editing blocked) → approved → live (direct draft edits blocked) → paused/completed, with admin review via POST /api/admin/review-content/chat/{chat_id} classifying content as SAFE (approved), REVIEW_RECOMMENDED (approved with MEDIUM flags) or UNSAFE (rejected, blocks launch on HIGH severity like PII solicitation, government ID, caste-based, explicit/racist content, or missing required media). Quotas declared in audience_payload are enforced by SuperJ targeting before the survey starts; the questionnaire does not re-collect NCCS, city or age. Delivery is via the SuperJ app on mobile and web — verified Indians, 60-90%+ completion, where people answer surveys in exchange for rewards — not via WhatsApp or email blasts.

Completed → Poseidon ingest POST /api/v1/ingest/ — zero manual export. The moment an admin marks survey_status=completed, the backend creates an ANALYTICS chat_turns row and asynchronously POSTs to POSEIDON_INGEST_URL/api/v1/ingest/ with body {chat_id, super_j_survey_id} and header X-Secret-Key: <JWT_DRAGON_SECRET>, fire-and-forget with 200 OK to the admin (analytics_data_status flips to failed on timeout/non-200 for ops to re-trigger, but completed is preserved). Poseidon then reads business_flow, routing_logic, audience_payload, formulated_goals and questions_json directly from PostgreSQL, fetches respondent answer JSON from SuperJ, cleans into Parquet at data/{chat_id}.parquet for DuckDB-on-Parquet analytics, writes back parquet_path, analytics_row_count, analytics_column_count, and analytics_data_status='ready', and generates the markdown/HTML/PDF report into chat_turns.analysis_result and survey_chats.analysis_result. The 5-phase per-query pipeline (Initialization with semantic cache 0.88 threshold and 78% repeat-hit at <200ms, Routing by intent/complexity SIMPLE/MODERATE/COMPLEX, Analysis via Data Profiler/Column Selector/Quality Checker over the Survey Knowledge Graph, Verification via three-layer deterministic+structural+self-critique at 99.1% accuracy, Reporting with confidence score and Quick Suggestions) and the 18-node report graph then operate on that Parquet asset — all without the user exporting a CSV. See the engine at poseidon analytics engine and the language coverage at multilingual survey tool India.