# Study 9 strengthening (ChatGPT local recommendations vs Google Maps). Frozen before the second wave was collected # (2026-09-27). Observational only: no content interventions exist in this study. seed: 20260927 bootstrap_reps: 2000 wave1: date: "2026-09-26" runs: [1] wording: original # "Who is the best {service} in {city}?" wave2: date: "2026-09-27" instrument: "DataForSEO LLM Scraper, ChatGPT consumer app, location = country (same as wave 1)" maps: "DataForSEO Google Maps SERP, depth 20, keyword 'best {service} in {city}', location = country (same as wave 1)" original_runs: [2, 3, 4] # same wording as wave 1, three runs minutes apart variants: # one run each; the type says what the variant changes p1: {type: semantic_paraphrase, template: "Which {service} in {city} would you recommend?"} p2: {type: semantic_paraphrase, template: "I need a good {service} in {city}. Who should I go to?"} p3: {type: list_format, template: "List the top {services} in {city}."} p4: {type: source_required, template: "Who is the best {service} in {city}? Please cite your sources."} # Services as asked (US uses "physical therapist" for physiotherapist, as in wave 1) plurals: dentist: dentists plumber: plumbers personal injury lawyer: personal injury lawyers accountant: accountants physiotherapist: physiotherapists physical therapist: physical therapists # Not testable in this study (documented, not simulated): content interventions, placebo/length controls, # fidelity-constrained interventions, retrieval and context inclusion (the consumer app exposes neither), # cross-engine field comparison (Gemini's capture returns text only, no structured business listings). # Added 2026-09-28, before the third collection. Purpose: separate run-to-run variation from day-level variation # (not to claim that ChatGPT changed). Same instrument and settings as wave 2. wave3: date: "2026-09-28" start_utc: "2026-09-28T16:15:00Z" # 24 h after wave 2 and at the same time of day, so day and time of day are not confounded original_runs: [5, 6, 7] # original wording only, three runs maps_pulls: [a, b] # two Google Maps pulls the same day (a before, b after the ChatGPT runs): same-day Maps baseline raw: cite/data/raw/day3_localrec/{llm,maps_a,maps_b} # Analysis revisions decided 2026-09-28 before the third collection was analysed: # - entity resolution is branch-aware: a shared registrable domain counts as the same business only when the street # key (number + first street word) agrees or one side has no address; otherwise "same company, other branch" # - wording-sensitivity outcomes: list shown, cites any source, business-set overlap with the original wording # (vs the original-vs-original baseline), identical-link share; the Maps-top-20 share is dropped (Maps query unchanged) # - coverage is reported in both directions with explicit denominators; citation traceability keeps an # "undetermined (page not fetched)" category in the denominator # Deviation log (after the model coders coded the targeted sample, before any human labels): # - 2026-09-28: the postcode of a Maps entry is now read from its full address text first, because Google's # structured zip field is sometimes wrong (item H030: zip "1811" = street number). General fix, not item-specific; # it moved 8 of 12,271 listings from other_branch to confirmed. Found through a validation item, so disclosed. # - 2026-09-28: the third collection (wave3) was cancelled by the study owner before it ran; no day-3 data exists. # The study ends with two dates (26 and 27 September).