- A single prompt, run once, is close to a coin flip. SparkToro and Gumshoe.ai's ~3,000-run study found under 1-in-100 odds that two runs of the identical prompt return the identical brand list.
- The University of St. Gallen's 45-day academic study (arXiv 2604.07585) put a number on the minimum sampling needed: at least 7 runs per prompt per day, and a 2–4 week rolling window before trusting a trend.
- IAB's own published measurement framework sets a hard floor from the standards side: fewer than 50 queries across an entire measurement program counts as merely exploratory, not even directional.
- What's actually stable, per SparkToro's data, is appearance frequency across a large batch of runs (55–77% for top brands, regardless of prompt phrasing), not the exact brand list or exact wording of any single run.
- The practical formula: enough prompts to cover real buyer intent, enough runs per prompt per day to average out daily noise, and enough weeks to separate a real trend from the platform's own week-to-week churn.
"How many prompts do I need to track?" gets asked far more often than "how many times do I need to run each one, and for how long?" The second question is the one two separate, disclosed-methodology studies have actually answered. AI visibility score covers what a score is and why a single one is unreliable; this piece is the specific arithmetic behind that claim, pulled directly from the two studies that measured it.
What did the St. Gallen convergence analysis actually recommend?
Researchers at the University of St. Gallen (Julius Schulte, Malte Bleeker, and Philipp Kaufmann) ran a 45-day measurement of ChatGPT, Perplexity, Gemini, and Google AI Mode, published as "Don't Measure Once: Measuring Visibility in AI Search (GEO)". Two findings anchor their recommendation: cited sources turned over roughly 65% from one day to the next for the identical brand and category, and identical prompts re-run at the same moment overlapped only 32–43%.
Their own convergence analysis translates that instability into a specific rule: trust a per-brand detection rate only once you've run at least 7 runs per prompt per day, and trust a trend only across a rolling window of 2–4 weeks. Both numbers come from the same paper, aimed at the same underlying problem: a measurement this noisy needs enough samples, taken over enough time, before a single number means anything.
Does SparkToro's ~3,000-run study point to the same conclusion?
SparkToro's Rand Fishkin and Gumshoe.ai's Patrick O'Donnell ran a different kind of study: 600 volunteers running 12 identical prompts through ChatGPT, Claude, and Google's AI surfaces, each prompt repeated 60–100 times per platform: nearly 3,000 runs in total. Two results, pointing in opposite directions:
- A single run is close to random. The odds of two runs of the identical prompt returning the identical brand list: under 1 in 100. The odds of the identical list in the identical order: under 1 in 1,000.
- Frequency across many runs is not random. Top brands in a category appeared in 55–77% of responses regardless of how the prompt was phrased, a real, stable signal, visible only once you're looking at dozens of runs rather than one.
That's the same shape of finding as St. Gallen's, arrived at independently: the exact output of any single run is close to noise, and the thing that's actually measurable is a frequency built from many runs, not a snapshot.
What does IAB's published framework add to this?
IAB's "Measuring Visibility in the AI Era" framework, published in August 2026, sets a floor from the standards side rather than the academic side: fewer than 50 queries across an entire measurement program counts as merely exploratory, not even directional, let alone decision-grade. Its quality-tier criteria go further: directional measurement should cover at least two of four intent types (informational, comparison, recommendation, transactional) with the distribution disclosed, and should be tested monthly or quarterly at minimum. Decision-grade measurement (the tier appropriate for a budget decision or an agency performance review) requires all four intent types, weekly or more frequent testing, and a documented variance range within a 7-day window.
Put next to the two academic studies, IAB's framework isn't a competing number: it's the same underlying instability, translated into a disclosure standard a buyer can actually check a vendor against.
What's the actual arithmetic to use?
Combining all three sources into one working formula:
| Dimension | Floor, per the research | Source |
|---|---|---|
| Runs per prompt, per day | At least 7 | University of St. Gallen convergence analysis |
| Measurement window before trusting a trend | 2–4 weeks rolling | University of St. Gallen convergence analysis |
| Total queries across the program | At least 50 to clear "exploratory"; more for directional or decision-grade | IAB Measuring Visibility in the AI Era |
| Testing cadence for decision-grade use | Weekly or more frequent | IAB Measuring Visibility in the AI Era |
| What's actually stable across many runs | Appearance frequency (55–77% for top brands in SparkToro's data), not exact wording or exact list | SparkToro / Gumshoe.ai |
None of these numbers are interchangeable with "the more the better": a prompt set built from real buyer language (covered in full in building a fixed prompt set) run 7+ times a day for 2–4 weeks tells you something real. The same number of runs spread across a single afternoon, or a huge prompt list run once each, doesn't. It just looks more thorough.
What does this mean for a report you're about to trust?
Before treating any AI-visibility number (a vendor's or your own) as a real reading rather than a single frame of a noisy signal, check it against the arithmetic above: how many runs per prompt, over what window, and how many total queries the whole program is built on. A number that can't answer those questions isn't necessarily wrong, but per IAB's own framework, an unwillingness or inability to disclose them should be treated as the answer, not a gap to give the benefit of the doubt on.
Frequently asked questions
What's the minimum number of runs per prompt before I can trust a detection rate?
Per the University of St. Gallen's own convergence analysis (Schulte, Bleeker & Kaufmann, arXiv 2604.07585): at least 7 runs per prompt per day. Below that, a detection rate is one sample from a distribution the same paper found overlapping only 32–43% between identical, simultaneous runs, not a stable reading.
How long a measurement window do I need before trusting a trend?
At least 2–4 weeks, per the same St. Gallen convergence analysis. A single day's data, even with 7+ runs per prompt, tells you about that day: the paper's own recommendation is a rolling window before treating any movement as a trend rather than noise.
Does SparkToro's ~3,000-run study tell you the same thing as the St. Gallen study?
Yes, from a different angle. SparkToro and Gumshoe.ai ran 12 prompts 60–100 times each across ChatGPT, Claude, and Google's AI surfaces (nearly 3,000 runs total) and found the odds of two runs of an identical prompt returning an identical brand list at under 1 in 100. But appearance frequency across the full set of runs was stable: top brands appeared in 55–77% of responses regardless of phrasing. Both studies converge on the same practical rule: single runs are close to random, repeated sampling is not.
Is there a single "right" number of prompts, runs, and weeks?
No universally validated number, but a defensible floor exists by combining both studies with IAB's published quality tiers: at least 7 runs per prompt per day, a 2–4 week rolling window, and (per IAB's Measuring Visibility in the AI Era framework) at least 50 queries across the whole program just to clear the bar for "directional," with decision-grade measurement requiring weekly or more frequent testing and a documented variance range.
Sources referenced
- Schulte, Bleeker & Kaufmann, "Don't Measure Once: Measuring Visibility in AI Search (GEO)" (University of St. Gallen, 45-day study across ChatGPT, Perplexity, Gemini, Google AI Mode)
- SparkToro / Gumshoe.ai AI recommendation consistency study (Jan 2026, ~2,961 prompt runs)
- IAB: Measuring Visibility in the AI Era (Aug 2026)
This piece is the practitioner arithmetic behind the volatility research in AI visibility score and the KPI foundation in AI visibility KPIs. For how to build the actual prompt set this cadence should be applied to, see building a fixed prompt set. For a worked example of a check that deliberately runs fewer times than this floor (and why that's a stated scope limit, not an error), see what is an AI Shortlist Scan?
