TL;DR

A prompt-level SEO experiment is a controlled test that measures whether a specific change (schema markup, content rewrite, entity clarification) actually shifts how LLMs like ChatGPT, Perplexity, or Gemini mention your brand. Without a rigid hypothesis framework, you are measuring noise, not signal. Jason Tabeling recently outlined the core methodology in Search Engine Land. We break it down, extend it with four gotchas he did not cover, and offer a free experiment template you can use today.

Key Takeaways

One variable per experiment keeps results interpretable and reproducible
Model updates invalidate past baselines because GPT-4.1 and GPT-4.2 may weight sources differently
Prompt drift is real: the same prompt returns different answers on different days, so run 5-7 repetitions across multiple days before drawing conclusions
Document everything: hypothesis, premise, theory, variable changed, model version, prompt set, number of runs, dates, and outcome
Non-English markets face higher prompt drift, meaning you need more repetitions to establish a reliable baseline

The typical AI visibility workflow looks like this: add schema markup, rewrite a product description, publish an llms.txt file, then check Perplexity a week later and declare victory if the brand shows up. The problem is obvious once you say it out loud. Nothing was isolated. Nothing was measured before and after under controlled conditions. The "improvement" might be a model update, a prompt drift artifact, or pure coincidence.

Jason Tabeling, writing in Search Engine Land on May 8, 2026, calls this the methodology gap. Brands invest serious budgets into Generative Engine Optimization programs but have no rigorous way to know if their changes did anything. His proposed fix: a hypothesis framework borrowed from scientific experimentation.

The Four Pillars of Prompt-Level Testing

1. Hypothesis documentation. Before touching anything, write down: what you expect to happen, why you expect it (the premise), the underlying theory, and what outcome would confirm or reject the hypothesis. Even when the world changes (a model update, a competitor move), these artifacts tell you which part of your reasoning broke.

2. Model update awareness. When OpenAI ships GPT-4.1 to GPT-4.2, your baseline is gone. Re-test. Record the exact model version. If you are using the OpenAI API, log the system_fingerprint parameter. For Perplexity, capture the model version header.

3. Prompt drift mitigation. Ask ChatGPT the same question twice in the same day and you may get different answers. This is not a bug; it is stochastic generation. The fix: run each prompt 5-7 times across 3 or more separate days before averaging results into a baseline. Single-run "tests" are anecdotes.

4. Variable isolation. Change one thing per experiment. If you simultaneously updated your JSON-LD, rewrote the homepage H1, and added three FAQ entries, you cannot attribute a citation lift to any single change. One change, one measurement cycle.

Four Gotchas the Original Framework Misses

Tabeling's framework is solid for English-language, US-market testing. Cross-border and multi-language scenarios introduce extra variables.

Geo bias in prompts. ChatGPT returns different answers depending on inferred user location. A prompt about "best CRM for small business" from Prague surfaces different brands than the same prompt from San Francisco. Lock the location variable via system prompt or API parameter, or your test results will be location-dependent noise.

Conversation memory pollution. If you test in a ChatGPT session that has prior conversation history or memory enabled, the model carries forward context from earlier exchanges. Always test in a fresh session with memory disabled.

Plugin and browse mode interference. When ChatGPT's browsing or plugin features are active, you are testing live web retrieval, not the model's parametric knowledge. Disable both unless retrieval behavior is explicitly what you are testing.

Model serving inconsistency. OpenAI and other providers sometimes A/B test different model revisions under the same model name. Two API calls to "gpt-4o" on the same day might hit different backend revisions. Log the system_fingerprint (OpenAI) or equivalent identifier so you can spot this post-hoc.

A Practical Experiment Template

Here is the structure we use internally for every GEO experiment:

Hypothesis: Adding FAQ schema to the /pricing page will increase Perplexity citation rate for "[brand] pricing" queries
Premise: Perplexity weights structured FAQ data for direct-answer synthesis
Theory: FAQ schema provides machine-readable Q&A pairs that reduce extraction effort for LLMs
Variable changed: FAQ JSON-LD added to /pricing (no other page changes)
Model + version: Perplexity (sonar-pro, build 2026-05-01); ChatGPT (gpt-4o, system_fingerprint logged per run)
Prompt set: 8 prompts, EN + CZ variants, covering pricing intent
Runs: 7 per prompt, across 4 days
Baseline: 3/56 runs cited brand before change (5.4%)
Outcome: 11/56 runs cited brand after change (19.6%), lift confirmed across both models

Automate the Measurement Layer

Running prompt-level experiments manually works for one-off tests. For ongoing GEO programs, you need automation: scheduled prompt runs, prompt drift detection, share-of-voice tracking across models, and alerts when a model update invalidates your baseline.

BeRecommended does exactly this. Set up your first automated GEO test in under 10 minutes, track citation rates across ChatGPT, Perplexity, and Gemini, and get notified when model updates shift your visibility.

FAQ

How many prompt repetitions do I need for a reliable baseline?

For English-language prompts, 3-5 repetitions across 2 or more days is a reasonable minimum. For non-English markets (Czech, Slovak, Polish), prompt drift tends to be higher due to smaller training data representation, so aim for 5-7 repetitions across 3 or more days.

Can I test multiple changes at once to save time?

No. Multi-variable experiments make it impossible to attribute results to any single change. If you must move fast, run sequential single-variable experiments with short measurement windows rather than bundling changes.

What happens when a model updates mid-experiment?

Your baseline is invalidated. Re-run the baseline prompts on the new model version before comparing against post-change results. This is why logging the exact model version on every run matters.

Does this framework apply to Google AI Overviews too?

Yes, with a caveat. AI Overviews and AI Mode are isolated systems with different citation logic, so test them as separate channels rather than aggregating results.

Sources

Jason Tabeling, "How to run prompt-level SEO experiments for AI search," Search Engine Land, May 8, 2026
OpenAI API Reference: system_fingerprint parameter documentation
BeRecommended internal experiment data, Q2 2026

Prompt-Level SEO Experiments: The Testing Framework Your GEO Program Is Missing