Ask ChatGPT for "the best running shoes for flat feet". Write down the brands it names. Ask again five minutes later. The list moves.
Nothing changed in those five minutes. Same model, same question, same morning. These assistants write a fresh answer every time, so a brand sitting near the edge of relevance drops in and out on its own.
A single reading is a sample
A rank tracker reads a search results page that mostly sits still. Check it Tuesday, check it Friday, and the page keeps its shape.
An AI assistant writes its answer new each time. We asked the same shopping questions several times inside one morning and compared what came back. Brands that almost always appear still didn't appear every time. Brands sitting on the edge came back closer to a coin flip.
Treat one of those readings as the answer and you'll report that a brand doubled its AI visibility on a morning when nothing happened.
Each assistant is noisy in its own way
On some assistants the answers vary inside the moment. Ask five times before lunch and you'll see a spread, so asking repeatedly and averaging settles it.
Others behave differently. Their answers inside one morning agree almost perfectly, then the whole day moves together. The assistant is consistent, and tomorrow it is consistently something else. Repeating the question five times in a row teaches you nothing there. You need readings spread across days.
Sample every assistant on one schedule and you'll get both cases wrong.
The contested questions keep moving
The sharpest finding came from watching the same questions over weeks. Queries we picked because they were contested had often settled by the time we looked again. The brand was either in the answer for good or out of it for good. Questions that had held steady had drifted into contention.
So the thing you measure won't hold still either. A quarterly snapshot of AI visibility describes a set of answers that already moved.
What to do about it
Ask more than once. Report a brand that appears half the time as a brand that appears half the time. Watch the questions that move more closely than the ones that sit still.
Measure it naively and you'll report movement that never happened. Lantern watches these answers every day and reports them as tendencies.
Ask twice. The second answer is the one that teaches you something.
Common questions
Why do AI assistants give different answers to the same question?
They write each answer fresh instead of reading a stored ranking. Small differences inside that process change which brands reach the list, and brands near the edge of relevance move the most.
How many times should I ask before I trust the result?
More than once, and the right number depends on the assistant. Some vary inside a single morning, so readings taken close together help. Others hold steady all day and then shift, so readings spread across days matter more.
My brand appears in half the answers. Did my AI visibility drop?
No. A brand near the edge of an answer set moves in and out on its own, so the honest reading is that it appears about half the time. Confirm a real drop across many readings before you act on it.
How often should I check?
Often enough to see the questions move. The contested set changes over weeks, so a quarterly check reports answers that already changed. Watch the questions that decide purchases in your category more closely than the rest.