A hypothetical store sells outdoor gear to consumers on Shopify. It meets the Catalog requirements, and some of its products show the Shop Promise badge. The owner, who has read about AI shopping, asks a fair question: is this working? Nothing in this scenario is real, and it is not a finding. It shows what to do once the eligibility question has been answered and the next one begins.
It helps to separate the kinds of evidence first. Eligibility is a documented rule: Shopify’s Catalog requirements page lists what keeps a store out, and the store appears to satisfy it. A ranking signal is a described mechanism. Shopify says badged products may rank higher and can receive a ranking signal, and the wording is conditional. An observation is something else again: what happens when a real buyer’s question is put to an assistant. The first two are in the owner’s hands already. The third has to be collected, and it is the only one that speaks to what a buyer saw.
Collecting it starts with the questions. A shopper does not use the store’s name or its internal categories. She describes a need, such as a lightweight waterproof jacket for a rainy hiking trip, a budget, and a constraint. Write several of these in shopper words, spread across the store’s product types, and fix them in advance. Fixing the set matters because the same set, repeated later, is how stability is assessed.
A small set is easier to run and to read, but it should cover different product types and different constraints, since an assistant may treat a budget question differently from a gift question or a durability question. Note what the set leaves out, so a result is not read as covering the whole catalog.
Next, choose the engines. A shopper might use ChatGPT, Claude, Gemini, or Perplexity, and each can answer the same question differently, so each should be kept as its own record. A blended figure can hide a store that appears on one engine and not another.
Run the set and keep the full text of every answer. The wording of a caveat often says more than the fact of a mention. For each answer, record whether the store’s products appeared and how: listed with no comment, listed with a caveat, listed after another seller is called a better fit, or recommended with a reason tied to what the shopper asked for. Note which sources the answer showed, if any, and whose they were.
Two measurement checks keep the record honest. Some responses can fail or go missing during collection, and those must stay separate from valid answers that did not mention the store. If the two are counted together, a collection problem can look like a finding about the store. A few missing responses do not automatically undermine a result, so the working check is whether enough of the planned responses succeeded to support what is being reported. The second check is repetition: running the same set later shows whether a result holds, while changed wording or new questions are a different diagnostic test whose results are not directly comparable.
Then comes the part that tends to be skipped, which is stating what the record cannot show. Shopify says AI agents get product information from Catalog. That is the platform describing its own system, and from outside, no one can see whether a given answer used it. A product appearing in an answer does not show that the badge caused it, since an assistant may draw on many sources. An absence does not show the badge failed. The record supports a narrow statement: for these questions, on these engines, on this date, the store’s products appeared or did not appear, and this is how they were presented. It does not show why. It also does not show what a buyer did next, because choice is an outcome the buyer produces and an answer alone does not reveal it.
Suppose the owner then decides to change something, for example by adjusting return rules so more products qualify for the badge. That is an intervention with an operational cost. Before making it, the owner should say what observation would count as an effect, such as the store’s products appearing in answers to the fixed questions on named engines, and then repeat the same set afterward. A before-and-after difference alone does not establish that the change caused it, because other things move at the same time.
Sometimes the evidence does not justify a change yet. The set may be small, the results mixed, or the owner may not yet know whether they hold. In our framework, WATCH is a legitimate result in that case. It means continuing to observe, setting a date to check again, and writing down what would change the decision. A store that can say that clearly is in a better position than one that changed its policies on the strength of a platform’s description.
FAQ
What should a store observe after it meets a channel’s eligibility rules? It should observe what a real buyer’s questions return on ChatGPT, Claude, Gemini, and Perplexity. Write fixed, shopper-worded questions, run the same set on each engine, and record whether the products appeared and how they were presented.
Why keep each engine’s results separate? Each engine can answer the same question differently. A blended figure can hide a store that appears on one engine and not on another.
How should I treat failed or missing responses? Keep them separate from valid answers that did not mention the store. A few missing responses do not automatically undermine a result, so check whether enough of the planned responses succeeded to support what you are reporting.
If my product appears in an answer, did the Shop Promise badge cause it? That cannot be concluded. Which sources shaped a given answer is not observable from outside, and a before-and-after difference alone does not establish cause.
When is waiting the right decision? When the evidence does not yet justify a change. In Axis Suite’s framework, WATCH is a legitimate result: keep observing, set a date to check again, and write down what would change the decision.
Is this article reporting a result from Axis Suite? No. The store and the scenario are hypothetical, and the article describes a method without reporting a result for any brand.
About Axis Suite Axis Suite is the independent intelligence layer that tracks what AI says about your brand and the observable evidence behind it, retrieval, narrative, recommendation, and corroboration signals, across engines including ChatGPT, Claude, Gemini, and Perplexity. Built by TrendAxis, Axis Suite works from a framework that separates Mentioned, Cited, Recommended, and Chosen. Axis Suite is measurement infrastructure, not a marketing quick fix. Explore the Proof Center.