Record a Baseline First: What to Observe Before an AI Usage Preference Changes

A hypothetical equipment-rental company publishes public how-to guides on its site. One explains how to choose a mini excavator for a backyard project. Another covers what size trailer a job needs. Leadership has decided, in principle, to restrict AI training use of those guides while keeping them eligible to inform answers. Nothing in this scenario is real, and it is not a finding. It shows the step that comes before any change: recording what is visible today.

The reason is simple. After a change, someone will ask whether it made a difference. Without a record of the earlier state, the answer rests on recollection, and recollection of what an AI answer said last month is unreliable. A baseline gives the later question something to compare against, and it takes less effort than most of the work it protects.

It is also worth being exact about what the baseline, and any later observation, can document. Suppose the company restricts training and then observes generated answers. Those observations can document visibility: whether the company appeared, how it was described, and which sources were shown. They cannot verify whether its content was excluded from training. Microsoft says it applies preferences as it crawls, and what happens inside its systems is not observable from outside. Answers are evidence about what a buyer sees. They are not evidence about enforcement.

Begin with the buyer questions. A buyer does not use the company’s name or its internal page titles. She describes a job: what to rent for clearing a small lot, how much trailer a project needs, what a weekend rental involves. Write several of these in buyer words, spread across the company’s guide topics, and fix them before anything runs. Vary them by stage, since a buyer early in a project asks what fits a job, while a buyer further along asks how two options compare. Note what the set leaves out, so a result is not read as covering everything the company publishes.

Then choose where to look. Microsoft’s announcement names Bing, Copilot and Microsoft 365, so a Microsoft experience such as Copilot is a relevant place to observe, and the record should say which Copilot product and which mode was used, because they can answer differently. ChatGPT, Claude, Gemini, and Perplexity belong in the record as context. A Microsoft preference does not govern them, so their results cannot show an effect of it, but they show what buyers see elsewhere. Keep every place’s results on its own line.

Run the set and keep the full text of each answer. For every answer, record two things separately. First, whether the company was mentioned: its name appears in the text. Second, whether one of its pages was cited: a page of the company’s appears among the sources shown, and the record should keep the address. The two can diverge. An answer can name the company with no citation of its pages, and it can cite a page without naming the company in the text. A mention does not show that the company’s content supplied the answer, and a visible link is not a complete record of where an answer came from. Note the date, the exact wording used, and how the company was presented: listed with no comment, listed with a caveat, or recommended with a reason.

Two checks protect the record. Responses can fail or go missing during collection, and those need to stay separate from valid answers that did not mention the company. Counted together, a collection problem can look like a finding. A few missing responses do not automatically undermine a result, so the working check is whether enough of the planned responses succeeded to support what is being reported. The second check is repetition: running the same set again shows whether a result holds, while changed wording or new questions are a different test whose results are not directly comparable.

Timing comes next. Microsoft says it will honor the new preferences as it crawls, starting February 1, 2027. The earliest point at which a comparison could say anything about the new preference is after that date and after the relevant pages have been recrawled. A comparison made sooner measures the passage of time. Because the announcement does not promise a recrawl date for any given page, the recheck date should be set with that uncertainty in mind.

When the comparison comes, it supports a narrow statement: for these questions, in these places, on these dates, here is what appeared. It does not show why. Other things change between two dates, including the engines themselves. A before-and-after difference alone does not establish that the preference caused it, and an unchanged result does not show a preference failed.

The baseline has a use even if the company decides to change nothing. It records what appears today, before any new preference is honored, and it gives the next review a starting point. When the evidence does not yet justify a change, WATCH is a legitimate result in our framework: keep observing, set a date to check again, and write down what would change the decision.

FAQ

What is a baseline in this context? It is a record of what a fixed set of buyer-worded questions returns today in each place you look. It includes the full answers, whether the company was mentioned, whether one of its pages was cited, and the date and wording used.

Can observing answers show that content was excluded from training? No. Observations can document visibility, meaning what a buyer sees. They cannot verify whether content was excluded from training, because a vendor’s internal processing is not observable from outside.

Why record a mention and a citation separately? They can diverge. An answer can name a company without citing its pages, and a mention does not show that the company’s content supplied the answer.

Which Microsoft experience should the baseline cover? A relevant Microsoft experience such as Copilot, which Microsoft names in its announcement. Record which product and mode was used, and keep ChatGPT, Claude, Gemini, and Perplexity as context only.

How should failed or missing responses be handled? Keep them separate from valid answers that did not mention the company. A few missing responses do not automatically undermine a result, so check whether enough of the planned responses succeeded to support what is being reported.

Is this article reporting a result from Axis Suite? No. The company and the scenario are hypothetical, and the article describes a method without reporting a result for any brand.

About Axis Suite Axis Suite is the independent intelligence layer that tracks what AI says about your brand and the observable evidence behind it, retrieval, narrative, recommendation, and corroboration signals, across engines including ChatGPT, Claude, Gemini, and Perplexity. Built by TrendAxis, Axis Suite works from a framework that separates Mentioned, Cited, Recommended, and Chosen. Axis Suite is measurement infrastructure, not a marketing quick fix. Explore the Proof Center.