ResourcesGuide
How to check what AI assistants say about your store: a product-answer log
Choose an important product question, save the complete answer and compare it with a dated reference. Keep consumer-app observations separate from simulated tests so each result tells you something useful.
Updated 3 min read
Save the full answer before changing anything. Keep the question, complete answer and dated reference together. That is enough to start a useful check of what ChatGPT, Gemini, Copilot or another assistant says about your store.
Start with one question your catalog operator can judge. If you are still choosing it, use the product-question playbook.
Decide what you are checking
Two questions create different records:
- Discovery: “Which stores sell adapters for this device?” Record which products and stores appeared in that session.
- Product detail: “Does this adapter support this exact device model?” Record the claim and the maintained fact that supports or contradicts it.
These are illustrative prompts. Use a real catalog question for your own check. A discovery omission is not a product-answer error, and appearing in one answer does not establish broad recommendation share.
For a Shopify store, a native Catalog search preview can help investigate a discovery record. Shopify explicitly treats that preview as directional because channels can rank results differently. It is not a preview of every consumer response. Shopify’s search-preview guidance.
Keep one row per answer
Use a spreadsheet, document or issue tracker you already maintain. The fictional examples below show how to review a record; they contain no merchant results. Link the full answer and reference rather than reducing them to a score.
Scroll sideways for columns out of view. Keyboard: focus the table and use the Left and Right arrow keys.
| Field | What to save |
|---|---|
| Question and purpose | Exact wording; why the distinction matters to the operator |
| App or test | Product/interface; displayed model or API identifier when known |
| Session | Date/time, signed-in state, prior dialogue, relevant settings and location supplied |
| Product identity | Product URL, exact model or variant, options and quantity |
| Answer | Full text or capture, with the links shown |
| Reference | Maintained fact, source owner, units, scope and capture date |
| Retrieval evidence | Actual source/tool response if available; otherwise “not available” |
| Assessment | Supported, contradicted, incomplete or unresolved, with the specific claim |
| Next action | Source correction, closer investigation, repeat check or no change |
A new conversation is useful for keeping prior dialogue out of the question you submit. Record the conditions anyway. A signed-out or private session does not establish that every source of context or personalization has been removed.
Preserve the qualification that changes the verdict
Illustrative example: an answer recommends an adapter for Model X, while the compatibility table lists Model X Pro only. Keep the exact names and the answer’s uncertainty. If the table is not exhaustive, absence may mean unknown, not incompatible.
Record the maintained source and its date alongside the answer. Links are useful leads, but an assistant’s explanation of its own process is not a verified tool response. For a source-to-response investigation, use the product-data guide.
Make the record useful for the next step
Keep each repeat as a new row. Note changes in question, session, source and configuration rather than overwriting the first answer. There is no fixed weekly schedule or universal sample size prescribed here.
Next step: take a disputed row to the diagnosis guide. If you have made a change, use the before-and-after method to compare the records.
Label consumer observations and simulations separately
A consumer-app row records what that app returned in that session. A simulated check records a defined model and tool setup. Do not label an API run as a consumer-app test because both use the same provider’s name. Shopify’s browser-tool documentation describes a storefront-session route distinct from server-side integrations.
For a StoreKnows result, keep the simulated-check label, model setup and available evidence. Identify a proposed-fix preview separately from a published version; a saved catalog tool is not live inventory. Automated grades still need review against the source.
Updated 13 September 2026: removed unsupported benchmarks, fixed effort and whole-catalog promises, and claims that simulated improvements establish consumer-app changes. Revised 2 October 2026 to keep this page focused on recording answers. StoreKnows is independently developed and not affiliated with, endorsed by or sponsored by Shopify or the AI platforms discussed.
Use the log for a fair comparison
Keep the same question and reviewed facts, and retain unchanged, worse and unresolved answers alongside improvements.
Open the blank answer-log worksheetQuestions this article answers
- How do I check what ChatGPT says about my store?
- Choose one important product question, record your session conditions and save the full answer with its date and links. Compare the specific claims with a dated product reference. Repeat as useful, preserving disagreements and unknowns instead of treating one result as a universal answer.
- Does a private browser window remove all personalization?
- Do not assume it does. Record whether you are signed in, the chat context, visible settings and location information you supplied. A fresh conversation reduces prior dialogue in your test; it does not prove the assistant has no other context.
- Is a simulated check the same as asking a consumer app?
- No. A simulated check tests a defined model and tool setup. A consumer-app observation records what that app returned in a particular session. One cannot certify the other's retrieval, product coverage or answer.