Skip to content

Crawler

StoreKnowsBot

StoreKnowsBot is the crawler StoreKnows uses to run a simulated AI shopper check against a public Shopify storefront. It identifies itself on every request as:

StoreKnowsBot/1.0 (+https://storeknows.com/bot)

What it reads

  • Public storefront pages (home, collection and product pages) and robots.txt.
  • Shopify's own public storefront tools, the same ones any shopping agent can call.
  • Nothing behind a login, a storefront password or checkout. No page bodies are stored beyond what a report needs.

How it behaves

  • One store at a time, at most one request per second and 15 requests per store, honouring robots.txt and any crawl delay it sets.
  • A store-initiated check (installed app) reads the catalog through the Shopify Admin API instead, with the scopes the merchant granted.

Opt out

Add User-agent: StoreKnowsBot / Disallow: / to your robots.txt, or email support@storeknows.com with your domain and we will exclude it.