Accuracy alerts and the AI shopping agent test

Two checks on Growth and Pro go beyond whether AI recommends you. Accuracy alerts catch AI assistants saying wrong things about your store. The agent test sends an AI shopping agent to find, choose and check out your products, and shows every step where it gets stuck.

What AI gets wrong (accuracy alerts)

After each full scan, the app compares what AI assistants say about your store with your real prices, stock and policies. Only clear contradictions are shown, under AI Visibility → What AI gets wrong.

Each issue shows what the answer said, what's actually true ("Actually: …"), and its kind: price, shipping, returns, product, stock, contact or other.

To fix one: AI assistants repeat what they read. Make the right facts clear on your product pages and policies, then re-check the question. Mark the issue Fixed, or Not an issue if it's wrong.

Open issues lower the Accuracy part of your AI readiness score, and new ones show on Home and in your alerts.

The AI shopping agent test

AI assistants like ChatGPT, Gemini and Copilot can now shop for people: they search a store, pick a product and take it to checkout on their own. If an agent can't find a product, can't tell which variant is in stock, or can't answer a question about returns, it moves on to another store.

The test sends an AI shopping agent to your store, through the same public Shopify connection real agents use. Find it under AI Access → Agent test, choose up to 10 products, and click Test. It takes a minute or two.

For your whole store, it checks:

• Can agents find your store's shopping tools (Shopify's agent profile)?
• Does your store have instructions for AI agents (agents.md)?
• Can the agent tell a shopper your return policy, and your shipping costs and times?

For each product, it tries to:

1. Find it by name, and by what it is (for example, searching its category)
2. Read its details: description, photos and category
3. Choose a variant that's in stock
4. Add it to a cart and work out the total
5. Go to checkout

The agent never places an order.

Reading the results

Agent checkout success is the share of tested products the agent could take all the way to checkout. Each step shows Passed, Needs attention or Failed, with what to fix. For example:

• Not found by what it is: set the product's Shopify category and type, and use the words shoppers search for in its title and description.
• No variant available: restock it, or allow selling when out of stock.
• No answer about returns: write your refund policy in Settings → Policies with clear terms.
• Can't reach checkout: remove your store password when you're ready to sell.

Problems also appear in What to fix. Run the test again after each fix to check the agent gets further.

Both checks are on the Growth and Pro plans. See the pricing page.