SEO A/B Testing: How To Run One On Your Store Without Duplicate Pages

SEO A/B testing changes one thing on a group of pages that share a template and leaves a matched group of the same pages untouched. Over the same weeks, the test compares the two groups’ organic clicks from Google, and the difference between them is what your change did in search.

It tells you whether your change or something else moved the clicks. You rewrote your collection page titles in spring, organic clicks rose over summer, and Google ran a core update in between. In a split test that update lands on the control group as well and cancels out.

That makes it a different job from A/B testing your shoppers, which splits visitors on one page to see who buys.

I'm John Butterworth, founder of Mint SEO, and I've spent over 11 years in SEO sorting the changes that really moved rankings from the ones that only coincided with a good month. A split test is how I tell the two apart on a store. On Shopify you build it inside your theme, so every collection keeps one URL and Google never meets a duplicate. The first thing I'd test is a new title formula on your collection pages.

Your Store Can Run One If A Template Has Hundreds Of Pages With Search Clicks

A store is ready for SEO A/B testing once it has two groups of pages large enough for the ordinary week-to-week wobble in clicks to cancel out. In practice that means hundreds of pages built on one template.

That template is nearly always the collection page or the product page, because those are the pages you have in bulk. Your homepage and a handful of landing pages can't be split at all, since there is only one of each.

Those hundreds of pages only count if Google sends people to them. A page nobody reaches from search adds nothing to either group, and the smaller the change you're testing, the more clicks the two groups need before a difference shows.

Find A Template With Hundreds Of Collections Getting Weekly Search Clicks

Open the Performance report in Search Console, filter the pages to URLs containing /collections/ and set the range to the last three months. Export the page list and count how many collections picked up clicks in most of those weeks.

In my experience, a few hundred collections with steady weekly clicks is enough to test a big change such as a new title formula.

Forty collections, with the clicks piled onto six of them, is too thin to split into two matched groups.

Below The Bar, Run A Before-And-After Test On Half Your Collections

A store below that bar can still test, with weaker evidence. Change the titles on half your collections, leave the other half as they are, and compare the two groups' clicks for the weeks before and after the change.

Mark the day it went live with an annotation on the Search Console performance chart, which pins a short note to that date.

An update or a seasonal swing lands on both halves at once, and the untouched half shows you how big it was, which a plain before-and-after read can't do. With small groups, chance can still move one half more than the other. I treat a difference that holds for several weeks as a lead worth acting on and nothing firmer.

Make Your First SEO A/B Test A Title Test

Your first test should be a title test, because it needs a change big enough to move organic traffic across a whole group of pages. Changing the title formula on the variant group of collection pages does that from one template edit, while the control group keeps the old titles. The two groups' organic traffic then shows what the new titles did.

A new title formula can cost clicks as easily as it wins them, which is the reason to test it before it goes on every page. In one SplitSignal SEO experiment published by Semrush, "adding the category name to product page titles" lost the test group an estimated 8.2% of clicks against the control group. Without the control group, that loss would have looked like a slow month for organic traffic.

Meta descriptions, headings and structured data can be split-tested for SEO the same way, as can category page copy. An internal linking test works too, with a block of related-collection links added to the variant bucket only.

Build The Split In Your Theme, With One URL Per Page

Because SEO A/B testing compares organic search traffic between two groups of pages, the build is about the groups. One is a control bucket of collections you keep unchanged, the other a variant bucket you change, and each URL keeps the single version shoppers and Googlebot both see.

An SEO A/B test that shows Googlebot one version and shoppers another is cloaking, which breaks Google's spam policies even during a test. Splitting by page group never does it.

Alternate Your Collections Into Two Buckets By Search Clicks

Sort the collections you counted by search clicks over the last three months, highest first, and deal them alternately into the control bucket and the variant bucket.

A few collections usually carry a large share of a store's clicks, and a random split can drop three of your top five into one bucket and swamp the test result. Alternating down the sorted list gives the SEO test two buckets with the same mix of busy and quiet collections.

Before the test starts, chart the two buckets' weekly organic clicks for the previous three months. Their lines should rise and fall together, which tells you the control group can predict the variant group once the SEO test is live.

Change The Variant Bucket With An Alternate Template Or The SEO Title Field

Shopify gives you two ways to change the variant bucket on its own. For a layout or content test, duplicate the collection template in your theme and make the change in the copy. Assign that copy to the variant collections only. Shopify's template documentation confirms a template can be assigned to one collection or to many at once through the bulk editor.

For a title test, edit the search engine listing title on each variant collection. On a big catalogue it is quicker to put the new formula in the theme's title tag behind a collection metafield that only the variant collections carry.

Either way the change sits in the HTML Shopify serves, which Googlebot reads on its next crawl.

Keep Duplicate Products And Tag Manager Scripts Out Of The Test

Duplicating a product or collection to try a new version puts two pages in front of Google for one search, which is the cannibalisation store owners are right to worry about.

Sending an SEO test through a ?view= link fails for a different reason. The alternate template only appears on that parameter address, and the collection page Google ranks never shows the variant. The test ends up comparing clicks on pages that did not change.

A change made in Google Tag Manager or a cookie-based testing tool runs in the visitor's browser. Googlebot generally doesn't keep cookies, so it sees the version served to browsers without them, and a JavaScript change can reach Google late. Make the change in the theme, where the served HTML carries it.

How Long To Run An SEO A/B Test, And How To Read The Result

Expect the test to run for weeks before the difference between the control group and the variant group means anything. For small and medium-traffic sites, SEOTesting.com's guide puts a standard SEO split test at six to eight weeks to reach a statistically significant result. Plan the test's launch date with that window in mind, because a spring SEO test that runs eight weeks finishes in the build-up to summer demand.

Start The Clock When Googlebot Has Recrawled The Variant Bucket

Nothing in the test counts until Googlebot has recrawled the variant pages. Googlebot revisits your busiest collections far more often than the quiet ones, and the variant group reaches Google's index over several days or weeks after you publish the test change.

Run URL Inspection in Search Console on a spread of variant collections, from the busiest down to the quiet ones. Start the test clock in the week most of them show the new title in the crawled page.

Launch Between Google Updates, Not During One

A control group cancels most of what a Google update does to the test, because both groups live through it together. A core update can still move your clicks far more than a title test does, and the bigger that swing, the harder the test result is to read.

Before you pick a launch date, open Google's Search Status Dashboard, which lists every ranking update with the day it started and how long it ran. This year, a title test switched on in the week of 27th March or 21st May would have shared its first twelve days or so with a core update rolling out.

A spam update started on 24th September, the day before this article went out. Spam updates go after pages that break Google's spam policies, so a store doing nothing of the sort rarely drops in one. Competitors that do drop can still shuffle your positions while it rolls out, and that movement lands inside any test running at the time. I'd wait for the dashboard to show it finished, then launch and add the annotation.

Decide What A Win, A Loss And A Flat Result Mean Before Launch

Write down before launch what you'll do with each outcome. Once the result is in, those notes decide it: a win goes onto every collection. A loss is reverted on the variant bucket, and you've saved the whole store from it.

The third outcome, a flat result, is where plenty of tests end. A change you had a sound reason to make can still go live after one, since some changes only show at the next core update. Record it in your notes as untested, so nobody later counts it as a win.

To read the result, filter Search Console to each bucket's URLs and compare the two groups' weekly clicks with the weeks before launch. Act on that difference and on the rules you wrote down, and every call on the test stays a data-led decision.

Get The Winning Title Formula Onto Every Collection

A title test that wins hands you a formula, and the formula then has to be filled in collection by collection with the search term each one really ranks for. That per-page work is part of our On-Page SEO service, which covers titles and meta across a store's collection and product pages. It also takes in heading structure and internal linking.

If you'd like help working out which template your store can test and which change to try first, book a free consultation and bring your Search Console export.