A content test rewrites specific things on the page: a headline, a button label, an image, a block you want hidden. No code, no second theme, nothing to set up in Shopify first. It is the fastest test to build and the one most likely to fail without telling you.
Setting one up
On Modifications, each group gets a list of changes, applied in order. Use the picker — Pick on my store opens your live storefront, you click the thing you want to change, and it comes back with a selector and a note of what that element looked like when you chose it.
Four operations:
- Replace text
- The safe one. Swaps the words and touches nothing else.
- Replace HTML
- For markup you control. Reach for it only when text is genuinely not enough, because you are now responsible for the markup being valid.
- Set an attribute
- Changes an image source, a link target, an aria label. Name the attribute or the operation does nothing.
- Hide it
- Removes a block from view. Genuinely useful — “does this badge help or distract” is a good question.
Leave one group with no changes and that group is your store as it is. That is normally the control, though it does not have to be: comparing two rewrites against each other with nobody seeing the original is a perfectly good test.
The selector problem, stated plainly
A selector that stops matching does not produce an error. The change simply does not happen. Those visitors still count as having been in the test, so the group reports as “no different from the control” — which is indistinguishable from a real finding of no effect.
That is the single most expensive thing this kind of test can do: a confident decision made on evidence that was never collected. Everything below is about avoiding it.
Pick, do not type
The picker builds a selector by scoring the options, preferring id and data- attributes over class names, because themes rewrite class names on every update and rarely rewrite those. A selector you type by hand is usually the first one that worked in devtools, which is usually the fragile one.
Check it on a phone
Plenty of themes render a different DOM at mobile widths — a separate header, a drawer instead of a nav, a duplicated price block. A selector picked on a laptop can match nothing on the device most of your traffic is using. Open the preview link on your phone before starting.
Watch for something overwriting you
Changes are applied before the page is shown and then re-applied for a few seconds afterwards, because themes render sections, drawers and quick views late. What that cannot survive is another app that owns the same element and rewrites it continuously. Load the page and watch: if your wording flashes in and back out, something else owns it, and the fix is to change it at the source in your theme.
What to test, and what not to
Content tests are good at questions with a short causal chain. Does this headline sell the product better than that one. Does naming the delivery time near the button reduce abandonment. Does the badge earn its space.
They are poor at anything requiring layout to change. If the honest answer involves moving things around, that is a template or theme test — trying to reach it with a dozen DOM edits produces a test that is fragile in a dozen places and impossible to reason about when it breaks.
Before you start
- Open a preview link per group and confirm each one actually looks different. Preview reads what is saved, so save first.
- Check the mobile rendering, not just the desktop one.
- Run Check setup on the Preview tab. It loads your live pages and says which selectors match nothing, before a shopper finds out for you.
- Check the Targeting tab’s URL rules. A test scoped to one page will collect a fraction of the traffic you expect, which is fine as long as you expected it.
When it wins
Make the wording real: edit it in your theme or your product content, then end the test. Leaving it running means every shopper waits on our script for a change that could be in your store, and content tests are the easiest of all of them to make permanent.