Shopify serves storefront HTML from its own CDN cache. That single fact decides the architecture of every A/B testing app on the platform: the page arrives already rendered, so the variant has to be applied in the browser, after it lands.
Which leaves an unavoidable choice. Either the shopper sees the original for a moment before the variant applies — the flicker — or the page is hidden until the decision is made. There is no third option, and anyone claiming otherwise is either rendering server-side or not telling you.
Why the flicker is worse than it looks
A visible flash is not only ugly. It biases the result. The visitors most likely to see it are the ones on slow connections, and they are not a random sample — they skew mobile, skew older devices, and convert differently. A test that degrades for one segment is measuring that segment’s patience alongside your headline.
Why hiding the page is also worse than it looks
The usual fix is to hide <body> until the config arrives, with a timeout. Every millisecond of that is time a shopper spends looking at nothing, on a page that makes someone’s living. Done carelessly it is a conversion cost you are paying to measure conversion.
The four rules
1. Never hide without a pending change
If nothing is going to change on this page view, there is nothing to hide for. The worst version of anti-flicker blanks the page, waits, and then reveals the same page it would have shown anyway. A preview link is the clearest case: the group is already decided by the URL, so hiding costs a blank second and buys nothing.
2. Fail open, always
If the config never arrives — our backend is slow, a network blip, an ad blocker — the page reveals and the shopper sees the control. A test that fails closed takes the store down with it. The timeout is a guarantee, not a best effort.
3. Hide <html>, not just <body>
The script runs in <head>, before <body> exists. A rule scoped to body covers nothing until the parser reaches it, so a theme that paints its background on <html> gets a frame or two on screen regardless.
4. Never reveal into a navigation
Theme, template and redirect tests move the visitor to a different URL. Revealing before that navigation completes paints the outgoing page for its whole duration — which is exactly the flash the overlay exists to prevent, arriving at the end instead of the start.
The budget
Our loader is capped at 10 KB gzipped, asserted on every build against the file actually served rather than the file in the repository. It had been broken twice before anything measured it.
Configuration is mirrored to the browser’s own storage, so a returning visitor applies their variant with no network wait at all — the round trip only happens on a first visit, and it refreshes in the background for the next one.
The honest summary
Client-side testing on Shopify has a real cost, and it is measured in milliseconds on the critical path of somebody else’s storefront. The right response is to make that cost small and to be specific about it — not to claim it is zero.