What we have learned building this, including the awkward parts.
Short pieces on the mechanics of testing a Shopify store: the arithmetic, the tradeoffs, and the things a pricing page has no room to say.
- 22 September 20267 min read
Price tests and offer tests: what a shopper is charged, tested
Zinx Signal now tests prices and offers as well as pages. Two new test types in a category of their own, how the charge is kept equal to the page, and which plans include them.
pricingreleaseRead - 12 September 202611 min read
How an experiment is set up: every tab, and what each option does
Six tabs, one draft, one save. What Test groups, Modifications, Targeting, Measurement, Preview and Results each decide — and the options on them that change your answer rather than your setup.
guidesetupRead - 12 September 20266 min read
Running a content test: headlines, buttons and the selector problem
The cheapest test to set up and the easiest to break. How to pick elements that survive a theme update, and why a selector that matches nothing is worse than an error.
guidecontentRead - 12 September 20265 min read
Running a redirect test: two pages, one question
Split-URL testing on Shopify. Why the origin URL is the whole setup, what happens to your ad attribution, and the one mistake that would redirect your entire store.
guidecontentRead - 12 September 20265 min read
Running a styles and JavaScript test: power, and the guardrails
Inject scoped CSS and JS per group on the theme you already have. Nothing to set up in Shopify, and the one test type that can genuinely break a storefront.
guidecontentRead - 12 September 20267 min read
Running a theme test: the heaviest change you can make
Split traffic across two whole themes. The highest-value test and the highest-risk one — including the mistake that makes a control collect nothing at all.
guidecontentRead - 12 September 20266 min read
Running a template test: two page layouts, one product
Compare alternate Online Store 2.0 templates for a page type. Narrower than a theme test, so it reaches an answer sooner — and it leaves you something you can publish.
guidecontentRead - 12 September 20266 min read
Running an offer test: does the discount earn what it gives away?
Compare 10% off against free shipping against nothing at all, applied at the cart by Shopify rather than in the browser. The setup, and the one metric that answers it.
guidepricingRead - 12 September 202610 min read
Running a price test: the mechanism, the checkout, and what to believe
Charge different shoppers different prices for the same product, up or down, on any Shopify plan. How the charge is kept equal to the page, what the shopper sees at checkout, and how to read the answer.
guidepricingRead - 12 August 20265 min read
Anti-flicker: what it costs, and when hiding the page is wrong
Client-side A/B testing has to hide the page for a moment or show the wrong version. Both are bad. Here is the tradeoff, and the four rules we hold ourselves to.
performanceengineeringRead - 11 August 20266 min read
How many visitors does a Shopify A/B test actually need?
The honest number, worked out from the arithmetic rather than a rule of thumb, and what it means for a store doing under 10,000 visitors a month.
statisticsplanningRead - 10 August 20265 min read
Measure revenue, not clicks, and what that costs you
Click-through rate is easier to move, easier to prove, and frequently wrong. Why we report revenue per visitor instead, and the two things it makes harder.
metricsattributionRead