A headline that lifts add-to-cart rate by 12% and leaves revenue flat has not helped anyone. It has moved shoppers one step further down a funnel they were going to leave anyway, and it will show up in your testing tool as a win.
This happens constantly, and it has a boring cause: upper-funnel metrics are easier to move because they are easier to move. Urgency copy, a bigger button, a discount badge — all of them reliably increase intention and unreliably increase purchases.
Revenue per visitor is the honest headline
Revenue per visitor — total revenue attributed to a group, divided by the visitors bucketed into it — is the only common metric that cannot be gamed by moving people around inside the funnel. It captures conversion rate and order value in one number, which is exactly right, because a variant that converts 5% better at a 10% lower basket is a loss.
Once cost of goods is available, profit per visitor is better still. A variant that sells more of your worst-margin product is a revenue win and a business loss, and nothing but margin data will tell you.
What it costs: variance
Revenue is noisier than clicks. One shopper buying a £900 order moves the mean; nobody clicks nine hundred times. So a revenue test needs more traffic than a click test to reach the same confidence.
That is a real cost and worth stating plainly. The answer is not to retreat to click-through rate — it is to report the uncertainty honestly. Every group carries a credible interval, and the dashboard refuses to name a winner until the bands separate.
What it costs: attribution
Counting clicks needs nothing but the page. Counting revenue means following a shopper into checkout, which on Shopify is a place apps cannot reach directly — script tags were retired there, and the modern route is a sandboxed web pixel with no DOM access.
So the chain has to hold:
- The group is decided in the browser and stored in a first-party cookie.
- The cookie is written before any navigation a test triggers.
- The pixel reads it when an order completes and stamps the order with the group.
Every link in that chain is a place attribution can be lost, and being specific about them is more useful than claiming they never break.
One visitor, many orders
A repeat customer is counted once in the denominator and every one of their orders in the numerator. That is deliberate: if somebody buys three times, their loyalty should push revenue per visitor up for their group. Counting them as three visitors would flatten an excellent customer into three mediocre ones.
The tradeoff, stated
Measuring revenue means slower tests and a harder attribution problem than measuring clicks. We think that is the right trade, because the alternative is a fast, confident answer to a question that does not decide anything.
But it is a trade, and a tool that presents it as a free upgrade is selling you something.