Running a price test: the mechanism, the checkout, and what to believe

A price test charges each group a different price for the same product. It is the most valuable question most stores can ask — a few percent on price moves more margin than almost any layout change — and the one with the least room for error, because the failure mode is a shopper being charged something other than what they were shown.

Everything below follows from one rule we hold ourselves to: never show a price we cannot charge. When anything is uncertain, the shopper sees your ordinary price and pays it.

The three figures are live, and the second and third run the app’s own code. Describe your store in them and they will tell you how your test will charge, and what a shopper abroad will pay.

Price tests are on the Growth plan and above, and in the 14-day trial. They are in beta: they change what shoppers pay, so check the first few orders after you start one.

What a price is worth

A lower price sells more. That is not the question. The question is whether the extra orders make up for the margin given away on every order you would have had anyway — and the answer depends on a number nobody knows in advance: how hard your shoppers react to price.

At the defaults — a £99 overshirt that costs £40, converting at 3% — testing £89 with a moderate reaction lifts conversion to 3.52% and revenue per visitor from £2.97 to £3.13. Profit per visitor falls, from £1.77 to £1.72. £89 needs 3.61% just to break even on profit.

Push the test price below £74 or above £124 and the figure shows the confirmation the builder shows. Anything more than 25% from your list price stops and asks whether you meant it, because 8.90 typed for 89.00 looks exactly like a clearance price, and orders placed at the wrong price cannot be undone. You can confirm and carry on: a cap that could not be lifted is one people route around by lowering their list price instead, which is permanent and worse.

The cheaper arm winning on revenue is the most common wrong answer in price testing.

Up or down, on any plan

How the price is charged depends on your store. You do not choose: it is decided when you create the test, stamped onto it, and shown on the panel with the reason.

With every switch off — a store not on Plus, testing £89 against £99 — the price is charged as a discount: the test group pays £89 as “£99 less £10”. Turn on Plus and the same group simply pays £89. The rules, in order:

A product in the test is sold on subscription

Always the discount mechanism, Plus or not. Shopify applies a discount to a subscription line and never a cart transform.

Shopify Plus, or a development store

The line price is set directly at checkout by a cart transform. Prices go up or down, and the shopper sees no discount anywhere — unless another app on the store already customises the cart (usually a bundle app). Only one app’s change applies to a cart line, so the test falls back to the discount mechanism and says why.

Every other store

Shopify only lets an app charge less than the list price, so each group is charged as a discount from it. A price cut needs nothing else.

A rise, on the discount mechanism

The product’s price in Shopify is raised to the highest test price while the test runs, and every other group — the control included — is discounted back down from there. It needs the permission to change product prices, which the builder asks for only when a group goes above the list price.

• Other sales channels, POS and product feeds show the raised price while the test runs.
• So do B2B catalogs set as a percentage off your price. Catalogs with fixed prices are unaffected.
• The original price is put back when the test is paused or ended — unless you rolled out a winning rise, which stays.
• Every change we make to a price is recorded before it is made, and an hourly check puts back anything a failure left raised.
• If you uninstall the app while a test is raising prices, they stay raised. End the test first.

Setting one up

Approve the pricing permissions on Settings, Store connection.

Choose the products first. That happens once, above the groups, because which products are under test cannot honestly differ between arms. Search your store, add what you want — up to 50 variants — and each one’s current price is captured as the baseline.

Then set a price per group for each product:

• Bulk helpers — −5%, −10%, −15%, −20% and back to list — with rounding to .99, .95 or a whole number.
• Compare-at per group, if you want a struck-through price. It shows only where your theme already strikes one through.
• CSV download and upload, for a catalogue too long to price by hand.
• An empty box means that group pays the list price for that product. That is how a control is usually expressed.

Not every currency. A store whose own currency has three decimal places — KWD, BHD, OMR and the like — cannot run a price test yet. Every amount in the app is in hundredths, so a price to the exact fils cannot be set, and the builder refuses rather than approximates.

Do not touch your prices while it runs

Every group’s price was worked out against the list price at the moment you configured it. Change that in Shopify and the arithmetic no longer matches what shoppers see.

We check hourly and pause the test if a price has moved, rather than warning and carrying on. A warning nobody reads means shoppers continuing to be charged from a figure that has changed; a pause costs you a restart you can control.

Changing the test’s own prices is different: you can, but on a running or paused test it restarts the results. Visitors keep their groups; the numbers start again, because what was measured before the edit was a different test. The builder asks before it saves.

Who is in the test

Every market, in its own currency. You set prices in your store currency; a shopper in another market gets the same percentage change on the price their market already shows — after Shopify’s conversion, any market price adjustment, and any price you fixed for that market by hand. Their market’s ending is kept.

Test £89 against £99, and a shopper in Ireland who normally sees €114.99 sees €102.99 — and is charged exactly that at checkout. The same ratio takes ¥19,800 to ¥17,800 and keeps it whole. The figure runs the function the storefront and both Shopify Functions are checked against, so these are the numbers your shoppers would see. If you would rather set a market’s test price yourself, open Market prices on the panel.

Subscribers

A product sold on subscription is tested like any other, and the plan’s own saving stays on top of the test price. You choose whether a subscriber keeps their group’s price on every renewal or on the first order only. Ending the test does not change what an existing subscriber pays.

Business buyers

Kept out. A customer buying for a company is never bucketed and their prices are never rewritten, so their orders carry no group and are not counted. They pay their company’s catalog price, and the Functions skip a B2B cart regardless.

In-store and draft orders

POS sales and orders completed from Draft orders in the admin never count. Nobody in them saw your storefront.

Crawlers

Never included. Google Merchant Center crawls your product pages and compares what it sees to your product feed; a mismatch disapproves the product, and across a catalogue it can suspend the ads account. Bots see exactly what your theme rendered, which is your feed price.

Two notices in the builder are worth reading rather than dismissing. Google Shopping and Meta feeds keep your list price, so an ad may show a different price from the one a shopper in a test group is offered. And a price test limited to some countries carries a geo-pricing warning: charging different prices by country is restricted in places — within the EU the Geo-blocking Regulation limits it.

What happens on the page

No theme edits. Prices are found by their value — the number your theme rendered — rather than by selectors, so the product page, collection cards, quick views and the cart drawer are all rewritten without anyone telling us they exist.

• The test price appears only once the visitor’s group is on their cart, where Shopify reads it. If that cannot be confirmed, the page keeps your normal price and the visitor is left out.
• “Buy it now” is hidden on tested products, for every group. It builds a checkout without the cart, so it would charge the list price whatever the page said.
• If your theme draws a price the scan does not find, point at it with the picker under “Elements that show a price”.
• Custom code can ask what this visitor pays with zinxSignal.priceFor(variantId).

Headless stores are the exception to “no edits”. Your site draws its prices itself, so our price rewriting never loads there. Your components have to show the group’s price — the SDK’s usePrice() gives them the number the Function will charge — and the site has to pass our cart attributes, or nobody is charged the test price.

If you run a bundle or volume-discount app

Those apps draw their own prices on your product pages, from their own data. We cannot tell that widget’s price from your theme’s, so without help a page can end up showing two prices that disagree.

The Modifications tab has a place to point at that widget and say “leave this alone”, with the same visual picker the content tests use. Do it before you start. On Plus the bundle app’s cart transform also pushes the test onto the discount mechanism — figure 02 shows the switch.

Check it before you start

Open a preview link per group. On a tested product, every price is outlined: green where it changed, amber where it did not, with the reason on hover.

Then run Check setup on the Preview tab. It loads each tested product page from your live store, runs our price code against it, and tells you whether the price was found and changed — with what to fix where it was not.

What the shopper sees at checkout

On the discount mechanism, the £89 group sees a clean £89 on the product page and then £99 struck through with £10 off at the cart. During a rise, that includes the control: it is discounted from the raised price back to your original one.

A discount and a low price are not quite the same offer to a shopper. Strictly, you have measured “£99 with a discount” rather than “£89”. The add-to-cart decision happens on the product page where the price was clean, so the effect shows up in checkout completion rather than earlier — but it is real, and we would rather say so than let you find out from a result that does not replicate.

On the cart transform none of this applies: the price simply is the price.

Reading the result

Profit per visitor is the primary metric, falling back to revenue per visitor where product costs are not set. Figure 01 is why: dropping a price reliably lifts revenue per visitor and can destroy margin at the same time.

Profit per visitor has no spread recorded, so the results page gives no statistical verdict on it — it says so, and points you at the two metrics it can judge. Read profit per visitor for which price made more money, and revenue per visitor or conversion rate for whether the difference is real.

Conversion rate carries a note for the same reason. The cheaper group converting better is arithmetic, not a finding.

The two discount metrics are switched off for a price test on the discount mechanism, with the reason stated rather than silently hidden: our own price change lands in them, so the cheaper group would appear to be giving money away for doing exactly what you told it to.

When it wins

Roll it out to send everyone to the winning price, then finish it in this order: end the test first, then set the winning price in Shopify. The other way round, our discount would come off your new price as well, and changing a tested price pauses the test.

A test never makes a price permanent on its own. A test you pause or end without rolling out puts back anything it raised, so a forgotten test cannot quietly become your pricing. A rolled-out rise is the exception: ending it leaves the winning price where it is, because that is the price you chose. Setting it in Shopify is still the step that makes it yours.