Conversion audit after a site migration

When orders fall after a new site goes live, a conversion audit after a site migration shows whether the drop is real and which checkout step lost the sales. It matches tracked purchases to the order system first, then compares each step by device for matched weeks either side of launch and ranks the falls.

Last reviewed

Before and after, by step, example

Mobile delivery details step fell from 78.0% to 61.5% after launch

Example data
Tracking check first (all devices)
PeriodOrders in order systemTracked purchasesMatch
4 weeks before launch7,4807,31897.8%
4 weeks after launch6,4106,27197.8%

Result: the tracking gap is the same in both periods, so the funnel below can be compared.

Tracked purchases matched the order system at 97.8% before and after.

Funnel: mobile, matched weeks (launch week left out)

  1. Before 12.0%

    After 11.9%

    2Added to basket

    −0.1 points

  2. Before 55.0%

    After 55.5%

    3Checkout started

    +0.5 points

  3. Before 78.0%

    After 61.5%

    4Delivery details completed

    −16.5 points

  4. Before 92.0%

    After 91.8%

    5Order placed

    −0.2 points

Two-proportion normal approximation, unpooled standard error.

Ranked list (the audit's output)

  1. 1.

    Delivery details, mobile: −16.5 points (interval −18.1 to −15.0). About 960 mobile orders lost over the four weeks (interval 869 to 1,051). Suspected cause: the postcode lookup on the new delivery form.

  2. 2. to 4.

    Basket, checkout start and order placed: each interval crosses zero, so no change can be called.

Example data: an invented clothing shop's replatforming, mobile sessions only. Every step but one moved by less than a point, inside its interval. The delivery details step lost 16.5 points, which at the new site's order rate is about 960 orders in four weeks.
Show the numbers
Funnel: mobile, matched weeks (launch week left out), example data. Step conversion is sessions reaching the step ÷ sessions reaching the step before.
StepBefore: sessionsBefore: step conversionAfter: sessionsAfter: step conversionChange (points)95% interval (points)
1 Product viewed96,40095,800
2 Added to basket11,56812.0%11,40011.9%−0.1−0.4 to +0.2
3 Checkout started6,36255.0%6,32755.5%+0.5−0.8 to +1.8
4 Delivery details completed4,96378.0%3,89061.5%−16.5−18.1 to −15.0
5 Order placed4,56692.0%3,57191.8%−0.2−1.3 to +0.9

What this fixes

Orders are down and the agency says it's seasonalThe new site went live in September, orders fell, and the only comparison anyone has made is with August.
The dashboard says conversion halvedGA4 shows a far bigger fall than the order system, and no one knows which to believe.
Desktop looks fineThe overall rate hides a device split, and the fall is all on phones.
Everyone has a theorySomeone blames the new navigation and someone else the slower product pages, and neither has a number.

What the conversion audit hands over

The tracking checkThe check sets orders against tracked purchases for each period, by device where possible. If the gap changed, the report stops there.
The step tableIt gives every step by device for matched weeks, with the counts and formula behind each rate, as in the KPI library.
The ranked listSteps rank by the size of the fall, with intervals and orders lost, and any step whose interval crosses zero shows as no change.
The suspected causeFor each step that fell, the report says what changed and gives the evidence, such as a count of errors on one form field.

How a post-migration conversion audit runs

  1. 1

    Tracking

    The audit counts tracked purchases against orders either side of launch. A duplicate event, a missing one or a changed currency setting shows up as a gap that moved.

  2. 2

    Matching weeks

    The comparison uses equal full weeks either side of launch week, names any sale or bank holiday on one side only, and adds the same weeks a year earlier.

  3. 3

    Step comparison

    Each step gets compared on mobile, desktop and tablet with a 95% interval, so a step that moved by chance isn't sent to the developers.

  4. 4

    Handover

    The ranked list, the step table and the SQL go into your drive, so the query reruns after a fix.

Where replatforming audits go wrong

The drop that was a tagA purchase event that fires again on a refreshed confirmation page makes conversion seem to rise, and a tag missing on one payment route makes it seem to fall. The audit checks tracking before reading any rate.
Steps that don't line upAn old basket page set against a new basket drawer makes one of them look broken, so we agree a written map of old steps to new first.
One rate for every deviceA fall on mobile and a rise on desktop can net to almost nothing, so the audit splits by device before giving any overall rate.

Launch week stays out of every comparison, because it carries fixes and test orders.

Site migration audits by industry

EcommerceThe order system on each platform gives the audit a count the tags can't change.

When the cause is broken tags, the fix is tracking work. When the fix is a design choice, such as how the postcode lookup should behave, it goes on the experimentation and CRO roadmap as an A/B test.

Questions people ask

How soon after launch can the audit run?

Once there are full weeks on the new site to set against the same number before it, like the four and four in the example above. Fewer weeks give wider intervals, and the report says which steps it can't call yet.

Do you audit replatforming for ecommerce sites?

Yes. Orders come from the order system on both platforms, so the comparison holds when the two checkouts track differently, and steps match by meaning, such as basket page to basket drawer.

Our GA4 setup changed during the migration. Can you still compare?

Yes. The old steps map to the new by hand, and the mapping goes in the report. A step with no matching event gets compared from server logs or the order system, or left out and named.

Do you fix the step that fell?

No, whoever builds the site makes the fix. Once it ships, we compare that step again for matched weeks, or test it as an A/B test if the fix is a design choice.

When did your new site go live?

On the call we compare your weekly orders either side of the launch date and check whether the tracking can be trusted.

Book a call

Free 30 min call