CRO · Testing

A/B testing & experimentation.

Tests designed to reach significance — not to produce a nice-looking chart. Every experiment starts with a hypothesis and ends with a decision.

We plan the roadmap, build the variants, run the math and tell you what each result is worth — including the tests that lose.

Fact 1: 95%; Significance before a winner is called
Fact 2: 1 metric; Agreed before every test launches
Fact 3: 0; Winners called early

Free Revenue Audit

Your 3 biggest revenue leaks, on video, in 3 business days.

3 biggest leaks
on video
3 business days
free

What’s included

Plan, run and read every test, end to end.

A test is only as good as the question behind it and the discipline after it. We handle both, so your results hold up when you ship them.

01 · Plan

Choose the right test.

A roadmap built from research, scored on revenue impact and the traffic each test needs.

Hypotheses backed by research evidence
Impact and effort scoring
Sample size and run-time estimates
Primary and guardrail metrics
A roadmap ranked by revenue

02 · Run

Build it properly.

Variants designed, developed and QA’d across devices before a single visitor sees them.

Variant design and development
Cross-device and browser QA
A/B, split-URL and multivariate tests
Server-side where it matters
Traffic allocation and segment rules

03 · Read

Decide, then ship.

Results read honestly — winners built into production, losers written up so nobody retests them.

Significance and sample-ratio checks
Segment analysis by device and source
Annualized revenue impact
Winners shipped to production
A searchable log of every learning

Inside a test

Two versions. One question. An answer you can ship.

Same traffic, same period, one change. We run every experiment until the math — not the mood in the room — says it is done.

A · Control

$48.00

SMLXL

Add to cart

Conversion rate

2.31%

20,000 sessions

B · Variant

$48.00

SMLXL

Arrives Thu · Free 30-day returns

Add to cart — free returns

Conversion rate

2.58%

20,000 sessions

What you get

Four things, every test cycle.

Clear enough that anyone in the business can see what was tested, what happened and what it was worth.

Test brief
Hypothesis, variants, metrics and expected run time — agreed before launch.
One page · Per test
Live dashboard
Where every running test stands, without anyone calling it early.
Always on
Results write-up
What won, what lost, why, and what it is worth annualized at your traffic.
PDF · Per test
Learning library
Every test you have run in one place, so the knowledge stays when people leave.
Yours to keep

How it works

From question to decision.

A steady cadence of tests, each one run until the data says something — not until someone gets impatient.

Weeks 1–2ResearchWe find where the funnel leaks, so the first hypotheses come from evidence.
Week 3RoadmapIdeas scored on impact, effort and the traffic they need, then ranked.
2–4 weeks eachTestVariants built, QA’d and run until they reach significance.
Every cycleShip & learnWinners go live, learnings get logged, the next test launches.

The learning library

Every test, logged — including the ones that lost.

Wins get shipped. Losses get written up. Over a year, the log becomes a record of what your customers actually respond to.

Test	Hypothesis	Page	Result	Annualized	Status
T-018	Show the delivery date above the fold	PDP	+6.2% CVR	+$142k	Won
T-017	Remove the homepage carousel	Home	+0.4%	—	Inconclusive
T-016	Free-returns badge at checkout	Checkout	+3.9% CVR	+$88k	Won
T-015	Bundle builder on collection pages	Collection	−2.1% CVR	—	Lost
T-014	Sticky add-to-cart on mobile	PDP	+8.4% ATC	+$121k	Won
T-013	Reviews moved above the price	PDP	+0.8%	—	Inconclusive

18: Tests run this year
7: Winners shipped to production
11: Losses written up — never rerun
+$351k: Annualized from the winners shown

Illustrative test log — not a client result.

Who it’s for

A fit for some brands. Honest about the rest.

Best-fit signals

500+ orders a month
Tests run before, results never trusted
Decisions made by opinion, not data
A testing tool, but no one running it
A redesign planned, want to de-risk it
Growth team without a dedicated analyst

Not a fit

Under 500 orders a month
Want a winner by Friday
No development access to the store
Magento / Adobe Commerce

Below roughly 500 orders a month (about 50,000 sessions at a 1% conversion rate), a testing program produces inconclusive results. Brands under that floor get research-led fixes and implementation instead, or an honest referral.

Platform: Shopify & Shopify Plus
Revenue: $200k–$5M / month
Monthly orders: 500+ orders
Who we work with: Founders & growth leads

Case studies

Experiments that reached significance and the revenue they moved.

See all 24 case studies

Led our web refresh

“100xRetention has led a web refresh for our brand, Andrea Maack. Honestly, one of the most technically talented teams I've worked with across the board.”

Andrea MaackAndrea MaackAndrea Maack case study

FAQ

Before your first test.

Didn't see your question? Email us at 100xretention@gmail.com

How long does a test run?

Usually two to four weeks. We estimate the run time from your traffic before launch, and we do not stop a test early because it looks good.

What if a test loses?

It still earns its keep. A losing test stops a bad idea going live, and we write up why so nobody runs it again.

Do you use our testing tool?

If it is sound, yes. If you do not have one, we recommend one that fits your traffic and stack.

Can we test checkout?

On Shopify Plus, largely yes. On standard Shopify there are limits, and we will tell you exactly which ones before we plan.

Who builds the variants?

We do — design, development and QA — so your team is not pulled off its own roadmap.

How many tests run at once?

As many as your traffic supports without tests interfering with each other. We plan that per page, not per month.