Tests that run themselves and call the winner
Most stores do not run enough tests, and the ones they run are called too early. The experiments agent fixes both.
Where test ideas come from
Every other agent feeds the backlog:
Each idea is written as a hypothesis: "Because first-timers ask about irritation in reviews, adding a gentle-proof line under add to cart will lift add-to-cart rate for new visitors."
- The CRO agent finds that a page does not answer an objection.
- The ads agent sees one angle beating another.
- The email agent sees one email carrying a flow.
- The audit finds a pain point nobody is addressing.
What it tests
| What | |
|---|---|
| Email, in Klaviyo | Subject lines, content, offers, send time |
| Ads, in Meta | Creative, angle, hook |
| Pages, on your theme | Copy, section order, trust blocks, offers |
Calling winners honestly
If a test does not reach enough data, Tilly says so, rather than picking a winner by chance.
- A minimum sample size before any call
- A maximum run time, so tests do not drag on
- One primary metric per test, chosen up front
- A plain-language result: "Version B lifted add to cart by 11%. We are fairly confident this is real."
Tests that teach every agent
When a test finishes, the result is written into your brand profile. The next email, ad or page fix reads it. That is the difference between running tests and learning from them.
Questions people ask
What can Tilly test?
Email subject lines, content and send times in Klaviyo, ad creative and angles in Meta, and page changes on your theme.
How does it decide a winner?
It waits for a minimum sample and stops at a maximum duration. It never calls a winner early, and it tells you in plain language how sure the result is.
How much does a test cost?
10 credits, including writing the variants.
See this on your own store
Paste your store URL. The audit takes about a minute, costs nothing, and ends with a welcome flow you can read before anything is sent.