Customer.io Launches Experiments to Test Whole Automation Paths

Customer.io now lets marketers test whole automation paths, not just single messages. In its release notes dated 9 November 2026, the messaging platform introduced Experiments: controlled tests that run inside an automation, measured against a success metric the marketer picks, such as a conversion event or a message click.

Until now, testing a welcome series of three emails against one of five, or a reminder on day two against day five, meant building separate journeys or splitting audiences with random branches and comparing the results outside the workflow. Customer.io says Experiments keeps the whole test, its results and the choice of a winner inside the automation itself.

What Customer.io launched

Customer.io describes Experiments as controlled tests run on messages and other workflow actions in an automation. The marketer defines what success means, and Customer.io reports how each variant performs against that goal. Creating, tracking and managing the test all happen in the automation, without exporting data to an outside analytics tool.

Message variants

The first mode works like a classic A/B test on a single message: two or more versions of one email, compared on the chosen metric. The difference is that each variant can lead into its own sequence of messages and actions, so the test can measure which route converts more people, not only which subject line or design gets more clicks. To start one, the marketer clicks a message block and selects Experiments.

Pathways

The second mode compares multi-step paths. The marketer drags an Experiments block from Flow control in the Build menu and places different sequences behind it. Customer.io contrasts this with random cohort branches, which split people but leave the comparison to the user: with Experiments, results are reported in the automation. The release note gives two options for running the test: send people automatically down the winning pathway, or keep each person on the same variant across several journeys while the experiment runs, so that one customer does not see conflicting versions.

What the release note does not say

The release note does not explain how a winner is determined, for example whether Customer.io applies a statistical significance threshold, a minimum sample size or a fixed test duration. It names two success metrics, a conversion event and a message click, and lists no others.

Plan availability is not stated in the body of the note. The page metadata lists the Essentials, Premium and Enterprise plans, so customers should check their own plan before building a test that depends on the feature. Experiments run in automations only; one-off broadcasts are not covered.

The same set of November release notes also mentions that Design Studio has become the default email editor in Customer.io.

Why path tests matter for deliverability

Testing the shape of a sequence touches more than conversion. Each extra email in a flow is another chance for a recipient to ignore, delete, unsubscribe or report the message as spam, and mailbox providers such as Gmail, Yahoo and Outlook weigh those signals when they decide where the next message lands. A five-email welcome series that converts slightly better than a three-email one may still be the wrong choice if it pushes complaint rates up across the whole domain.

Path tests also change volume. Sending a share of new subscribers down a longer path raises the number of messages sent from the same domain and IP addresses, which senders on shared or newly warmed infrastructure should keep in mind before splitting a large audience. Comparing variants on list health metrics, not only on the success metric chosen in Customer.io, gives a fuller picture of the winner.

Action checklist

• Pick a conversion event rather than clicks when the goal is revenue or activation, since clicks can reward curiosity rather than results.
• Watch unsubscribes and spam complaints per variant: a longer sequence can win on conversions while costing you list health and sender reputation.
• Keep people on one variant across journeys when a customer could enter several automations, so they do not get mixed versions of the same program.
• Let tests run long enough before choosing a winner, since the release note does not describe any built-in significance threshold.
• Check your plan before designing tests around the feature.
• Document each test and its winner, so a later change to the flow does not quietly undo what you learned.

Sources

1. Customer.io release notes: Run experiments in your automations, 9 November 2026 (primary source)

Related on SMTPedia

• Customer.io review: pricing, features and deliverability
• Email automation vs drip campaigns: what you should know
• How to add a Customer.io unsubscribe link