Most direct-to-consumer brands describe themselves as data-driven and run creative like a lottery. A batch of assets goes live, one of them works, the team celebrates, and nobody can explain why it worked or how to produce another one deliberately. Six weeks later the winner fatigues, the account softens, and the cycle restarts from zero. The volume of testing is high. The accumulated knowledge is close to nil.
Marketing strategist Jaivin Karnani has watched this pattern repeat across performance accounts of very different sizes, and the diagnosis is consistent: the brands are testing creative without a system for learning from it. Testing produces winners. A testing system produces a growing body of knowledge about why things win, which is the only asset in performance marketing that appreciates.
Published: September 9, 2026. This article reflects Karnani’s operating pattern for structuring creative testing in direct-to-consumer performance accounts.
The Difference Between Running Tests and Having a Testing System
Running tests means launching variations and keeping the ones that perform. It works, in the narrow sense that it produces better ads than not testing. Its limitation is that every cycle starts from the same place, because nothing carries forward except a single surviving asset.
A testing system treats each test as a question with a recorded answer. The output of a cycle is not only a winning ad but a documented statement about what the audience responded to, which then constrains and directs the next round. After a year of running tests, a brand has a folder of winners. After a year of a testing system, it has a map: which promises land, which objections must be addressed before the sale, which formats suit which stage, which proof types are persuasive to this buyer.
The map is what makes production cheaper over time. It is also what survives a platform change, an algorithm update, or an agency transition, because it describes the customer rather than the channel.
Test Variables, Not Assets
The most common structural mistake is testing whole assets against each other. Five completely different videos go head to head. One wins. What was learned? Nothing transferable, because the five differed on a dozen dimensions simultaneously and there is no way to attribute the outcome to any of them.
The alternative is to hold a hypothesis constant and vary one dimension at a time, in a deliberate order. Karnani’s sequence moves from the variables with the largest effect to the smallest, because testing polish before positioning is how teams spend a quarter optimising something that was never going to work.
The order that tends to hold: the promise first, because what the ad claims matters more than how it claims it. Then the objection, because most non-buyers are not unpersuaded by the promise but blocked by a specific unaddressed doubt. Then the proof, because once someone wants the outcome and their objection is handled, they need a reason to believe. Then the format and hook, which determine whether any of the above gets seen. Then, last, production quality, which is real but is the smallest of the five and the most expensive to change.
Brands routinely run this list backwards, starting with production value and hooks, and conclude that creative testing does not work for them.
The Documentation Layer Nobody Builds
The reason most testing produces no compounding knowledge is that the knowledge is never written down in a form anyone can query later. It lives in the head of whoever was managing the account, and it leaves when they do.
The fix is smaller than it sounds. Every test gets a record with five fields: the hypothesis in one sentence, the variable being changed, what actually ran, the result, and the interpretation. The interpretation field is the one that matters and the one always omitted. “Variant B outperformed” is a result. “Buyers respond to a durability claim more than a design claim, which suggests the objection is about longevity rather than aesthetics” is an interpretation, and it is the thing that directs the next four tests.
The discipline that keeps this honest is refusing to write an interpretation the result does not support. A test that produced no clear signal should be recorded as producing no clear signal. Inconclusive tests are common and useful; inconclusive tests rewritten as insights are how teams build confident maps of imaginary territory.
Reading Results Without Fooling Yourself
Performance creative generates results that look decisive far earlier than they are, and the pressure to act on them is intense. Three habits protect against acting on noise.
The first is deciding the stopping rule before the test runs. What sample, what duration, what margin. A rule set in advance is a rule; a rule set after the numbers arrive is a rationalisation. Without it, teams reliably stop tests at the moment the number they preferred is ahead.
The second is watching the metric that matches the claim. Creative that promises an outcome should be judged on whether people who click go on to buy, not on click rate. A hook that generates clicks from people with no purchase intent will look like a winner on the metric most dashboards surface by default and will quietly degrade account economics.
The third is checking whether the win survives contact with a different audience. A variant that beats the control only within a narrowly targeted segment has told you something about that segment, not about the creative. Winners worth scaling repeat.
Fatigue Is a Production Problem, Not a Creative One
Every winning asset decays. Teams typically respond by producing another completely new concept, which returns them to the lottery. The more efficient response is to treat a validated concept as a template and produce variations within it: the same promise and the same proof, delivered by a different presenter, in a different setting, at a different length.
This is where the documentation layer pays for itself. A brand that knows why an ad worked can generate ten more that work for the same reason. A brand that only knows which file performed has to rediscover the answer each time.
It also changes the production budget conversation. Iterating within a validated concept is cheap. Originating new concepts is expensive and mostly fails. A testing system shifts spend from the expensive activity to the cheap one without reducing output.
Frequently Asked Questions
How many creative variants should a brand test at once?
Fewer than most teams run. The constraint is not budget but attribution: if the number of simultaneous variants exceeds what the available volume can meaningfully separate, the test produces a ranking that is mostly noise. It is better to answer one question properly than five ambiguously.
Does this approach work for brands with small budgets?
It matters more at small budgets, because small accounts cannot afford to buy their way to a winner through volume. Sequencing from promise to production is a way of spending limited testing capacity on the variables with the largest effect first.
How long should a validated concept be expected to last?
Long enough that the right question is how quickly variations can be produced within it, rather than when it will die. Decay is certain and its timing varies by audience size and frequency, so the durable advantage is production speed inside a known-good concept.
Who should own the testing record?
Whoever will still be there next year. In practice this means the brand rather than the agency. A testing archive held by a vendor is a body of knowledge the brand is renting, and it leaves with the contract.
Closing Note
Creative testing is one of the few places in performance marketing where a durable advantage is still available, and it is available precisely because most brands do not take it seriously as a system. They take it seriously as an activity. The difference shows up slowly and then all at once: two brands running the same spend on the same platform, one of them producing creative from an accumulated understanding of its buyer and the other still drawing from a hat. The first one gets cheaper every quarter. The second one is always starting over, and cannot work out why the last winner was so hard to replace.