Running a Growth Experiment Backlog

Most growth teams do not fail because they run bad experiments. They fail because they cannot reliably produce good ones, week after week, without the whole thing depending on one person’s memory and mood. The difference between a team that compounds and a team that stalls is almost always the same thing: whether it runs a real experiment backlog, or just keeps a running list of ideas it feels vaguely guilty about not getting to.

A growth experiment backlog is the engine of a growth team. It is where ideas enter, get shaped into something testable, get ranked against everything else competing for traffic, get run, get concluded, and, most importantly, get remembered. When that machine runs well, the team ships a steady stream of tested learning and the wins stack up on their own. When it runs badly, you get a graveyard of half-formed ideas, the same debates every planning meeting, and a suspicion that you are busy without being productive.

I have built and run this machine at scale. At Chegg I ran a high-velocity experimentation program on Optimizely across a landing system of 200+ pages, and lifted conversion 34%. None of that came from a single clever test. It came from a backlog that never ran dry and a process that turned raw ideas into conclusive answers on a predictable cadence. This post is about how to build that machine, and how it fits into the broader discipline of owning the number as a growth PM.

What a growth experiment backlog actually is

A backlog is not a wish list. A wish list is a place ideas go to feel acknowledged. A backlog is an operating system: a structured, prioritized, living inventory of testable bets, each in a known state, moving through a defined pipeline toward a conclusion. The distinction sounds pedantic until you have watched a wish list quietly kill a growth team’s momentum by making everyone feel like there is plenty to do when in fact nothing is ready to run.

The reason the backlog is the engine, and not just a document, is that experimentation is fundamentally a throughput problem. Growth compounds when you run more good tests per quarter and remember what each one taught you. Everything else in a growth team, the analytics, the research, the tooling, exists to feed that throughput. The backlog is where all of it converges into a queue of things you can actually launch. Get the backlog right and the team has a heartbeat. Get it wrong and you have a lot of activity with no pulse.

Where ideas come from

A healthy backlog is fed from many sources, and the health of the program shows up first in how varied those sources are. If every idea traces back to one person, you are not running a program, you are running that person’s intuition with extra steps.

The richest sources, in rough order of how often they earn their keep:

Analytics. Funnel drop-offs, pages with high traffic and poor conversion, steps where users stall. Quantitative data tells you where the money is leaking, which is where tests are worth running.
Qualitative research. Session recordings, user interviews, surveys, usability tests. This is where you learn why the drop-off happens, which is what turns a location into a hypothesis.
Support and sales. The people who talk to customers all day hear the same confusions and objections repeatedly. Those patterns are free, high-quality hypotheses that most teams ignore.
Prior tests. Every concluded experiment, win or loss, opens or closes adjacent bets. A win suggests three follow-ups; a loss rules out a direction. Your own results are one of the best idea sources you have, provided you actually logged them.
Team submissions. Anyone should be able to drop an idea in. Designers, engineers, marketers, and PMs all see things the data does not.

The principle that keeps this from becoming chaos: lower the bar to submit, raise the bar to run. Make it trivially easy to add an idea, a single form field if you like, because you want the widest possible funnel of raw material and you never want someone sitting on an idea because the process felt heavy. Then make it genuinely demanding to graduate an idea into something you will spend traffic on. The submission bar being low is what fills the top of the funnel; the running bar being high is what protects your scarce traffic from untested opinion. Confusing the two, either gatekeeping submissions or running everything submitted, is how backlogs go wrong in both directions.

The anatomy of a good backlog entry

A raw idea is not a backlog entry. “Try a different hero image” is a wish. It becomes an entry when it is shaped into a structure that forces the thinking a good test requires. Every entry worth its place in the queue has five things.

Hypothesis. A specific, falsifiable statement, not a task. “Changing X will cause Y because Z.” The because is doing the real work; it is what connects the change to a belief about user behavior, and it is what you learn about whether the test wins or loses. This is the discipline I go deep on in hypothesis-driven experimentation.

Evidence. The reason to believe the hypothesis. The analytics drop-off, the recordings, the support tickets, the prior result. An entry with no evidence is a hunch, and hunches should score low and usually wait.

Primary metric. The one number this test is designed to move, decided before the test runs, not after you see the data. Naming it up front is what keeps you honest when the results come in ambiguous.

Expected impact. A rough, honest estimate of how much the metric could move if the hypothesis holds, weighted by how much traffic or revenue sits behind the surface being tested. A win on a high-traffic template is worth more than the same win on a page nobody sees.

Effort. What it takes to build and ship. A change a marketer can make in the CMS today is worth more, all else equal, than one needing two weeks of engineering.

These five fields are not bureaucracy. They are the minimum information required to prioritize honestly, and the act of filling them in is itself a filter: a shockingly large share of ideas quietly reveal themselves as weak the moment you try to write the evidence line.

The pipeline: idea to logged learning

Every entry moves through a defined set of states, and knowing the state of each idea is half of what makes a backlog manageable. The pipeline I use:

Idea. Raw, submitted, unshaped. Low bar. Lives in an inbox.
Shaped hypothesis. Someone has done the work to write the five fields. Now it can be judged against everything else.
Prioritized. Scored and ranked, sitting in the queue in a defensible order, ready to launch when traffic frees up.
Running. Live, collecting data, with a pre-decided sample size and stop condition.
Concluded. The test has reached its endpoint and a call has been made: win, loss, or inconclusive.
Learning logged. The result and its interpretation are written into the knowledge base. This is the step that most teams skip and the one that matters most.

The value of an explicit pipeline is that it makes the state of the program legible at a glance. You can see how many ideas are shaped versus raw, whether the running lane is full or starved, and whether concluded tests are actually getting their learnings logged or dying quietly at step five. A backlog where everything piles up at “idea” is a backlog with no shaping capacity. One where things reach “concluded” but never “logged” is a backlog leaking its most valuable output.

Prioritization, briefly

Ranking the shaped ideas is its own discipline, and I will not repeat it here because it deserves its own treatment. The short version: score every candidate on the same axes, usually some version of impact, confidence, and ease, and let the score produce a defensible order so that the loudest voice in the room does not simply set the queue. The framework you pick matters far less than applying it consistently to every idea, including the executive’s.

The full method, including how to score confidence honestly and why traffic constraints do most of the real prioritizing, is in experiment prioritization. Read that as the companion to this piece. Prioritization decides the order; the backlog is the thing being ordered.

Throughput and cadence

A backlog is only an engine if it turns. The rhythm that keeps it turning is a set of weekly rituals, light enough to sustain and firm enough to force movement.

I run a weekly experiment meeting with a fixed agenda: review what concluded and agree on the call, confirm what is launching next, and shape the next batch of raw ideas into hypotheses so the prioritized lane never runs dry. Thirty to forty-five minutes, same time every week. The discipline is in the recurrence, not the length. A program that reviews and refills weekly develops a cadence you can feel; one that meets when someone remembers develops long stalls followed by frantic catch-up.

How many tests run at once is a question people get backwards. The instinct is to run as many as possible because it feels productive. The reality is that traffic is finite, and every test running simultaneously splits the same pool, so ten concurrent tests each get a tenth of your traffic and most never reach the sample size to detect a real effect. You end up with ten inconclusive tests instead of two or three clear answers. So the rule is to concentrate: run only as many tests at once as your traffic can power to significance, usually far fewer than you want to. This is the same discipline that underpins an A/B testing program that works, and it is the single most common thing high-energy teams get wrong.

The learning log is the real asset

Here is the part almost everyone underinvests in. The lasting value of a growth program is not the wins. It is the accumulated knowledge of what works and what does not for your specific product and users. That knowledge lives in the learning log, and the learning log is the real asset the backlog produces.

A win gives you a one-time lift. A logged learning gives you a permanent input into every future hypothesis. Six months in, a good learning log means new ideas arrive pre-informed: you already know your users respond to social proof but ignore urgency, that pricing-page tests move the needle and blog-page tests rarely do, that a certain kind of copy change has failed four times. That accumulated context is what makes a mature program’s tests better than a new team’s, and it is entirely a function of whether you wrote things down.

Log losses and inconclusive results with the same care as wins, because they are often more valuable. A loss tells you a belief you held was wrong, which redirects real effort. An inconclusive result tells you the surface may not be worth testing further. A log that records only wins is a highlight reel, not a knowledge base, and it quietly lets the team relearn the same lessons every few quarters as people forget or turn over.

Keep it alive: re-scoring and pruning

A backlog is a living thing, not a document you rank once. New evidence arrives constantly, and every concluded test changes the priority of everything related to it. So re-score on a cadence rather than treating the ranking as fixed. A hypothesis that scored low six months ago might jump the queue once a support trend or a related test result raises its confidence.

Pruning matters as much as adding. Ideas that have sat unshaped for a quarter, or that a concluded test has effectively answered, should be closed, not left to clog the view. A backlog nobody trusts because it is full of stale, half-dead entries is one people stop looking at, and an ignored backlog reverts to the loudest-voice-wins default the whole system exists to prevent. Keep it curated enough that the top of the queue always reflects the best-evidenced bets available right now.

Roles, ownership, and tooling

Someone has to own the backlog. Not own every idea, but own the process: that shaping happens, that scoring is consistent, that the weekly ritual runs, that learnings get logged. In most teams this is the growth PM, and it is a core part of the job rather than an administrative afterthought. A backlog with no clear owner degrades into a shared document everyone can edit and no one maintains. How this ownership fits the wider role is something I cover in building a growth PM roadmap.

Ownership does not mean centralization. The best programs have many contributors submitting ideas and a single steward keeping the machine honest. On tooling, resist the urge to overbuy. A well-structured spreadsheet runs a serious backlog perfectly well: columns for the five fields, the state, the score, and a link to the learning. Dedicated tools help once volume is high, but they are an accelerant, not a prerequisite. I have run a program across hundreds of pages on tooling far simpler than people assume. The rigor lives in the process, not the software, and a team waiting to buy the right tool before running a real backlog is usually just avoiding the harder work of doing it.

Common ways backlogs fail

After running this at scale, the failure modes are predictable, and naming them is half the cure:

The backlog as a wish list. Ideas accumulate but nothing is shaped, ranked, or run. It feels productive and produces nothing.
Ideas without evidence. A queue of hunches. Without evidence, prioritization becomes preference wearing a number, and confidence scores get inflated for whatever people already like.
No learning capture. Tests conclude and the knowledge evaporates. The single most expensive failure, because it forfeits the compounding that makes growth work.
Celebrating only wins. Treating losses as failures rather than information. This punishes the honest bets and biases the team toward safe tests that teach nothing.
Running everything at once. Scattering finite traffic so nothing reaches significance, mistaking the number of live tests for progress.
No owner and no cadence. Without a steward and a weekly rhythm, the machine simply stops turning, quietly, and no one notices until the pipeline is empty.

The short version

A growth experiment backlog is an operating system, not a wish list: a prioritized, living inventory of testable bets moving through a defined pipeline.
Feed it from many sources; lower the bar to submit and raise the bar to run.
Every entry needs a hypothesis, evidence, a primary metric, expected impact, and effort.
Move ideas through a clear pipeline: idea, shaped, prioritized, running, concluded, learning logged.
Prioritize consistently, run a weekly ritual, and concentrate traffic rather than scattering it.
The learning log is the real asset; capture losses and inconclusive results as carefully as wins.
Keep it alive by re-scoring and pruning, give it one owner, and do not wait for fancy tooling to start.

Build this machine and the wins stop being lucky. They become the predictable output of a system that turns ideas into tested learning, week after week, which is the whole job.

I am Deepanshu Grover, a Growth Product Manager in Paris. If your experiment backlog is a graveyard of half-formed ideas, connect on LinkedIn or get in touch.