A cart-before-the-horse problem
This situation represents a category error. And it's not about whether you consider the signals you're going to look for, whether you'll use tests and whatnot. Of course you should!
It's about when and why.
On the one hand, you could let an idea—lots, even—emerge through play and exploration, without knowing or caring how you’re going to choose between them. You've no clue beforehand what metric or criteria the idea is going to be “tested” on. You just have to come up with ideas.
On the other hand, your starting point could be to find out how the ideas are going to be graded. Then you use the grading system to reverse-engineer ideas that you're confident will beat the test. This is another beast entirely.
If your goal is to find genuinely effective new ideas, the second approach defeats the point of the test and introduces risk in the long run.
Why though? Isn't it sensible to start with metrics? Surely, if you can find a suitable metric to use as a starting point, there's nothing wrong with fitting ideas to the test?
The issue is, the ultimate “test” of your idea is going to be The Market. In all its complexity and ambiguity. There is no single or simple or even legible test that will 100% accurately reflect the future behaviour of the market.
The eyes (don't) have it
Any test you do has the potential to give you valuable signals and help you unfold your ideas over time. But the test can't do that for you if your goal is to get top marks, rather than seeking to learn more about your as-yet-unknown reality, and the interactions between your idea and reality.
A simple test that sounds un-game-able (yep, we've used it ourselves in the past) is: when you show someone your idea and they engage with it, do their eyes light up? This is a useful test criterion because it's tough to game – you can't make people's eyes light up by asking leading questions or cherry picking nice things they said about your idea, although you can certainly misread the eyes with enough motivated reasoning. But the problem can start further upstream. As soon as you ask “what’s going to make someone’s eyes light up?”, you’re already lost, trying to predictively reverse-engineer a response that comes about from surprise.
(Imagine trying to reverse-engineer how you could get a friend to think you got them a brilliant present by figuring out all the nerve impulses that will make their eyes light up. Eyes lighting up is an emergent consequence of lots of factors, many of which are outside your control or awareness.)
A brief meltdown
If you've ever been a designer or worked with designers, you'll recognise this scenario:
The designer busts a gut trying to get the project owner to specify exactly what they want. (“Like this and like that, but not like that?” ... “It will have an image? In your mind’s eye, what is that image?”)
The designer then uses the material they've gathered to structure the brief.
The painful part? This almost never works.
What happens is that the designer's solution gets rejected, and another design gets chosen that DOESN'T EVEN FIT THE OFFICIAL BRIEF, but instead fits the real brief that was hidden behind the attempts to clarify the brief. The real brief is always, "something unexpected that also feels obvious when I see it, and clearly I can't tell you what it is, or I'd expect it".
You might also recognise this as a maxim in all of software development. The client doesn't really know what they want until after you give it to them, and then it's not what you gave them.
Yup. Fist-clenching stuff, the absence of a logical, deductive path to get the right answer.
The snozzberries taste like snozzberries!
"There's no earthly way of knowing. Which direction we are going. There's no knowing where we're rowing. Or which way the river's flowing."
– William Wonka, founder, CEO, FMCG innovator and billionaire
It's like when we write these articles for The Reach. One word in front of the other. Not “to the brief”. Not the piece we'll end up publishing. We have to let rip on the raw material, then let the idea crawl out of the mess.
We pass the draft back and forth between us, figuring out what's working and what point we're making. "No, it’s not that, it’s—” (One of us has just now deleted four opening paragraphs that were throwing sand in the face of the point.)
Letting the idea reveal itself naturally by letting it take us where it needs to go, and then seeing and responding to how it interacts with the world. In our case, each other, followed by ... you.
Once the initial vertigo of uncertainty has faded, it feels better than structuring a piece to make a point that we suspect someone might want to hear and the writing comes out sterile and lifeless.
We see this tension with our clients too. They want to launch a product or service that their customers can't ignore because it has a soul. Or in less whimsical terms, because it's coherent. It has that innate characteristic of being a "thing" that makes sense because of its thing-ness. It hangs together. It's not just a set of A/B test wins and optimisations tied together with string.
But how do organisations tend to approach this challenge by default? Often it's by trying to grip tighter. Reverse-engineering for metrics that represent a stickman drawing of a soul.
The issue at work here is that folks are mostly waiting for someone else to say exactly what to do. It seems safer, it seems more efficient. But in uncertainty, there is no "someone else", and there is no "exactly what to do".
Create a playground ... inside a bubble
How do you give people the goal of being less focused on the goal?
One answer is to create bounded space for play.
Imagine two kinds of play:
First, you're sitting around with a group of buddies before you go to a playground (one that's, you know, open to grown-ups), trying to align on an optimally fun new game. When you've agreed on the game, you can go execute it.
Now imagine you and your buddies just go to the playground, start mucking about, and see what emerges. Some of what you try will be stupid and not-fun. Someone might skin their knees. But some fun new games will emerge through just doing stuff. Games you never would've thought of if you were sitting at home trying to reverse-engineer "fun". (Don't worry, later you're allowed to refine and optimise your favourite game.)
The trick lies in creating a structure that makes it easier to start exploring than to start reverse-engineering. This means removing the pressure to justify or measure everything, but it also means putting in place temporary constraints that encourage productive mucking about. Generate raw material. Let it interact with reality. Use testing to learn, not to engineer a win. See what unfolds.
This is what we call an Uncertainty Bubble, and we'll share more on those soon. They're a kind of setup that enables coherent new ideas to emerge. Ideas that you couldn't have otherwise had. Ideas that can't be reverse-engineered.
The Market will test you eventually anyway. It wants "something unexpected that also feels obvious", and you can’t reverse-engineer your way past that.
P.S. If this pattern feels familiar — if you've watched smart teams get stuck reverse-engineering their way into mediocrity — let's talk. We're collecting stories from leaders navigating uncertainty, and figuring out what actually works. Hit reply if you'd like to compare notes.
