Starting something isn’t the hard part – stopping it is. Clear, agreed goals and a more sceptical mindset from the start makes it easier to make the hard decisions before it’s too late.
Written by:
Lars K Jensen
Lars is a former journalist who has worked with digital development, cross-functional teams and product management in the media industry for more than 10 years.
He is Head of Digital & Product at Willmore and an advisor to publishers and organisations on digital matters and how to understand the users and how to use that knowledge.
Reach out:
Email him at lars@willmore.dk
Follow Lars on Twitter or connect on LinkedIn
KILL YOUR DARLINGS. It’s important to set success criteria for your work and the ideas, products and projects to decide to act on. The criteria help you value if that great idea was in fact so great.
But success criteria aren’t always the right way to go. What if we instead talk about survival criteria and pledged that we would stop the process or the project if it did not meet certain, clear demands? Whether you are developing a new product, concent or area of business or planning a redesign you need to have a serious conversation about what to achieve, before you are ready to commit the needed time, energy, resources and money.
Here, success criteria are important but they are in no way a guarantee that the necessary conversations will happen and the necessary decisions be taken. Let’s look at some of the risks associated with the success criteria approach.
Make it Measurable
One of the challenges when using success criteria, is that they easily end up as a footnote. One of the frequent reasons for this is that the criteria are not measurable. They are the easiest criteria to agree on – but it is also the ones that matter the least. Non-measurable success criteria can therefore also be a symptom of an organization that isn’t in harmony with itself.
Naturally you would never aim for something as intangible and silly as “it has to be nice” or “the best solution in the market”, but you just as might. When you can’t measure whether a set of criteria are met who they actually say is secondary. They are just words.
Un-measurable criteria mean that you don’t stand a chance of actually assessing whether you succeeded or not. After all, it makes a lot of difference in reaching 70 or 17 percent of the amount of subscribers you had to sign up before that important date.
It is alwaays a good idea to have something to measure against, so you have a baseline. For instance, you can’t evaluate the potential success of your revamped newsletter if you don’t include data from the old newsletter – things like:
- Has the opening rate increased?
- Have we lost some subscribers?
- Are we seeing less traffic from the newsletter now than earlier?
- Etc.
There will always be someone putting their heels in, when you start talking about making success criteria measurable and tell you that that is not how innovation works. Sometimes I have a tendency to do the same (remember that change is hard) and the argument isn’t without some truth to it – but it’s crucial that you are ablee to define measurable criteria. The litmus test is to measure the right things; the ones that make sense.
No one is motivated by KPIs and other targets who are just their, because there has to be something measurable. It has to be done properly.
And remember: It’s not just about you.
Use Your Users
Because there is another risk – and that is the risk that what you are building isn’t what the users need. It needs to have value for the users, so logically that is what you should be measuring.
Sure, some internal success criteria matter – like technical performance, reponse time, transfer between various systems etc. – and they need to be included as well. But it’s easier to upgrade on the technology than to try and convince people that they actually need what you’ve spent the last six months building for them.
Done the right way, it begins with finding out what the users and customers are missing and what pains they are experiencing when trying to get something done, either personally or professionally. From there it’s about finding out what to build – and if it is feasible.
Success criteria based on knowledge about your users are much easier to make measurable. And the best part: Now you are actually focused on delivering value – focused on being something to someone.
It’s actually quite simple: Fall in love with your users’s problems – not your own ideas.
If you – for example – know that it’s important for your users to be able to easily share content or data with their colleagues, it’s a great idea to think of ways to make that even simpler and better.
With something as relatively simple as a newsletter there are numerous data points: Number of recipients, opening rates, click rates, openings per recipient, reading time, traffic from newsletter etc. But which are the most important for your newsletter or email product? Well – it depends on what delivers the most value to your users.
Don’t Save the Best for Last
A classic mistake is to wait until just before launch before assessing whether something is ready for launch or not.
Let’s say that a publisher has developed a new section, “universe”, “portals” or a new concept to be part of a website. If the success criteria solely address how many sign up after launch after the process has run through, the development phase is lightyears away.
And by the way: The people working on the project have probably been assigned to other projects now, so just forget it.
These are some of the classic issues with what is popularly refererred to as the waterfall model. This is where a project is driven through various phases – typically conceptualizing, design, development, implementing, test and finally launch. If you can’t evaluate the product or project until after launch it’s as good as impossible to go back to an earlier stage and change something.
No Consequences
But the biggest disadvantage with success criteria – if you ask me – is how they are implemented. Way too often they are reduced to somewhat of a sideshow and either something that is agreed on upfront and then forgotten or done in the last moment because “oops, we need some success criteria”.
Sometimes it’s about politics in the organization (which can make or break everything…) but often it’s about the fact that stopping things is hard. Much harder than starting them.
And if something doesn’t meet the criteria… then what? It’s a failure, right? Well… Yeah but no but… It’s hard to decide once the wheels are in motion if you didn’t agree to it upfront.
Even though it might make sense in the moment to let a project or product live on even though it’s struggling, it’ll hit you in the long run. This is especialy a risk if you are prone to think in projects instead of products. Projects are something with a start and an end – and when a project is done, the next one starts.
With a project mindset you risk losing the overview of all the things you are launching and maybe – maybe not – succeeding with. If you work with a product mindset instead, you have Product Owners or Product Managers with the responsibility that their products are performing – or being shut down, if that’s what it takes.
Even the best development or R&D departments will succumb if the organization launches too much that isn’t good enough – and which will require maintenance and cost money without delivering proper value to the users and the business.
Harvard Business Review had an interesting piece on this recently, titled “Start Stopping Faster“:
“It’s one of the most common laments of executives struggling to increase their organization’s adaptability: ‘We are terrible at stopping work, even when it’s obvious that the work is a complete waste of time and money.'”
When working with products how do you ensure that a Product Owner is ready to close a product – if that’s the very product they own? The HBR article has an interesting take on this:
“Finally, giving people more opportunities if their current project fails reduces the likelihood that they’ll stick with a bad idea longer than they should. Successful companies build a strong and visible backlog of compelling opportunities. They make it clear that until existing projects that aren’t panning out have stopped, new initiatives can’t be launched. And they redeploy people from the former to the latter as a matter of policy and offer training to ease the transition. In time, the fear of missing out on something better starts to overpower the fear of loss.”
Succes Survival Criteria
What if we instead of success criteria (which expresses what we hope to achieve, often without consequences, to be successful) talk about what it takes for you to not kill the idea?
That’s the general idea behind “Kill Metrics”, which they are using at for instance Google X, where they pioneered this approach to innovation.
Google X is Google’s “Moonshot Factory” – that is the place where they work with some pretty crazy-sounding, far-feteched ideas. But you certainly don’t need to work with big and scary experimental innovation to learn from theme.
The article continues below ↓
David Rowan (who for a number of years was the Editor in Chef of the British edition of Wired magazine) writes about the Google X metrics in his book ‘Non-Bullshit Innovation’:
“To gain approval for an X project, a team must agree up front to a aset of ‘kill metrics’ – criteria that determine whether an otherwise promising project should be ended. Kill metrics help identify the riskiest parts of a project from the start, before the team is too emotionally invested”
Not only is this about finding and defining the hurdles, the project has to pass not to get closed. That mindset also forces you to focus on the most important and riskiest parts – just like it forces you to talk about them right at the start. Let’s be honest: The sooner you can stop a bad idea, the more hours, resources and money you can spare and instead spend on something that will deliver actual value.
There is a great difference in whether you are discussion what it takes for Project XYZ to be a success – or whether you are discussing what Project XYZ has to accomplish so you won’t kill it.
“The Secret to Moonshots? Killing Our Projects,” Google X boss Astro Teller wrote in a Wired article back in 2016:
“I have a secret for you. The Silicon Valley hype machine has created this myth of visionaries who effortlessly build the future. Don’t believe the hype. The moonshot factory is a messy place. But rather than avoid the mess or pretend it’s not there, we’ve tried to make it our strength. We spend most of our time breaking things and working to discover that we’re wrong. That’s it. That’s the secret. Run at all the hardest parts of a problem first. Ask cheerfully, “How are we going to try to kill our project today!” We’ve found a balance that’s working for us — allowing our unchecked optimism to fuel our visions and then harnessing enthusiastic skepticism and critical thinking as a way to breathe life, breathe reality, into those visions.”
It’s Kind of a Talent Show
It’s about seeing your pipeline less a a pipe where everything passes through and more like talent show where only the very best contestants make it through several elemination rounds, to the live shows – and in the end win.
It’s an expense to develop something – but mark my words, maintaining something is an expense as well. Especially if something isn’t delivering value and/or not performing. Well then, does it really need to exist?
It can easily cost you more than just cold cash to have a product or service running which is merely “good enough” – and maybe not even that – which you can’t pull yourselves together to pull the plug on.
In the comparison with the talent show, it’s the difference between having one winner which goes on a sold-out tour (once concerts are happening again, of course…) – or 15 winners who perform for in half-empty malls around the country. This first one is the funniest. It’s also what’s best for the business.
Maintenance Kills
I’ve both seen if myself and talked to several others who have described how the sheer execution force and even the abiliity to think, conceptualize, develop and launch news products is getting drained and bogged down by maintenance.
Way too often it is maintenance of products and services that aren’t even successes – and which should have been closed long ago.
Sometimes the reason for this is that a certain someone (usually someone at the C-level) has had an idea which circumvented the usual checks and balances in the organization and made it to launch. Here, the risk is that you end up with something which very few (sometimes no one) want to use – but without the tools, focus and time to get it shut down properly. Argh!
“Kill Metrics” need to apply to everyonee’s ideas. Even the boss’s.
Another problem with having too may poor products and services running is that they collect debt. Not the usual debt where they owe someone money – but debt in the way that they start lagging.
This is sometimes called “technical debt”, and it basically means that you are running on old technology, that might not even be supported anymore. Needless to say, this isn’t good.
A better word might be “product debt“, which tries to shift the (unconscious, perhaps) focus away from blaming developers and the IT development.
“Both organizations and product owners can fall victim to operating a feature factory. If you don’t see the context and what value you’re bringing to your customers, you will only build the things right.
Instead of building the right things.
Look at Airbnb, Twitter, Uber, or Spotify. You don’t question what these products do because they made conscious decisions regarding what they offer and what they don’t — thereby managing their product debt.
While it’s not possible to avoid it altogether, product debt is only dangerous if you let it pile up over time. The most important is to recognize its existence.”
I think everyone who works with the development of products, tech or the business knows the furstration of not being able to work on developing and generating valuee to thee users, because some ill-considered and half-baked solution is limping along somewhere in the tech stack and keeps breaking down.
“Let’s shut it down or do it the right way,” you say to each other. But you rarely get to do anything about it.
Get Busy
Whether you are a new organization (maybe a startup) or have been around for some years there really is no excuse for not jumping into the right way to do these things.
The “Kill Metrics” applies to new ideas and projects as well as the products and services you already have running.
But no matteer who you are, it starts at the top. If management and the C-level can’t see why, you might as well continue doing things the old, ineffective – and wrong – way. It’s that simple.
Luckily, “Kill Metrics” and survival criteria make great sense from a business purpose. So try and win them over by talking about the company could use it’s resources better, get a clear focus – and maybe even make an honest buck as well.
“Kill Metrics” actually means that you have to kill the idea if it isn’t good enough.
Next, you’ll need a place to start. Is it a new product/project or something existing that will get the new treatment as the first thing? Now is a great opportunity to settle what needs to be done with that frustrating solution you are still supporting despite the fact that no one is using it.
Have the guts to say “if X hasn’t happened until Y, we’ll shut it down”.
And when you end something, be sure to dismantle it in a way, so you keep the parts worth keeping. And make an internal note or memo on why you are closing it. That will come in handy when the idea re-surfaces (they usually do – sometimes with a “maybe if we just…” attached) and when a central factor might change everything.
Stay updated
We are here to connect and help the publishing and media industry through live events, morning talks and great content.
Sign up for our newsletter and join the pack:
