How to create a product experimentation culture

Have you ever considered why it is that perfectly good ideas for products have quietly disappeared, while mediocre ones have come to dominate their market? The answer is likely in their growth model.

Product-led growth is something that I have written about a couple of times already, because it really can be the key to success. When the product is just so compelling that its popularity spreads from user to user, and they are finding it meets their needs so well that purchases and upgrades are made without specific prompting, growth takes on its own momentum. Several interesting things happen when a true product-led growth approach is implemented: innovation is expedited, spend on advertising and customer support can be controlled, more users are retained, and users spend more.

Why product experimentation is important

 Central to product-led growth is the testing cycle and how it guides the direction of travel. With dependable data on how users respond to new features, it’s possible to make truly informed decisions about product development, and achieve the associated benefits. The advantages of a product-led growth approach can include:

• Speeding up the development cycle, to innovate faster and more effectively (partly by honing focus, and dropping unsuccessful ideas quickly)

• De-risking projects, by enabling better understanding of users wants and needs

• Capturing a true picture of users’ preferences, to provide a solid basis for decision-making

• Finding the best solution before sinking significant resources into actually building it

• Democratising the decision-making process; good ideas can come from anywhere within the company

Why is a culture of experimentation not just the norm?

 With all its benefits, product-led growth might seem an absolute no-brainer. But if that is the case, then why isn’t everyone doing it, and doing it successfully? The answer is, commonly, that they are not carrying out the volume of testing to achieve the insight required to make a difference. There are a few reasons for this, such as:

• Risk. There are some sectors in which this approach would not be at all appropriate, since getting it wrong would result in catastrophic consequences. For example, nuclear power station control systems are not the place to begin experimenting in a cavalier fashion! However, this type of barrier to product-led growth only applies to a small number of cases, of course.

• Cost. On a related note to the point about risk, if you can’t afford the cost of failed experiments (even a tiny ‘failed’ experiment), then you can move a lot more slowly and try to get everything right. Of course, this is a risk in itself as you might not hit on the right solution, or you may get beaten to the finish by a competitor.

• There is no downside to just doing it. This is another rarity, but in cases where there is no cost to building the whole thing, then it might be better to just do it, and not bother experimenting.

• Size. Very small companies might have insufficient people to build an experimentation culture in any kind of structured way (though that’s not to say they can’t adopt the mindset).

Apart from these notable exceptions (some of which are quite rare), the obstacles to a full product-led growth approach are often more about shared behaviours, beliefs and values. The predominant mindset in business is still that failures are wasteful, and that predictability and efficiency are important, which makes managers and leaders reluctant to allocate resources to anything experimental. But of course, without experimentation, innovation is hampered.

 With the right approach to experimentation, innovation can be both faster and better, so the (controlled) risks can certainly be worth it. Overcoming the aforementioned barriers can be a challenge, but one that is worthwhile tackling – and the way to do that is to tackle the prevailing culture within your organisation.

How can you embed a culture of product experimentation?

 The short answer is ‘make experimentation part of the everyday.’ To do this, broadly speaking you must create an environment that encourages creativity, where data trumps opinion, and where anyone can have an idea and run a test (not just those within R&D, cross-functional teams are pivotal in maximising chances of success). 

1. Establish a context in which people are free to experiment, and provide support to help people get to grips with ‘failing on purpose’. Make sure they are aware that it’s okay to fail, and that the learnings achieved by experimentation are more important than getting things right first time. When there is psychological safety within the team, and they know that there is no negative outcome triggered by getting it wrong, they will not be afraid to try new things, resulting in a much-improved capacity for creativity and innovation.

2. Build an experimentation team responsible for viability, comprised of a product manager (who also owns responsibility for customer value), a product designer (who owns usability) and a technical lead (who owns feasibility).

3. Encouraging curiosity across the workforce is critical. How can individuals use their knowledge and experience to suggest improvements or new features? Give them a way in which to do that, and set out how to show that they have moved towards your organisational goals. Be explicit about what the goals are, and how you will measure them.

4. It’s important to encourage a culture in which anybody can have a good idea, from the admin assistant to the CEO; this diversity of perspective supports the best possible chance of coming up with the right user-centric improvements. Leaders must be humble, and recognise that just because an idea didn’t come from the top, doesn’t mean it isn’t a good or insightful idea.

Once these steps have been followed to set the tone for a new culture of experimentation, you should create a process for people to follow when they have an idea they want to experiment with. The following steps should all be considered and included:

• Set a company-wide challenge in the form of a question that you want people to answer: would doing X drive more Y?

• Create a hypothesis document to record and share thinking. Our Blue Ocean Insight hypothesis template is a handy tool for this, drop us a line to receive a copy. Treat each hypothesis as a ‘fail’ until proven otherwise, which helps to ensure a data-led mindset, rather than getting too excited about the idea itself.

• Decide on the goals and objectives for your experiment (the BOI template helps with this).

• Decide on a small number of variables. The ‘small’ is key; you can’t play with 100 variables and try to tweak them all, as you won’t know which one made the difference.

• Decide what metrics you need to prove the idea is failing and then collect those signals (if you can’t prove a failure, then it is probably a success). So-called ‘pirate metrics’, ARRR play well here, as do HEART, while a mix of leading and lagging indicators can also be worthwhile – though watch out for ‘vanity metrics’, Goodhart’s Law, and confirmation bias. Also be aware of ‘the tyranny of the marginal user’, which can result in products actually becoming less useful, because the focus has been too heavily on customers simply using a product, rather than how well it fulfils their needs.

• Decide when you will have enough data to make an informed decision (and consider when to bail from the test if it tanks).

• Consider ethics. Although inserting a formal policing or ethical review board process is to be discouraged (it creates a bottleneck, and detracts from the team feeling empowered), some awareness and oversight is wise, to avoid a backlash such as experienced in 2012 when Facebook decided to undertake some tests that were effectively a kind of emotional manipulation.

• Check that you will get actionable answers. Think about the value of knowing the answer to your question. If you had the answer, what action would you take? If you aren’t sure, it probably doesn’t tell you much. Will you understand why a user likes feature A over feature B? If not, what will you do with the experiment?

• Identify your Ideal Customer Profile (ICP) and understand it (or them, if segmenting); try a combination of these tools:

  • User personas

  • User testing

  • Analytics and data analysis

  • Social media listening)

  • Competitor analysis

  • Customer Support interactions

  • Beta/Lab programmes

  • Once you have your ICP [Ideal Customer Profile] and your question, try testing your hypothesis and see if the data support your hypothesis.

• Once you have actionable answers, take the actions that had been planned out during the design of the test.

The efficacy of the product experimentation approach is evidenced by companies that have made the leap of faith involved in undertaking tests without knowing whether they will fail or succeed, and found they could accelerate their development cycle and provide a better user experience: Amazon, Microsoft, Google, and Booking.com – and many, many more – are all examples that prove product experimentation works.

If you’d like to talk in more detail about how to nurture a product experimentation culture, and reap the rewards of the product-led growth model, send us an email at info@blueoceaninsight.com

Is my team truly Product Led? Spotting the Red Flags

In this article we consider what questions a leader or investor should be asking a tech business to understand whether that business is truly product-led, and if it has positioned itself to reap the benefits of this approach. (For a recap on the concept of product-led growth, read my short guide here.)

An effective product-led team will:

 ●       Understand the risks of building a product and embrace and use these risks

●       Prioritise for success

●       Measure progress well and often

Tech and product can seem impenetrable to those not immersed in the day-to-day workings of a business. But by asking the right questions you can gain the insights you need. You can understand where a business has got to in its product-led journey.

As I go through the questions, I’ll highlight the Red Flags that you should look out for. They’ll help you to spot if a team is ‘walking the walk’ of product-led growth or just ‘talking the talk’.

Question 1 – Are you risky?

 Some tech organisations – and the people within them – feel their job is to avoid risk. But we can use risk to help us achieve things we couldn’t achieve otherwise. I like to think about risk in terms of a ‘contained blast radius’ – what’s the worst that could happen? If we can answer that question then it will tell us whether we can afford to accept the risk associated with a decision or course of action, even if it doesn’t turn out as we hope.

When thinking about developing a product, these are the key risks the team should be thinking about:

• Value – If the product doesn’t give value, nobody is going to buy it. The team needs to think hard about what value the customer is going to get. Will they view it as something they ‘need to have’ or is it just a ‘nice to have’? For a customer, value must be intrinsically linked to something they care about NOW, not something that might be important to them in the future.

•  Usability – Can we make the product usable and friction free? This risk grows where the team doesn’t talk to customers, use prototypes or focus groups to hone usability.

•  Feasibility – Is it technically feasible to build the product in a particular way? It is vital to work closely with technical teams to try and pre-empt future problems.

•  Business viability – Are we the right business to do this? Are there other competitors in the space who can simply do this better than us?

 I would ask a team to explain to me each of the key headings of risk, and how they deal with and mitigate each one.

Red Flags

• It’s a red flag to me if a team can’t describe the exact way the customer will derive value. I want to hear that the team understands their customer and the problem the product will solve for them.

• It’s a common scenario that teams misuse the concept of ‘minimum viable product’ or MVP. The purpose of using a MVP is not to cut corners or meet roll-out targets that would otherwise be missed. It’s a tool to get feedback as quickly as possible and uncover the value that you are – or should be – giving to customers.

• Sometimes, companies say their product is a ‘toolkit’ – something that serves many different purposes, and there are many different ways a customer can use it. When I hear that, I hear that the product doesn’t have a sharp focus. When you’ve nailed the product, it won’t be a ‘toolkit’ – it will a hammer, or a screwdriver, or whatever tool the customer needs, at the time, to solve their problem.

Question 2 – are you prioritising for success?

To break this question down, I would ask a team:

•  How do you prioritise? A key part of being product led is responding to what the customer wants. But sometimes you can have one ‘loud voice’ customer who can skew priorities unless an objective prioritisation process is followed. I advise teams to build up a quantitative analysis with a RICE score (Reach, Impact, Confidence, Effort[1]). Reach considers how many people will this feature affect within a given time period. Impact is more of a qualitative question – what impact will it have for customers? Confidence asks how confident are we about the Reach and Impact scores? Effort is a high-level estimate of the number of Person-Months that it might take to develop the idea. It isn’t a commitment to deliver on a given date, it is just an indication of the level of investment the organisation will have to make to deliver value to the customer.

• Where do you start with an idea? For this, we need to go back to first principles – what problem are you are trying to solve, and why? It involves clearly defining the problem, defining the solution, and defining what success looks like (for more on success metrics, see below).

Red Flags

• Alarm bells ring if I hear a team saying things that essentially mean, ‘We know better than our customers. They’ll understand in the future why X, Y or Z is important.’

• Even worse, a team might be thinking, ‘Our customers aren’t smart enough. They’ll get it at some point.’ In my experience, most customers understand very well what their problems are. Even if they don’t, why would they pay out for a product whose value is not clear or relevant to them NOW?

Question 3 – How do you measure success? (Or ‘Are we there yet?’)

I want to see teams taking a rigorous and scientific approach here, for example:

• Release measurements – when new capabilities are released, you can use the AARRR approach (aka ‘Pirate metrics’!) to measure:

  • Acquisition ­– how many people are discovering our new feature?

  • Activation – are they taking the actions we want them to - are they signing up for the new feature and using it?

  • Retention – do they want to keep the feature?

  • Referrals – do they like it enough to tell others about it?

  • Revenue – are they willing to pay for it?

•  Product measurement – Is my overall product hitting the mark? Various metrics can be used including:

  •  Net promotor score – This is calculated by asking the customer, using a 1-10 scale, how likely it is that they would recommend your product to a friend or colleague. A positive score only occurs where a company has more ‘promotors’ (who score 9 or 10) than ‘passives’ (who score 7 or 8) or ‘detractors’ (who score 0 – 6).

  • Churn / Customer lifetime – Churn is the rate at which customers stop doing business with a company. Customer lifetime is 1 divided by the churn rate. E.g. if your yearly churn is 10%, then this means the average customer lifetime is 10 years.

  • Net revenue retention – This shows the percentage of recurring revenue that is coming from existing/repeat customers over a set period (i.e. disregarding revenue from churn), and is useful to predict potential for expansion.

  • Time to value – This is the amount of time it will take for a customer to get the value they were expecting from your product. The shorter this is, the higher the likelihood of retaining that customers / avoiding churn.

  • CoCA – Cost of Customer Acquisition, i.e. the amount of money the business spends in order to get a customer to purchase its product (all sales and marketing spend, including salaries, bonuses, overheads etc).

  • LTV – Long term value or lifetime value. How much will we generate from the customer before they churn?

 Red flags

• Not measuring success at all, or not in any meaningful or quantifiable way.

• Measuring too many things - the ‘NASA control room’ approach! There are dozens of metrics that could be used to analyse the data you receive. But the problem is you can end up with too much going on and the important takeaways get lost in all the noise.

• Mixing product metrics with sales and marketing metrics. A good product ultimately delivers revenue, but the product itself needs to have its own metrics. It’s hard to ask an engineering team to increase revenue by 20%. But you can ask them to build something that solves a customer problem in a really effective way – and the revenue will follow.

• Lack of clarity. It’s a red flag when a team is unclear about the cost of customer acquisition, or doesn’t understand LTV, and has no plan to quantify or understand these.

Conclusion

 Where a team does have meaningful answers to these questions – or a clear plan as to how they will be answered – then this reassures me that the business has the mindset and approach I’m looking for. For me, this is when it gets truly exciting to be involved with a business – when the path ahead is truly product-led and bursting with potential.

 If you want to chat more about product-led growth, and the transformative power of using this approach in business, please contact me by emailing Matt Little at Blue Ocean Insight.

OKRs – Hearts and Minds

In this article we consider how to empower teams to achieve OKRs (Objectives and Key Results) as well as pitfalls to avoid.

In Part 1 we met Jay, who had decided to run a marathon and was setting some OKRs to help him achieve his goals around health and wellbeing.

Now Jay has another goal in his sights – he’s going to seek sponsorship for running the marathon to raise money in memory of his dad. But first he must get his team on board.

The only problem is that this involves getting his teenage daughters off the sofa – and off their phones!

Jay has a think about what OKRs to set.

For Kat (who plays for the school hockey team and is the “IT-savvy one”) he sets the following OKRs:

For Abby (the dreamy, “artistic one”), he goes with the following:

Jay is storing up trouble here!

As we saw in Part 1, Key Results should communicate where you want to get to over a specified time period, and they should be ambitious. But Jay knows the girls have busy and complicated social lives and he isn’t sure how long they will be able to keep up their fundraising efforts. So, he’s just going to see how they get on. He hasn’t set an overall fundraising goal, to be met by a specified date.

What he’s actually done here is set the girls a series of tasks (calling them Key Results) in the hope that this might move them towards a vague, unspecified goal.

Jay worries that Kat isn’t very creative so he hovers over her as she sets up the fundraising page, suggesting what photos and wording she should include.

He asks his wife, Sally, to supervise the baking and make sure Abby doesn’t get carried away and spend too much on the ingredients. (Abby isn’t impressed.)

Jay calls a team meeting after the first week.

Kat’s fundraising page is up and running and three people have signed up – Mum, Gran and Auntie Sue have pledged £50 each.

However, the website charges a fee of 7%. Abby helpfully points out that they would have been better off just asking Mum, Gran and Auntie Sue to hand over the cash. Kat gives her one of her withering looks, but admits she just went with the first fundraising site that came up on an internet search and hasn’t researched the pros and cons of different sites. She doesn’t admit that she thinks it’s “lame” to share posts about Dad running a marathon…

But progress against her Key Results is looking fine - the amount raised in the first month is high (thanks to Mum, Gran and Auntie Sue) and she’s set up her Twitter account to automatically share the link 20 times per week (mostly in the middle of the night, when nobody will see it).

Abby held her first bake sale last Friday after school but hardly anybody came along, and most people didn’t have any cash on them anyway. She ended up selling all the cakes to Gran afterwards at a fixed price of £50.

Her progress towards her Key Results is therefore looking okay for this first month, but she won’t be able to replicate this every time (without seriously affecting Gran’s waistline, anyway.)

Jay is running into problems with the task-based Key Results he has set.

They are not focused on top-level outcomes and so the girls are approaching them as a box-ticking exercise. On a superficial level, it looks like they’re making progress, but they have burned through the low-hanging fruit (Mum, Gran and Auntie Sue!) and will quickly run out of steam unless they find new ways of reaching potential sponsors.

Crucially, task-based OKRs can be overly restrictive. They deprive teams of the autonomy they need to innovate and optimise.

Jay realises that the OKRs aren’t inspiring the girls. They don’t communicate his vision. They don’t tell the story of what he’s trying to do.

He needs to win the hearts and minds of his team! So he reframes his Objective:

It’s a slightly scary Objective since they’ll need to raise a LOT of money. But the girls put down their phones – they’re finally interested. They explore the charity’s website and find an adorable dog – Larry - they’d love to sponsor.

Kat goes away and researches fundraising sites properly – she doesn’t like the idea of the money being eaten up in fees when it could be going towards sponsoring Larry.

Abby tries harder with her bake sales. She makes a poster with a photograph of Larry with his dog friends at the charity. She writes some heartfelt words about what a disability assistance dog might have meant to Grandpa, who was in a wheelchair for the last few years of his life.

Jay holds another team meeting in a couple of weeks. But things are not going as planned!

Kat’s fundraising page is being shared plenty of times (she’s even sharing it during the day now) but nobody is signing up.

Abby made £23 from her last bake sale, which is even less than the first time. Abby explains that nobody ever has any money on them, and they’re always in a rush after school.

Jay realises there’s still a problem here. Although he’s set an inspiring overarching Objective, the Key Results he’s set for Abby and Kat aren’t aligned.

Abby’s bake sales are being run in a vacuum - independently from Jay’s marathon efforts and the fundraising web page. Kat’s fundraising page isn’t taking off ­– she doesn’t have Abby’s skills of creating inspiring content, so people just scroll past.

Abby and Kat are working independently to follow their own goals – the task-based OKRs that Jay has set for them. They are working in silos oblivious to what the other is doing.

It is crucial to make sure that goals are aligned, company-wide.

In my Insight Paper, Silos: Why Focusing on Vision (not communication) is the Answer, I consider the example of a sales team who promises a new client an amount of the technical team’s time in order to clinch a new deal, which means that the technical team gets pulled away from their own team goal (product development) in order to fulfil the sales team’s promise. Resentment, frustration and stress inevitably build, leading to disconnection and further entrenchment of silos.

Melissa Perri’s excellent book,Escaping the Build Trap, considers this type of scenario in more detail.

Jay realises he must step back and stop micro-managing. He sets new Key Results which are aligned, measurable and time-bound – Abby and Kat must raise £3000 in sponsorship in the next three months.

Kat and Abby respond in unison: “Are you actually joking?”

He shuts them in a room with takeaway pizza and cold drinks, and goes out for a run. The best thing he can do now is to get out of the way.   

The girls realise the only way they can do this is to work together.

Kat (who Abby now refers to as “the Tech Team”) creates some QR codes for Abby to put on her posters. People can use their phones to scan the QR codes and go straight to the fundraising page.

If people donate £5 on the fundraising page, they get a “free” cake. Abby can now raise £5 per cake instead of selling them for 50p! People don’t need cash, and Abby decides that if the person shares, on social media, that they made a donation, they get a free chocolate-covered marshmallow – this will help Kat’s efforts to raise visibility of the page online.

Kat also makes the sneaky suggestion of holding the bake sales on a Saturday morning, after the school sports matches. This will open up the fundraising to people from other schools – and parents who have come along to spectate. And she knows from her experience on the hockey team that players will be starving after playing their matches and more likely to pay £5 for a piece of cake.

Mum realises she doesn’t need to hover over Abby. Instead, she asks what resources Abby needs (more baking equipment, more money for ingredients.)

Abby (now known as the “Content Queen”) helps Kat with the layout of the fundraising page. She adds pictures of Grandpa, and of Larry the dog, and some inspiring wording. Kat is much happier to share the page now that all photos of Dad in his running gear have been deleted!

Teams will self-organise and figure out the best way to achieve a common goal – as long as you give them the autonomy to get on with it.

Communication between teams is important, but this is something that will happen organically when their goals are properly aligned with each other.

This leads to cross-pollination of ideas, collaboration, and innovation. And ultimately, more efficiency and productivity.

At the next team meeting, Jay checks in on the girls’ progress towards their new Key Result (raising £3,000 by the end of three months).

Kat’s progress is looking very healthy now that she is no longer embarrassed to share the fundraising page, and its new content is inspiring people (not just family) to donate.

Abby is also doing fantastically. She’s doing a bake sale every Saturday now – there’s no shortage of hungry sports players.

The girls keep fundraising for all they’re worth. Jay trains harder…

Jay – eventually – chose brave, exciting OKRs that inspired his team and told the story of where he wanted this venture to go.

They are now entering the space where that story just might come true.

If you’d like to talk in more detail about how to set OKRs that will improve innovation, efficiency, productivity, morale and ultimately profitably in your business and how they fit with your KPIs, contact Matt Little at Blue Ocean Insight.

OKRs and KPIs – What’s the Story?

I’m often asked how to set “good” OKRs (Objectives and Key Results) and how these should fit around a business’s KPIs (Key Performance Indicators). These concepts are often confused or conflated and it’s easy for the meaning to get lost in the jargon.

KPIs are measures of the health of a business, but OKRs are more than this – they are measures of change and ambition. They tell the story of where you want the business to go and over how long.

Take Jay. He’s been glued to his desk for the last six months, fuelling himself with doughnuts and coffee. His dad died last year from diabetes complications, and he doesn’t want to go the same way.

Jay has been tracking his weight, his daily step count and his resting heart rate since he got a fitness watch last year. (He thinks of these as his KPIs, since they’re ongoing measures of his health.

But he’s realised that simply monitoring these isn’t enough. He isn’t getting any healthier.

Jay decides to set himself a goal – he’s going to take up running and get fit. He’s going to set himself some OKRs.

He decides on the following:

He proudly tells his family about his plans over dinner (takeaway pizza again – the fitness kick isn’t starting until tomorrow). His teenage daughters, Abby and Kat, barely look up from their phones.

Jay has already run into problems!

 An Objective (i.e. the “O” in the OKR) should be inspiring. Nobody is going to be motivated by a “who cares?” Objective.

And Key Results should be ambitious, outcome-driven and specific, communicating where you want to get to over a specified time period (e.g. hit £100m in sales by Q3, or become the city’s best-rated hotel on Tripadvisor by the end of the year).

Sally, Jay’s wife, thinks he could push himself harder. She has heard that OKRs should be slightly scary. What if he worked towards running the London Marathon later in the year?

Jay has a nagging feeling about this and is not sure why. But he follows Sally’s suggestion of making this his Objective - it certainly ticks the box of being scary!

He starts off well, and even manages to find some tasks for the first week that don’t involve getting off the sofa – he orders some designer running gear and creates a spreadsheet to keep track of his progress towards his Key Results. He invests in some “smart” kitchen scales that will sync to his fitness app to track his calorie intake in real time.

In the second week, he goes out running twice. He manages four times the next week. His running speed increases by 0.1 km/hour. He fills in his spreadsheet and feels a virtuous glow.

But when he goes to weigh himself, he is horrified to see his weight has gone up. While he’s having a beer to drown his sorrows, his daughter, Kat, comes in and asks what’s up (she knows a bit about fitness since she plays for the school hockey team).

When he explains, Kat rolls her eyes and gives him one of her trademark withering looks. Doesn’t he even know the basics about fitness? She explains that if you are building muscle, your weight can actually go up at first, since muscle weighs more than fat. Also, the amount of water in the body can cause weight to fluctuate from day to day.  She says he should measure body fat ratio instead. And he should stop obsessing over calories – that’s so last year.

Jay doesn’t want to fork out on a new set of scales that measures body fat ratio. He decides the quickest way to move towards achieving his Key Result would be to try intermittent fasting.

The next week, Jay’s spreadsheet shows that he’s dropped half a kilo in weight, and he completed seven runs. But the numbers don’t tell the true story - that he only ran round the block once each day! His drastic calorie-cutting has affected his motivation and he has no energy to do big runs. But since he’s only running a hundred metres or so, he can sprint round, so his speed in km/hr is looking pretty healthy.

Jay has fallen into a classic trap. If Key Results become merely a box-ticking exercise, this doesn’t help a business to move forward towards its goals and it can even store up problems for the future.

Take the example of a company that has developed a fitness app. They adopt a fantastic marketing plan and manage to achieve their Key Result of 1000 downloads by the end of Q1, but there is a known issue that it can be tricky to sync the app with certain older fitness watches.

They’ve been too busy to recruit people for their customer support team, so support queries are going unanswered. Their customer satisfaction rating nosedives.

A blow comes when Jay doesn’t get into the London Marathon – he hadn’t realised it was a ballot system. He tells Sally he’s going to forget the whole thing.

Sally suggests he could try and enter another marathon – the New York or Boston one maybe? He says he can’t be bothered – it would be too time-consuming and expensive. Sally asks an interesting question – why did he want to take up running in the first place?

Jay does some soul searching. He realises that running the marathon was never really the point. The outcome he really wanted was to have more energy and feel more positive about himself. Sally suggests he should reframe his Objective accordingly.

When deciding upon an Objective, keep asking “why?” until you find the answer that resonates; the one that tells the story of why you’re doing what you’re doing.

 For example, a mental health charity might initially decide its Objective is to increase the money it raises each year. But if they ask “why?” they might end up reframing that. Maybe they want to be the mental health charity that helps the greatest number of people. Or the mental health charity that has succeeded in raising public awareness of a particular issue.

Jay decides he should change his Key Results to align with his new Objective. His younger daughter, Abby, tells him about an app where you can chart your mood and energy levels and he decides to incorporate this into his plan.

Then something strange begins to happen... Jay finds he actually likes running! He has more energy now he’s stopped counting calories and he discovers some beautiful running trails through the countryside. He enjoys the headspace and listens to calming mindfulness tracks from his mood tracking app. He enters the Edinburgh marathon and is excited when he receives confirmation of his place.

This transformation has come about because Jay has successfully reframed his Objective and aligned his Key Results with this.

He has invested his attention and focus in feeling more positive about himself, as well as improving his physical fitness, and he’s even come up with some new tools to help his progress.

Jay checks in on his progress towards his Key Results. He’s consistently completing longer runs and his mood has improved – helped by the fact that he can almost squeeze into his University jeans now! He’s starting to feel like his old self again.

 He realises that his KPIs (the ones he’s been tracking with his fitness watch - weight, daily step count and resting heart rate) could be better aligned with his OKRs. Choosing the right ones to monitor will help ensure he ‘locks in’ the progress he has been making through his OKRs.

He decides on the following:

Jay is now so confident about achieving his OKRs that he decides he’s going to set himself a second Objective – to get sponsorship for running the marathon and raise money for charity in memory of his dad.

He knows he can’t do this by himself – he’s going to need a really strong, motivated team behind him.

Who can he possibly ask to help…?

TO BE CONTINUED….

In Part Two, we will consider how to empower teams to achieve OKRs, as well as pitfalls to avoid.

If you’d like to talk in more detail about how to set OKRs and KPIs that will improve innovation, efficiency, productivity, morale and ultimately profitably in your business, contact Matt Little at Blue Ocean Insight.

The Role of CPTO: a mythical beast?

This insight is the next in a series which highlights the evolution of technology and product teams within an organisation and begins to look at the future structure of an organisation and the inevitable (and welcome) confluence of the technology function and the product function.

 In previous insights, we have talked about the difference between a team that is effective and one that is efficient. Effective (building the right thing) was typically the responsibility of the Chief Product Officer (CPO) whilst efficient (building good software predictably) was normally seen as the remit of the Chief Technology Officer (CTO).

 But the world has moved on. People realised that this ‘siloed’ approach wasn’t working for the customer, and organisations started a journey away from Projects towards Product.

The roles of CTO and CPO

The Chief Technology Officer (CTO) and Chief Product Officer (CPO) roles were born in isolation and only now are we beginning to evaluate whether we need both roles in their current form or whether it might work better to combine the roles.

The role of CTO

Startups are usually founded by either a technologist, a domain or subject matter expert, or both. When a domain expert starts a company, he or she usually realises within the first year that they need a founding partner to help lead ‘technology’ - relying on contract developers alone just doesn’t cut it past the MVP (Minimum Viable Product).

The person who joins the exec team at this stage is often called the CTO and their responsibilities are:

●     Innovation

●     Project management

●     Assessing the technical feasibility of new opportunities

●     Managing engineers and other ‘technical’ team members

 Effectively, the CTO is responsible for the how of product development. Once your customer (often singular at this stage) decides what they want, the CTO leads the charge to find out how to build it and ensures that the product meets the customer’s needs.

 As the company grows, the number of customers increases and suddenly the CTO is faced with a challenge - different customers want different things; their priorities and timelines differ.

 The greatest product risk is then less about feasibility (can the engineers build what the customer wants) and becomes more about value (with our limited resources, what combinations of features gives us and customers the greatest return)?

 To understand how to achieve the greatest value, the CTO needs to be continually out talking to customers and creating a product strategy, whilst continuing to ensure that her/his team deliver what has already been agreed - and checking that the platform scales effectively, is secure, and has low downtime.

 It’s a tall order.

 At this point, a lot of organisations bring in a CPO (Chief Product Officer).

The role of CPO

The rationale behind this decision is simple (albeit sub-optimal). The CPO will decide what to build and the CTO can work out how to build it. People talk about the “healthy tension” between product and engineering.

The CPO defines the strategy (represented in the roadmap) and the CTO takes this and, with their team, creates the release plan.

Somewhere in the middle, of what and how, they agree on the why which is where they discuss the risks of an idea:

●     Value - can we make money from this?

●     Usability - can we present this to customers in a way that they can understand and use?

●     Feasibility - is it technically possible to build something that meets the customer’s needs?

●     Business viability - are we the right people to be trying to solve this problem for customers?

 It sounds like a simple and elegant solution, doesn’t it?

 Unfortunately, an issue arises here because no one person is responsible for what the customer needs. Product OKRs are inconveniently split between ‘technical objectives’ and ‘customer objectives’, and the team needed to deliver the customer’s needs is split between two parts of the company. The why becomes both the CTO’s responsibility and the CPO’s responsibility and yet, somehow, doesn’t fully belong to anyone.

Improvise and adapt

This less than rosy picture ignores the fact that good CTOs are often well-versed in product management. They use this understanding to help craft both a roadmap and a release plan, then lead the team to deliver the product, to monitor customer success and the operational success of a release, and use customer metrics to check on success.

 These CTOs are effectively operating as a CPTO although they don’t have the job title (yet).

 As an organisation comes to understand that success should be measured by outcomes rather than features or outputs, it realises the need for a closer alignment between the strategy for the product and the ability to deliver that product. And so, the role of CPTO is born.

 Sejal Amin at Forbes talks about the ‘product development culture’, and I think that term very nicely summarises the move by technical teams to better understand and deliver value to a customer, and the move away from the build trap which is epitomised by a continual push to deliver feature after feature in the hope that one of these will be ‘the one’ that achieves product-market fit and helps the company onto that hockey stick that they are all looking for.

So why doesn’t everyone have a CPTO?

 Earlier I talked about the fact that good CTOs are often well-versed in product management, and that is true. The problem is that in the Venn diagram of technology and product, there are currently few people who can comfortably and competently occupy the space in the middle that we are beginning to term ‘CPTO’.

 It isn’t that people don’t want the ‘person who can do it all’ but rather that it is very hard to find such a person.

Am I looking for a unicorn?

A unicorn (despite being the national animal of Scotland) is generally thought of as a mythical beast that has magical powers. Are CPTOs just mythical creatures that you aren’t likely to meet in the wild?

 This is a growing domain and there are shining lights out there leading the way and others are bound to follow. Organisations such as LinkedIn, Ebay and Conde Nast have actively sought exec-level CPTOs.

 There are those of us who have moved from CTO to a hybrid product/technology role and others are doing the same.

 As the opportunities grow, CTOs will broaden their experience into areas we currently call ‘product management’ and CPOs will step sideways into the domain of the CTO. Organisations who embrace this new role, and ultimately their customers, will be the ones who benefit.

---

This Insight paper was originally published on the Blue Ocean Insight website. You can view the original article here.

If you want to discuss the role of CPTO with the Blue Ocean Insight's Team, you can email the team at info@blueoceaninsight.com.

Roadmaps and release plans: the key to better product outcomes

I doubt I’m the only one thinking wistfully about travel at the moment, looking forward to a time when the world returns to some kind of normality, and we can spend our leisure time how we please. I also guess I’m not the only one planning for the next time we can enjoy carefree holidays abroad. I’m probably also not the only one looking to make my next holiday a truly epic one.

And I do mean epic – my wife and I are going to drive across America, coast to coast, east to west, in a Shelby Mustang. But we won’t be taking the most direct, time-efficient way to cross the continent, because that would defeat the object of the trip. The scenic route is the order of the day (every day; it’s going to take us months).

Planning for what matters: keeping it flexible

We want to visit as many states as possible while we’re in the US; we want to see the incredible colours of autumn (though they’ll call it ‘fall’) in New England, visit the Big Bend National Park in Texas for the unrivalled stargazing opportunities, and head to Wyoming to take in the incredible geology (geysers, waterfalls and petrified forest) of Yellowstone National Park. We plan to go into Louisiana to experience the vibrant culture of New Orleans, and visit Arizona to learn about the indigenous populations that have made the area their home for some 12,000 years – that’s just the start of our list, too!

I could tell you, in more detail, about more of the things we’d like to see and places we’d like to go. But what I can’t tell you is the exact dates we’ll be arriving in each place, or how long we’ll stay there. We’re not building an itinerary, or committing to accommodation bookings along our route. We’ll book the car hire (because it’s important to me that we get that particular vehicle, for nostalgic reasons – it won’t be the first time that I’ve done a US road trip in a Mustang!), and we’ll make sure we have a hotel room secured for the first few days (we’re factoring in a few days’ R&R, because nobody wants to begin the road trip of a lifetime with jet lag). I’ll also do some good research on AirBnBs on the East Coast, so we’ve got some idea of our next steps after the jet lag has worn off, and also pick out some special options in the Bay Area for the last few nights of our trip.

What does all this information about my holiday planning tell you? It gives you a clear idea of what’s important; of what I want to achieve, what the point is. From what I’ve shared above, you can tell that I want to enjoy the trip of a lifetime, in a relaxed way that makes the most of the opportunities that present themselves at the time (and works around any metaphorical bumps in the road – for example, a route diversion because of roadworks, or finding that a restaurant we wanted to visit is closed when we turn up). It also tells you that I want to make the most of those serendipitous moments that offer positive opportunities we couldn’t have planned for.

Trying to pin down a really specific schedule of when we’ll be where and doing what would be counterproductive. We’d just end up stressed and dissatisfied, distracted by negativity over ‘not sticking to the plan’ (because no complex plan ever pans out quite as one might expect, there are usually setbacks and obstacles, as well as unforeseen changes that shouldn’t be missed).

Roadmaps and release plans in product development

Why I am I telling you this story about my lockdown daydreaming and future trip planning? Because my cross-America trip planning is an excellent analogy for the product roadmap, and the reasons why a release plan should be undertaken relating to small chunks of project time, with a flexible mindset.

It’s common for people to confuse the concepts of roadmaps and release plans; the terms often get used interchangeably when they’re very different things. This not only hinders communications (encouraging working in silos) and subsequently the effectiveness of teams, but can also result in missed opportunities for improvements during the process, since with a release plan the focus is often (mistakenly) on ticking off the milestones that have been set out at the start. You lose adaptability.

A roadmap is a strategic view of where a product might go, and over what timeframe. It embraces the good product discipline of taking on board customer feedback and works well with Agile software development. A good roadmap is a plan, not a commitment.

Any good plan (or roadmap) starts with some certainties. So my roadmap for my trip of a lifetime would look something like this:

• The trip will last no more than three months.

• We’re going to fly to the East Coast of the USA to begin our journey.

• Not every car hire place will have a Shelby Mustang, and this is important to me, so I’d better book that early.

• It makes sense to book accommodation close to the airport for the first couple of nights, and spend the first week or so in New England, keeping the travel distances lower while we recover from the jet lag.

• I like using AirBnB, as you meet some interesting people. I’ll do a bit of early research on great-looking places to stay on the East Coast, so I’ve got some possibilities in my head for the first couple of weeks.

• I want to book somewhere really special for our last few days, so before we go I’ll research places to stay near San Francisco, maybe in Monterey – no need to make the booking just yet, though.

• I need to think about any concrete dates or restrictions, and know about them in advance, even though the plan itself will be loose. For example, perhaps it’s not possible to drive into all parts of Yellowstone, or maybe I want to be in New Orleans for Mardi Gras.

This roadmap tells you the WHAT of my trip, it’s the strategic plan, my statement of intention. It sketches out my priorities and includes the WHERE and the WHY. It details the high-level proposal (and thinks ahead to any hard constraints), but leaves enough wiggle room for reacting to feedback and new information.

In product terms, I think of my roadmap as a similar sort of list. It’s a document that answers the following questions:

• Where do you think this product should ultimately go?

• Given what you know about your customers’ needs now, where do you want to take the product in the short, medium and long term?

• Do you have any concrete deadlines to hit, that are beyond your control? (for example, there might be a piece of new legislation coming into force)

As a product manager, my main purpose would be to find out about all the opportunities that are out there, decide if they fit with my vision, and then prioritise those that do.

So a roadmap is about priorities and vision, but how do those get turned into an actual product?

Once you have decided where your product is heading, and where to start – based on the information available to you at the time – you need to start thinking about the HOW. How will you translate that vision into reality? That’s where your release plan comes in.

Your release plan is about delivery and execution. But how can you utilise a release plan without falling into the trap of becoming a slave to its milestones?

The impact and potential of the ‘cone of uncertainty’

The key to this approach is to embrace uncertainty. Uncertainty is a powerful concept; you need to navigate it to enter new markets or pivot to a new customer need. In fact, the very edge of certainty is often where opportunities exist.

There is, of course, the inherent uncertainty in any plan if you don’t have all of the information you need to guarantee 100% accuracy – and in the real world that just doesn’t happen; you can’t predict the future accurately. You can however model a ‘cone of uncertainty’, which can plot the likelihood of where a project might end up. Think about the maps you see when meteorologists model hurricanes. They show the likely course of the hurricane’s progress, and this is usually cone-shaped because the model necessarily loses accuracy the further ahead you project the data, resulting in a shape that begins narrow, and gets broader.

At the end of the cone that represents ‘now’ or ‘very soon’, the probability of the model being accurate is high – there are few variables that could have a material effect on the path in the very near-term. At the far end of the cone, it spreads out. Whilst you might have been able to model the major influences, a series of minor influences can build into an effect that little by little has a material impact on an outcome. This is known by various names, including ‘the butterfly effect’, and is often adopted when modelling something that cannot (yet) be modelled with complete accuracy. It is this effect that leads to your weather report telling you that there is a 50% chance of rain today – in other words, the model has reached a point where the intrinsic uncertainty of earlier events has left it unable to decide if it will rain or not.

Big up-front planning: the bad old days

Once upon a time in software development, business analysts believed that they could take an incomplete set of data and use that to build a plan outlining exactly where the project would be on any given day, even if that day was two years away. People looked at the 200+ page project specification and believed it must be true just because of the sheer volume of work that went into creating it.

But the weight or size of a book (or a specification) rarely tells you whether it is any good or whether the information it contains is accurate. It will only turn out to be ‘good’ if there is sufficient data, time and resources to draw highly accurate conclusions. Rather than set out such a detailed specification right at the beginning of a project, it is often better if the author simply accepts that the further in time they move on from their accurate data, the less accurate the conclusions will be.

Having worked on such waterfall (big, up-front design) projects, it is clear to me why many failed to deliver what was projected at the outset. Those responsible tried to ‘manage away’ the uncertainty rather than embrace it. Software development’s response to this inaccurate planning model was to embrace change and customer-led learning, and this brought about the birth of Agile software development movement.

Reaching the best possible outcomes

But how does knowing all this get me to my destination, whether that’s San Francisco or a completed product?

You have enough data and capability to take the first steps in the journey. You can (literally or metaphorically) create a detailed plan for flights to JFK, you can book the car and you can book accommodation for the first few nights. You can manage the costs with 100% accuracy for these stages.

In week two of your trip, you think that you know where you are going to stay, and you can be fairly accurate with your fuel and accommodation costs.

This is your execution plan (in software development language, a release plan). It is detailed and accurate and it defines the HOW of the first part of your journey. You could get people in your team to start looking at flights and accommodation and working to a concrete budget.

There is little point at this stage spending hours trying to plan the same level of detail for three months into your trip. At that time horizon, you are at the wrong end of the ‘cone of uncertainty’ and you only think that you know when you will arrive in California. It is too early to jump into booking where to stay as you’ll probably get the dates wrong and lose your deposit!

A release plan is best employed as a detailed view of part of your roadmap. In terms of the three-month trip of a lifetime it’s your arrangements for the first week or two of travelling, in software development it would perhaps cover the first six months of releases in a 24-month product roadmap. Primary responsibility for the roadmap lies with the product team, while it’s the responsibility of the engineering team to ‘zoom in’ to the roadmap to create a release plan. Everyone should have access to both and agree on the contents.

Embracing uncertainty, maintaining flexibility

So, in summary, the takeaways – other than ascertaining my fondness for American muscle cars, stunning vistas, and the melting pot that is American culture – are as follows:

• A product manager should embrace uncertainty and communicate clearly.

• A roadmap tells stakeholders (and potentially customers) about the vision for your product and highlights how you think that the product will evolve over the next 12-24 months.

• A release plan is a document which is created collaboratively between the product and engineering teams. It homes in on a specific part of the roadmap, and adds detail to a period of time, giving you more scope to effectively project from the data that you have at that time.

The model of ‘only planning as far ahead as you can’ is well understood and used in many fields, but frequently forgotten in product management. Uncertainty is unavoidable. But if you define the differences between a roadmap and a release plan, you can more easily embrace the inherent uncertainty in a complex environment and use it to your advantage.

If you’d like to talk in more detail about the differences between roadmaps and release plans, and how it’s possible to utilise the power of uncertainty to develop better products, send us an email.