The £500K Spike
Introduction
The point of a spike in software development is to investigate a problem and come up with potential solutions while evaluating the costs and benefits of the different options. A recommended path forward should also be included.
A spike investigation is supposed to de-risk a project. However, in some cultures that value harmony and visibility over substance and rigour, these investigations can become the very source of catastrophic risk.
What follows is the story of how a well-intentioned effort to explore six potential solutions instead created a costly emergency and the lessons every team must learn to avoid a similar fate.
The Dysfunctional Spike
The Assignment
The story starts with the Product Owner who wanted to see options for a new feature. The spike was assigned to one of the developers to explore possibilities and report back to the team.
The Response
The developer came back later in the sprint with an impressive-looking piece of work highlighting 6 different potential options. However the spike had one fatal flaw: there was no analysis of the pros and cons of any of the options. There was no consideration of the long-term maintenance of each of the options or how they would fit architecturally with the existing product.
Why did the developer do this? Why did they miss this flaw?
Their behaviour was perfectly aligned to the culture. In a team culture that consciously or subconsciously incentivises the 3V's: vibes, visibility and velocity, the developer's effort was exactly what was required. They receive social credit for visible, rapid output and being able to say "look how many things I thought of!".
The Leadership Failure
And indeed, their efforts were rewarded. On the face of it, it looked impressive, it looked thorough. Leadership, understandably lacking the required technical depth to spot the missing analysis, could only evaluate the performance of the work (its volume and visibility), not its substance.
The Decision by Democracy Farce
And, because nobody pushed back, one of the six options was chosen based on superficial appeal or the loudest voice."This one looks cool" rather than a data-backed decision based on costs and benefits of implementation.
The decision-making process was not a technical discussion regarding the pros and cons or the amount of effort required, it became a democratic vote based on visual appeal as there was nothing else to base it upon.
The spike, intended to provide clarity and a data-based way forward, resulted in a decision made on vibes in an information vacuum.
The Inevitable Collision
What happens next? If we play this forward for a minute, the decision will be turned into a story and will enter refinement.
Once there, rather than be able to have a thoroughly vetted decision to refine, the team now has to actually draw conclusions and consider the pros and cons of the 6 options in real-time.
The team discussion goes nowhere. Any of the 6 options are viable but all of them make someone in the team uneasy because they can see the undocumented disadvantages. The story gets parked for another day and a further spike item is raised for re-investigation and further analysis of the problem.
Why It's a £500,000 Mistake
The wasted spike has several different costs associated to it:
Direct Costs
These are costs that are fairly obvious: the salary for the spike and effort of creating the story, the salaries for the refinement meeting and then the duplication of these for the re-analysis.
Opportunity Costs
The time wasted in discussing this item during refinement which could have been used to bring in another item into sprint.
Costs of Delay
The feature's launch will now be delayed. This postpones the moment it can start generating value, solving a user pain point or capturing revenue. In a fast-moving market, the value of a feature is often tied to when it ships, not just what it is.
Trust Cost
Related to delay costs, if this feature was marketed or promised to (a subgroup of) customers, the delay will erode stakeholder / customer confidence in the team's ability to deliver.
Morale Cost
The frustration cost for the team who now have to re-look at the area again and clean up a predictable mess.
The Antidote
Remember what a spike is and why we use them. They are there to investigate unknowns and to return with a decision-ready recommendation based on a consideration of the advantages and disadvantages of a number of options.
A spike does not exist to diagnose the problem and stop there.
A simple framework I try to follow is as follows.
1. Define the decision-making criteria upfront
What are we basing our recommendation on? Is it the time it takes to dev, to test? Is it the cost of the proposed solution in terms of licensing, for example? Is it how easy the solution is to maintain? Or how easy it is to scale?
Without any baselines for comparison, we can't reasonably compare different options.
2. Evaluate each option against the defined criteria
A simple table showing the options on rows and the criteria on columns would suffice. (I tend to write out mine in paragraphs although I am trying to get better at the tabular approach.)
3. A clear recommendation
Having evaluated all the options, a spike should include a clear, unambiguous recommendation that the team can take forward.
4. Explicitly document the trade-offs
No option will be perfect. A recommendation should be a fair assessment of the option chosen including its drawbacks.
This framework will only work in a culture where the leadership rewards rigorous thinking rather than long lists of options that look like busy work.
Stop Performing, Start Deciding
A culture that allows substance-free work is a culture that is building on sand. The emergencies that appear subsequently did not "come from nowhere". They are predictable and self-inflicted.
The mark of a professional team is not in avoiding spikes. It's in conducting them in such a detailed, well-scoped, rigorous manner that it makes the subsequent work faster and more predictable.
Think of spikes as the building block for the development work that comes later. Your team's output is only as good as the quality of these investigation pieces. A good amount of upfront effort will prevent any sand from shifting later in the development.
Invest in the craft of thinking and you'll stop creating the emergencies you're hired to solve.