The Two Realities: Performance vs Substance

The Anecdote

As a data professional I once had a job dealing with health care data. One of the pieces of data we needed was NHS prescribing data which would be published monthly on each of their country websites. This data was published per country and each of the various countries (England, Scotland, Wales & Northern Ireland) would publish on different days of the month to their respective websites.

There was also a little bit of variance as to which day the new data would actually be uploaded onto a given site. For example, one month the England data might be available on the 20th, the next month it might be late and not available until 26th. There was no predictability to the day of the month or the time of day.

The manual way to ensure we got the data promptly was to bookmark the four NHS websites and refresh several times a day waiting for the new data to appear. And this was what our Head of Data would do.

An alternative might be some sort of automation script that would run and go to each of the four websites, check the latest data uploaded and download the documents automatically.

And as a side project I actually built this - a web-scraper program in Python with the following characteristics:

• It ran automatically every morning and navigated to each of the 4 NHS websites in turn
• The date of the latest published document would be checked against the latest month within its dictionary
• If there was a difference, it would then download the document
• Consequently its dictionary would then be updated with this new date
• The downloaded file(s) would be moved to a central repository
• A message to the user would be reported informing them what had happened

When I showed this programme to the CEO and Head of Data, their response was lukewarm. They could appreciate the technical ability but did not understand why it was needed. The Head of Data carried on doing his daily, manual checks.

The Types of Work

The manual way of checking daily and downloading and then repeating four times a month for each and every month sounds not only boring but error-prone and an inefficient use of time.

And yet I see individuals and organisations in general take this approach even when there is a quicker automatic alternative that takes the heavy lifting from them.

So why is this?

I believe it's because there is a cultural issue. A disconnect between the performance of work (what looks like effort) and the substance of work (what delivers actual outcomes). And the culture has a bias towards rewarding the former while exploiting the latter.

1. The Performance of Work (The Theatre)

The manual way is more effort. It requires context switching, it increases anxiety as the individual is aware they need to constantly do something.

The goal for the individual here is not efficiency or an easy life. It is to be seen, to be engaged and to be useful.

The behaviour which you may recognise is loud struggle, frequent updates, 'collaborative' meetings, visibility, "show and tells".

The currency is social capital, visibility and perceived dedication to the task and, by association, to the company.

2. The Substance of Work (The Craft)

The approach here is different. It's about delivering robust trustworthy solutions which work under pressure e.g. with high volumes or on edge cases.

The goal here is a quiet life. A one-and-done mentality which means you expend as little energy as possible and quickly move on to the next puzzle.

The behaviour is deep work, simplicity, silence and intelligent use of automation.

The currency is actual outcomes and a system that is healthy and stands up to scrutiny.

Why Performance Wins

Humans are social creatures. Drama always draws us in. So visibility, struggle and chaos always appeals and gets the limelight. It's easy to see and therefore easy to measure. Activity and noise are easily mistaken for progress.

It also feeds ego and the helplessness garners a feeling of need in others. If somebody is saying something is difficult, it must be. So they need help and attention.

By contrast the substance workers are silent and efficient. They very rarely have to draw other team members in. The work just gets done. And 100 future tickets are prevented. None of this is visible and indeed the silence leads to suspicion "they have got the easy work again" or "if they've done it that quickly, they must be skipping steps".

The Craftperson's Curse

Not only are substance workers quiet and independent while they do the work but their works leads to very little downstream work. They work reliably to fix the issue and their work is of such high quality that it doesn't break.

The crafter unintentionally becomes a threat to the whole performance: a silent elegant solution executed quickly is an indictment on the noisy, chaotic way that things have always been done.

At some level, leaders recognise this ability. And you end up being the dumping ground for problems the Theatre couldn't solve (spikes, bugs, performance issues). You become the cleanup crew.

Worse still, your spike recommendations are ignored because they were not fully understood by the Performers and/or the solution wouldn't provide the drama and busyness they seek.

In contrast the brittle solutions put in by performance workers break under strain and lead to bugs and follow up work. The initial reaction shows the shallowness and the lack of ability to assess the work previously done. There is usually surprise "oh, this bug has come from nowhere" and the additional work perpetuates the narrative: "see I told you it was really difficult" and gives the performer more fuel to be noisy about.

When Substance Leaves

In an environment which doesn't look deeply and values the performance worker and their visibility, the substance worker will eventually leave.

This will temporarily blindside the leader who had no idea their quiet, competent, easy-going employee was looking to leave.

The automations and robust systems they created begin to decay. The technical debt that the substance worker was secretly managing starts to become everyone's problem.

And this will be when the walls cave in. It will be discovered that the substance worker was the only one whose solutions had any long term value. When they leave, more bugs will 'come from nowhere' and more tech debt will be used to fix tech debt. This debt compounds into almost daily fire-fighting.

And this suits the performer. They get to be centre stage and exert themselves solving today's problem. They are held up as the hero only for it to return tomorrow and nobody put two and two together.

The Path Forward

For Craftspeople

To prevent ending up frustrated in an environment of performers, you must seek out places that value outcomes over chaos. I appreciate from my own experience this is easier said than done!

Look for clues such as "deep work", "ownership" and "craft". Be more public about your value by writing it down. Explain your thinking and decision-making in words, document your solutions. If possible document the value of your solutions in terms of time or money saved. Heck, start a blog writing about your thinking!!

There is massive value in a presence, a portfolio of work, a network of like-minded people, a blog outside the day job. This prevents the feeling of frustration and restriction present in your environment and helps build your wealth in a different economy. For a craftsperson this feeling of liberation and autonomy is crucial.

For Leaders

Good leaders will frequently have to be amateur psychotherapists as social dynamics are often at play. If the individual is insecure or anxious or new to the team, they may be trying to project confidence and understanding within the team. If they are overwhelmed often their speech is frantic and dramatic. If they are not good at reading others they may not realise they have lost their audience and continue rambling.

Learn to be a little bit apart and reserve judgment until the the work is shipped. Then judge on the quality of the work, not on the chaos or noise that was expended in completing it.

Imagine the company is your company and think about the expense in terms of salary alone of holding several meetings, collaborations and long status updates. Think about the cost of items having to go through several iterations. Does the item warrant this cost?

Help the craftspeople by allowing and protecting periods of deep work. Space where they have uninterrupted, focus time to be able to think and solve the problem. Remember data work is knowledge work, not a factory line. Allow your staff the time to do the hard thinking necessary.

Do not be seduced by noise and chaos. Understand that as humans we are programmed to be drawn towards drama. But instead notice the quiet ones. Be curious towards them - why are they quiet? What are their thoughts? Draw them into the conversation. Often their viewpoints will be sharper than anything heard up to now.

Conclusion

The most valuable work is often the quietest. It's the automation that saves a thousand future clicks, the robust code that prevents a hundred bug tickets or the elegant data model that makes every report fast and accurate.

The manual refresh story from the start was not about getting the data efficiently. It was about the ritual of getting the data. The Python script which automated the daily process was rejected in favour of the manual process. Not because it was a worse solution but because it made the process invisible.

The path forward for the craftsperson isn't to strip away your identity and mould into the noisy environment. Instead it's to build your own stage where your work is the proof and the applause is the quiet humming of a system that just works.