Posted tagged ‘benchmarking’

Lies, damned lies and dodgy statistics

May 16, 2012

Disraeli; credit where it is due…

There are some stories that are too good to pass up without further comment; even if they have little relationship to local government.

This from Yahoo news is one such gem. Under the headline ‘25,000 men need an obstetrician or gynaecologist every year’ the story, originating from the Telegraph, reports that:

Official figures from the NHS appear to have proved that old adage true, as they show that Britain has tens of thousands of ‘male mothers’. They discovered that 17,000 men were recorded as having been admitted to hospital for obstetric services -a specialism for pregnant women and their babies – and 8,000 to see a gynaecologist; while another 20,000 apparently needed to see a midwife.

They also identified a steady increase in the numbers of children and teenagers attending geriatric services, to over 3,000 between 2009 and 2010, and more than 1,600 adults over 30 using child psychiatry services.

The problem for the NHS, and for other public bodies who use statistics like this to run their services, is that bad data like this can then lead to bad decisions. Increasingly public services rely on large quantities of statistics.

This is entirely rational. After all, public services don’t have profit to base their decisions on and the other ultimate means of determining success are elections and they are fairly blunt instruments.


Bench Pressed

February 27, 2012

Because thinking of a different picture was too difficult

As regular readers of our blog will know we really love to start a debate and are perfectly comfortable with people telling us that we don’t know what we are doing. Thus, we were rather pleased with the debate following our post about benchmarking (smack my bench up) and thought that it was worth following up on some of it in today’s post.

As a reminder our post had argued the following:

To be honest I would be quite happy if I never saw another benchmarking report ever again.

Who knows? Managers might even be encouraged to focus on their own services rather than comparing themselves against others based on a series of arbitrary, inaccurate and ill-fitting indicators.

Not much room for nuance there eh?

So without further ado let’s begin with a quick point from twitter. Jonathan Green took exception with our contention that benchmarking is inherently wrong and asked the following (albeit rhetorical) question:

@WeLoveLocalGov if u can’t manage a meaningful helpful benchmarking exercise to help improve services should you be managing a service at all?

A fair point and there is undoubtedly an element of bad practice that underpins my dislike of benchmarking. However, I still think there is a wider question. I’m sure our managers could complete a decent benchmarking exercise (a concession on my part) but with all the difficulties I’ve already identified is it worth their while?

To which an interesting comment from Kriswith on the blog itself seeks to answer that point:

The problems is that if you collect performance data but don’t have any context for it then you have no idea if you need to be doing something about the results. One example I know of is a score from a national survey. My borough got 27% which in itself might have been a worry – until we found out that this was the third highest score in England. Without the benchmark we might have spent money on a problem that wasn’t really a problem.

You can always find a health warning attached to any benchmark but that doesn’t mean the context you get from comparisons with others is not still useful in understanding your own performance.


Smack my bench up

February 22, 2012

Well, a picture of an actual benchmark would be really dull

We’ve all been there; when trying to defend the performance / budget / branding / future direction of our service the boss has turned round and said something along the lines of: ‘how does that compare to our neighbours; what’s the benchmark?’

Thus begins another tedious, and absolutely pointless, round of comparison with local councils followed by an equally tedious, and equally pointless, round of explanations as to why the comparisons are actually not a good match with our local context.

I hate benchmarking!

I think it is a weakness of the human condition to constantly want to compare ourselves to others and in local government this inclination is fully played out in the world of performance benchmarking.

For the uninitiated, benchmarking involves comparing the performance of your service/local authority with that of another local authority. The measure you use to do this can come in a variety of different forms and there are whole armies of staff, brought up on a steady diet of performance targets under the last Labour Government to crunch these numbers and produce some sort of comparative figure.

So why is benchmarking a bad idea?


In Defence of Performance Figures

October 27, 2010


LG Worker to the rescue


I recently wrote a post trashing performance indicators: LG Worker responded in the comments section and so I invited him/her to write a post in defence of performance indicators… And, as if by magic: here it is!

When I first joined Local Government, several Authorities and several years ago, I had a 1 to 1 with a Colleague I saw more as a mentor.  As we sat in the local pub (after work I hasten to add), he turned to me and said, ‘Young Padawan, to progress your career, you should get some Performance Management experience.’  This statement was geeky in two ways.  The first was the reference to Star Wars, as far as I knew I was not a Jedi who had ‘the Force.’   The second was the reference to Performance.

As I was relatively new to Local Government and really to the world of work, I had no real understanding of what Performance was.  The title gave clues but not the full picture.  Anyway, soon after I found myself in the Council’s Performance Team.  Within six months of being there, I could not see the point of my work (how did it connected with the residents?) and had badly messed up a big project.  As this project crashed around me, I decided I needed to take a break and reassess my options.

In the long weekend I took, instead of deciding to give Performance up, I had an epiphany and saw not only how important Performance is but also how it linked back to our ultimate bosses, the residents.  In this period were Performance Indicators are being thrown out (I hear the usual bloggers of this site cheering), I want to defend them and show you why they are a tool we should be holding on to.

Their defence is provided by the four things they help provide; monitoring, accountability, benchmarking and partnership.