Bench Pressed


Because thinking of a different picture was too difficult

As regular readers of our blog will know we really love to start a debate and are perfectly comfortable with people telling us that we don’t know what we are doing. Thus, we were rather pleased with the debate following our post about benchmarking (smack my bench up) and thought that it was worth following up on some of it in today’s post.

As a reminder our post had argued the following:

To be honest I would be quite happy if I never saw another benchmarking report ever again.

Who knows? Managers might even be encouraged to focus on their own services rather than comparing themselves against others based on a series of arbitrary, inaccurate and ill-fitting indicators.

Not much room for nuance there eh?

So without further ado let’s begin with a quick point from twitter. Jonathan Green took exception with our contention that benchmarking is inherently wrong and asked the following (albeit rhetorical) question:

@WeLoveLocalGov if u can’t manage a meaningful helpful benchmarking exercise to help improve services should you be managing a service at all?

A fair point and there is undoubtedly an element of bad practice that underpins my dislike of benchmarking. However, I still think there is a wider question. I’m sure our managers could complete a decent benchmarking exercise (a concession on my part) but with all the difficulties I’ve already identified is it worth their while?

To which an interesting comment from Kriswith on the blog itself seeks to answer that point:

The problems is that if you collect performance data but don’t have any context for it then you have no idea if you need to be doing something about the results. One example I know of is a score from a national survey. My borough got 27% which in itself might have been a worry – until we found out that this was the third highest score in England. Without the benchmark we might have spent money on a problem that wasn’t really a problem.

You can always find a health warning attached to any benchmark but that doesn’t mean the context you get from comparisons with others is not still useful in understanding your own performance.

This view of the use of benchmarks was shared by quite a few of our commentators although a warning from Jeremiah was well put:

if your 27% score worries you then maybe you should be doing something about it – if everyone else is even worse, that doesn’t make it good enough.

I agree with this but recognise that the flipside is that in a time of crunched resources maybe the benchmarking is a good way of identifying what the priorities should be.

On a similar note one commentator observed:

That could almost be an argument against performance indicators completely…

I should probably make it clear that this was not an argument against using performance indicators which all managers should have as part of their management toolkit. However, I am a little sceptical about all national indicators, probably for the reasons listed in the original piece.

My scepticism was questioned by many of the commentators who argued that the key is how people use the benchmarks. One author pointed out that benchmarks should be used to ask questions rather than answer questions and another pointed to the penguin effect (trying to get to the middle of the pack but not striving to be at the front).

Both are good points well made but whilst I still don’t really like benchmarks I’m not foolish enough not to recognise the sane commentary of our readers. In particular I would like to point to the comments of mhuttch who provided the following six bullet points of guidance for what benchmark makers would do well to do:

1) be clear about what the benchmark is to be used for
2) involve the people that are going to read the reports in their design
3) agree with the same people where the greatest inaccuracies are likely to occur and agree what, if any, mitigating or ‘equalising’ approaches can be taken
4) guide and coach contributers throughout the data collection process
5) guide and coach their readers through what the reports tell them and, importantly, what they hide
6) be under no illusion that they can produce anything other than something that is ‘roughly right’ (which is better than exactly wrong).

Good advice.

I haven’t totally changed my tune on benchmarks but there are definitely two sides to every debate and I’d like to thank all the people who took time to share their expertise with me and the other readers of this blog.

Remember, we love to read your comments and are more than happy to put up guest posts so if you have any ideas please do drop us an e-mail.

Welovelocalgovernment is a blog written by UK local government officers. If you have a piece you’d like to submit or any comments you’d like to make please drop us a line at: welovelocalgovernment@gmail.com

About these ads
Explore posts in the same categories: We love the Council

Tags: , , , , ,

You can comment below, or link to this permanent URL from your own site.

2 Comments on “Bench Pressed”

  1. Ed Hammond Says:

    I completely agree with everything you said in the first post.

    Often, benchmarking is about comparing your own meaningless data against the meaningless data collected by somebody else, which isn’t going to produce any huge insights. Let’s talk about sharing information and insights as a sector more generally, recognising that benchmarking is only one element of this, that can only ever provide a partial picture. My worry is that PIs and benchmarking can often be elevated to some kind of pre-eminent status where any and all other types of evidence are superfluous and your PIs give you a single version of the truth.

    I do like those bullet points though.

  2. Performance Officer Says:

    Public Finance has, in a most timely fashion, put together a blog about how to do benchmarking – and despite my scepticism generally, I pretty much agree with all of it.

    The link is: http://opinion.publicfinance.co.uk/2012/02/getting-the-best-from-benchmarking/

    On the whole it makes similar points ie know exactly what you’re doing, and why, and realise that benchmarking won’t change anything, it can be a tool to promote change though, if only when used well.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 241 other followers

%d bloggers like this: