As regular readers of our blog will know we really love to start a debate and are perfectly comfortable with people telling us that we don’t know what we are doing. Thus, we were rather pleased with the debate following our post about benchmarking (smack my bench up) and thought that it was worth following up on some of it in today’s post.
As a reminder our post had argued the following:
To be honest I would be quite happy if I never saw another benchmarking report ever again.
Who knows? Managers might even be encouraged to focus on their own services rather than comparing themselves against others based on a series of arbitrary, inaccurate and ill-fitting indicators.
Not much room for nuance there eh?
So without further ado let’s begin with a quick point from twitter. Jonathan Green took exception with our contention that benchmarking is inherently wrong and asked the following (albeit rhetorical) question:
@WeLoveLocalGov if u can’t manage a meaningful helpful benchmarking exercise to help improve services should you be managing a service at all?
A fair point and there is undoubtedly an element of bad practice that underpins my dislike of benchmarking. However, I still think there is a wider question. I’m sure our managers could complete a decent benchmarking exercise (a concession on my part) but with all the difficulties I’ve already identified is it worth their while?
To which an interesting comment from Kriswith on the blog itself seeks to answer that point:
The problems is that if you collect performance data but don’t have any context for it then you have no idea if you need to be doing something about the results. One example I know of is a score from a national survey. My borough got 27% which in itself might have been a worry – until we found out that this was the third highest score in England. Without the benchmark we might have spent money on a problem that wasn’t really a problem.
You can always find a health warning attached to any benchmark but that doesn’t mean the context you get from comparisons with others is not still useful in understanding your own performance.
This view of the use of benchmarks was shared by quite a few of our commentators although a warning from Jeremiah was well put:
if your 27% score worries you then maybe you should be doing something about it – if everyone else is even worse, that doesn’t make it good enough.
I agree with this but recognise that the flipside is that in a time of crunched resources maybe the benchmarking is a good way of identifying what the priorities should be.
On a similar note one commentator observed:
That could almost be an argument against performance indicators completely…
I should probably make it clear that this was not an argument against using performance indicators which all managers should have as part of their management toolkit. However, I am a little sceptical about all national indicators, probably for the reasons listed in the original piece.
My scepticism was questioned by many of the commentators who argued that the key is how people use the benchmarks. One author pointed out that benchmarks should be used to ask questions rather than answer questions and another pointed to the penguin effect (trying to get to the middle of the pack but not striving to be at the front).
Both are good points well made but whilst I still don’t really like benchmarks I’m not foolish enough not to recognise the sane commentary of our readers. In particular I would like to point to the comments of mhuttch who provided the following six bullet points of guidance for what benchmark makers would do well to do:
1) be clear about what the benchmark is to be used for
2) involve the people that are going to read the reports in their design
3) agree with the same people where the greatest inaccuracies are likely to occur and agree what, if any, mitigating or ‘equalising’ approaches can be taken
4) guide and coach contributers throughout the data collection process
5) guide and coach their readers through what the reports tell them and, importantly, what they hide
6) be under no illusion that they can produce anything other than something that is ‘roughly right’ (which is better than exactly wrong).
I haven’t totally changed my tune on benchmarks but there are definitely two sides to every debate and I’d like to thank all the people who took time to share their expertise with me and the other readers of this blog.
Remember, we love to read your comments and are more than happy to put up guest posts so if you have any ideas please do drop us an e-mail.
Welovelocalgovernment is a blog written by UK local government officers. If you have a piece you’d like to submit or any comments you’d like to make please drop us a line at: firstname.lastname@example.org