Smack my bench up


Well, a picture of an actual benchmark would be really dull

We’ve all been there; when trying to defend the performance / budget / branding / future direction of our service the boss has turned round and said something along the lines of: ‘how does that compare to our neighbours; what’s the benchmark?’

Thus begins another tedious, and absolutely pointless, round of comparison with local councils followed by an equally tedious, and equally pointless, round of explanations as to why the comparisons are actually not a good match with our local context.

I hate benchmarking!

I think it is a weakness of the human condition to constantly want to compare ourselves to others and in local government this inclination is fully played out in the world of performance benchmarking.

For the uninitiated, benchmarking involves comparing the performance of your service/local authority with that of another local authority. The measure you use to do this can come in a variety of different forms and there are whole armies of staff, brought up on a steady diet of performance targets under the last Labour Government to crunch these numbers and produce some sort of comparative figure.

So why is benchmarking a bad idea?

1)      No-one ever compares like with like. Most of the things we compare are based on old Government performance measures. These measures are often put together in a slightly random way and involve self reporting of things which are not naturally quantifiable; like detritus. Thus, any measure we compare simply refers to the way I categorise something versus the way you categorise things. Likewise, when comparing costs no-one cuts their services up in the same way; my customer services might include a benefits team, yours doesn’t.

2)      The benchmarking doesn’t even compare like with like. Providing a service in my local authority involves very different challenges than services in other authorities; even ones that on the face of things are fairly similar. Yeah, sometimes things are similar but once you’ve controlled for all the differences and the interesting local contexts you’ve basically not got much left.

3)      The stats that are comparable are not often the ones that tell us the most about our services; instead they’re the ones other people collect or ones that the Government wanted us to collect.

4)      It really doesn’t matter anyway. If your local authority provides a better bin service than the one next door does the local resident give a damn on the day when their bin isn’t collected? Being in the upper quartile of authorities in the south west of England is not exactly a rallying cry for an election, or anything else to be honest.

I’m sure there are good answers to all of the above but to be honest I doubt it would be worth it.

People should not confuse my words with an argument against working with other local authorities or even with comparing our practices with our colleagues. However, comparing our performance with them is likely to either give us a false sense of our own success (leading to complacency) or making us believe we are worse than we are (leading to ignoring it or ignoring local considerations).

I appreciate that these comments sit at the fashionable end of current local government practice but on this one occasion I am very happy to be a fully signed up member of the ‘riding the cool wave’ brigade. To be honest I would be quite happy if I never saw another benchmarking report ever again.

Who knows? Managers might even be encouraged to focus on their own services rather than comparing themselves against others based on a series of arbitrary, inaccurate and ill-fitting indicators.

Welovelocalgovernment is a blog written by UK local government officers. If you have a piece you’d like to submit or any comments you’d like to make please drop us a line at: welovelocalgovernment@gmail.com

About these ads
Explore posts in the same categories: The future of Local Govt, We love the Council

Tags: , ,

You can comment below, or link to this permanent URL from your own site.

10 Comments on “Smack my bench up”

  1. LGWorker Says:

    That could almost be an argument against performance indicators completely…

  2. kriswith Says:

    The problems is that if you collect performance data but don’t have any context for it then you have no idea if you need to be doing something about the results. One example I know of is a score from a national survey. My borough got 27% which in itself might have been a worry – until we found out that this was the third highest score in England. Without the benchmark we might have spent money on a problem that wasn’t really a problem.

    You can always find a health warning attached to any benchmark but that doesn’t mean the context you get from comparisons with others is not still useful in understanding your own performance.

  3. somegardener Says:

    I can see your point about comparison that might or might not be appropriate. But that’s not to acknowledge that one of the big benefits of benchmarking is that it prompts you to collect information on how you do thinks and how much you spend on it. That sort of self awareness process can surely only be good.
    As for comparisons across a benchmark group – my bigger concern is the penguin effect – people heading for the middle/average so they can feel comfortable, rather than concentrating on finding out whether the “best” really have found out something worth learning.

    • mhuttch Says:

      Benchmarks to me are like dogs – they can be man’s best friend, but in the hands of irresponsible breeders and owners they are dangerous. They can never be, or should never be used as, anything more than a reference point.

      Those that produce them exaggerate their accuracy and comparability. They present data in ways that lead us to look at good/bad, slow/fast, cheap/expensive without considering the circumstances in which our service operates or the requirements and expectations of the communities we serve – league tables are a classic. Most people that pick up a benchmark report have no idea how to read or interpret them – again partly the fault of the people that produce them.

      Benchmark makers would do well to:

      1) be clear about what the benchmark is to be used for
      2) involve the people that are going to read the reports in their design
      3) agree with the same people where the greatest inaccuracies are likely to occur and agree what, if any, mitigating or ‘equalising’ approaches can be taken
      4) guide and coach contributers throughout the data collection process
      5) guide and coach their readers through what the reports tell them and, importantly, what they hide
      6) be under no illusion that they can produce anything other than something that is ‘roughly right’ (which is better than exactly wrong).

      Of course benchmarks are inaccurate, but I think that’s ok as long as it is understood that they are only ever one part of a bigger picture/body of evidence. The numbers are never the interesting thing – it is the story behind the numbers and these will be different in each place. The best benchmark producers follow the above approach. The very best are honest and up front about the flaws in their ‘product’ and be never ‘sell’ their benchmark as anything more than one useful piece in the jigsaw of information that supports good decision making.

  4. Jeremiah Says:

    I’d absolutely agree with encouraging managers to focus on their own services rather than relying on comparisons with others. So #kriswith, if your 27% score worries you then maybe you should be doing something about it – if everyone else is even worse, that doesn’t make it good enough.

    But if you rely on manager led improvement and don’t look at benchmarks, the problems are that (a) some managers are more inclined to hide poor performance than address it, and (b) if there are limited resources to undertake improvement projects, how do you prioritise them, other than by who shouts loudest? To deal with those you need some element of independent, top-down challenge and review. For all its flaws, benchmarking can give you some basis for judging how well a service is performing, and how much better it could plausibly get, that doesn’t depend totally on what the service manager tells you.

    And for me, knowing that some other councils perform better is only the start – the next questions are who are they, and what good ideas did they have that we could borrow? Because it shouldn’t be about defending, it should be about improving.

  5. brookc Says:

    The flaw is often that people are looking for benchmarking to ANSWER questions. When used well, its about provoking questions to ASK.

  6. Performance Officer Says:

    Ive never seen benchmarking used well. Inthe 27% case, that’s a perfect use for it (though I agree that it then sounds like potential complacency as a result – just because everyone’s rubbish doesn’t mean its ok to be rubbish).

    In every case of benchmarking it’s either been used as an excuse (our performance is poor, but it’s ok, everyone else is getting worse so it MUST be national factors out of our control) or explained away by those local differences, the pesky things (ok, often very valid things).

    it SHOULD be good for reference, and it SHOULD be good for target setting, at least initially. But in practice, it’s rarely any of that and usually takes vast quantities of effort to identify.


  7. [...] a reminder our post had argued the following: To be honest I would be quite happy if I never saw another benchmarking report ever [...]


  8. [...] Don’t use contrived statistics for benchmarking. Trying to create statistics to meet someone else’s requirements is a sure fire to make your [...]


  9. [...] seems a little obvious to us, but not to others perhaps.  Of course it always helps to have a good benchmark to measure against.  Local approaches to other things have borne fruit, not least the local [...]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 237 other followers

%d bloggers like this: