Not measuring up

On Tuesday I gave a talk in Paris to a group of business leaders. The topic was secrecy in organizations, which is something I am researching at the moment.  In the discussion session which followed – in a mixture of French and English and, probably, Franglais, so something may have got lost in translation – I was asked the following question: how do you measure secrecy?

There’s no easy answer to that, if indeed there is an answer at all, and much would depend on what is meant by secrecy anyway. But it was an interesting question because it is one which is very commonly asked, not just in relation to secrecy of course, but in relation to organizational issues of all kinds. To respond that it is difficult or impossible invariably provokes, as on this occasion, a disappointment. But what lies behind the demand for measurement?

The obvious answer, I suppose, is something like clarity or certainty; a desire for the tangible and the real. The irony, though, is that measurement almost always provokes heated debates. For a couple of examples, take some stories from today’s news.  An IPCC report makes ‘with 95% confidence’ predictions about global temperature change. But immediately it is published the figures are pored over and differentially interpreted by those on different sides of the debate about climate changes, its causes, and the responses to be made. Meanwhile, in the UK, the opposition Labour Party have proposed a cap on energy bills to which the immediate response has been to point out that in fact UK energy costs are amongst the lowest in Europe. But is that the relevant figure? Or should it be the cost relative to income, or the change over a given time period (and, if so, which time period)? Or should it differentiate between the proportion of the charge that goes to the energy companies and that which goes in tax? As these examples show, measurement does not result in clarity, but perhaps even the opposite: the provision of a measurement provokes a demand for more measurements, which in turn provoke further demands and debates.

In such debates two things are immediately obvious. One is that measurement cannot be separated from meaning: numbers never ‘speak for themselves’, but are susceptible to a wide range of interpretations. They depend on meaning which cannot itself be measured. The very choice of what we measure is dependent on some prior judgement that that is the thing which is meaningful. The second, closely related, thing is that this meaning is a matter of political contestation, and the meanings which prevail are an outcome of power. That is most plain in overtly political stories such as those just mentioned. But it is no less the case in supposedly apolitical contexts such as accounting within organizations, where what is supposedly neutral and ‘independent’ measuring inevitably rests upon interpretative meaning and power – something long recognized by critical accounting researchers.

In the third edition of my book, there are more statistics that in the previous editions and that is because in the new chapter five (where almost all these statistics occur), there is a more overt focus on politics and economics. So, for example, I give figures on things like executive pay and corporate tax avoidance. And the aim here, too, is to seek to advance particular meanings and particular politics. That is to say, measurement is a form of rhetoric: an attempt to persuade by controlling meaning and so to exercise power. But the important thing to note is that this is unavoidable. The question ‘how do you measure that’ is itself a form of rhetoric which is a challenge from the questioner. If, as in my talk on Tuesday, you fail to answer it then the implication, at least, is that you are talking about something meaningless.

The potential stigma of ‘not measuring’ exercises a powerful hold within organization studies which, in its mainstream incarnation, is infatuated with measurement. It is even more evident within economics, which has shifted wholesale from its roots in 19^th century political economy to being dominated by ever more abstruse mathematical modelling. (Deirdre McCloskey's book, The Rhetoric of Economics is great on this*). But because measurement always opens up new terrains of contestation about meaning (as in the examples above) this is a doomed enterprise. Indeed, economics, especially since the financial crisis, is increasingly criticised for being precisely because of its abstraction from reality – a criticism which is again a form of rhetoric that tries to position its opponent as meaningless.

Thus we should not approach this issue in terms of the presence or absence of measurement, but always consider what this presence or absence betokens. In other words, we should see questions of meaning and power as the primary ones and then consider what role measurement plays in the attempt to corral meaning and exercise power. I know this because in a survey 100% of those asked agreed with me. But now I suppose you will want to know the sample size.

 

* And note how throwing in a reference to a 'great' book is another rhetorically forceful way of justifying a point. Here meaning is claimed on the basis of an appeal to authority rather than to a measurement. There is no way of standing outside rhetoric in this sense: indeed commenting upon my own rhetorical devices is a way of advancing the argument I am making. And so is commenting upon my commenting ....