Friday, 27 September 2013

Not measuring up


On Tuesday I gave a talk in Paris to a group of business leaders. The topic was secrecy in organizations, which is something I am researching at the moment.  In the discussion session which followed – in a mixture of French and English and, probably, Franglais, so something may have got lost in translation – I was asked the following question: how do you measure secrecy?
There’s no easy answer to that, if indeed there is an answer at all, and much would depend on what is meant by secrecy anyway. But it was an interesting question because it is one which is very commonly asked, not just in relation to secrecy of course, but in relation to organizational issues of all kinds. To respond that it is difficult or impossible invariably provokes, as on this occasion, a disappointment. But what lies behind the demand for measurement?

The obvious answer, I suppose, is something like clarity or certainty; a desire for the tangible and the real. The irony, though, is that measurement almost always provokes heated debates. For a couple of examples, take some stories from today’s news.  An IPCC report makes ‘with 95% confidence’ predictions about global temperature change. But immediately it is published the figures are pored over and differentially interpreted by those on different sides of the debate about climate changes, its causes, and the responses to be made. Meanwhile, in the UK, the opposition Labour Party have proposed a cap on energy bills to which the immediate response has been to point out that in fact UK energy costs are amongst the lowest in Europe. But is that the relevant figure? Or should it be the cost relative to income, or the change over a given time period (and, if so, which time period)? Or should it differentiate between the proportion of the charge that goes to the energy companies and that which goes in tax? As these examples show, measurement does not result in clarity, but perhaps even the opposite: the provision of a measurement provokes a demand for more measurements, which in turn provoke further demands and debates.

In such debates two things are immediately obvious. One is that measurement cannot be separated from meaning: numbers never ‘speak for themselves’, but are susceptible to a wide range of interpretations. They depend on meaning which cannot itself be measured. The very choice of what we measure is dependent on some prior judgement that that is the thing which is meaningful. The second, closely related, thing is that this meaning is a matter of political contestation, and the meanings which prevail are an outcome of power. That is most plain in overtly political stories such as those just mentioned. But it is no less the case in supposedly apolitical contexts such as accounting within organizations, where what is supposedly neutral and ‘independent’ measuring inevitably rests upon interpretative meaning and power – something long recognized by critical accounting researchers.
In the third edition of my book, there are more statistics that in the previous editions and that is because in the new chapter five (where almost all these statistics occur), there is a more overt focus on politics and economics. So, for example, I give figures on things like executive pay and corporate tax avoidance. And the aim here, too, is to seek to advance particular meanings and particular politics. That is to say, measurement is a form of rhetoric: an attempt to persuade by controlling meaning and so to exercise power. But the important thing to note is that this is unavoidable. The question ‘how do you measure that’ is itself a form of rhetoric which is a challenge from the questioner. If, as in my talk on Tuesday, you fail to answer it then the implication, at least, is that you are talking about something meaningless.

The potential stigma of ‘not measuring’ exercises a powerful hold within organization studies which, in its mainstream incarnation, is infatuated with measurement. It is even more evident within economics, which has shifted wholesale from its roots in 19th century political economy to being dominated by ever more abstruse mathematical modelling. (Deirdre McCloskey's book, The Rhetoric of Economics is great on this*). But because measurement always opens up new terrains of contestation about meaning (as in the examples above) this is a doomed enterprise. Indeed, economics, especially since the financial crisis, is increasingly criticised for being precisely because of its abstraction from reality – a criticism which is again a form of rhetoric that tries to position its opponent as meaningless.

Thus we should not approach this issue in terms of the presence or absence of measurement, but always consider what this presence or absence betokens. In other words, we should see questions of meaning and power as the primary ones and then consider what role measurement plays in the attempt to corral meaning and exercise power. I know this because in a survey 100% of those asked agreed with me. But now I suppose you will want to know the sample size.
 
* And note how throwing in a reference to a 'great' book is another rhetorically forceful way of justifying a point. Here meaning is claimed on the basis of an appeal to authority rather than to a measurement. There is no way of standing outside rhetoric in this sense: indeed commenting upon my own rhetorical devices is a way of advancing the argument I am making. And so is commenting upon my commenting ....

2 comments:

  1. Very interesting piece, Chris. You make a very strong point for the political assumptions behind numbers. I wonder if it is possible to think of numbers in non-political ways. Here is an attempt: What about the sequence of prime numbers in maths? Where is the politics in this? (Maybe the view that Mathematics alone among sciences can address reality in total abstraction?)

    And another question that has long fascinated me. How does the rhetorical power of numbers compare to that of stories? What is more persuasive: an appropriately chosen set of statistics or a suitable story? Does the inclusion of more statistics in the latest edition of your excellent book suggest that you are coming to appreciate increasingly the power of numbers?

    ReplyDelete
  2. Thanks for these interesting thoughts, Yiannis. Here are the thoughts they provoked.

    The question of whether pure maths can be thought of politically is a challenging one (and I suspect, if I did but know it, one that has been written about by philosophers of maths) but my thought is that perhaps it is present in a certain form, in the way that many mathematicians talk about the ‘beauty’ of various mathematical phenomena and proofs. The sequence of primes might be one such example: it has an aesthetic quality that makes it remarkable. And that does involve something like saying that beauty is normatively desirable (as opposed to ugliness) which might take us to a kind of social judgment which in turn entails, at least potentially, a politics: why should beauty be regarded as positive?

    But I wouldn’t go to the stake for that rather tricksy response: what I think is more to the point is that what I am really talking about in the post is measurement, rather than numbers per se. And the political nature of measurement is more clear cut, most especially when what is at stake is the measurement of social things (but not just then, because all choices of what to measure and how are in some diffuse sense political).

    On the question of the relative rhetorical persuasiveness of stories versus numbers, I don’t think one can answer that in the abstract; what stories, what numbers and what context (and what is the quality of the story and of the stats)? In my previous post on this blog, on suicide, I refer to a paper which is the story of one suicide attempt and it has great rhetorical power precisely because it expresses an experience so graphically; but it gains power when set alongside the stats on suicide levels – and vice versa. But this made me think that there is something different about combining both types of rhetoric. If we have a story, and then provide a second story, or if we have some stats and then provide some more stats it seems to me that each new story or each new statistic adds to the rhetorical power of the argument But when we combine a story and some stats somehow we don’t just add to but ‘multiply’ the rhetorical power. Do you think this could be so?

    On my own stance, well I have never been ‘hostile to numbers’. I have always disagreed with those in the critical camp who regard ‘positivism’ as a priori uncritical. For example, research which collates statistics on industrial accidents and seeks to make causal links between those and, say, particular work practices (or forms of ownership, or legal jurisdictions or whatever) seems to me to be eminently critical; and arguably more politically and practically effective than much critical theorising. The greater use of stats in the new edition of the book of this blog I think probably just reflects the nature of the new material, rather than any shift in my appreciation of numbers.

    ReplyDelete