The world is changing fast and to keep up you need local knowledge with global context.
We are living in an era where metrics have become more important. Businesses have used it for a long time and how else would you measure the success of an enterprise if you do not consider metrics such as productivity, sales revenue, gross margins, profits and losses and overhead costs to name but a few. It has also been adopted by many companies and businesses to incentivise their work force; rewards are given for reaching pre-set milestones. But have our societies become too obsessed with metrics? The old farming analogy of weighing the pig does not make it fatter, seems relevant in this argument particularly when it comes to education. There is a school of thought that believes you should teach children and let them progress from year to year without constantly submitting them to tests and examinations. This is particularly relevant in countries like the United Kingdom where children sit national examinations repeatedly throughout their school years. The accuracy of metrics to determine the health of an education system is also a matter of debate. The matric results were released a couple of days ago in South Africa and the question could rightly be asked; is this metric a fair reflection of what goes on in the education system in South Africa. Equal Education believes it is a superficial barometer of the education system and does not reflect the huge dropout rate before matric. Professor Jerry Z Muller writes on the Aeon website republished below that he believes the obsession with metrics is counterproductive and it invites “gaming”. – Linda van Tilburg
Against metrics: how measuring performance by numbers backfires
By Jerry Zuller*
More and more companies, government agencies, educational institutions and philanthropic organisations are today in the grip of a new phenomenon. I’ve termed it ‘metric fixation’. The key components of metric fixation are the belief that it is possible – and desirable – to replace professional judgment (acquired through personal experience and talent) with numerical indicators of comparative performance based upon standardised data (metrics); and that the best way to motivate people within these organisations is by attaching rewards and penalties to their measured performance.
The rewards can be monetary, in the form of pay for performance, say, or reputational, in the form of college rankings, hospital ratings, surgical report cards and so on. But the most dramatic negative effect of metric fixation is its propensity to incentivise gaming: that is, encouraging professionals to maximise the metrics in ways that are at odds with the larger purpose of the organisation. If the rate of major crimes in a district becomes the metric according to which police officers are promoted, then some officers will respond by simply not recording crimes or downgrading them from major offences to misdemeanours. Or take the case of surgeons. When the metrics of success and failure are made public – affecting their reputation and income – some surgeons will improve their metric scores by refusing to operate on patients with more complex problems, whose surgical outcomes are more likely to be negative. Who suffers? The patients who don’t get operated upon.
When reward is tied to measured performance, metric fixation invites just this sort of gaming. But metric fixation also leads to a variety of more subtle unintended negative consequences. These include goal displacement, which comes in many varieties: when performance is judged by a few measures, and the stakes are high (keeping one’s job, getting a pay rise or raising the stock price at the time that stock options are vested), people focus on satisfying those measures – often at the expense of other, more important organisational goals that are not measured. The best-known example is ‘teaching to the test’, a widespread phenomenon that has distorted primary and secondary education in the United States since the adoption of the No Child Left Behind Act of 2001.
Short-termism is another negative. Measured performance encourages what the US sociologist Robert K Merton in 1936 called ‘the imperious immediacy of interests … where the actor’s paramount concern with the foreseen immediate consequences excludes consideration of further or other consequences’. In short, advancing short-term goals at the expense of long-range considerations. This problem is endemic to publicly traded corporations that sacrifice long-term research and development, and the development of their staff, to the perceived imperatives of the quarterly report.
To the debit side of the ledger must also be added the transactional costs of metrics: the expenditure of employee time by those tasked with compiling and processing the metrics in the first place – not to mention the time required to actually read them. As the heterodox management consultants Yves Morieux and Peter Tollman note in Six Simple Rules (2014), employees end up working longer and harder at activities that add little to the real productiveness of their organisation, while sapping their enthusiasm. In an attempt to staunch the flow of faulty metrics through gaming, cheating and goal diversion, organisations often institute a cascade of rules, even as complying with them further slows down the institution’s functioning and diminishes its efficiency.
Contrary to commonsense belief, attempts to measure productivity through performance metrics discourage initiative, innovation and risk-taking. The intelligence analysts who ultimately located Osama bin Laden worked on the problem for years. If measured at any point, the productivity of those analysts would have been zero. Month after month, their failure rate was 100 per cent, until they achieved success. From the perspective of the superiors, allowing the analysts to work on the project for years involved a high degree of risk: the investment in time might not pan out. Yet really great achievements often depend on such risks.
The source of the trouble is that when people are judged by performance metrics they are incentivised to do what the metrics measure, and what the metrics measure will be some established goal. But that impedes innovation, which means doing something not yet established, indeed that hasn’t even been tried out. Innovation involves experimentation. And experimentation includes the possibility, perhaps probability, of failure. At the same time, rewarding individuals for measured performance diminishes a sense of common purpose, as well as the social relationships that motivate co-operation and effectiveness. Instead, such rewards promote competition.
Compelling people in an organisation to focus their efforts on a narrow range of measurable features degrades the experience of work. Subject to performance metrics, people are forced to focus on limited goals, imposed by others who might not understand the work that they do. Mental stimulation is dulled when people don’t decide the problems to be solved or how to solve them, and there is no excitement of venturing into the unknown because the unknown is beyond the measureable. The entrepreneurial element of human nature is stifled by metric fixation.
Organisations in thrall to metrics end up motivating those members of staff with greater initiative to move out of the mainstream, where the culture of accountable performance prevails. Teachers move out of public schools to private and charter schools. Engineers move out of large corporations to boutique firms. Enterprising government employees become consultants. There is a healthy element to this, of course. But surely the large-scale organisations of our society are the poorer for driving out staff most likely to innovate and initiate. The more that work becomes a matter of filling in the boxes by which performance is to be measured and rewarded, the more it will repel those who think outside the box.
Economists such as Dale Jorgenson of Harvard University, who specialise in measuring economic productivity, report that in recent years the only increase in total-factor productivity in the US economy has been in the information technology-producing industries. The question that ought to be asked next, then, is to what extent the culture of metrics – with its costs in employee time, morale and initiative, and its promotion of short-termism – has itself contributed to economic stagnation?
- The Tyranny of Metrics (2018) by Jerry Z Muller is published via Princeton University Press.
- This article was originally published at Aeon and has been republished under Creative Commons.
Cyril Ramaphosa: The Audio Biography
Listen to the story of Cyril Ramaphosa's rise to presidential power, narrated by our very own Alec Hogg.