by GARRY CARNEGIE and LEE PARKER

Merlin Crossley comments in CMM on the use of metrics in higher education, raising important issues, especially during this COVID-19 period, CMM October

While accounting and performance measurement using metrics or, “key performance indicators” (KPI), is typically seen as “technical practice”, accounting and related metrics is also an influential “social practice” – for good or ill. This has proven to be particularly the case with universities.

According to Carnegie et al (2020), “increasingly, accounting has become an object of study less technical in nature, but rather as a pervasive, enabling and disabling social phenomenon”. Accounting is not only endemic throughout society and organisations, including public universities, but it constructs and can distort how they function (see, for example, Tsahuridu and Carnegie, 2018).

What does this mean? Accounting and performance measurement provide “a means to enact, establish or create altered or new conditions for the conduct and evaluation of our life (including work-life)” (Carnegie et al, 2020). Today accounting is increasingly recognised and understood as a key instrument of power and control in organisations and society. In such respects it is arguably being misused in universities.

According to Crossley, metrics exist at least in part on the grounds that “if you can’t measure it, you can’t manage it”. In management control, it is known that what gets measured may not be important, and what is/cannot be measured may actually be important or crucial. Staff subject to demanding performance measurement and management regimes typically focus strongly on their personal KPI without necessarily considering an organisation’s wider mission, culture and reputation.

Many disregard such “irrelevant” aspects of their work, considering their role fulfilled as long as their personal KPI are met or exceeded. The best longer-term interests of an organisation are often the forgotten casualty.

Crossley also states “ideally, transparently measuring things can provide protections against disputes and disharmony”. Disharmony is often triggered by ‘stretch’ KPI that are often unrealistic and will not be achieved.  Not all staff are medal winning “olympians” to coin a phrase.

When aggressive stretch KPI are incentivised through bonuses and other forms of prized benefits, such a sabbatical leave and conference funding, motivation and performance can be negatively impacted. Indeed, decades of research into organisational members’ motivation and reactions to target setting reveals that performance targets, set at ideal or lofty levels, demotivate and produce disastrous individual and institutional performance.

Metrics at universities have arguably driven exclusion whereby many staff feel undervalued, teaching standards are distorted in search of meeting stretch KPI, and entire specialist research subjects are neglected in response to metric measures biased against them (e.g. by means of lower journal rankings of specialist journals).

For instance, specialist environmental accounting journals may not be well ranked, but scholars and advocates in this field have personal missions to help create a better world by means of protecting the natural environment. Yet such scholars are often devalued, even when they are genuinely seeking to help answer “big questions” and to solve “wicked problems”. Is this not the unique mission of public universities?

It is often informally remarked within the corridors of universities that there are certain A* journals which are informally regarded as being at the upper end of all Australian Business Deans Council officially ranked A* journals.   This unofficial internal practice tends to reflect the discourse of self-appointed elites within the sector, who prescribe their own informal ranking systems within their universities or disciplines.

Such distorted metrics have almost completely replaced any reference to significant contribution to knowledge and have caused researchers to obsess about the status of the publishing media in which their findings are published. The findings and conclusions are no longer referred to or discussed, since this is not the main “game” or they no longer matter in an environment of short-termism.

At the national metrics level, despite governmental and university declarations of the value of interdisciplinary and multidisciplinary research, the regimented use of Field of Research (FoR) codes militates against it. This potentially closes off a significant prospect for answering big questions and resolving wicked problems.

While Crossley may be showing fair judgment in stating “the world is still not perfect,” an imperfect world is no justification for perpetuation of flawed performance measurement and control systems. It is rather an argument for no change, in anything, at any time. In addition, we suggest that he misses the mark by stating that “the measurements are always imperfect. One type of error involves inaccuracy”.

Other key problems include how measures are designed, interpreted and assessed. And even beyond these is the problem of how performance is managed and how decisions on the assessment of staff are made and implemented.

We do agree that Crossley is not far from the mark in stating “the bigger problem is that most metrics measure the wrong things. Most simply, numbers are designed to measure quantity but not quality”.  So-called quantity metrics are often dressed-up to masquerade as quality metrics.

We hasten to point out, however, that university management’s  enslavement to metrics as social practice, is not a matter for casual reflection on the art/science relationship as Crossley suggests. It is a matter of urgent societal concern, since it has the potential to undermine the purpose and ethics of universities, distort the pursuit of knowledge, muzzle independent thought and critique, truncate interdisciplinary and multidisciplinary approaches to addressing critical questions and problems, and even has the potential to reduce universities to simply regressing to becoming lowest common denominator vocational trainers.

Emeritus Professor Garry Carnegie RMIT University and Distinguished Professor Lee Parker, RMIT University and the University of Glasgow.

References:

Carnegie, G., Parker, L. and Tsahuridu, E. (2020), “It’s 2020: What is accounting today?”, Australian Accounting Review, forthcoming.

Tsahuridu, E. and Carnegie, G. (2018) “Accounting as a Social and Moral Practice”, IFAC Global Knowledge Gateway, 30 July, available here .