by JOHN DUMAY, JAMES GUTHRIE and ANN MARTIN-SARDESAI
In our submission to the Sheil review of the Australian Research Council we wholeheartedly support terminating Excellence in Research for Australia.i Scrapping it will save the sector $250m, considering academic time, professional staff time, processes, and submission requirements—also, costs for the ERA process within the ARC.
While the results of this exercise do not directly impact university finances (unlike in the UK), they have been taken seriously by our vice-chancellors and deans. Consequently, university executives continually set ambitious targets to improve their scores for each discipline.
Our published research highlights[ii] that many academics regard the ERA evaluations with trepidation as the benefits from a good score for their discipline are unclear and possibly non-existent to individual academics. In contrast, the costs of not meeting the targets are potentially severe or even unfair, including removing internal research funding allocations, reducing allocations for research in workloads, reducing conference funding, and performance management of academics as they are viewed as underperforming in research.[iii]
In our submission, we provide case study evidence about the accuracy and appropriateness of the current assessment process, especially for those disciplines where peer review rather than metrics is used to make these assessments. Our skepticism of the ERA peer-review process comes from observation of an apparent disconnect between increases in the quantity and quality of research outputs, including publications and income, and the score awarded within the Australian accounting disciplines.
In our case study in the commentary, we argue that the ERA peer-review process in accounting emphasises quantitative and positivist research in mainstream research topics. Also significant increases in the quality and quantity of research outputs will not necessarily improve discipline scores for a university [iv].
Our contribution to the review is not a complaint about how the ERA appeared to get accounting ratings amiss. Instead, it highlights how the peer review practices used to judge the performance of academics and their publications are, at least fragile.
The situation is not helped by a substantial lack of transparency in the current ERA process, whereby there is no need to justify scores for university disciplines externally. The previous ARC’s practice was to release a report at the end of each round, providing aggregated data for each discipline and the awarded scores for each university discipline. No rationale was provided for calculating that score for a given university. In our perspective, the process is akin to a student submitting an assignment, and the only feedback they receive is a mark without written comments and a report about how the class performed. Would an Australian public university do this ?[v]
Therefore, we agree with the Sheil Review consultation paper that:
“there is no doubt that ERA has been tremendously effective in shifting the focus of Australian research from an emphasis on quantity to quality of outputs, particularly in its early iterations, but it can be argued that ERA has achieved its initial purpose and that the time and resources involved may be better re-directed to other evaluation needs”[vi].
Emeritus Professor James Guthrie, Macquarie Business School, Professor John Dumay Macquarie Business School Dr Ann Martin-Sardesai, School of Business and Law, CQUniversity
***
[i] James Guthrie, John Dumay, and Ann Martin-Sardesai, (2022), Review of the Australian Research Council Consultation paper: An Academic Response to the Paper, Commentary on the ERA Review of the Australian Research Council Consultation paper 10 November 2022.
[ii] For instance, Martin-Sardesai, A., Irvine, H., Tooley, S. and Guthrie, J. (2017a), “Government research evaluations and academic freedom: a UK and Australian comparison”, Higher Education Research & Development, Vol. 36 No. 2, pp. 372-385.
Martin-Sardesai, A., Irvine, H., Tooley, S. and Guthrie, J. (2017b), “Organisational change in an Australian university: Responses to a research assessment exercise”, The British Accounting Review, Vol. 49 No. 4, pp. 399-412.
[iii] Martin-Sardesai, A., Guthrie J. and Parker, L. (2020), “The neoliberal reality of higher education in Australia: how accountingisation is corporatising knowledge”, Meditari Accountancy Research, https://doi.org/10.1108/MEDAR-10-2019-0598
[iv] Guthrie, J. Parker, L. Dumay, J. and Milne, M. (2019), “What Counts for Quality in Interdisciplinary Accounting Research: A Critical Review and Reflections”, Accounting, Auditing & Accountability Journal, Vol. 31 No. 1, pp. 2-25.
[v] James Guthrie and Brendan O’Connell (2022) Time to re-think peer review in ERA: The peer review practices used to judge the performance of particular academics and their publications are at least, potentially biased and fragile, Campus Morning Mail 11 April 2022. https://campusmorningmail.com.au/news/time-to-re-think-peer-reports-in-era/
[vi] ERA Review of the Australian Research Council Consultation paper, 10 November 2022.