by MARGARET SHEIL, SUSAN DODDS and MARK HUTCHINSON 

Since the Final Report of our Review of the Australian Research Council was published by the Minister for Education, the Hon Jason Clare MP, we have been asked about the future of research evaluation, in light of our recommendation to discontinue the two research evaluation exercises – Excellence in Research for Australia (ERA), which concentrates on quality, and the Engagement and Impact Assessment (EI) – given the on-going need to demonstrate the benefit of government, industry and (increasingly) universities themselves investing in research in Australian universities.

The Panel received considerable support in its broad consultations across the university sector, as well as research-facing industry, government and community groups, on its proposal to recommend the cessation of ERA and EI, without denigrating the value the exercises had delivered to date. We also received significant support for the continuation of the ARC’s evaluation capabilities. Some observers find a tension between these two recommendations, but they are in fact in alignment.

What we need now

At its conception, ERA was designed to be sufficiently rigorous and robust to enable the ERA results to drive the allocation of research block grants. Apart from a short period a decade ago, ERA has not determined research funding, so the exercise is significantly over-engineered for its current application. We are using a Ferrari to pop down the shops for milk and bread.

The nation’s research evaluation need today is not for a labour-intensive, retrospective, comprehensive research assessment exercise. Instead, we need a combination of a tailored quality assessment function aligned to research provider standards; a rigorous, prospective gap-and-opportunity analytic function, to ensure Australian research is pressing our advantages and addressing our shortcomings in research for national competitiveness; and a survey function that reports back the myriad of benefits enabled by the research, to the community that funds and sustains it. The sector certainly still needs a trusted assessment mechanism; it just needs a more sophisticated and better targeted one.

We envisage that these functions would be delivered through the continuation, evolution and extension of the ARC’s evaluation activities, utilising the extraordinary capability within the ARC and its capacity to convene expertise from across the sector and internationally. As we highlighted in our report, trusting in Australia’s ability built up over decades of enabling the very best Australian research and conducting multiple rounds of ERA has created a foundation of integrity, competence, care, and reliability.

It is expected that the ARC would lead the development and trialling of new methods of assessment that utilise advances in data analysis without sacrificing rigour or the required breadth of application across the disciplines, including the humanities and qualitative social sciences, and particularly to the Non-Traditional Research Outputs (NTROs) of the creative arts and design disciplines.

There are several distinct contributions the ARC is uniquely placed to make through this application of its advanced evaluation capability. These are: the verification of standards for TEQSA, for the benefit of universities and university colleges; research capability mapping, for the benefit of government and industry; and the demonstration of the value of publicly funded research, for the benefit of the government, the parliament and the public.

Threshold standards

To many in the university sector the most immediate question was how TEQSA would assess the standards of research without ERA, now that universities are required to meet quality standards in a proportion of fields in which they teach.

The Panel envisages that the ARC would still provide that assessment service, only with vastly greater efficiency and timeliness: instead of assessing the entire sector once every several years, an assessment process could focus on only those institutions seeking accreditation or requiring reaccreditation in a given year. There were 2,603 units of evaluation assessed in the last ERA round in 2018. By contrast, under our proposed arrangements the ARC could expect to examine around 60 units of evaluation per annum, with a process designed to establish whether an institution’s research quality meets the threshold standards.

Capability mapping

While TEQSA’s needs are certainly salient for the standing of individual institutions, the evaluation of national research capability in specific areas is of critical importance to industry and government. The ARC is perfectly positioned to assist the whole of government – not just the Education Minister – to gain a precise and accurate picture of the research capability that underpins the success of each portfolio in Cabinet.

In collaboration with his Cabinet colleagues, the Minister could request a capability mapping exercise in specific areas – such as that called for in the government’s new Quantum Strategy. The upcoming national robotics strategy will also benefit from a similar exercise, as well as other areas of rapidly advancing technology, such as artificial intelligence, and areas of emerging critical need, such as climate change mitigation.

In a trusted leadership role the ARC could identify emerging areas of strength and competitive advantage, as well as critical areas where we lag nationally, not only through its evaluation capability but through the high-gain lens of the research grant assessment process, which affords a highly magnified picture of capability, global competitiveness and conceptual innovation of any given non-medical field.

As two of the case studies in our final report illustrate – on quantum computing and solar cells – the ARC has conducted this work to outstanding effect in the past. It is no coincidence that these are the two top-of-mind fields identified by the Prime Minister to be ripe for local advanced manufacturing.[1] The factories of the future are built on the research of the present and the policy foresight of the past. We need the ARC to keep doing this sort of horizon-scanning, identifying the next generation of research frontiers to underpin our future industries.

Demonstrating value

The ARC is also well placed to demonstrate the effectiveness of public funding for research by highlighting the benefits of the work it has funded over time. By keeping track of outcomes in the years after award, the stories of the full impact of funded research can be told; including discoveries that transform our understanding of how the world works, new and valuable international collaborations on common problems, and breakthroughs that lead to new products, processes and policies.

Expertise

At the heart of this evaluation capability is the ARC’s deep expertise – both in-house and on-call. With board members, a CEO, executive directors, the College of Experts, other staff and advisory committees all experience in the conduct and management of research, the ARC will be well equipped to provide detailed, up-to-date and targeted advice to government on research across all its domains.

In responding to the review’s recommendations, the government has the chance to establish a sound footing for the ARC to function not only as the primary funder of the best Australian research, but as a trusted pathfinder and a sentinel on research for the whole of government.

Professors Margaret Sheil, Susan Dodds and Mark Hutchinson constituted the Panel for the federal government’s independent review of the Australian Research Council Act 2001, published last month.

Margaret Sheil is a former ARC CEO and Vice-Chancellor of the Queensland University of Technology

Susan Dodds is Senior Deputy Vice-Chancellor and Vice-President (Research and Industry Engagement) at La Trobe University

Mark Hutchinson is Director of the ARC Centre of Excellence for Nanoscale BioPhotonics at the University of Adelaide.

[1] Hon Anthony Albanese MP, quoted in Sean Kelly, ‘The Year of Living Cautiously’, The Monthly, June 2023 HERE