The Australian Research Council is reviewing its two research metric schemes, Excellence for Research in Australia, and Engagement and Impact. James Cook U has ideas
JCU’s submission to the review argues, “ERA, in its current form, is past its use-by date.” And there are “several refinements and developments” that “should be considered.”
Recommendations in a policy-rich submission include;
ERA
* expand the comparison base for Australian research performance, for example by including, Netherlands, Canada, UK, Singapore, South Korea, Sweden and New Zealand. “This would provide a ‘reality check’ to validate where Australian research sits.”
* switching peer-reviewed FORs to citation analysis, where “fair and robust”
* establish a separate report for Aboriginal and Torres Strait Islander research, using data from ERA, EI, NHMRC and ARC grant outcomes. The use of “world standard” measures for ATSI research is problematic; “the appropriate measure here is less academic quality per se (although sound scholarship is expected) but rather, quality in terms of fit-for-purpose to inform policy, implementation, and practice, including dynamic, generative cultural practice.”
EI
* provide for “external verification” of claims by including URLs for evidence
* use two-digit socio-economic objective codes as “the organising principle”
* as for the engagement indicators, “the fundamental driver of engagement is the quality, calibre, sustainability, and nature of the relationships between the end-users and the researchers/research groups. There are no simple or adequate metrics to quantify the nuances of such relationships particularly in a robust fashion consistent across institutions.”
Overall, if methodological flaws are addressed both exercises could usefully occur every three years, JCU suggests.
Get the word out
The ARC plans to release submissions to the research metrics review after it is out, which seems a bit late for a debate. So, CMM will report and/or link to, as many submissions as it can – send them in people.