by ANGEL CALDERON
A European centric ranking
U-Multirank is a global university ranking which allow users to identify an institution’s strengths and weaknesses and can assist students in choosing which university to study at. It includes a range of indicators that allow institutional comparison of performance across five dimensions of higher education and research activities: teaching and learning, research, knowledge transfer, international orientation, and regional engagement.
Some indicators are based on data that is self-reported by institutions. Bibliometric data is drawn from the Web of Science database and is prepared by the Centre for Science and Technology Studies (CWTS) at Leiden University, which also produces the Leiden Ranking. Data on patents is drawn from the CWTS-licensed edition of the PATSTAT database.
U-Multirank does not provide an overall score, and institutions are ranked into five performance groups: very good, good, average, below average, and weak.
The European Commission established U-Mr to provide a “European perspective” to the global rise of rankings from the ‘European perspective’. Development was undertaken by the Consortium for Higher Education and Research Performance Assessment (CHERPA), led by the Centre for Higher Education Policy Studies (CHEPS) in the Netherlands.
The first edition of appeared in 2014 and covered 850 institutions from more than 70 countries. The 2022 edition covers 2202 institutions from 96 countries. 40 of these institutions are from Australia; however, only six Australian universities opted in to participate.
Participation of Australian universities
U-Multirank is yet to captivate the attention of students, academics and university leaders in Australian universities. U-Multirank states that it provides detailed evidence on the performance of Australian universities but this does not stand scrutiny (below).
On a dimension-by-dimension basis, we can see that all 40 universities are measured on four of the six indicators which comprise the research dimension, but only four universities are measured in five out of the eight indicators which comprise the teaching and learning dimension.
In the knowledge transfer dimension, there are nine indicators. Australian universities are assessed in the two bibliometric indicators derived from Clarivate’s Web of Science. For the remaining indicators, assessment is either missing or patchy.
In the international dimension there are six indicators, of which two are not particularly relevant to Australian universities (i.e., degree programmes offered in a foreign language at bachelor and master level). Six Australian universities are rated in four of the six indicators in this dimension.
In the regional engagement dimension, there are six indicators. Once again, six Australian universities measured in four or more indicators.
The six Australian universities which either submitted institutional data to U-Multirank or have elected to participate are: Australian Catholic University, Queensland University of Technology, University of Queensland, University of South Australia, Western Sydney University and University of Wollongong.
When Australian universities elect to participate in the ranking, U-Multirank prefills institutional data drawn from the Commonwealth Department of Education, Skills and Employment. Then, it is up to university planners to complete the institutional questionnaire (including data at the subject level if they choose to do so).
How Australian universities perform
Let us take selected indicators from every dimension to gauge how Australian universities fare.
In the teaching and learning dimension, there is only one indicator for which all universities are assessed. In the bachelor graduation rate, Sydney College of Divinity is the only institution that is rated “A” (i.e. very good), 10 are rated “B” (i.e. good), 11 are rated “C” (i.e. average), and 18 are rated “D” (i.e. below average). University of Wollongong is the only Australian institution rated in pedagogically skilled teaching staff and digital education investment and is rated “A” in both indicators.
In the research publications (size normalised) from the research dimension, 15 Australian universities are rated “A”, ten are rated “B”, nine rated “C” and six are rated “D”. Of the six universities rated in external research income, three are rated “A”: University of South Australia, University of Queensland, and Queensland University of Technology.
In the co-publications with industry partners indicator from the knowledge transfer dimension, four institutions are rated “A” (Murdoch, Adelaide, Melbourne, and Queensland), 12 are rated “B”, 15 are rated “C”, eight are rated “D” and not applicable for one institution.
In the measures of regional engagement, there is hardly any meaningful data to make an informed assessment. However, we should give kudos to University of South Australia and University of Queensland for attaining a rank of “A” and “B” respectively, in the measures of bachelor graduates working in the region and master graduates working in the region.
In the measures of international orientation there is also limited information about our universities. Of the six universities rated on international academic staff, four are rated “A” and two are rated “B”.
Universities opt in to participate in the 30 subject areas which form part of this ranking. There is hardly any meaningful data to enable comparison of Australian universities at the subject level.
Prospective students better look elsewhere
Prospective students for Australian universities are better served to look elsewhere to find out which universities suits their preferred areas of interest.
Whilst I do not wish to make recommendations for which ranking schema or systems are better positioned to assist students in choosing institutions, I will say that university websites would be a great way to get started. There is also the ComparED website, which is an Australian government initiative, and draws data from the Student Experience Survey and the Graduate Outcomes Survey.
Four ways to improve the Nature Index
The Nature Index was launched in November 2014, covering 63 journals which form part of the Nature Group. The ranking has continued to be published annually, and since 2018 it covers 82 high-quality natural-science journals. The index covers the Nature journal which is pivotal for ranking in the Academic Ranking of World Universities.
The index ranks the world’s top 500 institutions, of which 15 are from Australia. The University of Queensland remains Australia’s highest ranked institution at 76th, down from 69th last year (CMM, 23 May 2021).
Of the 15 ranked Australian institutions, nine moved up in position and six moved down. We can say that our scientists maintained their research output in the natural and science disciplines during the two pandemic years.
This is an index which allows for corrections, adjustment of institutional hierarchies, and removal of retracted papers. People in planning and research offices are reminded to check the live website for results from previous years, instead of relying on the frozen annual tables that we all like to keep in our desktops.
Nature aims to revise the list of journals covered by the index every five years. Notionally the review is scheduled for 2023 which could bear an impact for the edition due in June 2024.
Here are four ways in which Nature could bolster its appeal and impact in the wider academic community:
* widen journal coverage of its index. There are more than 160 journals which form part of the Nature family.
* separate the count of papers published in the Nature journal as opposed to those published in the family of journals under the Nature umbrella.
* include the number and proportion of papers which address any of the United Nation’s Sustainable Development Goals.
* identify proportion to which an institution’s collaborations are with authors from more than world geographical region or across income groups (i.e., low, lower-middle, upper-middle, and high income).
Angel Calderon is principal advisor, planning and research at RMIT