by ANGEL CALDERON
The global rankings season starts in earnest next month. I expect there will be plenty of insights to report on as these releases happen.
To start with, this year marks the twentieth year of global rankings, with QS and Times Higher Education making substantive methodological changes to their rankings. We continue to see increased participation of institutions from around the globe in all rankings, but also see stronger criticisms made against them.
Changing research landscape
To set the scene for what lies ahead, let me highlight how the geography of knowledge production has changed over the past twenty years.
United States’ dominance is steadfastly waning, whilst China’s is rapidly rising. There are also many countries which were previously importers of educational services and relied on high-income economies to build their educational systems that are also improving rapidly, such as Malaysia, Indonesia, Iran, and Saudi Arabia.
According to Elsevier’s Scopus, the United States global share of scholarly outputs has declined from 28 per cent in 2003 to 18 per cent in 2022. In turn, China’s global share has increased from 5 per cent to 26 per cent over the same period, while India’s world share has increased from 2 per cent to 5 per cent. In contrast, mature Western European economies (such as Germany, France, and the United Kingdom) and Japan are experiencing a decline in world share, whilst countries such as Canada, Netherlands, Sweden, and Switzerland have remained relatively unchanged over the past two decades.
As for research impact, the United States is also weakening in standing and influence. Among the top 20 countries with the highest volume of scholarly outputs, the United States stands in the lowest quartile of citation impact. Meanwhile, China is yet to make inroads in citation impact, with a field weighted citation impact (FCWI) of 0.31 in 2022 – the lowest among the top 25 countries of scholarly outputs. Western European countries have consistently outperformed the United States in FCWI over the last two decades. Countries such as Egypt and Saudi Arabia have made significant progress in citation impact; but the progress made by emerging countries such as Malaysia, Iran, Brazil, and Turkey cannot be underestimated.
Australia is a lucky country. Over the past twenty years, Australia’s scholarly output growth was above the rate of the United States and the United Kingdom and many other Western countries, but significantly below the rate of China, Egypt, Indonesia, Iran, Malaysia, and Saudi Arabia. However, Australia’s growth rate over the past five years is weakening – an indication that the levels of research investment have been insufficient to maintain pace with emerging economies.
In terms of research impact, Australia has improved over the past twenty years, but it has weakened over the last five. Australia has stepped up ahead of the United Kingdom but is lagging Sweden, Switzerland and the Netherlands.
In the absence of increased investment in research and research training, Australia has benefitted from the fact that our researchers are active in international collaboration and largely publish in top quartile journals. Australia has also benefited from the weakening standing of universities from North America and Western Europe.
Twenty years of global rankings
The first global ranking was published in June 2003 by Shanghai Jiao Tong University’s Institute of Higher Education with its Academic Ranking of World Universities (ARWU), listing 13 Australian universities among the top 500. A year later, Times Higher Education (THE) and QS published the first commercial global rankings, listing 14 Australian universities among the top 200.
Since then, there have been numerous ranking schemas that have emerged, focusing on various aspects of university’ endeavors, including regional and subject rankings as well as specialized rankings (such as those focused on sustainability).
Let us look at what lies ahead when the global rankings get into full swing next month.
THE Impact Rankings
There were more than 1700 institutions globally which submitted data for the fifth edition due for release on June 1. Last year, there were 1406 institutions ranked compared to 1115 in 2021. In previous years, this ranking was released in April.
This ranking is focused on measuring universities’ social, environmental, and economic impact. It is designed to showcase how institutions are working towards the United Nations’ Sustainable Development Goals (SDGs).
Australian universities have done well since the first release of this ranking in 2019; although there was a shake up in the standing of our universities in the 2022 edition. Out of the 25 Australian universities included in the ranking, 13 moved down in overall standing, seven moved up, two remain unchanged in position (or band), and three are new entrants.
The downturn in performance for Australian universities was significantly influenced by two key factors: new entrants to this ranking, which in many cases outperformed universities that have done well in previous years, and methodological adjustments which THE made that adversely impacted universities. Evidence from institutions which was previously scored generously was reviewed in greater depth by THE’s validation team.
My main criticism of this ranking is the volatility we have seen in overall standing year after year. To address this point, THE will now use a rolling average of the last two years’ scores for the overall ranking table. For institutions that are new to the ranking this year, THE uses only the 2023 score. This is a positive development and I commend THE for addressing this weakness.
QS World University Rankings
The 20th edition is due on 27 June and will feature an expanded and refined methodology.
As part of the enhancement features, it will include a couple of measures which were used in the now-defunct QS Employability Rankings. The first measure is focused on graduate employment and the other measure is about alumni outcomes. For the latter, QS has identified the alma maters of those included in 150 lists of highly successful individuals, each measuring desirable outcomes in a particular walk of life. In doing so, QS can establish which universities are producing world-changing graduates in a range of fields of endeavor.
There will also be a measure focused on sustainability, taken from the QS Sustainability Rankings. QS will add the International Research Network (IRN) measure, which was introduced this year in the subject rankings and two years ago at the faculty level. QS has also used the IRN measure in the regional rankings.
With these enhancements comes a change in the weighting for indicators such as academic reputation, the faculty-student ratio, and citations per faculty. The changes being introduced by QS are welcome and long overdue.
Last year’s edition of QS WUR included 24 Australian universities ranked in the top 500, of which 12 moved up in position and 12 moved down. The enhancement made by QS appears to be mostly favorable to Australian universities. We also need to keep in mind that over the past three years we have experienced weakening scores in the academic reputation survey. Last year, I noted that all Australian universities declined on the employer reputation survey.
Leiden Ranking
The fifteenth edition of the CWTS Leiden Ranking comes out on June 21. This ranking measures the scientific performance of universities worldwide, using a set of bibliometric indicators which include impact and collaboration. This ranking also includes several measures to reflect the extent to which institutions have embraced open access publishing. It has also included a set of measures designed to measure gender balance.
There were 32 Australian universities included in the 2022 edition, compared to 26 in 2019 and 23 in 2015. Over the years, Australian universities have done well in this ranking. Because Leiden does not produce an overall ranking for institutions, it offers the possibility that every Australian university can make a valid claim of success on any of its 24 indicators of scientific impact and collaboration, or in any of the open access publishing and gender diversity indicators.
ARWU
The Academic Ranking of World Universities (ARWU), is always out on August 15. In the early years, ARWU only published a ranking for the world’s top 500 but in recent years has expanded to 1000 universities.
Last year, I commented that the ARWU has achieved its key goal in measuring the relative progress of Chinese universities towards world-class status. I also urged them to rethink this, – it has an inherited bias in that it is all about measuring prestige on a narrow methodology. Since the ranking started, the only change made to its methodology is that of the list of Highly Cited Researchers (HiCis), produced by Clarivate.
In the 2022 edition, 24 Australian universities were included in the top 500 compared to 13 when the ranking began in 2003. It will be interesting to see how Australian universities perform this year, considering Australia’s share of HiCis decreased slightly between 2021 and 2022. There is not much change in the relative output of Australian universities in term of papers published in Nature and Science.
Over the years I have argued that the volatility that we see in ARWU is driven by the annual update to the list of HiCis, which has enabled institutions to game this ranking. The list of HiCis refers to those researchers who are in the world’s top 1 per cent of researchers based on citations over a ten-year period.
Recently it was it was reported that Rafael Luque, one of Spain’s most cited scientists, was suspended from his full-time employment at the Universidad de Cordoba because he was also working as a researcher at the King Saud University. Over the past four years, Luque listed King Saud listed as his primary affiliation instead of Cordoba.
This is not an isolated incident, but it serves to illustrate a growing concern – the extent to which universities and national systems with plentiful resources are gaming global rankings by enticing researchers to switch affiliations. It is unfortunate that the value of the HiCi assessment is progressively losing its relevance.
THE World University Rankings
THE is also introducing a new methodology and the 2024 edition comes out on 26 September.
THE is adding three measures to its controversial citations pillar, which previously relied entirely on the FWCI. These additions will reinforce what THE calls an institution’s research strength, research excellence, and research influence. In some ways these new measures lessen the oddity that institutions with limited research output could skyrocket into the top of the rankings thanks to inflated citations.
The other measure being introduced is assessing how often a university’s research is cited in patents. This measure will sit along the industry pillar. As a result of these four changes, the teaching and research environment pillars will have reduced weighting. Furthermore, THE is adopting a normalisation approach for the measures which form part of the international outlook pillar. This approach will consider the population of a country when assessing these metrics.
Last year, there were 31 Australian universities included in the world’s top 500, up by two from 2021.
QS Sustainability Rankings
The second edition of this ranking is due out on December 5. This is a ranking designed to showcase how universities are taking action to tackle the world’s greatest Environmental, Social, and Governance (ESG) challenges.
The key thing is that there is no opt-in in this ranking, as QS is able to produce it without direct submission from institutions. One shortcoming from last year’s edition was the absence of measures which relate to the governance part of the ESG framework. QS is introducing a set of measures to assess institutions through this lens.
QS will also be assessing institutions on environmental education: whether an institution offers courses that teach specifically on climate science or environmental sustainability, but also whether these courses lead to the award of an award degree. These additions are welcome and will strengthen this ranking relative usability across institutions.
Last year, there were 12 Australian universities ranked in the top 100.
Parting thoughts
Over the past twenty years, global rankings have been normalized as instruments of a quantitative performance measurement. I have often argued on the need to improve methodologies, to add context to make rankings meaningful, and for rankings to be performed regionally and seen within their relevant context rather than on a meaningless global scale.
It is also important to highlight that university ranking schemas are commercially driven enterprises and resonate globally. Over the years we have seen that global rankings have distorted national and institutional policy, but also reproduce and legitimise inequalities.
I would like to encourage readers to consider the views expressed by Tiffany Nassiri-Ansari and David McCoy in their paper World-class Universities? Interrogating the Biases and Coloniality of Global University Rankings. They call for a more critical appraisal of the dominant ideas, narratives, and perspectives in education and research that are propagated by the more powerful institutions of the global north.
Angel Calderon is Principal Adviser, Policy and Research at RMIT. He is a member of the advisory board to QS World University Rankings