It’s entirely possible to create a university ranking based on off-the-shelf equity measures. But when researchers did it for Australian universities they found flaws

Tim Pitman (Curtin U) and colleagues from Australian Council for Educational Research created an equity ranking for Australian higher education and then scored universities on it. Some of the results put elite universities in top spots.

What they did: The team sourced data on six HE equity-measures, aspiration, academic preparation, access-participation, experience in first-year of study, progress through course and graduate outcomes and built a model to weight them. They then assessed 37 Australian universities according to the performance of low SES students.

What they found: Outcomes depended on how indicators were weighted. When they were all measured equally five of the top ten were Group of Eight institutions. “They achieved a high ranking due to their superior performance in the retention, completion and graduate outcomes of their equity students,” the authors explain.

When participation was scored higher the ranking was more predictable, with regional universities in the top seven places, although when there is not much between scores, ranking can “magnify or understate” performance.”  For example, on retention and completion a 3.5 per cent performance gap between two institutions meant they were 12 places apart.

And the data itself can create an accurate but misleading ranking; “institutions enrolling relatively few, but high-achieving, equity students could be perceived as performing better than institutions enrolling many more, but lower-achieving, equity-group students.”

What it means: If you want a ranking that demonstrates what is actually going on, it must be custom-built rather than use off-the-shelf data, the authors suggest.

“There are multiple ways in which a university can achieve higher education equity depending on institutional profile, historical legacy and the overall policy environment. Thus, while a single rankings index has the advantage of focusing stakeholder attention, the complexity of measuring institutional performance in regard to higher education equity may be better understood through the complementary use of a wider range of indicators.”

Tim Pitman, Daniel Edwards, Liang-Cheng Zhang, Paul Koshy, Julie McMillan,  “Constructing a ranking of higher education institutions based on equity: is it possible or desirable?,” Higher Education, January 2020