New methodology addresses “contentious” student satisfaction surveys

Adopting a new methodology approach could address concerns about student satisfaction surveys, according to new research.

A group of economics and social sciences experts has come up with a ‘fuzzy index’ approach that they claim is a game-changer and could be applied to student surveys across the world.

The research, carried out by the University of Stirling, Swansea University, the University of East Anglia and Technische Universität München, follows questions over the revamped National Student Survey (NSS) in the UK.

The popular NSS, introduced in 2005 and with almost 500,000 annual respondents, has sparked intense debate in the higher education sector whilst increasing in its influence. The survey recently eliminated the final ‘overall satisfaction’ question for Higher Education Institutions (HEIs) in England, although the question remains for HEIs in the rest of the UK.

A contentious step, it has been opposed by many in the sector, who argue that eliminating the overall satisfaction question means students, particularly international ones, will find it difficult to compare universities UK-wide.

Focusing on the overall satisfaction question, the researchers used NSS survey information and Higher Education Statistics Agency to develop and test their index measurement. 

They based their approach on a ‘fuzzy poverty’ theory used to define deprivation; rather than drawing a line at an income level and declaring everyone below it a poor, personal circumstances such as education, healthcare and housing are built into the model. In the same way, applying the ‘fuzzy’ approach to NSS takes into account factors such as quality of teaching resource availability, and opportunities for personal development.

In applying their methodology for the report, the researchers found that people who had graduated from a state school were more likely to give a more positive satisfaction score. It also found that using non-permanent, teacher-focused contracts decreased dissatisfaction rates.  

Professor Robert Webb of the University of Stirling Management School and one of the report’s co-authors said:

“Student satisfaction and league tables are a prominent and often contentious subject, both for the higher education sector and the public. That’s why this study could be a game-changer. The methodology may be ‘fuzzy’ but the result couldn’t be clearer. We have developed a robust, accessible and easily replicated index that not only provides a solution for NSS but can be universally applied in other surveys around the world.”

He added: “Our findings on the negative correlation between state school graduates and dissatisfaction scores, and the effect teaching contracts has on dissatisfaction rates, are important factors for all HEIs, and the sector as a whole.”

• Student dissatisfaction in Higher Education: a ‘fuzzy’ index approach by Steve Cook, Duncan Watson, Asha Webb and Robert Webb is published in Studies in Higher Education.