Complex issues surrounding the proctoring of online assessments

#RethinkingAssessment – The 2021 ICAI (@TweetCAI) Conference raised complex issues surrounding the proctoring of online assessments 

The International Center for Academic Integrity (ICAI) held its annual conference, which was fully remote this year, in early March.

A number of panels addressed the issue of proctoring online assessments, a phenomenon that has become widespread in the past year, as many colleges and universities have been closed to in-person education.

Online exams are unlikely to disappear completely even after the pandemic is over 

Although online assessments exploded in 2020, the complexities of online proctoring didn’t surface for the first time last year — online assessment has become more common in the past decade, alongside the growth in popularity of online degree programs.

Thus, commonly cited concerns about online proctoring, like its effects on individual privacy, aren’t new, though they have certainly become more pressing issues since the pandemic began. In addition, like many in-person activities that have moved to online modalities, online exams are unlikely to disappear completely even after the pandemic is over.

The topic of proctoring technology was addressed from various perspectives at the ICAI conference. Brian Clark, a supervisor of academic authenticity at Western Governors University (WGU), spoke about the importance and process of quality assurance for online proctored assessments.

Of course, as one of the largest online universities in the US, WGU has ample experience conducting these types of exams; it serves over 130,000 students and conducts over 200,000 online assessments per month. Thus, WGU has come up with a comprehensive method that Clark stated both assures “assessment security” as well as accounting for student satisfaction with the testing process.

Which behaviors and actions during an online exam need to be flagged?

Clark emphasized the importance of universities accurately assessing their testing volume in order to determine personnel needs, specifically whether they need to have dedicated staff reviewing online assessments. WGU has both dedicated proctoring staff, as well as partnering with a third-party provider.

It’s also crucial, Clark said, to determine which behaviors and actions during an online exam need to be flagged, such as cell phone usage or students speaking to other people in their physical environment — so working closely with the proctoring team is important, especially if proctors are third-party vendors, such as ProctorU.

Just as important, he added, is to assess methods in an ongoing manner to respond to comments and concerns by students and proctors that can identify areas of improvement.

Walking the line between academic integrity and privacy

A few other panels at the ICAI conference looked at the other side of online proctoring and assessment: concerns raised by students, such as the invasiveness of the process and possible unintended racial bias of the technology. Jennifer Lawrence and Kylie Day, of University of New England (UNE) in Australia, discussed, as their panel was entitled, “walking the line between academic integrity and privacy with online exams.”

Like WGU, UNE has a long history with distance learning — but their founding predates the internet. Founded in the 1950s, UNE has long catered to students living in remote regions of Australia. These days, 90% of UNE students are online, and most are women in their 30s who have children and/or jobs, and must conduct their learning in the evenings. UNE shifted to online exams in 2017, as it was a more flexible model that worked better for its student population.

Lawrence and Day reported that many UNE students were initially very anxious about e-proctoring, as they felt it was invasive for proctors to request to see different parts of the room (usually in their home) in which they were taking their exams.

While that aspect of online assessment is difficult to get around—as proctors must ensure that students don’t have access to unauthorized materials during exams—Lawrence and Day stated that students’ concerns about privacy often stem from misinformation, such as the idea that the software or proctor would continue to be able to monitor them from their computer after the exam was over or would collect their genetic information.

Does artificial intelligence determine the outcome of the exam?

There also tends to be a lot of anxiety, Lawrence said, about possible violations and whether artificial intelligence determines the outcome of the exam. For example, one student asked, “Will I fail the exam if my husband walks into the room?”

Once students are informed that UNE always has a human proctor involved in the process who is trained to determine whether the unusual behavior was a violation of academic integrity, students generally feel more reassured.

Lawrence and Day also stressed that exceptions to the rules for online exams can always be worked out — for example, if a student has a medical need to leave the room and go to the bathroom during the exam.

This is an issue Clark also brought up in his presentation: the need for flexibility with students. If students need to have small children with them during their exam, he said, the proctors are made aware of the situation, and it is not deemed a violation.

Unintended racial bias of proctoring technology

There were thornier issues related to online proctoring that were addressed at the ICAI conference. Ceceilia Parnther (St. John’s University) and Sarah Elaine Eaton (University of Calgary) presented on the impact of race in educational surveillance from the perspectives of students.

Parnther and Eaton have found that students of color tend to react differently to e-proctoring than their white counterparts, especially given what is known about algorithms that “view whiteness as normative,” Parnther said: “They view e-proctoring in many ways as a threat to mutual trust within academic communities,” and are distrustful of methods that resemble surveillance.

These students often come from communities that have been historically surveilled by government entities as a means of social control. As an example of this history, Parnther relayed that during slavery in the U.S. there were “lantern laws,” which forced Black, indigenous and mixed race people to carry lanterns to illuminate their faces and control their movement.

Students with dark skin who Parnther surveyed reported being asked during online assessments to lighten their rooms for the purposes of identifying them, or not being recognized because of a different hairstyle they were wearing.

Eaton shared a principle taught to her by an indigenous student she has worked with: “If the system is invisible to you, it’s because it was made for you. And if the system affects you, it’s because it wasn’t made for you.” This is what she thinks of when she hears from students that e-proctoring doesn’t bother them — they don’t see its effects because it was made for people who look like them.

Like with the students at UNE, Eaton and Parnther’s research with students of color found that they’re also concerned about what personal data is being collected and how long it’s going to be stored. But mostly they feel frustrated about having to take extra steps to authenticate themselves, particularly in a testing environment, which is already anxiety-provoking; they feel it’s an additional barrier to their success.

All in all, there are many complex issues related to proctoring online assessments.

While the ability to take exams remotely clearly removes accessibility barriers to many student populations, this method is still novel and inspires anxiety and fear among students.

It seems paramount that institutions clearly lay out the processes involved in e-proctoring and reassure students that AI won’t be making decisions regarding possible violations of academic integrity.

Higher education administrators should also be listening to the unique experiences of students of color with e-proctoring and solicit their input on how to improve processes in order to build trust.

Rebecca Bodenheimer is a freelance writer with a PhD from UC Berkeley

Rebecca publishes on a wide range of subjects, including education, and has written for Inside Higher Ed, CNN Opinion, LA Times, and many other media outlets.