A new report urges ExamSoft to change its features
Photo by: Cristiano Minichiello/AGF/Universal Images Group via Getty Images
Gabe Teninbaum, a professor at Suffolk University Law School, is calling on ExamSoft to fix a serious bug with its test-taking software: failure to recognize faces. Its a problem that can delay test takers or bar them from starting their exams altogether and per reports, it disproportionately impacts people with dark skin tones.
ExamSofts software records students while they complete remote exams and monitors for signs of academic dishonesty. Teninbaums report addresses an ExamSoft feature called ExamID, which aims to verify that test-takers are who they say they are. The first time a student logs into their exam portal, they upload a photo of themselves (their baseline image); theyre then prompted to take another selfie before beginning future exams, which the software checks against their original photo.
Researchhas found that facial-recognition algorithms consistently make more errors in identifying Black faces than they do white ones. And while those studies didnt focus on ExamSoft specifically, it doesnt appear to be an exception. Back in September, multiple non-white exam-takers told the New York Times that the software couldnt identify them due to poor lighting a problem that Teninbaum, who has light skin, wasnt able to replicate.
The @ExamSoft software can’t “recognize” me due to “poor lighting” even though I’m sitting in a well lit room. Starting to think it has nothing to do with lighting. Pretty sure we all predicted their facial recognition software wouldn’t work for people of color. @DiplomaPriv4All
Alivardi Khan (@uhreeb) September 8, 2020
@ExamSoft I FINALLY figured out ON MY OWN 4 hours later ( still on hold for an EXAM SOFT chat rep) HOW TO GET TO MY MOCK EXAM… ONLY TO BE TOLD THE SYSTEM CANT RECOGNIZE ME. I AM IN A BOARD ROOM WITH ALL OF THE LIGHTS TURNED ON AND THE SUN WAS STILL UP. MAKE IT MAKE SENSE. pic.twitter.com/7K3yKT1y4n
Futureesq1990 (@futureesq1990) September 17, 2020
Early this fall, Teninbaum set out to find a fix. He believes such errors add undue stress to an already stressful time period. These are students who are about to take a high-stakes exam with a lot on the line, and that is very unwelcome, Teninbaum said in an interview with The Verge.
Any time you go into an exam you just want to focus on the exam, he added. You dont want to feel like you have these added challenges.
Teninbaum also believes that optics matter; schools owe it to marginalized students not to rely on a category of software thats known to be discriminatory. Students deserve to feel that their institution is doing what it can to protect their rights, interests, and dignity, he says.
In his report, which is forthcoming in The Journal of Robotics, Artificial Intelligence, and Law, Teninbaum outlines the workaround he found.
He suggests that schools assign every student an identical generic, baseline image. Then, he proposes, they should ask ExamSoft to enable deferred identification, a feature built into the software that allows students to proceed with exams even if identification fails. This feature is basically hidden its not mentioned anywhere on ExamSofts website (at least, not that I could find). Only ExamSoft can turn it on.
In tandem, these tweaks will cause ExamSoft to misidentify every test-taker. But theyll still be able to proceed with their exams ExamSoft will send the selfies to the school afterwards, and instructors can manually verify everyone. We know who our students are, Teninbaum says. We can make sure the students are who they say they are and avoid subjecting students to these sorts of challenges.
He also suggests that ExamSoft make the deferred identification feature accessible to customers. The reporter urges ExamSoft to build this into a feature by which institutions can simply toggle on/off, thereby bypassing ExamID until such time that the technology matures into one that does not discriminate, his report reads.
Teninbaum hopes those changes will last beyond the COVID-19 pandemic, and can help students feel more comfortable taking remote classes. Its going to be a growing problem as people get more and more online for their education, he says.
Even so, hes only fixed part of the problem. Students have experienced a range of hiccups with ExamSofts proctoring software. Over 3,000 people who used the platform to take Californias bar exam in October had their videos flagged for potential rule violations nearly 36 percent of applicants who took the online exam. Users reported audio issues, and other technical glitches as well.
A group of six US senators including Richard Blumenthal, Elizabeth Warren, and Corey Booker wrote an open letter to ExamSoft in December, highlighting potential harms to students of color and students with disabilities, among numerous other concerns.
ExamSoft did not immediately respond to a request for comment.
