Researchers at Harvard University and the Massachusetts Institute of Technology have discovered a new form of cheating in massive open online courses that they say poses a “serious threat to the trustworthiness of MOOC certification.”
According to a working paper released Monday, learners have found a fittingly massive way of cheating their way to a good grade in a MOOC. A single MOOC can enroll tens if not hundreds of thousands of learners at once, and by creating multiple accounts, a single learner can “harvest” correct answers that they can use to earn a perfect score and a certificate showing their mastery of the topic.
Curtis G. Northcutt, Andrew D. Ho and Isaac L. Chuang, the researchers behind the working paper, have dubbed the cheating strategy CAMEO, short for “Copying Answers using Multiple Existences Online.”
Chuang, senior associate dean of digital learning at MIT, described the CAMEO strategy as “endemic to MOOC, and not something you can have in a brick-and-mortar classroom.” Accounts that use the strategy follow a “Dr. Jekyll and Mr. Hyde” pattern, he said. One account gets everything wrong; the other, everything right.
The researchers used click-stream data -- the record of what MOOC learners do while logged in -- to discover the new cheating method. To find connections between “harvester” and “master” accounts, they searched for instances where one account took a quiz and requested to see the answers followed by another in the same location answering correctly. To further refine their results, the researchers excluded locations where multiple users could be legitimately working on the same MOOC, such as Internet cafés and classrooms.
“Indeed, when you see this interweaving of one account from an Internet location getting a bunch of answers wrong, checking the solutions, then immediately followed by another account from the same location submitting the correct answers, you start to get a little suspicious,” said Ho, a professor at the Harvard Graduate School of Education.
Based on an analysis of 115 MOOCs offered by Harvard and MIT between fall 2012 and June 2015, 1,237 certificates, or 1 percent of the total certificates awarded, were earned by learners using the CAMEO strategy. The estimate is a “conservative” one, the researchers said, pointing out that their study doesn’t capture other forms of cheating.
The prevalence of cheating varied by discipline. Only 0.1 percent of the certificates awarded in computer science courses -- one of the most popular categories of MOOCs -- were earned using the CAMEO strategy, while courses in the government, health and social science category, at 1.3 percent, had the highest rate.
The strategy is mostly used by learners outside the U.S. While only 0.4 percent of the learners in the U.S. earned their certificates by using multiple accounts, 12 percent of learners in Albania did. Indonesia, Serbia, Colombia and China rounded out the list of the top five countries with the highest rates, registering at between 4 and 2 percent.
The findings “severely undermine” the potential of MOOCs to deliver high-quality education to and evaluate the work of thousands of learners at once, the paper reads, adding, “It is less cheating than the wholesale falsification of a certificate.”
MOOC platforms already have features in place to prevent this kind of cheating, the researchers said. For example, some MOOCs don’t allow learners to see the correct response to a quiz until after the assignment is due, while others generate random questions. In a subset of MOOCs with those features, the researchers found a minuscule 0.1 percent of certificates awarded to learners using the CAMEO strategy, compared to 1.2 percent in courses that lacked them.
Those features are not without flaws, however. MOOC instructors who decide to withhold the correct answer until after the deadline are forced to balance “trying to provide as much feedback as possible and protecting against people who might abuse that feedback to fake a certificate,” Ho said.
In a statement, a spokesperson for edX, which was co-founded by Harvard and MIT, said the MOOC provider has already launched those and other features to prevent cheating. “Perhaps most importantly, we have also introduced virtual proctoring to monitor student activity, a scalable and convenient method of preventing this and other forms of cheating in MOOCs,” the spokesperson said.
Coursera, another MOOC provider, declined to comment.
MOOC platforms already ban users from creating multiple accounts, but Chuang said the solution to stamping out the CAMEO strategy is not as simple as increasing enforcement of that particular term of service. There are legitimate uses of multiple accounts, he said, such as teachers creating one account for use in the classroom, a second for personal use and a third for learning with their children.
“If you disallow pseudonyms, it may take away those approaches,” Chuang said.
https://www.insidehighered.com/news/2015/08/26/harvard-mit-researchers-find-mooc-learners-using-multiple-accounts-cheat
No comments:
Post a Comment