Truthful Peer Grading with Limited Effort from Teaching Staff
2019/04/02
研討會日期 : 2019-04-02
時間 : 10:30
主講人 : Professor Anujit Chakraborty
主持人 : Professor Ritesh Jain
地點 : Conference Room B110
演講者簡介 : Professor Chakraborty received his Ph.D. in Economics from University of British Columbia in 2017. He is currently an Assistant Professor at University of California, Davis. His research fields are Economic Theory, Behavioral Economics, and Experimental Economics.
演講摘要 : Massive open online courses pose a massive challenge for grading the answer scripts at a high accuracy. Peer grading is often viewed as a scalable solution to this challenge, which largely depends on the altruism of the peer graders. Some approaches in the literature treat peer grading as a best-effort service of the graders, and statistically correct their inaccuracies before awarding the final scores, but ignore graders’ strategic behavior. Few other approaches incentivize non-manipulative actions of the peer graders but do not make use of certain additional information that is potentially available in a peer grading setting, e.g., the true grade can eventually be observed at an additional cost. In this paper, we use such additional information and introduce a mechanism, TRUPEQA, that (a) uses a constant number of instructor-graded answer scripts to quantitatively measure the accuracies of the peer graders and corrects the scores accordingly, (b) ensures truthful revelation of their observed grades, (c) penalizes manipulation, but not inaccuracy, and (d) reduces the total cost of arriving at the true grades, i.e., the additional person-hours of the teaching staff. We show that this mechanism outperforms several standard peer grading techniques used in practice, even at times when the graders are non-manipulative. We also run experiments to test the limits of existing peer-grading methods.
