A case study of peer-rater differences in Hong Kong
Li, T.
Abstract:
Assessing student performance in higher education has never been easy. Pedagogy to engage students in learning and make them own their learning has become increasingly popular. The paradigm of teaching has switched from teacher-centered to student-centered. Likewise, students’ involvement in assessments (self- and peer-assessments) is becoming more common.
Self- and peer-assessment have widely been researched and evidenced in enhancing and motivating student learning. Differences in assessment results between peers and instructor have been found insignificant. However, differences among peers have not been studied much.
The purpose of this paper is to investigate differences among peer raters. Do friends rate more leniently and not-so-friendly peers rate more stringently?
Data were collected from a business communication course delivered at a Hong Kong private higher education institute for fall 2013. The course had 108 students in 3 sections, and 28 groups were formed to work on a case study. Students were required to orally present their case studies. Presentation was graded by both peer-rater groups and the instructor.
For inter-rater differences, 22 of the 28 groups were rated significantly differently (P < 0.05) by their peers. However, the Cronbach’s alpha for all groups (n = 28) is 0.952 indicating high inter-rater reliability. When the 28 groups were analyzed separately, a total of 19 (67.86%) groups had Cronbach’s alpha below the benchmark 0.7 reliability threshold, but only three (10.71%) of them were significant (p < 0.05). Among the 94 anonymous questionnaires collected from students after the process, 18 students (19.15%) admitted they had awarded more marks to friendly groups; and 10 (10.64%) had awarded less marks to unfriendly groups. Six students have both awarded more and less marks to peers due to personal relationship. Discriminatory rating by individual students existed, but it was uncertain if final grades were affected.
Seventy-one (75.53%) students agreed that observing other students’ presentation helped, and sixty-seven (71.28%) students agreed that the exercise trained their critical thinking skills.
Peer-assessment not only enhances and motivates students’ learning, but also makes evaluation of student performance more transparent. This may reduce grade dispute between teacher and students. To reduce impact on final grade from favoritism and/or hostile grading among peers, instructors may consider removing the highest/lowest marks from peer-rater groups and reducing the weight of peer-assessment on the final grade. What is more important is what students learn from the process.
Keywords: Peer-assessment, discriminatory grading, peer raters
LINK
APA citation
- Li, T. (2014). A case study of peer-rater differences in Hong Kong. International Journal of Teaching and Education, 2(3), 112-126.