“Others have scored the same as me; do you want to change it”? Exploring rater dominance in negotiation rating sessions of EFL writing

Rater negotiation is a score-resolution method through which raters review and discuss performance samples to resolve rating discrepancies. The success of this method depends on raters getting equally engaged in negotiations. This study explored whether novice raters remain equally engaged in negotiations or rater dominance occurs. Eleven English teachers attended eight negotiation sessions. They scored ten writing samples independently using the IELTS rubric and then discussed rating discrepancies in groups. It has employed a mixed-methods approach to see whether any traces of rater dominance are observed or raters are equally engaged in negotiations. The chi-square test results for score changes indicated that only in Task Response category, raters were inequitably engaged. No dominance was observed for other dimensions. However, qualitative analysis of the negotiations revealed various patterns of rater dominance. Furthermore, the analysis of rater interactions in negotiation sessions indicated that rater dominance is a nonlinear construct demonstrated in interactions of raters during negotiation rating sessions. The findings illuminated that while some raters attempted to scaffold each other to form a unified understanding of scoring rubric by sharing the floor in discussion sessions, some tried to dominate other raters. The findings highlight the utility of negotiation, not just as a resolution method but a procedure with training effects for performance assessment in EFL contexts where access to expert raters is usually limited.