Monologic vs. Dialogic Assessment of Speech Act Performance: Role of Nonnative L2 Teacher's Professional Experience on Their Rating Criteria

Few, if any, studies have investigated the effect of professional experience as a rater variable and type of assessment as a task variable on raters’ criteria in the assessment of speech acts. This study aimed to explore the impact of nonnative teachers’ professional experience on the use of criteria in monologic and dialogic assessment of 12 role-plays of 3 apology speech acts. To this end, 60 raters were divided into 2 subgroups of raters with under and over 5 years of professional experience and rated the role-plays monologically and dialogically. A content analysis of the raters’ descriptions of the ratings showed 3 groups of criteria: the general criterion (appropriateness), pragmalinguistic criteria (linguistic features, L1 effect, paralinguistic features, directness, and adequacy), and sociopragmatic criteria (politeness, repair, truthfulness, promise, thanking, reasoning, personal trait formality, genuineness, and expression of apology). We also discovered that neither the more experienced nor the less experienced raters paid due attention to the sociopragmatic criteria in the monologic and dialogic ratings of pragmatic performances. Both groups of raters based their ratings primarily on the general criterion of appropriateness in the dialogic ratings. However, in the monologic ratings, the more experienced ones preferred pragmalinguistic criteria, and the less experienced ones opted for the appropriateness criterion. An analysis of the influence of the type of rating on the raters’ application of criteria showed that the raters differed in the use of all the 3 groups of criteria in the monologic ratings, whereas in the dialogic ratings, their difference in the application of criteria narrowed down to the sociopragmatic criteria. The findings have implications for teacher education programs on pragmatic assessment, urge considerations for the role of teachers’ experience in pragmatic assessment, and stress the inclusion of dialogic ratings in the assessment of speech acts for improving the quality of raters’ assessments.