Properties of Single-Response and Double-Response Multiple-Choice Grammar Items

The purpose of the present study is to compare the psychometric qualities of canonical single-response multiple-choice items with their double-response counterparts. Thirty, two-response four-option grammar items for undergraduate students of English were constructed. A second version of the test was constructed by replacing one of the correct replies with a distracter. The two test forms were analysed by means of the Rasch model. To score double-response items, two scoring procedures, dichotomous and polytomous, were applied and compared. Results showed that dichotomously scored double-response items were significantly harder than their single-response counterparts.Double-response itemshad equal reliability to the single-response items and had a better fit to the Rasch model. Principal component analysis of residuals showed that double-response multiple-choice items are closer to the unidimensionality principle of the Rasch model which can be the result of minimized guessing effect in double-response items.Polytomously scored double-response items, however, were easier than single-response items but substantially more reliable. Findings indicated that polytomous scoring of double-response items without the application of correction for guessing formulas results in the increase of misfitting or unscalable persons.