Comparison of the Multilevel Bayesian IRT Model and the Multilevel Classic Model: An Analysis of Advanced TIMSS Physics Test (2008)

Validity of the multilevel analyses with a focus on differences in learning theories (with both classis approach toward measurement and new approach toward measurement (IRT)) by means of various data has recently been studied. This study is aimed at determining the level of impact of schools on the scores gained by students involved in advanced TIMSS’s 2008 test. In order to achieve the chief goal of the study, the researchers adopted the data relating to administration of advanced TIMSS’s 2008 physics test, which assesses the course of teaching advanced physics to the high school seniors (pre-university students). The population and sample group of this study comprise the pre-university candidates of mathematics and physics taking advanced TIMSS physics test administered in the academic year 2007-2008. The sample size of Iranian students involved in this measurement equaled 2556 subjects. Results of the analyses generally suggest that firstly multilevel IRT (MLIRT) analyses are more powerful than multilevel true scores (MLTS) analyses in clarifying school differences. Secondly, the level of difference in the schools involved in advanced TIMSS math test reflected in intraclass correlation (ICC) is high in MLIRT analyses. Discussion and First, considering measurement error of each item in analyses within Bayesian framework and by means of Gibbs sampling can remarkably improve the power of multilevel analyses and lead to a significant rise in the ratio of the explicated variance. Second, there is too much educational difference and discrimination among schools which is largely due to school-level variables (such as those relating to teacher or school-related variables).