Reliability of Electrical Power Engineering PhD Admission Criteria's in Iran: Based on Generalizability Theory

The assessment process for the admission of qualified candidates applying for doctoral programs in Iran has always been a challenging issue, leading to debates among academic communities about the psychometric properties of the admission criteria. Using the generalizability theory as the framework, this study investigated the reliability-like coefficients of the secondary data measures and the criteria related to the Ph.D. admission program at 37 departments admitting Electrical Power Engineering candidates at the Ph.D. level in different state universities in 2018. Data were analyzed using mGENOVA software with onefacet 𝑝 • × 𝑖 °design. The results suggested that due to the high difficulty of items, omitted responses, and using scoring formula, four Subject Test Batteries (STB) and two General Test Batteries (GTB) did not have adequate generalizability and dependability coefficients. Two interview scores, which included different measures, showed better coefficients. However, little predictive value was found for the candidates’ undergraduate grade-point average (UGPA) and graduate grade-point average (GGPA) due to their restricted range. Indeed, broadly defining the underlying construct and including multiple measures in the assessment design for doctoral programs leads to more error components in the results. Therefore, it is not possible to determine an exclusive reliability index for the predictive measures involved in the test. However, high-stakes educational decisions for the classification of applicants may be accomplished with fewer false predictions by adjusting test difficulty, using partial scoring schemes without correction for guessing, specifying the underlying construct more thoroughly, and increasing the levels of the facets.