Designing and Validating Conceptual Framework for Non-cognitive Competencies Assessment

In contemporary education systems, the cultivation of non-cognitive skills and competencies has been prioritized, and their development is perceived as one of the most significant outcomes of educational processes. These competencies equip students to excel in social, emotional, and behavioral domains. The formal education system of the Islamic Republic of Iran (henceforth referred to as IRI), an ideological system rooted in religion, underscores that the development of non-cognitive competencies is as crucial as achieving academic success. According to the Fundamental Reform Document of Education (FRDE) in the IRI, students are expected to cultivate seven dimensions (religious, spiritual, physical, economic, affective, and aesthetic), and they are guided to enhance their relationships with themselves, God, others, and all creations. In alignment with these seven dimensions, the Upstream Documents are replete with educational objectives that extend beyond the realms of academic or cognitive categories. The attributes delineated in the Upstream Documents bear resemblance to those highlighted in Social Emotional Learning (SEL) (i.e., social-emotional and 21st-century skills/competencies) and character strengths and virtues. It is noteworthy that different conceptual models each have their unique advantages and limitations in the context of education. Despite the Upstream Documents assigning the responsibility of developing assessment models (result-based, process-based, and integrative) to educational leaders, a model specifically for non-cognitive educational goals has yet to be proposed. Furthermore, while non-cognitive competencies are frequently mentioned in the Upstream Documents, they have not been systematically examined and classified. The development of such competencies not only enhances students' academic achievement but also aids them in stress reduction, management of challenging situations, avoidance of risky behaviors, and the establishment of positive peer relationships. Research Questions: This study aimed to design a framework for assessing non-cognitive/non-academic achievement. The research questions explored were: What is the conceptual framework for non-cognitive achievement assessment based on the Upstream Documents, literature review, and educational experts, and is this framework valid?

The study was structured in two phases; the qualitative research phase involved the examination of three primary sources: 1) Upstream Documents of the Educational System of the Islamic Republic of Iran, 2) research literature, and 3) expert interviews. A total of three Upstream Documents and 24 literature documents were reviewed. The primary Upstream Documents included the Fundamental Reform Document of Education (FRDE), its theoretical foundation, and the Iranian National Curriculum. These documents were selected due to their integral role in the formal educational system of Iran, where all educational activities, from curriculum development to workplace policies, are expected to align with them. In relation to the second data source, databases were explored using keywords that reflected social-emotional skills, character strengths, and constructs stated in the Upstream Documents. The PRISMA diagram was utilized to organize the texts. Semi-structured interviews were conducted to gather data from field experts, with judgmental-purposeful sampling employed to select the participants. The sampling process continued until information saturation was achieved (n=10). In the quantitative phase, the framework was validated using a researcher-made questionnaire. The face validity, content validity (CVI), and reliability (Alpha) of the questionnaire were calculated and confirmed by the participation of n=30 experts.

The Summative Content Analysis, conducted using the Strauss & Corbin standard procedure via MAXQDA-2020 and the Colaizzi method, yielded four dimensions and thirteen categories. The identified dimensions encompass designing a strategic plan, collecting evidence/data, drawing conclusions, and determining consequences. The internal validity of the framework was confirmed through data analysis using a One-Sample T-test, which yielded a significant result (p < .05, DF = 29, t = 2.49). This indicates that the framework is statistically significant and reliable for assessing non-cognitive/non-academic achievement.

The study's findings indicate that, based on the three primary sources of information, several decisions need to be made to effectively assess non-cognitive competencies. The initial step in establishing an assessment system for non-cognitive competencies involves developing a strategic plan. Prior to initiating the assessment procedure, it is crucial to determine the approach to conceptualizing and instructing non-cognitive competencies, which will guide assessment activities. Decisions need to be made regarding which areas of interest are being integrated into the academic content or are planned to be taught through direct teaching methods. A range of non-cognitive competencies can be taught through the establishment of a positive learning environment and the implementation of general teaching activities. However, these critical issues have not been clearly addressed in the Upstream Documents. The second category, gathering data and evidence, encompasses practical assessment activities. Such practices can ensure reliable results. A key criterion to bear in mind is to avoid relying solely on a single tool or technique for data collection. While some of the methods mentioned above are found in the Upstream Documents, they are not specifically categorized in a way that teachers can confidently rely on to assess non-cognitive competencies. In the third category (conclusion), decisions should be made regarding data analysis and interpretation. Various forms of errors and biases should be considered before reaching a conclusion. The fourth category, consequences, pertains to the impact of assessment practices. Ideally, the results of the assessment should be used to enhance the quality of different levels of education systems. Policymakers and communities should consider the results of non-cognitive assessments. Important decisions could be made based on the results, such as where to target supportive resources to students or classrooms identified as expressing particular needs. The current status of Iran's education system in achieving its non-cognitive goals should be investigated, as it is unclear how successful the system has been in reaching these goals. The Organization for Educational Research and Planning (OERP), which is responsible for publishing and reviewing teachers' guidebooks, has the capacity to revise and enrich them. Non-cognitive competencies are multifaceted and should therefore be formulated and conceptualized precisely. Standards, criteria, and benchmarks should be established for assessing non-cognitive competencies. Initially, some critical decisions should be made regarding the instructional process. As non-cognitive competencies in teachers can directly impact their ability to teach these skills, universities responsible for teacher education, such as Farhangian and Shaheed Rajai Universities, should strive to enhance these competencies in teachers. Curriculum planning, implementing teaching, and assessment activities could be reformed based on the framework. Further discussions and suggestions were presented relevant to the educational system of Iran.