فصلنامه مطالعات اندازه گیری و ارزشیابی آموزشی
پیاپی 32 (زمستان 1399)

The various successes of organizations are related to knowledge management in terms of their ability to provide services. Therefore, examining the relationship between knowledge management and organizational effectiveness is very important. Researches have shown that, in general, knowledge management has a significant positive relationship with organizational effectiveness, but the extent of this relationship varies based on different studies. The purpose of this study was to determine the components of knowledge management firslty, and then to examine their impact on organizational effectiveness.

For these purposes, mixed methodology was used with the priority of qualitative method over quantitative one. In the qualitative stage, first based on theoretical and experimental background, the components of knowledge management were identified, and then with the help of 11 expert evaluators by performing three stages of Delphi panel, the degree of agreement on the components with Kendall coefficient (W) was determined. In the quantitative stage, based on the results of the qualitative part, a knowledge management questionnaire was prepared and distributed along with the organizational effectiveness questionnaire among 293 people who were selected by the convenince sampling and data were collected. The construct validity of both questionnaires was assessed by confirmatory factor analysis. In both questionnaires, the fit of the model with the data was acceptable.

The Cronbach’s alpha of knowledge management and organizational effectiveness questionnaires were 0.96 and 0.93, respectively. Indicators of knowledge management construct, with respct to the impact of this construct on them respectively are: knowledge application, knowledge internalization, knowledge externalization, knowledge sharing, knowledge creation and knowledge retention. Indicators of organizational effectiveness structure, with respect to the impact of this construct on them respectively are: employee performance, operational efficiency, client's mental image, innovation, profitability and service development. Structural equation modeling (SEM) and path analysis were used to analyze the quantitative data. The results revealed that knowledge management positively influenced organizational effectiveness, such that about %7.6 of its variance is explained by knowledge management, which is considered as the average effect size. Among the knowledge management components, only knowledge creation, knowledge retention, and knowledge application positively influenced organizational effectiveness. According to the results of path analysis among the components of knowledge management, only knowledge creation and knowledge retention have a positive effect on the organizational effectiveness variable (i.e. total organizational effectiveness score) (P <0.05), in the presence of these two variables other components have a significant effect, and they have no effect on the total score (P> 0.05). Together, these two variables accounted for about 20% of the effective variance (Table 10). The results of path analysis regarding the effect of different components of knowledge management on different components of organizational effectiveness showed that only three components of service development, profitability and innovation of organizational effectiveness are related to different components of knowledge management. In the service development component, only knowledge creation, knowledge retention and knowledge application have a positive and significant effect on it, so that about 41% of the variance of the service development component (dependent variable) by the components of knowledge creation, knowledge retention and knowledge application (independent variables) is explained. In the equity component, only the effect of the components of knowledge creation and application of knowledge is significant, so that about 32% of the variance of the profitability component is explained. In the innovation component, only the effect of the components of knowledge creation and knowledge retention is significant, so that about 18% of the variance of the innovation component is explained. There is a significant relationship between knowledge management effectiveness and organizational effectiveness in various studies.

It can be concluded that knowledge creation, retention and application need to be emphasized in achieving organizational effectiveness. Long-term planning to further enhance the effectiveness of knowledge creation functions; improving and strengthening programs for the effectiveness of knowledge storage functions; improving and strengthening programs for the effectiveness of knowledge sharing functions; long-term planning to further enhance the effectiveness of knowledge utilization functions; long-term planning to further enhance the effectiveness of knowledge internalization functions and long-term planning to further enhance the effectiveness of knowledge externalization functions are areas that need to be emphasized.

Developing qualified researchers is one of the principal functions of higher education systems in order for the mission of knowledge production and dissemination to be accomplished. To this end, research methods are taught at all levels of higher education systems across a wide range of disciplinary contexts. Honing research skills is a key to educational and career success of students, making them trained information consumers. Hence, developing a relevant, well-defined research method curriculum has become a global concern of higher education systems. Despite its importance, there is a dearth of literature on education and curriculum of research methods, calling for more scientific efforts on the pedagogy of research methods. Therefore, in this study, we examined elements of research methods curricula of graduate programs in the field of education. Particularly, we reviewed the research methods syllabi of graduate programs in the field of education of 10 out of 100 top universities of the world in terms of goals, contents, teaching methods, learning activities and assessment methods. Research questions Which goals do the top universities seek from teaching research methods in graduate programs in the field of education? What are the contents of research methods curricula in graduate programs in the field of education at the selected top universities? Which teaching methods are used in the selected universities to teach research methods in graduate programs in the field of education? Which learning activities are involved in research methods in graduate programs in the field of education at the selected universities? Which assessment methods the selected universities use in research methods in graduate programs in the field of education? Methods We followed a qualitative approach. Implementing content analysis, we analysed several syllabi of research methods of graduate programs at the world top universities. Using purposive, intensity sampling method, we selected 10 universities out of 100 top universities of the world ranked based on Shanghi university ranking. The syllabuses of the most recent courses of research methods in the field of education were collected from the websites of the universities. This resulted in a pool of 13 syllabuses dated from spring semester of 2010 to the fall semester of 2019.  We then analyzed the main curriculum elements of the syllabuses; i.e., the goal, content, teaching method, learning activities, and assessment method.        Results The most important goals of teaching research method at the selected universities were learning about various research methods in educational research and reviewing the pros and cons of each method; improving research procedural skills including identifying and formulating research questions; enhancing the capability to read and review the literature; promoting the understanding of philosophical foundations of research; improving the skills of planning and doing the research; and enabling students to correct their mistakes in their research projects throughout the course. As with content, the quantitative research methods curricula contained identifying the problem and research questions; research design; research variables; reviewing literature and theoretical frameworks; various types of quantitative research methods; measurement instrument; sampling; experimental design; data collection and analysis; descriptive and inferential statistics; and writing research report. Identifying research problem and questions; different types of quantitative research methods; measurement instruments; and data collection and analysis were the most important topics. Qualitative research methods content included characteristics of qualitative research; types of qualitative methods; research questions; research design; sampling; data collect tools; data analysis and research report among which qualitative data analysis and different types of qualitative research methods were of paramount importance. The syllabuses shared several common subjects regarding both qualitative and quantitative approaches including various classifications of methods and approaches to research; research principles and process; philosophical and theoretical foundations of research; and research ethics, the two latter being the most important issues. The selected universities used combinations of various teaching methods, among which problem-based and project-based learning were the most prevalent method to teach research methods. The curricula involved multiple types of learning activities such as preparing research proposal during semester; assessing and criticizing published articles; team-based and individual-based research projects; data analysis and reporting research findings; writing up an academic article based on the class project; and submitting weekly assignments with respect to various topics including effect size, random sampling, autobiography, obtaining research permission from university ethic committee and undertaking self-assessment. The most important topics were writing a research proposal and reviewing prior studies, respectively. Although the selected universities used mixtures of methods for assessment purposes, the most-emphasized criterion was the quality of the submitted research proposal. Discussion Learning research methods to be able to undertake a research is an all-important educational outcome for students of any discipline including education. Pedagogy developers and teachers should pay heed to the synchrony of all curriculum elements. Taking a balanced approach towards qualitative and quantitative research methods is a must to develop a high quality research methods curriculum. Regarding quantitative research method content, curriculum developers need pay more attention to the subjects of research problem identification; experimental and quasi-experimental research designs; validity and reliability of measurement instrument; and methods of data analysis. As with qualitative approach, providing a deeper understanding of the qualitative research methods; what distinguishes the methods from each other; and their applications in real world should be a vital concern of both planners and teachers. Philosophical and theoretical foundations of research and research ethics are also very important to be covered. Furthermore, active teaching methods are recommended to engage students in real research projects. Finally, process-based evaluation methods, e.g. assessing a research proposal submitted by student, critical contemplation on existing studies, etc., are advised in order to encourage learning.

In the 21st century, the importance of the higher education system in any society has become so critical that, according to Malcolm Gillis, today the fact that a nation be rich or poor greatly depends on the quality of higher education. But the hasty quantitative-oriented approach caused it to remain only within the limits of directives and regulations, and the qualitative dimensions of education were considered less. In the meantime, paying attention to outputs- which is equivalent to outcome, yield and results- and evaluation became more important because evaluation is part of the learning process, and the three streams of training, learning, and evaluation work together. Accodingly in 1985, Rabart Gagne formulated the theory of education based on cognitive principles which includes the learning conditions or the circumstances in which learning takes place. There are two crucial steps in applying this theory: The first step is to determine the type of learning output. Gagne has determined the five main outputs. The second step is to identify learning events or factors that make a difference in learning. In defective educational systems, the appraisal criterion is basically observable behavior and large-scale classroom exams with the use of achievement tests,and the goal is passing semesters and graduation,and learning actually does not take place.Consequently, when evaluation is in the service of learning, the learner has a deep understanding of his/her performance and evaluates his/her learning continuously and considers his/her effort-and not random or external factors-as the factor of his/her success. This issue and its importance is studied and discussed by many researchers, e.g.(Klein et al.,2018); (Melgozzo et al.,2017); (Sumble,2016); (Havnes et al.,2016); (Klimova,2014); and (Naseri ,2017). By examining their findings and considering the problems and obstacles, and insufficiancy of facilities in universities, including lack of appropriate and different tools in evaluating the learning outputs of students, especially in the field of humanities, in which the theoretical aspect, score and exam sheet is the criterion for evaluation, the researcher faced this question: What are the dimensions, components and evaluation indicators of the students’ learning outputs? Research

The present study examined the "analysis of the lived experience of university professors from evaluating the students' learning outputs".The research purpose is practical and research field includes all professors in the field of educational science in universities of Tehran province. In the qualitative stage, the snowball sampling method was used. Also, by continuously comparing the data until reaching the theoretical saturation, a semi-structured interview was conducted with 10professors.In the quantitative stage, using the Cochran's formula,193 out of 387of these professors were surveyed by simple random sampling. The tools for collecting data at qualitative stage are semi-structured interviews, field studies and library studies, and in the quantitave stage, a researcher-made questionnaire based on the five-scale Likert was designed to evaluate qualitative data. To examine the content validity, triangulation of data sources was used and professors who were interviewed as well as advisors and readers were also surveyed, and with the help of a questionnaire measured with Cronbach's alpha tool, the idea of other professors was applied to assess the reliability. In terms of method and manner of implementation, the research is a mixed (exploratory) method. In the qualitative stage, the phenomenological method based on Van Manen’s seven-scale model was used, and in the quantitative stage, to validate the model, the Confirmatory Factor Analysis and the Structural Equation Model were used. Data normality was assessed by K-S test and components were assessed by one-sample T-test. Statistical analyzes were performed by SPSS and Lisrel softwares, and the final model was extracted. Research

Here, the research questions are answered in two stages, qualitatively and quantitatively. General Question: What are the dimensions, components, and evaluation indicators of students’ learning outputs? First, to do the analysis, the whole transcripts of 10 interviews and document surveys between 2012 and 2020 were divided into 1405 initial codes, and after several screenings and continuous comparisons, 320 open codes were recorded.To follow the inductive path, the initial codes were classified into 87 sub-categories and 17 main categories (dimensions) and were subdivided into 5 constructs. Question 1-What are the characteristics of a learning output evaluation model? Constructs, dimensions, categories and indicators after review by professors were categorized as follows.  Table (1):5 extracted constructs and subdivisions No. of Indicators with Higher Priority According to Professors' Survey No. of Dimenstions No. of Catatories No. of Indicators Constructs 60 indicators 134 38 6 The nature of evaluation 10 indicators 28 9 3 The nature of the curriculum 45 indicators 70 17 3 Evaluator (professor) 28 indicators 51 14 3 The nature of learning outcomes 26 indicators 37 8 2 Evaluatee (student)   Question 2-What is the model for evaluating learning outputs? From the results of the qualitative part, a model was extracted indicating the evaluation of learning outputs is surrounded by 5 constructs, and each construct is connected to a number of dimensions, categories and indicators. Question 3-To what extent is the mentioned model valid? In the quantitative step to assess the validity of the model extracted in the qualitative part,at first by using central indicators, the scattering of variables were described, the normality of the data was checked, and the correlation matrix with 5constructs was defined as the input for Lisrel software. Factor Analysis Test was performed on (components of) each construct. As a result of standard parameter estimates, Chi-Square and RMSEA indices showed that the output of pattern fitting indices is statistically significant in each construct and all factor loads were accepted and prioritized due to being at a high level.The amount of Chi-Square in the degree of freedom of the models and the values of one-sample T-test showed that direct relationships exist. Therefore, the null hypothesis was rejected with 95% confidence for these components, and the research hypothesis was confirmed.

Finally,according to the prevalent conditions of Iranian higher education system and the lived experience of university professors, there was an attempt to design a model that depicts the evaluation of learning outputs in 5 constructs, 17 dimensions, 87 categories and 320 indicators that are all related. According to the professors, paying attention to all of them will have a positive effect on the evaluation of learning outputs since in the evaluation process, the role of the student, the professor, and the curriculum are influential, and disregarding each of them can change the output.These findings are consistent with the results of studies by (Klein et al., 2018); (Melgozzo et al., 2017); (Coates, 2016); (Paulini, 2015); (Alkuran,2012); and (Sufizadeh,2015). Therefore, we hope to see the improvement of the quality and effectiveness of the national higher education system by using the results of this study and other related researches.

Today, corruption has become a serious issue in the higher education system, and thus different researches have studied the issue. But what can be seen in these studies is that there is a lack of clear and acceptable definition of indicators and components of academic corruption among researchers. This issue has gone so far that even the definition of corruption depends on the social and cultural context. Thus, it seemed necessary to design a questionnaire to measure university corruption in Iran according to its context.

The purpose of this study was to determine the components of the Academic Corruption Questionnaire in the faculty members of Iranian higher education. In addition, the current research sought to assess the validity and reliability of the university corruption scale.

  The method of the present study was correlation research based on factor analysis. The population of the study included faculty members of the University of Tehran (Faculties of Educational Sciences and Basic Sciences), Allameh Tabataba’i University (Faculties of Psychology and Educational Sciences and Social Sciences), Amirkabir University of Technology (Electrical Engineering and Medical Engineering), Semnan University (Faculty of Humanities and Psychology Educational) and University of Kurdistan (Faculties of Humanities and Basic Sciences). In total, the research population was 420 faculty members. By using stratified random sampling method, a total of 200 faculty members were selected as the sample based on the Cochran's formula. The method of data collection in the research was field study. Namely, the questionnaires used were distributed among the faculty members and after completing and returning the questionnaires, the necessary data were extracted from them. The measurement tool in the present study was a researcher-made academic corruption questionnaire. The questions were designed in five ranges 1 (strongly disagree) to 5 (strongly agree). The questionnaire was given to 5 professors and experts of higher education, and they confirmed the content validity of the questionnaire and the appropriateness of the questionnaire items. Then the questionnaire was tested on 30 sample people and the problems related to translation were solved. In this study, Cronbach's alpha coefficient in the initial implementation was 0.90.

Factor analysis by principal component analysis and orthogonal rotation (varimax) was conducted and the analysis showed 4 dimensions included educational, research, communication and administrative-managerial factors. A total Cronbach's alpha coefficient was 0.88. And for other factors were calculated 0.82, 0.79, 0.89 and 0.72, respectively. In addition, confirmatory factor analysis using fit indices in 4 factors has been confirmed. Finally, this scale of academic corruption can be used as a valid and reliable tool in measuring the academic corruption of the higher education community. 

  Findings from exploratory factor analysis showed that all items of academic corruption are loaded in four components: academic corruption, educational corruption, research corruption, communication corruption and managerial-administrative corruption. The analysis of the materials showed that this scale has appropriate psychometric properties and all its materials effectively participate in the academic corruption scale.

Quality assessment is one of the most common reforms of higher education in different countries of the world, which has been expanding especially since the 1990s and under the influence of Neoliberalism. The purpose of this quality assessment was to make university accountable and transparent, and thus improve the quality of the university as a market-oriented institution. In recent years, these market-oriented reforms have prevailed in different countries of the world and their footprint can also be seen in the higher education of Iran. The purpose of the present research was to study quality assessment with a critical and philosophical perspective. To achieve this goal, conceptual analysis, qualitative content analysis and inferential method were used. First, an attempt was made to analyze quality assessment as a concept and to examine its anthropological foundations. The next step was to analyze the market-based quality assessment and related reforms in the supporting documents of Iranian higher education, and finally the compatibility of anthropological foundations of market-oriented quality assessment with the Islamic perspective has been studied. The findings of this study indicate that quality assessment in the core of marketisation relies on the anthropological foundations of Neoliberalism and especially homo economicus model. This model account man as a self-interested, utility-maximizer, and competitive being. From this point of view, economic rationality is the supreme model for all rational activity and should be guide of all other human activities. Marketisation is the result of the idea that its goal is to manage institutions such as universities in a market-oriented way, which is interpreted as New Public Management (NPM). Therefore, market-oriented quality assessment tries to evaluate the quality of the university in a way similar to commercial companies and to make the university transparent and responsive to the needs of customers. Thus in these model universities, for funding, have to compete with other universities and higher institutions in competitive higher education markets. Therefore, the student is a customer of knowledge and educational services, and the university attracts her/him through marketing and offers her/him the knowledge as a commodity. In higher education markets, economic rationality and cost-benefit logic control and regulate all relationship and process includes quality assessment. Examination of Iranian higher education support documents indicates the impact of the NPM and market-based quality assessment. In these documents, market-oriented reforms are accepted as one of the necessities of modernizing the university, which the university must accept them for transformation. In these documents, an attempt has been made to intertwine market-oriented reforms with Islamic-Iranian goals. Competitiveness of universities, uncontrolled development of privatization, development of commercialization logic, and market-based quality assessment in universities are examples of market-oriented reforms considered in Iran's higher education supporting documents. Of course, market-oriented reforms in Iran can be considered as a kind of quasi-Neoliberalism that, although it has retained some basic principles of Neoliberalism, but in the cultural context of Iranian higher education, it has taken a different form and has created new damages.  These reforms further seek to make the university financially independent from state funding, but at the same time increase the government's indirect control over the university by new methods of quality assessment. The commodification of knowledge, the student as a customer, the evaluation of quality on the basis of quantity and the production of objective results, coercive accountability and the resulting moral model have important cultural and philosophical meanings that have been severely criticized in the Islamic view. Market-oriented quality assessment leads to a kind of prformativity and quantification, and its result is a kind of academic ethics based on cost-benefit logic. In this view, knowledge, educational services and educational relations are evaluated based on the profitability. In this view, the religious and traditional conception of knowledge as a path to perfection and the moral values based on it are seriously questioned. At a market-oriented university, a university professor is a knowledge provider, and his value depends on the extent of his educational services and customer satisfaction.

Investigation of the bias and Differential Item Functioning of important tests like mathematics between different groups has appealed to many researchers in education; most of them, however, have examined mathematics as a general structure. Attempts were made in this research to make use of the capabilities of Cognitive Diagnosis Models (CDMs) for precise and more detailed investigation of the Differential Item Functioning in the mathematics test of the eighth grade (third grade of guidance school) (T. 2007), and also for investigation of the effects of the questions with Differential Item Functioning on students’ performance. The statistical population of the study is the same as the eighth grade statistical population in the 2007 TIMSS math test, which was conducted in the academic year 2006-2007. In the academic year in question, 1,475,368 third grade middle school students were studying in 29,956 classes. The sample of the present study was a part of the Iranian third year middle school students participating in the 2007 TIMSS, whose number is equal to 1137. For analyzing the data and responding to the research questions, the non-compensative model DINA, with the two parameters of guessing and slipping, was used. First, the q matrix was set using required attributes of the questions. Then, the data were fitted to the model, and the parameters were estimated for the two female and male groups. The Differential Item Functioning was then examined using the Wald test. The results demonstrated that of the 44 investigated questions, 26 were anchoring questions, and 18 (40%) were ones with DIF, where 12 out of the 18 questions with DIF had linear DIF, and 6 had nonlinear DIF. Of the 18 questions with Differential Item Functioning, 7 had Differential Item Functioning in favor of the female students who had achieved mastery over the attributes necessary for responding correctly to the questions, and 10 in favor of the male ones. One question here had Differential Item Functioning in favor of neither the group of females nor the group of males who had achieved mastery over the attributes necessary for responding correctly to the questions. Furthermore, of the 18 questions with DIF, 9 have Differential Item Functioning in favor of the female students who have not achieved mastery over the attributes necessary for responding correctly to the questions, and 7 in favor of the male ones, and 2 questions have Differential Item Functioning in favor of neither the group of females nor the group of males who have achieved mastery over the attributes necessary for responding correctly to the questions. It can be concluded that The Wald test is enabled in the Framework Cognitive Diagnosis Models (CDMs). In addition to the test linear DIF and nonlinear DIF To be used in identifying differential function in mastery and non -mastery Persons in attributes It is suggested by these results and those of the research in the area that question format can decrease or increase gender difference; test developers; therefore, should try to make the questions so that gender differences is minimized. By identifying the factors effective on gender differences, instructors, teachers, and educational policy-makers in the country will be able to design and formulate novel, creative curricula or improve current ones to make the educational environment and system of the country flourishing and rich.