Design and Psychometric Evaluation of User Satisfaction Questionnaire in Integrated Health System for Reliability and Validity

Evaluating the effectiveness of health information systems and users' satisfaction with system performance is important. The success of an information system is directly dependent on the system responding to user needs. Due to the importance of this issue, a tool has been designed to evaluate users' satisfaction with Integrated Health System.

This cross-sectional study was conducted in 2018 to design a questionnaire to assess the users' satisfaction with Integrated Health System. In the first step, the areas of user satisfaction were identified by reviewing literature and reliable scientific sources. Then a bank of questions was created and after classifying the indicators and identifying related questions the primary tool was developed in several areas. Face validity and content validity of the questionnaire were determined using CVI and CVR indices. Reliability over time was assessed by 20 users of in an Integrated Health System with two-week intervals.

The final questionnaire included demographic information and 55 questions about the components of a System Satisfaction Questionnaire. Questions related to each component included system interface, system functions, system performance, system information and statistics, system supportive services, System services, and and total satisfaction from the system. The average CVI and CVR for total questions were 0.84 and 0.73, respectively. For reliability over time, the coefficient (ICC) for the whole structure was 0.98. The reliability of the questionnaire evaluated by Cronbach's alpha and internal consistency method, that this value was obtained 0.914 for total questionnaire. Also, separately for each structure was obtained in the "good" range.

In this study, we tried to provide a brief and expressive user satisfaction assessment tool that required the least amount of time to respond. The high response rate of all questions indicates that designed tool achieved the objectives of the study. The most emphasized component by users is the system support component. Ease of use of the system, usefulness and support of the system from performing daily activities are other things that are more important for users.