Initial Development of User-Based Quality Evaluation Questionnaire of Smartwatch Technology for Applying to Healthcare

Smartwatches are a consumer wearable device offering a potential, practical, and affordable method to collect personal health data in healthy adults. For patients with chronic diseases, this would enable symptom monitoring and aid clinical decision making. Therefore, providing customized checklists to recommend smartwatches is beneficial. However, few studies have evaluated the practical functions of smartwatches and their influence on user acceptance. We aimed at developing a reliable tool to assess the quality of smartwatches from the users’ perspective.

To develop the smartwatch rating scale (SWRS), we conducted a comprehensive literature review as well as reviewed relevant websites. The SWRS includes 22 items for the usability (usability, functionality, safety, material, and display) and five items for the acceptance and adoption domain (satisfaction and intention). We measured the scale’s internal consistency and inter-rater reliability by evaluating seven smartwatches.

The overall scale demonstrated an excellent level of internal consistency (Cronbach’s alpha = 0.91), with each subscale’s internal consistency above good level (0.74 ~ 0.92). Inter-rater reliability using intraclass correlation coefficients (ICC) was at good level (2-way random ICC = 0.82, 95% CI 0.09 – 0.97).

The SWRS is reliable, which can meet the need for assessment of smartwatch technology for utilizing in personal healthcare. Accounting for users’ perspectives will help make the most of technology without impairing the human aspects of care, this study can help consumers choose a smartwatch based on their preferences and provide guidelines for developing user-friendly wearable devices aimed at health behavior changes.