Original Article
The development, usability, and reliability of the Electronic Patient Visit Assessment (ePVA) for head and neck cancer
Abstract
Background: Annually, over 65,000 persons are diagnosed with head and neck cancer in the United States. During treatment, up to 50% of patients become severely symptomatic with pain, fatigue, mouth sores, and inability to eat. Long term complications are lymphedema, fibrosis, dysphagia, and musculoskeletal impairment. Patients’ ability to perform daily activities and to interact socially may be impaired, resulting in poor quality of life. A pragmatic, clinically useful assessment is needed to ensure early detection and intervention for patients to report symptoms and functional limitations over time. We developed the Electronic Patient Visit Assessment (ePVA) that enables patients to report 42 symptoms related to head and neck cancer and 17 limitations of functional status. This manuscript reports (I) the development of the ePVA, (II) the content validity of the ePVA, and (III) the usability and reliability of the ePVA.
Methods: Usability was evaluated using the “Think Aloud” technique to guide the iterative process to refine the ePVA based on participants’ evaluations. After signing the informed consent, 30 participants with head and neck cancer completed the ePVA using digital tablet devices while thinking aloud about ease of use. All patient conversations were recorded and professionally transcribed. Reliability of the ePVA symptom and functional limitation measures was estimated using the Kuder-Richardson test. Convergent validity of the ePVA was evaluated using the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 global QoL/health scale. Transcribed qualitative data were analyzed using directed content analysis approach. Quantitative analyses consisted of descriptive statistics and correlation analyses.
Results: Among participants, 90% strongly agreed or agreed that the ePVA system was easy to use and 80% were very satisfied. Only minor usability problems were reported due to formatting and software “bugs”. Reporting of usability problems decreased in frequency over the study period and no usability problems were reported by the last 3 participants who completed the ePVA. Based on participants’ suggestions during the iterative process, refinement of the ePVA included increased touch sensitivity of the touch screen technology and customized error messages to improve ease of use. The ePVA also recorded patient reported symptoms (mouth symptoms: 93%, fibrosis: 60%, fatigue: 60%). The ePVA demonstrated acceptable reliability (alpha =0.82–0.85) and convergent validity (ePVA total number of reported symptoms and function limitations was negatively correlated with EORTC QLQ-C30 global QOL/health scale: r=−0.55038, P<0.01).
Conclusions: The ePVA was rigorously developed, accepted by patients with satisfaction, and demonstrated acceptable reliability and convergent validity. Future research will use data generated by the ePVA to determine the impact of symptom trajectories on functional status, treatment interruptions and terminations, and health resource use in head and neck cancer.
Methods: Usability was evaluated using the “Think Aloud” technique to guide the iterative process to refine the ePVA based on participants’ evaluations. After signing the informed consent, 30 participants with head and neck cancer completed the ePVA using digital tablet devices while thinking aloud about ease of use. All patient conversations were recorded and professionally transcribed. Reliability of the ePVA symptom and functional limitation measures was estimated using the Kuder-Richardson test. Convergent validity of the ePVA was evaluated using the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 global QoL/health scale. Transcribed qualitative data were analyzed using directed content analysis approach. Quantitative analyses consisted of descriptive statistics and correlation analyses.
Results: Among participants, 90% strongly agreed or agreed that the ePVA system was easy to use and 80% were very satisfied. Only minor usability problems were reported due to formatting and software “bugs”. Reporting of usability problems decreased in frequency over the study period and no usability problems were reported by the last 3 participants who completed the ePVA. Based on participants’ suggestions during the iterative process, refinement of the ePVA included increased touch sensitivity of the touch screen technology and customized error messages to improve ease of use. The ePVA also recorded patient reported symptoms (mouth symptoms: 93%, fibrosis: 60%, fatigue: 60%). The ePVA demonstrated acceptable reliability (alpha =0.82–0.85) and convergent validity (ePVA total number of reported symptoms and function limitations was negatively correlated with EORTC QLQ-C30 global QOL/health scale: r=−0.55038, P<0.01).
Conclusions: The ePVA was rigorously developed, accepted by patients with satisfaction, and demonstrated acceptable reliability and convergent validity. Future research will use data generated by the ePVA to determine the impact of symptom trajectories on functional status, treatment interruptions and terminations, and health resource use in head and neck cancer.
