The usefulness of the Electronic Patient Visit Assessment (ePVA)^© as a clinical support tool for real-time interventions in head and neck cancer

Introduction

Head and neck cancer (HNC) is composed of subtypes of tumors that occur in the lip, oral cavity, pharynx, larynx, and paranasal sinuses. The estimated incidence of HNC in the United States (US) has grown from 53,640 in 2013 to 65,410 in 2019. This increase is attributed to the rising incidence of oropharyngeal cancer associated with human papillomavirus (HPV) (1-3). The 5-year survival rate in HNC has also improved, increasing from 53% in 1979 to approximately 70% in 2011 (4). This improvement is partially the result of the survival benefit seen in persons diagnosed with oropharyngeal cancer associated with HPV infection. It is also the result of the increased intensity of treatment, with the addition of chemotherapy to radiation therapy and surgery (3,5,6). In general, 60% of those diagnosed with HNC will have locally or regionally advanced disease and receive a combined treatment of surgery, radiation and chemotherapy (7).

During treatment, patients with HNC experience severe symptoms and associated functional limitations, leading to difficulty swallowing, impaired eating, and hospitalization for feeding tube placement (8). Up to 50% of patients experience painful and debilitating oral ulcers during radiation therapy, which are susceptible to increased colonization of microorganisms (9). The loss of mucosal integrity universally results in levels of pain for which even opioids may not be effective and places patients at risk for focal secondary infections, bacteremias, and sepsis (10).

These treatment-induced symptoms and associated functional limitations can lead to malnutrition and weight loss, which cause poor treatment tolerance, delays or premature cessation in treatment, hospitalizations, or lower quality of life (QoL) (11-15). Managing symptoms and functional limitations during active treatment to avoid poor outcomes and a lesser QoL requires close monitoring of patients with frequent visits during treatment. After completion of treatment, patients are at risk for long-term symptoms and functional limitations, such as dry mouth, voice changes, difficulty swallowing, lymphedema, fibrosis at the treatment area, difficulty moving the neck and shoulders, and decrease in daily activities (8,16,17). Managing these symptoms is complex and involves integrating specialty services such as palliative and rehabilitation services. Using pragmatic mHealth technology for early detection of symptoms and functional limitations in HNC has the potential to improve patients’ QoL and possibly survival.

Evidence is mounting as to the clinical usefulness of mHealth technology, such as real-time electronic patient-reported outcomes (ePROs) measures, to improve the symptom burden, QoL, and survival in patients with cancer (18-21). In the emerging and dynamic team-based approach to delivering cancer care, collecting ePROs provides longitudinal monitoring of treatment of adverse effects, disease complications, functional limitations, and psychological states throughout the cancer therapy for all providers to use. Two randomized clinical trials compared ePRO measures versus standard care for patients either undergoing cancer treatment or lung cancer surveillance. These studies found that ePROs were cost-effective methods that were associated with a significantly better QoL and survival (18,19). Systematic reviews point to possible mechanisms for the clinical usefulness of patient-reported measures. These mechanisms include increased patient and clinician awareness of symptoms and improved communication between patients and clinicians that helps patients avoid emergency room visits and hospitalizations (22,23). Hallmarks of effective patient-reported measures in cancer are brevity (i.e., completed in less than 10 minutes), tailored to the patient, and tailored to the type of cancer (22). An ideal ePRO platform is clinically relevant, validated, reliable, and would offer patient and provider usability. Thus, to be clinically useful, patients and providers must accept the technology, demonstrated by their usage behavior (24,25).

The Electronic Patient Visit Assessment (ePVA)^© for HNC, using mHealth technology, was developed for patients to report their symptoms and functional limitations. The ePVA has a user-friendly interface with the ability to collect real-world data longitudinally that can be used to identify patients who may benefit from real-time interventions that can optimize patients’ health-related quality of life (HRQoL). The purpose of this paper is to present the development and usefulness of the ePVA as a clinical support tool for real-time intervention for patient-reported symptoms and functional limitations in HNC. The hypothesis for this study was the ePVA would be considered ready for further implementation if greater than 70% of eligible participants and providers demonstrated acceptance of the ePVA. The specific aims of this paper were to: (I) describe the development of the ePVA as a clinical support tool, (II) determine the usefulness of the ePVA, and (III) describe the use of the ePVA for real-time interventions in HNC practice through a clinical exemplar. We present the following article in accordance with STROBE guideline for observational studies (26,27). We present the following article in accordance with the STROBE reporting checklist (available at http://dx.doi.org/10.21037/mhealth-19-250).

Methods

Framework and design

This observational cohort study was designed to determine the usefulness of the ePVA as a clinical support tool for HNC. The theoretical framework guiding this study was the Technology Acceptance Model (24,25,28). This model shows that perceived ease of use and perceived usefulness influence the end user’s acceptance of technology. To be an effective clinical tool, the patient-reported measure should adequately capture patients’ symptoms and functional limitations and demonstrate acceptability by patients and providers. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was reviewed and approved by the NYU Perlmutter Cancer Center Protocol Review and Monitoring Committee and NYU Langone Health School of Medicine Institutional Review Board (s16-01308, s16-02037) and informed consent was taken from all individual participants.

Setting and participants

The study was conducted at a National Cancer Institute (NCI)—designated Comprehensive Cancer Center in the northeastern US. The study population was a convenience sample of persons diagnosed with HNC. Based on recruitment during a prior usability study of the ePVA (29), the study team anticipated enrolling between 50 to 90 participants. The ePVA includes items addressing both acute and chronic symptoms; therefore, to determine the usefulness of the ePVA, the study team purposefully recruited patients receiving cancer treatment and patients who had completed treatment at time of enrollment. The inclusion criteria also consisted of 18 years and older and English speaking. Exclusion criteria were conditions that prevented informed consent and inability to comprehend English.

Procedures

Between January 2018 and August 2019, 138 persons diagnosed with HNC were approached during or after completion of cancer treatment (see Figure 1). Two people were ruled as ineligible for the study because of pre-existing medical conditions, and 58 people declined to enroll. The primary reasons that people declined to enroll in the study were lack of time (40%) or did not feel well enough to participate (28%). Seventy-eight people consented to participate in the study. Three people did not complete the ePVA because of complications from either the HNC or other medical conditions, and judged as screen failures. Ultimately, 75 people participated in the study, consisting of 69% who were receiving cancer treatment (52 of 75) and 31% who had completed treatment (23 of 75) at enrollment. Based on observational data, there were no significant differences in sex, race, or ethnicity between those who consented and those who declined.

Figure 1 Participant enrollment. ePVA, Electronic Patient Visit Assessment; EORTC, European Organization for Research and Treatment of Cancer.

After informed consent, during the same oncology visit, participants completed the ePVA questionnaire (29), followed by the European Organization for Research and Treatment of Cancer (EORTC) Quality of Life Questionnaire (QLQ-C30) version 3.0 (30), using handheld digital devices with touchscreen technology. Of the 75 participants enrolled in the study, 69 completed both the ePVA and the EORTC. Six participants completed only the ePVA because of time constraints or fatigue. The study team approached participants who had follow-up visits within the 6-month study period to determine their willingness to again complete the ePVA. The team’s goal was to collect data at the participants’ next visit to identify data trends and collect longitudinal data. However, due to the nature of the treatment for HNC (i.e., intensity of a multi-modality treatment associated with adverse effects), the protocol allowed the flexibility of collecting this data within a 6-month time period without being a “deviation” to the protocol. This provided an opportunity for the study team to accommodate patients’ needs, record reasons for delays in collecting data, and establish optimal data time points in preparation for larger studies. All participants enrolled in the study received $10 gift cards. To determine their usage behavior in real world clinical settings, participants were not reimbursed for completing follow-up surveys.

On completion of the ePVA, the participants’ responses were stored on an encrypted server, downloaded to an ePVA report template, and automated to the HNC care team in an encrypted email. Providers acknowledged receipt of the ePVA reports by sending an encrypted email to the study team. The study team also verbally notified the HNC care team of any items that were considered actionable (e.g., difficulty breathing). The HNC care team used the ePVA reports to implement real-time clinical interventions according to their clinical judgment and knowledge of the participants. These interventions included scheduling follow-up visits for symptom assessment, pain medication adjustment, and referrals to specialized services (e.g., social work, rehabilitation). The study team recorded all completions of the ePVA and providers’ responses on Microsoft^® Excel^® databases and stored on encrypted servers. Participants did not receive the ePVA reports.

Development of the ePVA as a clinical support tool

ePVA

The ePVA has been described elsewhere (29). Briefly, the ePVA is a web-based mHealth tool patient-reported measure that asks questions about 21 categories of symptoms and functional status limitations common to HNC (i.e., pain, eye, ear, nasal, mouth, voice, fibrosis, edema, skin, gastrointestinal, fatigue, limitation in movement, sleep, breathing, difficulty eating or drinking, swallowing, communication, social activities, anxiety, depression, and daily activities). In total, the ePVA consists of 42 items to assess symptoms, 17 items to assess functional limitation, and text boxes for patients to type free text. The ePVA is built on a highly flexible web-based platform. The format contains conditional items to provide a tailored questionnaire that aligns with the patient’s health state. The underlying theory that guided the development of the ePVA is that symptoms and functional status affect patient outcomes, such as QoL (29,31). Findings from a previous usability study of the ePVA indicated acceptable reliability (alpha =0.82–0.85), and convergent validity with HRQoL (29).

Development of the ePVA as a clinical support tool

Formal focus group discussions with the interdisciplinary team that cared for patients with HNC occurred during the early stages of the development of the ePVA. The interdisciplinary team consisted of nurses, physician assistants, social workers, speech therapists, occupational therapists, physical therapists, and nutritionists. The interdisciplinary team reviewed aggregated ePVA data from a prior usability study (29), and confirmed that the ePVA provided additional information that helped providers care for patients. Thus, the HNC team asked to receive the ePVA reports in real-time. To this end, the ePVA began to be used as a clinical support tool to inform clinical decisions about symptom management.

In collaboration with the interdisciplinary care team, the study team developed a protocol to guide the use of the ePVA as a clinical support tool. According to this protocol, the data collected with the ePVA are reported to the care team immediately after the participant completes the ePVA (See Figure 2). The protocol also specifies that the ePVA data collected from patients undergoing chemotherapy are sent to the care team before participants leave the cancer center. The study team verbally contacts the care team when patients indicate critical symptoms (e.g., new onset of dyspnea). Using this protocol, the care team can make clinical interventions in real-time using ePVA data.

Figure 2 Protocol for using the ePVA for real-time interventions. ePVA, Electronic Patient Visit Assessment.

Variables

Clinical usefulness

The following variables were used to examine the usefulness of the ePVA as a clinical support tool for real-time interventions for patient-reported symptoms and functional limitations in HNC.

Symptoms and functional limitations

Patient-reported symptoms and functional limitations were collected using the ePVA. Symptoms were defined as the multidimensional experience of perceived indicators of abnormal biological or physiological changes (31-33). Function was defined as activities people do to meet daily needs, interact with friends and family, perform work, and maintain their health and well-being (34). The ePVA items consist of binomial questions (yes/no), representing the participants’ perception of the presence or absence of the symptom or functional limitation. A sum of items was used for analyses. Reliability of the ePVA as a patient-reported measure of symptoms and functional limitation was estimated using the Kuder-Richardson Formula 20 measure for questionnaires with binary variables (35). Convergent validity of the ePVA with HRQoL was evaluated by correlating the sum of symptoms and functional limitations with the EORTC QLQ-C30 global QoL/health scale scores.

HRQoL

HRQoL was measured by the EORTC QLQ-C30 global QoL/health scale. HRQoL was defined as the global and QoL aspects related to health (30,36). The EORTC QLQ-C30 global QoL/health scale is a frequently used valid and reliable measure in HNC studies. This subscale consists of two items that ask patients to rate their overall health and overall QoL on a scale from 1 (very poor) to 7 (excellent). The health and overall QoL ratings were summed then transformed to a scale ranging from 0 to 100, with 100 representing best HRQoL. Prior research has demonstrated that EORTC QLQ-C30 global QoL/health scores at 6 and 9 months after diagnosis of HNC are predictive of participants’ survival at 10 years after diagnosis (37).

Acceptability of the ePVA

Participant acceptability was measured as the proportion of those who completed the ePVA within 6 months of study enrollment (numerator) to the number who had two or more oncology visits during the duration of the study [i.e., 6 months (denominator)]. Provider acceptability was measured as the proportion of ePVA reports that one or more members of the HNC team acknowledged receiving by email (numerator) to the number of ePVA reports sent to the HNC care team (denominator). Acceptability was defined as the usage behavior of end-users (participants and providers) (24,28). The goal was that greater than 70% of participants answered the ePVA at two or more oncology visits within a 6-month period and the HNC care team read greater than 70% of ePVA reports.

Analysis

Qualitative and quantitative methods were used throughout the study. Descriptive statistics, consisting of means and frequencies, were conducted. Pearson correlation coefficient was used to determine the correlations between the EORTC QLQ-C30 global QoL/health scale and the sum of symptoms, functional limitations, and total number of symptoms and function limitations. Student’s t-tests were used to calculate the association between the EORTC QLQ-C30 global QoL/health scale and the categories of symptoms and functional limitations. All statistical analyses were performed on SAS 9.4 and STATA using complete case analysis.

After informed consent, discussions of the focus groups of interdisciplinary HNC team were audio recorded, and transcribed. Two authors (JVC, EL) used content analysis of the recorded focus group conversations to identify themes and principles that guided the development of the ePVA as a clinical support tool (38,39). The two authors discussed any disagreements until they achieved consensus. To provide an audit trail and ensure the reliability of decisions, the authors documented their decisions in Microsoft Office^® Excel^® and Word^® programs. All results were confirmed with the interdisciplinary HNC team.

Results

The study population consisted of 75 participants who answered the ePVA for a total of 189 completions (see Table 1). The participants were primarily male (71%), White (76%), Not Hispanic or Latino (69%), diagnosed with oropharyngeal or oral cavity cancers (53%), and undergoing treatment for HNC at the time of enrollment into the study (69%). For participants enrolled in the study, the treatment for HNC mainly consisted of a combination of surgery, radiation therapy, and chemotherapy (81%). At enrollment, participants were symptomatic, reporting on average 8.4±5.3 symptoms and 4.4±3.0 function limitations.

Table 1 Participant characteristics (n=75 participants)
Full table

Clinical usefulness

Reliability and convergent validity of the ePVA

Using Kuder and Richardson Formula 20 reliability measure (35), the data indicated acceptable reliability of the ePVA (alpha =0.85). Data from 69 participants who completed both the ePVA and the EORTC QLQ-C30 global QoL/health scale supported the convergent validity of the ePVA with HRQoL. Correlation analyses revealed that participants reporting the highest number of symptoms and functional limitations also reported significantly decreased EORTC QLQ-C30 global QoL/health scores. This finding indicates that participants with the greater number of symptoms and functional limitations on average experienced worse HRQoL (sum of symptoms: r=–0.50, P<0.0001; sum of functional limitations: r=–0.56, P<0.0001, sum of symptoms and function limitations: r=–0.57, P<0.0001) (See Table 2).

Table 2 Correlation between sum of symptoms, functional limitations, and HRQoL scores^† (n=69)
Full table

We also conducted Student’s t-test analyses to identify the categories of the symptoms and functional limitations that had a strong association with participants’ perception of HRQoL. These analyses found participants that reported their current health limited their social activities averaged the lowest EORTC QLQ-C30 global QoL/health scores (limited social activities: mean =60.5, t=4.6, P<0.0001). This finding indicated the strong effect of social isolation on participants’ perception of their HRQoL (see Table 3).

Table 3 Relationship between ePVA categories of symptoms, functional limitations, and HRQoL scores^† (n=69)
Full table

Acceptability

Participants

The ePVA met the study goal for participant acceptance. Among 75 participants enrolled in the study, 64 participants had follow-up visits within 6 months of study enrollment. Among these 64 participants, 92% (59 of 64) completed the ePVA at one or more subsequent follow-up visits within the 6-month study window. The six participants who did not complete the ePVA at subsequent visits were experiencing disease progression or treatment complications. The key to this high rate of completion of the ePVA was the study team’s accommodation of participants’ needs during treatment—22% (13 of 59) of those who completed the ePVA at multiple visits deferred at least once until a more convenient time. Participants’ reasons for the delay in completing the ePVA were primarily because of side effects related to cancer treatment (e.g., fatigue, nausea) or schedule conflicts (e.g., transportation, time of treatments).

Providers

The ePVA met the study goal for provider acceptance. After participants completed the ePVA, 189 reports were automated to the HNC team. Providers acknowledged reading 89% (169 of 189) of ePVA reports. The proportion of ePVA reports read by the HNC team increased over time. For example, among the last 10 participants enrolled in the study, providers acknowledged receipt of 100% (21 of 21) of the ePVA reports.

Clinical exemplar

This clinical exemplar, based on observations and anecdotal events, describes the use of the ePVA as a clinical support tool.

Initiation of treatment

Mr. G is a 62-year-old male diagnosed with stage II oropharyngeal cancer. The treatment plan consisted of three cycles of Cisplatin chemotherapy and 7 weeks of daily radiation therapy. On the day that he began chemotherapy and radiation therapy, Mr. G completed the ePVA before starting treatment. Using the ePVA, Mr. G reported experiencing mild pain at the mouth, rated 3 on a scale between 0–10 (10 is worst), described as soreness. However, he also reported anxiety and depression. The HNC care team reviewed the ePVA data with Mr. G. They confirmed with Mr. G that acetaminophen was adequate pain medication for now. The HNC care team contacted the social worker, who visited with Mr. G during his initial treatment visit. The social worker stated that Mr. G’s anxiety and depression is appropriate for a new diagnosis of cancer. However, the social worker planned to continue to monitor Mr. G throughout the treatment.

During cancer treatment

Mr. G completed the ePVA during week 5 of 7 of radiation treatments. He scored his pain as ten at his mouth, described as tingling, soreness, and aching. He reported fatigue, difficulty talking and eating, nausea/vomiting, limitations in his social activities, anxiety, depression, and difficulty climbing stairs. The HNC care team discussed the ePVA report with Mr. G. They noted that he had dropped nearly 10 pounds over the last 2 weeks. Together with Mr. G, the HNC care team decided to change his pain medication from acetaminophen with codeine to a fentanyl transdermal patch, thus decreasing the number of medications that Mr. G needed to swallow. They emphasized to Mr. G the need to eat. They scheduled additional sessions of intravenous fluids that included anti-nausea medications to help Mr. G maintain adequate hydration. The HNC care team planned to evaluate Mr. G on Friday afternoon to anticipate any needs that he may experience over the weekend.

After cancer treatment

One year after completing his treatment, Mr. G is in the office for a routine follow-up appointment. His imaging studies demonstrated that he continues to be free of cancer. He completed the ePVA during his appointment with the HNC care team. Using the ePVA, he reported edema and neck fibrosis at the radiation site, dry mouth, diminished social activities, and mild depressive symptoms. The HNC care team reviewed the data collected with the ePVA with Mr. G. Overall, he felt well and was very happy to hear about the imaging studies. However, he was experiencing long-term side effects that can occur after treatment. The HNC care team referred Mr. G to rehabilitation for speech and swallow therapy and instructions for home exercises to manage lymphedema and fibrosis at the neck. They also referred him to the HNC social support group to exchange information about self-management of symptoms with others living with HNC.

Discussion

Given the severe symptom burden of HNC, mHealth technology may help with patients’ symptom management. Accordingly, the ePVA was developed as a patient-reported measure for symptom and function assessment for HNC. The findings from this study support the reliability, convergent validity, and acceptability of the ePVA in recording patient-reported symptoms and functional limitations. Thus, the ePVA may be useful to support real-time clinical interventions in HNC.

This study adds to the body of literature by demonstrating the development of the ePVA as a clinical support tool to facilitate patient-centered cancer care and critical clinician-patient interaction at the point-of-care. This use of the ePVA evolved through collaboration with the HNC team and patient engagement. The immediate availability of the ePVA data facilitated automated reports of actionable items to the HNC care team that resulted in real-time interventions for patients. To this end, providers were able to integrate the ePVA reports into their supportive care of their patients.

The ePVA is designed as a user-friendly mHealth tool for both patients and providers. Patients can complete the assessment during intense cancer treatment. For the most part, participants undergoing 7 weeks of combined chemotherapy and radiation therapy could complete the ePVA at 2 to 3-time points despite the intensity of their treatment. However, the study team learned that successful real-world data collection required flexibility to accommodate patients’ needs. Further, participants who deferred completing the ePVA at specific time points may be experiencing actionable symptoms and functional limitations. Thus, the study team began reporting to the HNC care team when participants deferred completing the ePVA.

To design a user-friendly tool for providers, the study team closely collaborated with the HNC care team. Together, they developed procedures to facilitate the use of the data collected with the ePVA for real-time interventions during busy work hours. The high proportion of ePVA reports acknowledged as read by providers conveys that the HNC care team valued the information collected with the ePVA.

This study affirms the principles of technology adoption—that perceived ease of use and usefulness of new technology directly affect people’s usage behavior (24,28). Any mHealth technology clinical application that is viewed as increasing providers’ workload while yielding little additional information to guide patient care will likely meet barriers to implementation. In addition, having patient-reported data that are quantitative, validated, easily captured, and recorded has the potential to add decision support value and improve the management of patients.

Further, how patient-reported data are presented and situated within electronic health records may impact the successful use of mHealth technology in clinical practice. Clinical interfaces should be easy to read and integrated into the electronic medical record. For example, mHealth technology that requires extra steps, such as the opening of additional screens to access patient-reported information, will meet challenges in uptake by health care providers. Emerging technologies interfacing secure connections compliant with the Health Insurance Portability and Accountability Act (HIPAA) of 1996, such as Fast Healthcare Interoperability Resources (FHIR), may facilitate the use of mHealth technology in real-world settings seamlessly across multiple applications and end-user devices. For example, FHIR may allow clinicians to view data from third-party applications in an embedded window in electronic health records without opening additional screens or requiring other log-ins (40).

Implications for research

An important area of ongoing research in mHealth technology is to determine best uses of patient-reported data in real-world settings. The complexities of cancer care require a team approach to quality management of the delivery of multiple dimensions of data to facilitate the interaction between the clinical staff and the patient. Some patient-reported data are delivered to providers as email alerts of escalating symptoms or as clinical summaries during clinical visits (18-20,41). Other mHealth technology applications offer evidence-based recommendations to patients to enhance their self-management of symptoms (20,21,42). For our study, we designed a report of the ePVA data that successfully provided a global picture of the participant’s health status for the HNC care team. This global picture facilitated the use of the ePVA to inform real-time clinical decisions about symptom management. Future research can investigate the optimal delivery approach that best informs real-time clinical decisions.

Another vital area of research in patient-reported data is identifying the best measurement design to uncover the actionable symptoms and functional limitations that require real-time interventions. Symptom measurement design should move beyond information gathering and to configure measures with branching question capability and quick-scoring interpretation to facilitate patient decision support and shared decision making. Some researchers have used pragmatic approaches, such as establishing minimally important differences (MIDs). These differences include one half of a standard deviation, scores in the lower quartile, or scores that are two standard deviations from the mean. Other methods include cut scores based on simple statistics, such as t-tests, or receiver operating curve analyses for measures with similar domains (43). We designed the ePVA to consist of binomial questions. Thus, patients decide the presence or absence of their symptoms. More work is needed to compare pre-determined cut scores with patient reports of presence or absence of symptoms to determine the clinically relevant patient-reported data. Ultimately, to meet the needs of increasing demands in cancer care, these electronic platforms should not only produce automated reports but engage and refer patients automatically to appropriate members of the care delivery team in real-life routine practice conditions.

Implications for policy

An important policy issue is whether the growing use of mHealth technology applications promotes health inequalities. Clinical applications of mHealth technology have the potential to expand health resources to underserved populations who own smart phones. Yet, these technologies may be disproportionately available to educated patients and to those in urban academic centers, such as the population that enrolled in our study (44). Further, people who distrust technology may be less likely to use the technology. To this end, mHealth technology may disproportionately benefit people who are more advantaged or people who live in developed countries. Suggested methods to overcome these barriers are to develop studies that include the planned recruitment of diverse study participants, outcomes relevant to health equity, and subgroup analyses by demographic groups (40).

Another important policy issue is patient privacy. As more mHealth technology applications become connected to medical records, linking biological data with personal data, these concerns will need to be addressed to move the field forward (40). Patients will need to have confidence that the information generated by mHealth technology is protected (45). Some solutions to address these issues are to encourage patient ownership and the sharing of their data. Further, researchers can include patients as partners in the design and conduct of research studies that merge personal data with health records (45).

Strength and limitations

The strength of this research is the strong provider and patient acceptance of the ePVA. Some limitations to the study included restricting the completion of the ePVA to point-of-care. For example, the majority of potential participants who declined to enroll in the study expressed a desire to participate. Still, they did not have the time to complete the survey during a busy oncology visit. Therefore, future research could include investigating whether recruiting and retaining participants is enhanced by expanding access to the ePVA through electronic health record systems, especially during this era of COVID-19 infections. This expansion would enable participants to complete the ePVA remotely before their oncology visit. Other limitations of this research include that it was conducted at a single urban site and the recruitment consisted of a convenience sample. Therefore, researchers should be cautious in applying the study findings to other types of cancer or institutions. Still, the significant findings of this study support the need for future research to determine the effectiveness of the ePVA as a clinical support tool to decrease the symptom burden and improve the QoL for patients with HNC.

In conclusion, patients with HNC undergo intense treatment of surgery, radiation therapy, chemotherapy, or some combination of these treatments. During treatment, patients experience painful and debilitating symptoms and functional limitations that can delay or decrease cancer treatment. After treatment, patients may develop long-term symptoms and functional limitations that decrease HRQoL. Accordingly, we developed the ePVA for HNC as a mHealth tool for patients to report their symptoms and functional limitations with the goal to alleviate severe toxicities that emerge between visits by facilitating communication using an electronic platform. The analyses supported the reliability, validity, and acceptability of the ePVA A clinical exemplar demonstrated the use of the ePVA that results in real-time interventions to support patients clinically. The study findings support future translational research to enhance the usefulness of the ePVA in real world settings for early interventions that decrease symptom burden and improve the QoL of patients with HNC.

Acknowledgments

Funding: JH Van Cleave and MRF: Louis and Rachel Rudin Foundation Interdisciplinary Pilot Project Award (Van Cleave, PI/Fu, Co-Investigator); 2017 Palliative Care Research Cooperative Group Investigator Development Pilot (funded by National Institute of Nursing Research U24NR014637) (Van Cleave, PI/Fu, Co-I); New York University Research Challenge Fund (Van Cleave, PI/Fu, Co-I); John A. Hartford Foundation Change AGEnts Action Award (Van Cleave, PI/Fu, Co-I). BLE: USA National Institutes of Health/National Cancer Institute (P30CA006927 Fox Chase Cancer Center Support Grant).

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editor (Mei R. Fu) for the series “Real-Time Detection and Management of Chronic Illnesses” published in mHealth. The article has undergone external peer review.

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at http://dx.doi.org/10.21037/mhealth-19-250

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/mhealth-19-250). The series “Real-Time Detection and Management of Chronic Illnesses” was commissioned by the editorial office without any funding or sponsorship. MRF serves as an unpaid Guest Editor of the series and serves as an unpaid editorial board member of mHealth from Aug 2018 to Jul 2020. MRF reports grants from Louis and Rachel Rudin Foundation Interdisciplinary Pilot Project Award, from NIH/2017 Palliative Care Research Cooperative Group Investigator Development Pilot Award (funded by National Institute of Nursing Research U24NR014637); from New York University Research Challenge Fund, and from John A. Harford Foundation Change AGEnts Action Award, during the conduct of the study. BLE reports grants from USA. National Institutes of Health/National Cancer Institute (P30CA006927 Fox Chase Cancer Center Support Grant), during the conduct of the study. JHVC reports grants from Louis and Rachel Rudin Foundation Interdisciplinary Pilot Project, from 2017 Palliative Care Research Cooperative Group Investigator Development Pilot (funded by National Institute of Nursing Research U24NR014637), grants from NYU University Research Challenge Fund, grants from John A. Harford Foundation Change AGEnts Action Award, during the conduct of the study. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was reviewed and approved by the NYU Perlmutter Cancer Center Protocol Review and Monitoring Committee and NYU Langone Health School of Medicine Institutional Review Board (s16-01308, s16-02037) and informed consent was taken from all individual participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Siegel R, Naishadham D, Jemal A. Cancer statistics, 2013. CA Cancer J Clin 2013;63:11-30. [Crossref] [PubMed]
2. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2019. CA Cancer J Clin 2019;69:7-34. [Crossref] [PubMed]
3. Chaturvedi AK, Engels EA, Pfeiffer RM, et al. Human papillomavirus and rising oropharyngeal cancer incidence in the United States. J Clin Oncol 2011;29:4294-301. [Crossref] [PubMed]
4. National Cancer Institute. Cancer Stat Facts: Oral Cavity and Pharynx Cancer. Available online: http://seer.cancer.gov/statfacts/html/oralcav.html
5. Chaturvedi AK, Engels EA, Anderson WF, et al. Incidence trends for human papillomavirus-related and -unrelated oral squamous cell carcinomas in the United States. J Clin Oncol 2008;26:612-9. [Crossref] [PubMed]
6. Ringash J. Survivorship and quality of life in head and neck cancer. J Clin Oncol 2015;33:3322-7. [Crossref] [PubMed]
7. National Comprehensive Cancer Network (NCCN). Head and Neck Cancers version 3.2019. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines). 2019.
8. Rosenthal DI, Mendoza TR, Fuller CD, et al. Patterns of symptom burden during radiotherapy or concurrent chemoradiotherapy for head and neck cancer: a prospective analysis using the University of Texas MD Anderson Cancer Center Symptom Inventory-Head and Neck Module. Cancer 2014;120:1975-84. [Crossref] [PubMed]
9. Vasconcelos RM, Sanfilippo N, Paster BJ, et al. Host-microbiome cross-talk in oral mucositis. J Dent Res 2016;95:725-33. [Crossref] [PubMed]
10. Viet CT, Corby PM, Akinwande A, et al. Review of preclinical studies on treatment of mucositis and associated pain. J Dent Res 2014;93:868-75. [Crossref] [PubMed]
11. Elting LS, Keefe DM, Sonis ST, et al. Patient-reported measurements of oral mucositis in head and neck cancer patients treated with radiotherapy with or without chemotherapy: demonstration of increased frequency, severity, resistance to palliation, and impact on quality of life. Cancer 2008;113:2704-13. [Crossref] [PubMed]
12. Elting LS, Cooksley CD, Chambers MS, et al. Risk, outcomes, and costs of radiation-induced oral mucositis among patients with head-and-neck malignancies. Int J Radiat Oncol Biol Phys 2007;68:1110-20. [Crossref] [PubMed]
13. Trotti A, Bellm LA, Epstein JB, et al. Mucositis incidence, severity and associated outcomes in patients with head and neck cancer receiving radiotherapy with or without chemotherapy: a systematic literature review. Radiother Oncol 2003;66:253-62. [Crossref] [PubMed]
14. Murphy BA, Beaumont JL, Isitt J, et al. Mucositis-related morbidity and resource utilization in head and neck cancer patients receiving radiation therapy with or without chemotherapy. J Pain Symptom Manage 2009;38:522-32. [Crossref] [PubMed]
15. Alfieri S, Ripamonti C, Marceglia S, et al. Temporal course and predictive factors of analgesic opioid requirement for chemoradiation-induced oral mucositis in oropharyngeal cancer. Head Neck 2016;38:E1521-7. [Crossref] [PubMed]
16. Hanna EY, Mendoza TR, Rosenthal DI, et al. The symptom burden of treatment-naive patients with head and neck cancer. Cancer 2015;121:766-73. [Crossref] [PubMed]
17. Murphy BA, Deng J. Advances in supportive care for late effects of head and neck cancer. J Clin Oncol 2015;33:3314-21. [Crossref] [PubMed]
18. Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA 2017;318:197-8. [Crossref] [PubMed]
19. Denis F, Basch E, Septans AL, et al. Two-year survival comparing web-based symptom monitoring vs routine surveillance following treatment for lung cancer. JAMA 2019;321:306-7. [Crossref] [PubMed]
20. Berry DL, Hong F, Halpenny B, et al. Electronic self-report assessment for cancer and self-care support: Results of a multicenter randomized trial. J Clin Oncol 2014;32:199-205. [Crossref] [PubMed]
21. Kolb NA, Smith AG, Singleton JR, et al. Chemotherapy-related neuropathic symptom management: a randomized trial of an automated symptom-monitoring system paired with nurse practitioner follow-up. Support Care Cancer 2018;26:1607-15. [Crossref] [PubMed]
22. Yang LY, Manhas DS, Howard AF, et al. Patient-reported outcome use in oncology: A systematic review of the impact on patient-clinician communication. Support Care Cancer 2018;26:41-60. [Crossref] [PubMed]
23. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 2013;13:211. [Crossref] [PubMed]
24. Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manag Sci 2000;46:186-204. [Crossref]
25. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989;13:319-40. [Crossref]
26. STROBE. STROBE checklists, version 4 as published in Oct/Nov 2007. Available online: https://www.strobe-statement.org/index.php?id=available-checklists
27. Medicine PLOS. Observational studies: getting clear about transparency. PLoS Med 2014;11:e1001711. [Crossref] [PubMed]
28. Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 1989;35:982-1003. [Crossref]
29. Van Cleave JH, Fu MR, Bennett AV, et al. The development, usability, and reliability of the Electronic Patient Visit Assessment (ePVA) for head and neck cancer. mHealth 2019;5:21. [Crossref] [PubMed]
30. Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993;85:365-76. [Crossref] [PubMed]
31. Lenz ER, Pugh LC, Milligan RA, et al. The middle-range theory of unpleasant symptoms: an update. ANS Adv Nurs Sci 1997;19:14-27. [Crossref] [PubMed]
32. Fu MR, McDaniel RW, Rhodes VA. Measuring symptom occurrence and symptom distress: development of the symptom experience index. J Adv Nurs 2007;59:623-34. [Crossref] [PubMed]
33. Feinstein AR. Symptoms as an index of biological behaviour and prognosis in human cancer. Nature 1966;209:241-5. [Crossref] [PubMed]
34. Leidy NK. Functional status and the forward progress of merry-go-rounds: toward a coherent analytical framework. Nurs Res 1994;43:196-202. [Crossref] [PubMed]
35. Kuder GF, Richardson MW. The theory of the estimation of test reliability. Psychometrika 1937;2:151-60. [Crossref]
36. Wilson IB, Cleary PD. Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA 1995;273:59-65. [Crossref] [PubMed]
37. Aarstad HJ, Østhus AA, Aarstad HH, et al. General health-related quality of life scores from head and neck squamous cell carcinoma patients obtained throughout the first year following diagnosis predicted up to 10-year overall survival. Eur Arch Otorhinolaryngol 2018;275:207-17. [Crossref] [PubMed]
38. Creswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. 2nd ed. Los Angeles: Sage, 2011.
39. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005;15:1277-88. [Crossref] [PubMed]
40. Sim I. Mobile devices and health. N Engl J Med 2019;381:956-68. [Crossref] [PubMed]
41. Basch E, Deal AM, Kris MG, et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016;34:557-65. [Crossref] [PubMed]
42. Cooley ME, Abrahm JL, Berry DL, et al. Algorithm-based decision support for symptom self-management among adults with cancer: results of usability testing. BMC Med Inform Decis Mak 2018;18:31. [Crossref] [PubMed]
43. Blackford AL, Wu AW, Snyder C. Interpreting and acting on PRO results in clinical practice: lessons learned from the PatientViewpoint system and beyond. Med Care 2019;57 Suppl 5 Suppl 1:S46-51.
44. Veinot TC, Mitchell H, Ancker JS. Good intentions are not enough: How informatics interventions can worsen inequality. J Am Med Inform Assoc 2018;25:1080-8. [Crossref] [PubMed]
45. Parikh RB, Schwartz JS, Navathe AS. Beyond genes and molecules - a precision delivery initiative for precision medicine. N Engl J Med 2017;376:1609-12. [Crossref] [PubMed]