Assessing the feasibility of a chatbot after ureteroscopy

Introduction

Each year, hundreds of thousands of patients undergo ureteroscopy (1), a common procedure whereby urologists use a flexible endoscope to treat kidney stones. Approximately, 18% of patients who undergo a ureteroscopy will have an unplanned visit either to the emergency department or with their provider shortly after their procedure (2). Many of these follow-up visits are for common, and expected, post-operative symptoms such as flank pain, hematuria, and dysuria (3). Though common, these symptoms can still concern patients. While some concerns may be addressed through preoperative counseling and post-operative discharge materials, technologies such as “chatbots” have the potential to improve the dissemination of information in a convenient and timely manner. A chatbot is an automatic text-messaging tool that communicates information to patients based on questions they submit. Unlike traditional paper-based discharge instructions, the chatbot’s interface allows providers to present large amounts of detailed information in a succinct, on-demand, manner.

Despite the potential benefits, it is unclear if patients will accept and use chatbots after kidney stone surgery. While it seems likely that patients will find the chatbot useful to address post-op concerns (e.g., pain and/or discomfort associated with the placement of a stent, painful urination, bleeding from a stent), patients may not use the chatbot for several reasons. First, patients may have other resources (e.g., Google search, written post-operative instructions) be more readily available. Second, patients may struggle to navigate the technology. Finally, patients may not find the information in the chatbot useful.

While chatbots have not been widely studied, other researchers have evaluated the use of mHealth technologies for post-operative care. For example, a team of investigators evaluated the use of a smartphone application to improve monitoring of lung transplant recipients post-transplant (4). They found that patients who used the smartphone application have better self-reported, health-related quality of life and self-management behaviors than those who received traditional discharge instructions. Others have looked at the use of smartphone applications for home-monitoring of symptoms and symptom severity for patients who underwent an ambulatory lumbar discectomy (5); they found that self-monitoring via this application was well received, and it minimized the need for in-person visits for post-operative patients.

We build on these prior studies by sharing our experience with a pilot study where we used a chatbot to address common symptoms during patients’ recovery after ureteroscopy. The results of our study will help inform how well patients engage with chatbots and how they can be used to improve the delivery of post-operative care.

Methods

The aim of this pilot study was to understand the feasibility of using a chatbot for recovery after ureteroscopy. To develop the chatbot, we interviewed nurses and physicians and reviewed patient education materials. The information gathered provided the tool’s content, and a design team constructed the patient interface for the chatbot.

To use the chatbot, patients were instructed to text a number located on a custom-designed business card provided to patients after the procedure. In response, they received a welcome message and link to click to begin a dialogue with the chatbot. They were then asked to select from a list of commonly experienced symptoms or complications after the procedure. The chatbot then provided a detailed answer along with a continued set of responses to provide and receive more information (Figure 1).

Figure 1 A mock up of the chatbot interface.

Sampling, study setting, and recruitment

We identified a convenience sample of twenty adult patients who underwent ureteroscopy for nephrolithiasis at the University of Michigan from January 25, 2018 to September 5, 2018. Prior to starting the pilot, we determined that interviews with twenty patients should be sufficient for us to answer the questions in our interview guide. Patients who received the chatbot were contacted 1–4 weeks post-surgery and asked if they were willing to participate in an interview.

Data collection and analysis

Patients who agreed to participate underwent a semi-structured telephone interview by a single researcher (Steven B. Goldenthal) using an interview guide. During the semi-structured interview, we assessed a variety of aspects related to patients’ post-surgical recovery. We assessed each patient’s overall impressions of their recovery and resources used to address concerns during their recovery. We asked patients if they used the chatbot, and if they did not, why. If they indicated that they used the chatbot, we assessed whether patients felt that it met their needs, and if they were amenable to receiving future post-surgical care via a chatbot. The interview guide is provided as a figure in the Supplement I. Patient responses were documented in notes, and three authors (Emma Steppe, Steven B. Goldenthal, Chad Ellimoottil) reviewed the notes to identify common themes across the post-operative experiences.

IRB statement

This study was deemed exempt from IRB review because it was a quality improvement initiative.

Results

Over the course of the pilot, we contacted and interviewed 20 patients provided with the chatbot after undergoing a ureteroscopy. Patients interviewed ranged from 31 to 79 years of age and lived anywhere from 2 to 247 miles away from our institution. Of these patients, 16 had a ureteral stent placed.

Symptoms after ureteroscopy

Patients identified a number of common symptoms and issues during recovery. These included, in order of frequency: discomfort and pain with stent in place (n=7); painful, bloody, and difficult urination (n=6); and infection or other systemic symptoms (n=1). The chatbot included all of these symptoms and related information in its response options.

Usage patterns of chatbot

Seven of the twenty patients activated the chatbot during recovery. Among patients who did not use the chatbot, reasons included misplacing the follow-up materials with instructions for chatbot use (n=6), relying on follow-up with the clinic or on discharge papers provided by the clinic (n=4), being unable to activate the chatbot (n=2), and not using text messaging (n=1).

Perceived benefits of using chatbot

One patient used the chatbot for information about painful urination after unsuccessfully trying to connect to personnel at the clinic. He stated that it alleviated his concerns about the pain he was experiencing. Another patient used the chatbot for moderate pain, but as the pain worsened, the patient didn’t use it again and instead called the clinic for more information. One patient outlined potential future uses for chatbot, indicating that it could be very useful for non-emergent issues through which the patient can receive an answer instantly and does not have to wait to connect to someone at the clinic. This patient also detailed a way to improve the patient experience with the chatbot, stating that it does not have an option to indicate when the patient has received the necessary information and desires to end the encounter. Rather, the chatbot returns to the beginning options for interaction. Select patient quotes about the chatbot are included in Figure 2.

Figure 2 Patient comments.

Discussion

According to our interviews, patients experience a common set of symptoms post-ureteroscopy, all of which were covered by the chatbot. Even though the chatbot provided information about a comprehensive set of symptoms, many patients chose not to activate the chatbot for a variety of reasons. Patients who did use the chatbot found it to be a convenient method for finding informative content related to their symptoms. However, they experienced some usability issues. Overall, chatbots appear to be a useful and convenient way of providing recovery information to patients post-ureteroscopy, but improved usability and more seamless integration into the flow of care are necessary to increase patient engagement.

Previous studies have found that patients commonly experience symptoms such as fever, flank pain, hematuria, and dysuria post-ureteroscopy (3). It is also common for patients who have a ureteral stent to experience pain or discomfort associated with the stent (6). The interviewed patients reported similar symptoms and appear to be consistent with the existing literature.

The benefits of using messaging tools to deliver health care across other specialties and in other industries has been well characterized. In psychotherapy, synchronous text-based interventions are a feasible and well tolerated (7) means for providing mental health care to patients. Additionally, messaging tools’ immediacy and availability make them convenient for patients. In the field of colorectal surgery, teams have evaluated the use of text messaging in post-operative care using standardized text messages to inquire about clinically relevant information (body temperature, signs of bowel obstruction/venous thromboembolic disease, pain, or absence of symptoms) on days one, three, and five post-surgery. Response rates were high, patients were very satisfied receiving care in this manner, and this system led to early, timely detection of postoperative complications (8).

Similar to these other use cases of chatbots, patients who did utilize the chatbot during their recovery found it valuable and indicated that they would use it following future procedures. However, most patients did not activate the chatbot, usually because it was one of many resources at their disposal. In light of these observations, integrating the chatbot more directly into patients’ post-operative care appears to be an opportunity to increase usage and enable patients to experience its value.

This study has several limitations. The first limitation is that we interviewed a small sample. However, this was an exploratory study to assess how patients respond to chatbots and identify themes in their recovery. The second limitation, which was unintentional, is that we had low chatbot activation and use among patients. Despite low levels of patient engagement with the chatbot, this study revealed the importance of improving its usability. If we continued this study, we would expect to see similarly low levels of use, so hope to improve the tool before continuing to evaluate its potential benefits in post-surgical care. Another limitation is that this was a single-institution study, meaning it may not be generalizable to other practices. However, our sample of patients included individuals from a wide variety of ages, locations, and technical backgrounds.

These limitations notwithstanding, our study has several implications. Chatbots have the potential to alleviate physician and staff burden to address common concerns after ureteroscopy. However, two key takeaways surrounding the current chatbot offering arose from our study. First, patient activation of the chatbot was low, so it will be necessary to increase patient awareness of the chatbot. Secondly, it is essential that providers ensure the chatbot is usable to patients and easily activated by or for patients at appropriate times during their recovery. Chatbots have the potential to provide significant information in a convenient and timely manner, keeping information readily available at patients’ fingertips. They also have the potential to alleviate post-operative concerns and eliminate healthcare waste, such as unnecessary follow up visits or calls to clinic. The information that we obtained from this study allows for the future improvement of chatbots post-surgery especially for patients undergoing kidney stone surgery.

Chatbots are increasingly adopted across many industries (9), including health care (10), and patients appear to be receptive to these new methods of care delivery. Automated text-based communication provides patients with easily accessible information regarding their care and another touch point to stay connected and engaged with their care team. Leveraging this technology in a patient-centered way has the potential to improve post-operative care delivery for patients and providers across a variety of specialties.

Supplement I Semi-structured Interview Guide

Do you think you had a good recovery after kidney stone surgery?

Do you feel like the instructions you were given after surgery were sufficient?

• Did you have to call the office for more instructions?
• Did you use other resources during your recovery?

What was the most uncomfortable part of your recovery?

Dr._____ gave you a chatbot. Do you remember the tool? Did you use it?

• If you did not use it, what were your reasons for not doing so?

If yes, do you feel that it addressed some of your needs during your recovery?

• In what ways was it helpful?
• In what ways was it not helpful?
• Did you use the internet to look up information regarding your recovery?
• Did you make additional follow-up appointments with Dr. _____ outside of your regularly scheduled follow-up visits?

How would you feel about receiving basic post-surgical care via a chatbot after future surgeries?

Acknowledgements

This work was supported by the Telehealth Research Incubator project grant (MPrOVE Research Challenge Grant).

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: This study was deemed exempt from IRB review because it was a quality improvement initiative. Verbal informed consent was obtained from the patients for publication of this manuscript and any accompanying images.

References

1. Urologic Diseases in America, NIDDK. Available online: https://www.niddk.nih.gov/about-niddk/strategic-plans-reports/urologic-diseases-in-america. Accessed August 2, 2018.
2. Scales CD Jr, Saigal CS, Hanley JM, et al. The impact of unplanned postprocedure visits in the management of patients with urinary stones. Surgery 2014;155:769-75. [Crossref] [PubMed]
3. Aboumarzouk OM, Kata SG, Keeley FX, et al. Extracorporeal shock wave lithotripsy (ESWL) versus ureteroscopic management for ureteric calculi. Cochrane Database Syst Rev 2012.CD006029. [PubMed]
4. DeVito Dabbs A, Song MK, Myers BA, et al. A Randomized Controlled Trial of a Mobile Health Intervention to Promote Self-Management After Lung Transplantation. Am J Transplant 2016;16:2172-80. [Crossref] [PubMed]
5. Debono B, Bousquet P, Sabatier P, et al. Postoperative monitoring with a mobile application after ambulatory lumbar discectomy: an effective tool for spine surgeons. Eur Spine J 2016;25:3536-42. [Crossref] [PubMed]
6. Joshi HB, Newns N, Stainthorpe A, et al. Ureteral stent symptom questionnaire: development and validation of a multidimensional quality of life measure. J Urol 2003;169:1060-4. [Crossref] [PubMed]
7. Hoermann S, McCabe KL, Milne DN, et al. Application of Synchronous Text-Based Dialogue Systems in Mental Health Interventions: Systematic Review. J Med Internet Res 2017;19:e267. [Crossref] [PubMed]
8. Carrier G, Cotte E, Beyer-Berjot L, et al. Post-discharge follow-up using text messaging within an enhanced recovery program after colorectal surgery. J Visc Surg 2016;153:249-52. [Crossref] [PubMed]
9. West E. Travel Bots: Are They Here to Stay? Business Travel News. April 2017:9-14. Available online: http://www.businesstravelnews.com/Technology/Travel-Bots-Are-They-Here-to-Stay
10. Chatbot Market Size And Share Analysis, Industry Report, 2014-2025. Available online: https://www.grandviewresearch.com/industry-analysis/chatbot-market. Accessed August 10, 2018.