Suicide risk assessment training using an online virtual patient simulation

Introduction

Suicide is a significant public health problem in the United States; in 2017 47,173 deaths were classified as suicides, suicide rates increased in nearly every state from 1999 through 2016 (1), and it remains the 10^th leading cause of death in the US. Furthermore, suicides and self-harm injuries cost the nation approximately $70 billion per year in direct medical and work loss costs (2). Recognizing that preventing suicide takes an aggressive, multifaceted, multicomponent approach, and acknowledging the evolving evidence that exists for effective practices, the U.S. Surgeon General and the National Action Alliance for Suicide Prevention updated and released the revised National Strategy for Suicide Prevention (NSSP)^. The revised NSSP includes Goals 8 and 9 which focus explicitly on health care as a setting to reduce suicides.

Data suggest that health care can and must play a more active role in preventing suicide. Approximately 83% of people who die by suicide have seen a health care provider in the year before their death (3), and 45% had contact with their primary care provider in the month before their death (4). More individuals with behavioral health concerns, including suicide, are seen in primary care than in specialty behavioral health (5). Likely due to lack of training, limited support, weak or nonexistent protocols, and packed schedules, primary care providers are missing opportunities to identify and respond to those at risk.

Despite the frequency of these encounters and the significance of suicide risk, many health and behavioral health professionals still do not receive the necessary training and/or have the confidence needed to effectively interact with suicidal individuals (6). Silva et al. (7) found that about 53% of behavioral health care professionals across seven states reported no previous suicide prevention or assessment training. A self-report study from Roush et al. (8) identified that over 30% of mental health professionals did not ask every patient about suicidal thoughts or behaviors in first visits. Despite the evidence that now exists for interventions that work—standardized and routine screening and assessment (9), collaborative safety planning (10), reducing access to lethal means (11), treatment that directly targets suicidal thoughts (12), and follow-up during acute care transitions to reduce suicide (13)—they are vastly underutilized. However, only a small percentage of health care systems in the U.S. to date have adopted, trained staff on, and embedded these best practices (14).

This lack of expertise impacts the ability of practitioners to provide comprehensive quality care for individuals at risk for suicide (15). Recognizing the variability that exists among health care professionals in delivering safe and effective suicide care, the National Action Alliance released Recommended Standard Care for People with Suicide Risk. These recommendations, one of which is screening and assessment in primary care for those with known key risk factors for suicide, are low-cost, feasible, high-value, and evidence-based. However, as previously stated, practitioners often never receive training or support in how to properly screen or assess for suicide, and consequently, patients are missed. Research has shown that once behavioral health providers receive evidence-based training in suicide prevention, they are more likely to self-report increased confidence in working with people at risk for suicide (16).

Comprehensive training for suicide risk assessment goes beyond asking questions on an assessment form. Merely asking about the presence of suicidal thoughts does not capture the nuance of suicide risk, and risk may be imminent even if it is not communicated to the provider due to a variety of factors (e.g., stigma, chaotic setting, or fear of hospitalization) (17). Therefore, it is critical that practitioners are trained to build rapport and provide compassionate care to people in crisis to provide a safe and open dialogue for people to talk about their suicidal thoughts and behaviors. Once a provider has gathered the information elicited in a suicide risk assessment (i.e., prior attempts, risk factors, precipitating events), a risk formulation is necessary to understand the way the acute, chronic, and historical risk factors combine to inform the collaborative clinical decision-making process (18). The complexity of providing suicide care indicates that innovative training techniques are needed to increase the ability of practitioners to confidently and effectively identify and intervene with patients who may be at risk for suicide.

Online virtual patient simulations (VPS) allow practitioners the opportunity to experience working with patients in a safe, risk-free environment. When used as a training modality, VPS can be particularly useful because they allow for repetition in practice with difficult scenarios a practitioner may face, which can help to develop critical knowledge and expertise (19-22). VPS can be used to teach complex skills and provide opportunities for the practice, as well as assessment, of skills (23). In-person role-plays and live practice sessions with standardized patients are widely used in health education for training conversation-based skills. However, there are barriers to this modality, such as the cost of bringing in trainers or paid actors for in-person trainings, logistics of coordinating in-person trainings with busy providers, and productivity losses when staff are taken away from patient care to complete trainings. Another drawback to in-person role-plays and live practice is that participants often experience discomfort and fear practicing in front of their peers (20).

VPS training is a modality which allows providers to practice communication with patients without fear of judgment, which is critical to increasing a provider’s comfort around talking about the subject of suicide with their patients. Simulation-based training has demonstrated feasibility in teaching suicide-related knowledge and skills (24), and has been shown to increase the perceived ability to identify patients at risk for suicide and engage them in treatment planning (25), as well as the preparedness, likelihood, and self-efficacy to identify individuals experiencing psychological distress such as depression and suicide ideation (26). The purpose of this study was to test the feasibility and acceptability of a simulation-based training—a novel online VPS developed using SIMmersion^® PeopleSim^® technology—as well as assess pre-post changes in suicide-related knowledge from a pilot of the VPS with 20 practitioners.

Methods

Design

Prior to recruitment, this study was submitted for review to the Institutional Review Board (IRB) at the Institute for Family Health which deemed the project (protocol #2288) to be exempt according to 45CFR46.102(d). We recruited practitioners (n=20) who work at a Federally Qualified Health Center (FQHC) in the northeastern United States prior and subsequent to their use of the VPS. The FQHC provides fully integrated primary and behavioral health care to over 100,000 patients a year, regardless of ability to pay, including patients who are uninsured. These practitioners included physicians (n=9), social workers (n=4), nurses (n=3), practitioners who identified their role as “other” (n=2), and practitioners for whom this information was missing (n=2). For the study requirements, participants completed an online program that consisted of the following components: demographics questionnaire, knowledge pre-test, two role-play sessions with the VPS, knowledge post-test, and the TEQ. Upon completion of the program, participants received a $100 stipend.

To evaluate feasibility, we calculated the proportion of recruited participants who used the VPS. To evaluate acceptability, we asked participants to rate their experiences with the VPS using a Training Evaluation Questionnaire (TEQ) which included nine items rated on a 7-point Likert scale (α =0.74). Examples of items in this scale include “How easy was the training to use?” (range: extremely easy to extremely difficult), “How helpful was the training?” (range: extremely helpful to extremely unhelpful), and “How relevant was the training?” (range: extremely relevant to extremely irrelevant). To assess suicide-related knowledge, we developed two parallel knowledge assessments, each with ten multiple-choice questions. Five questions from each test were written to assess participants’ subject knowledge and focused on these content areas: primary care initiative for suicide prevention, goals of a suicide risk assessment, structure of a risk assessment, suicide risk assessment topics, and assessment follow-up. An example of a question from this assessment includes “The goal of suicide assessment is to:” with the following choices: “(I) identify individuals who may be at risk for suicide, (II) determine if a patient has depression and prescribe medication, (III) confirm ideation and convince patient to seek immediate in-patient treatment, and (VI) confirm suspected risk and estimate immediate danger”. The last five questions focused on conversation knowledge and covered these areas: rapport, gathering details, and understanding patient language. An example of a question from this assessment includes “All of the following are elements that help build rapport with a patient who is thinking about suicide, except:” with the following choices: (I) assuring the patient that you can relate to his/her struggles, (II) creating a collaborative relationship, (III) demonstrating empathy, and d. avoiding judgement.

Data analysis

Descriptive statistics were examined to analyze participant demographics, feasibility, and acceptability (TEQ). To assess change in knowledge, we conducted a paired samples t-test to compare mean differences in practitioners’ scores from pre- to post-training. Analyses were conducted in SPSS Statistics, version 22 (Advanced Statistics).

VPS: suicide risk assessment training

The VPS in this study, “Suicide Prevention: Assessing Risk with Taye Banks,” uses SIMmersion^® PeopleSim^® technology to train practitioners. The PeopleSim^® technology provides learners with the opportunity to interact with a video-recorded actor by selecting from hundreds of questions and statements, allowing the conversation to flow naturally. This variety also ensures that there is no one “right” or “wrong” choice at any point in the conversation, moving users beyond judgment training (i.e., Which of the statements is best?) to conversational skill development (i.e., What should I say next?). Non-branching logic creates dynamic links between the user questions and character responses, allowing learners to try new approaches and experience different outcomes each time the simulation is used. In addition, learners receive real-time support and feedback from an on-screen coach who provides feedback and insights into what the virtual client is thinking.

The VPS tested in this study aims to train practitioners to more effectively interact with at-risk individuals in the health care setting by allowing them to practice talking with a fictional patient, 20-year-old Taye Banks, who self-identified that she was thinking about suicide (Figure 1). In the simulation, Taye comes to the clinic to obtain a note saying she is cleared to return to work. The role of the learner/practitioner in the VPS training is to encourage Taye to share details about her thoughts in an effort to assess Taye’s immediate risk of suicide, so appropriate next steps to keep Taye safe can be identified (Note: development of a suicide care management plan is not included in this VPS and is planned for future modules). To facilitate an effective conversation, the learner/practitioner must identify Taye’s existing stressors and protective factors; inquire about Taye’s current suicidal thoughts including if she has made a plan to kill herself, selected a method, has access to her chosen method and/or other lethal means, and has intent to kill herself; determine the frequency, duration, and controllability of Taye’s suicide ideation; and discuss past suicide ideation. Additionally, the learner/practitioner must build enough trust and rapport that Taye feels comfortable sharing these details. To build that rapport, they must validate Taye’s experiences and feelings; demonstrate empathy and compassion; avoid passing judgement on Taye’s thoughts or actions; develop a feeling of collaboration with Taye; practice active listening; and utilize key motivational interviewing techniques of open-ended questions, affirmations, reflections, and summaries.

Assessing Risk with Taye Banks has three components: a text-based introduction with scenario information and review materials, a simulated conversation with Taye that changes each time a new conversation begins, and integrated feedback that provides both in-the-moment and after-action guidance. After reviewing the introduction, learners/practitioners begin the conversation, and the simulation randomly selects one of four different versions of Taye Banks (based on risk level) to present to the practitioner. Although Taye’s basic information and reason for coming to the clinic will remain the same, the number of stressors, protective factors in Taye’s life, and the frequency and intensity of her ideation change depending on her risk level. While, initially, Taye’s willingness to disclose her stressors is inversely correlated with her risk level, the questions the learner/practitioner asks will impact her willingness to share. This correlation was developed to emulate patient behavior and was recommended by both clinical experts and experts with lived experience who identified that individuals at lower risk-levels are typically more willing to share than higher-risk individuals who have often experienced negative reactions (e.g., judgment, minimization, or hospitalization) from health care providers. As they build rapport, as described above, Taye becomes more open about her thoughts of suicide, and if they hurt rapport, Taye becomes more withdrawn and may even end the conversation. The on-screen coach provides non-verbal and text-based guidance to help improve their skills (Figure 1). Following the conversation, learners/practitioners answer a series of questions that allow them to practice documenting the conversation in a patient chart, and then they receive both quantitative and qualitative feedback for the objectives.

Figure 1 Screenshot of virtual patient simulation, “Assessing Risk with Taye Banks”.

Results

Table 1 describes the demographics of the sample of practitioners. Of the initial sample of 20, 14 participants completed the full VPS training, four participants began, but did not finish the protocol, and two participants never logged on to begin the protocol. Of the participants who began the protocol, 15 (83.3%) had prior suicide prevention training and 14 (77.8%) had prior training specifically related to suicide assessments. With respect to feasibility, participants used the VPS for 21 to 95 minutes, with an average time of 45.88 minutes. As indicated in Table 2, in regards to acceptability, participants rated the training favorably on all nine questions with an average score on the 7-point Likert-scale of 5.82 and ranging from 5.29 to 6.23. Pre-test scores ranged from 4 to 8 (out of 10) and post-test scores ranged from 6 to 9 (out of 10). As shown in Table 3, there was an average gain of 1.86 points on the knowledge scale among the practitioners in the sample from pre- to post-training. According to the t-test analysis the difference in scores was statistically significant [t (df =13) = −6.32, P<0.001)].

Table 1 Demographics (n=18)
Full table

Table 2 Training Experience Questionnaire summary (n=14)
Full table

Table 3 Participant knowledge assessment scores (n=18)
Full table

Discussion

The VPS tested in this study was feasible and acceptable to the pilot sample of practitioners, as the majority of practitioners completed the VPS, and those who complete the VPS reported satisfaction with the training experience. As evidenced by the statistically significant gains in practitioners’ skills in suicide risk assessment, the results offer preliminary evidence that the training model may be an effective way to teach practitioners how to identify and intervene with patients who may be at risk for suicide. The VPS tested in this study is particularly innovative in its ability to train practitioners in how to build rapport and provide compassionate care to people in crisis. This is especially critical in light of the National Action Alliance Suicide Attempt Survivors Task Force’s recommendations from their report, The Way Forward, which outlines best practices for supporting suicidal individuals, including promoting collaborative care, respecting the dignity of a person in crisis, and countering stigma, shame, and discrimination.

The increase in suicide-related knowledge following participation in the VPS training is particularly notable given the extensive training required by the FQHC for behavioral health providers. Providers within the behavioral health department are already required to take a full-day training on Assessing and Managing Suicide Risk (AMSR) in addition to completing online modules on the Columbia Suicide Severity Rating Scale (C-SSRS), safety planning, counseling on access to lethal means, and providing structured follow-up to patients expressing suicide ideation. These trainings are typically completed within the first three months of employment. Among primary care providers, nurses, and medical residents at this FQHC, training in suicide prevention is less formal and extensive. When a patient is in need of more thorough assessment, safety planning, or means restriction counseling, a behavioral health provider is typically sought. Thus, a potential strength of this simulation is that it may increase knowledge in suicide risk assessment across a broad range of provider experience with suicide prevention.

Release of compliance standards for suicide care have recently been accelerated (27-29), making suicide risk assessment training of utmost importance. In November 2018, Joint Commission released revised national Patient Safety Goal 15.01.01 (30) that will be effective July 1, 2019 to screen all suicidal patients using a validated screening tool. Suicide care is now on the forefront of professional organizations such as American Association of Pediatrics (AAP) and the American Medical Association (AMA). For instance, the AMA recently adopted Resolution 312 which states that the AMA will “engage with the appropriate organizations to facilitate the development of educational resources and training related to suicide risk of patients, medical students, residents/fellows, practicing physicians, and other health care professionals, using an evidence-based multidisciplinary approach” (31). As such, health care providers, including primary care practitioners, are held accountable to use effective tools and interventions for suicide prevention.

Now that the use of evidence-based approaches for suicide prevention has become a priority among professional agencies, finding effective ways to train an optimal number of practitioners is essential. VPS represent one such training that is easily accessible and allows for repetition in skill building as well as a safe space to practice difficult interactions with patients. Our findings which demonstrate the feasibility, acceptability, and preliminary effects of VPS are tempered by some limitations. Four participants did not complete the satisfaction questionnaire, the pre-post knowledge questionnaire tested gains in knowledge vs. skills, and we do not have evidence of how this training impacted patient outcomes. Building on this research, we are developing additional VPS modules to allow practitioners to practice other suicide prevention skills such as developing safety plans, minimizing access to lethal means, and increasing motivation to engage in treatment. Future research will evaluate the effect of this suite of VPS trainings on practitioner behaviors and patient outcomes in a fully-powered historically controlled trial.

Acknowledgments

Funding: This work was supported by the National Institute on Mental Health of the National Institutes of Health under Award Number R44MH114710. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Footnote

Conflicts of Interest: KHM O'Brien, S Fuxman, N Tirone, and J Goldstein Grumet are employed by Education Development Center and L Humm is employed by and owns shares in SIMmersion, LLC. Both Education Development Center and SIMmersion may benefit from sales of the product once commercialized. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was submitted for review to the Institutional Review Board (IRB) at the Institute for Family Health which deemed the project (protocol #2288) to be exempt according to 45CFR46.102(d).

References

1. Stone DM, Simon TR, Fowler KA, et al. Vital signs: trends in state suicide rates—United States, 1999–2016 and circumstances contributing to suicide—27 states, 2015. MMWR Morb Mortal Wkly Rep 2018;67:617. [Crossref] [PubMed]
2. WISQARS (Web-based Injury Statistics Query and Reporting System. Center for Disease Control Injury Center 2018. Available online: https://www.cdc.gov/injury/wisqars/index.html
3. Ahmedani BK, Simon GE, Stewart C, et al. Health care contacts in the year before suicide death. J Gen Intern Med 2014;29:870-7. [Crossref] [PubMed]
4. Abed Faghri NM, Boisvert CM, Faghri S. Understanding the expanding role of primary care physicians (PCPs) to primary psychiatric care physicians (PPCPs): enhancing the assessment and treatment of psychiatric conditions. Ment Health Fam Med 2010;7:17. [PubMed]
5. Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry 2002;159:909-16. [Crossref] [PubMed]
6. National Action Alliance for Suicide Prevention: Clinical Workforce Preparedness Task Force. Suicide Prevention and the Clinical Workforce: Guidelines for Training. National Action Alliance for Suicide Prevention 2014. Available online: https://theactionalliance.org/resource/suicide-prevention-and-clinical-workforce-guidelines-training
7. Silva C, Smith AR, Dodd DR, et al. Suicide-related knowledge and confidence among behavioral health care staff in seven states. Psychiatr Serv 2016;67:1240-5. [Crossref] [PubMed]
8. Roush JF, Brown SL, Jahn DR, et al. Mental health professionals' suicide risk assessment and management practices. Crisis 2018;39:55-64. [Crossref] [PubMed]
9. Mann JJ, Apter A, Bertolote J, et al. Suicide prevention strategies: a systematic review. JAMA 2005;294:2064-74. [Crossref] [PubMed]
10. Stanley B, Brown GK, Brenner LA, et al. Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department. JAMA Psychiatry 2018;75:894-900. [Crossref] [PubMed]
11. Zalsman G, Hawton K, Wasserman D, et al. Suicide prevention strategies revisited: 10-year systematic review. Lancet Psychiatry 2016;3:646-59. [Crossref] [PubMed]
12. Stanley B, Brown G, Brent DA, et al. Cognitive-behavioral therapy for suicide prevention (CBT-SP): treatment model, feasibility, and acceptability. J Am Acad Child Adolesc Psychiatry 2009;48:1005-13. [Crossref] [PubMed]
13. Luxton DD, June JD, Comtois KA. Can postdischarge follow-up contacts prevent suicide and suicidal behavior? Crisis 2013;34:32-41. [Crossref] [PubMed]
14. Bongar, B. The suicidal patient: Clinical and legal standards of care (3rd ed). Washington, DC: American Psychological Association, 2013.
15. Schmitz WM Jr, Allen MH, Feldman BN, et al. Preventing suicide through improved training in suicide risk assessment and care: An American Association of Suicidology Task Force report addressing serious gaps in US mental health training. Suicide Life Threat Behav 2012;42:292-304. [Crossref] [PubMed]
16. LoParo D, Florez IA, Valentine N, et al. Associations of suicide prevention trainings with practices and confidence among clinicians at community mental health centers. Suicide Life Threat Behav 2019;49:1148-56. [Crossref] [PubMed]
17. Silverman MM, Berman AL. Suicide risk assessment and risk formulation part I: A focus on suicide ideation in assessing suicide risk. Suicide Life Threat Behav 2014;44:420-31. [Crossref] [PubMed]
18. Berman AL, Silverman MM. Suicide risk assessment and risk formulation part II: Suicide risk formulation and the determination of levels of risk. Suicide Life Threat Behav 2014;44:432-43. [Crossref] [PubMed]
19. Fleming M, Olsen D, Stathes H, et al. Virtual reality skills training for health care professionals in alcohol screening and brief intervention. J Am Board Fam Med 2009;22:387-98. [Crossref] [PubMed]
20. Albright G, Adam C, Serri D, et al. Harnessing the power of conversations with virtual humans to change health behaviors. mHealth 2016;2:44. [Crossref] [PubMed]
21. Lateef F. Simulation-based learning: Just like the real thing. J Emerg Trauma Shock 2010;3:348. [Crossref] [PubMed]
22. Urresti-Gundlach M, Tolks D, Kiessling C, et al. Do virtual patients prepare medical students for the real world? Development and application of a framework to compare a virtual patient collection with population data. BMC Med Educ 2017;17:174. [Crossref] [PubMed]
23. Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Medical Education 2009;43:303-11. [Crossref] [PubMed]
24. Lancaster PG, Moore JT, Putter SE, et al. Feasibility of a web-based gatekeeper training: Implications for suicide prevention. Suicide Life Threat Behav 2014;44:510-23. [Crossref] [PubMed]
25. Albright G, Bryan C, Adam C, et al. Using Virtual Patient Simulations to Prepare Primary Health Care Professionals to Conduct Substance Use and Mental Health Screening and Brief Intervention. J Am Psychiatr Nurses Assoc 2018;24:247-59. [Crossref] [PubMed]
26. Bartgis J, Albright G. Online role-play simulations with emotionally responsive avatars for the early detection of Native youth psychological distress, including depression and suicidal ideation. Am Indian Alsk Native Ment Health Res 2016;23:1-27. [Crossref] [PubMed]
27. The Joint Commission. Sentinel Event Alert, Issue 56: Detecting and Treating Suicide Ideation in All Settings 2016. Available online: https://www.jointcommission.org/assets/1/18/SEA_56_Suicide.pdf
28. Commission on Accreditation of Rehabilitation Facilities. Quality Practice Notice: Suicide Prevention in CARF-Accredited Organizations: Advancing Clinical and Service Workforce Preparedness 2016. Available online: http://www.carf.org/QPN_SuicidePrevention/
29. Council on Accreditation. Release of Updated Standards Promoting Best Practices in Suicide Prevention. Available online: http://coanet.org/about/whatsnew/news-detail/article/112/
30. Joint Commission announces new National Patient Safety Goal to prevent suicide and improve at-risk patient care. Joint Commission News Details, 2018. Available online: https://www.jointcommission.org/joint_commission_announces_new_national_patient_safety_goal_to_prevent_suicide_and_improve_at-risk_patient_care/
31. Increasing awareness of suicide risks to save lives. Press Releases. American Medical Association, 2018. Available online: https://www.ama-assn.org/press-center/press-release/increasing-awareness-suicide-risks-save-lives