Impact of an online depression prevention intervention on suicide risk factors for adolescents and young adults

Introduction

Suicide is the second leading cause of death among American adolescents (1). Nearly one in ten adolescents report having attempted suicide at least once (2), and one in seven report experiencing suicidal ideation (3). Additionally, there is a significant worry that current statistics underestimate suicidal attempts by adolescents (3). Adolescents who have attempted suicide show ongoing risk of death by suicide in adulthood and/or future mental health problems that affect functioning throughout adulthood (2).

Risk factors for adolescent suicide commonly include: prior suicidal behavior (3-6), exposure to death by suicide within ones peer group or family (3,5,6), hopelessness (7-11), low self-esteem (12,13), social isolation (8,14-17), and depression (3,5,11,18-20). Additionally, 21 percent of teenagers with symptoms of depression experience suicidal ideation, which is over three times the rate of adolescents without any symptoms of depression (21). These results indicate that while major depressive disorder is a major risk factor for suicide, individuals experiencing either early stages of the illness or lower levels of depressive symptoms are also at risk for suicidal thoughts and behaviors.

To date, very few studies have demonstrated decreased rates of deaths by suicide or suicide attempts by adolescents following intervention. Recent meta-analyses revealed a general difficulty in treating adolescent suicide at the population level (22-25). Interestingly, studies have shown that smartphone applications can reduce suicidal ideation in adult populations (26). Such apps along with internet sites are considered behavioral intervention technologies (BITs) (27) which can be implemented at a population level to reduce risk factors for suicide (28,29). Such implementation in a structured, efficient method, such as through Internet interventions, may reduce impractical costs associated with traditional methods of providing psychological support to large groups of individuals.

Internet interventions use digital and electronic technology to address cognitions and behaviors in an effort to improve mental and physical functioning and wellness (27). Internet interventions can be scaled up such that large numbers of users are engaging with the intervention simultaneously and can be disseminated globally at an extremely low cost (30,31). Interventions provided without human contact (via a facilitator, provider, or coach) are considered unsupported (31,32).

Competent Adulthood Transition with Cognitive-behavioral and Interpersonal Training (CATCH-IT) is an internet-based and minimally supported depression prevention program (31,32) that aims to provide a selective preventative intervention for adolescents with subclinical depression. CATCH-IT has been found to effectively reduce depressive symptoms at post-intervention and one-year follow-up (33-36) and yield high parental approval for the intervention (37). Additionally, it has been found to be cost-effective, costing about one third of comparable CBT groups targeting adolescent depression per client (38).

As part of a broader effort to curtail the rise of adolescent suicide, this study aimed to examine whether an online depression prevention program, CATCH-IT, can reduce risk factors for suicide such as suicidal ideation, hopelessness, social isolation, and low self-esteem in a cost-effective, broadly accessible manner.

Methods

Data was collected as part of a previously completed study that evaluated the effect of a brief motivational interview on completion of the CATCH-IT program, by comparing two groups of adolescents: (I) Group 1 received primary care physician motivational interview + CATCH-IT program and (II) Group 2 received brief advice + CATCH-IT program (33,35,36). In the present study, all analyses included the entire sample, without being separated by condition, to determine the overall efficacy of CATCH-IT on adolescent suicidal ideation and risk factors for suicide.

Participants

Participants were recruited from 13 primary care sites across four states in the United States South and Midwest regions. Youth between the ages of 14 and 21 were screened using the core depression symptoms items on the Patient Health Questionnaire-Adolescent (PHQ-A) (39), and those who expressed depressed mood, anhedonia, and/or irritability were contacted by phone for eligibility interviews that used the full PHQ-A.

Eligibility interviews were conducted 1 to 2 weeks following initial screening. Youth were given a monetary incentive (US $75–100, depending on the number of visits necessary to determine the presence of exclusion criteria) to participate. Because the initial data sample was collected to evaluate the effect of motivational interviews on CATCH-IT completion rates, participants were excluded if they reported frequent suicidal ideation or intent for safety reasons, or if they met full criteria for a depressive disorder. Past substance use was not an exclusionary criterion, as few adolescents endorsed use in the baseline questionnaire. Individuals who met criteria for a depressive disorder were excluded from the study and referred to treatment. In total, 83 youth aged 14–21 years [mean =17.5; standard deviation (SD) =2.04] were involved in this study, of whom 56.2% were female and 41% identified as an ethnicity other than Caucasian.

Materials and measures

CATCH-IT

The CATCH-IT program consists of 14 self-guided, online modules that use techniques from cognitive-behavioral therapy (CBT) and interpersonal psychotherapy to teach skills for increasing resiliency against depressive disorders and decreasing vulnerability to depression (31,33). The program did not specifically target aspects of depression or suicidality, instead focusing on CBT and interpersonal factors related to depression (e.g., pessimism, indirect communication). In focusing on reduction of behaviors related to depression, the program sought to decrease vulnerability to and increase protection from depression (33).

Baseline questionnaire

At both baseline and post-intervention, participants completed a Likert-type questionnaire during phone interviews. The measure assessed symptoms of depression and anxiety, social functioning, family functioning, academic functioning, perceived coping ability, and other relevant mental health disorders (33).

CES-D

At both baseline and post-intervention, participants completed the Center for Epidemiological Studies Depression Scale (CES-D) (40), a 20-item measure that assesses depression across several subscales (depressed mood, somatic, happy, and interpersonal). The CES-D has been validated on male and female adolescents with depression and demonstrated high sensitivity and specificity (41).

Risk factors for suicide variables

Risk factors for suicide included: suicidal ideation, hopelessness, low self-esteem and social isolation. These specific factors were selected due to opportunity for analysis in the sample—while others may also have been relevant in assessing risk for suicide (e.g., prior suicidal behavior), they occurred at a low rate within the sample, making analysis impossible. Risk factors for suicide were assessed by summing items at baseline and post-intervention questionnaires, along with responses to the CES-D, related to the relevant theoretical construct (i.e., suicidal ideation, hopelessness, low self-esteem, and social isolation). Items for suicidal ideation were selected with particular focus on desire to escape, in alignment with Wenzel and Beck’s (42) cognitive model of suicidal behavior, specifically cognitive processes associated with psychiatric disturbance and those associated with suicidal acts which combine to increase the likelihood of suicidal behavior (42). Hopelessness was developed using items associated with negative expectations of one’s capability to succeed in the future (7). Items used to create the sum score for low self-esteem include those focused on comparison of oneself to others and/or a sense of inadequacy or incompetence (12). Finally, items related to thwarted belongingness and disconnection to others made up the social isolation variable (43). The components of each sum score can be found in Table 1. Items were reverse coded when appropriate. Because items came from different Likert-type scales, they were standardized prior to being combined into sum score variables. Changes in risk factor variables were determined by subtracting post-intervention scores from baseline scores. Negative change scores reflected improvements in suicidal ideation and risk. Using guidelines from Gliem and Gliem (44), internal consistency of the risk factor variables was considered adequate for further analysis. See Table 2 for a complete list of Cronbach’s alpha values of the variables.

Table 1 Components of variables analyzed
Full table

Table 2 Internal consistency of variables used in analysis
Full table

Usage

Due to the high rate of attrition among other BITs (45), previous studies using the CATCH-IT program defined intervention dosage as the time spent on the website (e.g., story or survey pages), number of modules completed, percentage of questions answered, and number of characters typed into response boxes (35,46). In the present study, intervention usage was measured with three variables—the total number of modules completed, the amount of time spent on the website, and the number of characters typed into response boxes—consistent with previous CATCH-IT studies.

Twenty-two participants (26.5%), after being recommended CATCH-IT by their primary care provider, did not complete any program modules. Twenty participants (24.1%) completed between 1 and 5 modules. Seventeen participants (20.5%) completed between 6 and 13 modules. Twenty-four participants (28.9%) completed the entire 14-module CATCH-IT program (33,46). To compare the full effect of CATCH-IT on risk factors for suicide, participants who finished all 14 modules were classified as “CATCH-IT completers” and all other participants as “CATCH-IT non-completers”.

Procedure

Data was analyzed from a phase II clinical trial comparing brief advice versus motivational interviewing with CATCH-IT completion rates and related severity of depressive symptoms (33,46). Further description of the procedures and findings of the original study can be found in previous publications by the authors (33,35). The study found the program to be associated with lower scores on measures of depression following usage of CATCH-IT. In addition, those who received motivational interviewing by their primary care provider showed further improvements (33,35).

Analysis

A paired samples t-test compared baseline suicidal ideation to post-intervention suicidal ideation. A multivariable regression analysis was conducted using change in suicidal ideation as the dependent variable and changes in hopelessness, low self-esteem, and social isolation as the independent variables, controlling for baseline values of risk factors for suicide, age, and gender. Lastly, a multivariable linear regression analysis was conducted using change in suicidal ideation as the dependent variable and three usage variables (number of modules completed, total time spent on the website in seconds, and number of characters typed into response boxes) as the independent variables. The effect of time was accounted for by controlling the equation for baseline values of risk factors for suicide, age, and gender. Because of the potential overlap of items in risk and usage variables, multicollinearity tolerance was calculated in each regression analysis as well.

Results

The internal consistency of the variables of the risk factors for suicide were generally classified as acceptable, with a range between 0.67 (low self-esteem at baseline) and 0.87 (social isolation at baseline) (see Table 2).

Mean suicidal ideation across all participants decreased by 3.3% [P<0.05; d =0.22 (paired observations statistic used), small effect size]. For a summary of change in scores for each of the risk factor variables, see Table 3. Additionally, when analyzing only those who completed all 14 modules (n=24), mean suicidal ideation decreased by 8.8% (P=0.01; d =0.60, moderate effect size). The risk factor model used to analyze predictors of change in suicidal ideation was significant and explained 42.12% of the variance in suicidal ideation change. The findings demonstrated that change in suicidal ideation was associated with changes in low self-esteem (P<0.05). Interestingly, hopelessness and social isolation were not associated with suicidal ideation (see Table 4 for specific values of the regression analysis). Regarding dosage and suicidal ideation, the regression model was significant (R² =0.11; P=0.04), but no variables in the equation had significant standardized beta weights. These findings indicated that the number of modules completed (β=0.05, P=0.82), time on the website (β=−0.30, P=0.15), and number of characters typed (β=−0.08, P=0.64) were not related to changes in suicidal ideation.

Table 3 Change in risk factor variables from baseline to post-intervention
Full table

Table 4 Risk variable predictors of change in suicidal ideation (N=83)
Full table

Discussion

This study evaluated the potential utility of an online depression prevention program in affecting adolescent risk factors for suicide. There was a significant change in suicidal ideation in adolescents at risk for depression after using CATCH-IT. Interestingly, for those who completed the entire program the effect size was moderate, but for CATCH-IT non-completers the effect size was small, and lower than the one reported in school-based adolescent suicide prevention programs (47). While Internet interventions reach many individuals, high attrition rates present a challenge (45). Attrition levels may reduce effect sizes found in this analysis and other studies. Following further research, CATCH-IT could be implemented as a universal intervention to reach many individuals who lack mental health care, producing a small impact for a large population and producing a more significant effect for individuals who complete the entire program.

Regarding the predictors of change in suicidal ideation, the model highlighted the importance of self-esteem in addressing adolescent suicide. Higher self-esteem often predicts lower suicidal ideation (13,48,49), which supports Baumeister’s (12) theory linking low self-esteem with adolescent suicide. This theory posits that low self-esteem results in a negative perception of one’s ability to manage problems that arise, leading to a desire to escape via suicide (12). With increased self-esteem, CATCH-IT participants may have felt more capable to manage thoughts and feelings related to depression or other problems in their lives, leading to a decreased desire to escape life.

Despite its well-established role in suicidal risk, it was surprising to see that changes in hopelessness were not significantly related to changes in suicidal ideation (5,7-9,11,50,51). Because change in hopelessness was measured concurrently with change in suicidal ideation in this study, potential delayed effects of changing cognitions would not have been measured. Additionally, the mean change in hopelessness was extremely small (baseline value not significant) and therefore made it more difficult to identify a related change in suicidal ideation. Previous researchers reported significant changes in hopelessness among CATCH-IT participants (35). However, that study measured hopelessness using a single item question and did not include the major components (i.e., affective, cognitive, and motivational components) that were included in this analysis (52-55). The lack of consistency between a single-item measure of hopelessness versus our more broad measure of hopelessness highlights the need for future research using full-scale measures of hopelessness [e.g., Beck Hopelessness Scale (BHS)] (56).

Although social isolation has been identified as a risk factor for adolescent suicide in previous studies (8,57,58), change was not significantly related to changes in suicidal ideation among CATCH-IT users. As social isolation among adolescents may be influenced by family relationships (59), analysis comparing both types of relationships among CATCH-IT participants may identify further effects of the program on adolescent suicide ideation. Other related variables for decreases in suicidal ideation were not present in the current analysis. Several authors have proposed the presence of significant mental illness (e.g., mood or schizophrenia spectrum disorders) and substance use as additional factors related to adolescent suicidal ideation (4,5,60-62), which were not measured in this study. Largely absent from analysis was the effect of an adolescent’s level of depression on suicidal ideation, and it may be valuable for inclusion in future research.

While utilization of the CATCH-IT program was significant in reducing suicidal ideation among participants, the proposed usage variables (number of modules completed, total time spent on the website, and number of characters typed into response boxes) were not individually significant in contributing to the regression model while controlling for the others. Interestingly, there was a moderate effect of the program among those who completed all 14 modules, but the number of modules completed was not significantly related to change in suicidal ideation. There may be other factors related to completion affecting change in suicidal ideation, such as variance in initial severity of risk. Previous analyses of the CATCH-IT sample found a significant effect of these variables on participant depression (35), indicating that increased participation in the study can lead to greater clinical effects.

CATCH-IT’s efficacy could be better understood by analyzing the manner in which adolescents completed the intervention as intended by investigating program fidelity, which is measured by participant adherence to the CATCH-IT program and can be moderated by several factors (63). A moderating factor such as participant responsiveness to the intervention may explain the non-significance of the traditional CATCH-IT dosage measures. Additionally, there may be aspects of clinical care that are effective in reducing symptoms of depression that may not be effective with adolescent suicidal ideation (24). While CATCH-IT and other interventions may be effective in reducing depressive symptoms (46,64), renewed efforts in providing support for adolescent suicide and dismantling research when comparing depression to suicide interventions are warranted. These efforts would align with current need for the development of suicide-specific online- and computer-based interventions recognized by others (65,66). Ethical concerns have been identified by several researchers (67) and addressed in several promising intervention studies that primarily target adolescent suicidality (66).

Limitations

Adolescents expressing frequent or severe suicidal ideation or intent were excluded from the original CATCH-IT study. While this decision was clinically appropriate, removing those at extreme risk of suicidal behavior produces an artificial ceiling effect for the variable of interest. Future studies may find increased significance of findings by including those deemed at heightened risk for suicide along with increased safety measures (e.g., greater use of telephonic safety check-ins). In doing so, such a study may be able to provide further evidence supporting the use of BITs in reducing risk factors for suicide in adolescents.

Main variables of interest were created by combining items from existing measures, which may not have adequately covered the intended construct. Despite adequate internal reliability, it was impossible to measure concurrent and construct validity through comparison with known measures of suicidal ideation, hopelessness, self-esteem, and social isolation. Additionally, using intent-to-treat to reduce bias in the data may have been overly conservative, especially in the case of studies with high rates of attrition (68). This secondary analysis attempted to control for confounding variables to the extent that it was possible, but several could not be controlled (e.g., academic achievement, quality of familial relationships, etc.). While change in low self-esteem was found to have a significant relationship with changes in suicidal ideation, it is not possible to ensure causality at this time. Additionally, the findings in these analyses may have been impacted by low power. With a sample size of 83, only 24 participants completed all 14 modules of CATCH-IT, meaning unusual reactions to the program could have drastically affected the results. Power analysis indicates that a sample size of 129 would be needed to detect an effect in future studies. Finally, because the data used in this study did not include a control condition, it is impossible at this time to determine interactional effects of time within the regression analysis. Further controlled research with a larger sample size will be needed to determine with more confidence the effect of CATCH-IT on suicidal ideation. Despite these limitations, this study makes a contribution to the literature by identifying a novel manner for addressing adolescent suicide which can be flexibly applied over a broad population.

Future directions

The potential for widespread rollout of depression prevention programs that can help to prevent adolescent suicide exists. This study provides initial evidence for and serves as a stepping stone to the development of future studies as well as the refinement and dissemination of new online programs. The potential value of mental health online prevention programs for adolescent suicide risk should be considered by service developers and providers. Current adolescent suicide prevention programs are focused either on individual risk factors or suicide as a whole, and population suicide rates have not decreased (1,69,70). Future research with online depression prevention interventions for adolescents should include standardized suicide risk measures (e.g., Columbia-Suicide Severity Rating Scale; Posner et al., 2011) that specifically address suicidal ideation, hopelessness, self-esteem, and interpersonal well-being in its design. Follow-up studies with these measures would provide additional evidence for the value of CATCH-IT and similar interventions. Because CATCH-IT showed initial evidence for a significant decrease in suicidal ideation among adolescents receiving minimal support, further research examining the impact of variables levels of support for Internet interventions is needed.

Conclusions

Participants who completed CATCH-IT yielded a moderate reduction on suicidal ideation and partial CATCH-IT completers yielded a small reduction in suicidal ideation. These findings provide initial evidence that an online depression prevention programs can reduce risk factors for suicide. Furthermore, different levels of implementation could be used to approach those at risk for experiencing depression. Low-risk individuals may benefit from this online depression prevention program as a standalone intervention, and those with higher levels of risk could benefit from human support aiming to increase completion of CATCH-IT, and consequently increasing the potential effect of the intervention.

Acknowledgements

The authors would like to acknowledge both Nathan Bradford and Blake Fagan for their hard and important work in creating the dataset used for this paper.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: Original data collection was approved by IRB at the University of Chicago (ID: 13240B). Current analysis was approved by IRB at Palo Alto University (ID: 17-016-H). Written informed consent was obtained from all parent or legal guardians of participants by study staff.

References

1. Center for Disease Control and Prevention. 10 Leading Causes of Death by Age Group, United States, 2010.
2. Brière FN, Rohde P, Seeley JR, et al. Adolescent suicide attempts and adult adjustment. Depress Anxiety 2015;32:270-6. [Crossref] [PubMed]
3. Miller DN, Eckert TL. Youth suicidal behavior: An introduction and overview. Sch Psychol Rev 2009;38:153.
4. Chérif L, Ayedi H, Sahnoun F, et al. Psychopathologie des tentatives de suicide chez les adolescents. Neuropsychiatr Enfance Adolesc 2012;60:454-60. [Crossref]
5. Bolognini M, Plancherel B, Laget J, et al. Adolescent’s Suicide Attempts: Populations at Risk, Vulnerability, and Substance Use. Subst Use Misuse 2003;38:1651-69. [Crossref] [PubMed]
6. Roberts RE, Roberts CR, Xing Y. One-Year Incidence of Suicide Attempts and Associated Risk and Protective Factors Among Adolescents. Arch Suicide Res 2010;14:66-78. [Crossref] [PubMed]
7. Beck AT. Hopelessness as a predictor of eventual suicide. Ann N Y Acad Sci 1986;487:90-6. [Crossref] [PubMed]
8. Daniel SS, Goldston DB. Hopelessness and lack of connectedness to others as risk factors for suicidal behavior across the lifespan: Implications for cognitive-behavioral treatment. Cogn Behav Pract 2012;19:288-300. [Crossref]
9. Kwok SY, Shek DT. Hopelessness, parent-adolescent communication, and suicidal ideation among Chinese adolescents in Hong Kong. Suicide Life Threat Behav 2010;40:224-33. [Crossref] [PubMed]
10. Minkoff K, Bergman E, Beck AT, et al. Hopelessness, depression, and attempted suicide. Am J Psychiatry 1973;130:455-9. [PubMed]
11. Reifman A, Windle M. Adolescent suicidal behaviors as a function of depression, hopelessness, alcohol use, and social support: A longitudinal investigation. Am J Community Psychol 1995;23:329-54. [Crossref] [PubMed]
12. Baumeister RF. Suicide as escape from self. Psychol Rev 1990;97:90-113. [Crossref] [PubMed]
13. Sharaf AY, Thompson EA, Walsh E. Protective effects of self-esteem and family support on suicide risk behaviors among at-risk adolescents. J Child Adolesc Psychiatr Nurs 2009;22:160-8. [Crossref] [PubMed]
14. Boivin M, Hymel S, Bukowski WM. The roles of social withdrawal, peer rejection, and victimization by peers in predicting loneliness and depressed mood in childhood. Dev Psychopathol 1995;7:765-85. [Crossref]
15. Campos RC, Besser A, Abreu H, et al. Personality vulnerabilities in adolescent suicidality: The mediating role of psychological distress. Bull Menninger Clin 2014;78:115. [Crossref] [PubMed]
16. Qualter P, Munn P. The separateness of social and emotional loneliness in childhood. J Child Psychol Psychiatry 2002;43:233-44. [Crossref] [PubMed]
17. Serafini G, Muzio C, Piccinini G, et al. Life adversities and suicidal behavior in young individuals: a systematic review. Eur Child Adolesc Psychiatry 2015;24:1423-46. [Crossref] [PubMed]
18. Cheng Y, Tao M, Riley L, et al. Protective factors relating to decreased risks of adolescent suicidal behaviour. Child Care Health Dev 2009;35:313-22. [Crossref] [PubMed]
19. Cleary KM. Risk factors for completed adolescent suicide: Implications for prevention. Palo Alto University, CA; 2002.
20. Hall-Lande JA, Eisenberg ME, Christenson SL, et al. Social isolation, psychological health, and protective factors in adolescence. Adolescence 2007;42:265. [PubMed]
21. Balázs J, Miklósi M, Keresztény A, et al. Adolescent subthreshold-depression and anxiety: Psychopathology, functional impairment and increased suicide risk. J Child Psychol Psychiatry 2013;54:670-7. [Crossref] [PubMed]
22. Calati R, Courtet P. Is psychotherapy effective for reducing suicide attempt and non-suicidal self-injury rates? Meta-analysis and meta-regression of literature data. J Psychiatr Res 2016;79:8-20. [Crossref] [PubMed]
23. Hawton K, Witt KG, Taylor Salisbury TL, et al. Interventions for self-harm in children and adolescents. Cochrane Database Syst Rev 2015.CD012013. [PubMed]
24. Ougrin D, Tranah T, Stahl D, et al. Therapeutic interventions for suicide attempts and self-harm in adolescents: Systematic review and meta-analysis. J Am Acad Child Adolesc Psychiatry 2015;54:97-107.e2. [Crossref] [PubMed]
25. Tarrier N, Taylor K, Gooding P. Cognitive-Behavioral Interventions to Reduce Suicide Behavior: A Systematic Review and Meta-Analysis. Behav Modif 2008;32:77-108. [Crossref] [PubMed]
26. Witt K, Spittal M, Carter G, et al. Effectiveness of online and mobile telephone applications (“apps”) for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysis. BMC Psychiatry 2017;17:297. [Crossref] [PubMed]
27. Mohr DC, Schueller SM, Montague E, et al. The Behavioral Intervention Technology Model: An Integrated Conceptual and Technological Framework for eHealth and mHealth Interventions. J Med Internet Res 2014;16:e146. [Crossref] [PubMed]
28. Embry DD. The Good Behavior Game: A best practice candidate as a universal behavioral vaccine. Clin Child Fam Psychol Rev 2002;5:273-97. [Crossref] [PubMed]
29. Gladstone T, Marko-Holguin M, Henry J, et al. Understanding Adolescent Response to a Technology-Based Depression Prevention Program. J Clin Child Adolesc Psychol 2014;43:102-14. [Crossref] [PubMed]
30. Bunge E, Jones M, Dickter B, et al. Information systems and technology. In: Patel S, Reicherter D. editors. 1st edition. Psychotherapy for Immigrant Youth. Springer International Publishing, 2016:127-45.
31. Muñoz RF, Bunge EL, Chen K, et al. Massive Open Online Interventions A Novel Model for Delivering Behavioral-Health Services Worldwide. Clin Psychol Sci 2015;4:194-205. [Crossref]
32. Schueller SM, Tomasino KN, Mohr DC. Integrating Human Support Into Behavioral Intervention Technologies: The Efficiency Model of Support. Clinical Psychology: Science & Practice 2017;24:27-45. [Crossref]
33. Van Voorhees BW, Vanderplough-Booth K, Fogel J, et al. Integrative internet-based depression prevention for adolescents: a randomized clinical trial in primary care for vulnerability and protective factors. J Can Acad Child Adolesc Psychiatry 2008;17:184. [PubMed]
34. Van Voorhees BW, Ellis J, Stuart S, et al. Pilot study of a primary care Internet-based depression prevention intervention for late adolescents. Can Child Adolesc Psychiatr Rev 2005;14:40-3. [PubMed]
35. Van Voorhees BW, Fogel J, Reinecke MA, et al. Randomized clinical trial of an Internet-based depression prevention program for adolescents (Project CATCH-IT) in primary care: 12-week outcomes. J Dev Behav Pediatr 2009;30:23-37. [Crossref] [PubMed]
36. Saulsberry A, Marko-Holguin M, Blomeke K, et al. Randomized clinical trial of a primary care internet-based intervention to prevent adolescent depression: One-year outcomes. J Can Acad Child Adolesc Psychiatry 2013;22:106. [PubMed]
37. Iloabachie C, Wells C, Goodwin B, et al. Adolescent and parent experiences with a primary care/Internet-based depression prevention intervention (CATCH-IT). Gen Hosp Psychiatry 2011;33:543-55. [Crossref] [PubMed]
38. Ruby A, Marko-Holguin M, Fogel J, et al. Economic analysis of an Internet-based depression prevention intervention. J Ment Health Policy Econ 2013;16:121-30. [PubMed]
39. Johnson JG, Harris ES, Spitzer RL, et al. The patient health questionnaire for adolescents: validation of an instrument for the assessment of mental disorders among adolescent primary care patients. J Adolesc Health 2002;30:196-204. [Crossref] [PubMed]
40. Radloff LS. The use of the Center for Epidemiologic Studies Depression Scale in adolescents and young adults. J Youth Adolesc 1991;20:149-66. [Crossref] [PubMed]
41. Garrison CZ, Addy CL, Jackson KL, et al. The CES-D as a screen for depression and other psychiatric disorders in adolescents. J Am Acad Child Adolesc Psychiatry 1991;30:636-41. [Crossref] [PubMed]
42. Wenzel A, Beck AT. A cognitive model of suicidal behavior: Theory and treatment. Appl Prev Psychol 2008;12:189-201. [Crossref]
43. Van Orden KA, Witte TK, Cukrowicz KC, et al. The interpersonal theory of suicide. Psychol Rev 2010;117:575-600. [Crossref] [PubMed]
44. Gliem JA, Gliem RR. Calculating, interpreting, and reporting Cronbach’s alpha reliability coefficient for Likert-type scales. In: Midwest Research-to-Practice Conference in Adult, Continuing, and Community Education; 2003 [cited 2017 Feb 7]. Available online: https://scholarworks.iupui.edu/handle/1805/344
45. Eysenbach G. The Law of Attrition. J Med Internet Res 2005;7:e11. [Crossref] [PubMed]
46. Van Voorhees BW, Fogel J, Pomper BE, et al. Adolescent Dose and Ratings of an Internet-Based Depression Prevention Program: A Randomized Trial of Primary Care Physician Brief Advice versus a Motivational Interview. J Cogn Behav Psychother 2009;9:1-19. [PubMed]
47. Strunk CM, King KA, Vidourek RA, et al. Effectiveness of the surviving the Teens® suicide prevention and depression awareness program: an impact evaluation utilizing a comparison group. Health Educ Behav 2014;41:605-13. [Crossref] [PubMed]
48. Jang JM, Park JI, Oh KY, et al. Predictors of suicidal ideation in a community sample: Roles of anger, self-esteem, and depression. Psychiatry Res 2014;216:74-81. [Crossref] [PubMed]
49. Overholser JC, Adams DM, Lehnert KL, et al. Self-esteem deficits and suicidal tendencies among adolescents. J Am Acad Child Adolesc Psychiatry 1995;34:919-28. [Crossref] [PubMed]
50. Chang HJ, Lin MF, Lin KC. The mediating and moderating roles of the cognitive triad on adolescent suicidal ideation. Nurs Res 2007;56:252-9. [Crossref] [PubMed]
51. Cole DA. Psychopathology of adolescent suicide: Hopelessness, coping beliefs, and depression. J Abnorm Psychol 1989;98:248. [Crossref] [PubMed]
52. Aish AM, Wasserman D. Does Beck’s Hopelessness Scale really measure several components? Psychol Med 2001;31:367-72. [Crossref] [PubMed]
53. Beck AT, Weissman A, Lester D, et al. The measurement of pessimism: the hopelessness scale. J Consult Clin Psychol 1974;42:861-5. [Crossref] [PubMed]
54. Rosenfeld B, Gibson C, Kramer M, et al. Hopelessness and terminal illness: The construct of hopelessness in patients with advanced AIDS. Palliat Support Care 2004;2:43-53. [Crossref] [PubMed]
55. Yip PS, Cheung YB. Quick assessment of hopelessness: a cross-sectional study. Health Qual Life Outcomes 2006;4:13. [Crossref] [PubMed]
56. Granö N, Oksanen J, Kallionpää S, et al. Specificity and sensitivity of the Beck Hopelessness Scale for suicidal ideation among adolescents entering early intervention service. Nord J Psychiatry 2017;71:72-6. [Crossref] [PubMed]
57. Beautrais AL, Joyce PR, Mulder RT. Precipitating factors and life events in serious suicide attempts among youths aged 13 through 24 years. J Am Acad Child Adolesc Psychiatry 1997;36:1543-51. [PubMed]
58. Topol P, Reznikoff M. Perceived peer and family relationships, hopelessness and locus of control as factors in adolescent suicide attempts. Suicide Life Threat Behav 1982;12:141-50. [Crossref] [PubMed]
59. Kaminski JW, Puddy RW, Hall DM, et al. The Relative Influence of Different Domains of Social Connectedness on Self-Directed Violence in Adolescence. J Youth Adolesc 2010;39:460-73. [Crossref] [PubMed]
60. Fleischmann A, Bertolote JM, Belfer M, et al. Completed Suicide and Psychiatric Diagnoses in Young People: A Critical Examination of the Evidence. Am J Orthopsychiatry 2005;75:676-83. [Crossref] [PubMed]
61. Goldston DB, Daniel SS, Erkanli A, et al. Psychiatric diagnoses as contemporaneous risk factors for suicide attempts among adolescents and young adults: Developmental changes. J Consult Clin Psychol 2009;77:281-90. [Crossref] [PubMed]
62. Substance abuse and suicide risk among adolescents. Eur Arch Psychiatry Clin Neurosci 2012;262:469-85. [Crossref] [PubMed]
63. Carroll C, Patterson M, Wood S, et al. A conceptual framework for implementation fidelity. Implement Sci 2007;2:40-9. [Crossref] [PubMed]
64. Twomey C, O’Reilly G. Effectiveness of a freely available computerised cognitive behavioural therapy programme (MoodGYM) for depression: Meta-analysis. Aust N Z J Psychiatry 2017;51:260-9. [Crossref] [PubMed]
65. Rice SM, Goodall J, Hetrick SE, et al. Online and Social Networking Interventions for the Treatment of Depression in Young People: A Systematic Review. J Med Internet Res 2014;16:e206. [Crossref] [PubMed]
66. Rice S, Robinson J, Bendall S, et al. Online and social media suicide prevention interventions for young people: a focus on implementation and moderation. J Can Acad Child Adolesc Psychiatry 2016;25:80-6. [PubMed]
67. Lakeman R, FitzGerald M. The Ethics of Suicide Research: The Views of Ethics Committee Members. Crisis 2009;30:13-9. [Crossref] [PubMed]
68. Leykin Y, Aguilera A, Torres LD, et al. Interpreting the Outcomes of Automated Internet-Based Randomized Trials: Example of an International Smoking Cessation Study. J Med Internet Res 2012;14:e5. [Crossref] [PubMed]
69. Calear AL, Christensen H, Freeman A, et al. A systematic review of psychosocial suicide prevention interventions for youth. Eur Child Adolesc Psychiatry 2016;25:467-82. [Crossref] [PubMed]
70. Rew L, Young C, Brown A, et al. Suicide Ideation and Life Events in a Sample of Rural Adolescents. Arch Psychiatr Nurs 2016;30:198-203. [Crossref] [PubMed]