Maintained effectiveness of a smartphone-based chatbot coach (BalanceUP) for mental well-being in headache sufferers: preliminary findings from a single-arm 6-month follow-up study

Introduction

Background

Headaches, particularly migraine and tension-type headaches (TTH), are among the most prevalent health issues globally (1,2). These conditions lead to significant physical, social, and mental burden, especially among young adults and middle-aged women (3). The economic impact is substantial: in Europe, the average annual expenses per individual associated with headache, encompassing both direct costs (e.g., treatment and medical assessments) and indirect costs (e.g., missed work days and decreased productivity), were calculated to be €1,222 for migraine, €303 for TTH, and €3,561 for medication-overuse headache (4). Given this high burden, defined as the summation of all negative consequences (5), there is an urgent need for adequate headache management.

Current treatment guidelines emphasize a multimodal approach to headache management, combining both pharmacological and non-pharmacological strategies (6). While medications are crucial in migraine management, their effectiveness is often limited by side effects leading to early termination (7). Moreover, pharmacological approaches alone fail to address the underlying psychological and behavioral components of headache conditions. Behavioral therapies, particularly cognitive-behavioral therapy (CBT), have emerged as promising complementary approaches with minimal side effects (8). Guidelines recommend incorporating psychological techniques such as education, self-monitoring, self-management strategies, social skills training, relaxation techniques, and mindfulness alongside medication (6,8).

The importance of behavioral approaches is underscored by evidence that pain patients’ cognitive, emotional, and behavioral coping responses significantly influence their long-term health outcomes (9). These pain responses, shaped by individual characteristics and social environment, are modifiable and therefore present promising targets for health behavior change interventions. Key resilience factors in managing headaches include pain self-efficacy, pain acceptance, and distress tolerance, which promote adaptive behaviors linked to sustained well-being and higher quality of life (10,11). This is particularly relevant for individuals experiencing high stress levels or psychological comorbidities, as research has shown that stressful events often precede headache attacks (12).

Rational and knowledge gap

Despite the documented effectiveness of behavioral therapies (13-15), significant barriers to access persist, including inadequate knowledge about treatments, limited availability, and stigma associated with headaches and mental health (16). These barriers disproportionately affect individuals from under-represented backgrounds or those facing socioeconomic challenges (17). Additionally, poor awareness about migraine among both affected individuals and healthcare providers continues to compound the overall burden (18).

Digital interventions, particularly smartphone-based solutions, have emerged as promising tools to overcome these treatment access barriers (19,20). Electronic headache diary apps have gained prominence for tracking headache-related data (21), with a recent study demonstrating their utility when combined with machine learning for headache prediction (22). Smartphone apps can deliver guideline-compliant therapeutic options (13), including psychoeducation, relaxation techniques, and stress reduction. Among these digital solutions, conversational agents (CAs), or chatbots, show particular promise in supporting headache management (23). CAs have demonstrated positive results in terms of acceptance and effectiveness across both clinical and non-clinical populations, with their ability to provide personalized intervention delivery enhancing engagement and adherence (24-26).

Building on these developments, we previously developed BalanceUP, a smartphone-based CA coaching intervention specifically targeting individuals with regular headaches. Our initial randomized controlled trial (RCT) demonstrated significant short-term effectiveness (27) with improved well-being [depression and anxiety, Cohen d =−0.66, 95% confidence interval (CI): −0.99 to −0.33]. Further, reduced psychosomatic symptoms, perceived stress, absenteeism and presenteeism, and improved self-efficacy and pain coping, with effects ranging from medium to large (Cohen d =0.43 to 1.05). The intervention showed high acceptance, with 65% of participants using the unguided coaching as intended, and no negative side effects were reported.

Objective

While these initial results are encouraging, the long-term effectiveness of app-based behavioral interventions, including headache management, remains largely unexplored (19,28,29), despite guidelines recommending the use of mobile health (mHealth) in headache treatment (30).

To address this knowledge gap, the present study evaluates the preliminary sustained effectiveness of the BalanceUP intervention through a follow-up assessment of the original RCT participants. We hypothesize that the positive effects observed immediately post-intervention will be maintained at follow-up, as measured by sustained improvements in well-being, psychosomatic symptoms, stress levels, work productivity (presenteeism/absenteeism), and pain coping. Given that the waitlist control group received the intervention after their waiting period (31), this study focuses on within-group changes over time rather than between-group comparisons, with potential biases addressed in the Methods and Limitations and suggestions for future work sections. CBT-based treatment is most effective for migraine sufferers experiencing high levels of stress (13). Additionally, self-efficacy (32) and coping (33,34) might be linked to treatment effectiveness. We, therefore, explored whether perceived stress, headache-related self-efficacy, coping, work productivity at baseline were associated with well-being improvements over time. To our knowledge, this represents the first follow-up study evaluating the sustained effectiveness of a CA-based coaching intervention for headache sufferers.

Methods

Study design

This preliminary follow-up study employed a one-group pretest-posttest design to assess the sustained effectiveness of the BalanceUP intervention. We conducted a within-group analysis, using data from three time points: before the intervention (T1), after completing the intervention (T2), and 6 months after the start of the intervention (T3).

We used data from our prior RCT, where participants were initially allocated to an intervention or waitlist control group, as described in detail elsewhere (27). Participants in the intervention group started the coaching program immediately, while those in the waitlist control group were offered the same coaching program after a waiting period of 42 days, corresponding to the mean duration of the coaching intervention. This waitlist design in the original RCT was chosen due to the challenges of implementing a sham intervention or active control (35). As this follow-up lacks a sustained control condition, we cannot fully disentangle intervention effects from natural symptom fluctuations over time. Maintaining a separate control group was considered but deemed ethically unfeasible, as all participants were ultimately offered the intervention (36).

Following previous studies (37,38), this follow-up evaluation included all participants who completed the coaching program, regardless of their initial RTC allocation. To enhance statistical power, we merged data from the waitlist control group with the intervention group, using their post-waiting period assessment (collected before starting the intervention) as their T1 measurement to align baseline time points across participants. While this approach improves power, it introduces potential biases. The waitlist group, despite receiving the same intervention, may have experienced different contextual influences, such as varying expectations or external factors during the waiting period. To assess the robustness of our findings, we conducted a sensitivity analysis using only participants from the original intervention group (n=45), examining whether the patterns of change were similar when excluding waitlist control participants. Despite these limitations, we believe this study offers valuable insights into the intervention’s sustained effects and informs future controlled research.

Informed consent was obtained electronically within the app after study information. Pre- and post-intervention data were collected within the BalanceUp app and via the in-app survey tool LimeSurvey (V3.4) by self-assessment. Additional self-reported outcomes and usage data were gathered during the coaching phase. For the follow-up data collection, participants received an email containing a link to the LimeSurvey questionnaire. Notably, there was no human contact throughout the study period, except in cases of technical issues.

The study was approved by The Swiss Ethics Committee Zurich (Swiss Ethics BASEC-Nr. Req-2021-01365). It does not fall within the scope of the Human Research Act (HRA). It was registered at the World Health Organization (WHO)-accredited German Clinical Trials Register (DRKS00017422) and Swiss Ethics BASEC-Nr. Req-2021-01365. This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments.

Study participants

Eligible participants were adults (aged ≥18 years) owning a smartphone (iOS or Android) with access to the Internet, with fluent German language skills, regular headaches for at least three months, and suffering from a minimum of four days/attacks incidents per month. These criteria were assessed within the app by self-report, eligible participants were asked to provide electronic informed consent. Participants were self-recruited in Switzerland, Germany, and Austria between April and November 2022 (German Speaking parts). The link to the study website was posted via social media by headache organizations, the Swiss Headache Society, a Swiss health insurance company, headache self-help groups, and health care institutions. The study website contained information about the study (written study information and video clip; www.zhaw.ch/psychologie/balanceup), participation, registration, and links to app stores to download the BalanceUP app. Alternatively, the app could be searched directly in app stores.

BalanceUP coaching

BalanceUP is developed for iOS (Apple Inc.) and Android (Google LLC.) applications using the MobileCoach platform. Participants communicated through a pre-defined chat interface with their digital CA coach, supplemented by videos, working materials, and illustrations. The coaching intervention is rooted in the cognitive behavior change migraine therapy manual [Cognitive-Behavioral Therapy for Migraine Management (Kognitiv-verhaltenstherapeutisches Migränemanagement); MIMA] (39-41). The intervention aims to support a balanced lifestyle through psychoeducation, cognitive restructuring, relaxation, behavioral activation (i.e., dealing with fear avoidance and triggers), and lifestyle changes (e.g., sleep, physical activity) (8). Furthermore, it integrates various behavior change techniques (42). Behavior change techniques are active intervention components (e.g., goal setting, action planning) designed to facilitate behavior change and positively influence engagement. By empowering participants to proactively reflect and modify behaviors, emotions, thoughts, and beliefs, the aim was to improve their mental well-being while ensuring low-threshold access and scalability.

Expanding upon MIMA, BalanceUP comprises seven consecutive modules (headaches, relaxation, balance, fear, coping, trigger, stress), each containing three to four sub-units. Participants engaging with the CA could opt to complete a module in a single session or distribute the units over multiple sessions based on individual preferences, allowing for completion within 24 to 60 days. Post-coaching, participants retained access to conversational interactions, relaxation exercises, video clips, and other materials. BalanceUP consistently incorporates various coaching elements, including task feedback, psychoeducation, behavior reflection, setting behavioral intentions, and action planning (43).

The intended use of BalanceUP, defined as the extent to which an individual needs to engage with the content to achieve maximum benefit (44), was assessed by reaching the outro to complete the post-survey. Participants were not required to review all materials; certain modules, units, or elements within a unit could be skipped based on diagnosis (self-reported migraine vs. TTH) and prior knowledge. Typically, participants had the option to bypass psychoeducation and proceed directly to behavior reflection within a unit. While relaxation exercises, educational videos, and worksheets were available, they were optional. This flexibility accommodated personal preferences and variations in participants’ desired outcomes, aligning with recommendations for digital health interventions (45,46).

Outcomes

Mental well-being

Mental well-being was defined as the primary outcome, measured by the Patient Health Questionnaire Anxiety and Depression Scale (PHQ-ADS) (47). The PHQ-ADS scores can range from 0 to 48, with higher scores indicating more severe depression/anxiety, 3 to 4 points are considered as minimum clinically important difference. Cut points of 10, 20, and 30 indicated mild, moderate, and severe levels of depression/anxiety, respectively.

Depression

The Patient Health Questionnaire-9 (PHQ-9) (48) measures depression and consists of 9 items to evaluate depressive symptoms, rated on a 4-point Likert scale ranging from 0 (not at all) to 3 (nearly every day). Higher scores indicate higher symptom severity, a score ranging from 0 to 4 indicates no symptoms of depression, scores from 5 to 9, 10 to 14, 15 to 19, and 20 to 27 indicate mild, moderate, moderately severe, and severe depression, respectively.

Anxiety

Symptoms of generalized anxiety were measured by the General Anxiety Disorder-7 (GAD-7). It comprises of 7 items, and answers are rated on a 4-point Likert scale ranging from 0 (not at all) to 3 (nearly every day). Higher scores indicate higher levels of anxiety, total score ranges from 0 to 21. Scores from 0 to 4, 5 to 9, 10 to 14, and 15 to 21 denoting minimal, mild, moderate, and severe anxiety, respectively.

Somatic symptoms

We measured somatic symptoms with the Patient Health Questionnaire-15 (PHQ-15) (49), a 15-item self-report questionnaire that can be scored on a 0 (not impaired) to 2 (severely impaired) scale. A total score of 15 or more indicates a high level of impairment due to somatic symptoms (50).

Stress

Perceived stress was collected by the German version of the Perceived Stress Scale-10 (PSS-10) (51), 10-item can be rated on a scale from 0 (never) to 4 (very often) where higher scores reflect a higher level of perceived stress. Scores ranging from 0 to 13, 14 to 26, and 27 to 40 indicate low, moderate, and high perceived stress, respectively.

Self-efficacy

To measure headache-related self-efficacy, the German short version of the Headache Management Self-efficacy scale (HMSE-G-SF) (52) was applied. The measurement comprises 6 items that assess self-efficacy beliefs related to headaches. Answer scales range from 1 (do not agree) to 7 (agree); higher summed scores imply higher self-efficacy expectations and scores below 19 indicate below-average self-efficacy expectations compared to other headache sufferers.

Intention to change behavior

We assessed participants’ intention to change behavior by the stage of behavior change according to the health action process approach (HAPA) (53). It distinguishes three stages of change: nonintenders, intenders, and actors. In this study, participants had to indicate if they had recently been using psychological techniques for headache treatment by choosing one of the five possible answers: “(I) no—and I do not intend to do so (nonintender); (II) no—but I’m considering it (nonintender); (III) no—but I have the intention to do so (intender); (IV) yes—but it’s not easy (actor); and (V) yes—and it’s easy (actor)”.

Absenteeism and presenteeism

To assess headache-related absenteeism and presenteeism at work, we used 4 out of 5 questions from the Migraine Disability Assessment (MIDAS) (54). Due to the study’s duration and potential recall bias (54), participants reported days absent due to headaches and days with reduced productivity due to headaches of at least 50% (in work or school and household tasks) over 1 month instead of 3 months (55).

Pain processing

We measured pain coping strategies by the German Questionnaire for the Assessment of Pain Processing (questionnaire to assess pain management; Fragebogen zur Erfassung der Schmerzverarbeitung) (56). Part one assesses the participants’ cognitive and behavioral coping strategies with 24 items; the second part assesses the psychological impairments caused by pain with the help of 14 items (not used in this study). The cognitive coping subscale includes the dimensions “action planning skills”, “cognitive restructuring”, and “experience of competence”. The behavioral coping subscale includes “mental distraction”, “counteracting activities”, and “rest and relaxation techniques”. Answers can be scored from 1 (not at all true) to 6 (always true), with higher scores indicating better pain processing.

Sociodemographics

Age, sex, level of education, self-reported headache diagnosis, additional headache tracking app usage, concurrent psychotherapy, and commitment to the program were attained at baseline.

Post hoc power calculation

We conducted a post hoc power calculation based on the primary outcome PHQ-ADS (mental well-being) based on a repeated measure analysis of variance (ANOVA) (within-group). Based on two previous studies evaluating follow-up data of behavioral treatment for headaches (37,38), we assumed small to medium effect size (f =0.2) for the primary outcome. Statistical power calculation using G*Power³ software revealed a power of 0.95 based on a sample size of 69, with an alpha of 0.05 and based on three measurements.

Statistical analysis

The analysis was performed using SPSS (version 28.0, IBM Corp.). To analyze changes over time for the primary outcome (PHQ-ADS) we included only participants who completed the coaching program and all surveys (complete cases from both intervention and waitlist control groups, n=67). For the main analyses, we converted the data from survey 2 of the waitlist control group (postwait) to T1 (pre-intervention) in SPSS. To assess the robustness of our findings, we conducted a sensitivity analysis using only participants from the original intervention group (n=45), examining whether the patterns of change were similar when excluding waitlist control participants. The analysis was done by using a repeated measure ANOVA with PHQ-ADS as the dependent variable, and time as a within-subject factor. In the case of non-sphericity, we applied the Greenhouse-Geisser correction. To account for multiplicity for post-hoc comparison, we applied the Sidak adjustment. Secondary outcomes [PHQ-9, GAD-7, PHQ-15, PSS, questionnaire to assess pain management; Fragebogen zur Erfassung der Schmerzverarbeitung (FESV), HMSE, MIDAS, HAPA] were analyzed accordingly.

Calculations of within-group effect sizes (Cohen d) were based on the pooled standard deviation and labeled as small (d=0.2), medium (d=0.5), and large (d=0.8).

To explore whether perceived stress, headache-related self-efficacy, presenteeism, absenteeism, and coping (baseline) predicted the change of the primary outcome, we performed linear regressions. Initially, a separate analysis was performed for each predictor variable, with the change from pre-intervention to follow-up in PHQ-ADS scores set as a dependent variable. Subsequently, we combined the predictors and also utilized a stepwise selection to determine their combined predictive power.

Results

Participant flow and baseline characteristics

In the original study, out of the 404 downloads, 198 individuals completed the baseline survey and were randomized into intervention (n=110) and control (n=88) groups. In this study, we used baseline data of 179 participants: all 110 participants from the intervention group, and 69 participants from the control group who started the coaching intervention after the waiting time. Of these, 35.8% (64/179) discontinued using the app, while 64.2% (115/179) completed the BalanceUP coaching, and 37% (67/179) provided follow-up data. The full participant flow is depicted in Figure 1.

Figure 1 Participant flowchart.

As shown in Table 1, most of the participants were women (157/179, 87.7%) with a mean age of 39.22 (SD 11.87) years, and more than half had a university degree (95/179, 53.1%). Migraine was the most prevalent self-reported diagnosis, accounting for 74.3% (133/179) of the sample. Approximately half of the participants (87/179, 48.6%) reported using smartphone diaries to track their headaches. Few participants (30/179, 16.8%) reported attending concomitant psychotherapy while using the BalanceUP app. In general, participants reported an average sum of 6.72 (SD 7.27) days of work per month missed due to headaches, and 11.95 (SD 10.23) days per month when their performance was reduced by 50% or more (including work, school, and household). Compared to other individuals who suffer from headaches (52), participants reported average levels of headache-related self-efficacy. On average, participants were categorized as “intenders”, suggesting they had the intention to change their behavior, unlike “nonintenders” or “actors”. On average, participants suffered from mild depression (8.98, SD 4.68), mild anxiety (6.58, SD 4.18), and moderate psychosomatic symptoms (10.92, SD 4.86). There was no difference between groups for any of the outcomes.

Effectiveness

Complete data (per-protocol analysis)

The results of the repeated measure ANOVA revealed evidence for a treatment effect over time for the primary outcome (F_1.7,113.2 =27.15; P<0.001; η² =0.29). Mental well-being improved from pre-intervention to post-intervention with a medium-large effect (d =−0.72) and was maintained at the 6-month follow-up with a medium effect (d =−0.54). Table 2 presents the changes from pre-intervention (T1) to post-intervention (T2) and from pre-intervention to follow-up (T3), along with Cohen d effect sizes.

Regarding secondary outcomes, we found evidence for improvement over time for depression (F_1.7,112.8 =30.32; P<0.001; η² =0.32), anxiety (F_1.8,120.6 =13.84; P<0.001; η² =0.17), somatic symptoms (F_2,132 =30.62; P<0.001; η² =0.32), stress (F_2,132 =18.53; P<0.001; η² =0.22), headache related self-efficacy (F_1.8,119.7 =23.75, P<0.001; η² =0.27), stage of behavior change (F_2,132 =17.68; P<0.001; η² =0.21), absenteeism (F_2,118 =4.00; P=0.02; η² =0.06), presenteeism (F_2,122 =5.43; P=0.006; η² =0.08), and cognitive and behavioral pain coping (F_1.6,105.1 =28.42, P<0.001, η² =0.30; F_1.7,113.7 =24.80, P<0.001, η² =0.27).

Comparison between time points (pre-intervention to post-intervention; pre-intervention to follow-up) revealed sustained intervention effectiveness. Effect sizes from pre-intervention to post-intervention for secondary outcomes were medium to large, except for absenteeism and presenteeism, which showed small effects, absenteeism was not significant. Effect sizes from pre-intervention to follow-up were medium, except for absenteeism and presenteeism, where small effects were observed. Please also see Figure 2 for a visual representation.

Figure 2 Representation of effect sizes (Cohen d) of outcomes related to psychological functioning and headaches (n=62). T1: pre-intervention. T2: post-intervention. T3: follow-up.

Sensitivity analyses with participants from the original intervention group (n=45) revealed similar patterns for the primary outcome (F_2,88 =14.55; P<0.001; η² =0.25), with mental well-being improving from pre-intervention to pos-tintervention with a medium-large effect (d =−0.69) and maintained at 6-month follow-up with a medium effect (d =−0.47). However, two differences emerged: improvements in anxiety and presenteeism were not maintained at follow-up in this subgroup [anxiety: mean difference (MD) =−0.73, 95% CI: −1.57 to 0.10; P=0.08; presenteeism: MD =−2.15, 95% CI: −4.78 to 0.48; P=0.11]. Effects ranged from small to large (d =−0.26 to 0.87), changes from pre-intervention (T1) to post-intervention (T2) and from pre-intervention to follow-up (T3), along with Cohen d effect sizes can be found in Table S1.

Explorative

The results of individual linear regressions revealed significant associations between perceived stress (r=0.43, P<0.001), absenteeism (r=0.38, P=0.002), presenteeism (r=0.38, P=0.003), and cognitive coping (r=−0.32, P=0.008) with change from pre-intervention to follow-up in well-being (PHQ-ADS). However, headache-related self-efficacy (r=0.02, P=0.85), and behavioral coping (r=−0.17, P=0.18) were not significantly associated with well-being changes. When including all predictors in the analysis, the R² for the overall model was 0.31 (adjusted R²=0.23). In the stepwise regression, stress, and presenteeism remained in the model, resulting in an R² of 0.27 (adjusted R²=−24). For detailed results, please see Table 3.

Discussion

Key findings

This follow-up study evaluated the sustained effectiveness of BalanceUP, a smartphone-based, CA-delivered coaching intervention for headache sufferers. The primary finding was a maintained improvement in mental well-being at 6-month follow-up (d =−0.54), though slightly reduced from post-intervention effects (d =−0.72). Secondary outcomes showed sustained improvements across multiple domains: depression, anxiety, somatic symptoms, stress, self-efficacy, and pain coping (d =0.24 to d =0.70). However, due to the lack of a control group in our study, these conclusions remain speculative.

Our sensitivity analysis using only the intervention group provided important insights into the robustness of our findings. While most effects were consistent across analyses, the non-maintenance of anxiety and presenteeism improvements in the intervention-only group suggests potential selection effects. Participants who completed the waiting period might represent a particularly motivated subgroup, having maintained their commitment despite the delay. This waiting period could have built up positive expectations about the intervention’s benefits, potentially enhancing its effectiveness through increased engagement.

The sustained improvements in well-being likely result from several mechanisms. First, the intervention’s comprehensive approach, combining CBT principles with relaxation exercises and behavior change techniques, appears to foster lasting improvements in coping skills and self-efficacy (57). The accessibility of relaxation exercises through the app might have contributed to sustained effectiveness, aligning with face-to-face therapy outcomes. Improvements in lifestyle and psychological functioning are critical for headache sufferers, as stress, anxiety, and depression can exacerbate symptoms and hinder coping (12,58).

The finding that participants with higher baseline stress and presenteeism showed greater improvements in well-being may be explained by several factors. Those with higher initial stress levels had more potential for positive change and possibly stronger motivation to engage with intervention strategies. Additionally, participants maintaining work attendance despite symptoms (high presenteeism) likely had more opportunities to implement learned coping strategies in real-world situations, reinforcing their effectiveness through regular practice. Contrary to expectations (59), participants experienced similar improvements in well-being regardless of their baseline self-efficacy levels, suggesting that the BalanceUP intervention was equally effective across varying initial levels of self-efficacy. Alternatively, participants with medium levels of self-efficacy may benefit differently than those with low or high levels, complicating the ability to detect a straightforward predictive relationship.

Comparison with prior work

Our findings can be compared across three relevant domains: traditional CBT interventions for headaches, digital headache interventions, and smartphone and CA-delivered mental health interventions. Regarding traditional CBT, which is recommended for preventive headache treatment (15,60), our results align with established effectiveness patterns. The BalanceUP coaching app is based on the face-to-face CBT migraine management manual MIMA, which demonstrated both feasibility (40) and efficacy (41). Klan et al. found significant within-group improvements at 4-month follow-up for headache days, disability, self-efficacy, and pain acceptance, though not for emotional distress, comparable to our findings. The 12-month follow-up revealted significant imrovements for headache days, disability, emotional distress, self-efficacy, and pain acceptance.

In the broader digital intervention landscape, evidence of sustained effectiveness remains limited (61). Recent research shows promising but varied results. A smartphone-delivered mindfulness program demonstrated reduced migraine-related disability at 6 months (62), though generalizability is limited by sample size. More comparably, an RCT of online behavioral training (63) found improvements in migraine attacks for both intervention and control groups (d =0.66 and 0.52), with maintained improvements in migraine-related self-efficacy (d =0.61), locus of control (d =0.31), and quality of life (d =0.47) at 6 months, aligning with our effect sizes.

CA-delivered health interventions present the most direct comparison but rarely report follow-up data. This study evaluated its sustained effectiveness, an area largely unexplored in digital headache interventions (64). A meta-analysis of CA effectiveness in mental health (65) identified only one study with extended follow-up, showing non-significant depression reductions at 12 months (66). Similarly, a digital CBT study for insomnia (67) found significant immediate effects but struggled to maintain clinically meaningful differences in depression and anxiety at follow-up, paralleling our findings regarding anxiety maintenance. A meta-review of mHealth interventions (68) emphasized the importance of evaluating the long-term effects of smartphone-based interventions at various follow-up points. It was noted that mHealth interventions can often be accessed over an extended period without a dedicated end date, making post-intervention or follow-up assessment more challenging to define. Linardon et al. (29) also highlighted the need for more studies assessing the long-term effectiveness of mHealth interventions, noting that most studies’ follow-up periods do not extend beyond 12 weeks.

Limitations and suggestions for future work

Due to the methodology of the original study, there was no control group for this follow-up evaluation. Participants from the original RCT were offered the opportunity to participate in the BalanceUP coaching program after a 4-week waiting period, which makes causal inferences about the intervention and patient-reported outcomes impossible. Additionally, the effect sizes reported in this study are likely overestimates of the true treatment effect relative to an active control or usual treatment, potentially influenced by regression to the mean.

Despite high engagement with the unguided coaching program, less than half of all participants completed the questionnaires at the 6-month follow-up. This selective follow-up population increases the risk of selection bias, underscoring the need for further research to assess the sustained effectiveness of chatbot-based interventions. Notably, there was no human contact or guidance throughout the coaching, including during the follow-up period, and participants were not involved in any multimodal treatment or additional interventions. In a study assessing smartphone-based migraine behavioral therapy (69), less than half of participants provided data at the 3-month follow-up. Therefore, our analysis was based on participants with complete data only. Although completers and dropouts only differed significantly in terms of application of behavior change at baseline, attrition bias may still influence the results, limiting generalizability. This suggests that those who completed the program may have been at a more advanced stage of behavior change, making them more willing to engage with and apply behavior change techniques. This readiness might have contributed to sustained participation and the observed positive outcomes, while those less prepared for change may have been more likely to drop out, potentially skewing the results. To mitigate this, incorporating new content within the app and encouraging self-monitoring could support sustained use and data collection (60), as regular reminders via push notifications might enhance follow-up engagement.

Furthermore, participants with fewer headache-related symptoms pos-tintervention might have been more inclined to complete the follow-up questionnaires, potentially biasing estimates of the sustained treatment effect. Results might also be subject to recall bias. In this study, the migraine diagnosis was self-reported, which is important to consider as migraines are often underdiagnosed, and many individuals may be unaware they have the condition. Collaborating with clinical practices could help ensure the program reaches those in need.

At the beginning of the study, there was no consensus on the definition of digital therapeutics, and medical regulations have since become more stringent and cautious (70), refraining from collecting headache days or attack frequency. Despite these changes, we assessed headache-related outcomes using measures of absenteeism and presenteeism. Absenteeism showed significant improvement at follow-up. This indicates the sustained effectiveness of unguided CA coaching. These results are consistent with findings from Schaetz et al. (71), where individualized telecoaching via smartphone was similarly effective in terms of absenteeism and presenteeism.

Finally, the BalanceUP coaching was available only in German, limiting its accessibility for non-German speakers. This language restriction may unintentionally exclude individuals from linguistically diverse backgrounds, potentially exacerbating existing healthcare disparities for populations already facing barriers to headache care. Furthermore, most of the participants in this study were women (87.7%). While this aligns with the known higher prevalence of headaches in women (72,73), our sample demonstrates a considerably stronger gender imbalance than would be expected from epidemiological data (74). This overrepresentation of women may reflect gender differences in healthcare-seeking behavior, willingness to participate in digital health interventions or potential bias in our recruitment methods. The gender imbalance limits the generalizability of our findings to male populations with headache disorders. Lastly, the homogeneity of our sample in terms of language and cultural region therefore represents another limitation for the global generalizability of our findings, particularly to non-Western populations where headache management approaches and adoption of digital health interventions may differ (75). Cultural adaptations by modifying intervention components that align with a target audience’s cultural norms, beliefs, and values may increase the reach and engagement of digital health interventions, but the evidence is limited (76).

Conclusions

This preliminary follow-up study suggests that BalanceUP, a smartphone-based CA intervention, may support sustained improvements in mental well-being and secondary outcomes among headache sufferers at 6-month follow-up, particularly for those with higher baseline stress and presenteeism. Our exploratory analyses highlight how motivation and timing may influence treatment success, as revealed by sensitivity analyses. While these initial results support the potential of digital coaching approaches, the methodological limitations of our single-arm design with substantial attrition require cautious interpretation. Future research should incorporate active controls and more robust follow-up assessment strategies to evaluate the sustained effectiveness of smartphone-based CA interventions in headache management better while exploring how factors such as motivation and timing influence treatment outcomes.

Acknowledgments

The authors thank participating users and to the technical team for their dedicated effort.

Footnote

Data Sharing Statement: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-25-9/dss

Peer Review File: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-25-9/prf

Funding: This study was funded by Zurich University of Applied Sciences, Switzerland.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-25-9/coif). T.K. is affiliated with the Centre for Digital Health Interventions, a joint initiative of the Institute for Implementation Science in Health Care, University of Zurich, the Department of Management, Technology, and Economics at ETH Zurich, and the Institute of Technology Management and School of Medicine at the University of St. Gallen. CDHI is funded in part by CSS, a Swiss health insurer, Mavie Next, an Austrian health insurer, and MTIP, a Swiss digital health investor. T.K. is also a cofounder of Pathmate Technologies, a university spin-off company that creates and delivers digital clinical pathways. T.K. holds neither shares nor any formal role with Pathmate Technologies. Also, neither CSS, Mavie Next, MTIP, nor Pathmate Technologies were involved in this research. A.R.G. is a a board member of the Swiss Headache Society, treasurer of the Swiss Pain Society, and has received honoraria for consulting or speaking from AbbVie, Lundbeck, Novartis, Pfizer, TEVA and Organon in the last 36 months. He has received payment for expert testimony from Pfizer and TEVA. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was approved by The Swiss Ethics Committee Zurich (Swiss Ethics BASEC-Nr. Req-2021-01365). This study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. Informed consent was taken from all the participants.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 2017;390:1211-59. [Crossref] [PubMed]
2. Steiner TJ, Stovner LJ, Birbeck GL. Migraine: the seventh disabler. J Headache Pain 2013;14:1. [Crossref] [PubMed]
3. Stovner LJ, Nichols E, Steiner TJ, et al. Global, regional, and national burden of migraine and tension-type headache, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol 2018;17:954-76. [Crossref] [PubMed]
4. Linde M, Gustavsson A, Stovner LJ, et al. The cost of headache disorders in Europe: the Eurolight project. Eur J Neurol 2012;19:703-11. [Crossref] [PubMed]
5. Steiner TJ, Terwindt GM, Katsarava Z, et al. Migraine-attributed burden, impact and disability, and migraine-impacted quality of life: Expert consensus on definitions from a Delphi process. Cephalalgia 2022;42:1387-96. [Crossref] [PubMed]
6. Diener HC, Gaul C, Kropp P. Therapie der Migräneattacke und Prophylaxe der Migräne, S1-Leitlinie, 2018. In: Deutsche Gesellschaft für Neurologie. editor. Leitlinien für Diagnostik und Therapie in der Neurologie. 2018. Available online: www.dgn.org/leitlinien
7. Blumenfeld AM, Bloudek LM, Becker WJ, et al. Patterns of use and reasons for discontinuation of prophylactic medications for episodic migraine and chronic migraine: results from the second international burden of migraine study (IBMS-II). Headache 2013;53:644-55. [Crossref] [PubMed]
8. Amatrudo G, Kengetter J, McCrea S, et al. Cognitive Behavioral Therapy for the Management of Episodic Migraine. Curr Pain Headache Rep 2023;27:471-7. [Crossref] [PubMed]
9. Neeb L. Diagnostik und Therapie des Kopfschmerzes vom Spannungstyp, S1-Leitlinie, 2023. In: Deutsche Gesellschaft für Neurologie. editor. Leitlinien für Diagnostik und Therapie in der Neurologie. 2023. Available online: www.dgn.org/leitlininen
10. Stonnington CM, Kothari DJ, Davis MC. Understanding and Promoting Resiliency in Patients with Chronic Headache. Curr Neurol Neurosci Rep 2016;16:6. [Crossref] [PubMed]
11. Kurtses Gürsoy B, Köseoğlu Toksoy C. Psychological Resilience and Stress Coping Styles in Migraine Patients. Neuropsychiatr Dis Treat 2023;19:63-72. [Crossref] [PubMed]
12. Seng EK, Martin PR, Houle TT. Lifestyle factors and migraine. Lancet Neurol 2022;21:911-21. [Crossref] [PubMed]
13. Kropp P, Meyer B, Dresler T, et al. Entspannungsverfahren und verhaltenstherapeutische Interventionen zur Behandlung der Migräne. Schmerz 2017;31:433-47. [Crossref] [PubMed]
14. Lee HJ, Lee JH, Cho EY, et al. Efficacy of psychological treatment for headache disorder: a systematic review and meta-analysis. J Headache Pain 2019;20:17. [Crossref] [PubMed]
15. Sullivan A, Cousins S, Ridsdale L. Psychological interventions for migraine: a systematic review. J Neurol 2016;263:2369-77. [Crossref] [PubMed]
16. Langenbahn D, Matsuzawa Y, Lee YSC, et al. Underuse of Behavioral Treatments for Headache: a Narrative Review Examining Societal and Cultural Factors. J Gen Intern Med 2021;36:3103-12. [Crossref] [PubMed]
17. Befus DR, Irby MB, Coeytaux RR, et al. A Critical Exploration of Migraine as a Health Disparity: the Imperative of an Equity-Oriented, Intersectional Approach. Curr Pain Headache Rep 2018;22:79. [Crossref] [PubMed]
18. Viana M, Khaliq F, Zecca C, et al. Poor patient awareness and frequent misdiagnosis of migraine: findings from a large transcontinental cohort. Eur J Neurol 2020;27:536-41. [Crossref] [PubMed]
19. Minen MT, Torous J, Raynowska J, et al. Electronic behavioral interventions for headache: a systematic review. J Headache Pain 2016;17:51. [Crossref] [PubMed]
20. Stubberud A, Linde M. Digital Technology and Mobile Health in Behavioral Migraine Therapy: a Narrative Review. Curr Pain Headache Rep 2018;22:66. [Crossref] [PubMed]
21. Göbel H, Frank B, Heinze A, et al. Gesundheitsverhalten von Migräne- und Kopfschmerzpatienten bei digitaler Therapiebegleitung mit der Migräne-App. Schmerz 2019;33:147-55. [Crossref] [PubMed]
22. Stubberud A, Ingvaldsen SH, Brenner E, et al. Forecasting migraine with machine learning based on mobile phone diary and wearable data. Cephalalgia 2023;43:3331024231169244. [Crossref] [PubMed]
23. Neeb L, Ruscheweyh R, Dresler T. Digitalisierung in der Kopfschmerzbehandlung. Schmerz 2020;34:495-502. [Crossref] [PubMed]
24. Jakob R, Harperink S, Rudolf AM, et al. Factors Influencing Adherence to mHealth Apps for Prevention or Management of Noncommunicable Diseases: Systematic Review. J Med Internet Res 2022;24:e35371. [Crossref] [PubMed]
25. Gaul C, van Doorn C, Webering N, et al. Clinical outcome of a headache-specific multidisciplinary treatment program and adherence to treatment recommendations in a tertiary headache center: an observational study. J Headache Pain 2011;12:475-83. [Crossref] [PubMed]
26. Wallasch TM, Kropp P. Multidisciplinary integrated headache care: a prospective 12-month follow-up observational study. J Headache Pain 2012;13:521-9. [Crossref] [PubMed]
27. Ulrich S, Gantenbein AR, Zuber V, et al. Development and Evaluation of a Smartphone-Based Chatbot Coach to Facilitate a Balanced Lifestyle in Individuals With Headaches (BalanceUP App): Randomized Controlled Trial. J Med Internet Res 2024;26:e50132. [Crossref] [PubMed]
28. van de Graaf DL, Schoonman GG, Habibović M, et al. Towards eHealth to support the health journey of headache patients: a scoping review. J Neurol 2021;268:3646-65. [Crossref] [PubMed]
29. Linardon J, Cuijpers P, Carlbring P, et al. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 2019;18:325-36. [Crossref] [PubMed]
30. Diener HC. Migräne vorbeugen und behandeln. Heilberufe 2023;75:28-9.
31. Freedland KE, King AC, Ambrosius WT, et al. The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel. J Clin Epidemiol 2019;110:74-81. [Crossref] [PubMed]
32. Nicholson RA, Hursey KG, Nash JM. Moderators and mediators of behavioral treatment for headache. Headache 2005;45:513-9. [Crossref] [PubMed]
33. Chiros C, O'Brien WH. Acceptance, appraisals, and coping in relation to migraine headache: an evaluation of interrelationships using daily diary methods. J Behav Med 2011;34:307-20. [Crossref] [PubMed]
34. Russo A, Santangelo G, Tessitore A, et al. Coping Strategies in Migraine without Aura: A Cross-Sectional Study. Behav Neurol 2019;2019:5808610. [Crossref] [PubMed]
35. Lutz J, Offidani E, Taraboanta L, et al. Appropriate controls for digital therapeutic clinical trials: A narrative review of control conditions in clinical trials of digital therapeutics (DTx) deploying psychosocial, cognitive, or behavioral content. Front Digit Health 2022;4:823977. [Crossref] [PubMed]
36. Elliott SA, Brown JS. What are we doing to waiting list controls? Behav Res Ther 2002;40:1047-52. [Crossref] [PubMed]
37. Mérelle SY, Sorbi MJ, van Doornen LJ, et al. Lay trainers with migraine for a home-based behavioral training: a 6-month follow-up study. Headache 2008;48:1311-25. [Crossref] [PubMed]
38. Voerman JS. Long-term follow-up of home-based behavioral management training provided by migraine patients. Cephalalgia 2014;34:357-64. [Crossref] [PubMed]
39. Klan T, Liesering-Latta E. Kognitiv-verhaltenstherapeutisches Migränemanagement (MIMA): ein Behandlungsmanual zur Krankheitsbewältigung und Attackenprophylaxe bei Migräne. Göttingen: Hogrefe; 2020.
40. Klan T, Liesering-Latta E, Gaul C, et al. An Integrative Cognitive Behavioral Therapy Program for Adults With Migraine: A Feasibility Study. Headache 2019;59:741-55. [Crossref] [PubMed]
41. Klan T, Gaul C, Liesering-Latta E, et al. Efficacy of Cognitive-Behavioral Therapy for the Prophylaxis of Migraine in Adults: A Three-Armed Randomized Controlled Trial. Front Neurol 2022;13:852616. [Crossref] [PubMed]
42. Michie S, Richardson M, Johnston M, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013;46:81-95. [Crossref] [PubMed]
43. Palmer S, Whybrow A. Handbook of Coaching Psychology: A Guide for Practitioners. 2nd Edition. London: Routledge; 2018.
44. Kelders SM, Kok RN, Ossebaard HC, et al. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012;14:e152. [Crossref] [PubMed]
45. Sieverink F, Kelders SM, van Gemert-Pijnen JE. Clarifying the Concept of Adherence to eHealth Technology: Systematic Review on When Usage Becomes Adherence. J Med Internet Res 2017;19:e402. [Crossref] [PubMed]
46. Donkin L, Christensen H, Naismith SL, et al. A systematic review of the impact of adherence on the effectiveness of e-therapies. J Med Internet Res 2011;13:e52. [Crossref] [PubMed]
47. Kroenke K, Wu J, Yu Z, et al. Patient Health Questionnaire Anxiety and Depression Scale: Initial Validation in Three Clinical Trials. Psychosom Med 2016;78:716-27. [Crossref] [PubMed]
48. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606-13. [Crossref] [PubMed]
49. Kroenke K, Spitzer RL, Williams JB. The PHQ-15: validity of a new measure for evaluating the severity of somatic symptoms. Psychosom Med 2002;64:258-66. [Crossref] [PubMed]
50. Toussaint A, Hüsing P, Kohlmann S, et al. Detecting DSM-5 somatic symptom disorder: criterion validity of the Patient Health Questionnaire-15 (PHQ-15) and the Somatic Symptom Scale-8 (SSS-8) in combination with the Somatic Symptom Disorder - B Criteria Scale (SSD-12). Psychol Med 2020;50:324-33. [Crossref] [PubMed]
51. Klein EM, Brähler E, Dreier M, et al. The German version of the Perceived Stress Scale - psychometric characteristics in a representative German community sample. BMC Psychiatry 2016;16:159. [Crossref] [PubMed]
52. Graef JE, Rief W, French DJ, et al. German Language Adaptation of the Headache Management Self-Efficacy Scale (HMSE-G) and Development of a New Short Form (HMSE-G-SF). Headache 2015;55:958-72. [Crossref] [PubMed]
53. Schwarzer R. Modeling Health Behavior Change: How to Predict and Modify the Adoption and Maintenance of Health Behaviors. Appl Psychol 2008;57:1-29.
54. Benz T, Lehmann S, Gantenbein AR, et al. Translation, cross-cultural adaptation and reliability of the German version of the migraine disability assessment (MIDAS) questionnaire. Health Qual Life Outcomes 2018;16:42. [Crossref] [PubMed]
55. Gantenbein AR, Agosti R, Kamm CP, et al. Swiss QUality of life and healthcare impact Assessment in a Real-world Erenumab treated migraine population (SQUARE study): interim results. J Headache Pain 2022;23:142. [Crossref] [PubMed]
56. Geisser E. FESV. Fragebogen zur Erfassung der Schmerzverarbeitung. Göttingen: Hogrefe; 2001.
57. French DJ, Holroyd KA, Pinell C, et al. Perceived self-efficacy and headache-related disability. Headache 2000;40:647-56. [Crossref] [PubMed]
58. Agbetou M, Adoukonou T. Lifestyle Modifications for Migraine Management. Front Neurol 2022;13:719467. [Crossref] [PubMed]
59. Paersch C, Recher D, Schulz A, et al. Self-Efficacy Effects on Symptom Experiences in Daily Life and Early Treatment Success in Anxiety Patients. Clin Psychol Sci 2025;13:178-94. [Crossref] [PubMed]
60. Lee K, Kwon H, Lee B, et al. Effect of self-monitoring on long-term patient engagement with mobile health applications. PLoS One 2018;13:e0201166. [Crossref] [PubMed]
61. Noser AE, Gibler RC, Ramsey RR, et al. Digital headache self-management interventions for patients with a primary headache disorder: A systematic review of randomized controlled trials. Headache 2022;62:1105-19. [Crossref] [PubMed]
62. Grazzi L, Telesca A, Rizzoli P. Management of chronic migraine with medication overuse by web-based behavioral program during the COVID-19 emergency: results at 12 months. Neurol Sci 2022;43:1583-5. [Crossref] [PubMed]
63. Sorbi MJ, Kleiboer AM, van Silfhout HG, et al. Medium-term effectiveness of online behavioral training in migraine self-management: A randomized trial controlled over 10 months. Cephalalgia 2015;35:608-18. [Crossref] [PubMed]
64. Singh B, Olds T, Brinsley J, et al. Systematic review and meta-analysis of the effectiveness of chatbots on lifestyle behaviours. NPJ Digit Med 2023;6:118. [Crossref] [PubMed]
65. Abd-Alrazaq AA, Rababeh A, Alajlani M, et al. Effectiveness and Safety of Using Chatbots to Improve Mental Health: Systematic Review and Meta-Analysis. J Med Internet Res 2020;22:e16021. [Crossref] [PubMed]
66. Pinto MD, Greenblatt AM, Hickman RL, et al. Assessing the Critical Parameters of eSMART-MH: A Promising Avatar-Based Digital Therapeutic Intervention to Reduce Depressive Symptoms. Perspect Psychiatr Care 2016;52:157-68. [Crossref] [PubMed]
67. Chan WS, Cheng WY, Lok SHC, et al. Assessing the Short-Term Efficacy of Digital Cognitive Behavioral Therapy for Insomnia With Different Types of Coaching: Randomized Controlled Comparative Trial. JMIR Ment Health 2024;11:e51716. [Crossref] [PubMed]
68. Goldberg SB, Lam SU, Simonsson O, et al. Mobile phone-based interventions for mental health: A systematic meta-review of 14 meta-analyses of randomized controlled trials. PLOS Digit Health 2022;1:e0000002. [Crossref] [PubMed]
69. T Minen M. Smartphone-based migraine behavioral therapy: a single-arm study with assessment of mental health predictors. NPJ Digit Med 2019;2:46. [Crossref] [PubMed]
70. Chen X, Luo Y. Digital Therapeutics in Migraine Management: A Novel Treatment Option in the COVID-19 Era. J Pain Res 2023;16:111-7. [Crossref] [PubMed]
71. Schaetz L, Rimner T, Pathak P, et al. Employee and Employer Benefits From a Migraine Management Program: Disease Outcomes and Cost Analysis. Headache 2020;60:1947-60. [Crossref] [PubMed]
72. Stovner LJ, Andree C. Prevalence of headache in Europe: a review for the Eurolight project. J Headache Pain 2010;11:289-99. [Crossref] [PubMed]
73. Stovner LJ, Zwart JA, Hagen K, et al. Epidemiology of headache in Europe. Eur J Neurol 2006;13:333-45. [Crossref] [PubMed]
74. Al-Hassany L, Haas J, Piccininni M, et al. Giving Researchers a Headache - Sex and Gender Differences in Migraine. Front Neurol 2020;11:549038. [Crossref] [PubMed]
75. Nittas V, Daniore P, Chavez SJ, et al. Challenges in implementing cultural adaptations of digital health interventions. Commun Med (Lond) 2024;4:7. [Crossref] [PubMed]
76. Spanhel K, Schweizer JS, Wirsching D, et al. Cultural adaptation of internet interventions for refugees: Results from a user experience study in Germany. Internet Interv 2019;18:100252. [Crossref] [PubMed]