Developing culturally-adapted mobile mental health interventions: a mixed methods approach

Introduction

African American young adults suffer from negative mental health symptoms (e.g., depressive and anxiety symptoms) at either similar or greater rates than that of White Americans, yet they are seven times less likely to have access to or receive effective treatments (1). Given that mental illness contributes to 90% of suicides and that African American young adults are more likely to attempt suicide (1), the disparity in mental health treatments within these communities is a concerning public health issue. Fortunately, mobile-health (mHealth) technology may transform mental health services and address disparities in mental healthcare. mHealth technologies are already being used in a variety of health contexts (i.e., diabetes management), and can take forms ranging from mobile web-based apps or platforms to wearable technologies (i.e., Fitbit watches). A meta-analysis of mHealth for mental health has shown that it can effectively reduce symptoms related to depression, anxiety, and substance use (2).

For mHealth to be effective for African American young adults, it is important that these applications are culturally-adapted to incorporate the unique sociocultural risk and protective factors that influence mental health for this population. Two such sociocultural factors that have a large impact on the mental health of African American young adults are experiences of racial discrimination and racial identity beliefs. For instance, perceived experiences of discrimination are linked to psychological distress (3). Yet, not all African Americans who experience racial discrimination are impacted in the same way, and research has shown that racial identity can both protect against experiences of racial discrimination (4,5), as well as promote better psychological outcomes (6).

There is an urgent need for mHealth technology for mental health symptoms for African American young adults. It is imperative that these technologies are culturally adapted so that they address sociocultural factors such as racial discrimination and racial identity beliefs, especially since these factors have been shown to influence perceptions of mental health treatments (7). In light of these limitations, the current study sought to utilize focus groups of African American young adults to inform the development and design of future culturally-adapted mHealth applications for mental health symptoms.

Background

Barriers to traditional mental health treatments among African American young adults

As stated above, there is a large disparity in the access to and utilization of effective mental health treatments among African American young adults, which may be contributing to rising rates of suicide among this group (1,8,9). Prior to conceptualizing possible solutions to this health disparity, it is important to consider relevant barriers to traditional mental health treatments among this population. Of note, extant literature suggests that disparities in access to care and utilization of effective mental health treatments may stem from a variety of factors, including financial factors (inadequate insurance, low socioeconomic status), stigma, a lack of educational resources about mental health, and a lack of minority providers or providers trained to treat patients from multicultural backgrounds (8). For example, research has shown that discrimination from providers and the historical experience of racism have led to a mistrust of mental health providers within African American communities (1). This mistrust usually manifests in the form of misdiagnosis, inadequate treatment, and lack of cultural competence by health care professionals.

In exploring the barriers specific to African American young adults, it has been shown that African American young adults do not view mental health symptoms as chronic, which reduces their likelihood to seek services (10). Additionally, African American young adults are also less open to treatment because they believe that they can self-manage their symptoms (10). Finally, compared to other age groups, African American young adults have high levels of stigma related to seeking mental health treatment (11). Taken together, these barriers to traditional mental health treatments highlight the urgent need to explore alternative methods to deliver effective mental health treatments to this population.

The promise of mHealth technologies

mHealth technologies may transform the delivery and utilization of mental health services and address disparities in mental healthcare among African American young adults. Research has shown that mHealth technologies have now been utilized in a variety of health contexts (e.g., diabetes management) (12) and have shown positive outcomes (13). Research has also begun to highlight how mHealth may transform mental health service delivery (14,15) and can lead to additional benefits when compared to traditional therapy (16). In fact, in the past decade, several efforts have been made to use mHealth to address negative mental health symptoms such as depressive symptoms, substance use, and bipolar symptoms, or to promote psychological well-being, such as improved coping strategies and mindfulness (17-21). For instance, Reid and colleagues (20) designed an mHealth intervention for mental health symptom assessment and management for adolescents and young adults. In their randomized control trial, they found that the mHealth application led to significant reductions in depression, anxiety, and stress symptoms when compared to an attention control group (20).

Since then, meta-analyses have shown that mHealth for mental health can effectively reduce symptoms related to depression, anxiety, and substance abuse (2) as well as promote more adaptive mental health practices (i.e., mindfulness, meditation) and promote better well-being (19,22). In regards to psychotherapy, mHealth for mental health can be used in a variety of ways. For example, some have used mHealth technologies as an added benefit to support and facilitate treatment (23), whereas others have used mHealth as the primary intervention for psychological symptoms (24). Given that mHealth has been shown to be effective for treating psychological symptoms, and given the flexibility of its usage, mHealth technologies may help to address current disparities in access to and utilization of mental health services among African American young adults (1,8). Yet, given the associations between sociocultural risk and protective factors and psychological well-being among this population, it is important that mHealth technologies consider how these factors may impact the effectiveness and utilization of these products.

Sociocultural risk and protective factors that influence psychological well-being

As stated above, in order for mHealth to be effective, it may be important to understand the unique sociocultural risk and protective factors (i.e., racial discrimination and racial identity, respectively) that influence psychological well-being among African American young adults. First, offline and online racial discrimination experiences are prevalent issues for African American young adults (25,26). Studies have shown that African Americans continue to experience more racial discrimination than any other ethnic/racial group in the U.S. (25). Even more alarming, these experiences of discrimination are linked to negative psychological outcomes (i.e., decreased self-esteem and decreased positive outcomes, increased anxiety, depressive symptoms) (3).

In contrast, racial identity beliefs, or the significance and qualitative meaning of race to the self-concept of an individual (27), can both protect against these experiences of discrimination and promote positive psychological outcomes among African American young adults (4-6). More specifically, racial identity beliefs, especially positive feelings about one’s race, are positively associated with academic adjustment, well-being, social functioning, and self-esteem, and negatively associated with depressive symptoms, externalizing and internalizing problem behaviors, and health risk outcomes (6,28). Additionally, racial identity pride and belonging may lead to a general sense of well-being and buffer against racism for African Americans, and reduce distress when one is exposed to a negative event that may be a function of discrimination (4).

Limitations of existing research and study rationale

Given disparities in access to and utilization of mental health treatments among African American young adults, mHealth technology for mental health symptoms may address this health disparity. As mHealth technologies are developed for this population, it is important that they are culturally sensitive to the unique sociocultural risk and protective factors (racial discrimination and racial identity beliefs, respectively) that influence the psychological well-being of those who identify as members of this group. For example, a study of mHealth created for African American rural families showed that when culturally-adapted mHealth technology is designed, it is just as effective as treatment as usual. This study also suggested that culturally-adapted mHealth may be more effective than traditional treatments as racial-ethnic minorities were more likely to use mHealth than attend in-person sessions (24).

Even more important, research must understand how sociocultural factors may directly impact the utilization of mHealth technologies. For example, racial discrimination has been linked to higher levels of stigma related to mental health treatments, whereas stronger racial-ethnic identity beliefs have been linked to lower levels of stigma related to mental health treatment seeking (7). Going forward, it will be imperative that future mHealth interventions take into account the associations between racial discrimination, racial identity beliefs, mental health, and mental health treatment seeking attitudes.

Unfortunately, despite the utility of mHealth technologies for treating psychological symptoms, African American young adults are still underrepresented in mHealth research, despite their high ownership of smartphones (29), and their common use of smartphones to access the Internet (30). Similarly, previous studies of mHealth have centered majority White samples (23). Doing so limits our understanding of how mHealth should be developed and designed for this group to effectively target disparities in access to and utilization of mental healthcare. Going forward, it is important for researchers to consider methods that incorporate the lived experiences of African American young adults (i.e., focus group interviews) in the design and development of mHealth platforms. This will allow researchers to explore and highlight which features and content are relevant to be included in effective mHealth interventions for this group. Finally, as outlined by Stephan and colleagues (31), mHealth development for mental health symptoms should be informed by a review of the literature, including a review of current mHealth for mental health treatment. Yet, as highlighted above, African American young adults are underrepresented in mHealth research, so the effectiveness and acceptability of current mHealth options for psychological symptoms among this group is unknown. Moving forward, researchers should explore if current mHealth technologies for mental health are acceptable for this population in an effort to assess if there is an urgent need for novel, culturally-adapted applications.

Current study objective

The current study addresses an important gap in the literature by utilizing focus groups of African American young adults to inform the future development and design of culturally-adapted mHealth applications for treating mental health symptoms. The study utilized a mixed-methods approach, in that quantitative (i.e., surveys) and qualitative (i.e., focus groups) methods were used to explore the research questions of interest. With regard to the quantitative methods, the current study explored the associations between racial discrimination, racial identity, and attitudes regarding utilizing mHealth interventions. Based on prior studies (4-6,25,26), it was expected that there would be positive associations between racial discrimination, racial identity, and attitudes regarding utilizing mHealth interventions, and negative associations between racial discrimination, racial identity, and attitudes regarding utilizing mHealth.

In terms of the qualitative approach, this study explored: (I) what are African American young adults’ perceptions of the associations between race-related factors (e.g., racial identity, racial discrimination, etc.) and their mental health and mental health treatment seeking attitudes, (II) what features do African American young adults desire within culturally-adapted mHealth interventions, and (III) how do African American young adults experience and evaluate current mHealth options. This study utilized an inductive approach to explore themes, which means findings relied on the recorded data from the participants, rather than a priori assumptions or frameworks (32).

Researchers have found that the construction of mHealth interventions should be a patient-centered process, and that mHealth interventions are more successful if the intended audience is incorporated in the initial development and design process (12). In light of these findings, an important feature of this study is that it utilizes a sample of African American young adults to inform the development of future mHealth applications for treating African American mental health.

A mixed methods approach is best in that the focus group narratives were used to provide a safe space for the discussion of experiences of racial discrimination, beliefs about racial identity, and how both factors influence mental health and mental health treatment seeking attitudes. Additionally, focus groups allowed for collaborative brainstorming among participants on the most effective features to include in an mHealth intervention for African American young adults. Next, focus groups allowed for active testing and immediate feedback on the utility of existing mHealth interventions to assess the utility of the creation of novel mHealth applications for this population. Finally, quantitative measures allowed for the objective exploration of how experiences of racial discrimination and racial identity beliefs influence perceptions of mental health treatments via mHealth.

Methods

Participants

Participants were 38 African American young adults recruited from the Chapel Hill, Durham, and Raleigh, North Carolina area. To be eligible to participate, participants had to self-identify as African American, be between the ages of 18 and 25, and have access to and/or own a smartphone. Data were collected over a three-month period during the summer of 2018. Participants included 29 women and 9 men, with an average age of 21. Although each participant identified as African American, 11 participants also identified with other ethnic backgrounds, such as St. Marteen, Pacific Islander, Nigerian, Liberian, Jamaican, and Ghanaian, which represents the heterogeneity of the African American experience and perspectives that were represented in the study. The majority of participants identified their family’s socioeconomic status as Middle Class (n=18), followed by Working Class (n=14), and Upper Middle Class (n=6). All participants were either in college, or had obtained at least a Bachelor’s degree. Finally, all participants owned a mobile smartphone and indicated that they used mobile applications “several times a day”.

Procedure

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by institutional ethics board of the University of North Carolina at Chapel Hill (Ethics Approval No. 18-0915) where the research was conducted and informed consent was taken from all the participants. Following university Institutional Review Board approval, participants were recruited via university listservs, word-of-mouth, and social media groups (i.e., GroupMe message channels aimed at African American young adults in the aforementioned areas). Eligible participants were randomized into one of twelve mini-focus groups that ranged from three to five participants per group. In general, focus groups allow researchers to highlight contrasting views within subgroups, and they are able to highlight which views are common across groups (33). Mini focus groups have the same advantages as larger focus group and were chosen as the ideal focus group format as they more easily mirror the natural dynamics of a conversation between acquaintances and allow for more interactions between participants and in-depth exploration of complex issues, such as mental health (34). Additionally, mini focus groups allow for better control of dominating individuals and the probing of less responsive group members (34).

All groups were conducted in the UNC Department of Psychology and Neuroscience. At each focus group session, research assistants led participants through the informed consent process, and participants completed the quantitative measures via the Qualtrics online survey software as they waited for the focus group session to begin. The quantitative measures included demographic information, as well as surveys that assessed experiences of racial discrimination, racial identity beliefs, and attitudes towards mHealth technologies for mental health.

Quantitative measures

Sociodemographic Information

Participants reported sociodemographic data, which were used to highlight the characteristics of the participants. This information consisted of gender, age, race/ethnicity, socioeconomic status, and frequency of mobile-app use.

Experiences of Racial Discrimination

Experiences of racial discrimination were measured using both the Daily Life Experiences Scale (DLE) (35), a self-report scale to assess past experiences with offline racial discrimination over the past six months (α=0.94), and the Online Victimization Scale (36) (OVS; α=0.82) which was used to assess past experiences with online racial discrimination (ORD) over the past year. The DLE subscale is a self-report measure of the frequency and bother associated with 18 independent microaggressions participants have experienced due to their race on a scale from 1 = never happened to 5 = once a week or more. The OVS consist of items assessing both vicarious and individual, or personally-mediated, experiences of ORD, from 1 = never happened to 6 = a daily basis. Previous studies have illustrated that both measures have reliable and valid psychometric properties within similar samples of African American young adults (36-39).

Racial Identity Beliefs

Racial identity beliefs were assessed using a shortened version of the Multidimensional Inventory of Black Identity (MIBI-S) (40). Responses on the MIBI-S assess the three stable dimensions of Black racial identity put forth by Sellers and colleagues (27). Specifically, the Centrality scale, which consists of four items, measures the extent to which being African American is central to participants’ definitions of themselves or self-concept (i.e., “In general, being Black is important to my self-image”). Next, the Regard scale is composed of two subscales assessing Public and Private Regard. The Public Regard subscale consists of four items that measures the extent to which participants feel that other ethnic/racial groups have positive feelings toward African Americans (i.e., “Overall, Blacks are considered good by others”), whereas the Private Regard subscale measures the extent to which participants have positive feelings toward African Americans in general (i.e., “I feel good about Black people”) and consists of three items.

The Ideology scale is comprised of four subscales: Assimilationist, Humanist, Oppressed Minority, and Nationalist. The Assimilationist subscale, which consists of four items, assesses the extent to which participants emphasize the similarities between African Americans and mainstream American culture/systems (i.e., “Blacks should strive to be full members of the American political system”). The Humanist subscale, composed of four items, measures the extent to which respondents emphasize the similarities among individuals of all racial-ethnic backgrounds (i.e., “Blacks should judge Whites as individuals and not as members of the White race”). The Oppressed Minority subscale, which also consists of four items, measures the extent to which participants emphasize the similarities between African Americans and other ethnic-racial minority groups (i.e., “The racism Blacks have experienced is similar to that of other minority groups”). Finally, the Nationalist subscale, which consists of four items, measures the extent to which participants emphasize the uniqueness of being African American or Black (i.e., “Whenever possible, Blacks should buy from other Black businesses”). Participants were asked to respond to each of the items using a 7-point Likert-type rating scale from 1 = strongly disagree to 7 = strongly agree, and previous studies have illustrated the construct and predictive validity for the MIBI (41).

mHealth attitudes

To assess participants’ attitudes towards mHealth interventions, the primary investigator created a novel scale for the purpose of the study (α=0.99). The scale consisted of 13 items that assessed three dimensions: participants’ attitudes towards mental health and utilizing mental health treatment, participants’ attitudes towards the appropriateness of utilizing mHealth interventions for treating psychological symptoms, and participants’ desire for culturally-adapted mHealth interventions for mental health. Sample questions included: “It is important to offer mental health services through a mobile-health app”, “A mobile app would be useful for finding treatment and resources for coping with negative mental health symptoms”, “My mental health is important to me”, and “The African American/Black community would benefit from a mobile mental health app geared towards us”. Each item was rated on a 7-point Likert scale, from 1 = strongly disagree to 7 = strongly agree.

Qualitative approach

The primary investigator served as the primary focus group facilitator, and research assistants served as note takers. After introductions and rapport building, the focus groups revolved around a variety of topics and the focus group facilitator followed a semi-structured focus group guide that was created based on the research questions of interest. Each primary question was also followed by optional probing questions that allowed the focus group leader to go more in-depth on topics that of were particular interest of the group. The focus group guide included some of the following primary questions:

• What is your perception of mental health and mental health treatments?
• What barriers/obstacles get in the way of finding help for mental health?
• Have experiences of discrimination been related to mental health symptoms for you or someone you know?
• If there was an app that promoted mental health in African Americans, what kinds of features should it have?

Finally, at the end of the focus group, each participant evaluated current mHealth applications for mental health that were available on the market to assess the need for a new, culturally-adapted mHealth intervention for this population. Prior to the focus groups, a team of research assistants conducted a literature review and review of the “Apple App Store” and “Android App Store”. Only technologies that targeted mental health and were free to use were selected for the initial review process. From there, each research assistant tested each application and rated them based on ease of use, relevance to mental health, evidence of effectiveness, and cultural relevance. Based on these subcategories and total ratings, the following applications were selected to use for the focus groups: CBTi, Mood Coach, STAIR (all three were selected due to high ratings on evidence base and/or ease of use; created by the U.S. Department of Veterans Affairs), and the Safe Place (selected for high ratings on cultural relevance; created by Jasmin Pierre). All four applications were available in the “Apple Store”, and eligible participants were instructed to download the applications prior to focus group sessions. Research assistants ensured that each participant had access to each application prior to each focus group, and research assistants had back-up hardware with the applications accessible if participants were unable to download or access the application during the session. Participants were given 10 to 20 minutes to test each application during focus groups, and the final section of the focus group involved the participants providing feedback on the applications. Examples of the questions used to guide the evaluation of these current applications are:

• Which apps did you like the most? The least? Why?
• What features did you like and dislike?
• What would make the apps more accessible to African American young adults?

Sessions lasted on average 90 minutes, and each participant was compensated $25 at the end of each focus group. The focus group facilitator and research assistants debriefed after each focus group session. Each group was audio-recorded, and later uploaded to a secure network that were later sent to a professional transcription company for verbatim transcription.

Statistical analysis

Quantitative data analysis

The quantitative data were explored primarily through descriptive statistics, such as means, standard deviations, and correlational analyses.

Qualitative data analysis

Interpretation of the qualitative data were guided by components of thematic analysis by Braun and Clarke (32). For instance, we determined the key findings within each of the overarching research questions using an inductive approach to interpreting respondents’ answers. Additionally, we utilized an open coding process as described in studies such as Hudson and colleagues (42), and relied on an inductive approach to interpret participants’ answers (32). For instance, the primary investigator and research assistants first reviewed a subset of the transcripts and debriefing notes. From there, the team independently created notes, highlighted themes and accompanying texts, and created an initial codebook and an initial list of codes. Codes were created by identifying, coding, and sorting text segments into higher order themes based on patterns in the data (43). From there, the research team met to compare notes and the initial codes and codebook, and resolved any discrepancies, disagreements, or new codes that emerged in the initial codebook creation process to increase inter-rater reliability. As a result, a final list of codes emerged based on this team approach, and the first author independently coded the remaining transcriptions.

Finally, thematic analysis was used to identify recurring themes across focus groups (32). This study utilized an inductive approach to explore themes, which means the codebook relied on the recorded data from the participants, rather than a priori assumptions or frameworks. The primary investigator explored overarching themes across the focus groups and combined these findings with the quantitative data and extant literature to explore the meaning of the obtained data (44). More specifically, recurrent themes were summarized, revised, mapped, and interpreted to explore the research questions highlighted above (e.g., what features do African American young adults want in a culturally-adapted mHealth intervention).

Results

Quantitative analyses

Offline and online racial discrimination experiences, racial identity variables, and attitudes towards mental health and mHealth treatment

Quantitative analyses consisted of examining means, standard deviations, and correlations among offline racial discrimination, individual and vicarious experiences of ORD, racial identity variables, and mental health/mHealth attitudes (Table 1). On average, participants responded that they experienced offline racial discrimination between once and once a month over the past six months (M =2.61, SD =0.93), and 97.3% of the sample experienced at least one instance of offline racial discrimination. In terms of individual ORD, participants reported experiencing it between never and once over the past year (M =1.71, SD =0.58), and 78.4% of the sample experienced at least one instance of individual ORD. In contrast, they reported that they experienced vicarious ORD between a few times a month and a few times a week over the past year (M =4.20, SD =1.21), and 97.3% of the sample experienced at least one instance of vicarious ORD.

Regarding racial identity, participants reported high levels of private regard (M =6.59, SD =0.67), racial centrality (M =6.01, SD =0.73), assimilationist ideology (M =5.93, SD =0.84), and nationalist ideology (M =5.31, SD =0.70). They also reported moderate levels of oppressed minority (M =4.80, SD =0.92) and humanist (M =4.41, SD =0.77) ideologies. In general, participants reported low levels of public regard (M =3.26, SD =1.01). Finally, regarding attitudes towards mental health and mHealth treatments, participants reported strong positive feelings towards utilizing mental health services (M =6.11, SD =0.75), positive attitudes towards using mHealth for treating mental health symptoms (M =5.89, SD =0.75), and a strong desire for culturally-adapted mHealth technologies (M =6.35, SD =0.60).

Next, the associations between racial discrimination and attitudes regarding utilizing mHealth interventions were explored using zero-order correlations among these constructs. Of note, more frequent experiences of vicarious ORD were positively associated with a stronger desire for culturally-adapted mHealth interventions for mental health (P=0.32). Offline racial discrimination and individual ORD were not associated with mental health treatment seeking attitudes, attitudes towards using mHealth, or desires for culturally-adapted mHealth. Next, those who felt positively about being African American (i.e., high private regard) and those who emphasized the uniqueness of the Black experience (i.e., high nationalist ideology) were more likely to have positive attitudes towards mental health and mental health treatments (P=0.02 and P=0.004, respectively). Furthermore, higher levels of private regard were associated with more positive attitudes towards the appropriateness of utilizing mHealth for mental healthcare (P=0.006). Higher levels of private regard, as well as nationalist ideology, were also associated with stronger desires for culturally-adapted mHealth applications for mental health (P=0.05 and P=0.03). Moreover, those who emphasized similarities between African Americans and mainstream American culture/systems (i.e., high assimilationist ideology) also tended to have stronger desires for culturally-adapted mHealth applications for mental health (P=0.01).

Qualitative analyses

African American young adults’ perceptions of race-related factors, mental health, and mental health treatment seeking attitudes

The first qualitative research question explored African American young adults’ perceptions of the associations between their racial identity and experiences of discrimination, mental health, and mental health treatment seeking attitudes (Table 2).

Theme one: “Feeling like everyone expects you to fail is exhausting”

First, participants described a variety of unique stressors that African American young adults face that impact their mental health. These experiences included instances of microaggressions and interpersonal discrimination (e.g., negative classroom experiences with professors and peers, police encounters, and discrimination in healthcare settings). Focus groups also highlighted experiences of vicarious discrimination (i.e., witnessing family members experience unfair policing/police brutality, etc.). They often noted that these experiences led to increased hypervigilance in their daily activities (i.e., being “on edge”), feelings of isolation, and other negative emotional reactions (i.e., anger, sadness, frustration). Additionally, participants noted that negative societal standards or stereotypes of what it means to be an African American have negatively impacted their mental health. They discussed their awareness of historical oppression and trauma experienced by African Americans, intersecting negative experiences between their racial, gender, and/or sexual identities (i.e., experiencing the “strong black woman stereotype”, experiencing both racism and homophobia), as well as “tokenism” (i.e., having to represent “all black people” in academic or professional spaces). Similarly, participants noted how expectations about how African Americans should cope with stress impacts their psychological well-being. For example, participants described notions of a “forced resilience”, or being expected to “push through” or cope with stress without succumbing to it (i.e., “You don’t cry, you just bite the bullet”), and these beliefs were related to treatment seeking attitudes:

Participant 7: Yeah I think it’s just all about not showing any vulnerabilities in the black community and just trying to set yourself up for or make yourself seem stronger than you actually may be. Which may be detrimental in the long run because if you don’t seek help when you need it the most, then you’re gonna become even more vulnerable than you are once you started experiencing any mental health issues.

Participant 8: Yeah, I agree. I think it’s just like, you kind of don’t wanna show weakness. You wanna show that you can handle yourself and then people kind of take that too far.

Additionally, participants discussed how interracial contexts can also negatively impact mental health. For example, they discussed the pressure of being minoritized or “the only one” in majority White academic and/or professional spaces, and the pressure of “having to work harder because of being a minority” in majority White spaces. More specifically, participants highlighted that environments where there is either a lack of community for African Americans, or a lack of African Americans present, leads to feelings of isolation and exacerbates maladaptive mental health symptoms. Taken together, the above findings highlight the myriad of ways that social and cultural stressors and expectations that are unique to African Americans lead to psychological distress, maladaptive coping symptoms, and feelings of isolation that may impact attitudes towards seeking mental healthcare.

Theme two: “Being Black is lit”

Focus groups also discussed a variety of culturally specific protective factors that contribute to the psychological well-being of African American young adults. Broadly, groups discussed how their racial identity and past experiences of discrimination has led to a feeling of “resiliency” (e.g., feeling optimistic in the face of stressors, increased self-efficacy as a result of overcoming adversities). They described that professional accomplishments contribute to increased psychological well-being in that it often relates to succeeding despite negative experiences related to individual, structural, and cultural racism. For instance, one participant stated, “I feel accomplished because there are a lot of barriers…I’m in UNC, and I see other black people here. It makes me feel proud, I feel even better about myself.” Additionally, having various cultural and ethnic backgrounds from the African diaspora also contributed to positive self-esteem for participants, and many of them noted how being a first-generation African or Caribbean American, in addition to identifying with African American culture, contributed to feelings of increased pride and happiness about being Black in this country.

Focus groups also often discussed how being African American allows for opportunities to experience increased social support from community spaces, such as from churches or in predominately Black spaces and/or social organizations (i.e., feeling connected to other Black young adults due to similar cultural experiences, interests, and beliefs). For instance, these associations are highlighted in this focus group excerpt:

Participant 3: I think just in general the pride in your culture and the pride in your family and even though you face hardships, the fact that you get through them and have community that’s trying to encourage you and build you up to get through it together, it’s that sense of camaraderie that you get when you walk on a campus like this or NC state or anywhere else and you see another face that looks like you. It’s just lit.

Participant 4: It’s lit.

Participant 3: And we created America, basically.

The protective function of social support also extended to seeking mental health treatment. For example, participants discussed how they felt as if they received adequate social support regarding seeking mental health treatment and coping with mental health symptoms from their African American young adult peers. They described how friend groups offered a “safe space” to show vulnerability. They also described being encouraged by their peers, and often how friends provided advice and support for seeking mental health services when participants felt unable to talk to their parents or other older adults. Additionally, participants discussed that increased conversations on social media platforms among African American young adults have begun to reduce their perceived stigma around mental health and seeking mental health treatments. Overall, the current findings highlight that past experiences of discrimination and increased connection and social support from same-race peers can lead to increased feelings of racial pride or positive feelings about being Black, which are also linked to overall psychological well-being (i.e., “If you don’t feel good about your race, then you can’t be happy”). Additionally, social support and a strong connection to same-race peers may also influence positive attitudes towards mental health and seeking mental health treatments.

Theme three: “Drink tea and pray”

In terms of how race-related factors negatively impact mental health treatment seeking attitudes, focus groups discussed a variety of culturally specific barriers to utilizing mental health services (e.g., stigma, lack of social support, exclusive reliance upon non-medical and/or religious solutions, etc.). First, participants described how stigma around discussing mental health, both among peers their age as well as among their family members, is a factor that often led to a reluctance to seek mental health services. Second, participants described a general lack of awareness about mental health and a perceived lack of need to seek mental health services among Black communities and families. They described that over time, this lack of awareness may lead to decreased utilization of mental health services. Importantly, this lack of awareness was also associated with a misunderstanding about the nature of mental health and mental health services. For example, focus groups often highlighted how a lack of education about mental health among their parents and older African American generations often led to misinterpreting the causes or presentation of symptoms as laziness or controllable.

Third, participants described how these factors often led to not only decreased utilization of mental health resources, but also a perceived lack of social support from family members due to them being unaware of signs of psychological distress. This was often described as “generational differences” in understanding and utilizing mental health treatments, such that African American young adults felt as if their generation supports mental health treatment seeking, but that older generations did not:

Participant 51: Right. It’s never taken seriously with the family. As older generation, but yeah, that goes back to saying, like, the generational thing. Like, I see now, like, our whole- our generation is always on social media, talking about mental health and saying how they- giving out tips of how they do things everyday and stuff. And, but if you- I do that with my family, then it’s like completely weird.

Participant 32: Yeah. It’s weird because then my parents will be like, oh you can share anything with us-

Participant 51: No, we can’t.

Participant 32: And be like, “Yeah, not feeling well today.” I don’t want to hear it. I’m like, “but well, I tried.”

Participant 51: (laughs)

Participant 32: (laughs) Like, you know? I can’t like, I don’t know. But um, yeah. It’s kind of hard, especially if, well, at least I know for my parents specifically, it’s kind of just like I think that doesn’t really exist. It’s like oh, it’s all in your head, like, you just go do some work.

Participant 51: Mm-hmm (affirmative).

Participant 32: Go do some readings. You’ll be fine. Think about it later. Then if you just come up and mention it again, they’ll just be like, “Oh, you’re complaining.” I’ll be like, “Okay, well. I’m done with this conversation. (laughs) Okay.”

Finally, focus groups often discussed a variety of family socialization messages they received growing up about mental health that also decreases African American young adults’ motivation to seek mental health services. For example, participants discussed receiving messages related to hiding mental health symptoms to appear strong (i.e., maladaptive coping strategies), as well as relying exclusively on religious or non-medical interventions to cope with mental health symptoms (i.e., “drink tea and pray”). Taken together, focus groups often noted how these interactions were viewed as non-supportive and ultimately influenced by parents’ lack of education about mental health services and mistrust of mental health providers. Interestingly, these interactions are not specific to young adults from traditional African American families, and many first-generation African and Caribbean young adults also reported similar non-supportive messages and interactions:

Participant 14: Or like with me, I’m Nigerian and my parents are also Nigerian and just the way ... Okay so like they relate mental health to being, they call it madness.

Participant 17: Same in Uganda culture.

Participant 14: So it’s weird cause if you say if someone ... they relate it to madness. It’s more of a joke... but it’s just weird because then there’s different layers to mental health. Kinda like your grandma, like they just brush it aside like, “Oh it’s a problem”. But I think with that they think it’s crazy, like you can’t control, it is what it is, more of like you can’t fix it, you can’t get support…

Taken together, these findings highlight that a lack of education around mental health and mental health services, which is likely influenced by systemic and structural factors that affect both young adults and older generations, has led to a decreased likelihood of utilizing traditional mental health services. More importantly, it appears that this lack of education influences maladaptive socialization messages received from family members related to how individuals understand and cope with psychological distress, which may also impact treatment utilization.

Theme four: “Doors are open for white people”

Focus groups also highlighted that African American young adults are often at a disadvantage when seeking mental health services as compared to their White peers. More specifically, participants often noted how White young adults have increased access to mental health services. For instance, participants often discussed that they perceived that White young adults and their families/parents have more knowledge about mental health services and symptoms, and that conversations around mental health are not stigmatized within White communities. They also perceived that White young adults had more social support from others around seeking mental healthcare, and more positive attitudes towards utilizing mental healthcare services. One participant even highlighted how although the rates of psychological distress between African American and White young adults are similar, sociocultural barriers may lead to current disparities in mental healthcare:

Interviewer: Do you think that mental health is different for African Americans than it is for White Americans?

Participant 15: I would probably say, like, the rates for African Americans and other races are probably about the same or maybe slightly lower, give or take some, but I feel like it’s more so the conversation around…mental health in African American communities because there’s a huge stigma. I feel like often times it’s pushed aside or something that’s not discussed, or something, um, that people feel like you need to be ashamed of and they see it as a sign of weakness, not something that a lot of people go through. Um, so I, to answer your question, I don’t think there’s, like, a huge disparity. I think it’s just the nature of how we have those discussions.

These perceived racial differences may even present unique challenges for bi-racial African American young adults, as one bi-racial participant noted that when discussing mental health problems and treatments, they have an easier time receiving social support from their White family members as compared to their African American family members. Altogether, these findings highlight how cultural and structural factors (e.g., reduced stigma and increased education about mental health within White communities) may contribute to disparities in utilizing mental healthcare between White and African American young adults.

Theme five: “The only Black person I saw was the receptionist”

Finally, participants described a variety of systemic barriers to seeking mental health services, such as a lack of finances to cover treatment (e.g., costs of sessions being too expensive, lack of adequate insurance), and a lack of availability of services (i.e., lack of providers in predominately Black and/or low-income communities). They also described how a lack of therapists of color or Black mental health providers is often a barrier to seeking mental health services:

Participant 19: There’s a lack of black representation. I know at least what I’ve seen as far as reading and seeing things on the internet, as far as mental health resources and providers, it did seem very whitewashed. I know for me, with mental health, just like with any other health provider, I’m going to feel a lot more comfortable if I can go to somebody that I know will understand my particular racial differences. Sometimes that’s not available.

Participant 27: And then again, I feel mental health is becoming on the forefront now specifically, but it’s definitely geared towards white people. So, we’re still being felt like we’re on the back-burner, even though it’s prevalent now, taking care of your mental health and things.

Even more concerning, when college students knew about accessible mental health resources on campus (i.e., Campus Psychological Services; CAPS) that they could utilize despite the systemic barriers above, they often discussed how CAPS was limited for students of color. Participants often discussed that they found the brief/problem-focused psychotherapy format of CAPS unappealing, or that they felt as if CAPS did not offer a “safe space” for students (i.e., limited confidentiality given the likelihood of being seen by other students/campus members). Most importantly, many focus groups noted how they did not perceive that CAPS could offer culturally relevant services:

Participant 29: Alright, great. I think another main barrier is the lack of African Americans in that aspect. It’s like, for instance, for me, when I went to CAPS, every person I talked to was white. There was not a black person except at the reception desk. So like, that might be a barrier because I might be trying to talk about something they wouldn’t understand, so why would I speak about it if you can’t understand, and I actually, like, understand anything I’m going through…

Taken together, these findings highlight that systemic barriers to utilizing mental health services (i.e., financial barriers), in combination with a lack of providers of color and perceived lack of culturally relevant services, often contributes to decreased utilization of mental health services among African American young adults.

Desired features in culturally-adapted mHealth interventions

The second qualitative research question explored the features African American young adults desire within culturally-adapted mHealth interventions. The focus groups also explored desired topics and content related to mental health and culture, as well as factors that would impact utilization of mHealth interventions (Table 3). First, focus groups often highlighted a variety of standard features that could be considered interventions for promoting adaptive mental health symptoms. For example, they stated that they would desire daily inspirational quotes related to psychological well-being and/or positive quotes related to being African American, the ability to engage in meditation and/or deep breathing exercises, minigames related to de-stressing, and the ability to access forums and/or discussion boards to build social support and discuss mental health related topics with other African Americans (i.e., often described as a “safe space”). Focus groups also discussed the desire for a “resource bank” that included coping and self-care strategies, psychoeducation about mental health symptoms, frequently asked questions about seeking mental health treatments, as well as the ability to journal (i.e., receiving daily journal prompts). Participants described wanting applications to be able to provide a variety of recommendations, such as being able to recommend Black or culturally-sensitive mental health providers in their area, or being recommended media (i.e., music, podcasts, videos, etc.) related to increasing their psychological well-being.

Most importantly, focus groups often discussed desiring the option to personalize applications, as well as have the ability to create and manage a profile. This profile would house their recommendations, current treatment plans, demographic information, personal preferences (i.e., favorite music, movies, television shows, etc.), as well as track their progress (i.e., monitoring changes in psychological symptoms). Based on their profile and how they use applications, participants discussed wanting personalized “suggested activities” to promote well-being. They also wanted their profile to have the ability to create a “support system” within applications, and the ability to connect to other users in an effort to create community and find social support during periods of psychological distress. Finally, participants described wanting to be able to assess their “warning signs” (i.e., symptoms of psychological distress), so that applications can keep progress of these symptoms and recommend interventions (both through the application or in the community) if these symptoms became severe.

Focus groups also discussed a variety of features related to racial identity that they would want incorporated into mHealth interventions. For instance, they discussed desiring activities to reaffirm and explore their Black identity (i.e., minigames or short readings describing positive qualities of the Black experience, positive trends in Black culture). Additionally, they discussed desiring content that promoted racial identity development, such as positive narratives about the meaning of being Black, such as “counter narratives” that would combat the negative stereotypes that exist in American culture. Finally, focus groups discussed desiring special topics within the forums/discussion boards that are related to racial identity and discussing the meaning of being Black, and an opportunity to discuss and/or read about common experiences that African American young adults encounter. Overall, focus groups often discussed how racial identity development should be personalized to each individual, and often described it as a “journey” or a process towards developing a “global identity”, which highlights the importance of such applications to take into account the heterogeneity among Black racial identity beliefs.

Focus groups also highlighted the need for applications to directly provide resources for coping with experiences of racial discrimination. For example, groups discussed wanting to be able to report instances of individual discrimination (i.e., workplace discrimination, online discrimination), as well as resources related to obtaining legal advice or utilizing interventions for dealing with individual and structural experiences of discrimination. Additionally, focus groups described wanting resources to be better able to “identify” subtle experiences of discrimination or microaggressions, as well as content related to understanding how experiences of racial discrimination (i.e., police brutality) may influence mental health symptoms. Finally, participants often discussed how applications should be able to help individuals process their emotional reactions to experiences of discrimination, provide affirmations (i.e., presenting positive quotes related to being Black, such as offering positive cultural history facts) after experiences of discrimination, and also teach problem-solving skills or possible solutions related to encountering future experiences of discrimination or microaggressions.

Desired topics and content

Second, focus groups discussed topics and content that they would desire in culturally-adapted mHealth interventions for mental health. As noted above, focus groups participants desired content related to racial identity and race-related stress/racial discrimination. Moreover, participants described wanting psychoeducation on daily stressors that affect mental health (family/relationship stress, academic stress, and the association between stress and mental health), as well as common psychological symptoms (e.g., anxiety symptoms, depressive symptoms and mood disorders, panic attacks, personality disorders, and ways to self-assess symptoms responsibly). Given the rising rate of suicide among African Americans (1) participants also described desiring content related to understanding and recognizing suicidality among African Americans. Given the age group and the transition between high school and college, some focus group participants also described desiring contents related to their high school experiences, such as coping with bullying (particularly on social media), building self-esteem, transitioning through puberty, and understanding the effects of social media on mental health. Finally, focus groups discussed desiring content related to coping with psychological distress (e.g., recovery steps from substance abuse or severe depression, self-care strategies), as well as content related to self-empowerment (e.g., resources related to increasing one’s self-image and self-esteem).

In regard to how to facilitate adaptive mental health practices, focus groups also described a variety of strategies that culturally-adapted mHealth should promote. For example, they highlighted the importance of promoting self-care practices (i.e., increasing one’s self-awareness of their mental health symptoms, journaling, and meditation/mindfulness practices). They also discussed increasing communication about mental health among their peers, as well as increasing their social support. They also desired activities unique to the Black experience to decrease psychological distress (i.e., culturally-relevant behavioral activation options such as attending a culturally-relevant concert). Finally, despite non-supportive family socialization messages related to religion noted above, some participants did highlight the desire to have content related to using religion and spirituality to promote psychological well-being.

Application utilization preferences and considerations

Next, the focus groups explored factors that would impact utilization of culturally-adapted mHealth interventions and ways to increase utilization of such technologies among African American young adults. For instance, participants described the importance of anonymity and confidentiality, such that users should have the option to hide their profile if they are only interested in browsing content or utilizing resources (versus interacting with other users in forums, creating a social support network, etc.). Additionally, participants described wanting to utilize the application flexibly in a variety of ways. For example, participants often described wanting to be able to use the application both as an individual intervention (i.e., utilize resources independent of external psychotherapy), as well as an adjunct to treatment (i.e., utilizing resources with the guidance of a mental health provider). In line with this, focus groups often described positive attitudes towards professional assistance from mental health providers when utilizing mHealth interventions. For example, many participants often noted that they would desire the option to access a “hotline” or have phone sessions with mental health service providers during crises or for consultation sessions. Additionally, focus groups described wanting the format of the application to be flexible, such that it should be an application that can be accessed via traditional web browsers, as well as an application formatted specifically for mobile devices and accessible through services such as the Google Play/Apple App Store (formatted for iOS and Android mobile devices).

Focus groups also discussed desiring that such applications be used to recover from psychological distress, as well as a preventative intervention to maintain/promote psychological well-being. Moreover, focus groups often described wanting professional “moderators” to monitor application usage (i.e., removing/suspending accounts that spread false or negative information, facilitating local “meet-ups”, monitoring and removing fake profiles). Importantly, participants highlighted that in order to make such applications accessible to African American young adults, these applications should be free to download and utilize. Participants also described wanting applications to be connected to other mHealth technologies (i.e., FitBit, Apple Watch, Calm, fitness trackers, etc.), in an effort to better manage their overall well-being. In contrast, they discussed wanting limited connectivity to social media applications (i.e., Twitter, Facebook, etc.), because of concerns related to confidentiality or being harassed due to discussing mental health topics.

Given the variety of barriers to mental health treatment seeking among African American young adults, focus groups also discussed a variety of ways that culturally-adapted mHealth applications could be advertised to increase their utilization. First, they described advertising the application via social media sites. They also described how users should be able to advertise the application among their own networks (i.e., having the ability to send a link to download the application to friends or family members). They also desired community demonstration events, such as opportunities to interact with these applications on college campuses in public demonstration spaces. Finally, they described desiring “promotional challenges” to increase utilization of applications (i.e., receiving an award after utilizing the app for a certain amount of time), as well as testimonials and/or reviews from other users.

Second, focus groups often discussed ways to increase motivation to continually engage with culturally-adapted mHealth interventions among African American young adults. For example, focus groups discussed how receiving periodic notifications or “push messages” to open the application, utilize self-care strategies, or engage in other activities (i.e., mindfulness), would motivate them to engage with applications. They also discussed how being able to track progress and receive rewards (i.e., reward points, “leveling” system, etc.) as they utilize the application over time would ensure that they continuously engage with such technologies. They also described that increasing the appeal of the application, both visual appeal (i.e., enticing designs and personalization options), as well as continuously updating the application with cutting-edge research on African American mental health, would increase motivation for utilizing the application over time. Finally, focus groups discussed desiring to be able to share the application and its content with others in order to increase awareness about mental health and mental health treatment seeking to further help build a community within the application.

The evaluation of current mHealth interventions

Finally, focus groups actively tested and evaluated a variety of current mHealth interventions for mental health (i.e., the CBTi, Mood Coach, STAIR and Safe Place applications), and provided their feedback as well as general considerations for improving current mHealth interventions (Table 4). It is important to note that all interventions received both positive and negative feedback, and the purpose of participants’ evaluations were to highlight features that they found particularly appealing, effective (or in contrast, confusing or ineffective). As a result, these user experiences may help tailor future mHealth interventions.

Overall, participants across all twelve focus groups endorsed liking the Safe Place application the most (twenty-five references of the “Liked the Safe Place” code), followed by the Mood Coach (twenty-two references), and the STAIR application (thirteen references). The CBTi application was liked the least amount of times (three references). Across these applications, focus groups highlighted a variety of features that they found appealing or effective. For example, they liked having access to inspirational quotes, popular media, and community forums/discussion boards (the Safe Place). More specifically, the Safe Place often emerged as the most liked application due to its cultural relevancy, as focus groups often described that the content and design was clearly related to Black culture. They also liked the ability to self-assess their psychological symptoms (Mood Coach, STAIR), and receive recommendations for both activities and interventions (Mood Coach). They also liked having the ability to access a recommended list of providers by city and state (the Safe Place). Participants often described applications’ layouts (i.e., visuals, ease of navigation, clarity of content) as appealing factors of applications (Mood Coach & STAIR). Additionally, participants highlighted that psychoeducation about common psychological symptoms (all four applications), the ability to receive notifications (Mood Coach, STAIR, CBTi), the ability to engage in deep breathing and thought monitoring (STAIR), and the ability to set goals and measure progress (Mood Coach & STAIR) as positive aspects of the available applications.

In contrast, participants across all focus groups mentioned disliking the CBTi application the most (ten references of the “Disliked CBTi” code), followed by the Safe Place (five references). Participants described disliking the Mood Coach and STAIR applications overall the least amount of times (one and two references, respectively). In exploring what aspects of these applications participants disliked, unappealing layouts (i.e., “appearing bland”, lack of color, etc.) emerged as the most common complaint (twenty-seven references) for both the CBTi and Safe Place applications. They also described that the length and quantity of articles on the Safe Place, the tutorial mode associated with the Mood Coach, and the quantity of activities on STAIR were overwhelming and/or confusing at times. Additionally, they also described desiring more accountability from applications (Mood Coach, CBTi), such that they felt as if they would not commit to intervention plans if there were a lack of notifications, reminders, or overall encouragement for engaging with the applications.

Participants also described that each application lacked features that they found desirable. For example, in terms of the Safe Place, participants wanted the application to incorporate more psychoeducation about serious mental illness as well as overall well-being (i.e., relationship health, physical health, etc.), the ability to journal, and the ability to search through resources. Furthermore, in terms of the CBTi, STAIR, and Mood Coach applications, participants often described desiring more psychoeducation and content relevant to African Americans and the Black experience. Finally, participants noted that for all applications, they would be more effective if they had the ability to interact with professional mental health therapists either virtually or through discussion boards/forums. In sum, to further improve the tested applications, participants often noted that they would either desire that the CBTi, STAIR, and Mood Coach applications were more culturally relevant, and/or they desired combining the cultural relevance of the Safe Place with the evidence-based interventions and visual appeal from the other three applications.

Discussion

Given the disparities in access to and utilization of mental health services among African American young adults (1,8), mHealth for mental health provides a promising opportunity to increase access to and engagement with evidence-based mental health services. Though limited, technology-based interventions for African Americans (i.e., African American parents/families) highlights the initial effectiveness and acceptability of mHealth (24). Unfortunately, African American young adults are still underrepresented in mHealth research (29), which limits our understanding of how mHealth should be developed and designed for this group. As a result, the current study utilized focus groups of African American young adults to inform the development of future culturally-adapted mHealth applications for treating mental health symptoms.

Next steps for culturally-adapted mHealth for African American young adults

First, in line with extant research (8,45), focus groups discussed how systemic barriers specific to African Americans (i.e., lack of insurance or ability to afford therapy), coupled with a lack of desirable local providers (i.e., lack of African American providers or culturally sensitive providers), leads to disparities in utilizing mental health services. Even more concerning, participants felt as if they were at a disadvantage to their White young adult peers, given that these barriers do not exist for them combined with more positive attitudes towards mental health among White young adults (10). As a result, the findings suggest that the development of free/affordable, culturally-adapted mHealth specifically for African American young adults is an urgent need and may help reduce disparities in utilization of mental healthcare services. These technologies may also build feelings of self-efficacy for coping with psychological distress among this population.

Second, in line with a large body of research (45,46), the findings also highlight that a variety of race-related stressors lead to psychological distress and influences maladaptive coping symptoms among African American young adults. The creation of culturally-adapted mHealth interventions may provide resources to aid African American young adults in coping with these unique stressors (i.e., techniques to reduce feelings of hypervigilance after experiences of vicarious discrimination). Furthermore, the findings highlighted that these stressors also led to feelings of “resiliency” and positive racial identity beliefs which were related to overall psychological well-being. This may be unique to African Americans in this developmental period, as young adulthood is a period of increased stress for African Americans that is sometimes associated with negative outcomes (47), especially given that learning how to cope with race-related stress and committing to one’s racial identity may be an important task during this period. As a result, one of the primary tasks of culturally-adapted mHealth interventions for this group should involve highlighting these associations and supporting this group through this developmental period as they grapple with race-related stressors while developing their racial identity.

Moreover, not only did negative race-related experiences increase participants’ connection to their racial group, these positive racial identity beliefs also increased their connection to other African American young adults. These connections allowed for increased perceptions of social support and increased positive feelings about mental health treatments. In line with these qualitative findings, the quantitative results suggested that both private regard (i.e., positive feelings about being Black and positive feelings about other Black people) and nationalist ideology (i.e., emphasizing the uniqueness of the Black experience) beliefs were positively associated with positive attitudes about mental health and mental health treatments. Taken together, culturally-adapted mHealth should also facilitate a “community space” or opportunities for young adults to connect with each other within applications to further facilitate psychological well-being and mental health support.

Additionally, the findings revealed that another key task for culturally-adapted interventions will be to increase awareness and understanding of mental health and mental health treatments. Similar to quantitative studies of young adults (10,48), stigma and lack of knowledge were barriers to mental health treatment, whereas support from friends facilitated mental health treatment seeking. In contrast to these previous studies, family interactions did not facilitate mental health treatment seeking among this sample, and findings highlights intergenerational barriers to seeking mental healthcare and receiving social support from family members. These barriers are likely influenced by cultural factors, given the history of mistrust of healthcare providers (1). As a result, within culturally-adapted mHealth, psychoeducation may help dismantle the negative socialization messages about mental health that young adults receive from others (i.e., hiding mental health symptoms). Additionally, strategies or interventions that increase communication between African American young adults and their family members about mental health and treatments may both help improve perceived social support from family members, as well as begin to decrease stigma and negative beliefs about mental health among older generations.

Culturally-adapted mHealth versus traditional mHealth: features and content

As the survey results suggest, African American young adults strongly supported the appropriateness of mHealth interventions for treating mental health symptoms, and had strong desires for utilizing culturally-adapted mHealth applications. These findings are in line with systematic reviews of mHealth technology (2), and extends previous research in that it is among the first to suggest that African American young adults in particular perceived mHealth applications as a useful vehicle for seeking mental health treatment. More importantly, this further highlights how culturally-adapted mHealth may circumvent the sociocultural barriers to treatment highlighted above by enhancing access to evidence-based monitoring and self-help for mental health (2).

Furthermore, the features highlighted above, as well as the feedback on current mHealth interventions, should guide future culturally-adapted mHealth interventions for this population. This is important given that in light of the feedback on current mHealth options (i.e., desiring more cultural relevancy from evidence-based applications), no current free mHealth applications appear to meet the needs of African American young adults. Despite this, some features currently available within applications are in line with what has been found desirable in other studies of mHealth, such as the ability to use applications daily, the importance of convenience, the ability to view one’s progress and track goals, and having an application that is easy to use (49-51). Similarly, the negative feedback about the tested applications were also in line with findings from other studies that highlight undesirable components of mHealth, such as the negative perception of applications with confusing content or features and a potential lack of motivation to engage continually with mHealth. Similar to previous studies, having applications connected to online social media was also disliked/not desired by participants in this study (49). This study extends this research by highlighting additional, culturally-relevant features (e.g., ability to find African American mental health providers, activities that promote racial identity exploration) and content (e.g., resources related to coping with racial discrimination), that should be in culturally-adapted mHealth. These findings also suggest ways to improve motivation and engagement among African American young adults, such as creating advertisements on social media and featuring positive testimonials or reviews from those who have used the applications.

Whereas other studies of young adults found that aspects such as constant notifications/reminders were undesirable (49), this sample encouraged the use of reminders to encourage continued engagement with applications. Furthermore, previous studies found that young healthy adults believe that mHealth should be for those in severe distress (49). Yet, in our focus groups with exclusively African American young adult participants, they highlighted the importance of utilizing mHealth to maintain psychological well-being in order to prevent severe psychological distress. These positive attitudes towards engaging with mHealth may be reflective of overall positive attitudes towards mental health and the desire for culturally-adapted mHealth given disparities in mental healthcare and a perceived lack of cultural relevancy among traditional mHealth offerings.

The community aspect (i.e., forums and discussion boards) was also another desired feature unique to this sample. In fact, other reviews of mental health mHealth found that users were unwilling to discuss mental health topics in community forums (18), but the studies reviewed did not include majority African American samples. It could be that for African American young adults, given the link between social support, connection with same-race peers, and psychological well-being, having a community aspect should be a key feature of culturally-adapted mHealth (with the caveat of respecting users’ anonymity and confidentiality). One way this may be accomplished is by integrating features common to social media applications into mHealth community forums within future applications. Others have highlighted that one challenge of mHealth is utilizing social media in a way that makes interventions more engaging and provides social support, given the limited desire of having these applications connected to social media (49). The findings from this study highlight that a community page that functions similar to social media (i.e., “Twitter feeds”, ability to post updates and interact with others, etc.) was still desired by participants and may be a way to bridge this gap of using social media functions to facilitate engagement in mHealth while building community among African American young adults.

Finally, traditional mHealth apps have been shown to improve communication between clients and mental health providers (21). Specifically, Reid and colleagues (21) found that mHealth applications help providers better understand clients’ symptoms, aids in tailoring discussions around the clients’ presenting problems during consultations/therapy, and facilitates the development of rapport. Fortunately, the current study findings suggest that African American young adults desire professional assistance and the ability to communicate with mental health providers while using culturally-adapted mHealth applications. As a result, culturally-adapted mHealth utilization may actually help increase utilization of in-person mental health services over time.

Racial discrimination, racial identity, and culturally-adapted mHealth

In addition to the utilization factors that focus groups discussed that might affect engagement with mHealth, it is also important to explore how individual-level sociocultural factors (i.e., previous experiences of racial discrimination and racial identity beliefs) might affect mHealth utilization. For example, existing literature found that more frequent experiences of discrimination led to increased stigma towards mental health treatment, whereas stronger levels of ethnic identity are related to decreased stigma (7). This study is among the first to explore how these factors may be associated specifically with attitudes towards culturally-adapted mHealth technologies. In contrast to previous literature (7), experiences of vicarious ORD were positively associated with a stronger desire for culturally-adapted mHealth interventions for mental health. It could be that since these experiences are frequent for this sample, and since they occur exclusively online, they may influence desires for culturally-relevant resources to cope with these experiences via mHealth technologies. For example, focus groups participants noted how they often received positive messages about mental health and seeking mental healthcare on social media. In light of this, as African American young adults navigate social media, though they are exposed to frequent experiences of vicarious ORD, they may also be receiving culturally relevant, positive messages about mental health and treatments from other African Americans. In turn, these experiences on social media may be influencing a desire for culturally-adapted mHealth, specifically those that address coping with these forms of discrimination.

Next, higher levels of private regard were associated with more positive attitudes towards the appropriateness of utilizing mHealth for mental healthcare. Higher levels of private regard and nationalist ideology were also associated with stronger desires for culturally-adapted mHealth applications for mental health. In regards to private regard and nationalist ideology, it could be that these beliefs may be reflective of having a strong connection to African Americans and the Black experience (27) which may be related to desiring technologies that are created specifically for African Americans, as well as using mHealth to facilitate connections to other African Americans. As highlighted by the focus groups findings, positive feelings about being Black are related to increased feelings of connection to other African Americans and positive experiences related to receiving social support from other African Americans, which may also support why these beliefs are related to mHealth technologies. More specifically, other researchers have highlighted that online spaces may become “safe spaces” to discuss race issues, process race-related stress, and effectively serve as an online “safe haven” (52). As a result, private regard and nationalist ideology beliefs may be related to strong desires for having a “safe space” within culturally-adapted mHealth.

Finally, although participants only had moderate levels of assimilationist ideology (i.e., beliefs related to emphasizing the similarities between African Americans and mainstream culture), higher levels of these beliefs were also related to stronger desires for culturally-adapted mHealth. This association may be related to the qualitative findings that indicate participants’ perceived disadvantage at being able to access mainstream mental health services compared to their White counterparts. For example, participants may desire using culturally-adapted mHealth to begin to feel as equipped to cope with or prevent mental illness as they perceive their White peers are. Additionally, it could also be that African American young adults desire culturally-adapted technologies in an effort to feel less isolated as they navigate and seek to integrate within predominately White spaces.

Limitations and future directions

Though this mixed methods study offered several findings that will positively impact the future development of culturally-adapted mHealth technologies, a few limitations must be noted.

Regarding the qualitative analyses, the primary author coded many of the transcriptions after the team-based approach of creating the initial codebook. As a result, we are unable to provide data related to inter-rater reliability. In terms of the quantitative analyses, given the small sample size, it will be difficult to make any substantial inferences about the correlational associations, but these exploratory findings can begin to establish some preliminary evidence for the associations under investigation. Also, the majority of participants were African American young adult women, so the results presented may be more relevant to culturally-adapted mHealth with this population. Future studies should incorporate more African American young adult men into mHealth research. Additionally, participants completed quantitative measures (including measures of racial discrimination) prior to participating in focus groups. On one hand, this may introduce a priming effect given that in focus groups, participants were asked to recall experiences of racial discrimination and how it affected their mental health. On the other hand, previous research has indicated that a limitation of self-report measures of racial discrimination is that individuals may have difficulty recalling these experiences due to the emotional pain of racism and subsequent denial of these experiences (53). In other words, although the priming effect of completing the quantitative measures first may be a limitation of this study, it is also a possible strength of the study in that it may have also helped participants better recall specific experiences of racism prior to focus groups, which contributed to our findings.

Next, usability and acceptance issues are prevalent issues for mHealth development (23). Participants in the study were college-educated, had access to smartphones and used smartphone applications often, and had positive attitudes towards using mHealth for mental health. As a result, this sample may not accurately represent the beliefs and attitudes of other African American young adults. For instance, on one hand, young adults from rural backgrounds, or who did not attend a four-year college, and/or who use technology less often, may have different attitudes towards seeking mental health treatments and using mHealth for mental healthcare. On the other hand, findings from studies with more heterogenous samples may be similar to this one, as one qualitative study of community-dwelling African American men found that they held positive attitudes towards mental health treatment seeking and did not perceive stigma as a barrier to help seeking (54). In the end, to ensure that culturally-adapted mHealth is designed to be effective for all African American young adults, future studies are needed with more heterogenous populations of African American young adults.

Additionally, as highlighted above, the developmental period of young adulthood may be a unique period where individuals grapple with racial identity development and learning how to cope with race-related stress. Experiences during this period may be directly related to desires of having access to culturally-adapted mHealth technologies to facilitate this developmental process. Going forward, future studies should also explore best practices and desired features or content that are relevant for designing culturally-adapted mHealth for African American children and families, as well as older African American adults. In line with this, many participants in focus groups highlighted how they perceived a need for culturally-adapted mental health resources for high school students (i.e., dealing with bullying, building a self-esteem prior to college, etc.).

Finally, the major strength of this study is that it offers a variety of implications specific to the development of future mHealth technologies, such as specific features, content, preferences desired by African American young adults. Though the study did not intend to offer recommended next steps for the development of a specific intervention, researchers can flexibly integrate the findings from the current study with existing frameworks to facilitate the development of such technologies (55). In fact, others often highlight that the initial phases of mHealth development require focus group research with the intended target (55), so these findings can be utilized by potential developers to facilitate the swift development of culturally-relevant mHealth.

Another strength of this study is that it provides feedback about the acceptability and usability of current mHealth apps for mental health from the perspective Black young adults. This can help researchers and developers tailor future mHealth interventions for this population. Furthermore, it is important to note that of the affordable, available mHealth options that exist, very few, if any, culturally-adapted mHealth interventions currently exist for this group. Even more concerning, research shows that there are very few evidence-based mHealth options for mental health in general. For example, one previous study found that most mHealth applications are not evidence-based, with their development being driven by commercial and economic goals versus scientific contributions (2). In fact, at the time of this study, only one free culturally-adapted mHealth intervention was found available online (i.e., the Safe Place). Though it is culturally relevant, it is unclear if it was developed to incorporate evidence-based interventions or activities. In support of this observation, findings often highlighted how African American young adults desired for the Safe Place to incorporate more evidence-based resources and features that were found in apps such as the Mood Coach and STAIR (which are applications designed from evidence-based resources by the U.S. Department of Veteran Affairs). Taken together, there is an urgent need to not only design culturally-adapted mHealth in line with the recommendations highlighted above, but it is imperative that these technologies include evidence-based interventions in an effort to positively impact current disparities in access to and utilization of effective mental health services among this group.

There is also a dearth of information on the effectiveness of evidence-based mHealth interventions (2,56). Studies also show that the effectiveness of technological interventions can vary depending on the format of the intervention (i.e., videoconferencing has been shown to be more effective as compared to online support groups as a primary intervention) (56). In light of this, once culturally-adapted mHealth interventions are created, researchers should conduct randomized-control trials to evaluate these applications in an effort to improve their effectiveness and begin to disseminate the most effective interventions. It is important to consider that African American young adults in this sample desired for culturally-adapted mHealth to have the option to utilize professional assistance (though they did not perceive this to be a requirement for mHealth to be effective). They also noted that not having this option was a weakness of some current mHealth applications. In light of this limitation, future research should explore if culturally-adapted mHealth as a primary intervention is as effective, or even more effective, as compared to culturally-adapted mHealth that is used in conjunction with a mental health provider (51,57).

Conclusions

In summary, the current study highlights that there is an urgent need for mHealth technology for mental health symptoms for African American young adults. Given the unique sociocultural experiences and barriers that impact mental health and treatment seeking attitudes, it is imperative that these technologies are culturally-adapted so that they meet the unique needs of this population. The findings from the current study can be viewed as recommendations to effectively design future culturally-adapted mHealth interventions, with the hope of reducing current disparities in the mental health outcomes, as well as disparities in the access to and utilization of mental health treatments, among African American young adults.

Acknowledgments

Funding: This study was supported by UNC Department of Psychology, Robert Wood Johnson Foundation Health Policy Research Scholars Program, Institute of African American Research at UNC.

Footnote

Data Sharing Statement: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-19/dss

Peer Review File: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-19/prf

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-19/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by institutional ethics board of the University of North Carolina at Chapel Hill (Ethics Approval No. 18-0915) where the research was conducted and informed consent was taken from all the patients.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. US Department of Health and Human Services. Agency for Healthcare Research and Quality. National Healthcare Disparities Report. Rockville (Maryland): Agency for Healthcare Research and Quality; 2013 Feb (06-0017).
2. Donker T, Petrie K, Proudfoot J, et al. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013;15:e247. [Crossref] [PubMed]
3. Schmitt MT, Branscombe NR, Postmes T, et al. The consequences of perceived discrimination for psychological well-being: a meta-analytic review. Psychol Bull 2014;140:921-48. [Crossref] [PubMed]
4. Brondolo E, Brady Ver Halen N, Pencille M, et al. Coping with racism: a selective review of the literature and a theoretical and methodological critique. J Behav Med 2009;32:64-88. [Crossref] [PubMed]
5. Willis HA, Neblett EW Jr. OC symptoms in African American young adults: The associations between racial discrimination, racial identity, and obsessive-compulsive symptoms. J Obsessive Compuls Relat Disord 2018;19:105-15. [Crossref]
6. Rivas-Drake D, Seaton EK, Markstrom C, et al. Ethnic and racial identity in adolescence: implications for psychosocial, academic, and health outcomes. Child Dev 2014;85:40-57. [Crossref] [PubMed]
7. Cheng HL, Kwan KL, Sevig T. Racial and ethnic minority college students' stigma associated with seeking psychological help: Examining psychocultural correlates. J Couns Psychol 2013;60:98-111. [Crossref] [PubMed]
8. American Psychological Association. Mental health disparities: African Americans [Internet]; 2014. Available online: https://work.cibhs.org/sites/main/files/file-attachments/american_psychiatric_association_mental_health_disparities_by_race.pdf?1597951366
9. Lasser KE, Himmelstein DU, Woolhandler SJ, et al. Do minorities in the United States receive fewer mental health services than whites? Int J Health Serv 2002;32:567-78. [Crossref] [PubMed]
10. Narendorf SC, Munson MR, Ben-David S, et al. Race and gender differences in attitudes toward help seeking among marginalized young adults with mood disorders: A mixed-methods investigation. Psychiatr Rehabil J 2018;41:277-89. [Crossref] [PubMed]
11. DeFreitas SC, Crone T, DeLeon M, et al. Perceived and Personal Mental Health Stigma in Latino and African American College Students. Front Public Health 2018;6:49. [Crossref] [PubMed]
12. Holtz BE, Murray KM, Hershey DD, et al. Developing a Patient-Centered mHealth App: A Tool for Adolescents With Type 1 Diabetes and Their Parents. JMIR Mhealth Uhealth 2017;5:e53. [Crossref] [PubMed]
13. Mulvaney SA, Anders S, Smith AK, et al. A pilot test of a tailored mobile and web-based diabetes messaging system for adolescents. J Telemed Telecare 2012;18:115-8. [Crossref] [PubMed]
14. Jones DJ. Future directions in the design, development, and investigation of technology as a service delivery vehicle. J Clin Child Adolesc Psychol 2014;43:128-42. [Crossref] [PubMed]
15. Kazdin AE, Blase SL. Rebooting Psychotherapy Research and Practice to Reduce the Burden of Mental Illness. Perspect Psychol Sci 2011;6:21-37. [Crossref] [PubMed]
16. Musiat P, Tarrier N. Collateral outcomes in e-mental health: a systematic review of the evidence for added benefits of computerized cognitive behavior therapy interventions for mental health. Psychol Med 2014;44:3137-50. [Crossref] [PubMed]
17. Bardram JE, Frost M, Szántó K, et al. Designing mobile health technology for bipolar disorder: a field trial of the monarca system. In Proceedings of the SIGCHI conference on human factors in computing systems; 2013:2627-36.
18. Nicholas J, Fogarty AS, Boydell K, et al. The Reviews Are in: A Qualitative Content Analysis of Consumer Perspectives on Apps for Bipolar Disorder. J Med Internet Res 2017;19:e105. [Crossref] [PubMed]
19. Lane N, Mohammod M, Lin M, et al. Bewell: A smartphone application to monitor, model and promote wellbeing. In 5th international ICST conference on pervasive computing technologies for healthcare; 2012 Apr 17.
20. Reid SC, Kauer SD, Hearps SJ, et al. A mobile phone application for the assessment and management of youth mental health problems in primary care: a randomised controlled trial. BMC Fam Pract 2011;12:131. [Crossref] [PubMed]
21. Reid SC, Kauer SD, Khor AS, et al. Using a mobile phone application in youth mental health - an evaluation study. Aust Fam Physician 2012;41:711-4. [PubMed]
22. Eisenstadt M, Liverpool S, Infanti E, et al. Mobile Apps That Promote Emotion Regulation, Positive Mental Health, and Well-being in the General Population: Systematic Review and Meta-analysis. JMIR Ment Health 2021;8:e31170. [Crossref] [PubMed]
23. Luxton DD, McCann RA, Bush NE, et al. mHealth for mental health: Integrating smartphone technology in behavioral healthcare. Prof Psychol Res Pr 2011;42:505. [Crossref]
24. Murry VM, Berkel C, Liu N. The Closing Digital Divide: Delivery Modality and Family Attendance in the Pathways for African American Success (PAAS) Program. Prev Sci 2018;19:642-51. [Crossref] [PubMed]
25. Chou T, Asnaani A, Hofmann SG. Perception of racial discrimination and psychopathology across three U.S. ethnic minority groups. Cultur Divers Ethnic Minor Psychol 2012;18:74-81. [Crossref] [PubMed]
26. Keum BT, Miller MJ. Racism in digital era: Development and initial validation of the Perceived Online Racism Scale (PORS v1.0). J Couns Psychol 2017;64:310-24. [Crossref] [PubMed]
27. Sellers RM, Smith MA, Shelton JN, et al. Multidimensional model of racial identity: a reconceptualization of African American racial identity. Pers Soc Psychol Rev 1998;2:18-39. [Crossref] [PubMed]
28. Smith TB, Silva L. Ethnic identity and personal well-being of people of color: a meta-analysis. J Couns Psychol 2011;58:42-60. [Crossref] [PubMed]
29. Smith A. U.S. Smartphone Use in 2015 [Internet]. Pew Research Center. 2015 [cited 2022 Dec 13]. Available online: https://www.pewresearch.org/internet/2015/04/01/us-smartphone-use-in-2015/
30. Zickuhr K, Smith A. Digital differences [Internet]. Pew Research Center. 2020 [cited 2022 Dec 13]. Available online: https://www.pewresearch.org/internet/2012/04/13/digital-differences/
31. Stephan LS, Dytz Almeida E, Guimaraes RB, et al. Processes and Recommendations for Creating mHealth Apps for Low-Income Populations. JMIR Mhealth Uhealth 2017;5:e41. [Crossref] [PubMed]
32. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77-101. [Crossref]
33. Morgan DL. Focus groups. Annu Rev Sociol 1996;22:129-52. [Crossref]
34. Greenbaum TL. Moderating focus groups: A practical guide for group facilitation. Sage Publications; 1999 Nov 18.
35. Harrell SP. The racism and life experience scales. Unpublished manuscript. 1994.
36. Neblett EW Jr, Bernard DL, Banks KH. The moderating roles of gender and socioeconomic status in the association between racial discrimination and psychological adjustment. Cogn Behav Pract 2016;23:385-97. [Crossref]
37. Seaton EK, Neblett EW, Upton RD, et al. The moderating capacity of racial identity between perceived discrimination and psychological well‐being over time among African American youth. Child Dev 2011;82:1850-67. [Crossref] [PubMed]
38. Tynes BM, Umaña-Taylor AJ, Rose CA, et al. Online racial discrimination and the protective function of ethnic identity and self-esteem for African American adolescents. Dev Psychol 2012;48:343-55. [Crossref] [PubMed]
39. Tynes BM, Del Toro J, Lozada FT. An unwelcomed digital visitor in the classroom: The longitudinal impact of online racial discrimination on academic motivation. School Psych Rev 2015;44:407-24. [Crossref]
40. Martin P, Wout D, Nguyen H, et al. Investigating the Psychometric Properties of the Multidimensional Inventory of Black Identity in Two Samples: The Development of the MIBI-S. Unpublished Manuscript. 2008.
41. Neblett EW Jr, Carter SE. The protective role of racial identity and Africentric worldview in the association between racial discrimination and blood pressure. Psychosom Med 2012;74:509-16. [Crossref] [PubMed]
42. Hudson DL, Eaton J, Banks A, et al. “Down in the Sewers”: Perceptions of depression and depression care among African American men. Am J Mens Health 2018;12:126-37. [Crossref] [PubMed]
43. Morgan DL, Krueger RA. The focus group kit Thousand Oaks. London und New Dehli (6 Bände); 1998.
44. Dill LJ, Mahaffey C, Mosley T, et al. “I Want a Second Chance” Experiences of African American Fathers in Reentry. Am J Mens Health 2016;10:459-65. [Crossref] [PubMed]
45. Williams DR, Mohammed SA. Racism and Health II: A Needed Research Agenda for Effective Interventions. Am Behav Sci 2013;57:10. [Crossref] [PubMed]
46. Williams DR, Mohammed SA. Discrimination and racial disparities in health: evidence and needed research. J Behav Med 2009;32:20-47. [Crossref] [PubMed]
47. Hurd NM, Sellers RM, Cogburn CD, et al. Racial identity and depressive symptoms among Black emerging adults: the moderating effects of neighborhood racial composition. Dev Psychol 2013;49:938-50. [Crossref] [PubMed]
48. Gagnon MM, Gelinas BL, Friesen LN. Mental health literacy in emerging adults in a university setting: Distinctions between symptom awareness and appraisal. J Adolesc Res 2017;32:642-64. [Crossref]
49. Dennison L, Morrison L, Conway G, et al. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. J Med Internet Res 2013;15:e86. [Crossref] [PubMed]
50. Harrison V, Proudfoot J, Wee PP, et al. Mobile mental health: review of the emerging field and proof of concept study. J Ment Health 2011;20:509-24. [Crossref] [PubMed]
51. Chan SR, Torous J, Hinton L, et al. Mobile Tele-Mental Health: Increasing Applications and a Move to Hybrid Models of Care. Healthcare (Basel) 2014;2:220-33. [Crossref] [PubMed]
52. Tynes BM, Garcia EL, Giang MT, et al. The racial landscape of social networking sites: Forging identity, community, and civic engagement. ISJLP 2011;7:71.
53. Lewis TT, Cogburn CD, Williams DR. Self-reported experiences of discrimination and health: scientific advances, ongoing controversies, and emerging issues. Annu Rev Clin Psychol 2015;11:407-40. [Crossref] [PubMed]
54. Ward EC, Besson DD. African American men’s beliefs about mental illness, perceptions of stigma, and help-seeking barriers. Couns Psychol 2013;41:359-91. [Crossref]
55. Matthews M, Doherty G, Coyle D, et al. Designing mobile applications to support mental health interventions. Handbook of research on user interface design and evaluation for mobile technology. IGI Global; 2008:635-56.
56. Mohr DC, Burns MN, Schueller SM, et al. Behavioral intervention technologies: evidence review and recommendations for future research in mental health. Gen Hosp Psychiatry 2013;35:332-8. [Crossref] [PubMed]
57. Ben-Zeev D, Brian RM, Aschbrenner KA, et al. Video-based mobile health interventions for people with schizophrenia: Bringing the "pocket therapist" to life. Psychiatr Rehabil J 2018;41:39-45. [Crossref] [PubMed]