Mixed-methods study of development and design needs for CommitFit, an adolescent mHealth App

Introduction

Background

Adolescent obesity continues to be a significant public health concern in the United States with 22.2% of children ages 12 to 19 years have obesity in 2020, as categorized by a body mass index (BMI) greater than the 95th percentile (1) increased risk of developing obesity-related diseases such as heart disease, type 2 diabetes mellitus, hypertension, and cancer (2). Adolescents are described as those who are 12 to 19 years of age.

Mobile health applications (mHealth apps) have the potential to influence health behaviors. mHealth apps are defined as an application, hosted through mobile technologies, that monitor and support the health of users. Yet, effective mHealth applications created for adolescents are limited. As a result, we created the CommitFit mHealth app in collaboration with adolescent users. CommitFit is an mHealth app, in development, that focuses on lifestyle and health behavior modifications in adolescents. While the CommitFit mHealth app may improve weight, overweight and obese adolescents are not the only group that could benefit from the improved health behaviors.

Rationale and knowledge gap

mHealth apps have shown potential to improve health behaviors and improve short-term BMI in adolescents (3). However, most current mHealth apps lack supportive scientific evidence (4,5) and were not developed with the guidance of health care experts (4,5). Current literature has revealed gaps in adolescent specific mHealth app development. mHealth apps, particularly those in the commercialized space, lack relevant user involvement in the development process, are scaled down from adult-targeted mHealth apps, and lacks sufficient guidance on the preferred design elements and user requirements of adolescents (6). Without adolescent involvement in the design process, mHealth apps cannot truly capture what motivates adolescents to continue using these apps, as shown by the lack of regular use (6). Additionally, most mHealth apps are limited in scope and only address a specific to one health goal (i.e., physical exercise). The development of a consolidated adolescent-targeted mHealth app (involving multiple health goals within one app) has the potential to improve a wider array of health outcomes (7,8). Thus, the CommitFit mHealth app was developed using a bottom-up approach with early adolescent feedback during several stages of development and health expert guidance.

Adolescents perceive mHealth app interfaces designed for adults to be complex and visually dull, which discourages adolescent usage (9). Adolescents have unique mental and physical health needs and are at increased risk of eating disorders and social stigma (9). Thus, there is a substantial need to be mindful of potential harms associated with developing a mHealth app that could be used as a weight management (10) and intervention tool for adolescents. As a result, emphasis on positive health behaviors and lifestyle changes can be advantageous to mitigate the risk of harm.

There are many mHealth apps developed for adult populations, however, there are few available commercialized children and adolescents-targeted mHealth apps that address the distinctive needs of this population during app development. A systematic review observing app development found that only 22.5% of studies involved the targeted users before developing (9). The lack of impactful user feedback is problematic because there are only a few apps, despite the multitude apps available, that engage adolescents sufficiently enough for them to use them long enough to develop meaningful health behavior changes. Only 21% of adolescents report downloading an mHealth app and of these, only 8% report using it regularly (6). Often, these adolescent-targeted mHealth apps are adapted from adult applications (9). Unfortunately, the popularity of adult-targeted mHealth apps is not always translated to adolescent populations. Adolescents are unique and often neglected as consumers of mHealth apps. The adolescent population are consumers who prefer technology developed for them, as opposed to having an mHealth app that was scaled-down from an adult-targeted application. As a result, mHealth apps are most impactful when developed using a bottom-up approach (11) and direct user feedback (12).

One concept that adolescents have identified as important to engage their use of mHealth apps is gamification. Gamification is defined as a set of techniques that engage users and protect against loss of interest in mHealth apps (13,14), such as rewarding users with points or badges. Gamification is a popular motivational technique that encourages users to continue to use an mHealth app through the implementation of competition, such as a point system, avatar, leaderboard, etc. (14). Despite literature on the use of gamification by adults, current literature fails to deeply explore the nuances of gamification techniques that are impactful for and preferred by adolescent users.

Objectives

This study has one objective and three aims. The objective of this study is to describe the co-development process working with adolescent users. The aims are as follows: (I) to explore the visual design and functional requirements when developing the CommitFit mHealth app, (II) to uncover the gamification techniques that incentivize adolescents to set and achieve healthy lifestyle goals, and (III) to identify adolescent expectations when using the CommitFit mHealth application. We utilized an iterative user-centered design and development framework focused on using a bottom-up approach, as opposed to scaling down an adult-targeted mHealth application. We present this article in accordance with the COREQ reporting checklist (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-35/rc).

Methods

We used a mixed method approach to describe and better understand the design and user needs and expectations of adolescent mHealth users. Our user-centered and human-system design process employed adolescent interviews and surveys to best develop the app based on user feedback. User-centered design is an iterative process which includes the users in each stage of the design process (15); human-system design approach is based on the analysis, design, and evaluation of a system from a user’s perspective. The integration of meaningful user contributions enhances the app by ensuring an intuitive user interface. Employing iterative user-centered design when developing mHealth apps increases the likelihood that an app will be engaging, effective, and usable for the user (16,17).

The CommitFit mHealth app

CommitFit is an mHealth app, in the development phase, that is designed to empower adolescent users to set and achieve healthy lifestyle goals. To co-develop CommitFit, users provided feedback on the app in a stepwise fashion and allowed us to iterate the design multiple times. Design elements and functions were continually improved based on feedback from users, and updated prototypes were presented to subsequent study subjects. The initial wireframe prototype was developed and guided by health technology and medical experts (18-22). All preceding prototypes of the CommitFit mHealth app were influenced by adolescent feedback through a previous qualitative study involving focus groups of adolescents and caregivers, leading to evolutions of the visual interface.

The basic function of the app is to allow adolescent users to select a maximum of two of five health behavior goals including: (I) increase physical activity, (II) increase the intake of fruits and vegetables, (III) decrease the consumption of sugar-sweetened beverages, (IV) increase water consumption, and (V) improve overnight sleep. These goals were selected based on clinical experience working with adolescents with obesity (18-20,23,24) (ASB, RK, AT) and with prior research on child obesity. After goal selection, users choose a daily goal based on their current health behavior, for example, a user who currently consumes no fruits/vegetables may set a goal of 2 servings per day. Users will use the “logging goals” functionality of the app to monitor their daily behaviors and be prompted to log their goal daily with a customizable reminder. To motivate app users, they will be able to earn points to encourage progress towards their goal or to their recommended level (gamification).

Paper wireframes were adapted into static prototypes and assets using Adobe Photoshop. The alpha build of the CommitFit mHealth app was developed using XCode, an integrated development environment created by Apple. This initial build was developed for iOS compatible devices, specifically for iPhones. The live alpha build was presented to the last three interviewees using TestFlight. TestFlight by Apple is an application that hosts unreleased mobile iOS apps in the alpha- and beta-testing phase; this allows developers to correct bugs and glitches, as reported in feedback from testers.

The first alpha build describes the pre-release first iteration of an application, which is typically dedicated to the testing process to determine usability and design (25). Typically, the alpha version is not considered “feature complete” and lacks fully implemented features and functions (26). Testing of alpha builds may reveal major deficiencies in the system and helps to evaluate actual application flow. Unfortunately, too often, face-to-face communication during the alpha stage is neglected or varies in several studies resulting in applications that do not meet user’s needs (27); real-time feedback from our interviews provided valuable information and insight to make improvements to usability.

Adolescent semi-structured interviews and task analysis

Adolescents were recruited to perform task analysis and semi-structured interviews to iterate the CommitFit app static prototype, until it was honed to the best iteration of the app to be programmed. Interviews were used to explore adolescent preferences with visual app design and functionality. Interviews were conducted online for one-hour using Zoom and transcribed verbatim from the recorded audio. Interviews were hosted by researchers (ASB, PG, KTB) with experience conducting adolescent interviews, and at least one observer (LF and/or RJK).

Recruitment occurred in three stages. The first stage identified eligible candidates using the electronic health record (EHR), from a single-family medicine and pediatric clinic in the MU Health Care system where one of the researchers (ASB) works as a physician, who met the inclusion criteria: age 13 to 15 years old to speak English fluently, read at the 6th grade level or higher, and be proficient with smartphone app use to participate. Candidates were excluded if they had a pre-existing severe mental health diagnosis (other than mild or controlled anxiety and/or depression), intellectual disabilities, or eating disorders. Adolescents with a normal BMI were included in the study, as the CommitFit mHealth app aims to improve the user’s lifestyle and health behaviors. The second stage of recruitment occurred from provider referrals of identified populations. Stratified sampling was utilized with a goal of obtaining at least 10% (1) of participants of African American race, 10% from a low-income family (Medicaid recipient), and 10% from rural community, and with an even distribution of gender. Adolescents who participated in previous developmental focus groups were excluded from the recruitment process for adolescent interviews to avoid priming bias. Adolescents were enrolled in the study on a first come-first serve basis until ten adolescents were enrolled. The Institutional Review Board of the University of Missouri Health Science approved the study plan (No. 2054598). The parent or legal guardian of the adolescent was contacted via email or telephone for recruitment. Informed consent was obtained from all adolescents and their parents or legal guardians prior to partaking in the study. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

All participants received compensation in the form of a $50 e-gift card of their choice to three popular vendors, for participating. The adolescents were recruited from varied demographic backgrounds including race, household income (Medicaid insurance), gender, and rurality. We intentionally sought to recruit a diverse sample of adolescents.

We presented the static prototype, using Microsoft PowerPoint and TestFlight, of the CommitFit mHealth app to the interviewees to receive specific feedback about the app. This feedback allowed us to develop the best prototype iteration before programming the alpha build of the CommitFit app. Feedback from the interviews was transcribed verbatim from the Zoom recording. The transcripts were inductively coded by two researchers (KTB, PG) using the Dedoose software. Interviewees were generally asked open-ended questions to allow for organic conversation, as shown in Table 1. Interviewees were asked close-ended questions, such as, “do you like the general design of the app?”

For the task analysis, a moving prototype that simulated the flow of the CommitFit mHealth app was presented via Microsoft PowerPoint for a structured and controlled task analysis. The first seven interviews were conducted with static slides the researcher advanced acting as the “computer” at the user’s instruction. The last three were presented with the first interactive alpha build via TestFlight and directed to vocalize their movements and opinions of CommitFit aloud. This process revealed issues with consistency and expectations of the apps flow within pre-existing conventions.

System usability scale and questionnaire

In succession to the adolescent interviews, we utilized a modified System Usability Scale (SUS) (28) to assess the functionality of features, the ease of use, and the behavioral intent of users. To quantify usability, we utilized the SUS scale of one to five with one being strongly agree, very satisfied, or very likely; and with five being strongly disagree, very dissatisfied, or very unlikely. This usability scale was adapted from the SUS (29,30), a well-established and proven method for measuring usability of systems. Additional questions were added to the questionnaire to scope the success of CommitFit specific features. The SUS has been described as a “quick and dirty” method to assess the usability of an industrial system evaluation and has been an industry-standard since its conception in 1996. Despite being used in several previous adolescent-targeted studies (31-35), including those producing mHealth apps, the use of a SUS has not been widely validated in adolescent's populations. The general use of the SUS, particularly in adult populations, is found to repeatedly have a Cronbach’s alpha greater than 0.80 (36). Additional questions were added to the questionnaire to evaluate the success of CommitFit specific features.

The first part of our survey consisted of a modified SUS (a Likert scale). The SUS works by asking participants five pairs of competing questions: one question will be positively framed; the next question will be negatively framed (such as question 1 vs. question 2 in Table 2). This allows researchers to capture an accurate result of user options on visual user interface design.

To calculate the score, all odd numbered questions (negatively framed) are summed into X and all even numbered questions (positive framed) are summed into Y. We then applied the following equations, respectively: X₀ = X − 5 and Y₀ = 25 − Y. Next, values are applied to the last equation: SUS score = (X₀ + Y₀) × 2.5.

The SUS is scored by converting each question into a point value (I.e., strongly disagree is equal to 1 point). Once each question is assigned a value, they are added together and then multiplied by 2.5 to convert the original scores of 0–40 to 0–100. While scores range from 0–100, these are not percentages and should be considered only in terms of their percentile ranking. In general, a SUS score above 68 would be considered above average (28-30).

The SUS and questionnaire were combined into one main survey and was electronically distributed to participants via Qualtrics^©, immediately following the adolescent’s interview/task analysis. The SUS survey consisted of ten items, while the rest of the questionnaire consisted of 19 additional items. The additional items surveyed were the features and functionalities (such as the point system, avatars, etc.) related to the CommitFit app. The questions in the additional questionnaire are presented in Table 3.

Results

Participants

A total of ten adolescent users participated in both the interview and surveys; all adolescent participants completed both the interview and accompanying survey. These participants ranged in demographics. Improved lifestyle and health behavior modifications are a key element of CommitFit. While CommitFit may improve weight, overweight and obese adolescents are not the only group that could benefit. Thus, the weight classification of adolescents was not an exclusion criterion, except those who were severely underweight to avoid potential eating disorder exacerbation. Participants ranged from the ages of 13 to 15 years of age, with an average age of 13.6 years old. Five participants identified as male, three as female, and two preferred not to answer. One participant was from a rural area. Two participants had obesity (BMI >95th%) and one participant had overweight (BMI 85–94.9th%), as identified by data in their EHR. One participant was low-income, as determined by Medicaid enrollment. The participants’ self-reported demographic data can be observed in Table 4.

Qualitative data

During adolescent interviews and task-analysis several inductive themes were uncovered, including visual elements (color usage, logos), navigation, and user-interface preferences. Functionality and features were also a large focal point and concern of adolescents, as they cited the following themes gamification, additional features, tutorial, home screen accessibility, reminders and notification, and recommendations.

Color

Based on interview feedback, the use of color was a powerful tool to keep adolescents interested and engaged in content.

ID 10: “Making the images more vibrant or something might help a little bit, it just kind of looks bland when I open into it. It doesn’t make me, like, want to do those achievements.”

ID 9: “I like the fact that it’s colorful. I feel like that's a good part about it.”

Navigation and icons

We also found that adolescents preferred icons within the app that fit pre-existing convention. For instance, the use of a home icon must lead to a home or dashboard page and auto progression after clicking a button. Changes from these conventions often confuse users, including adolescent users in our interviews. Accessory icons to enhance the design of the CommitFit mHealth app were developed from general interview feedback regarding the preference for mature and colorful designs, as presented in Figure 1.

Figure 1 Design board featuring icons and logos.

ID 7: “[Home icon leads] back to the dashboard... with like a set goal, leaderboard, and all that stuff.”

ID 7: “So, in my mind, I imagined that under, if you clicked one, it would bring you to a different page [automatic button progression].”

ID 10: “I think that normally you would get used to just clicking it [button] and it would automatically show up... The two arrows at the bottom and the home thing, and those in every screen because they do different things; however very similar, but they’re in different places and they look very different... Just making it as consistent as possible helps people understand more intuitively understand the app.”

Interviewees especially commented on the importance of ease of use when navigating.

ID 1: “I think having the home page is good so I can—it’s like a starting point—so, I can go from there. I think it’s great”

ID 2: “There’s like the main menu part [Dashboard]... That’s a good layout, ...so it’s clear like to where to go if you’re new to the app or don’t know anything about it.... It was pretty easy to know where stuff would be...It was really easy to navigate and, like, know where everything was and, like, what they’re, like, what to press to get to different places, like the arrows in the home button.”

ID 3: “I had never used it, I had never seen [the CommitFit app] before, it was pretty easy to know where stuff would be.”

ID 4: “It was really easy to navigate and, like, know where everything was and, like, what to press to get to different places—like the arrows in the home button and the ‘x’s and stuff.”

Results from later interviews, compared to earlier ones, supported the design changes made during CommitFit’s iterative process. Adolescents were able to easily navigate the interface.

ID 10: “I think it’s straightforward, it’s good.”

ID 4: “I think it looks good. It was easy to understand.”

Sophisticated interface

The first static prototype of the CommitFit mHealth app was a very simplified interface. However, the adolescent users identified that the interface and its depictions were too juvenile and oversimplified during previous focus group discussions and interviews. Adolescent users expressed a desire for a more sophisticated interface, specifically citing graphic stylization.

ID 10: “It’s kind of more like childish, maybe... like the Avatar and just the pictures [are childish]… I like the person [avatar icon].”

ID 5: “Almost too childish, a little bit... comparing it to other apps like the Fitbit apps, how those are a little more, in their design are a little—I don’t really know how to describe it, because, like, the shapes of the boxes and the colors, and I like the colors, but it makes it seem a little more like those very ‘for young kids’ apps, like elementary school.”

Therefore, we consolidated the CommitFit app to a more mature and consistent interface, as depicted in Figure 2 of the design evolution.

Figure 2 Overview of the iterative design process as user feedback was incorporated (A) Wireframes developed by health experts, (B) overview of first digital design iteration (presented during focus groups), (C) overview of intermediate digital design iteration after four interviews/task analysis (presented to interviewees), and (D) overview of final design iteration post-interviews and surveys.

Gamification

Interviewees recommended that to ensure a wide range of engagement, CommitFit should be designed to feature several diverse types of gamification including points, badges, customizable avatar, and leaderboard. Participants were explicitly asked what types of gamifications they would like to see or prefer. However, some participants organically discussed what gamification they liked within the CommitFit app, while also making some suggestions for gamification.

ID 6: “Maybe like a streak system. So, if you log on like multiple days in a row then it, like, first day you get one point, and second day you get two points.”

ID 10: “Honestly, the challenges; it’s good idea. Streaks would like keep you motivated to like, keep it. It’s kind of like Snapchat, you know you don't want to lose your streak type thing.”

ID 9: “The customizable avatar was like my favorite... I like to avatar thing and so I feel like that would be something that would make me interested in it, ‘cause usually it’s just like tracking stuff, just not really like any rewards, I guess...I think stuff like that [points], that’s probably what would make me use it more.”

ID 2: “Well, definitely, like enticing like rewards for sure would definitely be [encouraging use]. 'Cause, like, aside from personal drive to get to get it, you know like want to do. But like, definitely the point system is good. So, I guess just like whatever the points can do, like the making sure like the points can like get stuff that is like worth trying to get the points for 'cause then you know you wanna like come back and get it.”

Interviewees also suggested that the leaderboard be limited to goal types or a certain length of time (i.e., a week) to protect users from discouragement.

ID 7: “I’ve played games where the people at the top you can never catch up to them and it’s just not very fun... So, possibly if leaderboards for different time frames, maybe this week or this month, or like just global or overall. Well, it would be nice just so like, “Uh, because if you just have it only for overall people, just lose interest ‘cause they’re like, oh, I can never catch up now just so having it reset or having a new thing every week allows people to say, “oh, I didn’t do so well this week. I can try again next week.”

Additional features

Adolescents also indicated that they preferred the CommitFit mHealth app to have the ability to be personalized, customizable, and provide an educational tool for health resources.

ID 7: “Just being able to change different aspects is something that because people situations are extremely diverse, so being able to like tweak little things for each person would help so much. ‘Cause obviously no one is the same, so this blanket thing wouldn’t necessarily help everyone, so being able again that would be quite a bit more work, but something that I think would be beneficial to some people where being able to.”

ID 9: “I think it’s [avatar] pretty cool. Usually, you don’t really like have customizable stuff... I like stuff like that. I like customizing stuff.”

ID 5: “I really like the resources section with the healthy food ideas. And, also, like how there’s like exercise, ideas and stuff like that. That is something that really stands out to me.”

Tutorial

Additionally, users expect either a welcome page or tutorial when first using the CommitFit mHealth app. This is also due to pre-existing design conventions.

ID 7: “Probably welcoming, so I know for instance, in Google products they have when you first open. The screen there like, ‘this is how this works, and then you can login at the bottom, or something like that or possibly like a screen that shows just the basic interface and then like little possible popups that explain different elements of the UI [user interface]… At the beginning, when you’re first using the app, tell the user what the points are specifically used for ‘cause at the beginning the points were mentioned and they were shown, but I didn't know what that meant.”

Reminders and notification

Adolescent interviewees stated that notifications and reminders were important to encourage use. Preferences for morning versus evening notifications were varied; however, adolescents agreed that users should have control over their notifications, such as being able to disable or set time preferences for notifications.

ID 2: “If it reminded you maybe, like, two or three times throughout the day, just to like, make sure you’re doing your goal and stuff. So, like whenever it reminds you the little thing pops up.”

ID 3: “I think more things that would make it more interactive [to improve the CommitFit app]. Maybe it’ll give you like a reminder, like, you haven’t gotten your points in or something for today.”

Goal setting recommendations

To help protect adolescents from setting health behavior goals that are unsafe or unattainable, the CommitFit mHealth app uses app-generated recommendations. When selecting a goal, users are limited in the amount they can enter and the allotted points awarded to users are fixed; for instance, a user that sets and eats five servings earns the same points as someone who sets and eats one serving. This allows users at multiple levels to compete and to personalize their goals. CommitFit-made goal recommendations were positively received by interviewees.

ID 7: “Yeah, ‘cause a lot of people don’t necessarily know what they like. They have an intent of, like, ‘oh, I need to eat more healthy food’, but they don’t really know what that means. So, having that general baseline of oh so this is how much I should probably be doing and then being able to obviously improve up to that level is helpful.”

Quantitative results

The highest scores were reported in “I thought the CommitFit app was easy to use” (Item 3) and “I would imagine that most people would learn to use this CommitFit app very quickly” (Item 7) with mean scores of 4.4 (strongly agree) and 4.6 (strongly agree), respectively. The lowest scores were reported for positively framed questions in “I found the various functions in this CommitFit app were well integrated” (Item 5) mean scores of 3.2 (somewhat agree). 90% of surveyed adolescents found the app to be easy to use (either strongly agree or somewhat agree). 100% of adolescents surveyed found the app (electronic prototype or A1) to be easy to learn to use. Additionally, most adolescents (≥70%) were satisfied with the features. The results of the modified SUS can be observed below in Figure 3.

Figure 3 Adapted SUS for surveying the usability of the CommitFit app in the adolescent population. (A) Positively framed questions, (B) negatively framed questions with mean score and standard deviations. SUS, System Usability Scale.

The SUS uses a scaled system, in which we use a scale from 1 (strongly disagree) to 5 (strongly agree) (29,30). The results from the modified SUS suggest that adolescents found the application easy to use with an average score of 78.5 (a score above 68 indicates good usability). The SUS is graded from A to F; six participants scored CommitFit as A (>80.3), two participants scored it as B (>68–80.3), one participant scored it as D (51–68), and one participant scored it as F (<51).

To supplement the SUS results, we also surveyed the satisfaction of features that are specific to the CommitFit app (i.e., logging goals, gamification, etc.). This allowed us to determine the most effective gamification techniques and features, as valued by our participants. The questionnaire included 19 items on a Likert scale of 1–5 (1 was either poor, very unlikely, or very dissatisfied; 5 was either excellent, very likely, or very satisfied). The average mean scores for the three categories (Look and Feel, Intent of Use, and Features and Functionalities) are displayed in Figure 4.

Figure 4 Average mean scores of the additional items within the additional questionnaire, presented in three categories: features and functionalities, intent to use components, and the general look and feel.

Mean scores of the CommitFit Features and Functionality questionnaire reveals that of the included features adolescents strongly prefer social competition with friends (item 24; mean of 4.50) and family (item 25; mean of 4.60), app made recommendations (item 13; mean of 4.40), and gamification techniques (item 26; mean of 4.50). Additionally, adolescents agree that CommitFit protects users from self-esteem harm (item 17; mean of 4.30). Adolescents place the least value on the earning points, in general (item 16 mean score of 3.90). Interestingly, adolescents responded positively to the prospect of discussing their health behaviors with their doctors (item 27; mean of 3.80) and/or allowing their doctors to access this information via a health behavior record (item 28; mean of 3.80). All items, responses, mean, and standard deviations are listed in the Table S1.

Discussion

Key findings

The goal of this study is to describe the co-development process working with adolescent users. The three aims of this study include: (I) to explore the visual design and functional requirements when developing the CommitFit mHealth app, (II) to uncover the gamification techniques that incentivize adolescents to set and achieve healthy lifestyle goals, and (III) to identify adolescent expectations when using the CommitFit mHealth application. In doing so, the design and functional user requirements for mHealth app are described (9,13,37) and function (algorithms, expert generated content) preferences with mHealth apps align and expand with current literature on this topic (9,13,37). These findings can inform future researchers and app developers about adolescent needs and preferences, as identified by adolescent users.

Limitations

There were several limitations to our study including its small sample size (n=10). The post-interviews survey was completed by the same ten interviewees. Small sample groups can lead to erroneous generalizations. Despite our small sample size, data saturation was reached in interviews. Data saturation was reached when no additional novel feedback was collected from interviews, which occurred after ten participants.

Additionally, six participants scored CommitFit as A, two participants scored it as B, one participant scored it as D, and one participant scored it as F. We identified that the F scored SUS could have been a result of user error. The SUS presents items in an alternating fashion, positive framed questions are presented in odd numbered items while negatively framed questions are presented in even numbered items. The theme of questions is paired with both a positive and negative framed question; for example, items 7 and 8 address a similar topic, like the use of the app, but item 7 is positively framed, and item 8 is negatively framed. Therefore, answers for positive framed items should be, to some degree, inverted from the answers for negative framed items, serving as an internal check for consistency. This was not the case for the participant who scored the app as F. These results not only highlight the need for a larger population size to determine if this is a true outlier or an erroneous entry, but also for the need to ensure that questions are at the level of literacy appropriate for adolescent populations. Although the SUS has only been validated in adults, not adolescent populations, it has been used in multiple adolescent studies.

Our interviews and task analyses were limited by the developmental (static and then alpha prototype) stages of our app. While this was important for the CommitFit mHealth app development process, additional design and function preferences could be better explored with a fully developed app from the beginning. However, certain elements crucial to CommitFit, such as gamification and goal setting, may not be translated to other mHealth apps.

Comparison with similar research

We found that adolescents prefer vibrant colors, bold sophisticated graphics, and modern interfaces that are easy to navigate. A study performed by Chan, Kow, and Cheng in 2017 found that the incentives of app usage are the following: interface design and multimedia content, customization, and rewards (9). They found that adolescents preferred to use apps that are visually appealing with vibrant and/or bold colors, easy to use, rewards (prizes or points), customized, and allowed for competition between friends. These findings echo our similar findings that adolescents prefer a bright, colorful, and simple app, as well as the components of gamification (i.e., competition, points, etc.) and customization. This implies that usage of these design elements is universally desired among adolescents and not specific to CommitFit. Results from the SUS and additional questions support that the design elements of CommitFit improved its appeal and usability.

During adolescent interviews and task analysis, several themes were uncovered, including visual elements (color usage, logos), navigation, and user interface. It is crucial that the color is impactful, yet not overwhelming to avoid distraction (38). Since our adolescent users desired a colorful design (9), complementary colors were used to provide a colorful contrast and a visually interesting component. The use of complementary colors assists in the accessibility of colorblind users by providing a sharp contrast. Additionally, it has been shown that apps with icons that utilize a higher degree of colorfulness, proper complexity, and slight asymmetry lead to a greater number of downloads (39).

Explanations of findings

The core of the human-system design approach is based on the analysis, design, and evaluation of a system from a user’s perspective. These systems, including the development of the CommitFit mHealth app, are specifically tailored to human interactions and interventions. To provide the user with the most ergonomic design, it’s important to acknowledge the wants, needs, and interactions for which the target audience desires.

Functionalities and features were also a large focal point for adolescents, as they preferred the following themes: gamification, additional features, tutorial, home screen accessibility, reminders and notification, and recommendations. We found that adolescents not only preferred bright and bold color schemes but desired a sophisticated and easy-to-navigate interface. As shown in the additional questionnaire, adolescents were satisfied with the navigation of the CommitFit mHealth app (item 15; mean of 4.00) and the general look and feel (item 29; mean of 3.50).

Additionally, adolescents preferred robust gamification techniques; therefore, every user could find an external motivator to accomplish health goals and to incentive app usage. Gamification can be a crucial element for user engagement, which can lead to improved efficiency of outcomes (40) from using the CommitFit mHealth app. These findings and identified themes are supported by the results of the surveys and the variation of scores involving specific gamification techniques. Adolescents score all gamification techniques positively (item 26; mean of 4.50): the most well-received technique was competition among family (mean of 4.60) and friends (mean of 4.50), followed by points with a mean score of 3.90. Based on feedback from participants, we designed CommitFit to have points that can be exchanged for app-based incentives such as gear or clothing for the user’s customizable avatar, improving user-engagement.

Based on results from interviews and surveys, the functional requirements that users most care about are system usability and the ability to customize the app (i.e., avatar customization, controlling notifications, and other settings). This conclusion is supported by the encouraging positive mean graded scores of the SUS results, validating the general visual design interface. The additional questionnaire targeted specific questions regarding features and functionalities, the intent of use, and look and feel. Adolescents especially ranked the Features and Functionalities category of the additional questionnaire of the CommitFit mHealth app positively with a mean score of 4.11.

Implications and actions needed

Developing mHealth apps for adolescents requires nuance. While we focused on improving health outcomes via attainable and realistic goals that form changes in habit, these goals must be developed with care. As voiced by adolescents, weight loss goals and recommendations can be harmful. As a result, weight loss is not the explicit feature of the CommitFit mHealth app, nor is the app weight loss centric (i.e., it does not require users to perform weigh-ins or to set weight goals). The CommitFit mHealth app aims to empower youth to control their health outcomes by setting and tracking health-related goals. Likewise, the app was designed to be used by adolescents with healthy weights to promote positive health behavior, goal setting and obesity prevention, as well as adolescents with increased weight. These goals can lead to weight loss, through improved health habits, but that is not an explicit goal presented to adolescents. When developing the CommitFit mHealth app for adolescents, developers had to consider the potential harm the app can cause. The success of this harm prevention is reflected in the positive score of items 17 in the additional survey, “How satisfied are you with CommitFit not harming the user’s self-esteem?” with a mean score of 4.30. However, it is important to monitor the use and outcomes of mHealth apps like CommitFit to prevent unintended consequences for this vulnerable population.

In the future, we hope to link the data collected from the CommitFit mHealth app to a provider-facing, ambulatory, EHR tool. This will allow for effective, safe, and monitored weight management and lifestyle interventions to occur to help address child obesity. Future iterations of the CommitFit app may additionally include functions to synchronize data from wearable devices such as Garmin^© or Fitbit^©. Additional research is now needed to explore the clinical effectiveness of the CommitFit mHealth app, as a health and lifestyle intervention.

Conclusions

In conclusion, based on the interview and survey results of this mixed-methods study, adolescents prefer vibrant colors, modern, easy-to-use interface, gamification and rewards, customizable and personalized features, and simple but mature graphics. Complying with adolescent user feedback on design and functional elements optimizes the usability and desirability of an mHealth app. Adolescents were especially motivated by gamification techniques in maintaining their interest in the CommitFit mHealth app and their health behavior goals. By utilizing adolescent-approved designs and features, user engagement can be enhanced. As a result, improved user engagement can lead to increased compliance to health goals set in the CommitFit mHealth app; thus, leading to positive health changes, such as weight loss or management.

Acknowledgments

Funding: The research reported in this publication was supported by the University of Missouri-Columbia, TRIUMPH (Translational Research Informing Useful and Meaningful Precision Health) grant. The content is solely the responsibility of the authors and does not necessarily represent the official views of TRIUMPH or the University of Missouri-Columbia.

Footnote

Reporting Checklist: The authors have completed the COREQ reporting checklist. Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-35/rc

Data Sharing Statement: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-35/dss

Peer Review File: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-35/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-35/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The Institutional Review Board of the University of Missouri Health Science approved the study plan (No. 2054598). Informed consent was obtained from all adolescents and their parents or legal guardians prior to partaking in the study. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Stierman B, Afful J, Carroll MD, et al. National health and nutrition examination survey 2017–March 2020 prepandemic data files development of files and prevalence estimates for selected health outcomes. National Health Statistics Reports 2021; NHSR No. 158. Available online: 10.15620/cdc:10627310.15620/cdc:106273
2. Jensen MD, Ryan DH, Apovian CM, et al. 2013 AHA/ACC/TOS guideline for the management of overweight and obesity in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and The Obesity Society. J Am Coll Cardiol 2014;63:2985-3023. [Crossref] [PubMed]
3. Chen JL, Wilkosz ME. Efficacy of technology-based interventions for obesity prevention in adolescents: a systematic review. Adolesc Health Med Ther 2014;5:159-70. [Crossref] [PubMed]
4. Rivera J, McPherson A, Hamilton J, et al. Mobile Apps for Weight Management: A Scoping Review. JMIR Mhealth Uhealth 2016;4:e87. [Crossref] [PubMed]
5. Wearing JR, Nollen N, Befort C, et al. iPhone app adherence to expert-recommended guidelines for pediatric obesity prevention. Child Obes 2014;10:132-44. [Crossref] [PubMed]
6. Chen E, Moracco KE, Kainz K, et al. Developing and validating a new scale to measure the acceptability of health apps among adolescents. Digit Health 2022;8:20552076211067660. [Crossref] [PubMed]
7. Anderson M, Jiang J. Teens' social media habilts and experiences. Available online: https://www.pewresearch.org/internet/2018/11/28/teens-social-media-habits-and-experiences/, 2018 [Accessed 20 June 2022].
8. Vaterlaus JM, Aylward A, Tarabochia D, Martin JD. “A smartphone made my life easier”: An exploratory study on age of adolescent smartphone acquisition and well-being. Computers in Human Behavior 2021;114:106563. [Crossref]
9. Chan A, Kow R, Cheng JK. Adolescents' perceptions on smartphone applications (apps) for health management. Journal of Mobile Technology in Medicine 2017;6:47-55. [Crossref]
10. Hermawati S, Lawson G. Managing obesity through mobile phone applications: a state-of-the-art review from a user-centred design perspective. Personal and Ubiquitous Computing 2014;18:2003-2023. [Crossref]
11. Psihogios AM, Stiles-Shields C, Neary M. The Needle in the Haystack: Identifying Credible Mobile Health Apps for Pediatric Populations during a Pandemic and beyond. J Pediatr Psychol 2020;45:1106-13. [Crossref] [PubMed]
12. Jusoh S. A survey on trend, opportunities and challenges of mHealth apps. Int J Interact Mob Technol 2017;11:73-85. [Crossref]
13. Jeminiwa RN, Hohmann NS, Fox BI. Developing a Theoretical Framework for Evaluating the Quality of mHealth Apps for Adolescent Users: A Systematic Review. J Pediatr Pharmacol Ther 2019;24:254-69. [Crossref] [PubMed]
14. Schmidt-Kraepelin M, Toussaint PA, Thiebes S, et al. Archetypes of Gamification: Analysis of mHealth Apps. JMIR Mhealth Uhealth 2020;8:e19280. [Crossref] [PubMed]
15. Interaction Design Foundation, User centered design. Available online: https://www.interaction-design.org/literature/topics/user-centered-design, 2020 [Accessed 10 May 2022].
16. Biediger-Friedman L, Crixell SH, Silva M, et al. User-centered Design of a Texas WIC App: A Focus Group Investigation. Am J Health Behav 2016;40:461-71. [Crossref] [PubMed]
17. McCurdie T, Taneva S, Casselman M, et al. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol 2012;49-56. [Crossref] [PubMed]
18. Williams A, Turer C, Smith J, et al. Adoption of an Electronic Medical Record Tool for Childhood Obesity by Primary Care Providers. Appl Clin Inform 2020;11:210-7. [Crossref] [PubMed]
19. Williams AS, Ge B, Petroski G, et al. Socioeconomic Status and Other Factors Associated with Childhood Obesity. J Am Board Fam Med 2018;31:514-21. [Crossref] [PubMed]
20. Williams AS, Patel PM, Beucke NL, et al. Community-Based Medical Student Nutrition Counseling Training for Low-Income Families. PRiMER 2018;2:5. [Crossref] [PubMed]
21. Gable S, Tosh AK. Children, youth, and parents: screening for obesity risk with the spectrum of physical activity. Mo Med 2014;111:44-8. [PubMed]
22. Proffitt R, Glegg S, Levac D, et al. End-user involvement in rehabilitation virtual reality implementation research. J Enabling Technol 2019;13:92-100. [Crossref] [PubMed]
23. Braddock A, Koopman RJ, Smith J, et al. A Longitudinal Effectiveness Study of a Child Obesity Electronic Health Record Tool. J Am Board Fam Med 2022;35:742-50. [Crossref] [PubMed]
24. Braddock AS, Phad A, Tabak R, et al. Assessing Racial and Ethnic Discrimination in Children: A Scoping Review of Available Measures for Child Health Disparities Research. Health Equity 2021;5:727-37. [Crossref] [PubMed]
25. Hai-Jew S, Alpha testing, beta testing, and customized testing. In: Hai-Jew S. ed. Designing instruction for open sharing. Cham: Springer International Publishing; 2019:381-428.
26. Naeem F, Syed Y, Xiang S, et al. Development, testing and reporting of mobile apps for psycho-social interventions: lessons from the pharmaceuticals. J Med Diagn Meth 2015;4:1000191.
27. George G, Haas MR, Pentland A. Big data and management. Academy of Management Journal 2014;57:321-6. [Crossref]
28. Brooke J. SUS-A quick and dirty usability scale. Usability Evaluation in Industry 1996;189:4-7.
29. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. International Journal of Human–Computer Interaction 2008;24:574-94. [Crossref]
30. Lewis JR. The System Usability Scale: past, present, and future. International Journal of Human–Computer Interaction 2018;34:577-90. [Crossref]
31. Parish-Morris J, Solórzano R, Ravindran V, et al. Immersive virtual reality to im-prove police interaction skills in adolescents and adults with autism spectrum disorder: preliminary results of a phase I feasibility and safety trial. Annual Review of CyberTherapy and Telemedicine 2018;16:50-6.
32. Starling R, Nodulman JA, Kong AS, et al. Usability Testing of an HPV Information Website for Parents and Adolescents. Online J Commun Media Technol 2015;5:184-203. [Crossref] [PubMed]
33. Newton A, Bagnell A, Rosychuk R, et al. A Mobile Phone-Based App for Use During Cognitive Behavioral Therapy for Adolescents With Anxiety (MindClimb): User-Centered Design and Usability Study. JMIR Mhealth Uhealth 2020;8:e18439. [Crossref] [PubMed]
34. Grasaas E, Fegran L, Helseth S, et al. iCanCope With Pain: Cultural Adaptation and Usability Testing of a Self-Management App for Adolescents With Persistent Pain in Norway. JMIR Res Protoc 2019;8:e12940. [Crossref] [PubMed]
35. Langlet B, Maramis C, Diou C, et al. Formative Evaluation of a Smartphone App for Monitoring Daily Meal Distribution and Food Selection in Adolescents: Acceptability and Usability Study. JMIR Mhealth Uhealth 2020;8:e14778. [Crossref] [PubMed]
36. Vlachogianni P, Tselios N. Perceived usability evaluation of educational technol-ogy using the System Usability Scale (SUS): A systematic review. Journal of Research on Technology in Education 2022;54:392-409. [Crossref]
37. Frontini R, Sousa P, Dixe MA, et al. Designing a mobile app to promote healthy behaviors and prevent obesity: analysis of adolescents' preferences. Inform Health Soc Care 2020;45:327-41. [Crossref] [PubMed]
38. McCulloch V, Hope S, Loranger B, Rea P. Children and mobile applications: how to effectively design and create a concept mobile application to aid in the management of type 1 diabetes in adolescents. INTED2016: 10th annual International Technology, Education and Development Conference; 2016; Valencia, Spain.
39. Wang M, Li X. Effects of the aesthetic design of icons on app downloads: evidence from an android market. Electronic Commerce Research 2017;17:83-102. [Crossref]
40. Mustafa AS, Ali N, Dhillon JS, et al. User Engagement and Abandonment of mHealth: A Cross-Sectional Survey. Healthcare (Basel) 2022;10:221. [Crossref] [PubMed]