Engage, Develop, Learn: a pilot randomized clinical trial to test if a mobile application can enhance home support for early cognitive development among children living in poverty

Introduction

Responsive parenting to promote early childhood development is a key determinant of future academic success and quality of life. Children living in poverty are at increased risk for delayed cognitive, language, and social-emotional development when compared to their higher income peers (1-9). These delays may be mediated by providing supportive, loving relationships with caregivers. We previously suggested that mobile health (m-Health) interventions could promote positive caregiver interactions and improve child development among low-income families (10,11). m-Health interventions, such as text messaging and mobile applications (apps), are now widely available and used. However, parents with less income, lower education, lower health literacy, and those who do not speak English are less likely to engage with m-Health solutions, thus reducing the potential effectiveness of these interventions among populations who could most benefit (12-14).

To study the effect of a m-Health intervention on caregivers’ child development promoting behavior, we created an app, Engage, Develop, Learn (EDL), and conducted a pilot randomized clinical trial among low-income caregivers of children ages 12 to 24 months. Our objectives were to measure user engagement with the EDL app, determine if engagement resulted in sustainable behavior change, and determine if certain features of the app successfully incentivized caregivers to talk, read, play with, and praise their young children. We present the following article in accordance with the CONSORT reporting checklist (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-13/rc).

Methods

Study design

This pilot randomized clinic trial of the EDL intervention was conducted from September 2017 through November 2019. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Colorado Multiple Institutional Review Board (Protocol No. 15-2164) and informed consent was taken from all the study subjects.

EDL development

The EDL intervention was guided by the Theory of Planned Behavior and Social Cognitive Theory. The Theory of Planned behavior suggests that intention is a powerful predictor of behavior and is influenced by a person’s attitudes toward the behavior, norms, and perceived control to carry out said behavior (15,16). Social Cognitive Theory endorses that self-efficacy is a critical determinant of behavior change, which includes one’s skills and self-confidence to perform a behavior, and can be broken into smaller, more manageable steps (16,17). EDL intended to deliver accessible and understandable positive parenting content in the daily environment, thus increasing caregivers’ self-efficacy by influencing attitudes and norms. We developed the messaging structure of the EDL app with input from a parent advisory board whose members were from similar sociodemographic background as study participants and a multi-disciplinary team of experts including pediatricians, a speech pathologist, and a child psychologist. A user experience designer helped design the app with feedback from the parent advisory board.

EDL content and features

EDL content delivered short messages aimed for the child’s specific age. Messages were written at a 6^th grade reading level and were personalized to reflect child’s first name, age, and gender. Messages included information about child development and games/activities encouraging caregivers to engage with their children with a focus on four categories of child development-promoting behaviors: talking, reading, playing, and praise. Messages were sourced from Bright by Text (https://brightbytext.org), Learning Games and Language Power (18,19). Our team also developed new messages and activities based on expert knowledge and experience. All messages were positively framed to maximize effectiveness and impact and sought to use the ecological moment to help caregivers implement change (20-22). Private Facebook groups were established in English and Spanish. Study staff posted daily messages to share content, complement participant activities delivered via the app and stimulate participants to comment, answer questions, or share experiences.

EDL utilized a point system to enhance user experience and subsequent engagement. Features implemented in the EDL app included:

• Goal setting: caregivers were invited to set a shared reading goal;
• Reading log: a timer allowed caregivers to track the duration spent reading with their child. The reading log provided caregivers a screen to report this time. The built-in EDL function used time/log data to track progress toward the caregiver’s reading goal and award points;
• Points: caregivers received 1 point for each minute of time spent reading and up to 15 points for indicating that they completed an activity focused on increasing talking, playing, or praise;
• Rewards: caregivers were eligible to receive a prize, a children’s book, once obtaining a pre-determined number of points.

Recruitment and participants

Study participants were recruited from a large, urban pediatric clinic serving mainly low-income families that participates in the Reach Out and Read and Healthy Steps programs. Caregivers of children between the ages of 12 and 15 months who were low-income [based on insurance status of child being Medicaid or Children’s Health Insurance Program (CHIP)], had less than college level education, had a smart phone, and speak English or Spanish were invited to participate. Caregivers whose children were born prior to 36 weeks gestation, had conditions known to affect child development, screened positive on the Children with Special Health Care Needs Screener (23), or were already receiving developmental therapies were excluded. We did not exclude children who received Reach Out and Read books and/or Healthy Steps, because we aimed to conduct a pragmatic trial that reflects the ‘real life’ context for families. We wanted to understand if the EDL app could augment the care already provided by pediatric primary care clinics.

After consenting to participate in the study, caregivers were randomized using a computerized algorithm in Research Electronic Data Capture (REDCap) to one of two parallel groups: EDL or an attention control which was an injury prevention text messaging program. Participants were randomized with a 1:1 allocation ratio, using a block size of 4. The random allocation sequence (i.e., randomization table) was generated in SAS and uploaded to REDCap by the data analyst. Once the randomization table was uploaded to REDCap, the randomization functionality allowed no access to the random allocation sequence by anyone. The randomization status of each participant was displayed in REDCap once the “Randomize” button was clicked for each consented participant. Caregivers who completed the eligibility survey received a $10 gift card. Since this was a pilot study with the goal of determining feasibility and sample size needed for a future trial, we determined sample size based on available resources.

Data collection

Intervention delivery and data collection visits

After their initial recruitment in the clinic setting, intervention delivery and baseline data collection occurred over two consecutive visits in participants’ homes or community locations per recommendations of the parent advisory board. The EDL intervention was delivered by a bilingual ‘Infant Development Advocate’ who was a community member hired and trained to help caregivers download the app, explain its use, and discuss key concepts for promoting child development. Professional research assistants (PRAs) delivered the control intervention by setting up caregivers’ phones to receive injury prevention text messages. Child development measures were collected by PRAs during the first visit and the randomly assigned intervention was delivered and injury prevention measures were collected by PRAs during the second visit. Study participants received $30 and $40 gift cards following the first and second visits, respectively. Participants and those collecting the data were not blinded to group assignments.

Follow-up data collection occurred within 3 months of the child’s second birthday and was conducted by PRAs in the study participants’ home, community location, or clinic. For participants who were unable to complete a visit, parent-reported follow up measures were collected by telephone. Study participants received a $50 gift card for completing the follow up visit. Participants who were randomized to receive the EDL app were also invited to participate in key informant interviews by telephone. Participants who completed the key informant interviews received a $20 gift card.

Measures

Caregivers were administered the STIMQ Toddler at baseline and follow up. The STIMQ is a self-reported measure of parent-child activities that promote cognitive stimulation (24,25). The subscales used to quantify parental behaviors were: reading activities (READ) which assesses frequency of shared reading, number and diversity of books and associated activities; parent involvement in developmental activities (PIDA) which assesses parent engagement in teaching activities such as naming body parts, stacking blocks, pointing to and naming items at the grocery store; and parent verbal responsibility (PVR) which assesses parent-child verbal interactions. We hypothesized that use of the EDL app would improve the READ, PIDA, and PVR subscale scores by increasing parental knowledge and self-efficacy and specifically encouraging talking, reading, playing, and praise.

A questionnaire based on the Theory of Planned behavior (15,16) and developed by the study team with pre-testing from the parent advisory board was used to assess parental beliefs, attitudes and norms about target development-promoting behaviors at baseline and follow-up. Caregivers were asked to report their responses on a 6-item scale with the options of ‘less than once a month’, ‘about once a month’, ‘a few times a month’, ‘about once a week’, ‘a few times a week’, and ‘every day’.

Primary outcome measures included the STIMQ Toddler and the questionnaire developed by the study team described above, as well as results of key informant interviews with caregivers randomized to the EDL group described below. Control group measures were secondary outcome measures and included: (I) a survey regarding prior injury prevention education, current safety practices, knowledge of poison control numbers/clinic numbers/cardiopulmonary resuscitation (CPR)/first aid, report of child injuries during the past year, and attitudes and beliefs about safety practices; and (II) a home observation checklist completed by the PRA. Control results are not reported in this manuscript focused on child development.

Statistical analysis

Chi-square tests and t-tests were performed to assess potential differences between the sociodemographic characteristics of the EDL and injury prevention groups at baseline. Descriptive statistics were used to describe technology use among participants and quantitative measurements of engagement with the app. All analyses were conducted as intention-to-treat, meaning that those who were randomized to receive the EDL app were analyzed in the intervention group regardless of their actual use of the app. The nonparametric Wilcoxon Signed rank test was used to assess change in STIMQ scores between baseline and follow up within groups. Difference between groups for change in STIMQ scores from baseline to follow up was assessed using the nonparametric Wilcoxon rank sum test. Fisher’s exact test was used to determine whether the change in reading and playing behaviors between baseline and follow up was significantly different between groups.

Qualitative analysis

All study participants who participated in the data collection visits and received the EDL intervention (n=25; see Figure 1) were contacted with an invitation to participate in key informant interviews using a standardized protocol (up to 2 phone calls, 2 text messages, and 2 emails to the primary contact and 1 phone call and 1 text to the secondary contact). One bilingual PRA conducted semi-structured interviews of the 13 caregivers who received the EDL intervention and agreed to participate in the interviews. The interview guide was developed by the study team based on existing published literature regarding barriers and facilitators to app use and behavior change. The guide focused on usability of the app itself and factors outside of the app that affected its use, message content, likes and dislikes about the reading log and activities, and perceived effect of the app on development promoting behaviors and child development (see Appendix 1). The interviews were transcribed and translated (Spanish to English). The codebook was developed using an iterative process in which four members of the study team (MC, CG, MA, and CC) reviewed sample transcripts and suggested codes then met to discuss the codes and reach agreement about codes and their definitions. A final codebook was agreed upon after review of three transcripts. All 13 transcripts were coded by two members of the study team (MC and CG) who met regularly to ensure consensus regarding codes. Then the study team used a summative content analysis approach where common codes and combinations of codes were grouped, summarized, and placed in context (26).

Figure 1 CONSORT flow diagram. *, one caregiver didn’t have the baseline visit #2 but received the follow-up home visit. EDL, Engage, Develop, Learn.

Results

As shown in Figure 1, 50 caregivers initially randomized to receive the EDL app, 31 completed the first and 25 completed the second visit and downloaded the app. Of the 50 caregivers initially randomized to the control group, 38 completed the first and 34 completed the second visit and enrolled in the injury prevention text messaging intervention. Fourteen caregivers in the EDL group and 30 in the control group completed follow up 1 year after enrollment. We conducted intention-to-treat analyses; however, low participation in the data collection visits outside of the clinical setting prevented us from including all 100 of those who initially agreed to participate in the study because we lacked data from those who did not participate in the data collection visits.

Study participants

A total of 100 caregivers were recruited for the study from September 2017 through January 2018 during a pediatric clinic visit. Table 1 describes the sociodemographic characteristics of the study population. There were no significant sociodemographic differences between the caregivers randomized to the EDL vs. control group. Most caregivers were mothers with 2 grandmothers and 1 grandfather participating. Table 2 shows the baseline reported technology access and use among study participants.

EDL engagement

Six caregivers in the EDL intervention group (24%) remained engaged with the app throughout the 10-week period opening most messages delivered most weeks. One caregiver opened messages the first 3 weeks and then ceased engagement and six opened messages for the first 2 weeks then ceased engagement. Six caregivers only opened messages the first week (likely during the study visit) and never engaged with the app again. Three caregivers engaged with the app more sporadically, but total weeks of engagement throughout the 10-week period remained low with caregivers opening messages only 2 to 3 of the 10 weeks. Eight caregivers (32%) earned enough points to receive a reward. Two caregivers earned one reward (50 points), two earned two (100 points), one earned four (300 points) and three earned five (500 points). Only 12 English-speaking and 3 Spanish-speaking caregivers joined the private Facebook group with two English-speaking and no Spanish-speaking caregivers engaging on Facebook.

Behavior change

Table 3 shows the STIMQ data collected at baseline and follow up. Among those for whom we had both baseline and follow up STIMQ subscale scores, we found that the PIDA and READ scores increased from baseline to follow up for both groups (READ P=0.001 EDL, P=0.004 injury prevention; PIDA P=0.004 EDL, P<0.0001 injury prevention). There were no statistically significant differences between participants assigned to EDL vs. the control group. There was no significant change in PVR scores between baseline and follow up for either group.

Parental beliefs, attitudes, and norms were not statistically different between the EDL and control groups at baseline or follow-up. At baseline, 19% (6/31) of caregivers in the EDL group and 39% (15/38) in the control group reported reading to their children daily, and 77% (24/31) in the EDL group and 95% (36/38) in the control group reported playing with their children daily. At follow up, the reported frequency of reading increased for 57% (8/14) of caregivers in the EDL group and 38% (11/29) in the control group and the reported frequency of playing increased for 7% (1/14) in the EDL group and 4% (1/29) in the control group. These changes from baseline to follow-up were not statistically different between groups.

Barriers and facilitators to EDL app use

Among the 13 study participants who received the EDL app and agreed to participate in semi-structured interviews, all 13 set a reading goal (they were prompted to do this when they first downloaded and opened the app) but only 5 used the app enough to earn one or more rewards. Among the 5 who earned rewards, 4 received 4 or more rewards (300 or more points) indicating regular engagement with the app, and 1 received only 1 reward (50 points) indicating limited engagement with the app. The remaining 8 of those interviewed did not receive any rewards (<50 points) indicating minimal engagement with the app. Content analysis from the semi-structured interviews among 10 English-speaking and 3 Spanish-speaking participants focused on 4 themes: “positive app features”, “behavior change”, “barriers to use”, and “incentives and rewards”. Illustrative quotes are provided in Table 4.

Positive app features

Most caregivers interviewed found the app easy to use. Most caregivers also reported that they liked the content of the messages and activities. A few caregivers specifically mentioned that they liked receiving messages that explained why specific behaviors or activities were helpful to their child’s development. Many caregivers also appreciated having a reading log and timer so they could track weekly reading minutes. A few caregivers specifically mentioned that they liked being able to set a goal and track their progress toward that goal.

Behavior change

Caregivers reported that the app encouraged changes in the way they read with their children and talked to them throughout the day. The most common behavioral changes endorsed were increased talking and engagement in dialogic reading. One caregiver gave a specific example of how she had increased her talking to her child. Another gave an example that showed she had learned to engage in dialogic reading. Another reported playing more with her child after using the app. Finally, a few caregivers indicated that they had changed how they praised their children based on what they learned from the app. For example, one mother described how she was more vocal about when she was proud of her child.

Barriers to use

Engagement with the app was low for the majority of 13 caregivers we interviewed. Upon further questioning the most common barrier to use, reported by 8 caregivers, was inability to access or download the app without assistance due to forgotten passwords, phone change, phone reset, or phone not functioning properly. Two caregivers reported that they often forgot to use the app. One mother reported that the app did not allow her to engage with others.

Incentives and rewards

Most of the caregivers we interviewed reported knowing that they could earn points for completing activities and logging reading minutes. The majority reported that points were “easy” to earn, but most did not earn any rewards or could not remember if they had earned rewards. One caregiver reported that she was earning points but did not know about rewards. A few caregivers explained that they learned about points and rewards at the home visit but were still confused about how to earn or receive rewards.

Discussion

We conducted a pilot randomized trial to determine if the EDL app could successfully promote behavior change among low-income caregivers of young children. Our objective was to determine if certain features of the EDL app could enhance engagement with the app and incentivize caregivers to talk, read, play with, and praise their children. We experienced challenges with completing visits to deliver intervention and collect data among our study population. Only half of the caregivers who were initially randomized to receive the EDL intervention actually completed the data collection visits and, among the 25 caregivers who downloaded the app, only 6 (24%) engaged with it for the entire 10-week study period.

We did not find statistically significant changes in self-reported behaviors and STIMQ scores between caregivers who were randomized to receive the EDL intervention vs. the control intervention using an intention to treat analysis. While not statistically significant, we did find that more caregivers who were randomized to receive the EDL intervention increased the target behaviors based on self-report compared to the control group, and our qualitative data suggest that those who engaged with the EDL app changed their behavior based on what they learned from the app.

Previous studies of m-Health interventions to promote positive parenting behaviors have often relied on a convenience sample without a control group (27,28). A strength of our pilot study is that participants were randomized to receive either the EDL or attention control (injury prevention) intervention. The control group and the intervention group received a similar volume of content reducing the likelihood that the changes we observed in the intervention group were a result of the Hawthorne effect. Our selection of only caregivers who had not graduated from college and whose children had public health insurance was also unique, enabling us to focus on a subset of the population that has been traditionally harder to engage with m-Health solutions. A cohort study of the Text4Baby program where women were randomly selected to be offered the text messaging program found similar results to ours in terms of engagement. About half of those who were randomly selected attempted to enroll to receive text messages, and those who successfully enrolled were more likely to have at least some college education (12). Similarly, a recent systematic review of digital health interventions to increase physical activity found that these interventions were not effective among participants with low socio-economic status (14). Another strength of our study was that usage data collected through the app gave us objective data to measure app usage while semi-structured interviews allowed for the interpretation of our quantitative findings in an explanatory sequential manner.

Despite using strategies known to promote app engagement to develop the EDL intervention, only 24% of users demonstrated sustained engagement with the app at 10 weeks in our study (29,30). This is consistent with previous studies of digital self-help interventions that have shown rates of completion or sustained engagement beyond 6 weeks that range from 0.5–28.6% (31). Specifically, 3 therapeutic persuasiveness items—rewards, adaptive design, and on-going feedback—have been shown to significantly correlate with user retention (29). While we did not employ an adaptive design for this pilot, the EDL app utilized rewards and on-going feedback. While a total of 32% of users were able to earn at least 1 reward, incentives did not seem to promote app use among the caregivers we interviewed. Most of these users did not earn rewards despite reporting that points were easy to earn. This may be because rewards were not emphasized within the app, and some caregivers were unaware of the potential to earn rewards or how to receive rewards once they were earned. Some caregivers did not earn rewards despite reporting that they were aware of the gamification features that allowed them to earn rewards. This suggests that rewards might be less persuasive in encouraging app use and retention among our target population, or that the rewards selected (children’s books) were not persuasive.

We found that few caregivers engaged with Facebook though one caregiver reported that connecting with other mothers was important to her and she did not feel like the app provided sufficient contact with other caregivers. This suggests that perhaps social media is not an effective platform to facilitate these connections and face to face or real time contact via an on-line video platform may be more successful in establishing connections and promoting engagement.

Our pilot study had limitations. Though we initially recruited 100 study participants as planned for this pilot, many of those initially recruited in a clinic setting did not continue with the study outside of the clinic setting resulting in a smaller sample size than originally planned. In addition, we experienced high attrition with more attrition among the group randomized to receive the EDL app. Despite our use of methods known to encourage study retention, the 1-year follow up visit was only completed for 14 in the EDL group and 29 in the attention control group. For future studies, we would consider routinely conducting data collection directly before or after well child care visits in the clinic setting, when possible, given the difficulty we experienced with collecting data outside of the clinic setting. This difficulty resulted in small sample sizes, baseline differences for the outcome of interest between the EDL intervention and control groups, and differential attrition. To address these issues, we would also consider emphasizing the need for long term participation at the time of recruitment, decreasing the monetary incentive for completing the recruitment process, and increasing the monetary incentive for completing follow up data collection. An additional limitation was that our measures of talking, reading, playing, and praise were based on parent/caregiver-reported measures rather than direct observation.

The findings from our pilot randomized trial suggest that the EDL app may increase caregivers’ child development promoting behaviors if they are able to actively engage with the app. However, we also found that the majority of low-income caregivers with less than college education whom we recruited for the study did not engage with the app. The most common barrier to engagement was difficulty accessing the app. Some of the features that caregivers who did access the app found to be most be useful could be delivered via text messaging. Therefore, focusing on text messaging may be a solution to overcome the access barriers to using an app. We considered providing phones and/or data plans to study participants to decrease m-health access barriers, but chose not to because of our desire to conduct a pragmatic trial that reflected the ‘real life’ context of the families we thought were most likely to benefit from the intervention.

Our findings imply that, while m-health interventions may be one way to promote positive parenting, they are not a panacea for reducing disparities in early child development as they may not effectively reach caregivers of the children most in need of an intervention to promote child development. If the goal is to reduce disparities in child development among low-income families, we suggest that m-health interventions may not be adequate on their own and should be studied as a way to augment other interventions, such as Reach Out and Read, the Video Interaction Project, and home-visiting programs that have been shown to successfully increase child-development promoting behaviors among caregivers. Most m-Health that is commercially available is designed as a stand-alone intervention. Little has been done to explore how mHealth in pediatric intervention can be designed to supplement, enhance, and intensify effects of in-person interventions. We have seen evidence that mHealth interventions when linked to in vivo interventions can achieve this (32). Researchers have emphasized that the benefit of mHealth is likely in generating small effects through intervention that can reach exponentially greater audiences than in person intervention (33). A key could be to find a way to build on the success of a strong in-person intervention with booster or enhanced content delivered digitally.

Acknowledgments

Erin McBurney contributed to recruiting participants and data collection.

Funding: Development of the Engage, Develop, Learn mobile application and its evaluation were funded by Walton Family Foundation [Nonprofit 501(c)(3)], Daniel’s Fund [Nonprofit 501(c)(3)], Frankenburg-Camp Fund [Nonprofit 501(c)(3)], and the Gantz Family [Nonprofit 501(c)(3)].

Footnote

Reporting Checklist: The authors have completed the CONSORT reporting checklist. Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-13/rc

Trial Protocol: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-13/tp

Data Sharing Statement: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-13/dss

Peer Review File: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-13/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-22-13/coif). CAD received payments from Walton Family Foundation, Daniel’s Fund, made to her institution to conduct the BB3 project which supported a small portion of her FTE to work on this project; received payments from Children’s Hospital Foundation, which were made to her institution to support part of her FTE to conduct other research; and she also reports that owning stock in the Amgen company. The other authors report that Development of the Engage, Develop, Learn mobile application and its evaluation were funded by Walton Family Foundation [Nonprofit 501(c)(3)], Daniel’s Fund [Nonprofit 501(c)(3)], Frankenburg-Camp Fund [Nonprofit 501(c)(3)], and the Gantz Family [Nonprofit 501(c)(3)]; there are no conflicts of interest with any of the above organizations. SB also reports that was on the board of Bright by Three/Bright by Text and the Gantz Family Foundation at the time of this study.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Colorado Multiple Institutional Review Board (Protocol No. 15-2164) and informed consent was taken from all the study subjects.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Shonkoff JP, Phillips DA. editors. From Neurons to Neighborhoods: The Science of Early Childhood Development. Washington, DC: National Academies Press, 2000.
2. Arkin E, Braveman P, Egerter S, et al. Time to act: Investing in the health of our children and communities. Princeton: Robert Wood Johnson Foundation, 2014. Available online: https://www.rwjf.org/en/library/research/2014/01/recommendations-from-the-rwjf-commission-to-build-a-healthier-am.html
3. Bradley RH, Corwyn RF. Socioeconomic status and child development. Annu Rev Psychol 2002;53:371-99. [Crossref] [PubMed]
4. Braveman P, Sadegh-Nobari T, Egerter S. Early childhood experiences: Laying the foundation for health across a lifetime. Princeton: Robert Wood Johnson Foundation, 2008. Available online: http://hdl.handle.net/10244/613
5. Hair NL, Hanson JL, Wolfe BL, et al. Association of Child Poverty, Brain Development, and Academic Achievement. JAMA Pediatr 2015;169:822-9. [Crossref] [PubMed]
6. Hart B, Risley TR. Meaningful differences in the everyday experience of young American children. Baltimore: Paul H. Brookes Publishing, 1995.
7. Cunningham M, Bonnell LN, Mehner L, et al. A cumulative risk index for use in the medical home. Child Care Health Dev 2021;47:471-6. [Crossref] [PubMed]
8. Lurie LA, Hagen MP, McLaughlin KA, et al. Mechanisms linking socioeconomic status and academic achievement in early childhood: Cognitive stimulation and language. Cogn Dev 2021;58:101045. [Crossref] [PubMed]
9. Salhi C, Beatriz E, McBain R, et al. Physical Discipline, Deprivation, and Differential Risk of Developmental Delay Across 17 Countries. J Am Acad Child Adolesc Psychiatry 2021;60:296-306. [Crossref] [PubMed]
10. McClure C, Cunningham M, Bull S, et al. Using Mobile Health to Promote Early Language Development: A Narrative Review. Acad Pediatr 2018;18:850-4. [Crossref] [PubMed]
11. Selby E, Allabyrne C, Keenan JR. Delivering clinical evidence-based child-parent interventions for emotional development through a digital platform: A feasibility trial. Clin Child Psychol Psychiatry 2021;26:1271-83. [Crossref] [PubMed]
12. Gazmararian JA, Elon L, Yang B, et al. Text4baby program: an opportunity to reach underserved pregnant and postpartum women? Matern Child Health J 2014;18:223-32. [Crossref] [PubMed]
13. Cunningham M, Bull S, McNulty MC, et al. Does a text-messaging program to promote early childhood development reach the highest risk families? Mhealth 2018;4:55. [Crossref] [PubMed]
14. Western MJ, Armstrong MEG, Islam I, et al. The effectiveness of digital interventions for increasing physical activity in individuals of low socioeconomic status: a systematic review and meta-analysis. Int J Behav Nutr Phys Act 2021;18:148. [Crossref] [PubMed]
15. Ajzen I. Behavioral interventions based on the theory of planned behavior. 2006. Available online: https://people.umass.edu/aizen/pdf/tpb.intervention.pdf
16. Glanz K, Rimer B, Viswanath K. editors. Health behavior and health education: theory, research, and practice. 4th ed. San Francisco: Jossey-Bass, 2008.
17. Bandura A. Social cognitive theory: an agentic perspective. Annu Rev Psychol 2001;52:1-26. [Crossref] [PubMed]
18. Camp BW. Language Power. Baltimore: Brookes Publishing Co., 1999.
19. Sparling J, Lewis I. Learning Games. Washington, DC: Teaching Strategies, Inc., 2008.
20. Petty R, Barden J, Wheeler S. The Elaboration Likelihood Model of Persuasion. In: DicClemente R, Crosby R, Kegler M. editors. Emerging Theories in Health Promotion Practice and Research. 1st ed. San Francisco: Jossey-Bass, 2002: 71-99.
21. Petty R, Wegener D, Chaiken S. The elaboration likelihood model: Current status and controversies. New York: Guilford Press, 1999.
22. Schneider TR, Salovey P, Apanovitch AM, et al. The effects of message framing and ethnic targeting on mammography use among low-income women. Health Psychol 2001;20:256-66. [Crossref] [PubMed]
23. The Child and Adolescent Health Measurement Initiative. Children with Special Health Care Needs (CSHCN) Screener. Available online: https://depts.washington.edu/dbpeds/Screening%20Tools/CSHCN-CAMHIScreener.pdf
24. Dreyer BP, Mendelsohn AL, Tamis‐LeMonda CS. Assessing the child's cognitive home environment through parental report; reliability and validity. Early Dev Parent 1996;5:271-87. [Crossref]
25. Dreyer BP, Mendelsohn AL, Kruger HA, et al. StimQ, a new scale for assessing the home environment: reliability and validity. Arch Pediatr Adolesc Med 1994;150:abstr 47.
26. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005;15:1277-88. [Crossref] [PubMed]
27. Olson KB, Wilkinson CL, Wilkinson MJ, et al. Texts for Talking: Evaluation of a Mobile Health Program Addressing Speech and Language Delay. Clin Pediatr (Phila) 2016;55:1044-9. [Crossref] [PubMed]
28. Wang CJ, Chaovalit P, Pongnumkul S. A Breastfeed-Promoting Mobile App Intervention: Usability and Usefulness Study. JMIR Mhealth Uhealth 2018;6:e27. [Crossref] [PubMed]
29. Baumel A, Kane JM. Examining Predictors of Real-World User Engagement with Self-Guided eHealth Interventions: Analysis of Mobile Apps and Websites Using a Novel Dataset. J Med Internet Res 2018;20:e11491. [Crossref] [PubMed]
30. Montagni I, Mabchour I, Tzourio C. Digital Gamification to Enhance Vaccine Knowledge and Uptake: Scoping Review. JMIR Serious Games 2020;8:e16983. [Crossref] [PubMed]
31. Fleming T, Bavin L, Lucassen M, et al. Beyond the Trial: Systematic Review of Real-World Uptake and Engagement With Digital Self-Help Interventions for Depression, Low Mood, or Anxiety. J Med Internet Res 2018;20:e199. [Crossref] [PubMed]
32. Bull S, Devine S, Schmiege SJ, et al. Text Messaging and Teen Sexual Health Behavior: Long-term Follow-up of a Cluster Randomized Trial. Comput Inform Nurs 2017;35:549-53. [Crossref] [PubMed]
33. Bull SS. Sexual Assertiveness Skills and Decision-Making in Adolescent Girls: Moving to Replication, Scale, and Digital Health Impact. Am J Public Health 2018;108:18-9. [Crossref] [PubMed]