A randomized controlled field trial of iBsafe—a novel child safety game app

Introduction

Unintentional injury is a leading cause of childhood death and disability in the United States (US) and worldwide (1-3). Kindergarten-aged children have some of the highest injury rates. Two of the top ten causes of non-fatal injury among 5- and 6-year-old US children are bicycle injuries and dog bites (3). Bicycle injuries and dog bites together account for over 50,000 emergency department visits and 1,000 hospitalizations of 5- and 6-year-old children each year, with combined costs approximating $250 million (1,3,4).

Multimedia education can increase child knowledge of bicycle safety and dog bite prevention (5-8) and interactive games can improve children’s understanding and management of their health (9). Mobile technology is a rapidly expanding field with the potential to deliver interactive safety programs. There are currently 7.5 billion mobile device subscriptions worldwide (10). Seventy-seven percent of adults in the US own smartphones with recent increases in ownership among low-income families (11). Despite recommendations to limit screen time, half of young children regularly use mobile devices, most often to play games (12). Thus, we posit that injury prevention programs embracing mobile platforms with an interactive game experience can have a positive preventative impact, simultaneously reaching large populations with rapid dissemination.

We conducted a pilot randomized controlled field trial of iBsafe—an “interactive Bike and Bite safety” game application (app). Founded in behavioral theory, iBsafe was designed to teach kindergarten-aged children evidence-based/expert-guided strategies to be safe on bicycles and around dogs; its programming tracks game play data for correlation with outcomes. Details of iBsafe development and content are described elsewhere (13). This study aimed to determine the effect of iBsafe on child safety knowledge and translation of knowledge to skills practice. Secondary outcomes were to evaluate the frequency of safety discussions between children-parent dyads during the trial, and to assess iBsafe acceptability. We hypothesized that children who received iBsafe would demonstrate increased bicycle and dog safety knowledge and perform more safety skills than controls.

Methods

Design

This randomized, controlled, single-blinded, field trial of iBsafe was conducted between November 30^th, 2013 and February 23^rd, 2014. The local institutional review board approved the study (ID: 2013-3991).

Participants

Potential child participants were identified from a limited chart review of recent 5- or 6-year-old physicals performed at a local pediatric community group practice. Child-parent/guardian dyads were excluded if either were known to be non-English speaking; and/or the child was: developmentally/emotionally incapable of understanding their involvement, a sibling of a child already enrolled in the study, or deemed unsuitable for participation by their pediatrician.

Recruitment

Potentially eligible dyads were mailed a letter from their pediatrician outlining the study. Also enclosed was a stamped, return-addressed, “Do Not Contact” postcard for the parents to mail if they did not want to be contacted. Dyads for whom a postcard was not received within 10 days of the initial mailing were contacted by study personnel. During this phone contact, personnel explained the study, assessed interest in participation, and scheduled study procedures.

Enrollment and randomization

Enrollment occurred at the baseline visit during which informed written consent was obtained from all parent or legal guardian participants by study staff. Using a computer-generated, simple random numbers table (produced in advance of the study start), consented participants were sequentially and irreversibly assigned one of two conditions: (I) iBsafe group, and (II) control group.

Setting

Study procedures occurred over 13 consecutive weekend sessions at one of the institution’s outpatient facilities where our simulated safety lab was maintained. Separate areas were available for parents and children to undergo study procedures independently.

Data collection and outcome measurements

Baseline assessment

Parents completed questionnaires that assessed sociodemographics; mobile device use; child bicycle- and dog-related experiences/injuries; bicycle and dog ownership; previous injury prevention education; and current injury prevention practices including discussions about safety. Children completed a 20-item knowledge pre-test with text and picture questions about bikes and dogs. This test was read to the child in the absence of the parent, and answers were recorded by research staff.

Field trial intervention

After baseline assessment, children received their assigned study packets, which were taken home for 1 week. Packets had identical exteriors. Intervention packets included a loaner Apple iPod Touch device with the installed iBsafe app, a wall charger and power cord. iPod Touch devices were programmed to be a closed/locked system, ensuring that the only activity the device could perform was the iBsafe game app. Dyads were instructed that use of the device was voluntary; however only the child enrolled in the study could do so. Control packets included American Academy of Pediatrics (AAP) nutritional pamphlets, intervention packets did not (14).

Follow-up assessment

Follow-up assessment occurred 1 week after randomization. Study packets were returned and parents completed surveys to assess injury prevention practices and safety discussions during the trial. Children received the identical 20-item knowledge post-test in the same manner as the baseline assessment. Intervention dyads completed iBsafe acceptability surveys.

All child participants underwent identical skills testing procedures. Because one cannot put children at true risk of injury, testing occurred in our simulated safety lab. This environment had multiple stations representing the different safety strategies promoted within iBsafe game play. Stations entailed real props, giving children an opportunity to engage in simulated experiences. Bicycle safety props included helmets, bicycles, and simulated streets/sidewalks. Dog safety props involved live dogs in settings such as behind a fence, eating, and tethered. To ensure child safety, we used certified therapy dogs from our pediatric institution’s dog visitation program. Three volunteer dogs and their owners were selected a priori so that dogs would have similar characteristics. Specifically, we used Golden Retrievers dogs since these were perceived as less threatening in our prior work (6). While each testing session utilized just one therapy dog, all dog owners were trained on testing procedures to ensure consistency of the dog’s behaviors at each station (e.g., dog lying down in “stay” while at the tethered station). At each station, children were prompted to demonstrate or role-play what they would do. A trained research coordinator who was blinded to the child’s randomization group independently observed, scored and documented the child’s skill performance by assessing the child’s verbal responses and non-verbal actions.

Study conclusion procedures

At the end of each testing session, iBsafe data were downloaded from each iPod Touch device. All study participants received AAP bicycle safety and dog bite prevention pamphlets. Control participants were given the option to take home an iBsafe study packet for a week if the child desired to play the iBsafe game.

Statistical analysis

Data were managed using REDCap electronic data capture tools (15). Bicycle safety and dog bite prevention knowledge test scores and safety skills assessments were calculated pre- and post-intervention for each child participant. Scores were calculated by summing the number of correct answers to the 20 knowledge test questions and separately the number of correct responses at the skills stations. Any missing responses were coded as being incorrect. Missing answers were minimal. One case was missing the post-intervention dog bite prevention safety skills assessment. Eight cases were missing answers to one question on the assessments, and two cases were missing answers to three questions.

A sample size of 60 (30 in each arm) was selected a priori based on recommendations for conducting pilot testing (16). To account for within subject variability as well as between subject variability, a repeated measures ANOVA was used to compare knowledge tests pre- and post-intervention between groups, differences between groups with 95% confidence intervals are reported. The Mann-Whitney U-test was used to compare safety skill assessment scores between groups. All statistical analyses were conducted using SPSS 22.0 (IBM Corporation, Armonk, NY) and R (base). Some graphics were created using R (3.2.4).

Results

Study population

To enroll 60 children, we screened 283 charts and mailed 201 invitations. Six percent of those mailed invitations returned “Do Not Contact” postcards. Nearly 80% of eligible participants who were interested in participating enrolled in the study. All 60 dyads completed the study and were included in this analysis (Figure 1).

Figure 1 Recruitment and study flow of iBsafe trial.

No substantial differences in demographics or mobile device use were noted between groups (Table 1). Over 90% of parents were Caucasian and the majority had a college degree. More than half of children were male; most were 6 years old. One hundred percent of children used mobile devices; the majority spent 30 minutes to 1 hour on these devices daily. Nearly 80% of parents reported that “game play” was their child’s most frequent mobile device activity. Both parent groups strongly endorsed the notion that children can learn from mobile apps (>80%) and felt it was important or very important to have mobile app games for child safety (>70%).

Table 1 Study population demographics and mobile device use/perceptions by randomization group
Full table

Bicycle and dog-related experiential histories between groups were also similar (Table 2). Over 95% of children in both groups rode bicycles; 25% of children had been injured while doing so. While most parents felt that bicycle safety education was important, only 20% of children had received formal bicycle safety education. For dog-related questions, 90% of children were regularly around dogs and nearly half of homes had dogs. More than 13% of children had previously suffered a dog bite. Although 100% of parents felt it was important or very important for their child to learn how to be safe around dogs, only three children had ever received formal dog bite prevention education.

Table 2 Study population safety experiential history/perceptions by randomization group
Full table

Knowledge tests

Both bicycle and dog safety knowledge scores increased with exposure to iBsafe (Figure 2). At baseline, the iBsafe group had significantly lower bicycle safety knowledge scores than the control group (7.8±1.4 vs. 8.7±0.9; difference −0.9; 95% CI: −1.586 to −0.348; P=0.003). Post-intervention, the iBsafe group had significantly higher bicycle safety knowledge scores than the control group (9.2±0.9 vs. 8.7±1.0; difference 0.5; 95% CI: 0.056 to 1.011; P=0.029). Baseline dog safety knowledge scores for both groups were similar (iBsafe 5.7±2.7 vs. control 6.1±1.9; difference 0.4; 95% CI: −1.601 to 0.801; P=0.508). Post-intervention, the iBsafe group had significantly higher dog safety knowledge scores than the control group (8.2±2.1 vs. 6.7±1.8; difference 1.5; 95% CI: 0.551 to 2.582; P=0.003).

Figure 2 Comparison of pre-/post-intervention bicycle and dog safety knowledge between groups.

Safety skills assessment

The median number of demonstrated safety skills was higher in the iBsafe group for both bicycle and dog-related components (Figure 3). The median number of bicycle safety skills in iBsafe participants was 5, compared to 4 in control participants (difference 1.0; 95% CI for difference in medians 0.52 to 1.48, P=0.007). The median dog-related safety skills in iBsafe participants was 5, compared to 3 in control participants (difference 2.0; 95% CI: for difference in medians 1.29–2.71; P

Figure 3 Comparison of post-intervention bicycle and dog safety skill scores between groups.

Safety discussions

The frequency of safety discussions between parents and children was also impacted by the intervention. At baseline, approximately 80% of parents in both groups reported rarely or never having bicycle safety discussions. Post-intervention, 53% of iBsafe parents reported rarely or never having bike safety discussions whereas control responses remained the same. For dog safety conversations pre-intervention, 83% of both groups reported rarely or never having discussions. Post-intervention, 33% of iBsafe parents reported rarely or never having dog safety discussions as compared to 67% of controls.

Acceptability

Child and parent acceptability of iBsafe was high; 87% of iBsafe children “very much” liked the game and two-thirds stated they would want to play it again. Seventy percent of iBsafe parents reported their child played the game at least five times during the trial. Most parents (83%) rated the game as “good” or “very good”. Nearly all parents felt it taught their child to be safe riding bikes (97%) and around dogs (93%). The majority endorsed the notion that iBsafe would help keep their child safe while riding bikes (77%) and around dogs (72%). One-third of parents were “very likely” to buy iBsafe or another mobile safety game app, while the remaining two-thirds stated they would be “somewhat likely” to do so.

Discussion

Child injury is a significant cause of morbidity and mortality in the US and globally. Though safety programs exist, there remains a disparity in the type and frequency of injury prevention education children receive. In our study of a highly educated population whose children have regular pediatricians and among whom there is near universal parental desire for their children to learn how to stay safe, formal safety education is lacking and children are experiencing preventable injuries. For example, most of our child population rode bikes and 25% had been injured while doing so, yet only 20% had ever received bicycle safety education. Similarly, 13% of children in our study had already suffered a dog bite and although 90% of them were around dogs regularly, only 5% had ever received dog safety education. Thus, there is considerable need for improved child safety education models that are both effective and have expansive reach.

Evidence-based, interactive mobile app games such as iBsafe could help solve this child safety education gap and potentially improve outcomes. Our findings show that iBsafe is effective at improving child safety knowledge and increasing safety skills. Specifically, children who played iBsafe had significantly higher bicycle and dog safety knowledge than controls, despite having similar or lower baseline knowledge in these respective areas. Since knowledge does not always translate into skills or behaviors and ethical evaluation of injury risk should not place children in harm’s way, our innovative testing procedures gave children the ability to engage with bicycles and dogs to demonstrate their skills without true risk. Based on blinded review of children’s skills and behaviors in this environment, children who played iBsafe executed significantly more bicycle and dog-related safety skills compared to controls, providing evidence for iBsafe’s potential to reduce injuries.

We also note that iBsafe dyads had near triple the frequency of safety conversations for both topics, while controls reported decreased frequency of bicycle safety conversations and slight increases in dog safety conversations. While it is possible that the increased attention iBsafe parents paid to discussing safety had an impact on children’s scores in our testing environment, the finding that iBsafe stimulates safety conversations in families reinforces its potential benefits. Increasing safety conversations can educate children, expand parental awareness of their child’s risk and possibly increase supervision, which is associated with less risk-taking behaviors and fewer child injuries overall (17).

This study supports that educational game apps such as iBsafe can leverage the already extensive use of mobile devices among children. At baseline, 100% of children in our study regularly used mobile devices with game play as their most frequent activity. Among children given the opportunity, iBsafe was extremely well accepted. The vast majority of children “liked playing the game very much” and most wanted to play it again. Interactive game apps that teach children evidence-based content have the potential to provide essential information and be an alternative to other non-educational mobile game play.

Although iBsafe content currently focuses on two of the top 10 injury mechanisms burdening US kindergarten-aged children, the implications of this study are more expansive. Mobile safety game apps are limited in availability, but they are desired and can reach massive markets. Irrespective of randomization group, more than 80% of parents believed that children can learn from mobile game apps and over 70% strongly endorsed the need for mobile game apps to teach children how to be safe. Furthermore, parents whose children played iBsafe were extremely positive with most rating iBsafe as “good” or “very good” and nearly all stating it taught their children and would help keep them safe.

Given the burden of injury in the US and worldwide, pediatricians and injury experts must adapt the approach to educating children and families about safety. The effects of media use in childhood is complex and multifactorial with ever-increasing evidence of risks and benefits, thus establishing healthy media use plans for families to include screen time limits is recommended for optimal child health and development (18). Mobile safety interventions should not replace conversations between providers and patients about keeping children safe, nor should they supersede the most important safety practice of parental supervision. However, with the lack of child safety education even amongst those who appear to have the best access, safety experts should consider working with mobile markets to develop and bring evidence-based, effective safety interventions to children’s fingertips.

This study had several limitations that temper findings. Although we demonstrated positive benefits of iBsafe game play, our study was not powered to assess for specific differences or to evaluate possible modifiers of the effect of game play on safety knowledge and skills. Our sample size was, however, well within recommended sample sizes for two-armed pilot trials to aid in determining future trial effect sizes (19). Our population was also relatively homogeneous and results may not be generalizable. Although child mobile device use in the US is near universal irrespective of age, ethnicity, parental educational attainment or socioeconomic status (20), it is unknown if iBsafe is effective in all populations. Our knowledge tests were developed from safety recommendations espoused by the AAP and while the question set has been previously used (6,21-23), the test has not been explicitly validated. Additionally, our simulated safety street and utilization of live therapy dogs has not been tested. This environment did, however, provide a safe environment in which to systematically evaluate children’s behaviors without real risk of injury. This methodology offers an exciting and innovate model of using live therapy dogs in the testing environment. While our results may not completely capture true knowledge or behaviors in a real environment, we contend that the changes observed in the iBsafe group reflects an increased awareness of safety knowledge as a result of the mobile app game play, and demonstrates the potential for direct translation to skills and injury prevention. Further assessment of iBsafe and its end-user data will help understand game play variance and impact on outcomes. Finally, whether knowledge and safety skills acquired translates into real life behavior and decreased injury rates remains unknown. We encourage future evaluation of iBsafe or other mobile safety game apps to determine effectiveness in reducing injuries.

Conclusions

This study demonstrates that iBsafe, a novel mobile safety game app, is effective at improving safety knowledge and skills among kindergarten-aged children, increases frequency of discussions about safety, and is extremely well accepted by children and parents. Safety game apps can leverage children’s ubiquitous use of mobile devices and bring safety education to those who might not otherwise be exposed. If systematically developed, tested and implemented, evidence-based mobile safety game apps have the potential to decrease child injury burden both in the US and globally.

Acknowledgements

The authors thank Christopher Bolling, MD and the pediatricians of Pediatric Associates of Northern Kentucky for their assistance in recruiting participants; Edith G. Markoff, PhD, Cincinnati Children’s Dog Visitation Program Coordinator, for helping coordinate study volunteers; and our three study volunteers and their amazing therapy dogs—Beckie Russo and Peach, Karen Renz and Chloe, and Martha Coen-Cummings and Star—for their time, enthusiasm and dedication to this project.

Funding: This work supported by the Cincinnati Children’s Hospital Medical Center Division of Emergency Medicine and in part by an Institutional Clinical and Translational Science Award, National Institutes of Health (NIH)/National Center for Research Resources (NCRR) Grant Number 8UL1-TR000077.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The local institutional review board approved the study (ID: 2013-3991). Enrollment occurred at the baseline visit during which informed written consent was obtained from all parent or legal guardian participants by study staff.

Disclaimer: Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.

References

1. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control (NCIPC). WISQARS Ten Leading Causes of Fatal Injury Reports (accessed 2018 Aug 29). Available online: http://webappa.cdc.gov/sasweb/ncipc/leadcaus10_us.html
2. Peden M, McGee K, Sharma G. The Injury Chart Book: A Graphical Overview of the Global Burden of Injuries. Geneva: World Health Organization 2002 (accessed 2018 Aug 29). Available online: http://apps.who.int/iris/bitstream/10665/42566/1/924156220X.pdf
3. Centers for Disease Control and Prevention, National Center for Injury Prevention and Control (NCIPC). WISQARS Ten Leading Causes of Nonfatal Injury Reports (accessed 2018 Aug 29). Available online: http://webappa.cdc.gov/sasweb/ncipc/nfilead2001.html
4. Centers for Disease Control and Prevention (CDC), National Center for Injury Prevention and Control (NCIPC). WISQARS Cost of Injury Reports (accessed 2018 Aug 29). Available online: https://www.cdc.gov/injury/wisqars/index.html
5. McLaughlin KA, Glang A. The effectiveness of a bicycle safety program for improving safety-related knowledge and behavior in young elementary students. J Pediatr Psychol 2010;35:343-53. [Crossref] [PubMed]
6. Dixon CA, Pomerantz WJ, Hart KW, et al. An evaluation of a dog bite prevention intervention in the pediatric emergency department. J Trauma Acute Care Surg 2013;75:S308-12. [Crossref] [PubMed]
7. Meints K, de Keuster T. Brief report: Don't kiss a sleeping dog: the first assessment of "the blue dog" bite prevention program. J Pediatr Psychol 2009;34:1084-90. [Crossref] [PubMed]
8. Schwebel DC, Morrongiello BA, Davis AL, et al. The Blue Dog: evaluation of an interactive software program to teach young children how to interact safely with dogs. J Pediatr Psychol 2012;37:272-81. [Crossref] [PubMed]
9. Lieberman DA. Management of chronic pediatric diseases with interactive health games: theory and research findings. J Ambul Care Manage 2001;24:26-38. [Crossref] [PubMed]
10. The World Bank. Mobile Cellular Subscription (accessed 2018 Aug 29). Available online: http://data.worldbank.org/indicator/IT.CEL.SETS/countries/1W?display=graph
11. Pew Research Center. Mobile Fact Sheet. 2017 (accessed 2018 Aug 29). Available online: http://www.pewinternet.org/fact-sheet/mobile/
12. Common Sense Media. Zero to Eight: Children's Media Use in America. 2011 (accessed 2018 Aug 29). Available online: https://www.commonsensemedia.org/research/zero-to-eight-childrens-media-use-in-america
13. Dixon CA, Ammerman RT, Dexheimer JW, et al. Development of iBsafe: A Collaborative, Theory-based Approach to Creating a Mobile Game Application for Child Safety. AMIA Annu Symp Proc 2014;2014:477-85. [PubMed]
14. American Academy of Pediatrics (AAP). Growing Up Healthy: Fat, Cholesterol, and More. 2012 (accessed 2018 Aug 29). Available online: https://patiented.solutions.aap.org/handout.aspx?gbosid=156463
15. Harris PA, Taylor R, Thielke R, et al. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42:377-81. [Crossref] [PubMed]
16. Johanson GA, Brooks GP. Initial Scale Development: Sample Size for Pilot Studies. Educ Psychol Meas 2010;70:394-400. [Crossref]
17. Morrongiello BA, House K. Measuring parent attributes and supervision behaviors relevant to child injury risk: examining the usefulness of questionnaire measures. Inj Prev 2004;10:114-8. [Crossref] [PubMed]
18. Chassiakos YR, Radesky J, Christakis D, et al. Council on communications and media. Pediatrics 2016;138:e20162593. [PubMed]
19. Whitehead AL, Julious SA, Cooper CL, et al. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res 2016;25:1057-73. [Crossref] [PubMed]
20. Kabali HK, Irigoyen MM, Nunez-Davis R, et al. Exposure and Use of Mobile Media Devices by Young Children. Pediatrics 2015;136:1044-50. [Crossref] [PubMed]
21. Dixon CA, Mahabee-Gittens EM, Hart KW, et al. Dog bite prevention: an assessment of child knowledge. J Pediatr 2012;160:337-41. [Crossref] [PubMed]
22. National Highway Traffic Safety Association (NHTSA). Cycling Skills Clinic (accessed 2018 Aug 29). Available online: https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/811260app.pdf
23. Institute BS. Toolkit for Helmet Promotion Programs (accessed 2018 Aug 29). Available online: https://www.helmets.org/