Understanding potential uptake of a proposed mHealth program to support caregiver home management of childhood illness in a resource-poor setting: a qualitative evaluation

Introduction

The global proliferation of mobile phones has contributed to the expansion of mobile health (mHealth), which incorporates mobile devices into the promotion and delivery of health services and information. Of the nearly 7 billion mobile phone subscribers worldwide, mobile phone use has experienced the greatest growth in low- and middle-income countries; mHealth, therefore, presents a novel opportunity to address significant barriers to health in marginalized and resource-limited populations (1-3). mHealth programs can be designed to extend the reach of a healthcare system by improving access to health information and promoting behavior change, especially among vulnerable groups. Tools have been developed for a variety of public health applications, such as treatment adherence, disease surveillance, emergency medical response, data collection, and point-of-care support (3).

The promise of mHealth in reaching vulnerable populations has garnered attention in the area of maternal and child health. While under-five child mortality rates have dropped dramatically since 1990, child survival remains a critical priority as an estimated 5.9 million children under the age of five died in 2015. Leading causes of death are largely preventable and could be avoided with broader deployment of existing prevention and treatment interventions (4). Therefore, mHealth can be leveraged to impact child health outcomes by influencing caregiver behaviors to improve delivery of interventions through education and enhanced access to health information (5). mHealth tools have thus far been deployed in this arena to improve vaccination coverage, enhance skilled delivery attendance, advance early disease detection, and encourage preventive behaviors, such as hand washing and insecticidal net use (6-8).

Although the field of mHealth has attracted rich inquiry in recent years, a more robust body of evidence is needed to support the implementation and scale-up of mHealth initiatives (9,10). Formative evaluations are sparse in current research, which is a prominent concern because implementation outcomes, such as feasibility, acceptability, and intervention reach, are rarely robustly explored (11-13). Access to mobile phones and feasibility of mobile phone use are highly important aspects to assess in the target audience prior to mHealth program implementation, alongside health needs of the community, health literacy and comprehension, and relevance of proposed program content (14). Further, there is a paucity of formative and implementation evaluations that utilize qualitative methodology, which can provide a more thorough understanding of cultural factors shaping the adoption and success of mHealth interventions (5).

The goal of the present study is to contribute a qualitative formative evaluation of a proposed mHealth program designed to support caregivers of young children in a resource-limited setting. We explore the factors that may support or hinder the adoption and use of an mHealth intervention in this community. A detailed analysis of the behavioral, affective, normative, and control beliefs that caregivers have when contemplating using an mHealth service can build our understanding of potential facilitators and barriers to participation in order to shape mHealth program development.

Methods

Study setting

Study participants were recruited from the Ciudad de Dios Centro de Salud (health center) service area in the Cono Norte region of Arequipa, Peru. Cono Norte is an informal settlement of approximately 30,000 people on the outskirts of Arequipa. Families in Cono Norte commonly lack access to stable housing, potable water, and environmental health services.

Study participants

We conducted 25 semi-structured interviews with a purposive sample of female caregivers. Women were eligible for the qualitative study if they had participated in a previous survey (n=220) about feasibility and acceptability of utilizing mobile phones to improve management of common childhood illnesses conducted by members of this team. Survey participants were recruited from a cluster-randomized block sample of households in the catchment area of Ciudad de Dios Centro de Salud in the Cerro Colorado District, and were eligible to participate if they were women over 18 years of age with at least one child under the age of 5 years, a regular resident in the household, and able to provide informed consent (15).

Procedures

Local field staff with considerable experience in household interviewing conducted elicitation interviews in Spanish with study participants in a private area of their home. A semi-structured interview guide, informed by the integrated health behavior model (16), was developed (see translation, Figure S1). Interviewers first asked open-ended questions about attitudes towards using mobile phones, specifically text messaging, for health-related purposes. Participants were asked to describe the type of content that would be beneficial to them in a proposed, hypothetical mHealth program for child health, and desired program features and functionality. Interviewers also inquired about who would participate in the program and for which community members it would be most useful. Mothers were asked about text messaging skills, perceived approval to participate in the mHealth program by peers, and any concerns they had about the initiative. A second interview was conducted with each participant to explore attitudes and beliefs around use of oral rehydration practices to prevent and treat dehydration during diarrheal episodes (results are not reported in this paper). Each interview lasted approximately 30 minutes and was audio recorded and transcribed by native Spanish-speaking members of the research team in Peru. Study participants were given a small basket of food items in appreciation for their time. The Institutional Review Boards of the University of Pennsylvania and the Universidad Peruana Cayetano Heredia approved this study.

Figure S1 Semi-structured elicitation interview guide exploring attitudes and beliefs around a proposed mHealth intervention to improve child health

Data analysis

Interview data were de-identified and imported into NVivo 10.0 software (QSR International, Melbourne, Australia) for management. To maintain integrity and preserve cultural nuances embedded in the transcripts, the research team analyzed the data in Spanish. Based on an initial close reading of the interview transcripts, a codebook was developed by the research team (TC, RF, AB). After a Spanish-speaking member of the team (TC) coded all transcripts, a subset of seven representative transcripts was selected and independently coded by a Spanish-speaking research assistant. Discrepancies in the coding were resolved by consensus and reviewed with members of the research team. Inter-coder reliability was evaluated using the Kappa statistic and review of percent agreement by code. The mean Kappa was 0.93, with a range of 0.61–1.0, indicating substantial to almost perfect agreement (17). This was supported by a percent agreement analysis that yielded 99.6% (range, 94.4–100%) agreement. We then summarized codes, examined relationships among codes, and organized them into emergent themes.

Results

Caregiver perspectives emerged along two main themes: (I) potential facilitators of participation in the proposed mHealth program; and (II) potential barriers to mHealth program uptake. Summative statements are followed by participant quotations and associated translations.

Potential facilitators to caregiver participation in proposed mHealth program

Benefits of two-way communication with healthcare providers

To make better decisions about the health of their children and families, the participants expressed interest and enthusiasm around the potential to engage in two-way communication with healthcare providers. One caregiver mentioned:

“…Envíame o pregúntamelo… prácticamente me convertiría en un hospital andante… […They send me (the information) or I will ask it… I would practically be converted into a walking hospital…]”

Another mother envisioned a scenario in which two-way communication would be useful. She described:

“Cuando mi bebe, por ejemplo, amanece con vómitos, en ese ratito yo puedo mandar mensaje en el sistema. (When my baby, for example, wakes up throwing up, in this moment, I can send a message through the system.)”

Caregivers indicated that the texted information would be most helpful if provided in a clear, direct manner. Participants were curious about the frequency with which they would receive messages, and more specifically, how quickly they would receive a response to child health inquiries. One mother said:

“…(supongamos que) le mando el mensaje, mando el mensaje al celular. ¿De inmediato me llega la respuesta? […(let’s say) I send them a message, I send the message on my cell. Does my response arrive immediately?]”

Although participants were told that the proposed mHealth program would function on the Short Message Service (SMS) platform, a few mothers suggested that the program additionally offer voice call features, such as an emergency hotline, as an alternative method of communication. One mother described:

“También sería bueno que nosotros llamemos y decir cómo nos pueden ayudar… también que ellos nos manden mensajes, nos manden mensajes de nutrición, cómo podemos ayudar debido a las enfermedades y todo eso. [It would also be good for us to call and to tell them how they can help us… also (it would be good) for them to send us messages, they send us messages about nutrition, how we can help the weak with illnesses and all of that.]”

One participant recommended that the program use unique access codes in order to prevent children and other individuals from unintentionally gaining entry into discussed personal health information.

Development of instrumental knowledge to care for sick children

Almost all caregivers expressed a strong desire for the proposed program to offer instrumental knowledge in the area of childhood disease management. When asked to describe knowledge gaps, many mothers identified needing to learn more about common illnesses, such as diarrheal disease, cold and flu, asthma, and malnutrition. One participant described:

“Cuando mi bebe no tiene apetito, yo debo de preguntar al doctor ¿por qué mi bebe no tiene apetito?, ¿qué le puedo dar?, entonces ya pues el doctor me dirá. (When my baby does not have an appetite, I need to ask the doctor, ‘why doesn’t my baby have an appetite? What can I give him?’ Then, the doctor will tell me.)”

Mothers specifically mentioned wanting more information on medications and therapies to use when children are sick. When asked what she would use the mHealth program for, one mother said:

“…para aprender… para saber del medicamento así, si es bueno, si es malo, qué cantidad; ahí también a veces para comunicarnos mejor, en casa no sabemos nada. […to learn… (we would use it) to know about medication, if it is good, if it is bad, how much to give; also at times (we would use it) to communicate better, in the case that we do not know anything.]”

They currently lack information on how much of a medication or therapy to give to a sick child and how to administer them. One caregiver stated:

“Me parece que es buena idea que nos den un poco más de orientación sobre las diarreas y cómo evitar… cuando uno está tan mal, qué puedo hacer, qué medicamento puedo tomar… porque cuando un niño está con diarrea, se deshidrata inmediatamente. [I think it is a good idea to give us a bit more guidance about diarrhea and how to prevent (it)… when one is very bad, what can I do, what medication can I give… because when a child is with diarrhea, he/she dehydrates immediately.]”

In addition, participants were interested in learning about adolescent and adult health. Specifically, participants mentioned a need for more information about diabetes, cardiovascular issues, kidney and liver problems, cancer, women’s health (pregnancy and menopause), emergency response and psychology.

Provision of support knowledge for childcare

Nearly all caregivers reported feeling distressed when their child is sick because they doubt their abilities to effectively manage the illness at home. Participants felt that the proposed mHealth program would be a helpful tool in providing critical information when needed. One mother stated:

“Es (el programa) una ayuda a la salud, es ayuda que nos ayuda qué sé yo a solucionar problemas de enfermedades o ayudar personas que tienen un accidente… [It (the program) is an aid for health, it is an aid that helps us know how to solve problems from illnesses or to help people who have an accident…]”

Caregivers felt that the proposed program would ease concerns about providing proper care and provide valuable support and guidance, especially in the case of an emergency.

“…lo sentiría pues como alguien en quien confiar, como un apoyo, sería un ahorra, un apoyo, un amigo, una amiga de confianza que puedes preguntar en cualquier momento, porque no siempre hay tiempo en el día, en la noche podría preguntar y yo sé que me van a responder. (…I am thinking it would be like someone to confide in, like a support, it would be a savior, a support, a friend, a friend with trust that you can ask a question to in any moment, because there is not always time in the day, in the night I could ask and I know that they would respond to me.)”

Among mothers who already access medical care, many felt that the proposed program could supplement limited encounters with the healthcare system. A few described these encounters as unfulfilling, and mothers often leave with unaddressed questions. One mother explained that her child had diarrhea, and despite having the opportunity to take him to a provider, she still had doubts about his recovery and questioned about how to manage his illness. She said:

“Mi hijo está con diarrea, qué puedo hacer? A veces llega uno a desesperarse. Ya le hice sus análisis y no sale bien qué tiene… le estoy dando sus medicamentos y no le hacen nada. Pero la doctora me dice que le siga dando, pero sigue igual. Ahí habría una duda. Entonces, le preguntaría eso, ¿qué puedo hacer?, ya ha recibido tratamiento, pero sigue mal, ¿qué es lo que se puede hacer? (What can I do, my son has diarrhea, what can I do? I go through desperation sometimes. They did analyses and they still do not know what he has. I am giving him his medication and it is not doing anything. The doctor told me to continue giving it to him, but it is still the same. So right there is a doubt. So, I would ask that, what can I do? He received the treatment, but he still feels bad, what is it that I can do?)”

Healthcare challenges in a resource-poor community

For families with few resources, it can be difficult to access health information and healthcare providers. One mother mentioned that she must go straight to the pharmacy when her child is sick because she is unable to afford consultation with a healthcare provider. Participants described that the proposed mHealth program would help families overcome barriers to healthcare, including travel time to the health clinic and difficulty obtaining appointments. One caregiver described:

“Porque yo mando un mensaje, no necesito ir al consultorio, el doctor me dice ‘sabes que señora, dele esto’… leo mi mensaje y el doctor me dijo así. [Because I send a message, I do not need to go to the (doctor’s) office, the doctor tells me ‘you know what, Ma’am, give him this’…I read my message and the doctor tells me like that.]”

Due to transportation barriers, many participants felt that families who live farthest away from the health clinic would be inclined to use the proposed program. Another caregiver explained:

“Vivimos tan lejos que no hay (como) llamar a una emergencia y el tiempo mismo que demora en venir, capaz que ya el enfermo está ya muerto. Más que todo en eso, en las informaciones que nos dan y de repente el apoyo que nos puedan brindar. [We live very far where there are no emergency calls, and perhaps by the time they (emergency personnel) arrive, the sick could be already dead. Overall with this, with the information that they give us, all of a sudden, we would have the support we need.]”

Others believed that mothers who work outside of the home would utilize the program because they do not have time to take children to the doctor. The proposed program was recognized by many participants as a way to save both time and money. One mother described:

“Gente como yo que trabajamos todo el día si podríamos usar, tenemos que usarlo, porque ya tienen a quién preguntar, ya no estás con esa duda ‘si le doy o no le doy un medicamente al bebe’ o ‘cuánto de medicamento tengo que darle,’ no se sabe, entonces cuando ya tienes a quién llamar o mandar mensaje de texto ya lo harías nomás. (People like me who work all day could use it, we have to use it, because they then already have someone to ask, you will not be with that doubt ‘do I give or not give the medication to the baby?’ or ‘how much medication do I have to give him/her?’ I don’t know, so when we have someone to call or to send text messages to, you will not hesitate to do it as they say.)”

A few mothers noted that there are some families in the area with more resources, and that because they have the means to take their children to the doctor, they would not necessarily need this program. While mothers were acknowledged as the ideal end-users of this program because they tend to spend the greatest amount time with their children, participants suggested that all community members participate in the program so that everyone is informed about health. Many mothers also felt that they would be empowered to share general health information and support with their extended family and community.

Potential barriers to caregiver participation in proposed mHealth program

Preference for in-person healthcare visits

A few participants suggested that visiting a healthcare provider in person is a unique experience. One mother said:

“En mi caso, sí lo usaría. Pero no es lo mismo que manden mensajes que un doctor lo vea. [In my case, I would use it (the program). But it is not the same to send text messages as it is for the doctor to see them.]”

Two caregivers mentioned that since they are accustomed to taking their children to the doctor when they are sick, they might utilize the program less than others. After the study survey was administered and prior to the elicitation interview, participants had the opportunity to discuss the program with family members and peers, some of which proposed the idea that in-person provider visits are better at times. Most of the mothers who described this concern said that their husbands, specifically, preferred that their children be taken to the doctor.

“Mi esposo siempre dice, ‘¿vas a hacerle caso al celular?, mejor llévalo a la posta o llévalo al doctor,’ es lo que diría mi esposo. [My husband always says, ‘are you are going to follow what they say via cellular (phone)?, it is better to take him to the clinic or to take him to the doctor,’ this is what my husband would say.]”

Even though preference for in-person provider visits over mobile communication emerged, mothers clarified that this would not be a major deterrent to their participation.

Program cost

Many mothers mentioned concerns about participation in the proposed mHealth program not being free of cost. Some participants were worried that the program cost would present a barrier to participation. One caregiver stated:

“Quisiera saber cuánto, si va a ser gratis o se va a cobrar. [I would like to know how much (the program will cost), if it (the program) is going to be free or if it is going to cost.]”

One mother was concerned about the potential costs of making phone calls. She explained that if she sends a text message and does not receive a response, she would have to call the service to obtain the information that she needs, and therefore, would need to purchase minutes that she cannot afford. Most participants who mentioned concerns about program cost said that family members and peers shared these uncertainties.

“Yo le comenté a mi hermana… ella también tiene sus hijos menores y me dice, ‘no creo, van a gastar mucho dinero mandando mensajes,’ esa es su respuesta. [I commented to my sister (about the program)… she also has young children and she said to me, ‘I do not think so, it is going to cost a lot of money sending (text) messages,’ this was her response.]”

Text messaging abilities

A few mothers admitted that they were not confident in their text messaging capabilities or commented on other mothers’ abilities. One caregiver admitted:

“El celular está bien, yo creo que es aprenderlo bien, pero hay señoras que no creo ni saben bien manipular el celular, ahí es el problema. (The cellular is useful, I think all you need is to learn how to use it, but there are women who do not think that or know how to use it well, this is the problem.)”

In particular, some mothers were concerned about not being able to write and send text messages. One mother said:

“Es que yo no sé escribir los mensajes, no sé mandar, yo quisiera que me enseñen eso… (It is that I do not know how to write the messages, I do not know how to send, I would like for them to teach me this...)”

Another caregiver explained:

“No sé manipular bien el celular para escribir o hay un celular que no hay para mandar mensajes… creo que habría que ver qué celular porque el mío no puedo, cuando escribo se van las letras, no sé. (I do not know how to use my cellular to write well or I have a cellular that is not able to send messages… I think I will need to look at my cellular because mine will not work, when I write, the letters go away, I don’t know.)”

However, almost all of these participants described ways to address these barriers. For example, one mother said that she was unable to write in general, but her daughter and husband knew how and could create a text message for her. Other participants explained that this program would encourage them to learn how to text message. A few suggested organizing workshops to teach mothers text messaging skills so that they could participate in the program.

Doubt surrounding program legitimacy

A main concern of the proposed mHealth program was legitimacy. With the majority of mothers enthusiastic about engaging in two-way communication with healthcare providers, some questioned the credentials of the individuals that would be responding to their inquiries. One mother asked:

“Esa es la más importante para mí… ¿Y quiénes son los que mandarían? ¿Doctores? [This is the most important to me… Who are those who will be sending (the messages)? Doctors?]”

Along these lines, a few participants were worried about the accuracy of the information that they would be provided.

“Me preocuparía que te manden falsos, que no es cierto, que es mentira. Parece que tantas personas pueden usar este programa para mandarte algo que no es cierto. [I would worry that they send me false (information), that it is not true, that it is a lie. I think that many people could use this program to send me something that is not true.]”

Some mothers wondered if they could be confident that the proposed program is not fraudulent. With this in mind, one participant wanted to know more about the true intentions of the proposed program and the overall purpose of the research conducted in this community. When asked if she had any doubts about the program, one mother explained:

“Sí, porque no conozco mucho, me gustaría saber dónde queda la oficina, me gustaría conocerlo a fondo [Yes, because I do not know much… I would like to know where the office is, I would like to become more familiar with the background (of the program)].”

Similar to other concerns mentioned, this doubt primarily stemmed from family members and peers. One mother said:

“Mis hermanos dicieron: ‘¿tú sabes con qué intenciones están viniendo o para qué te están sacando información?’ (My brothers said to me: ‘do you know with what intentions they are coming with or what do they want your information for?’)”

Nevertheless, the majority of mothers were still motivated to participate. Some said that providing their family members and peers with more information about the program could help change their views.

Discussion

While mHealth interventions have the potential to enhance access to health information and education, and ultimately improve health outcomes, a more comprehensive understanding of the factors that influence uptake of mHealth programs is needed. This study utilized qualitative methodology to explore beliefs and attitudes about a proposed mHealth intervention among caregivers in a resource-limited setting to further guide development of the proposed program. In previous work, we confirmed access to mobile technology, ability to utilize text messaging, and interest in communicating about health via mobile phones in this community (18).

Although results from our prior work in this community suggested that a one-way messaging program may ensure higher rates of caregiver participation due to limitations in writing and sending text messages (15), the present study found that caregivers were enthusiastic about the opportunity to engage in two-way communication with healthcare providers. This is consistent with existing literature that underscores the potential for two-way communication to engage the end-user and allow for targeted and tailored content (19). Therefore, it may be optimal to design the program with both one-way and two-way messaging functionality in order to allow those with skill limitations to participate and to serve those whose motivation to participate is driven by the opportunity for two-way communication. Lester et al. found that patients using an mHealth intervention designed to improve disease management were motivated to continue their involvement because communication from providers made them feel that someone cared about their health (20). Previous work also revealed that caregivers were interested in using mobile phones to share child health information and to seek help when caring for sick persons, but the qualitative analysis demonstrated that mothers had a specific need to learn more about childhood disease management and to feel supported in delivering home-based treatments (15). Although some mothers in this community reported having access to healthcare, many of these reports were fraught with frustration and confusion; therefore, this intervention may not only support caregivers in making more informed decisions about their children’s health, but also bolster dissatisfactory experiences with healthcare providers. Caregivers also described that learning child health information in both tailored and general formats would make them feel empowered and would cause them to want to disseminate information throughout their communities. Skinner et al. detected a similar sentiment among caregivers of HIV patients in South Africa who were already using an mHealth tool to facilitate communication with doctors and nurses regarding treatment adherence and patient health status (21). This evaluation confirmed that participants felt the mHealth program would be a beneficial tool, especially among those with the most limited access to resources.

Caregivers provided insights about concerns that may hinder their willingness to participate in the program, and therefore, must be considered during development and implementation in order to ensure high rates of participation. Although some mothers were concerned that this program could not substitute for in-person healthcare visits, the proposed program is not intended to do so. However, it may be important to communicate to caregivers that even if they are able to access the healthcare system, the program could still supplement their experiences and provide an alternative channel for health communication at the times they are unable to see a healthcare provider. Program cost was a major concern for most of the participants and findings suggest that this factor could deter caregivers from participation, which has been previously identified in the literature (18). Doubts surrounding program legitimacy among caregivers can be addressed through enhanced communication on the purpose of the program, transparency throughout the development process, and provision of education on the program when it is ready for deployment. Although participants reported sharing phones with family members, privacy was not discussed as major concern, which is a varied finding in existing research. A formative evaluation of an mHealth intervention to support post-abortion family planning in Cambodia did not find evidence of privacy concerns, while the importance of privacy and confidentiality was emphasized in a HIV medication adherence trial in Kenya (20,22). Shet et al. also reported a high proportion of mobile phone sharing among their target population, but did not find significant privacy concerns around receiving reminders for antiretroviral therapy adherence (23). Other significant barriers to mHealth intervention delivery in resource-limited populations, such as language variability and consistently changing phone numbers, identified in other formative research, were not detected in this study (24).

This study has several limitations. The peri-urban study population may not be representative of other resource-poor settings in Peru or in the developing world, or of communities in either urban or rural areas. However, Cono Norte is similar to other Peruvian peri-urban communities in that individuals who settle on the outskirts tend to be more disadvantaged than those living closer to the urban center, and therefore, generalizing these findings to other peri-urban areas may be appropriate. Mothers who agreed to participate in the interview may not be representative of the community, and their beliefs and attitudes may not predict actual behaviors. Mothers were introduced to a hypothetical mHealth program, and therefore, their perceptions of this hypothetical intervention may be different than if they had been able to actually test the program.

There is a crucial need to develop novel health education and promotion interventions that have the capacity to reach marginalized and resource-limited populations. Although the potential of mHealth has been recognized, and the literature reflects an increasing consideration of the end-user population and factors that make mHealth intervention rollout feasible, our understanding of the factors that facilitate and challenge the adoption of mHealth programs is limited. Since the success of mHealth programs hinges on the ability to design an intervention with a user-focused approach and within the context of the target population, this formative evaluation is a necessary component of program development. This study established that a critical need for improved access to child health information exists in this community and underscored that caregivers may greatly benefit from provision of instrumental and support knowledge to care for children with common illnesses. These findings coupled with evidence of widespread access to mobile phones, feasibility of use, and interest in using mobile phones to communicate about child health in this community will shape development of an mHealth program ultimately aimed at improving child health outcomes.

Acknowledgements

The authors thank Lina Margot Mollesaca Riveros, Amparo Marisol Toledo, Valerie Paz-Soldan, Matias Fernandez-Duque, and Michael Z. Levy for assistance with this study.

This work was supported by the Penn-Peru Pilot Grant Program of the University of Pennsylvania and the Universidad Peruana Cayetano Heredia.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The Institutional Review Boards of the University of Pennsylvania (No. 819463) and the Universidad Peruana Cayetano Heredia (No. 59473) approved this study. Informed consent was obtained from all participants.

References

1. International Telecommunication Union. ICT: Facts and Figures 2014. Available online: http://www.itu.int/en/ITU-D/Statistics/Documents/facts/ICTFactsFigures2014-e.pdf
2. Kahn JG, Yang JS, Kahn JS. “Mobile” health needs and opportunities in developing countries. Health Aff (Millwood) 2010;29:252-8. [Crossref]
3. Vital Wave Consulting. mHealth for development: The opportunity of mobile technology for healthcare in the developing world. Washington, DC & Berkshire, UK: UN Foundation-Vodafone Foundation Partnership; 2009: 1-66.
4. UNICEF. Committing to child survival: A promise renewed, 2015. Available online: http://www.unicef.org/wcaro/2009_2901.html
5. Higgs ES, Goldberg AB, Labrique AB, et al. Understanding the role of mHealth and other media interventions for behavior change to enhance child survival and development in low- and middle-income countries: An evidence review. J Health Commun 2014;19:164-89. [Crossref] [PubMed]
6. Domek GJ, Contreras-Roldan IL, O’Leary ST, et al. SMS text message reminders to improve infant vaccination coverage in Guatemala: A pilot randomized controlled trial. Vaccine 2016;34:2437-43. [Crossref] [PubMed]
7. Lund S, Hemed M, Nielsen BB, et al. Mobile phones as a health communication tool to improve skilled attendance at delivery in Zanzibar: A cluster-randomised controlled trial. BJOG 2012;119:1256-64. [Crossref] [PubMed]
8. Seidenberg P, Nicholson S, Schaefer M, et al. Early infant diagnosis of HIV infection in Zambia through mobile phone texting of blood test results. Bull World Health Organ 2012;90:348-56. [Crossref] [PubMed]
9. Lee SH, Nurmatov UB, Nwaru BI, et al. Effectiveness of mHealth interventions for maternal, newborn and child health in low- and middle-income countries: Systematic review and meta-analysis. J Glob Health 2016;6:010401. [Crossref] [PubMed]
10. Modi D, Gopalan R, Shah S, et al. Development and formative evaluation of an innovative mHealth intervention for improving coverage of community-based maternal, newborn and child health services in rural areas of India. Glob Health Action 2015;8:26769. [Crossref]
11. Mechael P, Batavia H, Kaonga N, et al. Barriers and gaps affecting mHealth in low and middle income countries: Policy white paper. Center for Global Health and Economic Development, Earth Institute, Columbia University; 2010.
12. Tomlinson M, Rotheram-Borus MJ, Swartz L, et al. Scaling up mHealth: where is the evidence? PLoS Med 2013;10:e1001382. [Crossref] [PubMed]
13. Whittaker R, Merry S, Dorey E, et al. A development and evaluation process for mHealth interventions: Examples from New Zealand. J Health Commun 2012;17 Suppl 1:11-21. [Crossref] [PubMed]
14. Marshall C, Lewis D, Whittaker M. mHealth technologies in developing countries: A feasibility assessment and a proposed framework. Herston, Australia: University of Queensland; 2013:1-47. (Health Information Systems Knowledge Hub Working Paper Series).
15. Calderón T, Martin H, Volpicelli K, et al. Formative evaluation of a proposed mHealth program for childhood illness management in a resource-limited setting in Peru. Rev Panam Salud Publica 2015;38:144-51. [PubMed]
16. Montano DE, Kasprzyk D. Theory of Reasoned Action, Theory of Planned Behavior, and the Integrated Behavioral Model. In: Glanz K, Rimer BK, Viswanath K. editors. Health Behavior and Health Education. Available online: http://www.med.upenn.edu/hbhe4/part2-ch4-integrated-behavior-model.shtml
17. Viera AJ, Garrett JM. Understanding interobserver agreement: The kappa statistic. Fam Med 2005;37:360-3. [PubMed]
18. Kaplan WA. Can the ubiquitous power of mobile phones be used to improve health outcomes in developing countries? Global Health 2006;2:9. [Crossref] [PubMed]
19. Gurman TA, Rubin SE, Roess AA. Effectiveness of mHealth behavior change communication interventions in developing countries: A systematic review of the literature. J Health Commun 2012;17 Suppl 1:82-104. [Crossref] [PubMed]
20. Lester RT, Ritvo P, Mills EJ, et al. Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): A randomised trial. Lancet 2010;376:1838-45. [Crossref] [PubMed]
21. Skinner D, Rivette U, Bloomberg C. Evaluation of use of cellphones to aid compliance with drug therapy for HIV patients. AIDS Care 2007;19:605-7. [Crossref] [PubMed]
22. Smith C, Vannak U, Sokey L, et al. Mobile Technology for Improved Family Planning (MOTIF): The development of a mobile phone-based (mHealth) intervention to support post-abortion family planning (PAFP) in Cambodia. Reprod Health 2016;13:1. [Crossref] [PubMed]
23. Shet A, Arumugam K, Rodrigues R, et al. Designing a Mobile Phone-Based Intervention to Promote Adherence to Antiretroviral Therapy in South India. AIDS Behav 2010;14:716-20. [Crossref] [PubMed]
24. Otieno G, Githinji S, Jones C, et al. The feasibility, patterns of use and acceptability of using mobile phone text-messaging to improve treatment adherence and post-treatment review of children with uncomplicated malaria in western Kenya. Malar J 2014;13:44. [Crossref] [PubMed]