The role of mHealth intervention on maternal and child health service delivery: findings from a randomized controlled field trial in rural Ethiopia

Background

The provision of consistent and quality maternal and child health (MCH) services is a challenge for Ethiopia where most of the population lives in the rural setup and service delivery constrained by meager human, material, and financial resources (1). Maternal and under-five mortality rates remain highest among the world, at 420/100,000 and 64/1,000, respectively (1,2). The major contributory factor for the high mortality rates are the low rate of health service utilization of antenatal care (ANC), delivery services, and postnatal care (PNC) which are reported at only 41%, 16% and 18% respectively (2). Ethiopia has a total fertility rate of 4.1 children per women, only 3 in every 10 currently married women use contraceptive and immunization rates of children ages 12–23 months for measles are 80%, with immunization for BCG 86%, Penta1 89%, and Penta3 79% (3,4).

The government runs the Health Extension Program (HEP) which is implemented at community level by a national cadre of about 40,000 Health Extension Workers (HEWs) selected, trained, and deployed at the health posts to provide basic preventive and curative health services including MCH services to the rural community (5). Promotion of care seeking behavior, institutional delivery and preventive care are major roles to be played by them facilitated by supportive supervision of health center heads (5). However, health services access and use are mainly constrained by shortage of professionals and subsequent work load, low level of awareness among the society and robust bureaucratic work procedures, among others (6).

The application of information and communication technology (ICT) to improve health service delivery in rural areas with limited resource is increasingly becoming an important notion in global health (7). Studies revealed that mobile-phone based technology has a potential to serve a range of functions like data collection and sharing, overcome inadequacies of health infrastructure, improve access to information and services, improve quality of care and supply chain management (7-9). In recent years, this technology has emerged on application in the maternal, newborn and child health (MNCH) program but the available bodies of evidence on MCH arena are very limited.

A 2014 review of mHealth evidence identified only two mHealth RCTs that addressed pregnancy health outcomes (8,9) while a study in low income urban population reported an increased rate of influenza vaccination using text messaging intervention compared with the usual care (10,11). Similarly, a study in rural Western Kenya showed that mobile phone based strategies is potentially useful platforms to deliver messages about immunization schedule (12). Initiatives like the Mobile Alliance for Maternal Action (MAMA) are currently launching more thorough research agendas into this topic, which is anticipated to provide the national health leaders to have more evidence to base their healthcare policymaking (13).

Therefore, this RCT aimed to determine whether a locally developed mobile-phone short message service (SMS) based data exchange application influencing HEWs can improve maternal health services, contraceptives utilization rates, and immunization coverage compared to traditional approach.

Methods

Study settings

The study was conducted in three woredas of the Guraghe zone, Southern Nations, Nationalities and Peoples Region of Ethiopia (SNNPR), with intervention and control arms [2 intervention-(full and partial) and 1 control] having sufficient mobile phone network coverage selected randomly, by lottery method, among the twelve socioeconomically and topographically comparable woredas of the zone.

Study design

The study is a community-based randomized control trial (RCT) with intervention and control woredas. A pre and post intervention survey was used to assess the outcome of FrontlineSMS software loaded mobile phone intervention in improving selected MCH indicators.

The changes were assessed after 13 months of intervention (September 2012–October 2013), measured by differences of comparable endpoints between a control woreda and two intervention woredas. In an effort to avoid cross-contamination of the intervention, adjacent woredas were excluded from the random selection, as were woredas without adequate mobile phone network coverage.

Intervention

FrontlineSMS based application was locally developed, customized and integrated with the mobile phone system and the central server. A short-code system and general service mobile (GSM) modem was subscribed from Ethio-Telecom. A local physical network installed at the center for eHealth data center and connected with the Ethio-telecom end point.

Mobile phone distributed to the intervention woredas as depicted:

• In Ezha (Treatment 1, partial intervention), all HEWs [48] received a mobile phone with a customized software application (Table 1);

Table 1 Baseline Population profile, health facilities and community health workers at intervention and control woredas
Full table
• In Abeshge (Treatment 2, full intervention), all HEWs [49] received a phone loaded with FrontlineSMS application and two vCHWs within each kebele (56 in total) received a mobile phone without the customized software application (Table 1);
• In the control woreda (Sodo), no mobile phones were distributed.

Moreover, the health office heads, HC heads and local research assistants were received training focused on the overall usage of FrontlineSMS system.

The details of the intervention and their intended function were as follows:

• Improve ANC and delivery services—the software intended to improve ANC and delivery outcomes:
• HEWs in Ezha and Abeshge registered every mother who needed ANC care on electronic Maternal Registration Form by entering name, ID number, and either last date of most recent menstrual cycle or months of amenorrhea. Submitting this form to the central database automatically calculated and the Expected Date of Delivery and sent, antenatal care appointments scheduled and reminder messages were also sent to HEW at weeks 14, 24, 30 and 36;
• In Abeshge (vCHW responsibilities): vCHWs used the phone for missed-call HEWs when they meet or hear of pregnant mothers who did not register.
• Improve vaccination coverage—the software-enabled phone was intended to address gaps related to immunization services:
• HEWs filled and submitted a Child Registration Form by registering every child at or under the age of 11 months. Based on the birth date, vaccination appointments were scheduled for each child at 6, 10, 14 weeks and 9 months, and reminders were sent to HEWs phone;
• A week before the Woredas monthly vaccination program, HEWs sends SMS containing the names and IDs of children in their Kebele who needed vaccination in the upcoming vaccination day and how many vaccines needed for the subsequent month to her immediate supervisor;
• After each vaccination day, the HEW submitted an electronic vaccine report form indicating the number of children immunized for each vaccine;
• In Abeshge (vCHW responsibilities): vCHWs miss-call the HEWs to arrange for mothers who needed to attend health post for vaccination.
• Facilitate emergency referrals: in the third use case, the software-enabled phone was intended to address problems related to facilitating the management of emergency patients.
• HEWs call HC in emergency referral situations so that HCs are prepared to receive patient;
• In Treatment 2 (vCHW responsibilities): vCHW missed-call HEW when birth is going on and no HEW is present, or if there are other emergencies that needed HEWs attention
• Minimize contraceptive stock outs—the fourth use case used for stock out control:
• HEW received weekly SMS reminder to submit a Stock Report Form, submit containing stock data on the four contraceptive products;
• Health Center supervisor who received SMS containing reported stock levels and revise weekly delivery schedule.

The project provided an air-time that worth 50 ETB and 25 ETB per month to HEWs and vCHW respectively at the intervention woredas to facilitate their communication with their client as well as their co-workers. Besides the project covered cost of SMS communication on a monthly basis.

Sample size

The following assumptions were used to determine the sample size; probability of making α error (two sided) at 0.05, 90% power, and the ratio between the intervention and control areas (n3/n2/n1), to be 1:1:1, with a 10% upward adjustment for possible non-response.

Based on the assumptions, the sample size required was computed approximately to be 538 for each woreda, and a design effect of 2 was considered to account for the intra-woreda correlation.

A cluster sampling technique was used and 30 mothers per cluster for a total of 36 clusters/woredas were taken as our sample size in the selection scheme, thus a total of 1,080 mothers/woredas were determined to be included in the survey. Kebeles (clusters) were selected based on the probability proportional to size (PPS) method so that the probability of being included in the sample was proportional to the total number of people residing in each cluster.

The first household within each kebele was chosen by spinning a pen on the ground and to go forward for the first house following the direction in which it points upon stopping. Subsequent consecutive households were then selected randomly. When the selected household did not have an eligible mother for interview, then the nearest household was included instead. In case eligible respondents were not available at the time of the survey, a revisit (2×) was arranged.

Study end points

The value of the intervention was measured by comparing the difference in MCH proxy and service indicators/endpoints:

• % change in ANC attendance: timing and number of visits;
• % change in Clean and safe deliveries;
• % change in Deliveries attended by skilled provider;
• % change in Stock-out of contraceptives at the health post level;
• % change in Immunization coverage (TT2, Penta1, Penta3, Measles);
• % of women visited by an HEW during pregnancy.

Data collection and processing

A pre-tested, structured questionnaire developed in Amharic was used as a data collection instrument. The survey was conducted twice, once at base line in August, 2012 and again after 13-month of the intervention. Training was provided for a team of data collectors that included a district field supervisor, local supervisor and data collectors at each woreda.

Inclusion criteria considered for study participants were: for post intervention survey, women of age 15–49 years who have at least a child less than one year, and for the baseline survey, women of 15–49 years and who have a child less than 5 years.

Data were analyzed using SPSS v16 and Minitab v1.0.01. Descriptive analyses were conducted including testing for the presence of statistical significance of proportions comparisons between the intervention and control.

Ethical considerations

The study was ethically cleared by the Institutional Review Board of College of Health Sciences at AAU. Study permission was acquired at all levels of governmental administration. Informed consent was obtained from all participants before enrolling them as study participants. Confidentiality and anonymity of the information obtained was maintained at all levels of data handling.

Results

Response rates for each of the three woredas—Ezha (Treatment 1), Abeshge (Treatment 2), and Control (Sodo) were, 1,065 (98.6%), 1,073 (99.35%), and 1,080 (100%) at baseline, and 1,066 (98.7%), 946 (87.6%), and 1,077 (99.72%) after the intervention respectively.

Socio-demographic characteristics

Detailed demographic characteristics of respondents are displayed in Table 2. No major differences in the population profile were found among the studied woredas or across the two different survey phases.

Table 2 Socio-demographic characteristics of respondents (% of total respondents per woreda)
Full table

Intervention outcome on ANC service utilization

The proportion of ANC service utilization for the last pregnancy in Ezha, Abeshge and Sodo woreda was high at both baseline and after intervention (98.93%, 96.30%, and 86.94% and 98.50%, 98.76% and 93.10% respectively) (Table 3).

Table 3 Comparison of selected ANC utilization parameters at baseline and post intervention
Full table

Besides, HEWs ANC follow up at mothers home increased from 5.21% at baseline to 29.75% after thirteen months of intervention in Abeshge woreda. On the other hand it showed a decrease from 0.65% at baseline to 0.30% after intervention in Sodo woreda. The level of ANC visit time is depicted in Table 3.

The proportion of mothers receiving more than four ANC visits increased significantly in the intervention woredas (Ezha P=0.001, Z=17.96 and Abeshge P=0.001, Z=4.04), and showed insignificant decrease, in the control woreda (Table 4). Specifically, in Abeshge it increased from 15.8% at baseline to 31.5% and in Ezha it increased from 45.32% at baseline to 59.84% after intervention; where as in the control woreda of Sodo a decrease from 24.48% at baseline to 23.27% after intervention were registered (Figure 1).

Table 4 Mothers ANC visit—test and confidence interval for two sample proportions during & post intervention
Full table

Figure 1 Rate of completion of more than four times ANC visit at Baseline and post intervention. ANC, antenatal care.

Intervention outcome on delivery service

Table 5 show results of selected delivery and PNC utilization parameters at baseline and post intervention. Home delivery reduced in all three study woredas from 2012 to 2013. Home delivery in the intervention woredas, Abeshge and Ezha was 50.70% and 61.57% in 2012. This is reduced to 35.82% and 33.73% respectively after the intervention. Nevertheless, in the control woreda, it reduced from 72.80% to 58.46% in 2012 to 2013 respectively. The rate of home deliveries alone was found higher in the control woreda than in the intervention woredas (higher negative Z-values for comparisons of Ezha vs. Control and Abeshge vs. Control, Table 6). Deliveries conducted in the presence of an HEW in the Ezha woreda increased significantly, at a P value less than 0.050, whereas the control woreda experienced a statistically significant reduction in deliveries attended by an HEW, at a P value less than 0.001.

Table 5 Comparison of selected delivery and PNC utilization parameters at baseline and post intervention
Full table

Table 6 Mothers place of delivery; test and confidence interval for two sample proportions during & post intervention
Full table

More considerably referral services were observed from health post to health center by HEW in Ezha woreda (3.85 to 23.08). Referral of mothers for pregnancy and delivery complication by HEW in the other intervention woreda, abshege is reduced (19.51 to 15.65). The control woreda showed only 2% proportional increase.

Regarding delivery at the health center, Ezha woreda has shown more momentous increment from 23.44% at baseline to 55.07% after intervention. Health institution delivery was high in the intervention woredas compared to the control woreda. This variation has also statistical significance difference with P value <0.001(Table 6).

Health professional assisted deliveries at baseline were 26.79%, 41.96% and 21.79% while after intervention it increased to 55.23%, 63.54% and 52.05% for Ezha, Abeshge and Sodo respectively.

Intervention outcome on PNC service utilization

The timing of mother’s first PNC visit by community health worker was improved significantly in the intervention woredas. Although PNC timing in Abeshge woreda was more favorable at baseline than Ezha, women who receive their first PNC visit 1–2 weeks after birth still increased. Consequently, those who receive PNC in 3–4 weeks after birth diminished, and those who receive PNC more than 4 weeks after birth reduced to 0 percent (Table 5).

Family planning service utilization

Contraceptive prevalence rate (comparing mothers who have under 1 year age child) was found to be 97 (33.3%), 95 (44.8%) and 107 (37.81%) at baseline and 473 (45.83%), 300 (37.08%) and 481 (45.51%) for Ezha, Abeshge, and Sodo woreda respectively after the intervention. No statistical difference was noted before and after intervention and also among the study woredas (Table 7).

Table 7 Pre and post intervention age adjusted contraceptive prevalence rate and preferred contraceptive out of stock
Full table

Table 7 also shows the status of preferred contraceptive out of stock at health post level. It showed no change in the full intervention woredas, Abeshge and the control woreda, Sodo. However, a significant reduction from 16.96% to 8.24% was found at the baseline survey and after the intervention respectively.

Child immunization coverage

As shown in Figure 2, the full vaccination coverage was 88.63%, 76.19% and 81.82% at base line and 58.31%, 58.72% and 62.98% for Ezha, Abeshge and Sodo woreda respectively after intervention. This finding showed that the vaccination rates decreased across all three woredas and analysis of the cause revealed that mothers had several reasons for their child not being fully immunized, including being “too busy” as reported by 14.0 percent of surveyed mothers in Ezha, 4.6% in Abeshge, and 23.0% in Sodo. Further, vaccinators were sometimes absent at the immunization site as reported by 4.4% of surveyed mothers in Ezha, 8.6% in Abeshge, and 32.0% in Sodo.

Figure 2 Rate* of vaccination utilization, at baseline and post intervention. *, Comparing under 1 year of both survey/age adjusted utilization rate.

Discussion

This study measured the effect of SMS-based mobile phone application on selected MCH outcomes. It showed an increase in the proportion of mothers receiving more than four ANC visits in the intervention woreda (Table 3, Figure 1) unlike the control woreda. The proportion of ANC visits being conducted by HEWs at mother’s homes (preferred follow up method in rural set up) increased significantly in the intervention woreda.

Reduction in the instance of home deliveries, which is an important contributor to maternal death and, an increase in deliveries conducted in the presence of HEW were registered in the intervention woredas. Besides, an increase in referrals from health post to health center was observed across all woredas, although the increases in the two intervention areas were more pronounced (and absolutely higher) than the control group. The increase in referrals from health post to health center was statistically significant in Ezha woreda as a decrease was reported at P value less than 0.05 in the control woreda.

A decrease in contraceptive utilization rate was documented across all woredas and the intervention brought no effect on the contraceptive prevalence rate and management of contraceptive stock out. This decrease (instead of expected increase) is probably explained by the fact that the baseline survey included all mothers with at least one child under the age of five, and the post-intervention survey included only mothers with a child under the age of one (since our intervention was relevant only to women with a child under the age of one). Typically, women do not start up birth control again immediately after giving birth so that perceived decrease in the data was observed.

Since 2007, the World Health Organization has recommended that every pregnant woman receive at least four ANC visits during a healthy pregnancy in minimizing maternal mortality in developing countries, yet, in Ethiopia the UNFPA had reported that only 28 percent of women received at least one ANC visit, and only 12 percent received the full recommendation of four ANC visits (14,15). In agreement with this study, Lund et al. showed that in Zanzibar more women in the intervention group (44%) received the recommended four or more ANC visits than the control group (31%) and the odds for receiving four or more ANC visits were more than double for women benefiting from the mobile phone intervention (OR, 2.39; 95% CI, 1.03–5.55) (16).

In Ethiopia, obstetric complications including abortion, obstructed labor, sepsis, hemorrhage, and hypertension, which are directly related to the quality of delivery and PNC services, accounts for 85% of maternal deaths (17). Better health outcomes resulted from deliveries assisted by skilled health attendants and from deliveries that occur at health facilities (18). Our mobile phone application’s ability to send delivery “alerts” of pregnant mothers to HEWs, as well as the simple capacity to text and call vCHWs who operate at levels closer to households, lead to increased instances of safe deliveries and increased frequency of PNC service utilization. Although it is difficult to draw direct comparison, similar findings were reported by Lund in a study done in Zanzibar where 60% of women in the intervention group versus 47% in the control group delivered with skilled attendants (18).

Strong evidence were documented about mobile phones role in improving access to reproductive health services including dissemination of quality family planning information and services to different population segments and minimizing costs of accessing health care (7,19-22). Even though this study assumed that mobile phone application could serve as a stock management tool for contraceptives and thereby minimize contraceptive stock-outs, and could help to improve access to contraceptive utilization it had shown no impact on contraceptive prevalence rate and contraceptive stock out management.

Immunization is a proven tool for controlling and eliminating life-threatening infectious diseases (23,24). Various studies investigated factors directly associated with childhood immunization such as maternal education, exposure to media and maternal health care utilization (25-27). Mobile phone based SMS reminders were reported as a solution and potentially useful platform to deliver messages that prompts patients about immunization appointment days, improving clinic attendance, and had shown positive impact on increasing childhood immunization rates (11,12,28,29). However the current intervention did not impact on the level of child immunization rate.

Analysis of the reason why mothers do not fully immunized indicated several reasons, which included mothers being “too busy”, and “vaccinators were sometimes not available at the immunization site”. However, the most likely reason for the lack of impact of this intervention on the level of immunization rates is that there was a major change in the vaccination (contraceptives also) delivery process that occurred at health centers and woreda level across the country at the middle of the implementation of this intervention, which was neither foreseeable nor avoidable

Study limitations

We identified some limitations that affected the intervention’s ability to influence health service delivery outcomes.

First, there was a major change in the vaccination and contraceptive delivery processes across all health centers in the middle of the intervention. These significantly limited the capacity of the intervention to influence immunization and contraceptive utilization rates.

Secondly, there were also some problems with the mobile phones and mobile network. Recurrent failures of the mobile network for various reasons, including failure at the server end had potentially profound effects on the level of use. In addition, failure and loss of mobile apparatus by HEWs and absence of functional solar panel chargers that can keep phones consistently charged and useable meant that some Kebeles did not benefit from the intervention, which ultimately influenced the outcome of the pilot.

Thirdly, logistic problem encountered in availing timely project phones to the newly assigned health center heads and HEWs reduced the rate of utilization of the mobile system which significantly limited the intended benefits of the intervention. We believe this led to data not being submitted regularly and contributed to a level of frustration with the program that might lead some not to utilize it effectively.

Conclusions

The study showed that SMS based mobile telephone intervention could indeed improve the effectiveness of frontline HEWs in rural Ethiopia, primarily in the area of improving access to ANC, Delivery Services, and PNC. Number of ANC visit, percentage of delivery attended by health workers improved and it also facilitated the work process of frontline health workers. No significant impact was observed in the rate of contraceptive utilization and immunization coverage. The authors believe issues stated in the study limitations contributed to this luck of significant impact. In addition to the studies contribution to the general body of knowledge in mHealth application; the following recommendations can be drawn for future implementation.

Initiate systematic awareness program on the potential of mHealth on health for the concerned stake holders, specially based on the outcome of the current study and particularly on the technology used, the potential success and feasibility of the intervention, the impacts of the study results, and the experience learned from the current pilot intervention.

Based on the lesson learned from the current and other similar initiative scaling up mHealth use to other geographic areas and different health care application is recommended.

Acknowledgements

The work was funded by World Bank and African Development Bank in collaboration with Addis Ababa university.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethnical Statement: The study was ethically cleared by the Institutional Review Board of College of Health Sciences at AAU. Study permission was acquired at all levels of governmental administration. Informed consent was obtained from all participants before enrolling them as study participants.

References

1. Population growth and universal access to reproductive health. United Nations Department of Social and Economic Affairs Population Division; 2014. Population Facts No. 2014/6. Available online: http://www.un.org/en/development/desa/population/publications/pdf/popfacts/PopFacts_2014-6.pdf
2. Central Statistical Agency (CSA). Ethiopian Mini Demographic and Health Survey. Addis Ababa, Ethiopia; 2014. Available online: https://www.unicef.org/ethiopia/Mini_DHS_2014__Final_Report.pdf
3. The Last Ten Kilo Meters Project. Extended program on Immunization (EPI) coverage in selected Ethiopian zones: a baseline survey for Last 10 K’s routine immunization improvement initiatives. Addis Ababa: JSI Research & Training Institute, Inc.; 2015. Available online: http://l10k.jsi.com/Resources/Docs/EPI-baseline-report.pdf
4. Ethiopian Demographic and Health Survey. Addis Ababa: Central Statistical Agency; 2011. Available online: https://dhsprogram.com/pubs/pdf/PR10/PR10.pdf
5. FMOH (2010) Health Sector Development Program IV, 2010/11-2014/15. Addis Ababa: Federal Ministry of Health; 2010.
6. Kaba M, Atnafu A, Bisrat A, et al. Mobile phone technology: an opportunity for improved health services. The Ethiopian Journal of Health Development 2015;29:137-9.
7. McQueen S, Sarah K, Nicole P, et al. mHealth Compendium. Arlington, VA: African Strategies for Health project, Management Sciences for Health; 2012.
8. Jareethum R, Titapant V, Chantra T, et al. Satisfaction of healthy pregnant women receiving short message service via mobile phone for prenatal support: A randomized controlled trial. J Med Assoc Thai 2008;91:458-63. [PubMed]
9. Sharma R, Hebbal M, Ankola AV, et al. Mobile-phone text messaging (SMS) for providing oral health education to mothers of preschool children in Belgaum City. J Telemed Telecare 2011;17:432-6. [Crossref] [PubMed]
10. Higgs ES, Goldberg AB, Labrique AB, et al. Understanding the Role of mHealth and Other Media Interventions for Behavior Change to Enhance Child Survival and Development in Low- and Middle-Income Countries: An Evidence Review. J Health Commun 2014;19:164-89. [Crossref] [PubMed]
11. Stockwell MS, Kharbanda EO, Martinez RA, et al. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. JAMA 2012;307:1702-8. [Crossref] [PubMed]
12. Wakadha H, Chandir S, Were EV, et al. The feasibility of using mobile-phone based SMS reminders and conditional cash transfers to improve timely immunization in rural Kenya. Vaccine 2013;31:987-93. [Crossref] [PubMed]
13. Mobile Alliance for Maternal Action (MAMA). Mobile Alliance for Maternal Action “2015 Research Agenda”. Available online: http://www.unfoundation.org/what-we-do/issues/global-health/mobile-health-for-development/mama-research-agenda.pdf
14. World Health Organization (WHO). Maternal Mortality in 2005. Geneva: World Health Organization; 2007. Available online: http://www.who.int/whosis/mme_2005.pdf
15. United Nations Population Fund (UNFPA). State of the World Population, 2004: The Cairo consensus at Ten: Population, reproductive health and the global effort to end poverty. New York: UNFPA; 2004.
16. Lund S, Nielsen BB, Hemed M, et al. Mobile phones improve antenatal care attendance in Zanzibar: a cluster randomized controlled trial. BMC Pregnancy Childbirth 2014;14:29. [Crossref] [PubMed]
17. Federal Democratic Republic of Ethiopia, Ministry of Health (FMoH). Reduce Model: An advocacy tool for accelerated reduction of maternal and newborn morbidity and mortality in Ethiopia. Addis Ababa: Ministry of Health; 2003.
18. Lund S, Hemed M, Nielsen BB, et al. Mobile phones as a health communication tool to improve skilled attendance at delivery in Zanzibar: a cluster-randomised controlled trial. BJOG 2012;119:1256-64. [Crossref] [PubMed]
19. Foreman, M. The challenges of integrating family planning and maternal/child health services. Washington, D.C.: Population Reference Bureau; 2011.
20. Singh S, Darroch JE. Adding it Up: Costs and Benefits of Contraceptive Services-estimates for 2012. New York: Guttmacher Institute and United Nations Population Fund; 2012.
21. L’Engle KL, Vahdat HL, Ndakidemi E, et al. Evaluating feasibility, reach and potential impact of a text message family planning information service in Tanzania. Contraception 2013;87:251-6. [Crossref] [PubMed]
22. Sylla AH, Robinson ET, Raney L, et al. Qualitative Study of Health Information Needs, Flow, and Use in Senegal. J Health Commun 2012;17 Suppl 2:46-63. [Crossref] [PubMed]
23. World Health Organization (WHO). State of the World’s Vaccines and Immunization. 3rd edition. Geneva: World Health Organization; 2009.
24. World Health Organization (WHO). Millennium development goals: Immunization’s contribution to reaching the Millennium Development Goal on child survival. Geneva: World Health Organization; 2012.
25. Owino LO, Irimu G, Olenja J, et al. Factors influencing immunisation coverage in Mathare Valley, Nairobi. East Afr Med J 2009;86:323-9. [PubMed]
26. Toikilik S, Tuges G, Lagani J, et al. Are hard-to-reach populations being reached with immunization services? Findings from the 2005 Papua New Guinea national immunization coverage survey. Vaccine 2010;28:4673-9. [Crossref] [PubMed]
27. Bbaale E. Factors influencing childhood immunization in Uganda. J Health Popul Nutr 2013;31:118-29. [Crossref] [PubMed]
28. Szilagyi PG, Schaffer S, Shone L, et al. Reducing Geographic, Racial, and Ethnic Disparities in Childhood Immunization Rates by Using Reminder/Recall Interventions in Urban Primary Care Practices. Pediatrics 2002;110:e58. [Crossref] [PubMed]
29. Kaewkungwal J, Apidechkul T, Jandee K, et al. Application of Mobile Technology for Improving Expanded Program on Immunization Among Highland Minority and Stateless Populations in Northern Thailand Border. JMIR Mhealth Uhealth 2015;3:e4. [Crossref] [PubMed]