ABC Epatite Web App: raising health awareness in a mobile world

Introduction

Increasingly the internet has become a common goto for health-related information. As lockdowns have been implemented for the COVID-19 pandemic, internet use has grown and become more sophisticated reaching a far greater share of the populations involved than ever before. However, this has highlighted the peculiar aspects inherent in this environment; the resulting infodemic has brought to the forefront the need to improve health literacy globally (1).

Low health literacy levels have been shown to lead to increased health risks (2-5) therefore numerous countries have set out to measure and improve literacy in this sector (e.g., the M-POHL; https://m-pohl.net/). Awareness campaigns, especially those conducted online, have proven to be particularly effective in better informing people on health in general as well as on specific diseases (6). The role of digital technologies will therefore become ever more central (7) as has been recommended by the WHO health system guidelines (8). This change furnishes the opportunity of continuously refining messages and creating visual arguments in an adaptive, iterative fashion, enabling one to tailor content according to user competences (9).

The fruition of online content, however, has overwhelmingly shifted from the original desktop technology to a mobile one. At no time in history has a technological innovation had such a rapid uptake as that of the smartphone. Currently there are over 5 billion unique smartphone users (https://datareportal.com/reports/digital-2019-global-digital-overview) up from near negligible in the first stages a decade ago. This has occurred at a global level encompassing people with incredibly diverse economic possibilities from the rich to the poor. This has created an opportunity for reaching out to people in a far more efficient and economically feasible way than before, while increasing the risk of a widening digital inequality gap (10).

Originally conceived in April 2018 as a handy method for online code documentation, Vuepress (https://vuepress.vuejs.org/) has quickly grown to be a popular open source way to create modern static websites with Progressive Web App (PWA) capability (https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps). These PWAs are JavaScript applications reliant on HTML and CSS for the visual component aspects and typically distributed by website URLs similarly to web pages. Typically, these apps enable offline functionalities in modern browsers and can be installed on devices using the add to homescreen (https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Add_to_home_screen) functionality in modern browsers, so as to give users a native app like experience. Essentially Vuepress provides simple Wordpress functionalities within a Vuejs JavaScript framework, in which content is written in markdown. Markdown facilitates the process of enriching content with markup by using a simple and quite natural syntax. Vuejs is a popular JavaScript framework, which powers Vuepress’s functionalities. The Vuepress development tools are cross platform relying on a Node.js (https://nodejs.org/en/) environment and are assembled from modular npm packages. Once installed, the development process can be mostly confined to editing configuration and content page files. An integrated local web server enables real time editing and testing. The configuration file enables one to define the navigational menu structure of the app in a hierarchical manner. In the build process a distribution package with minified JavaScript and necessary supporting HTML and CSS are created. This package is then uploaded to a hosting network service e.g., Content Distribution Network (CDN, https://en.wikipedia.org/wiki/Content_delivery_network) or a web server. Once a new version of the app is uploaded, users on the app are notified with a pop-up prompting them to update the app so as to access the new content; this update is immediate as the new content has already been downloaded before the prompt is shown.

Raising awareness on health and wellness related topics is particularly well suited to the smartphone platform (11,12); not only are they ubiquitous but they are indeed far more personal than personal computers (PC) as people tend to develop an intimate bond with their phones (13,14). Due to the relatively small visual space available in smartphones, it is possible to determine the degree of engagement that each single user has with the content: the amount of time spent on each specific section, the continuity with which it is accessed and the route followed can be analyzed in an unobtrusive way (15). User engagement metrics are fundamental tools in assessing the quality and degree of effectiveness of an awareness campaign (16-18), enabling one to update content to reflect the actual user behavior in real-world conditions. Dynamically tailoring content to specific user categories so that appropriate levels of detail may be delivered according to specific groups is an important advantage of this technology. This approach also allows in-field testing of alternative formulations of similar content targeting randomized samples of users so as to compare the effectiveness of the presentation of the material: this is generally referred to as A/B testing.

A specific challenge that should be tackled is message comprehension: high user engagement does not, by itself, imply that the information is correctly understood. This problem is obviously not confined to mobile awareness campaigns. Comprehension can be assessed by a number of means such as quizzes, achieving desired goals or behavioral changes, etc. An awareness app might leverage the playful aspect of self-testing to enhance user engagement as users are often well disposed to challenging themselves with quizzes or games, if these are sensed to be satisfying.

User attraction is also an area of major importance in awareness actions. This essentially comes down to determining the primary target audience and finding a cost effective method of getting them on-board.

This paper presents the case study of a health campaign for mobile devices launched in Italy in 2019 aimed at raising viral hepatitis awareness. The research project “ABC epatite. Sviluppo di una app per la prevenzione delle epatiti virali e per la consapevolezza dei comportamenti a rischio”, winner of the 2018 Digital Health Program of Gilead Science Italia, was carried out by the National Research Council. The project entailed the development of a free Italian language PWA for providing current and scientifically validated information on infections caused by hepatitis A, B and C viral strains (HAV, HBV, HCV), especially promoting behavioral best-practices, such as what high-risk activities should be avoided, and furnish skills to users so as to help them navigate the fake news infodemic. The app is aimed at the general public with a particular focus on more vulnerable people and youngsters. A specific landing page for schools has been implemented with a simplified tour of the content so that a tailored outreach program can be effectively achieved for children as they represent a fundamental target of this awareness action.

We present the following article in accordance with the MDAR reporting checklist (available at http://dx.doi.org/10.21037/mhealth-20-158).

Methods

Background

The World Health Organization (WHO) has set a goal of eradicating viral hepatitis by 2030; the fulcrum of this strategy relies on achieving a target of reducing new infections by 90% and deaths by 65% between 2016 and 2030 (https://www.who.int/hepatitis/publications/hepatitis-service-coverage-targets/en/; https://www.who.int/hepatitis/news-events/7-steps-to-hepatitis-elimination/en/). Globally, over 250 million people have chronic hepatitis B, and more than 70 million have chronic hepatitis C; these two viruses cause serious liver damage, hepatic cirrhosis and cancer resulting in immense socio-economic damage throughout the world. WHO estimates at 1.34 million yearly deaths caused by HBV and HCV place these just below tuberculosis (1.4 million) and above HIV (1.1 million) in the sad rank of mortality by infection. An important aspect is that while tuberculosis, HIV and malaria have peaked and are now on the decline, hepatitis deaths have increased by over 20% from year 2000 (https://www.who.int/publications/10-year-review/hepatitis/en/). According to a WHO study, over 4 million premature deaths might be prevented by 2030 through the combined actions of vaccinations, testing, medicines and educational campaigns (https://www.who.int/health-topics/hepatitis/viral-hepatitis-in-the-world-map#tab=tab_1).

The ABC Epatite project directed at Italian speakers draws upon the guidelines and information provided by the Italian Ministry of Health and the Istituto Superiore di Sanità (the Italian National Institute of Health). Promoting awareness at a national level is considered to be an important step in eradicating viral hepatitis according to the “Piano nazionale per la prevenzione delle epatiti virali da virus B e C” (the Italian national plan for the prevention of viral hepatitis B and C, http://www.salute.gov.it/portale/documentazione/p6_2_2_1.jsp?lingua=italiano&id=2437). Targeting the public in general and particularly younger age groups for awareness raising actions is recognized to be a priority (https://www.cittadinanzattiva.it/files/rapporti/salute/epatite-c-siamo-report-2016.pdf).

In recent years, there has been a general shift towards using the internet as a primary source of information, this being particularly true in youngsters. The percentage of the total population that uses the Internet (the so-called Internet penetration) in Italy is 92% (in 2019), and growing at a considerable rate (+27% in 2019). The amount of time per day spent on the Internet via mobile devices is constantly increasing worldwide year by year; health & fitness mobile apps appear as a category in the top 10 consumer spending rankings on both Google Play and Apple Store (https://datareportal.com/reports/digital-2019-global-digital-overview). Therefore, the mobile first approach shows great promise both in Italy and globally.

Study design

From conception, this project has relied on a metaphorical expression—“ABC”—to emphasize an immediate educational intent underlying the presentation of the principal A, B and C forms of the Hepatitis Virus central to the project message. Furthermore, the web app’s domain: epatite.web.app is effective in declaring use and purpose.

While in the industry a number guidelines have emerged (e.g., Google’s Material Design guidelines, https://material.io/design/guidelines-overview), little research has been conducted to define the specific elements that constitute effective designs in general (i.e., 19). Design aspects that facilitate and predict user engagement have been proposed by Garett et al. (20), on the basis of a literature review of relevant studies on the topic. Aspects including navigation, graphics, organization, utility, purpose, simplicity, and readability have been a principal concern in the design study of this project. On the basis of these concepts, the design study for “ABC Epatite” has evolved around achieving three major aims: being (I) user-centric; (II) trustworthy; (III) and pursuing an ethical approach in all research phases.

Two major audience groups were identified as being students and adults, leading to a user design model catering to these user types. Specific cognitive architectures were implemented by designing visual arguments and presentations according to the defining characteristics of each group.

User-centric approach

A user-centric approach has been central in the organization of the logical and graphical elements of the app (21-23). Starting from the main app theme a teal primary color was chosen as would be expected from a reliable informational source (24). From a set of proposals, a selection was made which resulted in a modern simple flat design logo heavily weighted to the ABC message. This logo is given prominence in the landing pages in order to reinforce the user’s association with the app; fundamental as PWA are often added to a smartphone’s home screen where icon logos are the principal means of accessing the app. Furthermore, in line with industry guidelines on user retention, app loading time was reduced in order to achieve a less than 3 second time to first meaningful render.

Basic graphical elements draw attention to essential core ideas, e.g., by using the concept of “cards”. These are analogues of physical paper cards providing a rich tactile user interface by a play of shadows. Nuanced color decoration added to cards aids users in perceiving the level of risk associated with the information contained in a card; a typical green-amber-red color scheme being used to signal positive-warning-danger message highlights. Central aspects of each section were emphasized by introducing images in order to produce a visual note on the relevant key content aspects.

Typography has an important role in convening the tone of the delivered message; whereas serif fonts tend to have a formal note, sans-serif ones tend to feel more relatable (25). Furthermore, on the smaller screens of mobile devices, they are more readable. For this reason, a sans-serif family was chosen (e.g., Roboto on Android).

In order to ensure inclusivity, so that content is accessible to all, including people using assistive technology (e.g., screen readers), the app uses HTML Accessible Rich Internet Applications (ARIA) attributes (https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) and a web tool (Lighthouse, https://developers.google.com/web/tools/lighthouse) was used to generate an audit report verifying the app’s excellent score on this aspect.

In order to differentiate between adult and student groups, specific landing pages were introduced: one clearly catering to school children, the other to adults. The school specific route leads users through a quite structured experience, as might be expected in classroom environments with content being presented in an appropriate direct fashion. Specifically, for primary schools (ages 6–11) the landing page specifies that the activity should be done in the classroom or with the children’s family, while for secondary schools (ages 11–19) browsing the app autonomously is also feasible. Although in principle it would be possible to determine age-related data for each individual user, we choose not to do so for the scope of this project.

Navigational flexibility was reduced to a sequence of 5 clearly set steps. This enables students to have a global awareness not only on how much content will be provided but also on how much of the material has been covered at any given point, improving concentration. On the other hand, adults entering from the main landing page have at their disposal a full set of navigational routes, linear from page to page, hierarchical by menu, indirect by search.

The chosen editorial line was to prepare material in conformance with the educational and research nature of the project, while at the same time, emphasizing simplicity, readability, and learnability in the argumentative strategies. In line with recent studies, the content was developed around the following main concepts: organizational coherence, lexical accuracy, accountability, reliability and audience respect (26).

The discursive strategy followed a structure in which a set of questions was addressed for each specific hepatitis viral infection (A, B, C). The pivotal aspects of each revolved around the following eight questions: (I) what is it; (II) how is it contracted; (III) what are the symptoms; (IV) what are the associated risks; (V) how is it diagnosed; (VI) what are the therapies; (VII) what prevention measures can be undertaken; (VIII) how widespread is it both in Italy and globally.

Trustworthiness

In any serious awareness campaign, it is essential to ensure that the information presented is scientifically accurate, based on trustworthy material and to be as up-to-date as possible. Taking into account that currently many users are overwhelmed by an information overload (27) and subjected to online misinformation (28,29), ensuring content quality is of paramount importance. In order to achieve this, a consistent effort was made to develop quality content based on selected bibliographic references as well as providing users with reputable hyperlinking material for further reading.

A Committee of Medical Experts, external to the project, was established to correct and validate the content. A second external working group composed of non-expert volunteers was invited to assess the content over a set of aspects including proofreading, comprehension, and overall user experience.

Another aspect that was taken into account was that the information provided had to be practical and relevant to the context of each target user group, so as to promote responsible behavior. To balance out the emotional impact of the app, appropriate colors were chosen to convey the nature of the information. Furthermore, the use of an alarmist language tone was avoided. For example the original text “In casi rari l’epatite A può evolvere in epatite fulminante, con un alto tasso di mortalità. La mortalità nella popolazione generale è di circa 1 caso su mille ma sale fino a 18 casi su mille nei soggetti di età superiore a 50 anni” (in rare cases hepatitis A can evolve into it’s fulminant version which has a high mortality rate. Mortality in general population is 1 in 1,000 but rises to 18 in 1,000 for subjects over 50) was simplified and toned down to “In casi rari l’epatite A può evolvere in epatite fulminante e pericolosa” (in rare cases hepatitis A can evolve into it’s dangerous fulminant version). Essentially changing “lethal” to “dangerous” and removing statistical details specific to a rare condition.

A specific section dedicated to “Fake News” was provided, setting straight a series of untrue prejudices often found on the Internet. This may lead to a more inclusive approach reducing the burden of discrimination and social exclusion of those affected by these diseases.

Ethics

The Web App is thought out to empower the general public on developing a critical health skill set specifically in regard to viral hepatitis. By engaging people in a personal way using mobile technology, it strives to increase the field of open science initiatives in a concrete fashion as promoted by the European Commission (https://ec.europa.eu/research/openscience/pdf/ec_rtd_ospp-final-report.pdf#view=fit&pagemode=none).

An important aspect for evaluating the effectiveness of the content and implemented navigation is to analyze the flow that users follow in the app. Google Analytics (https://support.google.com/analytics/answer/6004245?hl=it) is a GDPR compliant service for this purpose and has the advantage of being capable of reporting user behavior in offline mode as well, a fundamental feature in PWAs as these are often used in this mode. In line with the GDPR [General Data Protection Regulation (EU) 2016/679; https://eur-lex.europa.eu/eli/reg/2016/679/oj] no personal data is collected and no third party cookies are installed while first part technical cookies are installed both for contributing to a positive user experience and for collecting statistical data. This analytics service (with anonymized IP addresses) was used to analyze the quantitative data acquired both to improve the app and for research purposes. The Content Delivery Network (CDN) system used is Firebase Hosting (https://firebase.google.com/support/privacy), which does however store IP addresses for security and is fully GDPR compliant. The data policy is directly available to users on the app detailing the nature of the data collection process and research aspects of the project.

Results

Implementing the App

A mobile first PWA health awareness app was implemented and made available through a link (https://epatite.web.app) together with an Android app version, which is available on Google Play. Diversified landing pages were implemented so as to cater to two target audiences: general public and schools. This furnishes each user group with a tailored experience most suitable to their characteristics. The overall aim is to cover the fundamental aspects regarding the main hepatitis viral strains and tying these to three Italian mnemonic concepts (A: Alimenti, B: Bio-liquidi, C: Contatto), so as to provide a quick reminder connecting these three principal viruses to their main vector: food, body fluids and blood-blood contact.

Raising awareness on health issues related to the general population requires an effective means of targeting a suitably interested audience. The approach taken by this project is to leverage on the school environment who may reach out to families and friends thus resulting in a multiplying effect. A special navigation path was set up in the app with content suitable for all ages. The school landing page sets out a simplified scholastic route for all, so that older children and families may navigate the main content together; there is, however, further material suitable for older age groups from middle school on. An initial campaign directed toward schools in nine Italian regions was conducted.

In Figure 1, some representative screenshots of the PWA as captured by a mobile device are shown. The theme has a clean modern look with fonts suitable for onscreen reading. The logo icon is given a prominent position on both the homepage and on the school landing page so that users who install the app with the Add to Homescreen action will readily find the app on their mobile devices. In the screenshots, on the app bar, the classical hamburger icon for accessing the menu and the search icon can be observed. The link to the project repository on GitHub is in a prominent position; during the beta development phase each page had an edit on GitHub link so that users could have suggested modifications to the content this being an especially useful feature for the editorial review phase.

Figure 1 Representative screenshots of the “ABC Epatite” PWA as captured by a mobile device.

An efficient CDN was used for app distribution so as to provide fast first load times regardless of where the user is located ensuring a positive experience. The app was audited using Lighthouse, an analysis tool that provides PWA-aware information so as to fix any remaining issues.

Vuepress was found to provide excellent functionalities out of the box and proved to be easily extendable with add on vue components. In particular, we developed a vue share component, which provided native sharing on mobile devices and email sharing on desktop PCs. The source code of ABC Epatite app is openly available on GitHub (https://github.com/cnr-isc-ttt/abc-epatite/) so that others may view it as a use case. Some of the images used in the app were copyrighted stock pictures, which we could not make available on the public GitHub repository. In our images directory we made two subdirectories, one with the original copyrighted pictures and a second one with their corresponding watermarked preview versions. The project.gitignore file was configured to avoid uploading the copy-restricted versions of the pictures, and a Unix link was used to rapidly switch between restricted and public versions of the images.

Converting a PWA to a native Android app is quite feasible through the Trusted Web Activity (https://developers.google.com/web/android/trusted-web-activity) mechanism. Using Android Studio we developed a simple wrapper to the PWA and published it on the Play Store (https://play.google.com/store/apps/details?id=it.cnr.isc.abcepatite) ensuring that the content available in the native android app was entirely synchronous to that of the main PWA content. This approach enables users who prefer installing apps from the Google Play store to have a seamless experience while eliminating content duplication.

Engaging schools

The initial educational campaign was directed toward engaging schools in nine Italian regions. Working with information provided by regional educational boards a database of about 6,000 emails for public schools was compiled encompassing 9 of the 22 regions in Italy. A mass email service provider (sendpulse, https://sendpulse.com/) was used to send emails to these schools. From the 11^th until the 21^st of October 2019, 5,296 emails were sent, 5,007 (95%) were delivered and 1,572 (31%) were opened.

The structure of the emails sent was planned to provide schools with a clear understanding of the purpose and nature of the project and for whom it was directed at. Multiple ways to interact with the email were provided from direct links to buttons targeting the school specific landing page or the GitHub source code repository (Figure 2). This allowed us to evaluate if and how the school administration staff interacted on receiving the email, the additional link to the GitHub repository serving as a means to evaluate to what depth the email was being read. Table 1 reports these results.

Figure 2 The email sent to 6,000 Italian schools.

Mass email delivery services provide detailed information on the outcome of each email sent, enabling a detailed comparison by region or school type, e.g., state versus private schools. One metric available is the ratio of email opened with respect to email received; this measures the effectiveness of the email heading. By this metric, state schools appeared to be more receptive to the initiative; of the 4,148 emails delivered to state schools 37% (n=1,563) of these were opened. In contrast, of the 1,064 emails delivered to private schools in Lombardy only 11% (n=113) were opened. Interaction with the opened emails is also an important metric with users preferring links (12%) to the button (7%) in the email; overall click-through ranged from 14% in Piemonte down to 6% in say Lombardy.

Click-through rates do not provide the full picture as can be seen when they are compared with analytic data. Email campaigns were spaced out so as to provide a method to correlate a campaign with PWA sessions. Having sent 598 emails at around 3:30 pm on Oct. 15^th, two peaks were observed in the times at which these were read, one a little after they were sent and a second the next morning. While only 25 of these lead to a click-through, 86 new users were reported in that on the 15^th and 16^th this being 3.44 times larger than the click-through. Considering 1,572 emails were actually opened and that from Oct 11^th to Nov 30^th 427 new PWA users were observed in the analytics data, a ratio of about 1 to 4 between new users and opened emails was achieved. This first part of the email campaigns ended on Oct. 21^st and a well-defined peak with 33 PWA users was observed on Nov 14^th which most likely can be attributed to a class being coordinated through the school navigation path. User flow data for sessions on Nov 14^th are shown in Figure 3. It is interesting to note that of the 38 on the school-landing page, 19 followed the canonical route to the school level introduction while 10 went ahead to the main introduction and followed that separate path.

Figure 3 User flow data for sessions on Nov 14th.

Conclusions

Mobile first awareness campaigns offer considerable advantages over other more traditional communication channels. By using open-source tools, the implementation barrier has dropped considerably. Smartphone based information sources are perceived to be personal and trusted thus enhancing the emotional dimension of the message and thereby enhancing its persuasive effect. Analytics furnish a tool to quantify the effectiveness of a campaign’s message in an unobtrusive manner. Initial results based on a campaign directed toward Italian schools in just under half of its regions have shown just how feasible it is to reach larger audiences.

This is an important result considering just how critical the question of health literacy is. As underlines by recent research: “since health literacy is closely related to social determinants of health, such as education, income, language barriers, and other physical, cultural, and environmental factors, building knowledge and skill related to self-care can mitigate the health impact of low health literacy” (9). To put a perspective on this, data shows that an average of 47% of the European citizens have a limited level of health literacy (30); data from HLS-EU Consortium 2012) resulting in difficulties in coping with self-management and health related behavior.

As the awareness campaigns are ongoing, it is very likely that the impact of the COVID-19 pandemic may influence some aspects of this ABC Epatite project, and thus we look forward to studying this evolution.

These initial findings show that mobile first campaigns, given their extensive reach and cost effectiveness, have the potential to provide quality communication, to promote knowledge acquisition and to stimulate proper behaviors on viral hepatitis, at least at a national level. A combination of educational awareness campaigns and communication initiatives, such as mass-media outreach, online courses (31) and our ongoing project, may help in reaching the goal of eradicating viral hepatitis by reducing infections and deaths. Further research is essential in identifying and targeting hard to reach audience groups as well as in investigating the effective impact of mobile health apps on multiple dimension parameters such as engagement, behavioral changes, patient-HCP relationship and hepatitis prevention, diagnosis, and treatment.

Acknowledgments

Funding: The research project “ABC epatite. Sviluppo di una app per la prevenzione delle epatiti virali e per la consapevolezza dei comportamenti a rischio” was supported by the 2018 Digital Health Program of Gilead Science Italia (id. 04107).

Footnote

Reporting Checklist: The authors have completed the MDAR reporting checklist. Available at http://dx.doi.org/10.21037/mhealth-20-158

Data Sharing Statement: Available at http://dx.doi.org/10.21037/mhealth-20-158

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/mhealth-20-158). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Paakkari L, Okan O. COVID-19: health literacy is an underestimated problem. Lancet Public Health 2020;5:e249-50. [Crossref] [PubMed]
2. Baker DW, Wolf MS, Feinglass J, et al. Health Literacy, Cognitive Abilities, and Mortality Among Elderly Persons. J Gen Intern Med 2008;23:723-6. [Crossref] [PubMed]
3. Cavanaugh KL, Wingard RL, Hakim RM, et al. Low Health Literacy Associates with Increased Mortality in ESRD. J Am Soc Nephrol 2010;21:1979-85. [Crossref] [PubMed]
4. Bostock S, Steptoe A. Association between low functional health literacy and mortality in older adults: longitudinal cohort study. BMJ 2012;344:e1602 [Crossref] [PubMed]
5. McNaughton CD, Cawthon C, Kripalani S, et al. Health Literacy and Mortality: A Cohort Study of Patients Hospitalized for Acute Heart Failure. J Am Heart Assoc 2015;4:e001799 [Crossref] [PubMed]
6. Cugelman B, Thelwall M, Dawes P. Online Interventions for Social Marketing Health Behavior Change Campaigns: A Meta-Analysis of Psychological Architectures and Adherence Factors. J Med Internet Res 2011;13:e17 [Crossref] [PubMed]
7. Qudah B, Luetsch K. The influence of mobile health applications on patient - healthcare provider relationships: A systematic, narrative review. Patient Educ Couns 2019;102:1080-9. [Crossref] [PubMed]
8. World Health Organization. WHO guideline: recommendations on digital interventions for health system strengthening. Executive summary. Geneva: World Health Organization, 2019.
9. Conard S. Best practices in digital health literacy. Int J Cardiol 2019;292:277-9. [Crossref] [PubMed]
10. Bol N, Helberger N, Weert JCM. Differences in mobile health app use: A source of new digital inequalities? Inf Soc 2018;34:183-93. [Crossref]
11. Fitzgerald M, McClelland T. What makes a mobile app successful in supporting health behaviour change? Health Educ J 2017;76:373-81. [Crossref]
12. Kumar S, Nilsen WJ, Abernethy A, et al. Mobile Health Technology Evaluation: The mHealth Evidence Workshop. Am J Prev Med 2013;45:228-36. [Crossref] [PubMed]
13. Roberts J. Too Much of a Good Thing: Are You Addicted to Your Smartphone?. Seattle: Amazon Digital Services LLC - KDP Print US, 2015.
14. David M, Roberts J, Christenson B. Too Much of a Good Thing: Investigating the Association between Actual Smartphone Use and Individual Well-Being. Int J Hum-Comput Interact 2017;34:265-75. [Crossref]
15. Verhoeff N. Mobile Screens: The Visual Regime of Navigation. Amsterdam: Amsterdam University Press, 2012.
16. O’Brien H, Cairns P. Why Engagement Matters: Cross-Disciplinary Perspectives of User Engagement in Digital Media. New York: Springer, 2016.
17. Wiederhold BK, Riva G. Annual Review of Cybertherapy and Telemedicine 2014. Fairfax, Virginia: IOS Press, 2014.
18. Holdener M, Gut A, Angerer A. Applicability of the User Engagement Scale to Mobile Health: A Survey-Based Quantitative Study. JMIR MHealth UHealth 2020;8:e13244 [Crossref] [PubMed]
19. Horvath KJ, Ecklund AM, Hunt SL, et al. Developing Internet-Based Health Interventions: A Guide for Public Health Researchers and Practitioners. J Med Internet Res 2015;17:e28 [Crossref] [PubMed]
20. Garett R, Chiu J, Zhang L, et al. A Literature Review: Website Design and User Engagement. Online J Commun Media Technol 2016;6:1-14. [Crossref] [PubMed]
21. De Vito Dabbs A, Myers BA, Mc Curry KR, et al. User-Centered Design and Interactive Health Technologies for Patients. Comput Inform Nurs 2009;27:175-83. [Crossref] [PubMed]
22. McCurdie T, Taneva S, Casselman M, et al. mHealth Consumer Apps: The Case for User-Centered Design. Biomed Instrum Technol 2012;46:49-56. [Crossref] [PubMed]
23. Schnall R, Rojas M, Bakken S, et al. A user-centered model for designing consumer mobile health (mHealth) applications (apps). J Biomed Inform 2016;60:243-51. [Crossref] [PubMed]
24. Alberts WA, van der Geest TM. Color Matters: Color as Trustworthiness Cue in Websites. Tech Commun 2011;58:33-44.
25. Gump JE. The Readability of Typefaces and the Subsequent Mood or Emotion Created in the Reader. J Educ Bus 2001;76:270-3. [Crossref]
26. Bigi S, Caporale C, Zagarella RM, eds. Politiche del linguaggio in medicina. Una prospettiva etica e linguistica [Language politics in medicine. An ethical and linguistic perspective]. ETS; 2020. Available online: https://www.ibs.it/politiche-del-linguaggio-in-medicina-libro-vari/e/9788846758736
27. Eppler M, Mengis J. The Concept of Information Overload: A Review of Literature From Organization Science, Accounting, Marketing, MIS, and Related Disciplines. Inf Soc 2004;20:325-44. [Crossref]
28. Del Vicario M, Bessi A, Zollo F, et al. The spreading of misinformation online. Proc Natl Acad Sci 2016;113:554-9. [Crossref] [PubMed]
29. Quattrociocchi W, Vicini A. Misinformation: Guida alla società dell’informazione e della credulità. Milano: Franco Angeli Edizioni, 2016.
30. Sørensen K, Pelikan JM, Röthlin F, et al. Health literacy in Europe: comparative results of the European health literacy survey (HLS-EU). Eur J Public Health 2015;25:1053-8. [Crossref] [PubMed]
31. Buller-Taylor T, McGuinness L, Yan M, Janjua NZ. Reducing patient and provider knowledge gaps: An evaluation of a community informed hepatitis C online course. Patient Educ Couns 2018;101:1095-102. [Crossref] [PubMed]