Advances in mobile mental health: opportunities and implications for the spectrum of e-mental health services

Introduction

Perhaps no emerging technology development dovetails better with the patient-centered care (PCC) than mobile health (mHealth). PCC was conceptualized in the early 1990s by Harvey Picker and the National Research Council (1), is championed by the Institute of Medicine (2) and focuses on quality, affordable, and timely care. Person-centered healthcare emphasizes the whole person or person behind the patient (3). These shifts emphasize participatory medicine, moving patients from being mere passengers to responsible drivers of their health (4) by shared decision-making—in line with international standards (5). The patient-reported preferences, experiences and outcomes (PRO) is becoming a standard method for health systems and guideline development. mHealth empowers, enables and engages patients and other healthcare participants better and “around” the patient rather than the acute care and outpatient clinic (ironically called the medical home).

People and patients are empowered by mHealth, telemedicine and other technology-based services, which may be conceptualized as a telemental health (TMH) spectrum of care (6). E-mental health (eMH) is a term that is relatively new and it has been defined as “mental health services and information delivered or enhanced through the Internet and related technologies” (7). However, there is no agreement on a field-specific definition. The terms TMH and telepsychiatry (TP) have typically been used for traditional MH care services provided synchronously by videoconferencing, or asynchronously (8,9). A review of the literature on eMH through 2010, with most of the research (76%) from the USA, Australia, or the Netherlands, found four primary areas of eMH service delivery: information provision; screening, assessment, and monitoring; intervention; and social support (10).

Globally, Internet use has grown dramatically over the past decade, with a jump up to 44% of the population in the USA (6); Africa, the Middle East, and Latin America are the fastest growing populations of use. Online health and MH information varies in quality and readability (11), but it has helped people by enhancing coping strategies, empowerment, and self-efficacy. Users report reduced feelings of anxiety and isolation, enhanced connectedness in the doctor-patient relationship, and ability to make decisions on health-related behaviors (12-14). The Internet and other technologies may be used as a primary option or may complement regular MH care services.

Two areas that are growing exponentially are mobile MH apps and social networking after a somewhat slow uptake of MH apps attributed to MH organizations being ineligible to receive federal start-up for IT infrastructure. Mobile MH apps offer: (I) portability for access anytime, anywhere, regardless of patient geography and transportation barriers; (II) an inexpensive option versus traditional desktop computers; and (III) additional features, e.g., context-aware interventions and sensors (15) with real-time feedback. MH app demand is high across census-designated areas, generations, and, to a degree, age, with less use by older adults (15). Stress reduction programs using an app are increasing due to popularity economical impact (16). Some of these enhance social networking, which is typically defined as web-based service that allow individuals to construct a public or semi-public profile within a bounded system, share a connection with specific users, and traverse other connections of others (17). Health behaviors have been shown to change with this medium (18).

MH providers need a framework (6) to meet the challenges and requirements that are emerging in care-related complex interactions between consumers, patients, caregivers and other participants (Table 1). MH providers face many challenges with these emerging technologies, and they, like many others in society, may fear the trends (19). First, providers are encouraged to screen what technology is being used, how, and when—and to keep up with the slew of new options patients are using. Second, they need to evaluate how good MH or psychiatry apps are (i.e., evidence-based?) for smartphones and if they are used in an evidence-based approach (20). Third, clinicians and patients need to decide if any or all of the technology is instrumental and monitored in clinical care; this may include long-term planning. Fourth, clinicians may need to help the patient use the “right” service at the “right” time (e.g., not using social media when expressing suicidal ideation)? Fifth, clinicians and patients should weigh the advantages (empowerment, in-time learning, increased self-efficacy) versus the liabilities? And, finally, clinicians may need document use of MH apps as part of treatment plans.

Table 1 Mobile health on a continuum of e-mental health services: goals, pros/cons and suggestions for clinical care
Full table

This paper will:

• Define mHealth, elucidate its roots in medicine, describe its philosophical approach, and link its components with service delivery and outcomes particularly related to mobile mental health (mMH);
• Compare and contrast mMH to a range of eMH services including TP, and describe how one employs it within a service delivery system—and how healthcare may be built around it;
• Provide an approach to clinical care, education/training, administration and evaluation so that quality care is provided and participants adapt well to incorporation of new technologies.

mHealth, mobile MH and MH/psychiatric apps

An overview of mHealth

The definition of mHealth has shifted from “unwired e-med” (21) to “emerging mobile communications and network technologies for healthcare systems” in 2003 (22) to “wireless communication technologies that transform health, healthcare and public health” (23,24). Recent data suggest that more than 90,000 consumer smartphone health applications (“apps”) are now available for download (25)—many of these are for MH. Few of these have been scientifically studied for benefits or potential risks or submitted to USA Food and Drug Administration for review or approval (25). An estimated 69% of the USA adult population track at least one health indicator (e.g., activity, weight, symptom), but only about 20% of track it long-term (26). Patients in primary care have comparable or greater rates of using mHealth options as the general population (e.g., smartphone ownership 55%) (27).

mHealth has two major foundations: flexibility; and integration (28). First, it is able to incorporate qualities often associated with conventional health communication methods, such as personalization, tailoring, interactivity, and message repetition at a relatively low cost. SMS text messaging, for example, is used for scheduling, automated responses, and monitoring. Second, a good example of using mHealth for system integration is the linkage of: a national health network, hospital and other acute care centers, home-based care, and mobile devices (26). Key features include:

• Voice/video calling: convenient way for clinicians and patients to remotely communicate;
• SMS and multimedia message services (MMS) with video clips/sound files for education;
• Multimedia functions for a range of learning opportunities;
• Inbuilt touch, motion and GPS sensors that simplify clinical assessment and enhance lifestyle and social activities;
• Device connectivity: practical and less error-prone than manual data entry.

Since that system is too elaborate for many, the smartphone or tablet PC is the core device that links clinicians with patients in their own environment (Figure 1) and helps patients to self-manage their diseases via bi-directional flow of information. Even better, wireless monitoring devices gather data from sensors, input that data into a mobile medical app on the smartphone, relay the information to a network (26) and prompt clinical decision support. Flow of information becomes 24 × 7, with feedback on progress, as well as reminders of healthy behaviors, scheduled appointments and medications. Many patients like SMS text, educational videos or motivational short video clips from providers.

Figure 1 Integration of information in the technology age through the mobile/smartphone and other technologies.

Ecological momentary assessment (EMA) is a particularly promising method for mMH care, in capturing more accurate accounts of a client’s emotions, functioning, and activity related to mood anxiety and smoking (29-31). This method involves the repeated sampling of naturalistic behaviors and experiences—in other words, it enhances assessment. EMA has evolved from paper-and-pencil diary methods (e.g., medication calendars) to current use of smartphones that capture immediate self-reports while respondents carry out their daily lives.

Examples of EMA commonly used are daily diary methods, signal-dependent reporting, and event-dependent reporting. Daily diaries report on events and mood at the end of the day and are subject to bias from recall and social desirability. Signal-dependent reporting involves the client reporting on symptoms at random intervals during the day in response to an alarm. Event-dependent reporting has the client report on symptoms after predetermined interpersonal or challenging events during the day. Of the three, signal-and event-dependent reports are more accurate and yet, they demand a level of engagement and motivation that may exceed the capacity of some participants (32). Smartphones and wearable sensors have better potential to capture an accurate picture of a patient’s symptoms in real time and are less intensive.

mMH and MH/Psych apps

Once again, trends in mHealth point to how things develop for mMH, but the latter’s evolution may be similar and/or different. A review of mMH studies showed that text messaging was used in a wide range of mental health situations, notably substance abuse (31%), schizophrenia (22%), and affective disorders (17%) (33). Text messages were used in four ways: reminders (14%), information (17%), supportive messages (42%), and self-monitoring procedures (42%); and in combination. Most papers described pilot studies, while some randomized controlled trials (RCTs) reported improved treatment adherence, symptom surveillance, appointment attendance, and satisfaction with management and health care services. SMS text messaging cannot be used as a remote counseling tool like other telemedicine devices (7), but even a few words and a simple message can have an important impact. Personalization, caring sentiments, and polite text are associated with more successful preventative messages (34).

EMA is particularly well-suited for and widely used in mMH. A predictive analytic approach and functional data analysis applied to EMA data connected changes in affect with subsequent risk of suicidal ideation (35). Once more predictive models are developed and validated, self-management interventions could assist individuals or their caregivers in responding to future risks. Patients with bipolar disorder, schizophrenia, and other serious mental illnesses accept and are capable of participating in EMA studies, even if they are not users of mobile devices; study completion rates have been high in these samples (36). A good example is greater concordance between smartphone-captured mood ratings and clinician-rated affective symptoms than between paper-and-pencil mood ratings and clinician ratings (37). More complex systems that elicit data on multiple aspects of symptom and present summary feedback to respondents in graphical form facilitate self-management. Moreover, repeated data collection also enables modeling of within-person trajectories and temporal sequences of behavior (38).

Psych apps are used for many functions, including to: (I) communicate with other patients, caregivers, social supports, or providers; (II) augment psychotherapy and medical support with journaling, diaries, symptom tracking tools, and psychoeducation between clinic appointments; (III) (smart) monitor, that is, to use tools to predict relapse behavior or worsening affective symptoms, through sensors and data activity; (IV) to practice self-assessment and care through reflection about their symptoms; (V) make learning more interactive than traditional paper homework; and (VI) organize long-term activities, moods, and therapy homework (20,39,40). Since patients often forget key events between visits, logging “symptoms, affect, behavior, and cognitions close in time to experience” helps with reporting of symptoms (41).

Various mobile apps, especially those focusing on self-help in dealing with anxiety disorders, wellness and stress reduction, have been adjusted so that various patient groups may benefit from them (42). One example is a “Fear Fighter”, computer guided self-exposure approach to treat phobia/panic developed at the end of last century (6). Exposure therapy is effective for phobia/panic but qualified and trained therapist resources are scarce. By using a computer-guided approach that makes most of the treatment suggestions, and still achieves formidable results, both patient and clinicians achieve benefits by saving time and enhancing health care efficiency. An app called PTSD Coach (http://www.ptsd.va.gov/public/pages/PTSDCoach.asp) has been designed by the National Center for Telehealth and Technology to help veterans learn about and manage symptoms that commonly occur after trauma (6). It also has direct links to support and help; such apps are not designed to act as a substitute for treatment.

Psych apps are used to supplement or complement psychotherapy. Journaling, diaries, symptom tracking tools, and psychoeducation add to in-person clinic appointments. These encourage self-assessment, reflection about symptoms; and make learning more interactive. Apps are both empowering and reinforce action toward illness-specific education, treatment resource location, and tracking of treatment progress (43). Soldiers prefer to complete psychometric measures [e.g., Patient Health Questionnaire (PHQ) or PHQ-9] and other military population measures by iPhone rather than paper or computer due to its interface, portability, and convenience (44).

One promising area is supporting patients in attendance to treatment, which is a common reason psychiatric treatment fail to produce intended outcomes. Unfortunately, only about half of all patients obtain psychiatric treatment (45) due to stigma and poor insight. Direct or remote education, motivation and support may increase attendance (i.e., treatment readiness), recognition of treatment benefits, and enhance collaboration between care providers—all contribute to a positive psychiatric treatment (46). Recent patient-centered strategies that increase patient attendance and adherence to treatment include simple mail, telephone or SMS reminders (47).

A search revealed 166 and 240 psychiatry apps on the Apple and the Android stores, respectively. Medical students (N=185; 66.7%) have between 1–5 medical smartphone apps, used mainly for classroom and clinic purposes; 95.2% of the students indicated that having a psychiatry smartphone application would be useful, preferably with textbook contents and clinical videos embedded (48); there is a scarcity of high-quality, comprehensive, textbook grade e-learning materials (48). App designers are rarely clinicians or trainees, but if they were, there may be better accuracy of the content (49) and buy-in to use apps (50). The barriers for clinicians are typically anxiety/fear and a lack of technical skills (e.g., coding in computer programming language) and time. As with the implementation with electronic health records (EHRs), the role of physicians is cursory input on workflow or employing a leader such that he/she may then influence peers; companies who are more progressive may include them for better design (and perhaps for marketing purposes).

Internet and other technology-based options for patients, caregivers and clinicians

The eMH spectrum of how people, patients, caregivers and providers use technology (6) (Table 1) and particularly the internet began long before the advance of, but now overlaps with, the evolution of mHealth. The spectrum provides context, though, and data on its components informs mHealth, particularly mMH. While it technically might not matter if people/patients access this material while stationary or mobile, it may be helpful to research the trends in this and understand the differences.

The users of the Internet are mostly female (86% vs. 73% of men) and seek information on diseases or medical problems, treatments or procedures, doctors or other health professionals, hospitals or other medical facilities, food safety or recalls, drug safety or recalls, and pregnancy and childbirth (8). Caregivers (a term used for adults who provide unpaid care to a parent, child, friend or other loved ones) usually have access to the Internet (79%) and of those, 88% look online for health information. One’s education affects use (89% of those with a college degree vs. 70% with a high school degree vs. 38% without a high school diploma). Income is a predictor as well (95% with household income $75,000+ and 57% with ≤ $30,000).

A systematic review of 18 studies of the effectiveness of young people aged 14–25 seeking online MH help (N=18) reported high satisfaction and higher use by females (51). A key avenue is consumer driven sites where individuals connect with others in the community who are experiencing similar medical issues [e.g., PatientsLikeMe (http://www.patientslikeme.com/)]. Young people with developmental challenges may have few traditional care options and feel more comfortable sharing experiences and trying to learn new behaviors anonymously or at a distance (52). Comfortable with Internet-based chats and groups, they may even express ideas of self-harm, negative affective states, or pessimistic cognitions of other peers (53). No studies have been done to see if these concerning declarations are to be taken literally and if they are shared with parents and/or professionals.

In a community sample in France, young adults were assessed for eMH patient-related factors, use of eMH care and the impact on use of conventional services for MH care (54). Factors were organized into: (I) predisposing factors (age, sex, educational attainment, professional activity, living with a partner, children, childhood negative events, chronic somatic disease, parental history of depression); (II) enabling factors (social support, financial difficulties, parents’ income); and (III) needs-related factors (lifetime major depression or anxiety disorders, suicidal ideation, ADHD, cannabis use). Overall, 8.65% (105/1,214) of participants reported seeking eMH care in cases of psychological difficulties in the preceding 12 months and 15.7% reported psychological difficulties. The likelihood of eMH care was positively associated with lifetime major depression/anxiety disorder and lifetime suicidal ideation; the predisposing factor of childhood life events was negatively correlated. EMH care did not hinder traditional care, but was associated with face-to-face psychotherapy.

Support groups and participation in a “community”

Most support groups are for consumers and patients, based on the following premises: (I) knowledge affects changes in behaviors; (II) peer support/feedback may induce such changes (or in some cases, the opposite); and (III) even informal contact by e-mail, chat or telephone with a health care provider feels personalized and affects such changes. Internet-mediated support groups can include specialized groups for individuals with disabilities or unique modes of experience (55).

Web-based support has coalesced in MH around certain consumers, patients and other (e.g., caregiver) populations. A summary of these populations (6) includes:

• Individuals with stigmatizing or rare illness with social isolation;
• Schizotypal personality disorder patients, who have interest in social interaction on the Web (6) and interpersonal relationships without the usual in-person difficulties;
• Military personnel re-entry into regular life, whose fear of stigma reduces help-seeking and who prefer technology-based platforms (e.g., 33% of personnel were more willing to use a technology-based platform for MH care than talk to a counselor in-person);
• In about 2/3 of studies, caregivers who use Internet-based services have significantly reduced stress and improved quality of life for MH disorders (14). They use interactive communities to bulletin board therapy groups. Family caregivers located in rural areas found e-health support to be beneficial in comparison with conventional caregiver support (56).

Structured information and tools for self-directed habit, lifestyle or illness changes

These tools typically target good habits/health promotion, disease prevention, and informal management of symptoms or problems. Techniques might include use of a diary, a questionnaire or survey to provoke reflection or “stepping back” to re-evaluate one’s assumptions in a conclusion. Exercise and substance (i.e., alcohol) logs are popular, mood assessments (MoodyMe https://itunes.apple.com/us/app/moody-me-mood-diary-tracker/id411567371?mt=8), and those that map behavior patterns across time, including triggers, diet, sleep and other related factors.

Young people may benefit from structured health information, web-based screening and assessment, and online treatment options—across many settings—as free-standing promotion sites, programs at school, and combination home/primary care settings or home/MH specialist settings. Many Internet interventions have been developed to provide broad MH promotion in children and adolescents: Kindertelefoon (www.kindertelefoon.nl), YooMagazine (www.Yoomagazine.net), Ciao, ReachOut (www.reach-out.org) and Walkalong (www.walkalong.ca).

Informal advice from health professionals without guidance

A common misconception is that psychotic patients are not eligible for remote consultations and they do not use of technology, in general. This is attributed to stimulus overflow and inability to deal with the abundance of information, difficulties with concentration during psychosis, lack of energy, paranoid ideas and fear of symptom provocation. However, they successfully use the Internet for information related to their illness and medication (e.g., side effects and the hope of finding better medication) (57,58). On the other hand, patients may feel the need to guard themselves against excess information that Internet frequently offers. Health promotion strategies are typically at freestanding websites.

Some of the above options, while not considered “care”, involve some oversight by MH providers (e.g., depression). This usually involves bulletin boards with occasional comments or steering by professionals. For example, in an asynchronous chat group with education, the provider can participate periodically (e.g., paper, video or other) based on the discussion to provide information, corrections of misunderstood concepts or distortions, or review of self-report measures with a follow-up piece of advice. The “best” outcome of one of these forums is when a patient is referred to see a professional when things are not simple or there is a perception by the facilitator that too many concurrent problems are at-hand.

In a recent study, researchers reviewed the public social networking accounts of college students to assess for symptoms of depression, finding that 25% exhibited depressive symptoms based upon the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) criteria, and 2.5% met the criteria for major depressive disorder. Online reinforcement from their friends may have made them more likely to discuss their depressive symptoms publicly via social networking sites (59).

Support and self-help programs are delivered via Internet especially to rural areas but also within urban environments—for patients and caregivers. These allow anonymous questions, offer relevant treatment ideas, and provide self-help interventions without stigma (e.g., severe mentally ill or individuals with drinking problem) (60). The range of initiatives for support for caregivers includes hotlines for consultation on key decisions (i.e., decision support), psychosocial/CBT (individual or group), problem solving training, coaching for positive parenting skills (e.g., Internet- or app-based follow-up assessment and engagement of treatment), and use of formal questionnaires to self-diagnose and refer loved ones (e.g., patient health questionnaire for depression; hospital anxiety and depression scale).

Traditional clinician-assisted decisions, telepsychiatric care and other evidence-based options

The least structured of these options is patient-doctor correspondence integrated with clinical care and the EHR. As the Internet increases level of knowledge and information amount regarding specific illness, the users may easier talk to their doctor regarding their specific conditions and potential treatment options (13). Schizophrenic patients especially perceive the shift in hierarchy to a more equal relationship. This may be attributed to a sense of partnership or shared decision-making, which equalizes the informational and power symmetry between doctors and patients—both parties share information and develop consensus in a decision (61). A study about active discussions regarding continuation or discontinuation of an antipsychotic depot medication in patients with schizophrenia led to 87% of 96 patients continuing medication—a very high rate. In this respect, a specific advantage for patients with psychosis is not having to face another person, but still being able to gain information and interact with others without feeling devalued or unsafe (58).

Internet-based cognitive behavioral therapy (ICBT) and other evidence-based treatments are most often for patients with depression and anxiety based on a new review (62) and past summaries (8). ICBT appears to be effective when delivered in clinical practice (i.e., guided by a qualified therapist (63,64). Effect size and recovery rates were comparable to, or somewhat superior to, in-person CBT (65). Internet-based cognitive therapy (CT) is often combined with text messages (mobile cognitive therapy; mCT) and therapist e-mail and telephone contact—this prevents relapse in depression, is acceptable and is feasible for both patients and therapists (66). Online MH interventions are also as effective as traditional in-person therapy for disorders such as depression and anxiety (67-69). In a 30-month study using CBT for social phobia research, the long-term effects of in-person delivered CBT was comparable to Internet-based treatment (68).

Asynchronous telepsychiatry (ATP) to primary care is feasible, valid and reliable in English and Spanish-speaking patients in primary care (9). Similar methods are used in radiology, dermatology, ophthalmology, cardiology and pathology. One ATP model uses a basic questionnaire for screening by the provider of the patient, video capture of that interview, and uploading of patient histories for a remote psychiatrist for review in a HIPAA-adherent manner (9). Diagnosis and treatment recommendations are made and PCPs implement care successfully about 80% of the time and the model is cost-effective.

Synchronous TP (STP) or TMH models of clinical care and education have pros and cons (6,7), including their level of overall intensity, cost, feasibility and depth of the relationship between the eMH provider, the PCP and patient. A range of low to high intensity models from tele-education to videoconferencing has been described (70-72). A systematic approach funded by a grant in the USA developed a multi-specialty phone and email teleconsultation system for adults and children with developmental disabilities (6).

The adult practice guidelines for TMH health and other such practice parameters cover the approach, scope, clinical, administrative and technical aspects of services for adults and a new one for children and adolescent patients is in progress. This is needed as child and adolescent mental healthcare clinicians contend with specialized populations (e.g., developmental disorders), family and systems work, a variety of treatment modalities (e.g., parent management, play therapy) and settings (e.g., corrections/juvenile hall, school). These guidelines, though, do not cover all the new nooks and crannies of technology innovations (e.g., communications between professionals and clients or patients via texting, e-mail, chatting, social network sites, online “coaching” or other non-MH services). They do offer suggestions as a starting place to consider adjustments and control quality for the new technologies (e.g., licensing, emergency management, mandatory reporting and ethical issues). For new technologies, verification of provider names, credentials and sources to check the information on the professional and the patient is even more important to avoid security breaches.

Clinical care, training/education, system administration: approaches and preliminary guidelines

Technology integrated into clinical practice

The new application of telehealth modalities to one’s practice must be carefully selected, discussed with patients, and adaptable to the rapidly changing literature (Figure 2). When first selecting which modalities to add or subtract from one’s practices, recommendations should be considered, as is with the addition or change of any medical protocol. Considerations when applying a new model include the following:

Figure 2 Tips on clinical, program and system issues, outcomes and evaluation related to new technology options. PHQ, Patient Health Questionnaire; AUDIT, Alcohol Use Disorders Identification Test.

• The patient. Depending on comfort, familiarity with technology and/or the provider, the individual patient may have varying degrees of receptiveness to a specific telehealth model. The patient’s willingness to engage and favorable opinion is a key factor to the success of implementation and efficacy on improved healthcare delivery. Evaluation should also consider which technology is accessible, practical and feasible given the patient’s access to electronic products. The patient’s familiarity may also play into the patient’s view of the clinician as a professional (i.e., some may prefer in-person interactions; others may feel they are receiving higher quality of care through technological adjuncts). When adding technology to one’s practice, it is key to be aware of primary and secondary languages of the patient, and when differences arise, an in-person or telephone-based interpreter may be needed. For asynchronous communication—which often involves “short-hand” or abbreviated words and/or symbols, the cultural context is also important.
• The disease. The technology modality chosen must be appropriate and effective for the natural history of the disease. Chronic diseases may have a severe impact on quality of life that may benefit from the support of an online support group chat. Chronic medical conditions requiring constant monitoring not feasible through in-person visits, such as diabetes mellitus or hypertension, could benefit from the use of wearable devices and/or the submission of data to the practitioner. For all diseases, patient understanding of pathophysiology and/or treatment regimens may be improved by the adjunct of at-home reading done by the patient through online portals. Thus, the appropriate technological modality should be applied to maximize individual patient benefit and avoid difficulties.
• The provider. Before offering contact, communication and “care” via additional technologies, the provider must ensure that he/she has the time and resources to provide and maintain the quality and consistency of care. It is suggested to discuss expectations of the new modality and if the telehealth modality is offered in replacement of some in-person services (i.e., synchronous technology, at-home reading rather than in-person educational sessions) or as an additional adjunct. Frankly, it may or may not be possible to provide the same level of care via the technology being added.

Training and education

To date, only TP competencies have been published (73) but there has been a call for social media, mHealth, psych and MH app and other competencies (6) (Figure 3). The TP were based on ACGME and CanMEDs, but simplified into three levels that better fit learner levels and across disciplines:

Figure 3 Creating a culture of technology use for patients, students, clinicians and administration.

• Novice or advanced beginner (e.g., advanced medical student, early resident, or other trainees);
• Competent/proficient (e.g., advanced resident, graduating resident, faculty, attending, or interdisciplinary team member);
• Expert (e.g., advanced faculty, attending, or interdisciplinary team member.

The areas described in the TP competencies are patient care, systems-based practice communication, knowledge and practice-based learning. An interdisciplinary group has developed a framework for TMH health competencies (74).

Several strategies help providers to build and maintain competencies. Providers and trainees may complete self-study in many ways. There is a range of online resources that provide dynamic information on the changing telemedicine landscape: (I) professional organizations (e.g., American Telemedicine Association); (II) telehealth resource centers; (III) federal resources; (IV) grant-supported resources; and (V) private companies. Increasingly, training programs are incorporating TMH health rotations and seminars to teach technological approaches to health care.

Developing an administrative approach

An approach to clinical care, training/education and system administration includes setting patient-centered goals, evaluation, quality improvement and many other steps for healthcare with a new technology-based (Figure 2). Program evaluation with contributions from all participants has become increasingly important to meet program, patient, provider, and externally driven administrative (e.g., Joint Commission) and reimbursement [e.g., Center for Medicaid and Medicare Services (CMS)]; more accountability is expected by both consumers and payers. The Institute for Healthcare Improvement (IHI) is assisting healthcare systems in their transformation to higher-quality systems (75). For example, one of its initiatives is the Triple Aim, which consists of: (I) better population health; (II) better patient experience of care and better quality and safety of care and (III) reduced cost. Contemporary program evaluation and outcome work is a substantial shift in philosophical approach for some, from seeing what happens with planned services to planning the outcomes and then designing the services—in advance. Now, it is patient- and outcome-centered, whereby the end product determines what is built or put in place —hence assessment includes satisfaction, technology, cost, clinical, process of care, and other outcomes—iterative feedback, adjustments and further study make it useful.

Parameters and methods fall into three basic frameworks that naturally overlap with one another: (I) research measures, in the form of feasibility, validity, reliability, satisfaction, costs and outcomes; (II) clinical care measures (e.g., mood questionnaires; habit diaries; utilization of health services); and (III) customized measures for technologies. Suggestions:

• Pick 1–2 things to measure rather than trying to measure everything (e.g., an app for substance); how frequently is the app used, frequency of near misses of or actual use of substances;
• Pick an outcome that has high heuristic value (e.g., substance relapse; averted suicide; frequency of increased visits cued by using an app);
• Adopt standardized measures already used in the literature; they typically have undergone multiple iterations, levels of review and psychometric testing;
• Use a readily available, easy to use self-report instrument or program;
• Collect data prospectively rather than retrospectively, with some exceptions;
• If possible, pick a regular evaluation interval (e.g., beginning and then 3-, 6- and 12-month).
• Follow guidelines, but assess their liabilities to anticipate problems, take corrective actions, and generalize findings among different patients;
• Identify who has the responsibility to prevent, identify, and correct the issues: patients, providers, or programs? If patient care requires increased responsibility, are clinicians ready, and what support do they need? If providers have to adjust roles and responsibilities, do it proactively, too.

Guidelines

Guidelines tangibly help by providing clinical criteria, protocols, algorithms, review criteria, and other components—all aimed to help clinicians make the best clinical decisions, avoid bad outcomes, and to provide an approach in uncharted circumstances. The need for an evidence-based guideline on the use of medicine-related apps has been suggested by several parties (76-78). When dealing with apps, different aspects apply and these depend not only on the context, but also on the different “levels” one needs to consider. Therefore, the proposed guideline is subdivided into three sections: level 1 considers the global app level, that is, its purpose, where it fits in or provides an alternative, conflicts of interest, and registration (if any); level 2 focuses on content and process based on evidence-based systems and reporting [e.g., PRISMA (79); so pre-selection of studies provided by an app should not be classified as a systematic review with extracted information prepared in a “take home” summary]; and level 3 considers structured, formal assessment with outcome data (77). The Healthcare Information and Management Systems Society (HIMSS) has created assessment guidelines for mobile technologies (80).

A summary of suggestions on how to use e-mail, social media, and other technologies (Figure 4) may be helpful, and though they are not evidence-based, a number of them have come from prominent organizations internationally (81-85). They fall back on sound in-person ethical, legal and other administrative procedures from in-person and technologies used for some time (e.g., telemedicine). For example, requests for contact between visits (e.g., texts, e-mails) are increasing due to time online, and if responded to, they should be sent during regular working hours to attend to expectation and boundary issues (6). Asynchronous written or email language is good for answering yes/no questions, trading a piece of information (e.g., confirming appointment, medication side effect); these methods do not afford vocal nuances and accompanying body language, which may lead to misinterpretations and other unexpected consequences. Other preliminary guidelines discuss concerns about patient privacy, professional image, confidentiality, and defined expectations for use in general. Many organizations have specifically made recommendations about professionalism and social media [e.g., The American College of Physicians, Canadian Medical Association (6)].

Figure 4 Synopsis of guidelines for the eMH spectrum of service. MMS, multiple messaging service.

Discussion

Perhaps no emerging technology development both fits with PCC—and provides challenges to providers—than mHealth. Patient participation, leadership and sharing of preferences, experiences and outcomes are becoming a standard in healthcare and it is a great opportunity for patients and providers to collaborate. The eMH spectrum provides some orientation to changes in technology and yet mHealth is moving so fast that it may outdate this conception. For clinicians, there are a variety of goals to integrating new technology into one’s practice and the question will be how fast is too fast to apply changes to one’s practice in order to avoid hasty changes and the need to have a plan, procedure and/or protocol.

Competencies for clinicians are needed for mHealth, social media and other new technologies in the eMH spectrum, similar to those in TP (73). Clinicians have to become aware of, adapt to, and clinically oversee some or all of these new technology options in order to provide the best care—this means adding to or upgrading all parts of clinical care (e.g., review of decision-making, new advisory roles to patients, greater complexity of care, hybrid models of care). This also impacts standards for professionalism, privacy/confidentiality, tracking of data, evaluation and general practice management. It is critical that clinicians increase their awareness and understanding of mHealth options to understand patients’ concerns, changes in the therapeutic relationship, and potential positive/negative effects on outcomes. MH providers may soon practice in-person, virtually, or both, but how they spend their time may change (e.g., 1/2 traditional, 1/4 review of tech results, and 1/4 interdisciplinary team leadership). Clinicians, clinical managers and administrators need to shift their philosophy—from seeing what happens—to designing the services in advance to achieve outcomes.

More research is needed on the application of new technologies to clinical care, with attention to methods, outcomes and linkage (if any) with other care options, particularly in the form of randomized trials and study of health service delivery models with an emphasis on effectiveness. Relatively few studies assess outcomes, compare in-person and eMH care, and or compare technology-based care options to one another; hybrid models of care have emerged, but have not been studied. A number of studies and projects are well underway to demonstrate the utility of combined mobile data collection to improve our understanding of psychopathology (86). ICT-4Depression, a European 7th Framework Program for Research and Technological Development (FP7) project, is collecting EMA through a combination of mobile phone and web-based self-report assessment, using wearable sensors and recording electrophysiological measures (87). An algorithmic computation of the data to predict a patient’s current and future mental health states occurs through a monitoring program with real-time support to patients through smartphones and the Internet (87). It is also likely that EMA data collected electronically and be tied to EHRs to enhance our knowledge regarding who responds to some treatments and who responds best to others.

A dilemma exists, currently, in which neither public nor private, top-down nor bottom-up and country-specific nor international approaches related to apps is providing a framework to develop, evaluate and regulate to mHealth care. The result is a chaotic mix of apps of varying degrees of usefulness, quality, effectiveness and danger. A common vocabulary and set of quality standards for the review of health apps would benefit both end users, industry participants and governments by encouraging developers to secure favorable ratings by meeting the standards. Creation and adoption of review standards by an international, interdisciplinary consortium could reduce many of the barriers currently keeping mHealth technologies from becoming routine in providing healthcare worldwide. Ideally, such a consortium would be open to all who are involved in healthcare, including consumers, clinicians, academia, business, technology, education, professional and advocacy organizations (88). Such a consortium could initially coalesce around developing definitions, standards and quality assessment methods, such as a toolkit for app review (89), along with ethical standards (90).

Conclusions

mHealth, telemedicine and other services are considered part of a TMH health or eMH spectrum of care. mHealth offers excellent access, portability and low cost options. Like web- and Internet-based resources, the options are remarkably popular with the public, patients and providers—this is a new era of medicine. Patients are empowered by increased access to information and their providers. Exploring options as part of the initial and longitudinal care helps patients initiate, participate and steer their care. Clinicians have to become aware of, adapt to, use sound clinical judgment, and serve new advisory roles to patients, as we are all challenged to keep the best of MH care while making it more accessible. Prioritization of outcomes and evaluation in the provision of clinical services is important—any time that participants start to try some new technology.

Acknowledgements

The authors would like to thank Academy of Psychosomatic Medicine, Telepsychiatry Interest Group; American Psychiatric Association, Telepsychiatry Committee; American Telemedicine Association, Telemental Health Interest Group; Coalition for Technology in Behavioral Sciences, Competency Group.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Frampton SB, Guastello S. Honoring the Life of a Pioneer in Patient-Centered Care:Harvey Picker, PhD (1915-2008). Patient 2008;1:73-5. [Crossref] [PubMed]
2. Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington: National Academies Press, 2001.
3. Miles A, Mezzich J. The care of the patient and the soul of the clinic:Person-centered medicine as an emergent model of modern clinical practice. Int J Pers Cent Med 2011;1:207-22. [Crossref]
4. Frydman GJ. Patient-driven research: rich opportunities and real risks. J Participat Med 2009, Available online: http://ojs.jopm.org/index.php/jpm/article/view/28/18
5. Witteman HO, Dansokho SC, Colquhoun H, et al. User-centered design and the development of patient decision aids: protocol for a systematic review. Syst Rev 2015;4:11. [Crossref] [PubMed]
6. Hilty DM, Chan S, Torous J, et al. New frontiers in healthcare and technology: Internet- and web-based mental options emerge to complement in-person and telepsychiatric care options. J Health Med Informatics 2015;6:1-14.
7. Christensen H, Griffiths K, Evans K. E-mental health in Australia: implications of the internet and related technologies for policy. Canberra: Commonwealth Department of Health and Ageing, 2002.
8. Hilty DM, Ferrer DC, Parish MB, et al. The effectiveness of telemental health: a 2013 review. Telemed J E Health 2013;19:444-54. [Crossref] [PubMed]
9. Yellowlees PM, Odor A, Iosif AM, et al. Transcultural psychiatry made simple--asynchronous telepsychiatry as an approach to providing culturally relevant care. Telemed J E Health 2013;19:259-64. [Crossref] [PubMed]
10. Lal S, Adair CE. E-mental health: A rapid review of the literature. Psychiatr Serv 2014;65:24-32. [Crossref] [PubMed]
11. Ferreira-Lay P, Miller S. The quality of Internet information on depression for lay people. Psychiatric Bulletin 2008;32:170-3. [Crossref]
12. Morahan-Martin JM. How Internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav 2004;7:497-510. [Crossref] [PubMed]
13. Murray E, Lo B, Pollack L, et al. The impact of health information on the Internet on the physician-patient relationship: Patient perceptions. Arch Intern Med 2003;163:1727-34. [Crossref] [PubMed]
14. Hu C, Kung S, Rummans TA, et al. Reducing caregiver stress with internet-based interventions: a systematic review of open-label and randomized controlled trials. J Am Med Inform Assoc 2015;22:e194-e209. [PubMed]
15. Torous J, Chan SR, Yee-Marie Tan S, et al. Patient Smartphone Ownership and Interest in Mobile Apps to Monitor Symptoms of Mental Health Conditions: A Survey in Four Geographically Distinct Psychiatric Clinics. JMIR Ment Health 2014;1:e5. [Crossref] [PubMed]
16. Luxton DD, Hansen RN, Stanfill K. Mobile app self-care versus in-office care for stress reduction: a cost minimization analysis. J Telemed Telecare 2014;20:431-5. [Crossref] [PubMed]
17. Boyd DM, Ellison NB. Social network sites: definition, history, and scholarship. J Comput-Mediated Commun 2007;13:210-30. [Crossref]
18. Laranjo L, Arguel A, Neves AL, et al. The influence of social networking sites on health behavior change: a systematic review and meta-analysis. J Am Med Inform Assoc 2015;22:243-56. [Crossref] [PubMed]
19. Ben-Zeev D. How I stopped fearing technology-based interventions. Psychiatr Serv 2014;65:1183. [Crossref] [PubMed]
20. Chan SR, Torous JB, Hinton WL, et al. Psychiatric apps: Patient self-assessment, communication, and potential treatment interventions. In: Mucic D, Hilty DM. editors. Key Issues in e-Mental Health. New York: Springer Publishing, 2015;217-29.
21. Laxminarayan S, Istepanian RS. UNWIRED E-MED: the next generation of wireless and internet telemedicine systems. IEEE Trans Inf Technol Biomed 2000;4:189-93. [Crossref] [PubMed]
22. Istepanian RS, Lacal J. Emerging mobile communication technologies for health: some imperative notes on m-health. In: Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2003. United States of America: IEEE, 2003;1414-16.
23. Steinhubl SR, Muse ED, Topol EJ. Can mobile health technologies transform health care? JAMA 2013;310:2395-6. [Crossref] [PubMed]
24. Free C, Phillips G, Felix L, et al. The effectiveness of m-health technologies for improving health and health services: a systematic review protocol. BMC Res Notes 2010;3:250. [Crossref] [PubMed]
25. Pelletier SG. Explosive growth in health care apps raises oversight questions. AAMC Reporter 2012, Available online: https://www.aamc.org/newsroom/reporter/october2012/308516/health-care-apps.htm
26. Honeyman E, Ding H, Varnfield M, et al. Mobile health applications in cardiac care. Interv Cardiol 2014;6:227-40. [Crossref]
27. Fox S, Duggan M. Tracking for health. Pew Research Center 2013, Available online: http://pewinternet.org/Reports/2013/Tracking-for-Health.aspx
28. Bauer AM, Rue T, Keppel GA, et al. Use of mobile health (mHealth) tools by primary care patients in the WWAMI region Practice and Research Network (WPRN). J Am Board Fam Med 2014;27:780-8. [Crossref] [PubMed]
29. Carlson EB, Field NP, Ruzek JI, et al. Advantages and psychometric validation of proximal intensive assessments of patient-reported outcomes collected in daily life. Qual Life Res 2016;25:507-16. [Crossref] [PubMed]
30. van Os J, Delespaul P, Barge D, et al. Testing an mHealth momentary assessment Routine Outcome Monitoring application: a focus on restoration of daily life positive mood states. PLoS One 2014;9:e115254. [Crossref] [PubMed]
31. Langdon KJ, Farris SG, Overup CS, et al. Associations between anxiety sensitivity, negative affect, and smoking during a self-guided smoking cessation attempt. Nicotine Tob Res 2016;18:1188-5. [Crossref] [PubMed]
32. Berkman ET, Giuliani NR, Pruitt AK. Comparison of text messaging and paper-and-pencil for ecological momentary assessment of food craving and intake. Appetite 2014;81:131-7. [Crossref] [PubMed]
33. Berrouiguet S, Baca-García E, Brandt S, et al. Fundamentals for future mobile-health (mHealth): a systematic review of mobile phone and web-based text messaging in mental health. J Med Internet Res 2016;18:e135. [Crossref] [PubMed]
34. Odeny TA, Newman M, Bukusi EA, et al. Developing content for a mHealth intervention to promote postpartum retention in prevention of mother-to-child HIV transmission programs and early infant diagnosis of HIV: a qualitative study. PLoS One 2014;9:e106383. [Crossref] [PubMed]
35. Thompson WK, Gershon A, O’Hara R, et al. The prediction of study-emergent suicidal ideation in bipolar disorder: a pilot study using ecological momentary assessment data. Bipolar Disord 2014;16:669-77. [Crossref] [PubMed]
36. Depp CA, Moore RC, Perivoliotis D, et al. Technology to assess and support self-management in serious mental illness. Dialogues Clin Neurosci 2016;18:171-83. [PubMed]
37. Depp CA, Kim DH, Vergel de Dios L, et al. A pilot study of mood ratings captured by mobile phone versus paper-and-pencil mood charts in bipolar disorder. J Dual Diagn 2012;8:326-32. [Crossref] [PubMed]
38. Cain AE, Depp CA, Jeste DV. Ecological momentary assessment in aging research: a critical review. J Psychiatr Res 2009;43:987-96. [Crossref] [PubMed]
39. Harrison V, Proudfoot J, Wee PP, et al. Mobile mental health: review of the emerging field and proof of concept study. J Ment Health 2011;20:509-24. [Crossref] [PubMed]
40. Hilty D, Yellowlees PM, Parrish MB, et al. Telepsychiatry: Effective, Evidence-Based, and at a Tipping Point in Health Care Delivery? Psychiatr Clin North Am 2015;38:559-92. [Crossref] [PubMed]
41. Moskowitz DS, Young SN. Ecological momentary assessment: what it is and why it is a method of the future in clinical psychopharmacology. J Psychiatry Neurosci 2006;31:13-20. [PubMed]
42. Torous J, Staples P, Shanahan M, et al. Utilizing a Personal Smartphone Custom App to Assess the Patient Health Questionnaire-9 (PHQ-9) Depressive Symptoms in Patients With Major Depressive Disorder. JMIR Ment Health 2015;2:e8. [Crossref] [PubMed]
43. Luxton DD, McCann RA, Bush NE, et al. mHealth for mental health: integrating smartphone technology in behavioral healthcare. Prof Psychol Res Practice 2011;42:505-12. [Crossref]
44. Bush NE, Skopp N, Smolenski D, et al. Behavioral screening measures delivered with a smartphone app: psychometric properties and user preference. J Nerv Ment Dis 2013;201:991-5. [Crossref] [PubMed]
45. Agyapong VI, Rogers C, Machale S, et al. Factors predicting adherence with psychiatric follow-up appointments for patients assessed by the liaison psychiatric team in the emergency department. Int J Psychiatry Med 2010;40:217-28. [Crossref] [PubMed]
46. Gonzalez J, Williams JW, Noël PH, et al. Adherence to mental health treatment in a primary care clinic. J Am Board Fam Pract 2005;18:87-96. [Crossref] [PubMed]
47. Zanjani F, Davis T, Newsham P, et al. Management of psychiatric appointments by telephone. J Telemed Telecare 2015;21:61-3. [Crossref] [PubMed]
48. Zhang MW, Ho CS, Ho RC. Methodology of development and students' perceptions of a psychiatry educational smartphone application. Technol Health Care 2014;22:847-55. [PubMed]
49. Barton AJ. The regulation of mobile health applications. BMC Med 2012;10:46. [Crossref] [PubMed]
50. Subhi Y, Todsen T, Ringstd C, et al. Designing web-apps for smartphones can be easy as making slideshow presentations. BMC Res Notes 2014;7:94. [Crossref] [PubMed]
51. Kauer SD, Mangan C, Sanci L. Do online mental health services improve help-seeking for young people? A systematic review. J Med Internet Res 2014;16:e66. [Crossref] [PubMed]
52. Berger M, Wagner TH, Baker LC. Internet use and stigmatized illness. Soc Sci Med 2005;61:1821-7. [Crossref] [PubMed]
53. Griffiths KM, Calear AL, Banfield M. Systematic review on Internet Support Groups (ISGs) and depression (1): Do ISGs reduce depressive symptoms? J Med Internet Res 2009;11:e40. [Crossref] [PubMed]
54. Younes N, Chollet A, Menard E, et al. E-mental health care among young adults and help-seeking behaviors: A transversal study in a community sample. J Med Internet Res 2015;17:e123. [Crossref] [PubMed]
55. Antze P. On the pragmatics of empathy in the neurodiversity movement. In: Lambek M. editor. Ordinary Ethics. New York: Fordham University Press, 2010;310-27.
56. Blusi M, Dalin R, Jong M. The benefits of e-health support for older family caregivers in rural areas. J Telemed Telecare 2014;20:63-9. [Crossref] [PubMed]
57. Schrank B, Sibitz I, Unger A, Amering M. How use patients with schizophrenia the Internet:Qualitative study. J Med Internet Res 2010;12:e70. [Crossref] [PubMed]
58. Koivunen M, Välimäki M, Pitkänen A, et al. A preliminary usability evaluation of Web-based portal application for patients with schizophrenia. J Psychiatr Ment Health Nurs 2007;14:462-9. [Crossref] [PubMed]
59. Moreno MA, Jelenchick LA, Egan KG, et al. Feeling bad on Facebook: Depression disclosures by college students on a social networking site. Depression Anxiety 2011;28:447-55. [Crossref] [PubMed]
60. Riper H, Kramer J, Smit F, et al. Web-based self-help for problem drinkers: a pragmatic randomized trial. Addiction 2008;103:218-27. [Crossref] [PubMed]
61. Hamann J, Leucht S, Kissling W. Shared decision making in psychiatry. Acta Psychiatr Scand 2003;107:403-9. [Crossref] [PubMed]
62. Huguet A, Rao S, McGrath PJ, et al. A systematic review of cognitive behavioral therapy and behavioral activation apps for depression. PLoS ONE 2016;11:e0154248. [Crossref] [PubMed]
63. Andersson G, Hedman E. Effectiveness of guided Internet-based cognitive behavior therapy in regular clinical settings. Verhaltenstherapie 2013;23:140-8. [Crossref]
64. Andersson G, Hesser H, Veilord A, et al. Randomised controlled non-inferiority trial with 3-year follow-up of Internet-delivered versus face-to-face group cognitive behavioural therapy for depression. J Affect Disord 2013;151:986-94. [Crossref] [PubMed]
65. Ruwaard J, Lange A, Schrieken B, et al. The effectiveness of online cognitive behavioral treatment in routine clinical practice. PLoS One 2012;7:e40089. [Crossref] [PubMed]
66. Kok G, Bockting C, Berger H, et al. Mobile cognitive therapy: Adherence and acceptability of an online intervention in remitted recurrently depressed patients. Internet Interventions 2014;1:65. [Crossref]
67. Amstadter AB, Broman-Fulks J, Zinzow H, et al. Internet-based interventions for traumatic stress-related mental health problems: A review and suggestion for future research. Clin Psychol Rev 2009;29:410-20. [Crossref] [PubMed]
68. Carlbring P, Nordgren LB, Furmark T, et al. Long-term outcome of Internet-delivered cognitive-behavioural therapy for social phobia: A 30-month follow-up. Behav Res Ther 2009;47:848-50. [Crossref] [PubMed]
69. Kiropoulos LA, Klein B, Austin DW, et al. Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT? J Anxiety Disord 2008;22:1273-84. [Crossref] [PubMed]
70. Yellowlees PM, Hilty DM, Marks SL, et al. A retrospective analysis of child and adolescent e-mental health. J Amer Acad Child Adol Psychiat 2008;47:1-5. [Crossref]
71. Fortney JC, Pyne JM, Turner EE, et al. Telepsychiatry integration of mental health services into rural primary care settings. Int Rev Psychiatry 2015;27:525-39. [Crossref] [PubMed]
72. Myers K, Vander Stoep A, Zhou C, et al. Effectiveness of a telehealth service delivery model for treating attention-deficit/hyperactivity disorder: a community-based randomized controlled trial. J Am Acad Child Adolesc Psychiatry 2015;54:263-74. [Crossref] [PubMed]
73. Hilty DM, Crawford A, Teshima J, et al. A framework for telepsychiatric training and e-health: competency-based education, evaluation and implications. Int Rev Psychiatry 2015;27:569-92. [Crossref] [PubMed]
74. Sunderji N, Crawford A, Jovanovic M. Telepsychiatry in graduate medical education: a narrative review. Acad Psychiatry 2015;39:55-62. [Crossref] [PubMed]
75. Institute for Healthcare Improvement. The Triple Aim. Optimizing health, care and cost. Healthc Exec 2009;24:64-6. [PubMed]
76. Chan S, Torous J, Hinton L, et al. Towards a framework for evaluating mobile mental health apps. Telemed J E Health 2015;21:1038-41. [Crossref] [PubMed]
77. Gonnermann A, von Jan U, Albrecht UV. Draft guideline for the development of evidence based medicine-related apps. Stud Health Technol Inform 2015;210:637-41. [PubMed]
78. Agarwal S, LeFevre AE, Lee J, et al. Guidelines for reporting of health interventions using mobile phones: Mobile health (mHealth) evidence reporting and assessment (mERA) checklist. BMJ 2016;352:i1174. [Crossref] [PubMed]
79. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions:explanation and elaboration. PLoS Med 2009;6:e1000100. [Crossref] [PubMed]
80. Arellano P, Bochinski J, Elias B, et al. Selecting a mobile app:Evaluating the usability of medical applications. 2012. Available online: http://www.himss.org/selecting-mobile-app-evaluating-usability-medical-applications-0
81. Hilty DM, Shoemaker EZ, Myers KM, et al. Issues and steps toward a clinical guideline for telemental health for care of children and adolescents. J Child Adolesc Psychopharmacol 2016;26:283-95. [Crossref] [PubMed]
82. Koh S, Cattell GM, Cochran DM, et al. Psychiatrists' use of electronic communication and social media and a proposed framework for future guidelines. J Psychiatr Pract 2013;19:254-63. [Crossref] [PubMed]
83. Farnan JM, Snyder Sulmasy L, Worster BK, et al. Online medical professionalism: patient and public relationships: policy statement from the American College of Physicians and the Federation of State Medical Boards. Ann Intern Med 2013;158:620-7. [Crossref] [PubMed]
84. Behnke S. Ethics in the age of the Internet. Monitor Psychology 2008;39:74.
85. Turvey C, Coleman M, Dennison O, et al. ATA practice guidelines for video-based online mental health services. Telemed J E Health 2013;19:722-30. [Crossref] [PubMed]
86. Areàn PA, Hoa Ly K, Andersson G. Mobile technology for mental health assessment. Dialogues Clin Neurosci 2016;18:163-9. [PubMed]
87. Warmerdam L, Riper H, Klein M, et al. Innovative ICT solutions to improve treatment outcomes for depression:the ICT4Depression project. Stud Health Technol Inform 2012;181:339-43. [PubMed]
88. Pullier MR, Daviss S. A call for a digital health consortium. J Tech Behav Sci 2017. [Epub ahead of print].
89. Maheu MM, Nicolucci V, Pulier ML, et al. The Interactive Mobile App Review Toolkit (IMART): A clinical practiceoriented system. J Tech Behav Sci 2017. [Epub ahead of print].
90. Jones N, Moffitt M. Ethical guidelines for mobile app development within health and mental health fields. Professional Psychology: Research and Practice 2016;47:155-62. [Crossref]