Clinical evaluation of e-Quit worRx: a mobile app to enhance smoking cessation shared decision making in primary care

Introduction

Smoking is the leading preventable cause of morbidity and mortality in the United States (1). Although numerous interventions improve the likelihood of successful smoking cessation and the resulting health benefits, most smokers relapse or require several intervention attempts before staying quit (2). Primary care providers (PCPs) have a unique opportunity to discuss evidence-based smoking cessation methods in a way that enhances abstinence rates (3-5).

Several cessation aids have been associated with increased cessation rates. These include behavioral interventions such as physician advice (RR 1.76), quit line telephone counseling (RR 1.41), nicotine replacement therapy (RR 1.60), bupropion (RR 1.62), and varenicline (RR 2.27) (6). With so many options for cessation support, it is important for clinicians to personalize evidence-based interventions that are both useful and appealing to patients. During primary care office visits with competing priorities, applying patient-centered outcomes research for any given problem can be challenging (7).

The increasing integration of technology into practice offers an exciting opportunity to address barriers that have limited discussions of smoking cessation information in primary care. For example, in a 2018 survey, 61% of physicians said they use “technology in nearly all of their interactions with patients to better educate and engage with them”. This included 46% using mobile apps and 44% using tablets in the exam room (8). Patients are generally accepting of tablet technology in the exam room; in one study, 84% reported no problems using tablets themselves at their PCP’s office (9). In another study, over half of patients preferred to receive and provide information using a tablet instead of paper forms, with researchers finding that the information collected was more complete and accurate (10).

To fulfill their potential, interventions and tools must be carefully designed and implemented. For example, computer-based assistance for stage-of-change counseling in primary care increases patient-reported physician counseling and is cost effective (11), but has minimal effects on actual smoking cessation in primary care (12). While this study attempted to minimize time commitment by distilling a summary for the physician to one page, it nonetheless required lengthy patient assessments (20 minutes) (11) and a physician training session (30–40 minutes) focused on counseling and interpretation of the summary (12). Another study yielded promising results for personalized electronic decision support for smokers, but included a 30–90 minutes electronic patient intervention and was in a residential setting for mentally ill patients, not in a primary care setting (13). A randomized controlled trial had positive results for cessation rates in primary care, but had a five-component intervention that included a physician tutorial, vital sign stamp, physician performance feedback, nicotine replacement therapy, and telephone counseling (14). Mobile applications can be thoughtfully designed with consideration to the range of general and health literacy among patients (15,16). Finally, elements of stages of change counseling and motivational interviewing have been successfully incorporated into a hand-held electronic point-of-care tool, but the tool did not emphasize comparative effectiveness findings or shared decision-making (SDM) (17). These studies demonstrate that in order to succeed in primary care an intervention must fit into office workflow and provide added benefit to the busy practicing physician.

Electronic technology, especially with incorporated decision support, holds promise to meet these emerging preferences while overcoming the primary barriers to improved dissemination of smoking cessation evidence. Among other strengths, these technologies (I) fit into the workflow, as many practices have now successfully integrated mobile apps and tablets into everyday practice as described above; (II) save clinician time by assessing, providing information to patients, and summarizing information for rapid review and incorporation into medical records; and (III) enable communication with patients across the literacy spectrum in a manner that is appealing and familiar. The combination of incentives for both clinicians and patients are powerful. To address these opportunities and challenges, we developed and tested a tablet-based m-Health application (e-Quit worRx™) to assist PCPs in disseminating patient-centered outcomes research evidence to support SDM about smoking cessation (18,19). Literacy level of our app was assessed and optimized during the development of the app, reported previously (19). The primary outcome of this pilot study was app feasibility in primary care from the patient and PCP perspective.

Methods

Approach

This project was guided by a novel conceptual framework described previously (19). This framework includes grouping of factors called Antecedents, Mediators, and Proximal and Distal Outcomes. Its guided study innovations, the intervention design, and the selection of outcomes. Antecedents were factors present prior to our study including patient characteristics and environment and provider/health system variables. Mediators included content of the intervention and the patient-provider encounter. Proximal and Distal outcomes selected for study were assumed to be influenced by both antecedents and mediators, but it was not the goal of the study to link any particular outcome to a specific mediator.

The first aim of this project, previously published, was to develop the app, incorporating feedback from multiple stakeholder groups (19). The second aim, described in this manuscript, involved a pilot study and clinical evaluation of the final mobile app, e-Quit worRx™, at three different primary care offices within the University of Cincinnati Heath Primary Care Network (PCN) in Cincinnati, Ohio.

For this practice-based pragmatic study, we chose a single crossover control design wherein each practice begins the study in a control period (Booklet group), followed by an intervention period (iPad group). This design was chosen because of challenges associated with other designs for this practice-based pragmatic trial. At each study site, we recruited and enrolled the control patients (Booklet group), then trained staff and PCPs on the use of the app, then recruited and enrolled all of the intervention patients (iPad group).

The trial consisted of a single study visit and a 12-week follow-up phone call. The study visit occurred at a previously scheduled visit with the patient’s PCP. Our study design aimed to incorporate the decision aid app into the current smokers’ waiting time for their PCP in the exam room, so the PCP need only review their responses and personal selections to finalize treatment choices. After the visit, the research nurse (RN), same for all 73 encounters, completed an exit interview with all enrolled patients. After the study period at each site, the research team conducted focus groups one with PCPs and another with medical support staff. Finally, we contacted all patients 12 weeks after their study visit for a short phone interview.

This study was approved by the University of Cincinnati’s Institutional Review Board.

Recruitment and enrollment

Goal patient enrollment was 72 patients. Inclusion criteria for patients were as follows: (I) age 21 or older; (II) smoked within the past 7 days; (III) smoked at least 100 cigarettes in their lifetime; (IV) English-speaking; (V) physically able to use a tablet; and (VI) not having plans to switch their PCP for the next 12 months. Participants were excluded if they had (I) current (past month) smoking cessation treatment use (already motivated and in treatment) or (II) an inability to provide consent. Fifty-two PCPs and 35 support staff also consented to participate in their respective clinical roles. The 72 patients (12 per group per practice) would come from three purposefully selected practices in our health system’s PCN. All three clinical practices in the study had Patient Centered Medical Home certification, indicating they already value SDM with patients. The three family medicine and internal medicine/pediatrics practices, included a hospital-based residency practice consisting of 56% African American patients, 63% Medicaid-covered patients, and only 10% of patients aged 65 and older; a suburban academic practice whose patients are 40% African American, 19% Medicaid, and 20% 65 and older, and another suburban academic practice with a larger proportion of older adult patients (27%) but fewer patients who are African American (28%) or Medicaid-covered (6%). This mix of practices helped ensure that we recruited patients of varying ages, genders, races, and socioeconomic status.

Patients were initially recruited via letters mailed to those who were smokers based on the electronic health record (EHR). We had no response from mailed letters. The RN also approached patients waiting for a visit and inquired if they currently smoked cigarettes. If there was an affirmative response, they were asked if they were interested in participating in the study. They were told that they did not have to change their smoking habits in order to be in the study. Once recruited, patients completed the informed consent process with the RN. Once patients were consented and enrolled, and prior to the clinical encounter in the exam room, the RN provided either a paper smoking cessation booklet (“Booklet group”) or the study iPad (“iPad group”). The RN collected demographics, details of smoking history, and some health history, from patients in the Booklet group. They were asked to read a standard 9-page smoking cessation booklet, “Help for Smokers and Other Tobacco Users,” from the U.S. Department of Health and Human Services (20). This booklet had previously been used in studies of decision aids (13). Demographics were collected from both groups for sample description purposes only (see Table 1). The Booklet group was a Treatment as Usual/Standard Care control but included the booklet as an attention control. It was a relatively basic government-produced pamphlet that included basic smoking risk information, sample encouraging and motivating statements, a list of medicines to ask about, and the national quit line number.

Table 1 Demographics of the study groups
Full table

Patients in the iPad group were handed the iPad with the study application. Patients were asked to work through the interactive iPad app, while waiting for their PCP. Patients were instructed to hand the iPad to the PCP when the PCP entered the exam room. Research staff was not be involved with rooming the patient or present during the clinical encounter in order to maximize real-world applicability.

e-Quit worRx

Our goal was to design a decision aid app that could be completed by the patient while waiting for their PCP to enter the exam room. This timeframe was likely to fit into most clinical flows. As will be mentioned in the Limitations section, we were not able to capture actual patient or PCP time commitment but qualitative data lent support to our assumption. On the iPad, the app guided the patient through each step, inputting personal information when prompted, including demographics, details of smoking history, and some health history (all Antecedents in the conceptual framework), stage of change information, and motivations to continue or quit smoking. These included the Fagerstrom Nicotine Tolerance Questionnaire (FTQ) (21), Smoking: Stage of Change-Short Form (SOC-SF) (22,23), and Smoking: Decisional Balance-Short Form (DBSF) (24). The user interface was designed through an iterative process of qualitative feedback and integration of design principles from the literature and was described previously (19).

Based on participants’ prior input, the app produced custom information about their smoking-related health risks, cigarette costs over time, dependence level, stage of change, and personal barriers and facilitators to quitting (Mediators). If applicable, based on their personal stage of change, they were presented with a comprehensive list of cessation aids including medications, therapy, and m-Health tools from which to choose. In order to allow for patient-centered selection of cessation aids of interest, the app included common interventions with strong evidence of efficacy (e.g., nicotine replace therapy, bupropion SR, varenicline, counseling and phone support) as well as other non-first line treatments such as cold turkey, self-help, mobile apps, acupuncture, and e-cigarettes.

Patients could click on items to get additional information on each (as a single level pop up). They could indicate it a method had already been tried and failed, and were prompted to select up to three cessation aids to compare. Information on comparative effectiveness, costs, and risks was then displayed side-by-side and patients could finalize their choices which would then be discussed with their PCP. In the case of patients who were not yet interested in specific treatment, they instead selected another element of motivation interviewing to discuss with their PCP such as barriers or personal motivators to quit.

Once selections were made, a static screen of the patient’s customized report, the “Provider Summary” was displayed for the PCP to review. Figure 1A illustrates a static screen shot of an example e-Quit worRx™ Provider Summary. This single custom report screen contained an easily understood organized summary of the patient’s responses. This information included their personal stage of change, personal pros and cons for smoking, health risks, and preferred and non-preferred cessation aids (or personal barriers and motivators if applicable based on their stage of change).

Figure 1 e-Quit worRx provider screenshots. (A) Provider summary; (B) treatment selection.

PCP interaction

Upon entering the room, the PCP conducted their clinical interaction with the patient. Booklet group patients were asked to discuss smoking cessation with their PCPs during their clinical encounter. iPad group patients were asked to hand the iPad to the PCP. Since this pragmatic intervention was embedded in real-life primary care clinic visits, in both groups, the PCP had the option of deferring the smoking cessation discussion if other health issues were more pressing.

In the iPad group, the PCP was able to view the Provider Summary, clicking on different tabs if needed for detail, and, after discussing with the patient, was able to advance to the following screen, “Treatment Selection,” to select from among the patient’s own choices or others, the cessation aid or aids to prescribe that day, if any (see Figure 1B). If applicable, based on stage of change, the selected plan for the patient might be “work in barriers to quitting” for example. Finally, PCPs completed three questions on the app evaluating SDM and the app itself.

Integration into practice and home

Once selected, the app communicated the cessation aid choices with the patient in the method preferred by the patient, either by printing a custom report at the office or e-mailing it to the patient via a secure e-mail. It communicated with the medical office by printing the report to be scanned. Though desirable, a direct link to the EHR was not a goal for this study. We did however design parallel EHR tools that allowed for rapid translation of the report information into the medical record by the PCP and to assist in accurate billing for counseling time and ordering medications and referrals as needed. The newly created order set included all information needed to supply the agreed upon treatment including dosing based on level of dependence and local contacts for counseling, acupuncture, etc.

Post-intervention patient assessment

Immediately after the clinical encounter, the RN opened up additional assessments on the app for the patient to complete in order to measure Proximal outcomes. All patients, as part of this “exit interview,” completed surveys on the iPad about SDM (SDM-Q-9), patient-centered communication, decisional conflict (SURE scale), and usability [modified Client Satisfaction Questionnaire-8, (CSQ-8)]. Data from these assessments formed the basis for determining the primary outcome of feasibility from the patient perspective. Due to minor design flaws not caught in time, some data were lost (see Limitations).

PCP and practice assessments

Participating PCPs were asked to complete a brief interview with the RN at the end of each clinical study day in which they had an enrolled a patient from either Group. A brief qualitative interview about the app and provider communication were also included as part of the exit interview. The RN also made daily research notes detailing issues or concerns with the intervention in the practice or the use of the iPad. At the end of the study period, depending on practice size, one or two focus groups were held at each site, preferably one group with PCPs and one group with staff, to assess the use of the app in the office setting. Data from these assessments formed the basis for determining the primary outcome of feasibility from the PCP perspective.

Patient follow-up

Patients were contacted by the RN or another team member by telephone at 12-week post study visit for a follow-up interview to assess Distal outcomes including self-reported abstinence as well as to complete another FTQ and SOC-SF. Table 2 contains a list of all instruments used including all Proximal and Distal outcomes.

Table 2 Study instruments
Full table

Statistical analysis

As a pilot, this study was designed with an emphasis on evaluating feasibility rather than an emphasis on the power of hypothesis tests (25) and our sample size was expected to be large enough to provide useful information about the various aspects of feasibility being studied (26).

Previous pilot/feasibility studies have included a wide range of numbers of subjects, depending on context, though one general rule suggests 30 (25) and another review found the mean to be 34 (27), therefore we felt that 36 control and 36 intervention patients would be reasonable.

For qualitative data, themes were identified using the “editing” technique (28) and interview transcripts were coded independently by 2–3 research team members. Emerging themes, in addition to a priori concepts were identified. For quantitative data, descriptive statistics were used to characterize the groups at baseline and Booklet group versus iPad group were compared using t-tests for means and Chi squares for proportions.

Results

Primary outcome

Feasibility of the intervention from the patient perspective was assessed during the exit interviews. A majority (95%) of iPad group patients said they would want to use an app like this at their PCP’s office in the future outside of a paid study (this question was not asked of Booklet patients). Patients in both groups endorsed that the app and booklet were easy to use (Figure 2). Feasibility from the PCP perspective was determined largely qualitatively through the end-of-study feedback sessions (Figure 3), but PCPs answered two questions in the app itself and in 35 of 37 cases (95%) endorsed that the app promoted SDM.

Figure 2 Select exit interview data.

Figure 3 Qualitative feedback on App from patient exit interviews and PCP and staff focus groups.

Demographics

Seventy-three patients were enrolled and completed the study visit (36 Booklet group patients and 37 iPad group patients). Demographics and smoking characteristics of Booklet and iPad groups were similar with no significant differences except for sex as noted in Table 1. Overall study patients included 50% white, 46% African American, and 4% other race and the majority (64%) were female. Education level ranged from less than high school to professional degree, but 73% made less than $50,000 per year and insurance payer varied widely. Thirty eight percent (38%) lived with another smoker. One-third (33%) of control patients and 31% of intervention patients were in the preparation stage of change at the time of the study visit (P=0.83). Study visits for these 73 patients included 22 of the 52 enrolled PCPs.

Exit interview data

More Booklet group patients stated that a decision was made about their smoking, even if the decision was to make no changes (97% vs. 67%, P=0.001) (see Figure 2). Although not significant, for patient-reported SDM (primary outcome), the mean SDM score (out of 100) for iPad group [76] was higher than for Booklet group [64] (P=0.16) and for decisional conflict the percentage for iPad patients (11%) was lower than the percentage for Booklet patients (22%) (P=0.21). Patient-reported time spent by their provider discussing smoking cessation was significantly higher for iPad patients (7.6 vs. 3.3 min; P<0.05).

As mentioned in Methods, PCPs could defer smoking cessation discussions during the visit. Based on patient-reported time spent data, this happened for 9 of the 36 Booklet patients (25%) but not once for iPad patients. There was no difference in endorsing “learned new ways to quit smoking” (56 vs. 57%, P=1.000) but, while not significant, the percentage of iPad patients endorsing that they “learned about smoking and addiction” was higher than for Booklet patients (42% iPad vs. 26% Booklet, P=0.20) and the percentage of iPad patients was more than the percentage of Booklet patients endorsing that “the app/booklet affected their conversation with their provider” (57% vs. 34% control, P=0.08).

Data collected from PCPs and staff

For 12 of the 37 iPad group patient encounters (32%), PCPs endorsed that using the app changed their smoking cessation plan at the visit but, as mentioned as part of the primary outcome, PCPs stated that in 35 of 37 cases (95%) the app promoted SDM. These questions were not asked after Booklet group encounters. Qualitative themes developed, and representative direct quotations, from analysis of PCP and staff focus groups, as well as patient exit interviews, are shown in Figure 3.

Twelve-week follow-up data

A majority (83%) of the 73 enrolled subjects completed their 12-week follow-up phone calls. Few differences were apparent between Groups. Among all enrolled patients, 88% were still smoking at the time of their follow-up phone call, though 79% had quit for at least 24 hours within the last year (asked as part of SOC-SF, could have been prior to enrollment) and 64% made a quit attempt since their study visit including 25% who used a cessation aid. Thirty-five percent (35%) of patients had seen their PCP to discuss smoking cessation since the study visit. No differences were found between the Groups in questions about the study visit being helpful in choosing a cessation aid or changing motivation to quit or confidence in cessation success with next attempt.

At the time of the follow-up calls, 22% of Booklet group patients but 46% of iPad group patients endorsed that they were seriously thinking of quitting smoking within the next 30 days (preparation) (P=0.11). Considering that at the study visit about 32% of both Groups were in preparation stage, the post-assessment stage of change represents a pre/post difference for iPad patients (67%) versus Booklet group patients (43%) having progressed to preparation stage. While non-significant (P=0.052), this potential difference in the proportion of subjects in each group that had a positive change was none-the-less encouraging.

Limitations

This study had a few predicted and unforeseen limitations. The design process described in detail separately (19) presented several challenges including navigating requests to our coders for repeated changes to both content and design, resolving conflicting feedback from our diverse group of stakeholders and even within our study group, realizing the time-intensity of editing content and code, and integration into a clinical setting.

The biggest unforeseen limitation was that some study data was lost and could not be analyzed due to a database error that we did not notice until data collection was completed. For example, nearly all cigarette quantity data were lost meaning we could not calculate Fagerstrom dependence levels. Also, we intended to collect patient time completing the app and PCP time reviewing app data and the coding failed to capture these data. The loss of these few data points did not significantly impede our analysis and did not affect the study visit at all since the data was displayed for the patient and provider, being lost in the data transfer to our research database. While we did not directly evaluate time commitment, qualitatively clinic staff and PCPs did not feel that the app slowed them down. Frequent iOS updates required updates to our e-Quit worRx™. These updates did not alter the content of the app or user experience; they just required a patch to bring the app back into compatibility with the iOS before it would work. Again, while this did not impede our trial, it could pose a problem for sustainability since any app will require ongoing information technology maintenance.

Challenges with our health system’s information technology department, the project timeframe, and limited budget were significant barriers. The app could not be fully integrated into the EHR so that patient selections and chosen interventions would automatically populate into the medical record. However, we were able to integrate into the clinic sites in several ways. We gained access to the network and Internet connection, allowing real-time secure data transfer to our database. We enabled automated Email messaging to patients at the end of the session summarizing their study data. In addition, we built new matching templates (i.e., SmartPhrases and a SmartSet order set) for our EHR (Epic, Epic Systems Corp.), so that PCPs could quickly copy over patient selections from the study. Though the existing clinic printers could not be used to print from our app, we placed AirPrint^® enabled printers at each site to allow printing summaries for patients and PCPs.

Finally, as a pilot trial our small sample size leaves open the possibility that many of non-significant mean differences, including the primary outcome of promoting SDM, may have reached significance in a larger study.

Discussion

Our primary outcome of feasibility was achieved based on quantitative and qualitative findings such as that providers in 95% of encounters felt that it promoted SDM and 95% of patients said that they would want to use an app like this at their PCP’s office in the future outside of a paid study.

Through this project, we successfully created a usable iPad app-based decision aid for use in primary care offices. Our app engaged patients and providers in smoking cessation conversations. It significantly increased time spent discussing smoking cessation and the likelihood that a decision was made at the time of the visit and, in a larger sample, may improve SDM. Determining whether or not a decision is made is necessary in order to identify whether decisional conflict is present. Therefore, arriving at a decision, no matter what the decision was, was seen as a positive outcome. PCPs strongly endorsed that the app promoted SDM and a non-significant mean difference was observed for patient-reported SDM. Non-significant differences were also observed in other measures including decreased decisional conflict and especially for intervention patients progressing into preparation stage of change at the time of their 12-week follow-up.

We successfully ran a pragmatic pilot trial in three primary care offices using a technology novel to many of the users. Overall, the app was easy to use but office flow concerns were raised. One possible solution to this would be integration with the patient portal so patients could complete the app prior to their visit.

Significance

To our knowledge this is the first iPad app-based decision aid for smoking cessation that was designed using evidence-based methods to be used in a clinical setting. If this technology can be further integrated into EHR systems we think acceptability to providers will increase and apps like these have the potential to improve patient-centered care, SDM and patient engagement and empowerment.

Acknowledgments

The authors acknowledge the contributions made by Dr. Nancy Elder for her clinical input, Balaji Baskaran and Nandita Subramanian for their efforts in programming the app, Dr. Josh Magee for his help during initial conceptualization of the project, Brett Harnett provided invaluable technical feedback, students Haley Boling and Kelsey Dirksing who supported the research team conducting literature searches and other tasks, and finally, the clinical staff at each location who worked with research staff and clinicians to accomplish the research objectives.

Funding: Funding was provided through Agency for Healthcare Research and Quality, Grant No. R21HS023994.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was approved by the University of Cincinnati’s Institutional Review Board (No. 2015-0880).

References

1. Fast Facts. Smoking & Tobacco Use 2014. Available online: http://www.cdc.gov/tobacco/data_statistics/fact_sheets/fast_facts/. Accessed (cite 2014 Jul 21).
2. Brandon TH, Tiffany ST, Baker TB. The process of smoking relapse. NIDA Res Monogr 1986;72:104-7. [PubMed]
3. Gorin SS, Heck JE. Meta-analysis of the efficacy of tobacco counseling by health care providers. Cancer Epidemiol Biomarkers Prev 2004;13:2012-22. [PubMed]
4. Katz DA, Muehlenbruch DR, Brown RB, et al. Effectiveness of a clinic-based strategy for implementing the AHRQ smoking cessation guideline in primary care. Prev Med 2002;35:293-301. [Crossref] [PubMed]
5. Stead LF, Buitrago D, Preciado N, et al. Physician advice for smoking cessation. Cochrane Database Syst Rev 2013;5:CD000165. [PubMed]
6. Patnode CD, Henderson JT, Thompson JH, et al. Behavioral Counseling and Pharmacotherapy Interventions for Tobacco Cessation in Adults, Including Pregnant Women: A Review of Reviews for the U.S. Preventive Services Task Force. Ann Intern Med 2015;163:608-21. [Crossref] [PubMed]
7. Jaén CR, Stange KC, Nutting PA. Competing demands of primary care: a model for the delivery of clinical preventive services. J Fam Pract 1994;38:166-71. [PubMed]
8. Physicians’ Perspectives: Patient Engagement Technology (and Pharma’s Role in It). Digital Heatlh Coalition. P. Available online: http://patientpoint.com/wp-content/upl.oads/2018/10/PatientPoint-DHC-Physician-Research-Infographic.pdf
9. Strayer SM, Semler MW, Kington ML, et al. Patient attitudes toward physician use of tablet computers in the exam room. Fam Med 2010;42:643-7. [PubMed]
10. Schlechtweg PM, Hammon M, Giese D, et al. iPad-Based Patient Briefing for Radiological Examinations-a Clinical Trial. J Digit Imaging 2014;27:479-85. [Crossref] [PubMed]
11. Smith MY, Cromwell J, DePue J, et al. Determining the cost - effectiveness of a computer - based smoking cessation intervention in primary care. Manag Care 2007;16:48-55. [PubMed]
12. Unrod M, Smith M, Spring B, et al. Randomized controlled trial of a computer-based, tailored intervention to increase smoking cessation counseling by primary care physicians. J Gen Intern Med 2007;22:478-84. [Crossref] [PubMed]
13. Brunette MF, Ferron JC, McHugo GJ, et al. An electronic decision support system to motivate people with severe mental illnesses to quit smoking. Psychiatr Serv 2011;62:360-6. [Crossref] [PubMed]
14. Katz DA, Muehlenbruch DR, Brown RL, et al. Effectiveness of implementing the agency for healthcare research and quality smoking cessation clinical practice guideline: a randomized, controlled trial. J Natl Cancer Inst 2004;96:594-603. [Crossref] [PubMed]
15. Boulos MN, Brewer AC, Karimkhani C, et al. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J Public Health Inform 2014;5:229. [PubMed]
16. Kamel Boulos MN. Social media and mobile health (Chapter 13, under Part B: Taking action to create and strengthen health literacy-friendly settings). Available online: http://www.euro.who.int/__data/assets/pdf_file/0008/190655/e96854.pdf
17. Strayer SM, Heim SW, Rollins LK, et al. Improving smoking cessation counseling using a point-of-care health intervention tool (IT): from the Virginia Practice Support and Research Network (VaPSRN). J Am Board Fam Med 2013;26:116-25. [Crossref] [PubMed]
18. Fowler FJ Jr, Levin CA, Sepucha KR. Informing and involving patients to improve the quality of medical decisions. Health Aff (Millwood) 2011;30:699-706. [Crossref] [PubMed]
19. Doarn CR, Vonder Meulen MB, Pallerla H, et al. Development and Usability Testing of e-Quit worRX, an iPad App for Smoking Cessation Counseling and Shared Decision Making in Primary Care. JMIR Form Res 2019;3:e11300. [Crossref] [PubMed]
20. U.S. Department of Health and Human Services PHS. Help for Smokers and Other Tobacco Users. Rockville, MD: 2008.
21. Heatherton TF, Kozlowski LT, Frecker RC, et al. The Fagerstrom Test for Nicotine Dependence: a revision of the Fagerstrom Tolerance Questionnaire. Br J Addict 1991;86:1119-27. [Crossref] [PubMed]
22. DiClemente CC, Prochaska JO, Fairhurst SK, et al. The process of smoking cessation: an analysis of precontemplation, contemplation, and preparation stages of change. J Consult Clin Psychol 1991;59:295-304. [Crossref] [PubMed]
23. Velicer WF, Fava JL, Prochaska JO, et al. Distribution of smokers by stage in three representative samples. Prev Med 1995;24:401-11. [Crossref] [PubMed]
24. Velicer WF, DiClemente CC, Prochaska JO, et al. Decisional balance measure for assessing and predicting smoking status. J Pers Soc Psychol 1985;48:1279-89. [Crossref] [PubMed]
25. Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract 2004;10:307-12. [Crossref] [PubMed]
26. Thabane L, Ma J, Chu R, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010;10:1. [Crossref] [PubMed]
27. Shanyinde M, Pickering RM, Weatherall M. Questions asked and answered in pilot and feasibility randomized controlled trials. BMC Med Res Methodol 2011;11:117. [Crossref] [PubMed]
28. Crabtree BF, Miller WL. Doing qualitative research. 2nd edition. Thousand Oaks, CA: SAGE Publications Ltd., 2004.