Assessing perceptions of patient-centered care during telehealth visits provided via rural county Extension offices: an implementation pilot study

Introduction

Telehealth is defined as using technology and methods of telecommunication to virtually deliver healthcare to patients (1). Telehealth is an important emerging tool for delivering remote, comprehensive, healthcare services to patients using technology. Telehealth can be delivered both asynchronously and as a synchronous visit, which refers to a live telemedicine visit (1). Between 2005 and 2017, there was a 300% increase in telehealth use in the United States (U.S.), with 80% of these telehealth users residing in urban areas (2). The coronavirus disease 2019 (COVID-19) pandemic further solidified telehealth as an important healthcare tool. Notably, rates of use have increased among key demographics, with 21.1% of older adults using telehealth services currently, compared to 4.6% pre-pandemic (3) and over 70% of U.S. physicians intend to continue using telehealth in their practice post-pandemic (4). Thus, telehealth is a growing strategy that allows patients and providers to connect using computers, smartphones, and tablets as a supplement to in-person healthcare visits.

Telehealth is documented as an effective way to deliver quality healthcare that yields positive patient outcomes among telehealth users. For example, a nurse-facilitated, telehealth intervention for patients with persistently poor diabetes control found that telehealth was effective in increasing diabetes self-care, self-efficacy, and improving Hemoglobin A1C levels among intervention participants compared to control participants (5). Moreover, a narrative review of the effectiveness of telehealth among adults 55 years or older living in rural areas found a significant increase in weight loss among participants who attended at least two sessions of the Medicare Diabetes Prevention Program (MDPP) via telehealth. The MDPP delivered via telehealth also provided social support and was found to be well received by participants (6). Due to the growth of telehealth, it is important to explore how it impacts aspects of the patient-clinician relationship, including communication and patient perceptions of care among diverse populations.

Patient centered care (PCC)

PCC is a dynamic approach to fulfilling health-related needs that integrates patient’s goals, needs, abilities, and opinions while empowering patients to take an active role in their care (7). The concept of PCC evolved from the 1950’s practice of client-centered therapy and over time has developed into a wholistic model of care (8,9). While a variety of definitions, measures, and applications of PCC exist, the overarching goal is to foster a trustworthy relationship between patients and their healthcare providers (10,11), where patients are understood and cared for as people, not just treated for their condition (12,13). Key domains like patients’ perceptions of their illness experience, shared decision making, the quality of the patient-doctor relationship, and considering the patients as a whole person are critical components of PCC (14).

When exploring PCC from a health systems perspective, researchers study how clinicians interact with each other across a patient’s care journey and how easily a patient transitions from one mode of care to another. Alternatively, several aspects of the patient-clinician relationship that are important for PCC focus on the patient’s viewpoint. Specifically, Ryan and colleagues (15) outline three domains of PCC that focus on the patient-clinician relationship: (I) healthcare process (e.g., the direct interaction between a patient and their healthcare provider, are concerns understood and met and communicated well by provider?); (II) context and relationships (e.g., ongoing relationship between the patient and their provider including care, interest and trust); and (III) roles (e.g., does provider adequately encourage and explain patient-provider roles including listening, management, and advocacy?).

A variety of positive health outcomes are linked to PCC such as improved medication adherence, higher satisfaction with care, and improvements in self-management of conditions (15). For example, in a systematic review and meta-analysis of 19 randomized-controlled trials examining the effectiveness of PCC interventions among patients with type 2 diabetes, the authors found significant reductions in HbA1c levels and improved secondary outcomes such as diet-control in intervention groups compared to controls (16). In the same metanalysis, PCC interventions delivered in a community setting had the largest effects on HbA1c levels (16). Thus, PCC is associated with positive patient health outcomes and can be an effective strategy to facilitate community-based, self-management of chronic diseases.

The role of telehealth on PCC

Because PCC is a critical component of positive healthcare delivery, understanding how PCC is experienced by telehealth users can help inform strategies for delivering high quality health care. Previous studies have indicated that satisfaction with PCC (e.g., interpersonal skills, rapport, clinicians’ clinical competence) is similar for telehealth and in-person visits (17,18). In a 2022 meta-analysis of telehealth outcomes (19), found that 83% of the studies showed at least equal effectiveness of telehealth when compared to clinic visits. Similarly, Paige et al. (20) found high general satisfaction with patient centeredness of telehealth visits despite also identifying several disparities in telehealth use (i.e., individuals from rural areas and older age adults were less likely to use telehealth) and satisfaction (i.e., current smokers and patients with high distress were associated with less satisfaction with patient centeredness and provider expression of empathy).

Access disparities and barriers to telehealth use in rural areas

Forty-six million U.S. adults (or about 13% of the U.S. populations) reside in rural areas (21). The United States Department of Agriculture (USDA) defines geographically rural areas as nonmetropolitan areas (counties) using parameters like population and housing density [e.g., USDA Rural Urban Continuum Codes (RUCC) range from 1–9 where 9 represents nonmetro counties with fewer than 5,000 population] (22).

There is slower uptake of telehealth care in rural populations which may be attributed to limited broadband access (23,24). In fact, 22.3% of people living in rural areas lack access to the internet (25). Inadequate internet access including lack of suitable devices, are common barriers to telehealth use identified in research with rural U.S. residents (26). Transportation is also a common barrier to accessing healthcare, and in one study, a primary care clinic found that nearly a fifth of patients (17.65%) cited transportation difficulties as their main reason for a ‘no-show’ (27). Accessible telehealth services can increase patient attendance while reducing the burden of cost, time, and loss of economic productivity (17,28).

Negative perceptions of telehealth may also be a barrier for some populations, including geographically rural populations. One study found that, although rural U.S. residents understood telehealth could increase their access to healthcare, they were concerned that telehealth would make it difficult to maintain a positive patient-provider relationship (29). It is important to note, that despite this perception, after these same adults had personal experiences using telehealth, they were receptive to using it again and generally had very positive experiences using telehealth (29).

Rationale and knowledge gap

Telehealth is an emerging technology that may be an important tool for healthcare delivery, especially for individuals living in geographically rural communities who experience unique barriers to access (e.g., transportation, internet connectivity) as well as excess morbidity and mortality from chronic illnesses. Furthermore, interactions with a clinician and low/high-quality patient-provider communication (e.g., exchanging information, responding to emotions, making decisions, enabling patient self-management, managing uncertainly, fostering relationships) can predict important patient outcomes like satisfaction and adherence to recommendations (30,31). Thus, exploring telehealth interventions in community settings can help identify opportunities to support high quality healthcare access that consider the unique confluence of patients’ risk factors, environment, and experiences (32).

Objective and research questions

This article reports on a pilot study using a mixed methods design to explore different aspects of PCC while delivering rural telehealth services to US adults in their own communities. Specifically, TytoCare telehealth kiosks were placed at several University of Florida, Institute of Food and Agricultural Sciences (UF/IFAS) county Extension offices in high-need rural areas to facilitate visits in collaboration with adult endocrinology clinics. We present this article in accordance with the STROBE reporting checklist (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-24-47/rc) and answer the following research questions:

• (RQ1): to what degree do telehealth visits conducted using TytoCare kiosks in a rural community setting facilitate patient-centered care?
• (RQ2): how do self-reported scores on the revised Patient Perceptions of Patient-Centeredness (PPPC-R) questionnaire compare with domains of PCC derived from qualitative analysis of synchronous, audio-recorded, telehealth visits?

Methods

This study used a unique combination of implementation and communication science frameworks to assess and define best-practices strategies for implementing telehealth in rural settings. Specifically, we used the Consolidated Framework for Implementation Research (CFIR) to guide the investigation of factors that support telehealth implementation of UF/IFAS county Extension offices. As CFIR is widely used to inform implementation research in both community and clinical settings, it was highly applicable to the current pilot goal of assessing the implementation telehealth visits conducted in high-need, rural, Extension offices in Florida. We also base this study on concepts of patient centeredness and related theories of patient-centered care and patient centered communication, often explored in the interdisciplinary field of health communication (33). Specific aspects of the patient-clinician relationships informed by these two frameworks inform study design and analysis.

Prior funding from a separate research team was obtained from the USDA (grant No. FL0719-A63) for the procurement, installation, and networking of telehealth kiosks in 13 high-need, rural, Florida counties. Across the 13 counties, estimated time, distance, and money saved on travel from these counties was 5 hours, 324 miles, and $178 for each round trip (derived from google maps from Extension office to the academic medical center location).

The current, mixed-method, cross-sectional, pilot study was funded by a Clinical and Translational Science Institute (CTSI) pilot grant with the goal of assessing the implementation of telehealth kiosks in a small sample of the original counties (4 counties out of the original 13). The four counties had an average estimated travel burden of 2 hours, 114 miles, and $63 per round trip visit from the county to the academic health clinic. Regarding rurality, the 2023 RUCC codes for the four counties range from 6 to 9. The 2020 total population for the four counties ranged from 8,226 to 28,303. The pilot study research team was led by two assistant professors who also serve as Health and Wellness State Specialists within the UF/IFAS Extension system (M.J.V., L.J.O.). The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Institutional Review Board of University of Florida (IRB# 202201079) and all participants provided written informed consent prior to participating in the study.

Patients conducted their telehealth visits using TytoCare health kiosks that were available by appointments made in collaboration with the research team and healthcare team. The TytoCare kiosk systems allowed for interactive video conferencing through a tablet (e.g., iPad) and provided accessories to facilitate remote patient monitoring and directly relay patient data to the health care provider conducting the telehealth visit (Figure 1). For example, a scale, blood pressure cuff, otoscope and small portable camera are available to collect information during the patient visit so that the patient does not have to read measurements or readings to the provider. Patients could operate the kiosks on their own, but there was high assistance provided (technical support) by the research team on site for all visits. Clinical staff were all remote during visits but available to communicate with the research team as needed before, during, and after the patient visit. Infographics were created by the team and were available at each visit (Figure 2).

Figure 1 Image of TytoCare telehealth kiosk and accessories: blood pressure cuff, exam camera, tongue depressor, pulse oximeter, and otoscope that allow the direct import of vitals visible to the clinician’s dashboard. Patients can use the kiosks to log in to their patient poral and join the telehealth visit with their provider. Patients can use accessories for self-measurements as directed by the clinician. This image is published with the participants’ consent.

Figure 2 Informational resources created by Extension faculty to facilitate telehealth kiosk use by patients. These were printed, laminated and placed next to the telehealth kiosks. UF/IFAS, University of Florida, Institute of Food and Agricultural Sciences.

Research team and implementation partners

The research team of the current pilot study included Extension state-level (M.J.V., L.J.O.) and county-level faculty (H.C.), academic healthcare center physicians/nurse practitioners representing endocrinology (W.T.D., T.J.B.) and primary care (M.S., L.S.), and an implementation scientist (R.G.S.). A postgraduate research coordinator was also part of the research team (C.M.). Additionally, regular communication was required with faculty and staff from adjacent systems including information technology (IT). The technology team (IT staff) were responsible for ensuring the systems were functioning, secure, and appropriately set up in connection with the academic health system portal and were an integral part of the implementation process. A regular meeting schedule was set up to facilitate communication among all stakeholders including the research team members and the IT team members.

Patient recruitment

UF health patients were invited to enroll in the pilot study using a complimentary multi-strategy recruitment plan. The first enrollment was in January 2022 and continued through September 2023. We monitored recruitment and enrollment data weekly with a target enrollment goal 20 adults equally distributed (n=5) across Black men, Black women, White men, and White women. Participants were eligible if they were (I) UF health patients between 18–75 years of age with diagnosis of type 2 diabetes; (II) agreed to be contacted for research opportunities they might qualify for (i.e., Consent2Share); (III) were already established patients with a UF endocrinology clinic; (IV) had an upcoming appointment they would consider changing to a telehealth visit at their local county Extension office; and (V) had or were willing to sign up for a patient portal account which was needed to access the telehealth kiosks. Finally, recruitment largely occurred in two ways: (I) clinic referrals; and the (II) the integrated data repository (IDR).

Clinic referrals

Clinical members of the research team T.G., M.S., and T.D. supported the recruitment/enrollment of their own and colleagues’ patients from UF Health clinics. Potentially eligible patients with upcoming, in-person visits scheduled with a UF Endocrinology clinic identified in this way were informed of the study and if interested contacted by the research team’s research coordinator to discuss the opportunity to participant in more detail and confirm screening eligibility. At that point, the existing visit was rescheduled to a telehealth visit at their local county Extension office.

IDR

Eligible patients were also identified using the IDR patient lists derived from the electronic medical record. Patient lists in Excel files, were delivered in recurring weekly reports. Patient lists were pulled based on the inclusion criteria and the criteria to facilitate scheduling the visit [i.e., aged 18 to 75 years old, previous diagnosis of type 2 diabetes (International Classification of Diseases, Tenth Revision, ICD10), had an upcoming visit scheduled at an endocrinology clinic within the next 4 weeks (clinic codes used were: GP MED ENDO SUF and GP MED SPEC ENDO MP]. Patient lists contained the following data elements: patient name, medical record number, scheduled visit date, phone number, email, MyChart/patient portal status, zip code, race, and gender. This information was only provided to researchers if patients were enrolled in the Consent2Share program, which is patient preapproval to be contacted for research opportunities.

To identify patients living near any of the four rural county offices where telehealth kiosks were located, we filtered lists by zip code, then sent lists to our clinical collaborator to review for patient suitability (e.g., ensure patient did not have multiple visits in the same day, did not need labs or another test requiring in person visit). The research coordinator then invited the remaining patients on the list to consider participating in the pilot study, which would require changing their upcoming visit from an in-person visit to a telehealth visit at their local county Extension office. If patients were interested, a follow-up telephone call explained the details of the pilot study, described telehealth Kiosks and patient requirements (need to have a patient portal account username and password).

Scheduling the visit

Once patients agreed to update their visit, the research team coordinated with the clinic staff to confirm the date, location, and time of telehealth visit and update visit details from “in-clinic visit type” to “TytoCare telehealth visit type”. Of note, the TytoCare telehealth visit type was added as a new visit type during the course of the pilot. This visit type was in addition to the already established telehealth visit type (not at the county Extension office).

The process of updating the visit from in-person to telehealth required structured, efficient, and timely communication with the endocrinology clinic staff responsible for updating the visit type in the system. It also required coordination with County Extension office staff, as each county office had specific days and times when telehealth could be offered. Once the visit was scheduled and the visit type was updated, the research team confirmed the appointment with the Extension staff at the local Couty Extension office. Additionally, we separately sent an email confirmation of the visit to the research team, the provider conducting the visit (who was also a member of the research team), the telehealth lead from the IT/technology team (who attended several of the first in-person visits to assist with any technical difficulties), and Extension office contact.

Data collection

Telehealth visits at County Extension offices

Participants were advised to arrive at the County Extension office 30 minutes before their scheduled telehealth visit. Upon arrival, patients were greeted by the research team members (who traveled from the college town to the county office prior to the appointment to set up). Patients were provided with an informed consent document and given the chance to ask the research team any questions before signing the document. Once written informed consent was obtained, research team members helped participants log onto the telehealth kiosk [e.g., signing into the patient portal (MyChart) with an existing username and password]. When patients did not have a MyChart account set up, a member of the research team would facilitate the creation of one with the MyChart Help Desk. Once logged on, the research team messaged the health care provider scheduled to complete the visit to confirm that the participant was logged on for the scheduled visit on the telehealth kiosk.

To record the visit, two portable audio-recording devices placed next to the telehealth kiosk to capture the conversation between patient and provider. The research team members left the private room during the patient visit, waited outside, and were available if the patient needed assistance. When the visit was complete, the audio recorders were turned off. The audio was downloaded to a secure server and professionally transcribed. Finally, patients completed the VeggieMeter and a paper questionnaire which was later entered into the secure web-based platform, Research Electronic Data Capture (REDCap), by a research assistant. The questionnaire covered several topics including the quality of their visit, general health information, and socio-demographics. Participants received remuneration ($35 cash) upon completion of the questionnaire.

Patient visit process notes and follow-up interviews

Guided by CFIR constructs, the research team documented contextual information about each telehealth visit via process (e.g., internet issues at the rural county office, patient issues with locating office, etc.). These notes were documented on a paper and then entered into Qualtrics. A subsample of telehealth eligible adults (both who completed a telehealth visit with the study and those declined to complete a telehealth visit) were invited to conduct a follow-up interview via Zoom or phone to assess perceptions of telehealth at the Extension office. Interview and process data will be analyzed and reported elsewhere to more fully explore barriers and enablers of telehealth use.

Measures

A post-visit questionnaire included several measures, such as the PPPC-R, Telehealth Usability Questionnaire (TUQ), Patient Health Questionnaire (PHQ; shown to impact disease management and progression) and other outcomes (e.g., demographics, prior telehealth use, self-efficacy, satisfaction, etc.). Participants also completed a Veggie meter finger scan.

PPPC-R

This validated measure covers three domains of a patient’s appointment experience that participants rate on a Likert Scale from 1 to 4 where lower scores represent more patient centered experiences and higher scores represent less patient centered experiences [e.g., “how well do you think your provider understood you today? (1—very well, 2—well, 3—somewhat, 4—not at all)”]. Domains can be scored individually or combined to give a total score for the three domains of PCC: (I) healthcare process; (II) context and relationships; and (III) roles. However, the authors recommend calculation of an overall mean score based on all 18 items pending additional validation of subscales (15).

TUQ

This validated measure assesses the usability of telehealth implementation and services that takes utility and usability of the technology into account across several domains (e.g., usefulness, ease of use, interaction quality, and satisfaction) (34). The measure includes 21-items rated on a Likert scale where 1—strongly disagree, 2—disagree, 3—somewhat disagree, 4—neither agree nor disagree, 5—somewhat agree, 6—agree, 7—strongly agree. For this project we did not assess 2 domains [i.e., interface quality (n=3 items) and reliability (n=4 items)], resulting in a 14-item version rated on a 5-point Likert scale (1 is strongly disagree and 5 is strongly agree). We also added researcher generated items to assess previous telehealth use (yes/no) and satisfaction with telehealth at Extension (how satisfied were you with how well; your privacy was respected, your time was spent during the visit, and with the information you received during the visit?) on a 5-point Likert scale.

PHQ-8

The PHQ-8 is shown to impact disease management and progression and measures aspects of depression (35). The measure asks, over the last 2 weeks, how often have you been bothered by any of the eight listed problems. Response options include (0—not at all, 1—several days, 2—more than half the days, 3—nearly every day). A final item asks how difficult any of the selected problems made it difficult for you to work, take care of things at home or get along with others (1 to 4, with not difficult at all to extremely difficult). Scoring is the sum of the 8 items (score range from 0–24) with a score of 10 or greater is considered major depression, 20 or more is severe major depression.

Social Determinants of Health (SDoH)

SDoH were measured using the ten item Health Leads Screening tool. This measures the presence or absence (yes/no response options) of several social need’s domains (food security, health care access, childcare, housing, companionship, and health literacy) (36).

Veggie meter

We measured fruit and vegetable intake using the Veggie meter (37). The Veggie meter is a spectroscopy carotenoid measuring device that allows for a non-invasive measurement of the skin carotenoid concentration by scanning the individual’s fingertip three times. The Veggie meter then displays a score, ranging from 0–800, with a higher score representing a higher intake of fruits and vegetables. It also shows how the individual’s fruit and vegetable consumption compare to a reference population.

CAHPS clinician & group visit adult survey

We asked participants to think of the current telehealth visit to complete the CAHPS. This measure captures aspects of the patient visit: including communication with doctors, did the visit start on time, and patient satisfaction ratings. Response options for most items are: Yes, definitely; Yes, somewhat; or No. One item captures an overall rating of patient satisfaction (e.g., using any number from 0 to 10, where 0 is the worst experience possible and 10 is the best experience possible, what number would you use to rate your most recent telehealth visit at the extension office?). Two items were adapted to specifically obtain perceptions of staff at Extension office: (I) “Staff at the Extension office may talk with you about your visit, help set it up, and remind you about your appointment. Thinking about your most recent visit, did you talk to staff at the Extension office? (yes/no) if yes, was the staff as helpful as you thought they should be? (Yes, definitely/Yes, somewhat/No)”. A second question asked (II) “Did you need instructions from the Extension staff about how to use the telehealth kiosk for this visit? (Yes/No) if yes, did this Extension office give you all the instructions you needed to use the telehealth kiosk for this visit? (Yes, definitely/Yes, somewhat/No)” (38).

Participant demographics

Age, gender, health insurance, education, income, zip code, and current chronic conditions, and 5 questions about access to broadband/internet use from the Health Information National Trends Survey (HINTS 5) which surveys how U.S. adults’ access and use health information (39,40).

Data analysis

Statistical analysis

Descriptive statistics are provided for quantitative variables including mean, standard deviation, minimum, and maximum for continuous variables and frequency and proportion for categorical variables. Note that missing values were not included. Raw data consisted of 21 participants. Standard summary statistics stratified by County were calculated and between-group comparisons were conducted on participant demographic variables. Specifically, Fisher’s exact tests were used for categorical variables, and Kruskal-Wallis Rank Sum tests were used for continuous variables. Only relevant to determining differences in participant characteristics between counties, P values smaller than 0.05 indicated differences between counties. Statistical data analysis services were obtained from the Research Design and Data Coordinating Center, which is a part of UF’s CTSI Biostats Shared Resources.

Qualitative analysis

Transcripts of audio recorded patient visits were thematically coded for a priori instances of patient centered communication based on definition and constructs from Ryan and colleagues’ description of the revised perceptions of patient-centeredness self-reported scores on PPPC-R (15). Coding was conducted by two trained researchers (V.W. and I.B.) and confirmed by M.J.V. Qualitative data was managed in Excel, which housed both the codebook and coded portions of transcribed patient and provider visit conversations.

Results

Overall, 21 participants completed telehealth visits, that lasted between 10 and 20 minutes, at one of four rural, county Extension offices. There were no differences in any self-reported participant characteristics based on the county of enrollment. The following enrollment obtained by county was: county one: 2 (9.5%), county two: 10 (47.6%), county three: 5 (23.8%) and county four: 4 (19%). Overall, there were more women than men and more White participants than Black participants. Some participants reported some vocational training or some college education, with no education completed above an associate degree. Most participants reported annual household incomes below $30,000 (Table 1). Of note, self-reported data from one participant is missing, as the questionnaire data did not save upon entry, prompting a protocol change to collect questionnaire data via paper for all subsequent participants.

Participants (n=20) who completed the PPPC-R questionnaire had an overall mean [standard deviation (SD)] score of 20.45 (4.66), with a range of 18–37. The lowest possible score on the PPPC-R is 18 and the highest possible score is 72, with lower scores indicating more/better patient-centered experiences during the visit. Mean (SD) scores for individual items within the 3 domains (i.e., healthcare process, context and relationship, and roles) are reported alongside corresponding segments of participant-provider dialogue captured during the telehealth visit (Table 2). All the self-reported PPPC-R scores were low (indicating higher quality perceptions of patient centeredness). Notably, only ratings of 1 or 2 were recorded on a 4-point Likert scale.

All transcripts of the audio-recorded telehealth visits, when analyzed, contained examples reflective of each of the three domains measured in the PPPC-R questionnaire. Consistent with the self-reported PPPC-R scores, the narrative exemplars of patient centeredness, derived from analysis of the transcripts, provide specific, contextual examples of high-quality patient-provider interactions. Most of the dialogue reflected positive interactions across each of the three factors (aka domains).

Factor 1: healthcare process

Many of the quotes coded to factor 1 (e.g., healthcare process) include a patient describing a concern and the provider responding with a tangible solution (e.g., prescribing a device/medication or ordering a lab or test to assess progression towards a goal). For example:

PatientBrad6: Also, I need a refill for my NovoLog.

Provider: Yes ma’am. Absolutely. So, I can definitely do that as well. Do you need any other refills?

PatientBrad6: It probably wouldn’t hurt to get the Lantus refilled as well. I mean I still have two pens left and I’m on a half a pen now, if that makes sense.

Provider: Yes ma’am, but we don’t want you to ever run out, so I’ll make sure I do that. We’ll get all those things ordered for you.

PatientBrad6: Okay.

In other narrative examples of factor 1: healthcare process, the patient expresses agreement or satisfaction with the provider’s suggestion (e.g., Provider: “How much walking are you doing?” PatientBrad5: “I’m not doing any walking and I need to. I’ll add that in.” Provider: “Okay.”) or there is an exchange and interaction related to collecting biometrics data (pulse, blood pressure, and weight) using the telehealth kiosks TytoCare accessories (e.g., Provider: “So they’re on your desk. You should see a blood pressure cuff. Can you take your blood pressure with the blood pressure cuff?” Brad6: “Yes, does it matter where it has to be?” Provider: “No, ma’am, it doesn’t matter which arm.” Brad6: “Okay.”).

Factor 2: context and relationship

Individual items on the PPPC-R questionnaire were scored low (more patient centeredness) with scores obtained in this pilot study ranging from 1–3 on the 4-point scale and all individual items with a mean score below 2. Likewise patient provider dialogue coded to this factor highlights interactions demonstrating the health providers concern for the person beyond their role as a patient, including concern for patient’s opinions and feelings, including but not limited to their medical condition. Narrative exemplars highlighted in Table 2 demonstrate trust building dialogue that reflect empathy for the burden of driving long distances for health care and discussion of patients’ experiences with food assistance programs.

Factor 3: roles

For factor 3: roles, we found scores ranged from 1 to 4 on the 2-item factor. Authors of the measure indicate that factor 3: roles, has the lowest amount of internal consistency, which is something to consider for the interpretation of the self-reported outcomes. Despite this, each individual item had mean scores close to 1 as can be seen in Table 2. Furthermore, dialogue in this factor reflects a discussion of roles that the patient and clinician will play in the patient’s care. Some discussions in this factor included setting priorities and making plans with explicit discussion of who will be involved. For example, the provider clarifying future visit plans was coded to factor 3 [e.g., Provider: “You will still have your visit with Dr. (redacted name) in March”].

Additional self-reported variables collected are reported in Table 3. These results demonstrate key descriptions of patient characteristics that are relevant for understanding the context in which telehealth visits were conducted and perceived by the patient. For example, some of the self-reported behaviors reveal a majority of study participants, 61.9% had never used telehealth before. Assessments of several social determinants of health revealed prevalence of issues related to food assistance access (52.4%), income insufficiency (42.9%), and lack of companionship (38.1%). In terms of telehealth usability and satisfaction with the visit, patient-reported scores indicated telehealth kiosks were perceived as highly usable and participants had high satisfaction with all aspects of the visit assessed.

Finally, there were several insights about system-level findings obtained from process notes collected during recruitment, scheduling, and on-site telehealth visits. These were factors like unanticipated events that disrupted implementation, aspects of information technology infrastructure, and work infrastructure as outlined in CFIR domains. There were several challenges with scheduling patients including establishing scheduling workflows, clerical errors with cancellations, issues with limited blocks of time to schedule visits, and needing to create new visit types in the scheduling system for Extension telehealth visits specifically (which were different than general telehealth). System-level challenges related to technology included many sites requiring hot spots, several instances of internet or Tytocare systems being down, and issues with logging into the telehealth kiosks (e.g., patients forgetting their passwords and emails addresses). Additionally, challenges with staffing and concerns for long-term sustainability were documented in process note. Many of these concepts were also addressed in follow-up interviews conducted with stakeholders. While briefly summarized here, the details of the fully analyzed process notes and stakeholder interviews will be presented in a separate article describing the nuances and implications of telehealth implementation from these perspectives.

Discussion

In this implementation pilot study, we assessed patient perceptions of the quality of telehealth visits conducted at rural, UF/IFAS, County Extension offices. Utilizing existing internet infrastructure and portable hotspots at Extension offices facilitated telehealth access and provided an opportunity to assess several aspects of implementation that serve as indicators of high satisfaction and high-quality care. In our pilot study, only 35% of participants had ever used telehealth prior to enrolling and 30% were not satisfied with their current internet speed suggesting there was a need to support access in the communities where the pilot study took place. This also supports the importance of exploring locations in rural communities that might successfully facilitate telehealth use.

Contrary to our findings, which demonstrate an overall positive patient experience with telehealth, a common barrier documented in the existing literature is the perception that telehealth visits will not provide the same high-quality interaction that patients come to expect from an in-person clinic visit. For example, Alexander and colleagues (41) explored patient perceptions of telehealth in rural North Carolina finding comfort and trust, among other barriers, must be addressed to increase patient willingness to use telehealth. A patient in their study indicated: “I feel like, that a lot of people would not feel comfortable unless it (doctor’s visit) was face-to-face, looking a person in the eye, and talking to that person about their health.” (pg. 4). Perceptions that telehealth visit would be less acceptable than an in-person visit was echoed as a common barrier across several studies (24). Despite this well documented perspective, our findings indicate an overall positive patient response to the real-time telehealth visit.

Regarding the specific construct of patient provider communication (PPC), our findings on PPC are overall positive as reflected on the PPPC-R scores. In contrast, conclusions in Nikolic et al. (26) indicate mixed findings on PPC among telehealth users. Such differences may be due to how this variable is measured and interpreted in both studies. For example, self-report scores obtained from the PPPC-R measures are collected in the current study compared to qualitatively derived insights on PPC in the study by Nikolic and colleagues. Thus, our findings add a slightly different perspective to the literature exploring perceptions of online PPC.

Additionally, because we obtain both self-report (PPPC-R scores) and real-time objective assessments (telehealth visit conversations recorded, transcribed, and analyzed qualitatively), we feel our findings provide a nuanced understanding of patient perceptions of telehealth. We found overall positive ratings across all domains of patient centeredness from self-reported data collected after a patient visit. The narrative examples of patient-provider interactions supplement and support the self-reported findings of high patient-centeredness. Transcripts revealed positive interactions across each of the three domains of patient centeredness: healthcare process, context and relationship, and roles. Thus, despite existing literature indicating that rural adult populations value receiving health information in a clinic, our findings suggest that upon completing a telehealth visit, rural telehealth patients in the current pilot study rated the experience very positively.

To date, very few studies in the U.S. explore how non-traditional settings (public spaces accessible and with internet) may be used to support telehealth visits. One study to explore this provided access at public libraries in rural areas to establish available spaces for telemedicine for the community (42). This study found that due to the lack of broadband in many rural homes, the availability of free high-speed internet for public use in libraries gave patients the ability to receive a form of care otherwise inaccessible to them. Similarly, Sundstrom et al. (43) explored perceptions of telehealth in rural South Carolina using a community center. The limited number of prior US based research on this topic may point to a gap in telehealth implementation research. Future research can explore how attempts at providing telehealth appointments from community-based settings near a patient’s home may help to mitigate rural access disparities that stem from poor or no internet access within patients’ homes or other access barriers.

This pilot study also suggests a possible non-traditional telehealth space for adults residing in rural communities, is local County Extension offices, which serve to bring educational resources out into the community to promote safe and healthy living. Furthermore, use of telehealth at high need rural county Extension offices not only provides access to care due to the scheduled visit but opens avenues to expand community engagement, as extension offices are a hub of resources and Extension agents (aka county faculty) considered knowledge brokers at the community level (44).

Key findings

Telehealth may be an effective strategy to alleviate healthcare access barriers related in rural areas. In this study, patients from rural communities, receiving care for type 2 diabetes via a single telehealth visit experienced high-quality visits. They reported scores on the revised PPPC-R known to be associated with high quality care and positive patient health outcomes. Self-reported findings were supported by thematically coded examples of key domains of patient centeredness derived from audio recordings of the patient’s telehealth visit.

Our findings imply that, from the patient’s perspective, community-based settings, such as local County Extension offices, may be appropriate for increasing telehealth access in rural areas. Telehealth was also found to be acceptable among patients seeking to conduct clinical visits closer to home. Specifically, findings indicate patients perceived the telehealth visits to be highly patient centered, which has important implications for counterarguments that telehealth visits are not as high-quality or as hands on as an in-person visit would be.

Understanding the role of PCC among telehealth users is valuable due to telehealth’s growth and because PCC is a critical component of positive healthcare delivery. Although our pilot study did not compare telehealth visit ratings with in-person clinic visits, findings did however indicate PCC ratings indicated high levels of patient satisfaction. Given that previous studies have indicated that satisfaction with PCC (e.g., interpersonal skills, rapport, clinicians’ clinical competence) is similar for telehealth and in-person visits (18) it is reasonable to suggest that rural adult telehealth users in this study had a patient experience that was, at least, as good as an in-person visit would provide.

However, it is still important to consider other implementation factors that may impact overall sustainability of telehealth in community-based settings. Particularly, during some patient visits in the current study, there were occasional technical issues regarding Wi-Fi, equipment access, and connectivity. Also, the research team was very hands-on in the appointment process, with several patients informally expressing that they would not have been able to complete the appointment without the help of the research staff. Finally, patients also seemed to imply that they saw the post visit questionnaire as an evaluation of the research team’s performance and wanted to be sure to “leave a good review”. Thus, the research team’s presence may have impacted ratings of the system’s actual useability which may have been different if the team was not there to help.

Strengths and limitations

Several aspects of the pilot study strengthened the findings. First, we leveraged existing infrastructure including telehealth equipment/systems obtained from a prior grant as well as the well-established network of Extension offices. Furthermore, we leveraged pilot funding to build an interdisciplinary team of researchers, clinicians, internet technology experts, and faculty across several relevant disciplines (public health, behavioral science, sociology, implementation science, and medicine). We also selected locations with previously noted disparities and limited resources to ensure we would reach a population that would benefit from the intervention. Finally, the audio-recorded telehealth patient visits allowed the research team to capture, real-time, objective data from telehealth kiosks without interrupting clinical workflow. This allowed us to supplement the self-reported data as well as the follow-up interviews discussing the patients’ reflections on the visit.

Limitations spanned from recruitment challenges to unique challenges of community-based work. First, some recruitment challenges were because one provider (nurse practitioner) saw all patients in the pilot study. Thus, some potentially eligible participants who preferred to see a different provider who may have participated otherwise were not enrolled. Additionally, providers from the endocrinology clinic did not engage in an intra-clinic patient referral protocol. Only one provider referred their patients residing in rural areas to the local Extension offices for telehealth services for upcoming appointments. There were also several scheduling challenges that arose as visits depended on the availability of the Extension offices, pilot study research staff, and the endocrinology provider. Thus, concerns with warm glow effect (45) and not wanting to evaluate the research team harshly may have impacted responses. These limitations and challenges represent situations that may have skewed participant responses. Finally due to the small sample size and study goals being to assess implementation, findings may not be generalizable to other populations

Comparison with similar research

Prior work has found that privacy and confidentiality concerns and lack of provider-patient connection are barriers to using telehealth that typically decrease and tend to not be concerned once participants can use telemedicine (46-49). Similarly, participants in the current pilot study rated the telehealth visit positively regarding privacy and satisfaction. It should also be noted that the current measure of patient centeredness used in this study (PPPC-R) as well as most self-reported measures are evaluations of the specific visit and may exclude other important contextual components that impact perceptions of patient centeredness. This point is discussed in a systematic review of several patient centeredness measures, where authors describe how all measures exclude components that may impact patient perceptions of patient centeredness and high-quality care that extend outside of the actual visit (50).

Assessments of PCC may benefit from the use of multiple assessments or by triangulation of findings. Especially in an environment where patients are conducting a visit using technology or from a non-traditional setting (e.g., using telehealth at home, a library, or another community location). For example, this implementation pilot study analyzed audio recording transcripts of the patient visit as well as follow-up interviews with patients to gain a better understanding of patient perceptions that may be unique to the telehealth context.

Explanations of findings

Rural residents experience both external and internal barriers to quality healthcare. External barriers, such as fewer healthcare providers and limited broadband access in rural communities have resulted in increased burden on rural residents as they seek access to care (25). For example, rural residents often drive longer distances to see a specialist (51). Additionally, internal barriers like patient perceptions of reduced quality patient-provider interactions during telehealth visits may contribute to lower telehealth usage among rural residents (24,29). Because telehealth can facilitate information-sharing and clinical interactions between patients and providers, we sought to explore ways this technology may help remove health care access barriers for adults in rural communities (52).

Overall, findings from this study demonstrate the ability of telehealth to reduce the burden associated with both external and internal barriers to quality care in rural communities. Reduction of external factors may have presented in the fact that county Extension offices transformed spaces to accommodate patients and equipment. Patients found that traveling to a local site where they were able to communicate with their provider in a private location using state of the art technology, saved them time, including time off work, and money when compared to the average travel distance of 1 to 1.5 hours for specialty care. Internal factors, similarly may be that patients enrolled in the current pilot study, overall had well controlled diabetes which meant all telehealth appointments were for routine maintenance. Implementation of telehealth care via County Extension offices was seen as acceptable by patients for this purpose.

While we reported overall positive outcomes in our pilot study, there were several other factors that could have impacted results. First, the single provider that conducted all visits was a nurse practitioner, who had a focused expertise in supporting/advocating for rural patients and engaging in telehealth, which may have positively skewed results. Additionally, the provider was aware that patients were being seen in a local community-based setting with access to telehealth equipment and accessories that would enhance the visit. The provider walked patients through steps to take their own weight and blood pressure (using blue tooth technology, data were automatically uploaded to patient charts). Thus, patients might have felt more empowered by learning how to check their numbers and by engaging with the provider in a new way. Second, the research team was very involved in the scheduling and facilitation of the in-person telehealth visits, thus findings might not reflect the telehealth visit alone, rather the whole visit including communications with the research team.

Despite several documented patient benefits, and overall positive patient experiences with telehealth in the current study, it is important to consider cost barriers or other challenges that may impact overall success of using telehealth in non-traditional settings. For example, this pilot study assessed the implementation of telehealth kiosks and systems that had already been set up in several rural counties. The process of setting up the kiosks, linking them to the University Health Systems Electronic Health Record and ensuring compliance with secure information exchange was facilitated by IT support from personnel overseeing these important precursors to implementation and assessment of implementation. Implementation of telehealth in a location outside of a patient’s home may be impacted by financial constraints. For example, in the 2016 National Emergency Department (ED) Inventory survey conducted in rural EDs, it was found that EDs that considered including telehealth services at their location were not able to do so because of the costs associated with implementing telehealth.

Implications and actions needed

Access to telehealth via community settings may be an important feature of effective telehealth in rural areas. Due to the paucity of research in this area in the U.S., there is a need to continue to explore potential impacts of such strategies. Also, the use of telehealth at high need rural county Extension offices may present additional benefits to a patient because Extension offices are a hub of resources and Extension agents considered knowledge brokers at the community level (53,54). The UF/IFAS Extension is a statewide, integrated network of offices and faculty. One Extension goal is making evidence-based knowledge available to communities through non-formal education and health programming. County offices are located throughout the state and serve as a hub for evidence-based information and offices for trained county faculty (55,56). Exploring the full context of multi-level impacts including across a variety of stakeholders will be important to understand the total impact that this strategy may have.

Conclusions

This pilot study demonstrates the capability of telehealth-based healthcare approaches, in rural community settings, to provide patient-centered care comparable to in-person visits. Adults residing in geographically rural areas reported high ratings of PPPC-R values which were mirrored in analysis of patient-provider communication encounters. Telehealth can impact rural patients’ healthcare access and is perceived as an option for high quality care among adults managing chronic conditions such as type 2 diabetes. To understand the full impact of telehealth on rural health disparities, it is imperative that future studies investigate patient-centered care using telehealth approaches.

Overall, this study allows for unique insights into the fields of health implementation science, and patient care. The telehealth pilot study has shown the feasibility and success in using Extension resources to create non-traditional healthcare settings that deliver high quality patient care to rural communities. This study provides a unique foundational framework that will allow for increased diverse resources to become accessible to rural adults seeking healthcare. Furthermore, this study shows the benefits of non-traditional telehealth for rural communities by not only removing accessibility barriers, but also creating high standards of quality and patient-centered care. Patients can receive personalized care through this telehealth initiative, creating a viable and accessible opportunity for rural communities seeking healthcare. These preliminary contributions suggest a need to further investigate the implementation of additional telehealth initiatives, as well as the benefits surrounding patient care through telehealth.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-24-47/rc

Data Sharing Statement: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-24-47/dss

Peer Review File: Available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-24-47/prf

Funding: This research was supported by the University of Florida Clinical and Translational Science Institute, which is supported in part by the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (award No. UL1TR001427). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://mhealth.amegroups.com/article/view/10.21037/mhealth-24-47/coif). All authors report that this research was supported by the University of Florida Clinical and Translational Science Institute, which is supported in part by the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (award No. UL1TR001427). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. M.S. received grant funding for serving as a site PI for Guardant Health, honoraria for serving as an Advisor for Exact Sciences, and teaching at the American Canadian School of Medicine. The authors have no other conflicts of interest to disclose.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Institutional Review Board of University of Florida (IRB#202201079) and all participants provided written informed consent prior to participating in the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Mechanic OJ, Persaud Y, Kimball AB. Telehealth Systems. 2025.
2. Barnett ML, Ray KN, Souza J, et al. Trends in Telemedicine Use in a Large Commercially Insured Population, 2005-2017. JAMA 2018;320:2147-9. [Crossref] [PubMed]
3. Choi NG, DiNitto DM, Marti CN, et al. Telehealth Use Among Older Adults During COVID-19: Associations With Sociodemographic and Health Characteristics, Technology Device Ownership, and Technology Learning. J Appl Gerontol 2022;41:600-9. [Crossref] [PubMed]
4. Callaghan T, McCord C, Washburn D, et al. The Changing Nature of Telehealth Use by Primary Care Physicians in the United States. J Prim Care Community Health 2022;13:21501319221110418. [Crossref] [PubMed]
5. Crowley MJ, Tarkington PE, Bosworth HB, et al. Effect of a Comprehensive Telehealth Intervention vs Telemonitoring and Care Coordination in Patients With Persistently Poor Type 2 Diabetes Control: A Randomized Clinical Trial. JAMA Intern Med 2022;182:943-52. [Crossref] [PubMed]
6. Ritchie ND, Turk MT. Enhancing access and impact of the Medicare Diabetes Prevention Program using telehealth: a narrative review. Mhealth 2024;10:10. [Crossref] [PubMed]
7. Ford S. Patient-centered Medicine, Transforming the Clinical Method (2nd edition). Health Expect 2004;7:181-2.
8. ROGERS CR. Significant aspects of client-centered therapy. Am Psychol 1946;1:415-22. [Crossref] [PubMed]
9. Laine C, Davidoff F. Patient-centered medicine. A professional evolution. JAMA 1996;275:152-6. [Crossref] [PubMed]
10. Hudon C, Fortin M, Haggerty JL, et al. Measuring patients' perceptions of patient-centered care: a systematic review of tools for family medicine. Ann Fam Med 2011;9:155-64. [Crossref] [PubMed]
11. Mead N, Bower P. Patient-centredness: a conceptual framework and review of the empirical literature. Soc Sci Med 2000;51:1087-110. [Crossref] [PubMed]
12. Epstein RM, Street RL Jr. The values and value of patient-centered care. Ann Fam Med 2011;9:100-3. [Crossref] [PubMed]
13. Levinson W. Patient-centred communication: a sophisticated procedure. BMJ Qual Saf 2011;20:823-5. [Crossref] [PubMed]
14. Stewart M, Brown JB, Weston WW, et al. Patient-centered medicine: transforming the clinical method. 4th ed. Boca Raton: CRC Press; 2024.
15. Ryan BL, Brown JB, Tremblay PF, et al. Measuring Patients' Perceptions of Health Care Encounters: Examining the Factor Structure of the Revised Patient Perception of Patient-Centeredness (PPPC-R) Questionnaire. J Patient Cent Res Rev 2019;6:192-202. [Crossref] [PubMed]
16. Asmat K, Dhamani K, Gul R, et al. The effectiveness of patient-centered care vs. usual care in type 2 diabetes self-management: A systematic review and meta-analysis. Front Public Health 2022;10:994766. [Crossref] [PubMed]
17. Osman S, Churruca K, Ellis LA, et al. The Unintended Consequences of Telehealth in Australia: Critical Interpretive Synthesis. J Med Internet Res 2024;26:e57848. [Crossref] [PubMed]
18. Agha Z, Schapira RM, Laud PW, et al. Patient satisfaction with physician-patient communication during telemedicine. Telemed J E Health 2009;15:830-9. [Crossref] [PubMed]
19. Edmiston KD, Alzubi J. Center for Insurance Policy & Research Report. 2022;
20. Paige SR, Bunnell BE, Bylund CL. Disparities in Patient-Centered Communication via Telemedicine. Telemed J E Health 2022;28:212-8. [Crossref] [PubMed]
21. Davis JC, Cromartie J, Farrigan T, et al. Rural America at a glance: 2023 edition (Report No. EIB-261). 2023 Nov. U.S. Department of Agriculture, Economic Research Service.
22. U.S. Department of Agriculture Service. Rural-Urban Continuum Codes. [updated 2024 Jan 22; cited 2024 Aug]. Available online: https://www.ers.usda.gov/data-products/rural-urban-continuum-codes.aspx
23. Chu C, Cram P, Pang A, et al. Rural Telemedicine Use Before and During the COVID-19 Pandemic: Repeated Cross-sectional Study. J Med Internet Res 2021;23:e26960. [Crossref] [PubMed]
24. Pullyblank K. A Scoping Literature Review of Rural Beliefs and Attitudes toward Telehealth Utilization. West J Nurs Res 2023;45:375-84. [Crossref] [PubMed]
25. Federal Communications Commission. 2020 broadband deployment report. Washington, D.C.; Apr. 24, 2020.
26. Nikolic-Khatatbeh J, Crowe J, Li R. Navigating telehealth challenges: rural perspectives on technology access and trust. J Rural Community Dev 2024;19:167-80.
27. Priya L, Carey P, Shafi F. Conversion of No-Show Patients to Telehealth in a Primary Medicine Clinic. Mo Med 2022;119:74-8. [PubMed]
28. Snoswell CL, Smith AC, Page M, et al. Quantifying the Societal Benefits From Telehealth: Productivity and Reduced Travel. Value Health Reg Issues 2022;28:61-6. [Crossref] [PubMed]
29. Holtz B, Mitchell K, Hirko K, et al. Using the Technology Acceptance Model to Characterize Barriers and Opportunities of Telemedicine in Rural Populations: Survey and Interview Study. JMIR Form Res 2022;6:e35130. [Crossref] [PubMed]
30. Epstein RM, Street Jr. RL. Patient-centered communication in cancer care: promoting healing and reducing suffering. National Cancer Institute, NIH Publication No. 07-6225. Bethesda, MD, 2007.
31. McCormack LA, Treiman K, Rupert D, et al. Measuring patient-centered communication in cancer care: a literature review and the development of a systematic approach. Soc Sci Med 2011;72:1085-95. [Crossref] [PubMed]
32. Noordman J, Driesenaar JA, van Bruinessen IR, et al. ListeningTime; participatory development of a web-based preparatory communication tool for elderly cancer patients and their healthcare providers. Internet Interv 2017;9:51-6. [Crossref] [PubMed]
33. Ishikawa H, Hashimoto H, Kiuchi T. The evolving concept of "patient-centeredness" in patient-physician communication research. Soc Sci Med 2013;96:147-53. [Crossref] [PubMed]
34. Parmanto B, Lewis AN Jr, Graham KM, et al. Development of the Telehealth Usability Questionnaire (TUQ). Int J Telerehabil 2016;8:3-10. [Crossref] [PubMed]
35. Kroenke K, Strine TW, Spitzer RL, et al. The PHQ-8 as a measure of current depression in the general population. J Affect Disord 2009;114:163-73. [Crossref] [PubMed]
36. Health Leads. Social Needs Screening Toolkit. 2018;1–21. Available online: https://healthleadsusa.org/wp-content/uploads/2023/05/Screening_Toolkit_2018.pdf
37. Radtke MD, Poe M, Stookey J, et al. Recommendations for the Use of the Veggie Meter® for Spectroscopy-Based Skin Carotenoid Measurements in the Research Setting. Curr Dev Nutr 2021;5:nzab104. [Crossref] [PubMed]
38. Dyer N, Sorra JS, Smith SA, et al. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Adult Visit Survey. Med Care 2012;50:S28-34. [Crossref] [PubMed]
39. Blake K. National Cancer Institute, NIH; NCI Hub. 2020. HINTS 5 CYCLE 4. Available online: https://hints.cancer.gov/docs/methodologyreports/HINTS5_Cycle4_MethodologyReport.pdf
40. National Cancer Institute. National Cancer Institute Website. Health information national trends survey: About HINTS. Available online: https://hints.cancer.gov/
41. Alexander DS, Kiser S, North S, et al. Exploring community members' perceptions to adopt a Tele-COPD program in rural counties. Explor Res Clin Soc Pharm 2021;2:100023. [Crossref] [PubMed]
42. DeGuzman PB, Abooali S, Jain N, et al. Improving equitable access to care via telemedicine in rural public libraries. Public Health Nurs 2022;39:431-7. [Crossref] [PubMed]
43. Sundstrom B, DeMaria AL, Ferrara M, et al. "People are struggling in this area:" a qualitative study of women's perspectives of telehealth in rural South Carolina. Women Health 2020;60:352-65. [Crossref] [PubMed]
44. Buys D, Editor G, Koukel S. The national framework for health and wellness: (re)framing the work of cooperative extension for the next century. J Hum Sci Ext 2018;6: [Crossref]
45. Monin B. The warm glow heuristic: when liking leads to familiarity. J Pers Soc Psychol 2003;85:1035-48. [Crossref] [PubMed]
46. Houser SH, Flite CA, Foster SL. Privacy and Security Risk Factors Related to Telehealth Services - A Systematic Review. Perspect Health Inf Manag 2023;20:1f. [PubMed]
47. Thomas NA, Drewry A, Racine Passmore S, et al. Patient perceptions, opinions and satisfaction of telehealth with remote blood pressure monitoring postpartum. BMC Pregnancy Childbirth 2021;21:153. [Crossref] [PubMed]
48. Puzzitiello RN, Moverman MA, Pagani NR, et al. Public perceptions and disparities in access to telehealth orthopaedic services in the COVID-19 era. J Natl Med Assoc 2021;113:405-13. [Crossref] [PubMed]
49. Zayde A, Kilbride A, Kucer A, et al. Connection During COVID-19: Pilot Study of a Telehealth Group Parenting Intervention. Am J Psychother 2022;75:67-74. [Crossref] [PubMed]
50. Hudon C, Fortin M, Haggerty JL, et al. Measuring patients' perceptions of patient-centered care: a systematic review of tools for family medicine. Ann Fam Med 2011;9:155-64. [Crossref] [PubMed]
51. Bolin JN, Bellamy GR, Ferdinand AO, et al. Rural Healthy People 2020: New Decade, Same Challenges. J Rural Health 2015;31:326-33. [Crossref] [PubMed]
52. Cortelyou-Ward K, Atkins DN, Noblin A, et al. Navigating the Digital Divide: Barriers to Telehealth in Rural Areas. J Health Care Poor Underserved 2020;31:1546-56. [Crossref] [PubMed]
53. Dwyer JW, Contreras D, Eschbach CL, et al. Cooperative Extension as a Framework for Health Extension: The Michigan State University Model. Acad Med 2017;92:1416-20. [Crossref] [PubMed]
54. Gutter MS, O'Neal LJ, Riportella R, et al. Promoting community health collaboration between CTSA programs and Cooperative Extension to advance rural health equity: Insights from a national Un-Meeting. J Clin Transl Sci 2020;4:377-83. [Crossref] [PubMed]
55. Harder A, Harder LK. Interpersonal Leadership Competencies of Extension Agents in Florida. J Agric Educ 2019;60:224-33. [Crossref]
56. Gutter MS. A Case Study of Extension in Florida. Fam Consum Sci Res J 2016;44:245-54. [Crossref]