“It depends:” a qualitative study on digital health academic-industry collaboration

Introduction

Information and communications technologies (ICT) designed to address health (1), (i.e., digital health), offer promise in health promotion and disease prevention. The growing body of evidence highlights the efficacy in using mobile devices, applications, computers, and wearables (e.g., sensors) for health-related initiatives. The use of technology, informatics, big data, and artificial intelligence increasingly supports chronic disease management, health-system improvements, and health behavior change (2). The emergence of the coronavirus 2019 (COVID-19) pandemic accelerates the adoption and acceptability of digital health at unprecedented rates (3).

Despite the opportunity to advance public health with technology, various challenges exist, limiting public health impact. Technology evolves rapidly, thus continually demanding interventionists, technologists, and researchers to adapt to challenges regarding reach, engagement, quality, and sustainability (4). Although the enthusiasm for digital health remains strong across sectors, many times academia and industry are “operating in siloes, rather than collaboratively” (5). These challenges further complicate the digital health field, impeding public health efforts.

Academic-industry collaborations (AICs) are often endorsed by global public health organizations to alleviate these challenges due to each sector’s unique strengths (1,5,6). AIC can be defined as a mutually beneficial partnership with two or more stakeholders joined in a collaborative undertaking to maximize impact (7,8). Academia (i.e., institution for higher education, research, and scholarship) demands scientific rigor, ethical human subjects treatment, and prioritizes hard-to-reach populations. Oftentimes, academia is timebound to historically slow institutional processes and declining research dollars. Academia’s’ impact is halted at the end of a grant cycle, thwarting dissemination and efforts to scale. Whereas industry (i.e., a for-profit technology-related corporation aiming to earn profit based on own interests) thrives in a ‘fail fast’ environment, designed with commercialization in mind. Oftentimes industry lacks clinical validation, prioritizes profit over impact, and products are primarily designed for affluent customers. To blend each sector’s strengths, a surge of innovative collaboration models have emerged specifically to digital health and health innovation (9), convening industry, and academia in a unique way.

Historically, AICs yielded lifesaving discoveries in research, pharmacology, and genomics. Biotechnology AIC merged large pharmaceutical companies and prestigious universities (e.g., Novartis International AD and Harvard University) for research and development efforts (7,10-12).Whereas most recently, digital health AIC expands on traditional R&D, yielding innovative mechanisms for collaboration particularly with smaller digital health enterprises (e.g., startups) (9,13).

Despite the emergence and endorsement of innovative digital health AICs, collaboration remains understudied (5,14-17). Little is known from the real world (18), particularly the mechanisms of collaboration, contextual factors, and partnership dynamics among highly collaborative organizations. In accordance with the MDAR checklist (available at http://dx.doi.org/10.21037/mhealth-20-140), the objective of the qualitative study was to initially explore investigate collaboration experiences between digital health companies and academic institutions.

Methods

Methodology

The research team employed a phenomenological approach (19) to investigate how academia and industry collaborate when working in digital health. Ethical approval was obtained by The Colorado Institutional Review Board and approved this study (IRB#: 19-2508). All study participants obtained and electronic consent prior to data collection. The study conformed to the provisions of the Declaration of Helsinki (as revised in 2013).

Conceptual model

The theoretical mechanisms of AIC in digital health remain underexplored. The conceptual model for the study merged key elements from the Business Model for the University-Industry Collaboration in Open Innovation (20) and a known collaboration measure in the literature, assessing the extent of collaboration (EC) and collaboration dynamics: relational, climate, expectations (REC) (21). (Figure 1) (22).

Figure 1 Conceptual model: digital health academic industry collaboration.

Settings/populations

Initially, participants were purposively sampled (23) from companies listed in a funding database curated by Rock Health, a San Francisco-based venture fund aggregating data from digital health startups in the United States (2011–2019) (24). Inclusion criteria included individuals from startups (also referred to as small/medium-sized enterprises) dedicated to a public health issue (e.g., behavioral health, chronic disease) who previously or currently partner with academia. We utilized a positive deviance framework to investigate high-functioning AIC in digital health (25-27). Positive deviance remains an appropriate theoretical framework to understand high performing entities in healthcare organizations (26,27). Purposeful maximum variation sampling (28,29) included a diverse sample of participants (e.g., project managers, academic directors, researchers, entrepreneurs, industry affiliates). All interviews occurred via telephone or using remote video conference calls.

Recruitment

We approached industry participants (n=36) using email and social media outlets (i.e., Twitter and Linkedin). Two participants declined due to busy schedules (i.e., COVID-19 response) and 10 individuals did not reply to recruitment message. Snowball sampling captured additional participants (both industry and academic). The study included an electronic consent process and offered no incentive for participation.

Data collection

All data collection occurred between March-June 2020, adhering to the Consolidated Criteria for Reporting Qualitative Research (COREQ) framework (30). While remaining reflexive (31), the investigator (KF lead author) captured experiences by survey and semi-structured interviews. Each interviewee completed an oral survey summarizing key points of the collaboration by collaboration mechanism (9), project stages (32), and collaboration types (33). Participants completed demographics questions to capture participant information and role within the AIC (i.e., race, gender, ethnicity, age, role, years at the organization, and seniority at the organization). Memoing, a note tasking approach for qualitative research, occurred during the interviews to record notes during the interaction.

We created a semi-structured interview guide to inquire about EC and REC dynamics, specifically examining the history of AIC, patterns of interpersonal relationships, models of operation, measures of success (e.g., results, output), facilitators, and barriers of partnership. The interview guides were pilot-tested to minimize participant burden and included roughly 10 questions (Table 1), yielding 30–60 minutes interviews. All interviews were audio-recorded and transcribed verbatim using Otter.ai software. A digital audit trail (34) documented the data collection and analytic process.

Analysis

All data was anonymized and analyzed by lead author (KF) using a qualitative management software (34), (i.e., Dedoose). The conceptual model informed the deductive coding approach, coding for questions in the interview guide around collaboration dynamics, facilitators, and barriers. A second coder, trained in qualitative methods, supported analysis for a sub-set of the data to ensure consensus and code alignment. To reach thematic saturation (23), the analysts iteratively organized data into themes, generating significant statements and group meaning. The analysis used direct interpretation during the analysis process and preliminary results were shared back to participants as a form of member checking to reduce bias and ensure accuracy of findings (35,36).

Results

We interviewed a total of 20 participants from 7 (35%) unique academic institutions and 13 (65%) industry affiliates. Of the industry participants, 9 (69%) obtained previous experience in an academic setting (e.g., academic appointments, graduate training). Table 2 describes the characteristics of participants. Interviewee ages ranged from 30-62 years old, with 5% of interviewees obtaining a bachelor’s degree (n=1), 20% master’s degree (n=4), and (75%) doctoral-level degree (n=15). The seniority of interviewees varied, 10% with no supervisory experience (n=2), 10% team leader (n=2), 55% director/investigator (n=11), and 25% executive/department leadership (n=5). Table 3 delineates the reported collaboration mechanism, project stages, and collaboration types from each participant.

Significant statements

Seven major themes emerged from the data: (I) collaboration evolves with time, relationships, funding, and evidence; (II) AIC demands strong relationships and interpersonal dynamics; (III) operational processes vary across AIC; (IV) collaboration climate and context matters; (V) shared expectations lead to a better understanding of success; (VI) overcoming challenges with AIC recommendations; (VII) AIC may help navigate the global pandemic. Additional illustrative quotes are found in Table 4.

Collaboration evolves with time, relationships, funding, and evidence

All participants reported the impetus for collaboration varied and evolved throughout the product or intervention life cycle. Some participants mentioned AIC started with a previous relationship (e.g., peer or previous colleague) and others mentioned the AIC was endorsed by company leadership (e.g., C-level executives and/or board members). When a C-level champion was described, participants highlighted how company values and mission naturally advocated for cross-disciplinary work and evidence-based practices, thus collaboration was “woven into the company culture/fabric.” (Participant 1004)

Of the digital health AIC described, many relied on partnerships early for design and development, specifically adhering to human-centered design processes and stakeholder engagement best practices. As digital health products matured and went to market, AIC shifted in approach, pivoting to controlled scientific trials. Academic participants mentioned as their health interventions reached an evidence-based status, dissemination efforts were facilitated by industry partners in the commercialization process. Many participants discussed a historical journey of AIC, with “ebbs and flows,” as some research projects did not require an external collaborator. (Participant 2008)

AIC demands strong relationships and interpersonal dynamics

Among the participants, interpersonal dynamics remained a consistently reported facilitator of digital health AIC. Many reported the necessity for strong relationships early in digital health partnerships; grounded in mutual respect, strong communication, awareness of the diverse values of collaborators, and assumption of good intent. Most participants mentioned the benefit of collaborators with experiences in both academia and industry to facilitate the relationship with empathy and understanding of diverse cultures.

Both academic and industry participants reported previously established relationships facilitated collaborations. The participants recalled AIC with previous academic colleagues, graduate school peers, or connections made through annual professional conferences, describing established relationships feel less transactional and more transparent than AIC without established relationships. Roughly half of the participants reported experiences where personnel changes impacted their digital health AIC, reporting when an advocate leaves, no one champions the relationship. Most participants reported they would prefer to keep the same collaborators over time as they learned each other’s work style. One participant mentioned, “you can’t establish rapport when you’re constantly changing your roster.” (Participant 1004)

In a couple of interviews, industry participants mentioned the nuances of straddling digital health AIC who were considered both partners and customers. While originated as AICs, the partnerships yielded future customers to the industry partners. After the collaborators conducted pilot or feasibility tests, the academic health centers or universities shifted to revenue-generating clients. Some industry partners mentioned these collaborations served as an investment to their product or offering.

Operational processes vary across AIC

Participants mentioned the variability when it came to the mechanism of AIC. “Every school [academic partner] is different. Unfortunately, our joke is ‘once you know one school, you [only] know one school” (Participant 2006). This was a common perspective shared among industry partners where they underscored how different each academic partner was—contextually and operationally.

Most participants reported the traditional processes (i.e., single industry partner and one academic institution) carrying out research studies on industry products. Some participants reported unique forms of collaboration involving diverse collaborators, either with non-profits, community partners, additional companies, or universities. Participants in one high-functioning AIC described their experience with multiple institutions as creating a “three-headed monster” (Participant 1007) which proved to be advantageous for role clarity, accountability, and democratic synergy among collaborators.

Both academic and industry participants described their internal teams, blending operations management, and research operations. Most collaborations obtained little interface with technical or product teams. Many participants mentioned two avenues for research, product (e.g., UX/UI, user feedback) and efficacy (e.g., scientific trials). When working in digital health, many industry participants mentioned an important distinction between product efficacy and product usability. Creating this delineation of research ensures the collaborators treat the digital health product like a static intervention; thus limiting researchers’ requests for product changes outside the product road map and avoiding the need to train all industry staff (e.g., sales team, customer service) in human subject’s research. Regardless of the research route, the role of strong project management and effective communication on both sides of AIC was reported among all participants.

High-functioning digital health AICs reported a collaboration point-person, operating as a broker between the academic and industry collaborators. Described differently from the project manager, the brokers were typically doctoral-level scientists employed by industry. These individuals managed many AICs, serving as a “conduit team member” (Participant 2003) who interfaces with both collaborators. This role obtained both an understanding of industry operations and academic processes. A few of the industry participants mentioned their companies maintain internal research departments, operating like contract research organizations (CROs), balancing the company mission, and advancing scientific discovery.

The extent of partnership varied among AIC, however, most participants mentioned the cadence of interaction depended on the maturity of digital health tool, stage of collaboration, and longevity and size of the project. Newer digital health products in development or design stages required larger collaborative kickoff meetings used for ideation and brainstorming. Many participants mentioned government or community partners were involved in the early stages of AIC. As partnerships progressed and relationships emerged, the cadence of meetings increased during the scope of work process; participants mentioned weekly/bi-weekly calls for project initiation and maintenance. For established collaborations with long-standing relationships, interactions were most prominent during onboarding and project initiation and less engaged during the conduct of work. This shift in communication interaction was reported by many scientific participants, saying it removed conflicts of interest and influence during trials and scientific discovery.

Funding for AIC varied among participants. Less established companies collaborated on traditional grant funding mechanisms like NIH grants and innovation opportunities like Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs. When companies outgrew the collaborative federal grants, participants reported relying on industry-sponsorship funding. More established companies and digital health products earmarked budget dollars from funding rounds for research and evaluation. Participants mentioned the importance of academic partnerships to avoid, mitigate, and disclose conflicts of interest based on digital health products. Many academic participants recommended not to “try to make industry pay for everything”, reporting that does not reflect a partnership and you’ll get “transactional experience.” (Participant 2002)

Collaboration climate and context matters

When participants recalled their most successful AICs, they reported an internal culture of shared understanding, co-created goals, clear expectations, and flexible boundaries during the execution of partnership. Despite a thriving internal setting for AIC, many participants reported external barriers thwarting initial efforts of partnership. For example, even with receptive and engaged academic principal investigators, institutions reported formal structures (e.g., expensive indirect costs, slow IRB review processes, and restrictive tech transfer policies) influencing AIC.

Some participants mentioned how companies are not set up to conduct research, therefore restructuring companies to treat human subjects different than customers. Some examples were shared to create separate processes for product improvements and scientific discovery. All industry and academic participants mentioned a certain level of cross-sector learning was required, education examples included: information on design thinking, commercialization, current reimbursement/policies, human subjects research, IRB processes, budget, and statistics/analytics (e.g., need for power calculations). Additionally, both academic and industry participants reported the need for authentically engaged parties during implementation, recommending a good study design and scalable digital health product only goes so far, “you need a champion at the implementation level to execute vision”. (Participant 2012)

Shared expectations lead to a better understanding of success

Participants mentioned digital health products and processes demand shared goals, requiring verbal discussions and contractual obligations. While goals depended heavily “on the impetus of the partnership,” many participants mentioned the importance of clearly articulating the objectives of the AIC. (Participant 1006) Various contractual agreements helped facilitate shared expectations of the partnerships, including the memorandum of agreements (MoA), scopes of work (SOW), data use agreements, non-disclosure agreements (NDAs), publication agreements, and co-created grant proposals with delineated milestones and performance measurements.

While these contractual agreements were required among all participants, other expectations (e.g., budget, timeline, and outcome objectives) were discussed among high performing collaborators. Many participants mentioned the need to establish expectations around role clarity, communication cadence, and shared roadmap that aligned with both sector’s needs. Participants reported alignment of expectations was critical for the success of the partnership. Some academic participants mentioned unlike traditional grant funding mechanisms, AIC does not obtain cost extensions therefore early negotiation of expectations is critical. In particular, the role of time remained a common talking point among participants

Many participants mentioned digital health product testing occurs too fast for typical academic timelines. Various strategies were mentioned to reduce the time needed for pilot tests, but most agreed its more challenging with formal clinical trials. Many industry participants agreed the “fail-fast mentality doesn’t work” (Participant 2017) in digital health as it leaves customers at risk. “People live in their tech bubble and people live in their academic or healthcare bubble. And if you’re actually open to listening, you can learn some really cool things. And I think there’s an opportunity: move fast and break things without actually breaking things when it comes to human health. And we need to totally abandon that [fail fast] in digital health, and make sure that we’re putting patient safety and outcomes at the forefront.” (Participant 2001)

Despite the aversion to ‘fail fast’ terminology, all participants mentioned: “time is the biggest resource of digital health companies.” (Participant 1003) Most industry participants mentioned time is so critical not because the company can generate revenue faster, but because digital health products are oftentimes behavioral interventions. Digital behavioral products lack federal regulation, therefore, oftentimes the inferior and non-evidence-based products go-to-market faster.

Lastly, definitions of success varied among AICs. “Each partnership and each collaboration is going to have its own measure of success.” (Participant 2001) Some participants reported a successful AIC involved both internal and external success. For example, internal success meant reaching product goals, engaging with customers, improving health outcomes, and creating actionable insights to improve the digital health tool. Whereas external success for most academic collaborators sought to fill a gap in knowledge or within a body of evidence. Beyond publications and product advancements, most AIC mentioned a successful collaboration was an enjoyable, fun, experience where a good relationship was established and maintained with the other sector. Generally, when asked about measuring success, most participants agreed “it depends and it’s complicated” (Participant 2010)

Overcoming challenges with AIC recommendations

Participants mentioned overcoming challenges in digital health AIC demands “investigator-, institution-, industry-, and implementation-fit”. (Participant 2002) Over time, all participants mentioned they learned the attributes of an ideal collaborator. Many participants leveraged their trusted network to recruit like-minded individuals and interested organizations. In contrast, three participants mentioned the importance of diverse collaborators and the benefits of engaging with organizations and individuals with different perspectives. Either way, “finding the right formula of priorities, aims, resources, and expertise” was mentioned by all participants. They reported there is no established roadmap or collaborator due diligence playbook, therefore making the process one of “the murkier areas of academic-industry partnerships.” (Participant 2003) In the collaborator recruitment stage, some participants mentioned recommendations to sus out ideal collaborators.

Many participants mentioned the need for industry to obtain a doctorate-level broker to facilitate the AIC. With university experience and understanding of scientific boundaries, these individuals can straddle the mission of the digital health company while driving for scientific discovery. Many industry collaborators mentioned they look for academic investigators with strong reputations in the scientific community, experience with industry-funding, and entrepreneurial mindsets. Some participants mentioned when a digital health product is a production from a co-creation or design collaboration, the collaborators are more invested in implementation, evidence-generation, and commercialization.

AIC helps navigate a global pandemic

All participants advocated digital health AIC despite the challenging trajectory of balancing diverse priorities and cultures. Most participants highlighted the need for collaboration during a global crisis, demanding an “all hands on deck” approach. (Participant 1007) Many times without prompt, participants mentioned the collaborative efforts needed to combat the coronavirus disease 2019 (COVID-19). Most industry participants reported they are well-positioned due to the demand for health technology and the evidence generated from previous AIC. Academic participants mentioned many research projects are on hold, thus pivoting to knowledge generation on COVID-19 related topics. “There is great work that can be done in these partnerships now. But I think sometimes it comes with getting past the stigma, just because your sponsor is a company and not the federal government [traditional NIH grants] that it somehow diminishes the scientific rigor of the work. It doesn’t have to be that way.” (Participant 2003)

Conclusions

Digital health remains a rapidly evolving field in public health practice, demanding cross-disciplinary partnerships. Refining collaborative processes between academia and industry are of interest among researchers and practitioners (5,6,14,37-42) to ensure marketed digital health products remain high quality and evidence-based. The study investigated experiences of AIC, specifically examining the experiences of academic and industry collaborators working in digital health. The thematic analysis found consensus on many collaboration dynamics related to relationships, context, and expectations. The findings also suggest the range of variability among mechanisms for collaboration, the EC, and unavoidable challenges to overcome among collaborators.

Digital health has many nuances in the public health space; however, the findings align with the literature on university-industry partnerships (43-45). Collaborative strategies such as shared goals, clear expectations, awareness of diverse priorities, and transparent communication align with other industries (11,45). Authentic relationships remain a key facilitator, even if opportunistic in the digital health field. Despite the alignment, the findings do not map onto a specific collaboration framework or communications model described in other industries (e.g., biotechnology, life sciences), suggesting future research may elaborate on models to account for nuances in digital health and public health fields (37).

Generally, most participants responded to interview questions probing on their digital health collaboration experiences with “it depends”. AIC in digital health varies across environments, mechanisms, and people. Highlighting the qualitative experiences depends on many contextual variables. Findings illustrated the various contexts and mechanisms for AIC, confirming the diversity of collaboration types in the innovation literature (43,46). Results demonstrate the diverse ways to collaborate across disciplines and their dependence on collaborator relationship status, the extent of the project, and phase of digital health product. Collaboration types, stages, and mechanisms were not mutually exclusive in this study. Many described partnerships experienced different strategies at different times in different situations. Digital health AIC may consider implementing best practices in complex adaptive systems (47) and complexity theory (48) to help facilitate evolving and highly adaptive collaborative landscapes. AIC is a dynamic process where interactions and relationships simultaneously affect delivery of digital health products or projects. The complexity of human interaction in parallel with the challenges in digital health create a constellation of variables to consider when executing these collaborations in the real-world.

Three common facilitators of AIC were (I) established collaborations from previous relationships, (II) the need for an industry scientific broker, and (III) the role of communication. First, established collaborations born from previous relationships were reported easier to initiate and implement. This finding aligns with community-based participatory research efforts (49) and the agile manifesto in software engineering (50). Second, high-functioning collaborations operated with a doctoral-trained industry champion, serving as a scientific broker in the AIC. The role of champions is prominent in implementation science (51),however the additional knowledge and experience (in both academia and industry) created a new level of expertise needed to successfully facilitate these partnerships. Third, the findings illustrate the critical role of communication and transparency, mirroring what is known in the inter-organizational healthcare (52) and communication literature (53). The friction points mentioned by participants regarding IRB, costs, cultures were reported learned over time and defeated by mutual exchange.

While the sampling strategy carefully considered a positive deviance approach, inherently, the methodology introduced potential selection bias, therefore, lacks generalizability. Additionally, qualitative methods remain open to interpretation. To manage this limitation, the investigator remained reflexive by bracketing biases throughout the process, specifically by utilizing Stakes (2005) checklist for quality case studies (54) and the COREQ framework. Despite these limitations, capturing real-world perspectives filled a gap in the digital health AIC literature while confirming what is known in broader university-industry partnerships. This work augmented a previous understanding of AICs to meet the evolving digital health context. Although promoting innovation through AIC is not new, the era of digital health obtains additional factors relevant to the success of collaborations. The speed of technical innovation (4,6,55), cross-disciplinary nature of digital health development (6,56,57), increased presence of health startups (58,59), and software regulation charts new territory for AIC (60,61). The study captured empirical data from academia and industry in digital health to fill this gap. Future research should investigate additional real-world AIC to generate frameworks or theories relative to digital health.

Digital health AIC provides promise in health innovation; however, they demand high functioning partnerships engrained in strong interpersonal relationships. Despite how often AIC are advocated for in the literature, findings illustrate the value and variability across digital health collaborations. AIC remain diverse and evolve as rapidly as health technologies, requiring flexible mechanisms of collaboration and intentional cultural fit. Although many challenges exist in digital health, findings report ways to leverage the complementary strengths of each sector to advance digital health efforts. While no partnership defeats all barriers in digital health, AIC catalyzes improved digital health tools, thus advancing scientific discovery, enhancing public health, and benefiting the economy.

Acknowledgments

In the spirit of open innovation, the research team acknowledges the participating companies and academic interviewees for their volunteered time and contribution to the study.

Funding: None.

Footnote

Reporting Checklist: The authors have completed the MDAR checklist. Available at http://dx.doi.org/10.21037/mhealth-20-140

Data Sharing Statement: Available at http://dx.doi.org/10.21037/mhealth-20-140

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/mhealth-20-140). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study conformed to the provisions of the Declaration of Helsinki (as revised in 2013). Ethical approval was obtained by The Colorado Institutional Review Board and approved this study (IRB#: 19-2508). All study participants obtained and electronic consent prior to data collection.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. World Health Organization. WHO Guideline: recommendations on digital interventions for health system strengthening. Guideline 2019.
2. Marcolino MS, Oliveira JAQ, Agostino M, et al. The Impact of mHealth Interventions: Systematic Review of Systematic Reviews. JMIR Mhealth Uhealth 2018;6:e23 [Crossref] [PubMed]
3. Webster P. Virtual health care in the era of COVID-19. Lancet 2020;395:1180-1. [Crossref] [PubMed]
4. Patrick K, Hekler EB, Estrin D, et al. The Pace of Technologic Change: Implications for Digital Health Behavior Intervention Research. Am J Prev Med 2016;51:816-24. [Crossref] [PubMed]
5. Hingle M, Patrick H, Sacher PM, et al. The Intersection of Behavioral Science and Digital Health: The Case for Academic-Industry Partnerships. Health Educ Behav 2019;46:5-9. [Crossref] [PubMed]
6. Michie S, Yardley L, West R, et al. Developing and Evaluating Digital Interventions to Promote Behavior Change in Health and Health Care: Recommendations Resulting From an International Workshop. J Med Internet Res 2017;19:e232 [Crossref] [PubMed]
7. D'Amour D, Ferrada-Videla M, San Martin Rodriguez L, et al. The conceptual basis for interprofessional collaboration: core concepts and theoretical frameworks. J Interprof Care 2005;19:116-31. [Crossref] [PubMed]
8. Fitz Harris LF, Toledo L, Dunbar E, et al. Program collaboration and service integration activities among HIV programs in 59 U.S. health departments. Public Health Rep 2014;129:33-42. [Crossref] [PubMed]
9. Melese T, Lin SM, Chang JL, et al. Open innovation networks between academia and industry: an imperative for breakthrough therapies. Nat Med 2009;15:502. [Crossref] [PubMed]
10. Crew B. Top 5 corporate-academic collaborations in biomedical sciences. In: This is what success looks like. Nature Index 2019. Available online: https://www.natureindex.com/news-blog/top-corporate-academic-collaborations-biomedical-sciences.
11. Ankrah SN, Al-Tabbaa OUniversities–industry collaboration. A systematic review. Scand J Manag 2015;31:387-408. [Crossref]
12. Perkmann M, Tartari V, McKelvey M, et al. Academic engagement and commercialisation: A review of the literature on university–industry relations. Res Policy 2013;42:423-42. [Crossref]
13. Jackson P, Richter N, Schildhauer T. Open Innovation with digital startups using Corporate Aceelerators–A review of the current state of research. Zeitschrift für Politikberatung 2015;7:152-9. [Crossref]
14. Abroms LC, Allegrante JP, Auld ME, et al. Toward a Common Agenda for the Public and Private Sectors to Advance Digital Health Communication. American Public Health Association, 2019.
15. Chen J, Pickett T, Langell A, et al. Industry-academic partnerships: an approach to accelerate innovation. J Surg Res 2016;205:228-33. [Crossref] [PubMed]
16. Steinmo M, Rasmussen E. How firms collaborate with public research organizations: The evolution of proximity dimensions in successful innovation projects. J Bus Res 2016;69:1250-9. [Crossref]
17. Rajalo S, Vadi M. University-industry innovation collaboration: Reconceptualization. Technovation 2017;62-63:42-54. [Crossref]
18. Morandi V. The management of industry–university joint research projects: how do partners coordinate and control R&D activities? J Technol Transf 2013;38:69-92. [Crossref]
19. Starks H, Brown Trinidad S. Choose Your Method: A Comparison of Phenomenology, Discourse Analysis, and Grounded Theory. Qual Health Res 2007;17:1372-80. [Crossref] [PubMed]
20. Ivascu L, Cirjaliu B, Draghici A. Business Model for the University-industry Collaboration in Open Innovation. Procedia Economics and Finance 2016;39:674-8. [Crossref]
21. Greenwald HP, Zukoski APAssessing collaboration. Alternative measures and issues for evaluation. Am J Eval 2018;39:322-35. [Crossref]
22. Ford KL. An Exploration of Digital Health Academic-industry Collaboration. University of Colorado at Denver; 2020.
23. Creswell J PC. Qualitative Inquiry & Research Design. Choosing Among Five Approaches. Thousand Oaks, California: Sage Publications Inc., 2018.
24. Rock Health: Full-service seed fund that supports startups working in digital health 2019. Available online: https://rockhealth.com/.
25. Marsh DR, Schroeder DG, Dearden KA, et al. The power of positive deviance. Bmj 2004;329:1177-9. [Crossref] [PubMed]
26. Bradley EH, Curry LA, Ramanadhan S, et al. Research in action: using positive deviance to improve quality of health care. Implement Sci 2009;4:25. [Crossref] [PubMed]
27. Walker LO, Sterling BS, Hoke MM, et al. Applying the concept of positive deviance to public health data: a tool for reducing health disparities. Public Health Nurs 2007;24:571-6. [Crossref] [PubMed]
28. Creswell JW, Klassen AC, Plano Clark VL, et al. Best practices for mixed methods research in the health sciences. Bethesda (Maryland): National Institutes of Health 2011;2013:541-5.
29. Palinkas LA, Horwitz SM, Green CA, et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health 2015;42:533-44. [Crossref] [PubMed]
30. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349-57. [Crossref] [PubMed]
31. Reflexivity: a practical guide for researchers in health and social sciences. Malden, MA: Blackwell Science, 2003.
32. Schacht WH. Small Business Innovation Research (SBIR) Program 2010. Library Of Congress Washington DC Congressional Research Service.
33. Frey BB, Lohmeier JH, Lee SW, et al. Measuring collaboration among grant partners. Am J Eval 2006;27:383-92. [Crossref]
34. Silver C, Lewins A. Using Software in Qualitative Research: A Step-by-Step Guide. 1 Oliver’s Yard, 55 City Road London EC1Y 1SP: SAGE Publications Ltd, 2014.
35. Munn Z, Porritt K, Lockwood C, et al. Establishing confidence in the output of qualitative research synthesis: the ConQual approach. BMC Med Res Methodol 2014;14:108. [Crossref] [PubMed]
36. Crowe S, Cresswell K, Robertson A, et al. The case study approach. BMC Med Res Methodol 2011;11:100. [Crossref] [PubMed]
37. Iyawa GE, Herselman M, Botha A. Digital Health Innovation Ecosystems: From Systematic Literature Review to Conceptual Framework. Procedia Comput Sci 2016;100:244-52. [Crossref]
38. Hostetter M, Klein S, McCarthy D. In: Taking digital health to the next level: Promoting technologies that empower consumers and drive health system transformation. Lorber D. editor. New York City, NY: The Commonwealth Fund, 2014.
39. Sucala M, Nilsen W, Muench F. Building partnerships: a pilot study of stakeholders’ attitudes on technology disruption in behavioral health delivery and research. Transl Behav Med 2017;7:854-60. [Crossref] [PubMed]
40. Hird N, Ghosh S, Kitano H. Digital health revolution: perfect storm or perfect opportunity for pharmaceutical R&D? Drug Discovery Today 2016;21:900-11. [Crossref] [PubMed]
41. Depasse JW, Chen CE, Sawyer A, et al. Academic Medical Centers as digital health catalysts. Healthcare 2014;2:173-6. [Crossref] [PubMed]
42. Desmond-Hellmann S. Improving Health With Partnerships Between Academia and Industry. JAMA Internal Medicine 2013;173:1051. [Crossref] [PubMed]
43. Draghici A, Baban C, Ivascu L, et al. editors. Key Success Factors for University-Industry Collaboration In: Open Innovation. 8th Annual International Conference of Education, Research and Innovation, Spain; 2015.
44. Golder-Buckley D, Way D, Glover M. Best practice strategies for successful innovation through university-business collaboration. Research Councils UK, 2015.
45. Rybnicek R, Königsgruber R. What makes industry–university collaboration succeed? A systematic review of the literature. J Bus Econ 2019;89:221-50. [Crossref]
46. Perkmann M, Walsh K. University–industry relationships and open innovation: Towards a research agenda. Int J Manag Rev 2007;9:259-80. [Crossref]
47. Innes JE, Booher DE. Consensus building and complex adaptive systems: A framework for evaluating collaborative planning. J Am Plann Assoc 1999;65:412-23. [Crossref]
48. Anderson P. Perspective: Complexity theory and organization science. Organization Science 1999;10:216-32. [Crossref]
49. Becker AB, Israel BA, Allen A. Strategies and techniques for effective group process in CBPR partnerships. Methods in community-based participatory research for health 2005;2:69-94.
50. Sandberg A, Pareto L, Arts T. Agile collaborative research: Action principles for industry-academia collaboration. IEEE Software 2011;28:74-83. [Crossref]
51. Hendy J, Barlow J. The role of the organizational champion in achieving health system change. Social Science & Medicine 2012;74:348-55. [Crossref] [PubMed]
52. Karam M, Brault I, Van Durme T, et al. Comparing interprofessional and interorganizational collaboration in healthcare: A systematic review of the qualitative research. Int J Nurs Stud 2018;79:70-83. [Crossref] [PubMed]
53. Lewis LK. Collaborative interaction: Review of communication scholarship and a research agenda. Ann Int Commun Assoc 2006;30:197-247. [Crossref]
54. Stake RE. Qualitative Case Studies. In: Denzin NK, Lincoln YS. The Sage handbook of qualitative research. Sage Publications Ltd., 2005: 443-66.
55. Murray E, Hekler EB, Andersson G, et al. Evaluating Digital Health Interventions: Key Questions and Approaches. Am J Prev Med 2016;51:843-51. [Crossref] [PubMed]
56. Hekler EB, Klasnja P, Riley WT, et al. Agile science: creating useful products for behavior change in the real world. Transl Behav Med 2016;6:317-28. [Crossref] [PubMed]
57. Sucala M, Ezeanochie NP, Cole-Lewis H, et al. An iterative, interdisciplinary, collaborative framework for developing and evaluating digital behavior change interventions. Transl Behav Med 2020;10:1538-48. [PubMed]
58. Garousi V, Petersen K, Ozkan B. Challenges and best practices in industry-academia collaborations in software engineering: A systematic literature review. Inf Softw Technol 2016;79:106-27. [Crossref]
59. Arora A, Belenzon S, Patacconi A, et al. The changing structure of American innovation: Some cautionary remarks for economic growth. Innovation Policy and the Economy 2020;20:39-93. [Crossref]
60. Duggal R, Brindle I, Bagenal J. Digital healthcare: regulating the revolution. British Medical Journal Publishing Group, 2018.
61. Shuren J, Patel B, Gottlieb S. FDA regulation of mobile medical apps. JAMA 2018;320:337-8. [Crossref] [PubMed]