White Paper

Evidence generation for digital health solutions

Matthew S. Prime, Saira Ghafur
Published on March 14, 2023 | 5 min read
evidence-generation

Key takeaways

  • Digital health solutions are rapidly growing and have the potential to improve health access as well as optimize operational efficiency and costs for organizations
  • Defining the evidence requirements for digital health solutions is complex and is highly dependent on the intended purpose of the solution itself
  • Bridging the evidence gap that exists today and generating robust evidence in a timely and cost-effective manner will empower healthcare transformation

Digital health solutions are rapidly growing and have the potential to revolutionize healthcare. This growth is reflected by the rising number of companies emerging in the sector and increasing amounts of investment in digital health over the last decade.1,2 Notably, the unique challenges generated by COVID-19 abruptly catapulted the positioning of using digital health solutions from an interesting potential opportunity to an immediate necessity.3

A wide variety of technologies – medical apps, clinical decision software, electronic health records, telehealth, health information technology, wearables devices, artificial intelligence, and machine learning – can be considered digital health solutions. These solutions have the potential to improve health access as well as operational efficiency and costs for healthcare organizations.

However, key questions remain:

  • How can healthcare providers or users of digital health solutions determine whether they deliver the value they claim?
  • What evidence should companies providing digital health solutions generate to prove it?

The complexity of defining evidence needed for digital health solutions

Given the spectrum of digital tools and technologies created for the healthcare market, there can be no single standard for what evidence is required for digital health solutions as a group. Evidence generation is highly dependent on the intended purpose of the solution itself.

evidence-generation

Digital health solutions with a medical purpose qualify as a medical device (also known as Software as a Medical Device (SaMD) in the US, Medical Device Software (MDSW) in the EU, or digital therapeutic). These solutions need regulatory approval to enter the market and require clinical evidence to show their safety and efficacy. The evidence required for these solutions include:4

  • Clinical association: Does the solution solve a real problem?
  • Analytical performance: Does the solution perform its calculations accurately and reliably?
  • Clinical performance: Does it produce clinically relevant data in its target population and environment?

This is not the case for digital health solutions with no medical purpose and therefore not considered a medical device such as electronic health records or clinical workflow solutions. To demonstrate the value of these solutions, as well as the additional benefits of digital solutions with a medical purpose, other evidence is of importance to allow various stakeholders to make decisions about their implementation. These include:4

  • Clinical value: Does the solution have a measurable impact on clinical outcomes (peer-reviewed evidence)?
  • Financial value: Does the solution show benefit as shown by health economics and outcomes research (if it directly impacts patient care), or show an enterprise benefit via a financial model?
  • Operational value: Does the solution provide operational efficiencies?
  • Experience value: Does qualitative research demonstrate its usability and adoption potential?

 

Watch the video below for a summary of the types of evidence that exist to demonstrate the value of digital health solutions.

Generating evidence for digital health solutions

Closing the existing gaps in evidence

Regardless of the type of evidence required, today many digital health solutions land on the market without generating appropriate data. One study analyzed all 224 US-based digital health companies focused on prevention, diagnosis, or treatment of disease that raised at least one round of venture funding of $USD 2 million or more between 2011-2020. This study revealed that 44% of the digital health companies had no regulatory filings nor clinical trials published. 5

There are various reasons why this may be the case. Many solutions entering healthcare today belong to the software developers and product experts of the tech world, rather than traditional health stakeholders. While these new players possess a wealth of expertise, they are not necessarily familiar with established clinical research principles and practices that are the backbone of the traditional biopharmaceutical industry. In addition, the emergency situation that COVID-19 caused for healthcare systems may have led to unchallenged tools on the market.6

Evidence generation can also be very expensive, and considering that in most markets reimbursement pathways that reward good evidence do not exist, there is often too little incentive for manufacturers to make this investment. Innovators may also not have access to the data that would help generate strong evidence for their solutions, and may not have the capacity to gather and store this in a secure manner. Furthermore, traditional research methodologies, such as randomized control trials – the gold standard for assessing drugs and other interventions – are ill-suited to many digital health solutions that share the same therapeutic aim.

This lack of rigorous evidence can result in the use of ineffective tools in the marketplace, and, perhaps even more worryingly, compromised patient safety.7,8 Furthermore, it dampens enthusiasm for and uptake of future digital solutions.

In order for technology to truly transform healthcare, the industry will need to find ways to bridge the evidence gap that exists today for digital health solutions and generate robust evidence in a timely and cost-effective manner. Fortunately, there are several innovative approaches and new methods that are already being used to help generate the quality evidence needed to demonstrate the true benefits of digital health.

Regulators and payors are starting to embrace and encourage these new and pragmatic approaches, including simulation-based methods and the use of Real World Evidence at the appropriate stages of the development cycle of products. Additionally, clear reimbursement pathways that reward innovators generating strong evidence are also being formally established in many countries.

Helping to reduce the barriers faced by digital health innovators today will contribute to a stronger global digital health ecosystem, where safer and more effective digital solutions will reach patients and clinicians more efficiently.

Aspire academy webinar series featuring Allison Guire

Generating evidence for digital health solutions

Email Icon

Get our latest insights

Join our community and stay up to date with the latest laboratory innovations and insights.

Subscribe now

Contributors

Matthew S. Prime headshot

Matthew S. Prime, BSc, MBBS, MRCS (Eng), PhD

Head of Evidence Generation, Roche Information Solutions
Dr. Matthew S. Prime trained as a doctor at University College London. He spent 12 years in clinical practice working as a surgeon in the NHS. He completed his PhD in healthcare policy focused on the enablers & barriers for healthcare innovation at Imperial College London. In 2013 he co-founded a digital health company to digitize care coordination for patients with traumatic injuries. Matt joined Roche in 2018 and today he is the Head of Evidence Generation for Roche Information Solutions. He has published extensively on digital health in the academic literature and is a member of the EU task force on the harmonization of the evaluation of digital medicine.
Saira Ghafur headshot

Saira Ghafur, MB ChB, MSc, MRCP (Resp)

Chief Medical Officer at Prova Health
Saira Ghafur is the CMO and a cofounder at Prova Health. She also holds the role of Digital Health Lead at the Institute of Global Health Innovation at Imperial College London and is a consultant Respiratory Physician at St Mary’s Hospital in London. Dr Ghafur has published extensively on a diverse range of digital health topics including cybersecurity, policy issues on the uptake of health tech and AI and has led the work to date on maximising the impact of health and care data on health and wealth.

Explore articles from our community

Join our community

Drive healthcare innovation with access to critical perspectives and exclusive events.

Healthcare Transformers delivers insights on the emerging trends shaping modern healthcare: digital health, patient experience, value-based care, and data security. Partnering with global experts and innovators, we tackle the industry's toughest challenges and explore actionable solutions.

Subscribe to our newsletter today to get invaluable perspectives delivered straight to your inbox.

References

  1. Cohen, Dorsey, Mathews, Bates and Safavi. (2020). NPJ digital medicine 3, 1-10
  2. Krasniansky, Evans, and Zweig. (2022). Report available at http://rockhealth.com/insights/2021-year-end-digital-health-funding-seismic-shifts-beneath-the-surface/ [Accessed February 2023]
  3. Williams et al. (2022) Eurohealth 28| Research paper available at  https://apps.who.int/iris/bitstream/handle/10665/351081/Eurohealth-28-1-29-34-eng.pdf [Accessed February 2023]
  4. Conroy, Fontana, Prime, Ghafur. (2023) White paper avaiable at https://healthcaretransformers.com/digital-health/current-trends/generating-evidence-for-digital-health-solutions/ [Accessed March 2023]
  5. Day, Shah, Kaganoff, Powelson, Mathews. (2022). J Med Internet Res 24, e37677
  6. Ghafur. (2022). Article available at https://www.wired.co.uk/article/digital-health-tools-science [Accessed February 2023]
  7. Wexler and Reiner. (2019). Science 363, 234–235
  8. Rotondi et al. (2019). J Med Internet Res 21, e14567