Evaluating digital health solutions: How to know which solutions will bring value to your organization?

• The development of digital health solutions is rapid and expanding fast, but their impact has been limited due to a general lack of evidence demonstrating benefits and a lack of involvement from relevant stakeholders
• A six-step approach to evaluate digital health solution quality has been developed with the help of stakeholders including payers, employers, investors, and health technology companies
• Clear definitions of different types of digital health solutions and associated evidence requirements would help healthcare organizations select the best solutions that will make the biggest impact on patient’s lives

We live in a world that is digitally transforming, and healthcare is transforming with it. Like our banks, our cars, or our personal entertainment, every aspect of our healthcare is being impacted by digital technology. Today, digital health already covers everything from telehealth services to wearables to electronic health records and more. These tools can assess health risks, help with diagnosis, and provide continuous monitoring of both individuals and populations. And the use of digital health solutions is exploding: the industry was estimated at $USD 211 billion in 2022¹ and is expected to grow by more than 18% by 2030, fueled by increasing acceptance, the need to reduce healthcare costs, and an increasing prevalence of chronic diseases.¹

However, this rapid rise also comes with a downside. As Drs Perakslis and Ginsburg stated in a recent editorial for the Journal of the American Medical Association, “digital health has the potential to improve health management, but the current state of technology development and deployment requires a “buyer beware” cautionary note.”²

There has been an explosion of digital health solutions and companies developing them – but how do you know which ones are high-quality and will make a difference to patients? To address the question, Dr. Simon Mathews, Assistant Professor of Medicine at Johns Hopkins Medicine and thought leader for improving care delivery and value through technology, joined Healthcare Transformers for an interactive session. During the session, he explored in detail how to best evaluate digital health solutions for any healthcare organization and provide insights on how to know which digital health solutions are right for your organization and will add the most value for your patients.

According to Dr. Mathews, given the explosion of digital health companies and solutions, it is not surprising that we see a disconnect between quantity and quality. In the technology world, entrepreneurs and tech leaders are always hunting for the next disruptive breakthrough. However, the problem in digital health is that there is currently no set of accepted standards as to how and what even defines a “quality” digital health solution. This lack of clarity, combined with the overwhelming amount of funding and activity in this space, means that people within the industry believe that their solutions are much more groundbreaking than they actually are. In fact, many of these “cutting-edge” solutions barely scratch the surface when it comes to having a true clinical impact.

The result, according to Dr. Mathews, is an overall low-quality rating for digital health solutions. In one example, he and his team examined the clinical robustness of the digital health industry across more than 200 US companies with greater than $2M in funding, measuring clinical robustness in the context of regulatory filings and clinical trials as a proxy for potential clinical rigor. Surprisingly, nearly half of the companies had a score of 0.

As it stands today, the digital health industry at large is not having the kind of impact on healthcare that it probably could – or should. The question is, how can we fix that and make sure we are developing, purchasing, and using digital tools that will make a real difference in patients’ lives?

Dr. Mathews has therefore been pushing for more rigorous evaluation criteria for digital health solutions. Informed by his experience working with several stakeholders including payers, employers, investors, and health technology companies, he has developed a six-step approach to evaluate digital health solution quality.

1. Understand the conventional gold standard for managing, treating, and diagnosing the condition of interest, and determine, recognize, and understand how improvement is measured for that condition. 

2. Identify whether the solution of interest uses the same approach identified in step 1, or if it chooses a different path or approach.

3. Examine the evidence that supports the broad approach that the solution is presenting. Even if the digital solution uses the gold standard approach, it is still worth the rigor of this step, because using the gold standard does not always mean the digital solution will meet the effectiveness threshold that you may be looking for. 

4. Dig deep into how the solution has been tested in a real-world or experimental setting. 

5. Conduct an examination of regulatory filings, where applicable.

6. Conduct an evaluation of the clinical team to see if it includes those with the experience to realistically manage and diagnose the condition of interest.

Using his six-step methodology as a guide, Dr. Mathews shared a real-world example of a digital health solution that has the potential to make a positive impact in his own field of gastroenterology: a wearable for the treatment of acid reflux.

Acid reflux is a common ailment where acid from the stomach rises into the esophagus, causing injury and painful symptoms. The gold standards for treating acid reflux are lifestyle and diet modifications, medications, and ultimately more invasive procedures, depending on the severity of symptoms or disease (step 1).

This digital solution aims to help drive a lifestyle modification that can help reduce the impact of reflux by vibrating at night and reminding the patient to turn on his or her left side when sleeping. This is considered an evidence-based lifestyle modification for treatment that comes straight from major society guidelines for the management of reflux (step 2). The solution, therefore, uses a high-evidence, high-quality approach.

The study itself was rigorous, in that it was randomized, controlled,  and double-blinded, including a sham control arm where one group received the intended vibration intervention to reduce reflux, while the other group also received the wearable, but it was programmed to provide random vibrations. This element of the experimental design reduced the potential placebo effect of using the wearable. Other positive factors of the study included similar and representative populations in both arms, and the use of validated, patient-reported outcome measures specifically designed for reflux (steps 3 & 4). There were no US regulatory filings as the company was based and operated outside the US (step 5). In addition, two authors of the publication are well-recognized in the field and participate in the company (step 6). Finally, the company has published their outcomes in a well-regarded, highly competitive, peer-reviewed journal.

Given this rigorous approach, Dr. Mathews believes we can be optimistic that this device will make the impact that it promises and might be a valuable tool in the management of acid reflux. However, he also shared the caveat that any solution still needs to meet non-clinical requirements such as cost, security, and usability.

Because the digital health industry remains a complex and evolving space, Dr. Mathews believes that we should still keep two things in mind based on the current landscape:

• First, we must remain skeptical of all digital claims. It is up to each of us to do our research and do the groundwork to verify that these solutions are working as they are intended 

• Second, we should never forget to ask the question: does this solution serve the patient’s need in a meaningful way? We can never lose sight of the patient who should be at the center of this equation

Dr. Mathews believes in the promise of digital health solutions and is a firm believer that the potential to make a massive patient impact is there. Of course, the final decision to ultimately adopt any digital health solution goes beyond clinical quality, and must also consider factors such as usability, technical quality, privacy, interoperability, security, and cost. However, to push digital health to achieve its full potential, we need to have a pragmatic, standardized way to evaluate digital health solutions for clinical quality and ensure they are making a true impact on patients’ lives.

The discussion continued during a live Q&A round guided by Dr. Matthew S. Prime from Roche. Some questions that Dr. Prime and Dr. Mathews tackled include:

• What is your perspective on pricing strategies for digital health solutions? 
• What are the practicalities of tying evidence standards to reimbursement?
• How do you explain the lack of application of good clinical testing practices?
• Why are most applications being created without proper discovery phases to gather opinions from users and patients?
• Why do you think that digital health solutions haven’t been used more for early detection and diagnostics since getting data in between clinical visits?

Watch the free live event recording now and learn more about how you can better evaluate digital health solutions so they make a bigger impact on your patients.