Clinical simulation for digital health solutions

• There is a gap in building evidence for digital health solutions, but the use of clinical simulation may be a way to help overcome it
• Clinical simulation situates end users in realistic clinical scenarios to perform tasks working with synthetic patient cases, and is a flexible, scalable, and agile method that can keep pace with the rapid update cycles associated with software solutions
• Clinical simulation is already being used successfully and could become a safe, fast, and cost-effective way to test digital health solutions and generate robust evidence

Digital health solutions are on the rise, and they hold great potential for enhancing health access and outcomes. They also have the potential to optimize operational efficiency and reduce costs for healthcare organizations. However, defining the evidence requirements for digital health solutions is a complex task and heavily reliant on the specific purpose of the solution.

Traditional methods of evaluation, such as randomized control clinical trials, are oftentimes ill-suited for digital health solutions. First, they are largely incompatible with the typically fast-moving and iterative nature of software development given their size and complexity. Furthermore, their strict inclusion criteria give little guarantee that results will generalize to real-world populations. In addition, conducting clinical trials requires substantial amounts of time, money, and other resources that are often not available to companies.

Watch a short animation about how clinical simulation can help generate evidence for digital health solutions

Given that digital health solutions with clinical value are fundamentally different from drug therapies, a middle ground must be found between two somewhat opposing traditions: rigorous clinical trials in healthcare and agile software development in the tech world. Evidence generation for digital health should therefore be treated somewhat differently, even if the goals of ensuring safety and efficacy are the same.

One new, pragmatic approach that promises to both develop robust evidence and suit the fast-moving, iterative needs of digital solutions is clinical simulation.¹ Clinical simulation studies require high-fidelity, synthetic patient cases generated according to specific traits that are relevant to the needs of the study and/or condition (e.g. age, disease status, ethnicity). In addition, knowledge of local clinical workflows is needed to recruit the right end users to perform the most relevant clinical tasks.

Watch a short animation on clinical simulation in practice

How does clinical simulation work? An example could be a smartphone app that is created for patients with multiple sclerosis (MS). In theory, this app would record objective data on patient symptoms between patient visits. This data would then be presented to the patient’s neurologist at their next visit to help inform their consultation and ongoing treatment options.

Because the app developer wants to generate strong evidence that this app might have a positive effect, they turn to clinical simulation. In this case, the research team creates realistic, synthetic patient cases, working closely with expert neurologists. The team then tests and refines the feasibility of the study methods with a small number of clinicians, including data collection and the validity of synthetic patient cases. Any refinements are taken into account.

Once validated, the research team defines an appropriate sample size to achieve meaningful results with the outcome measures. Using these synthetic cases, the app is then populated and used to facilitate consultations between real neurologists and patients (played by actors in this case) based on these synthetic cases. These consultations could be conducted in person or remotely to accurately reflect both kinds of real-world consultation scenarios.

During these mock consultations, the research team tracks how the neurologists interact with the app, noting any issues or challenges they have. They also measure quantitative outcomes, such as how much time was needed to make a clinical decision. These results are then used in several ways: to further inform product development and refine the app; to inform the design of real-world trials; and to help demonstrate the value of the solution to support widespread adoption.

There are several potential benefits to using clinical simulation in healthcare:

• Flexibility: Simulations allow for high levels of flexibility and scalability in study designs and executions.
• Accuracy: High-fidelity, synthetic patient cases may be used to replicate real-world healthcare settings.
• Agility: Methods can keep pace with the rapid update cycle associated with software solutions
• Cost-effectiveness: Remote, multi-site testing can be conducted at low cost using virtual communication platforms, and avoiding disruption to clinical settings can save on health resources.
• Data security and privacy: Synthetic patient data can be generated from real-world data using models designed to minimize privacy issues.
• Inclusivity: Simulation also allows the inclusion of high-risk patient profiles (usually excluded from studies for ethical reasons) and direct observation of scenarios that may be impossible in the real world. Medical simulation also makes testing sub-population data more feasible compared with randomized control trials.

Of course, these benefits are only possible when the quality of the simulations themselves is of sufficient quality. Researchers must therefore make every effort to closely replicate real-world systems to ensure their simulations are of high fidelity.

And while the evidence generated from clinical trial simulation can be robust, it is not the same as a true randomized clinical trial and, for some solutions, may not be sufficient on its own to obtain regulatory approval. This is particularly true for higher-risk solutions that are classified as medical devices.

Clinical simulation is already being used today as a valuable tool to improve quality and safety in healthcare.²  One recent use case within the digital realm was to test a new tool that automated the search for suitable clinical trials for cancer patients. Normally, this is a time-consuming process that involves searching multiple databases using patient-specific data (cancer stage, genomic alterations, treating institution’s location) to match patients to cancer clinical trials.³

The objective of this specific simulation study was to compare the efficiency, quality, and cognitive burden of a clinical trial match tool that was developed by Roche to streamline the process of searching for and identifying clinical trials for patients.³  This simulation involved generating synthetic patient data (mock data) that was based on real-patient data, but with altered names, dates, clinical history, medication, and patient outcomes.⁴

In this case, 25 clinicians and research staff were then recruited to match the 10 synthetic patient cases to clinical trials using both the new tool and the traditional way via the publicly available online trial databases.³ The goal was to compare the speed, usability, and effectiveness of this tool in matching patients to clinical trials, as well as to generally understand the suitability of using simulations as a way to test digital health solutions.³

Using the results, the research team was able to collect both quantitative insights on the performance of the clinical trial match tool and valuable user feedback – all without the need for implementation in clinical practice. This saved considerable time, as approval to use actual patient data would have required up to 3 months lead time.³

Beyond the actual performance of the solution, the team was also able to gather important data related to cognitive burden, record screen activity, and gather additional feedback from participants immediately after they conducted the trial matching exercises. None of this would have been practical or even allowed in a real-world clinical setting.³

Simulation technology in healthcare can go a long way towards improving the evidence behind the array of digital health solutions being developed today. Given their flexibility, speed, and cost, they allow solution developers to continue to iterate rapidly in the early stages and make adaptations before moving forward to create a more defined, high-impact product.

There are barriers to overcome. Even once the use of clinical simulations has been validated, that does not mean they will necessarily be immediately accepted by regulators or clinicians. In addition, simulations are also limited by our ability to cope with the inherent complexity of modeling living systems, though this will continue to evolve.⁵

However, these barriers are not insurmountable. The fact is the biomedical industry is not the only sector that deals with highly complex and potentially critical systems. In other industries, such as aerospace or nuclear industries, simulation is used extensively during both product development and assessment to overcome similar safety problems with mission-critical products.⁵ There is no reason why it can’t be the same for healthcare.

Simulation studies allow innovators to evaluate new digital health solutions and services in a safe, efficient, and cost-effective manner before deploying them into the real world, helping to bridge the evidence gap that exists between early product development and real-world deployment. If implemented appropriately, they can help to reduce the evidence deficit that exists for many solutions and support the growth of a strong, digital health and evidence-based healthcare system.