[ad_1]
We’re witnessing a significant shift in healthcare with the acceleration of AI-enabled medical devices. From improving detection and risk stratification to drug development and clinical workloads, they have massive potential to improve health equity.
To build these innovations, high-quality, real-world medical data is essential for training AI models. Bringing these solutions into the hands of clinicians and ultimately helping patients requires rigorous regulatory oversight. This process ensures that the technology is safe, effective, and ready for clinical use.
Why medical AI relies on data
Small, unrepresentative training datasets that are not drawn from the intended population hinder AI performance, leading to a range of issues downstream, including exacerbating existing bias, creating products that lack generalizability, and produce inaccurate outputs.
Sometimes small datasets are unavoidable, especially for rare diseases. However, their use beyond this context leads to underrepresentation and reduces an AI model’s ability to generalize across a population.
Algorithms trained on narrow population samples have limitations in predicting, detecting, and classifying conditions across broader patient groups, which exacerbates health disparities and leads to poorer patient outcomes. If datasets used to train and test a model are not representative of the intended population, the model may not produce accurate results, and no amount of testing will be able to properly validate results. Meaning that models cannot generalize beyond the groups for which they were trained. If extra information (such as labelling and clinical reports) is not presented or contains errors, models trained and tested on that data may have inaccurate outputs also.
Minimizing bias is an essential aspect of training and testing data for AI-enabled medical devices. Identifying and mitigating potential bias is also a key component that regulators are focused on. AI bias leads to a multitude of issues, however, large and adequately diverse training datasets can help mitigate it.
While training data is the first consideration, testing data must also be representative of the intended population. It should be high quality, diverse, and sufficiently large to ensure the model’s accuracy and practical usefulness. Training and testing data must also be appropriately independent to ensure that the tests truly assess the accuracy and effectiveness of the algorithm, providing evidence of real-world performance.
These challenges are not just technical; they impact how regulators assess the safety and efficacy of AI-enabled devices.
Why is this important for regulatory submissions?
Getting ready for regulatory submission is a key driver for clients reaching out to us. A common thread we see is the need to train and test their devices on data from different regions that they are entering. Regulators are increasingly requiring detailed information on the representativeness of the data for new medical devices.
Agencies, such as the U.S. Food and Drug Administration, issue guidance on how to address data management, specifically focused on training and testing data used to ensure the effectiveness, accuracy, and usefulness of medical devices. Ensuring transparency and responsible AI development is key to making devices that are effective and compliant with ever-evolving regulatory guidelines.
Often, regulators will need thorough documentation and records of how data was acquired, the split of data used for training versus testing, how it is processed, stored, and annotated, among a plethora of other information points. Good data practices from the start make it easier for developers to pull together the information needed for regulatory submissions. Knowing that the training and testing data have been properly sourced and managed, and are large and diverse, can help reduce the need for regulators to require further validation, as they are reassured that the device will work accurately in its intended population.
Slow access to the breadth and variety of data needed ultimately slows down regulatory submissions. Insufficient representation in the training and testing data can also be a reason for rejection of regulatory submissions.
Looking ahead
In the race to deploy AI in healthcare, speed matters. But speed without structure leads to setbacks. Medical AI developers who prioritize data early will be the ones crossing the regulatory finish line faster and more reliably.
In the current environment, better data isn’t only about better algorithms. It’s the key to getting to market faster, with better clinical performance, and improved patient lives.
Photo: Supatman, Getty Images
Joshua Miller is the CEO and co-founder of Gradient Health, and holds a BS/BSE in Computer Science and Electrical Engineering from Duke University. He has spent his career building companies, first founding FarmShots, a Y Combinator backed startup that grew to an international presence and was acquired by Syngenta in 2018. He then went on to serve on the board of a number of companies, making angel investments in over 10 companies across envirotech, medicine, and fintech.
This post appears through the MedCity Influencers program. Anyone can publish their perspective on business and innovation in healthcare on MedCity News through MedCity Influencers. Click here to find out how.
[ad_2]
Source link
