Taming AI Sprawl in Radiology

A radiologist pulls up a chest CT scan flagged for a pulmonary nodule. One artificial intelligence (AI) tool writes its findings into a picture archiving and communication system (PACS). A second posts a risk score into its own worklist. A third requires a separate login. Each has its own definition of urgent, and none of the results arrives at the same point in the read.

Lior Eshel, founder and CEO of TestDynamics, has seen this scenario play out regularly across health systems. “Radiologists want to use AI tools, but each one asks them to switch between models and interfaces,” he says. “That causes great anxiety and frustration. And when AI isn’t adding any value or efficiency, radiologists won’t use it.”

For AI to move forward, Eshel says radiology leaders must look beyond individual tools and consider how to manage a growing stack of AI without making radiologists the de facto integrator.

When More AI Creates More Friction

More than 1,500 AI algorithms have been cleared by the FDA, and most apply to medical imaging.¹ Eshel refers to these as “vertical AI solutions” designed to solve a specific problem. But as health systems deploy these tools separately, problems mount.

“The first tool changed something real,” Eshel says. “The second is where workflows break, the gain is smaller, and the cost is not only the license, but it’s another integration, another stream of alerts, and another confidence score.” By the third or fourth deployment, health systems “stop thinking about AI and imaging as products to buy and start thinking about how you take all these outputs, make them consumable, amalgamated, and surface only what’s relevant to the user,” Eshel adds.

That’s where solutions such as TestDynamics’ Satori come into play. A vendor-neutral platform, Satori integrates, validates, and continuously monitors AI tools within the existing clinical workflow. Radiologist-validated templates within Satori put AI findings into the proper clinical context, and clinicians have control over which results they see and don’t see.

“Radiologists like the solution because it saves them a lot of time and makes everything concise,” Eshel says. “If you need to switch the engine on the backend because it’s not performing well, the physicians don’t see any difference because the template will look the same.”

Workflow and Drift Considerations

When choosing AI solutions, radiology leaders should take a workflow-first approach by asking, “How do we enhance it but not change anything,” Eshel says. “[Satori] isn’t changing the viewer, worklist, or reporting system. We’re just adding capabilities on top of the workflow you’re currently using.”

Next, Eshel recommends that leaders focus on enhancing a single clinical use case with AI. “We will come in and suggest tools that might be able to cover these use cases versus sending them a list of 200 AI vendors and asking them to choose,” Eshel says.

Post-deployment, strong governance is essential. While regulatory bodies like the FDA grant clearance for new AI algorithms, they do not account for post-deployment drift. “FDA clearance reflects performance evaluated at a point in time, while local populations, scanners, protocols, and clinical conditions can change over time,” Eshel says. “I think that process is wrong because you have to have post-market surveillance data on the AI and how it performs in real life and in clinical setup moving forward.”

Satori bridges the gap by monitoring AI solutions over time, with alerts triggered when an algorithm’s performance starts to decay. “We can see the drift, or that end users aren’t agreeing with AI over time, and we will monitor that and share that information with the institution,” Eshel says.

Redefining Success with AI

The ideal AI-enabled medical imaging ecosystem of the future, Eshel says, won’t be judged by the number of AI tools deployed but by how well they’re integrated. “[Successful organizations] measure performance locally instead of accepting what was measured elsewhere,” he says. “They keep the ability to remove a tool without rebuilding a workflow. And they separate the people selling the algorithm from the people grading them.”

Reference

1.     U.S. Food and Drug Administration. Artificial Intelligence-Enabled Medical Devices. FDA. Updated June 16. Accessed Aug. 20. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-enabled-medical-devices