Shining a Bright Light on the Challenges and Opportunities of AI in Healthcare

Key to realizing the opportunities AI will provide is to clearly identify the concerns and how to address these, and further recognize that the “darkness” may simply be a function of an incomplete appreciation of what we do not yet fully understand.

In 2024, AI is already integrated in several healthcare systems: decision support tools and image analysis, for example. It is clear that we have crossed the figurative adoption Rubicon, and there is no reversing course. In reality, we know that there are some functions that AI simply does better than humans do (find small features on an X-ray, predict disease course) or more equitably (e.g., providing academic medical center–type care in rural areas) than we have the human resources to deliver.

While there is some early comfort with emerging regulatory regimens, the medical and legal communities still need to wrestle with very fundamental issues that AI foists on the healthcare ecosystem:

• How do we develop and properly validate AI-based systems?
• On what datasets are the AI product development based, and are these datasets truly representative of the population on which the innovations will be used?
• What expertise is required for effective regulation? Do the regulators need to retool?
• How should healthcare providers be trained or retrained in the use of these new tools?
• Is informed consent appropriate? What do patients and consumers need to know?
• How will the “standard of care” evolve over time?
• Where, when the care paradigm changes, does liability lie: with the human or with the device? Or both?

To begin to answer these questions, it is helpful to use the concept of OODA loop, a four-step process for effective decision-making in high-stakes situations. This paradigm, observe–orient–decide–act, was developed in a military context by Air Force Colonel John Boyd.

As with several issues related to AI and autonomy, paradigms from the military are helpful. We must consider where the human (physician or other healthcare provider) is relative to this loop. She may be in it, on it, or entirely out of it. Each position impacts regulatory claims (“indications for use”), provider literacy, informed consent, care pathways, and standard of care and liability.

Motivated by improved quality for more people at lower cost, AI can and will change the “standard of care.” The rate of change will be a function of three over-arching considerations:

• What is the magnitude of the improvement that AI can produce in safety and effectiveness?
• What is the level of stakeholder interest: Will specific providers embrace the tools or be reluctant to adopt them?
• How many people will be impacted?

One can imagine that where data demonstrate impact, there is the potential to reshape care delivery paradigms, perform procedures differently, and use new diagnostic tools. Outcome data and the generation of evidence will be paramount.

Healthcare lawyers will be central to this care paradigm evolution. In particular, legal expertise will be required to:

• Define and ensure access to foundational datasets while maintaining privacy.
• Lay out the intellectual property framework for AI algorithms and derived products.
• Weigh in on the redefinition of the standard of care.
• Certainly weigh in on the tort and product liability implications of the evolving standard of care.

While there is certainly significant uncertainty that some consider “the dark side,” the challenges and opportunities for the practice of medicine and the practice of healthcare law might rather be reframed as “not yet clearly light.”