FTC’s Approach to AI in Health Care
The FTC is taking proactive steps to become the leading AI regulator and to define appropriate practices in the AI context. Chair Lina Khan has been at the forefront of this initiative. Some of the most active efforts in this area have been a response to what FTC leadership believes was a failure in regulating privacy in the early days of the Internet. According to an op-ed by Khan, “the trajectory of the Web 2.0 era was not inevitable—it was instead shaped by a broad range of policy choices. And we now face another moment of choice. As the use of A.I. becomes more widespread, public officials have a responsibility to ensure this hard-learned history doesn’t repeat itself.”
In the same article, she laid out some key goals for policing AI, including:
- Strengthening consumer protection: Safeguarding users from deceptive and fraudulent practices enabled by AI, such as phishing scams, deepfake videos, and voice cloning.
- Promoting data privacy: Monitoring AI systems to ensure they adhere to data protection laws and prevent exploitative data collection or usage, protecting users’ personal information.
- Combating discriminatory practices: Ensuring AI systems don’t perpetuate or amplify biases and discrimination, which can lead to unfair treatment in areas like employment, housing, or access to essential services.
For health care, however, the approach has been even more aggressive (although still informal). In a February 2024 speech, Khan stated that sensitive personal data related to health, location, and web browsing should be “simply off limits” for training artificial intelligence models. Khan explained the FTC is creating “bright lines on the rules of development, use and management of AI inputs” and that companies that want to use data they’ve already collected for AI training must actively notify users of this change.
Potential Impact of FTC’s Approach on AI in Health Care
While developing appropriate consumer protections for AI is critical, this policy perspective from the FTC goes too far. It would have the impact of essentially prohibiting AI in health care—from having the broadly defined health care system use the last 100 years of critical data to benefit the health care system and all of the patients it serves. This approach would prevent enormous benefits for consumers. Khan asserts that health care data should be “off limits” for training AI and that new uses of data for AI purposes require some additional new notice and consent—a real-world impossibility for data accumulated over the past 100 years. Taking this position broadly would prevent many of the benefits of AI in health care for industry and consumers alike.
Furthermore, the proposed AI policy drastically breaks precedent of approaches to AI in the last decade. During the Obama presidency, the administration consistently stressed the importance of building and applying AI for the public good. More recently, President Biden’s mandate on AI has largely been that AI holds both promise and potential peril. The Biden administration has focused on “responsible AI in health-related fields.” For example, the administration gathered commitments from top health AI companies on the safe, secure, and trustworthy use and purchase and use of AI in health care. Part of the pledge includes dedication to the “FAVES” principles—that AI should lead to health care outcomes that are Fair, Appropriate, Valid, Effective, and Safe.
The ramifications of the FTC’s proposed AI policy approach would only further intensify existing concerns about the shortcomings of applying AI to health care contexts. For example, requiring consent to use anonymized health data for AI training in health care would perpetuate prevailing concerns about bias in existing health data against underrepresented patient subgroups, as some racial, ethnic, and gender groups could be more hesitant to consent to their data being shared. In addition, it would make it harder for medical device performance to be generalized to the entire intended patient population. It also can lead to overfitting of training data, where subsequent AI models cannot generalize as they fit too closely to the training dataset. In other words, the new FTC policy would pave the way for AI to leave behind the most vulnerable health populations, like racial and ethnic minorities, as well as people with disabilities.
Benefits of AI in Health Care That Proposed FTC Policy Would Block
AI has shown and continues to show potential to fundamentally change health care by providing patients more precise diagnoses and treatment recommendations tailored to their individual needs. For example, AI can help doctors and health care workers deliver higher-quality and more empathetic care to patients in communities across the country. It even can help develop solutions that advance health equity and access to care and make health care more affordable. Below are various ways that AI can improve the health care ecosystem.
AI Provides Higher-Quality Patient Care
AI’s broader adoption could help doctors and health care workers deliver higher-quality diagnoses and care to patients in communities across the country. AI can improve data processing, which in turn helps clinicians with identifying patterns and generating insights that might have been missed by a physician’s efforts alone. For example, clinicians have traditionally used tools like the Modified Early Warning Score (MEWS), which calculates the risk for clinical deterioration in a patient in the next few hours. AI would automate this process, assisting doctors in noticing any potential flags relating to a patient. Similarly, Google Health developed a program that predicts the onset of acute kidney injury two days prior to when the injury occurs. This is a huge improvement in patient care as, in current medical practice without AI, health care professionals are only able to detect kidney failure after it happens. Similarly, AI can help identify pulmonary nodules to ensure early detection of deadly tumors and lung cancer. In these scenarios, AI helps by automatically identifying pulmonary nodules and categorizing them as benign or malignant. Even for mammographies, AI can help in the interpretation of microcalcifications in breast tissue to assist in the early detection of breast cancer. Such health care AI technologies push the boundaries of patient care beyond what health care providers would otherwise be able to provide patients, and help provide patients more precise diagnoses and higher-quality health care.
Along with such improvements in diagnoses in patient care, health care AI also can democratize medical knowledge. AI can share the expertise and performance of specialist doctors to supplement the knowledge of generalist doctors. For example, new AI technologies can expand access to ophthalmology and radiology through AI image analysis, two specialties where wait time is generally quite long and expensive. This can especially help underserved communities gain access to the specialized care that otherwise might not have been possible.
The expansion of care is even more important in the United States where 30% of the population lives in a health care desert with insufficient access to primary care and hospitals. AI can help close this divide by providing health insights and diagnostics tools to expand access to health care in these underserved communities. One example of how AI could be used in these underserved communities is for labor, where AI technologies could help detect abnormalities during labor that could protect mothers and babies during childbirth by flagging issues in time for mothers to access hospitals hours away. In these communities, AI health care tools can be the difference between life and death.
AI Is Cost Saving for the Entire Health Care System
Researchers estimate that the adoption of AI in health care could lead to savings of $200 to $360 billion a year, roughly 5–10% of health care spending. For hospitals, these savings come from AI’s contributions to improving clinical operations, like optimizing operating rooms and detecting adverse events. The benefits are similar for savings for physician groups, where AI would benefit the management of continuity of care, like referral management. Health insurers would benefit from AI with claim management and automation of prior authorization. And most importantly, all these savings would, in turn, be passed to patients themselves. Private payers, for example, could save roughly 7% to 9% of their total costs, amounting to $80 to $110 billion in annual savings within the next five years.
AI Benefits Patient Comprehension Through Chatbots
AI can help patients make more informed health choices by better understanding their health conditions and needs through chatbots. One example is Penny, an AI-run chatbot used by the University of Pennsylvania Cancer Center. Cancer patients who are receiving oral chemotherapy at the University of Pennsylvania are contacted daily by Penny to confirm their medication plan for the day and to ask about their well-being and side effects. If Penny detects that any of the patient’s messages are reason for concern, Penny will alert clinicians at the University of Pennsylvania to contact the patient and potentially arrange an in-person visit. Penny’s use is twofold: She helps patients better understand complex medication schedules and also helps clinicians continue to be informed of the patients’ conditions after the patient is discharged. Such chatbots are being applied to various types of health care, such as to check in on pregnant mothers.
AI Automates Basic Tasks in Health Care, Saving Medical Providers Time
AI can automate many of the daily computer tasks that take up a doctor’s time, saving the health care provider time and allowing them to spend more clinical time with a patient. Health care providers spend a tremendous amount of time navigating electronic medical records, documenting patient data, and typing. AI systems can significantly help reduce this burden on doctors by queuing up relevant information in patient records and distilling recordings of medical appointments into patient data.
AI systems also can help save physicians time by identifying significant visual markers in imaging, for example, in radiology technology. Normally, this is a very tedious and meticulous task that takes hours of human physician labor. AI can automate this task, allowing physicians to spend more time with patients.
AI Can Produce Precise Quantitative Outcomes for Image-Recognition Tasks
One area for which AI has demonstrated significant benefit is medical image recognition tasks, which are traditionally reserved for physicians. AI can contribute to medical image interpretation by recognizing complex patterns in imaging data and providing quantitative assessment of the radiology images. Physicians are still responsible for interpreting these quantitative data to make a final qualitative assessment of the diagnosis, but AI quantitative data bolster this ultimate assessment. Such AI medical image assessments would be helpful in several medical fields that rely on medical images, including radiology, pathology, and dermatology.
AI Makes Designing Health Care Systems More Efficient
AI also can be an incredibly helpful internal tool for health care systems when it comes to allocating resources and designing health care systems. AI can help departments predict short-term staffing issues. It also can help with questions about resource allocation for patients through predictive inventory management. AI can be used for more efficient facility management, including HVAC systems and predictive maintenance, or for patient flow and scheduling optimization through predictive analysis of admissions, discharges, and transfers. All of these applications of AI could help health care facilities run more efficiently and maximize revenue, while also reducing wait time for patients.
AI also can make the health care system more efficient by automating administrative tasks. This could include things like patient data management such as EMRs and unstructured data analysis, or an AI-driven scheduling system for appointments. AI could even help with data security and compliance monitoring.
From a more practical perspective, AI can even help with non–health care basics in the health care ecosystem. For example, AI can enable faster payments and greater claims accuracy with insurance companies. This could ultimately come back to benefit patients if insurance companies trust health care systems’ claims more due to their increase in accuracy.
AI in Health Care Does Not Need to Produce the Entirety of the Answer
When considering the application of AI to health care, one important consideration is that AI does not need to produce the entirety of an answer or diagnosis. AI can be applied to specific decision points in the health care context that can help augment a health care provider’s larger diagnosis or task. AI in fact can be most helpful when it is applied to a specific context or decision that benefits the larger human decision-making process.
One example of this is how AI can provide doctors with real-time analytics of patient data through mobile devices. By collecting precise data, AI can expedite and optimize doctors’ abilities to make decisions on these data, with AI only providing data for the larger diagnosis led by the physician. Real-time data analytics collected by AI also could inform health care professionals of a patient’s status or declining condition. In these ways, AI can supplement a health care provider’s decision-making process without totally overtaking and being responsible for the entirety of a diagnosis.
AI Is Uniquely Capable in Drug Discovery, Which Can Be Lifesaving
AI can bring unprecedented assistance in the field of drug discovery, which can drastically reduce the time required to attain lifesaving drugs. Drug discovery is known to be expensive, inefficient, and often fraught with failure. AI can rapidly improve this process. For example, AI can make a digital twin of potential drug candidates and then can run millions and billions of simulations based on the twin drugs. Of course, clinical trials are still necessary before the U.S. Food and Drug Administration (FDA) can allow the drug to be open to clinical use, but the AI simulations can drastically streamline the long and arduous drug discovery process and increase the number of approved drugs.
Another aspect of AI in drug discovery is that AI helps precision design by determining which patients are potential responders and nonresponders during drug trials. In practice, this means AI performs in-depth analyses of participants before a trial to identify certain biomarkers in patients. This deep analysis helps researchers understand which patients are likely to respond to the target drug, and why others may not. This analysis is important because it helps drug manufacturers hone in on how a specific drug might help certain segments of the population and helps further target drugs to patient groups.
AI Is Predicted to Continue to Show Immense Contributions to Personalized Medicine
Beyond its current capabilities, AI promises even more helpful applications in a health care context. There are several emerging trends and potential impacts of AI in health care. One area is personalized medicine, where AI could help revolutionize treatment for patients with genetic conditions as AI could more precisely cater care plans to individual patients. AI also could integrate real-time health monitoring data into the tailored treatment plans to achieve more dynamic treatment plans. Another area AI is projected to impact in health care is AI-powered tools for health and sleep monitoring. AI could improve detection of sleep disorders and offer personalized treatment by designing intervention technologies to monitor, predict, and manage sleep issues. AI even could help with global health monitoring systems. AI could strengthen global health security by enabling response to disease outbreaks by integrating diverse data in real time and more rapidly adapt models to respond to emerging health threats. Although these AI technologies are not finalized yet, they are predicted to be discovered and applicable in the near future.
Existing Solutions That Can Address Concerns About AI’s Application to Health Care
Rejecting the proposed FTC approach does not mean that AI, in health care or otherwise, should not be regulated appropriately. In fact, there are currently several protections that already guide AI’s application to health care. For one, the academic community has started to develop reporting guidelines for AI clinical trials. These trials ensure the reliability of AI algorithms in the health care context and ensure the algorithms are safe and robust before being launched.
Additionally, the FDA approves marketed medical AI devices and publishes a summary document for each device, highlighting the performance data of the device’s evaluation study. The devices in the FDA’s list have all met the FDA’s overall safety and effectiveness requirements, which includes an evaluation of appropriate study diversity based on use and technological characteristics. Academics can then use the FDA data to further analyze and study the device approval process.
There is valid criticism that these evaluations have been retrospective studies and that samples are not always reported. These studies also spark concern that more prospective studies are needed to fully evaluate the impact of AI on health care practice. And that is exactly where efforts should be: working to improve the limitations of current health AI evaluation processes to better address vulnerabilities of these devices, through rigorous studies, across multiple clinical sites, and across representative patient populations. These are the solutions that will ensure health AI concerns are mitigated by reducing the risk of overfitting data and better capturing clinical outcomes. Taking AI off the table isn’t the solution.
AI Would Supplement Rather Than Replace Clinical Expertise
Health care companies already are cognizant of how AI tools are applied to supplement rather than replace clinical decision-making because of state scope-of-practice laws. State scope-of-practice laws prevent unlicensed individuals from practicing medicine. Health care AI companies therefore must ensure their AI tools do not creep into clinical decision-making positions in violation of these state rules.
A more accurate way to characterize AI is as a tool that can save health care providers time and allow them to spend more time with patients. AI is not taking over health care; rather, physicians can rely on AI to make better diagnoses and provide higher-quality patient care.
Achievable Steps That Can Be Taken to Address Bias in Health Care AI
There is a valid concern that AI health care–related decisions can be biased by gender or race or other important criteria. Such biased health AI outcomes are born when AI is trained on data that do not represent the population it aims to treat. To mitigate this risk, a large volume of data is needed to train the AI to more accurately infer health solutions for patients. The best way to achieve a higher volume of useful health care data, from a wide range of individuals, is to continue the current model of allowing health care AI to be trained on a diverse range of patient data. The FTC’s proposed policy directly perils a diverse range of patient data: The policy limits, rather than expands, the diverse range of data on which AI tools can train.
AI Tools Affect Doctors Differently
Research shows that AI tools impact clinicians in different ways, suggesting that some clinicians can adequately mitigate the negative consequences of AI while others might struggle to do so. This suggests that a one-size-fits-all solution, like a proposal to ban health care data from AI, would not meet any concerns about physician misuse of AI. One researcher states, “We find that different radiologists, indeed, react differently to AI assistance—some are helped while others are hurt by it.” The researcher suggests that radiologists therefore should be treated differently in interpreting clinician error with AI as AI might not interfere or contribute to radiologist performance uniformly. AI is not one-size-fits-all, and any misuse of AI can be physician specific.
Taking health AI off the table, as the FTC’s new policy mandates, is not wise. The most nuanced path forward is to realize the tremendous potential for AI to transform health care and health outcomes for all patients, and to engage AI makers, clinicians, patients, ethicists, and others in legal debates on how AI should be implemented in health practice. Broad use of existing data—from dozens of years of evolving treatment regimes—can provide a treasure trove of opportunity for appropriate development of useful and beneficial AI. Privacy and security protections are critical in using these data, along with other concerns about bias, discrimination, and, of course, overall accuracy. Simply banning AI in health care does not address concerns surrounding the use of AI and deprives multiple stakeholders—including the patient community—from the benefits AI in health care can offer.