Will AI Be Your New Doctor? Probably Not, Thanks to Recent Trends in State Regulation

As the use of artificial intelligence (AI) becomes increasingly commonplace, some have raised concerns about its use in medicine, particularly when AI assumes a role that is traditionally held by a physician or other licensed healthcare professional (HCP). Recently, states have begun to regulate AI in medicine, attempting to limit what they view as the software’s practice of medicine and improve transparency in health-related communications.

Trend watch: States regulate AI’s role in the practice of medicine

AI prior authorization denials

In June 2025, Nebraska became the latest state to pass legislation requiring a physician or other qualified HCP to provide final review of any health insurance claim before it can be rejected.¹ Six states – Arizona, California, Indiana, Maryland, Nebraska and North Dakota – now have such laws, with nearly two dozen more states introducing similar bills this year.

These laws generally have two components. First, they expressly prohibit AI (and other similar software tools) from denying, delaying or modifying healthcare services without the review and approval of an HCP who specializes in the relevant type of medicine. Second, if an HCP still wishes to use AI to assist in their review, these laws then dictate numerous criteria the AI must meet, including a guarantee that the AI will base its decisions on individual (rather than group) data and will not discriminate against particular groups of patients.

Increased regulation of prior authorization comes as a response to intense public scrutiny of the practice, with multiple veins of criticism. Roughly a quarter of prior authorization requests are denied,² a number some argue is higher than it should be because decisions are being based not on medical necessity but on the generation of profits for insurance companies. Courts have recognized that prior authorization based on medical necessity is a “medical decision,”³ so there is also concern that, by vesting prior authorization decision-making authority in someone other than a licensed HCP, insurance companies are practicing medicine without a license.⁴ Lastly, although 80% of prior authorization denials are eventually overturned, appealing denials delays treatment, wastes physicians’ time and can cause harm to patients.⁵

The proliferation of AI has caused some to raise concerns that insurers are using the software to keep prior authorization denials high, irrespective of medical necessity. Because AI software is not an HCP, some regulators believe that its determinations on medical necessity could be viewed as practicing medicine without a license.

AI chatbots

A few states have passed laws requiring all AI chatbots in the healthcare context to provide an upfront disclosure that the chatbot is AI rather than a real person. Although these laws have so far proven to be less popular than the laws related to AI in prior authorization, the chatbot disclosure laws that have been passed are quite comprehensive. California’s AI chatbot law, for example, requires healthcare entities to disclose the use of AI in any type of communication with patients, not just when those communications are related to medical advice.⁶ Colorado’s AI chatbot law goes even further, requiring disclosure of a business’s use of AI chatbots for all communications (not just healthcare communications) with Colorado residents, unless it would be “obvious” that the chatbot is AI. 

Other AI chatbot disclosure laws are aimed specifically at misrepresentation in the context of mental health or prohibit certain uses of data obtained by suppliers of mental health chatbots. Utah law now requires mental health AI chatbots to immediately disclose to users that the chatbot is AI technology, not a licensed mental health care provider, and prohibits selling or sharing individually identifiable health information in some circumstances.⁷ New Jersey, meanwhile, is currently considering a bill that would stop chatbot developers from marketing or otherwise presenting chatbots as licensed to provide mental health services.⁸ Additionally, Oregon – so far the only state to adopt such an approach – now has a law prohibiting chatbots from bearing the title “nurse” or any other similar title to ensure that AI chatbots do not present themselves to patients as HCPs. 

Note that these state laws do not contemplate other regulatory regimes that could govern the software, namely the federal Food, Drug, and Cosmetic Act, if the intended uses of the software render it a medical device.

Implications for clients

While AI interventions have many beneficial uses in the medical context, 2025 has brought a rise in state laws that challenge or limit the use of AI products in this way. These state laws require additional regulatory considerations, in addition to the regulation of software as a medical device by the US Food and Drug Administration at the federal level. Legislative trends suggest that the wave of state regulation is just the beginning. Almost all the new state laws discussed above were introduced for the first time within the past year, and passage of any of the dozens of proposed laws in the coming years could mark a paradigm shift in the use of AI in medicine. This is especially true if, like the bills introduced in Minnesota and Connecticut, any of these new laws fully prohibit the use of AI in the prior authorization process. 

Nevertheless, these new laws do not represent state regulators’ rejection of a role for AI in medicine. The vast majority of the prior authorization laws still allow HCPs to use AI in the prior authorization process as an assistive tool or a means of more quickly approving such claims.

AI chatbots are also still permitted, provided that the company deploying them provides the requisite reports and notifications. 

It is possible that federal legislation could nullify state law restrictions on AI in medicine. The “One Big Beautiful Bill Act,” recently signed into law, initially included a provision that would have instituted a 10-year ban on state-level regulation or limitation of AI technologies.⁹ Although this language ultimately did not make it into the final version of the law, the door remains open for the US Congress to determine whether new federal regulation of AI would be more appropriate than a patchwork of state laws.

For more insights into how AI in medicine regulations may impact your business, or for help tracking the legislative landscape, please contact any of the lawyers listed below.

Notes

1. Ensuring Transparency in Prior Authorization Act, LB 77 (Neb. 2025), https://legiscan.com/NE/text/LB77/id/3252446/Nebraska-2025-LB77-Chaptered.pdf.
2. Hallie Levine, “Prior Authorization: What Is It, When Might You Need It, and How Do You Get It?,” Harvard Medical School, Aug. 5, 2024, https://www.health.harvard.edu/staying-healthy/prior-authorization-what-is-it-when-might-you-need-it-and-how-do-you-get-it.
3. See, e.g., Murphy v. Board of Medical Examiners of State of Ariz., 190 Ariz. 441, 446 – 47 (1997).
4. Nick Thomas & Michael J. O’Neill, “Members Brief: Prior Authorization and the Practice of Medicine,” Legislative Budget Office’s Office of Research and Drafting, 134 (34), Mar. 16, 2022, https://www.lsc.ohio.gov/assets/organizations/legislative-service-commission/files/prior-authorization-and-the-practice-of-medicine.pdf.
5. Levine, supra note 2.
6. Health Care Services: Artificial Intelligence, AB 3030 (Ca. 2024), https://legiscan.com/CA/text/AB3030/id/2979649/California-2023-AB3030-Amended.html.
7. Artificial Intelligence Amendments, HB 452 (Utah 2025), https://le.utah.gov/Session/2025/bills/enrolled/HB0452.pdf.
8. AB 5603, 221st Leg., (NJ 2025), https://pub.njleg.state.nj.us/Bills/2024/A6000/5603_I1.PDF.
9. One Big Beautiful Bill Act of 2025. Public Law 119 – 21, 119th Congress, 4 July 2025. Congress.gov, https://www.congress.gov/bill/119th-congress/house-bill/1.

Cooley summer associate Thomas Dell also contributed to this alert.

This content is provided for general informational purposes only, and your access or use of the content does not create an attorney-client relationship between you or your organization and Cooley LLP, Cooley (UK) LLP, or any other affiliated practice or entity (collectively referred to as "Cooley"). By accessing this content, you agree that the information provided does not constitute legal or other professional advice. This content is not a substitute for obtaining legal advice from a qualified attorney licensed in your jurisdiction, and you should not act or refrain from acting based on this content. This content may be changed without notice. It is not guaranteed to be complete, correct or up to date, and it may not reflect the most current legal developments. Prior results do not guarantee a similar outcome. Do not send any confidential information to Cooley, as we do not have any duty to keep any information you provide to us confidential. When advising companies, our attorney-client relationship is with the company, not with any individual. This content may have been generated with the assistance of artificial intelligence (Al) in accordance with our Al Principles, may be considered Attorney Advertising and is subject to our legal notices.