AI Chatbot’s Medical Claims Draw Regulatory Scrutiny

On May 1, 2026, the Pennsylvania State Board of Medicine filed a complaint in the Commonwealth Court of Pennsylvania against Character Technologies, the corporate entity operating the Character.AI generative artificial intelligence platform.¹ The complaint raises immediate questions about state licensing board enforcement, but the regulatory picture it reveals extends further – to US Food and Drug Administration (FDA) oversight and an accelerating wave of state legislation targeting AI in healthcare. Character Technologies also faces a separate lawsuit brought by the Kentucky attorney general, which alleges that the company preys on children and leads them to self-harm.²

Background

The platform and the investigation

Character.AI is a generative AI platform with 20 million+ monthly users that allows users to create chatbot characters with specific personalities. A Pennsylvania Professional Conduct Investigator created an account, searched “psychiatry” and interacted with a character named “Emilie” described as a “Doctor of psychiatry.” Note that the character had approximately 45,500 user interactions as of mid-April, during which “Emilie” claimed to have medical credentials, offered to conduct a psychiatric assessment and represented that it held a valid Pennsylvania medical license, providing a fabricated license number.

Character Technologies does not hold a license to practice medicine in Pennsylvania.

The commonwealth’s case

Pennsylvania asserts that Character Technologies engaged in the unauthorized practice of medicine and surgery.³ The crux of the state’s allegations is that Character Technologies permitted its chatbot to hold itself out as a licensed psychiatrist by claiming a Pennsylvania license, using the title “psychiatrist” and providing a fabricated license number.

Character.AI contests the suit, reasoning that its user-created characters are fictional and intended for entertainment and roleplaying. The company points out that the platform includes in-chat disclaimers stating that characters are not real people and all statements should be treated as fiction, along with additional disclaimers warning users not to rely on characters for professional advice.⁴

Legal issues

State licensing

In Pennsylvania, medicine and surgery is defined as “[t]he art and science of which the objectives are the cure of diseases and the preservation of the health of man, including the practice of the healing art with or without drugs, except healing by spiritual means or prayer.”⁵ Medical doctors, including psychiatrists, as with most distinct healthcare professions (e.g., nurses, physician assistants, etc.), are licensed at the state level.

Further, Pennsylvania, like other states, prohibits the unauthorized practice of medicine, which includes:

1. Practicing medicine.
2. Purporting to practice medicine.
3. Holding forth as authorized to practice medicine through use of a title.
4. Otherwise holding forth as authorized to practice medicine.⁶

Given the breadth of these statutory prohibitions, the bar for demonstrating the unauthorized practice of medicine appears low. For example, a platform need not deliver clinical care in the traditional sense to run afoul of the statute; merely holding itself forth as authorized to practice medicine, whether through the use of a title, the assertion of credentials or other representations of licensure, may be sufficient. In this case, the complaint expressly alleges that the “Emilie” character represented that it was a medical doctor, claimed to have attended medical school at Imperial College London and to have been practicing psychiatry for seven years, asserted that it was licensed to practice medicine in Pennsylvania, and provided a fabricated Pennsylvania license number. Each of these allegations, standing alone or in combination, may be used as evidence that the chatbot held itself out as authorized to practice medicine.

‘Intended use’ and FDA’s medical device regulatory framework

The Character.AI matter also raises significant questions under federal law – specifically, whether a chatbot that performs diagnostic or treatment-related functions could be classified as a medical device⁷ subject to FDA oversight. Platform operators and their counsel should not assume that the absence of FDA enforcement to date reflects a settled regulatory position; to the contrary, the agency’s existing statutory and regulatory framework is more than sufficient to reach AI chatbot platforms with these types of functions, and the Pennsylvania complaint may accelerate federal attention to this space.

Under the Federal Food, Drug, and Cosmetic Act (FDCA), a product qualifies as a “device” if it is “intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease” or is “intended to affect the structure or any function of the body” – provided that, unlike a drug, it does not achieve its primary intended purposes through chemical action within or on the body and does not depend on being metabolized to achieve such purposes.⁸ Critically, FDA does not simply accept a company’s characterization of what its product is intended to do. Under 21 CFR § 801.4, a product’s “intended use” can be established by, among other things, its design, the circumstances surrounding its distribution, website claims, advertising, and oral and written statements. FDA evaluates the totality of the circumstances – how a product is actually used, what it actually communicates and what the objective evidence shows about the manufacturer’s intent.

Importantly, FDA regulates Software as a Medical Device (SaMD) in the same manner as other products, unless the software is subject to one of the statutory carve-outs from the 21st Century Cures Act, such as software intended for general wellness purposes.⁹ Thus, software that is intended for use in the diagnosis or treatment of a disease or condition is subject to regulation as a medical device under the FDCA.

While the FDCA may already provide a basis for reaching chatbot operators, enforcement to date has largely been driven by state attorneys general rather than FDA. That gap likely reflects issues of timing and resource constraints rather than any meaningful limitation in federal authority. In the current environment, states like Pennsylvania also appear more willing to devote their limited resources to enforcement in this space. For platform operators, that combination of latent federal authority and active state-level activity means the question is not whether regulatory scrutiny is coming, but how to be ready as it continues to evolve.

The best defense is a good offense

So, what can platform operators do now to get ahead of the regulatory curve? First, they can start with a regulatory risk assessment to map the landscape of applicable state laws across all jurisdictions in which the platform operates before deploying health AI features.

Based on that assessment, platforms can strengthen their regulatory position by calibrating their compliance practices either to the highest applicable state standards or to emerging national frameworks. The Federation of State Medical Boards, for example, announced in May 2026 the formation of a new workgroup charged with developing recommendations and model guidelines for state medical boards on the regulation of AI tools used in the practice of medicine. At the federal level, and as discussed further below, the Trump administration has also signaled its desire to establish a uniform federal framework for AI.¹⁰

Platform operators should also define and implement clear boundaries around what their AI systems can do in all healthcare contexts. This does not mean shutting down all health-related conversations, but it does mean drawing a line between providing educational information or a general wellness function and conduct or messaging that may appear to be providing clinical advice requiring a professional license, which is a distinction that matters equally under state unauthorized practice statutes and the FDA’s device classification framework. A chatbot offering generic stress-management tips will be analyzed differently than one that asks about symptoms, offers a diagnosis or recommends a treatment course. Those boundaries should be enforced through content moderation systems and model-level constraints, not through user-facing disclaimers alone, given that a company’s disclaimers may actually be used to demonstrate knowledge of the law and do not change a product’s status as a device under the FDCA.¹¹> Platforms that build these guardrails in before a regulator comes knocking will be in a far stronger position than those that wait and react.

Will the Character.AI case open the floodgates?

It is too early to say whether the Character.AI lawsuit will open the floodgates for state enforcement actions, but the conditions are there. State licensing boards now have a live case that hands them a roadmap for going after AI platforms whose responses stray into regulated territory. And they are not the only ones: A growing number of state legislatures have moved to regulate AI systems directly (e.g., California, Texas and Illinois), and more will follow.

These developments suggest a regulatory landscape that may become both broader and more varied over time – though federal pressure on state AI regulation is mounting. On December 11, 2025, President Donald Trump signed an executive order directing federal agencies to establish “a minimally burdensome national policy framework for AI.” While the order does not preempt existing state AI laws, it identifies several mechanisms for challenging state AI laws inconsistent with that policy, including Department of Justice litigation, Commerce Department review of “onerous” state laws, and a White House mandate to prepare a legislative recommendation establishing a uniform federal framework that would preempt state laws conflicting with the administration’s policy of sustaining and enhancing US global AI dominance through a minimally burdensome national framework.¹²

For now, state AI compliance obligations remain in effect. The scope of these regulations varies considerably from state to state, ranging from disclosure requirements mandating that users be informed they are interacting with an AI agent to data privacy obligations, advertising restrictions and other consumer protection measures. Of particular relevance to the issues raised by the Character.AI matter, Delaware recently enacted legislation that expressly prohibits a “nonhuman entity,” including an “agent powered by artificial intelligence,” from using professional titles or abbreviations associated with licensed healthcare professions, including, but not limited to, “advanced practice registered nurse,” “registered nurse,” “doctor” and similar designations.¹³ The Delaware law further prohibits the licensure of a nonhuman entity to practice medicine, nursing or related healthcare professions, and bars any such entity from engaging in the practice of medicine within the state. Legislation of this nature may reflect a growing desire among state legislatures to expressly address this practice in an attempt to rein in AI platforms that offer medical advice without state oversight – though their durability will depend on whether federal legal challenges to these laws materialize and succeed, or whether Congress moves to preempt them through a federal AI framework.

What makes the Pennsylvania case especially notable is how it started – not with a purpose-built health app, but with a single chatbot on a general-purpose platform that a state investigator found by searching “psychiatry.” The takeaway: Regulators are looking at what the AI actually says, and if those responses look like the practice of a licensed profession or the function of a regulated device, disclaimers may not be enough. That said, enforcement is not the only model. Some states have signaled a preference for regulatory partnership over litigation. Utah, for example, has entered into a regulatory mitigation agreement with mental health chat app ElizaChat, under a framework created by Utah law¹⁴ that allows companies to operate under agreed terms in exchange for regulatory flexibility. Whether other states follow Utah’s lead remains to be seen, but the gap between a regulatory partnership and an enforcement action may come down to whether the platform drew the lines itself before a regulator had to – or, where a regulator has already drawn them, whether the platform engaged constructively with those boundaries rather than ignoring them.

Notes

1. The Pennsylvania State Board of Medicine operates under the Pennsylvania Department of State, Bureau of Professional and Occupational Affairs.
2. Commonwealth of Kentucky ex rel. Coleman v. Character Technologies, Inc., No. 26-CI-00029 (Ky. Franklin Cir. Ct. filed Jan. 8, 2026).
3. In violation of Sections 422.10 and 422.38 of the Medical Practice Act.
4. Cailey Gleeson, “Pennsylvania Sues Character.ai Over AI Chatbot Allegedly Presenting Itself as Licensed Medical Professional,” Fierce Healthcare, May 7, 2026.
5. 63 Pa. Stat. Ann. § 422.2.
6. 63 Pa. Stat. Ann. § 422.10.
7. 21 USC § 321(h)(1).
8. Id.
9. 21 USC § 360j(o). See also, Cooley, “FDA Opens Aperture for Wearables in Latest General Wellness Guidance,” January 20, 2026.
10. “Ensuring a National Policy Framework for Artificial Intelligence,” Exec. Order No. 14365, 90 FR 58499, December 11, 2025).
11. See, e.g., United States v. 789 Cases of Latex Surgeons’ Gloves, 799 F. Supp. 1275, 1285 (D.P.R. 1992) (“Whether a product’s intended use makes it a device depends, in part, on the manufacturer’s objective intent in promoting and selling the product. All of the circumstances surrounding the promotion and sale of the product constitute the ‘intent.’ It is not enough for the manufacturer to merely say that he or she did not ‘intend’ to sell a particular product as a device.”).
12. “Ensuring a National Policy Framework for Artificial Intelligence,” Exec. Order No. 14365, 90 FR 58499, December 11, 2025. See also, Cooley, “Showdown: New Executive Order Puts Federal Government and States on a Collision Course Over AI Regulation,” December 12, 2025.
13. Del. H.B. 191, 153d Gen. Assemb. (2026).
14. UT Code § 13-72-302.