The Informed Consent Problem Nobody Is Talking About
Mental health AI has a consent problem it cannot disclose its way out of. The failure is not in the fine print. It is in the design.
There is a foundational principle in clinical practice that has governed the relationship between a practitioner and a patient for decades. It predates digital health, predates managed care, predates every wave of technological disruption the field has absorbed. Informed consent is not a form. It is not a legal formality or a compliance checkbox. It is the mechanism by which a person in distress retains their autonomy in the moment they are most likely to surrender it.
Mental health AI has largely abandoned it.
Not by intention, at least at the outset. Through a combination of commercial pressure, the genuine difficulty of translating clinical ethics into product requirements, and a field-wide tendency to treat consent as a problem for someone else to solve. The result is that millions of people are engaging with systems that perform therapy-like functions through design, language, and affect, without those people having any meaningful understanding of what they are actually interacting with. The field is debating disclosure frameworks when it should be demanding something far more foundational.
Not All Mental Health AI Carries the Same Obligation
This argument does not apply equally to every mental health-related AI tool. A scheduling assistant, a journalling prompt generator, a clinician-facing documentation tool, a regulated digital therapeutic, a general-purpose chatbot, and a companion system marketed for emotional support do not carry the same consent burden. The obligation rises with clinical function. When a system uses therapeutic language, responds to distress, encourages disclosure, simulates empathy, retains memory of personal material, exposes users to crisis situations, and cultivates relational trust over time, it has entered territory where ordinary consumer disclosure is no longer adequate. The ethical standard should be determined by what the product does to the user, not what the company chooses to call it.
Regulators have already arrived at a version of this position. The FTC’s September 2025 inquiry into AI companion products was not limited to tools that claim to provide therapy. It concerned systems that simulate human-like relationships, prompt trust, and may affect children and teenagers. Relational design itself is becoming a regulatory concern, independent of medical-device classification. That matters for every founder and governance team building in this space.
What Informed Consent Actually Is
The AMA’s Code of Medical Ethics is direct: informed consent to medical treatment is fundamental in both ethics and law. Patients have the right to receive information and ask questions about recommended treatments so that they can make well-considered decisions about care. Successful communication in the therapeutic relationship fosters trust and supports shared decision-making. Withholding information without the patient’s knowledge or consent, except in genuine emergencies, is ethically unacceptable.
The APA’s framework on informed consent in clinical practice goes further. It identifies three conditions for valid consent: the information must be given, the person must be competent to receive it, and the consent must be voluntary. The informed consent process promotes ethical practice by fostering respect for the client’s right to dignity, autonomy, justice, and integrity. It is not simply a mechanism of disclosure. It is the first clinical act. It establishes the nature of the relationship, and it allows the person to decide, with full understanding, whether to enter it.
Mental health is not a domain where this principle can be lowered. People arriving at a mental health resource are frequently in distress, often cognitively affected by that distress, and specifically susceptible to misreading what is being offered to them. The vulnerability that makes the therapeutic relationship powerful is the same vulnerability that makes uninformed engagement dangerous. You cannot separate these facts.
The Misconception Is Not Accidental
There is a well-established clinical concept called therapeutic misconception. It was first described in psychiatry in 1982 by Appelbaum, Roth, and Lidz, who documented that even educated patients in psychiatric research failed to understand the distinction between research participation and clinical treatment. Their concern was that participants overestimated what was being done for them and underestimated the risks they were accepting by not fully grasping that distinction.
Therapeutic misconception was identified as a problem in controlled research settings, where teams are obligated by ethics boards to obtain consent, where processes are reviewed, and where the gap between expectation and reality is generally modest. Mental health AI is deploying this phenomenon at population scale, without ethics board review, without informed consent processes, and in some cases with product design that actively reinforces the misconception rather than correcting it.
A paper published in Frontiers in Digital Health documents this directly: there can be significant misunderstandings about the exact purpose of a mental health chatbot, particularly in terms of care expectations. Ignorance or misunderstanding of the limitations of psychological AI chatbots may lead to a therapeutic misconception where the user underestimates the restrictions of such technologies and overestimates their ability to provide actual therapeutic support. The authors describe this as raising major ethical concerns that can exacerbate a person’s mental health.
They are right. They are also being conservative about what is happening.
A user who does not understand they are not in a therapeutic relationship cannot give informed consent to be in one. They are also not in a position to evaluate whether the support they are receiving is appropriate for their presentation, whether they should be seeking something different, or what the limits of the system they are relying on actually are. The clinical risk of this is not theoretical.
The Design Problem Is Upstream of the Disclosure Problem
Mental health AI companies understand that their products work better when they feel more human. This is the predictable output of a product development process driven by engagement metrics. Research shows that users attribute human traits to AI systems because human-like features enhance emotional connections and relatability, fostering trust. The same research notes that this trust, once established, persists and deepens over time.
The implication is unavoidable. If you design a system to feel like a caring, responsive human presence, and that design successfully creates the feeling of being heard, understood, and supported, you have created the conditions for therapeutic misconception at scale. The more effective the design, the more thoroughly it erodes the user’s ability to accurately assess what they are actually engaging with.
The data on this is consistent. A large-scale retrospective study of 36,070 Woebot users found that participants established working alliance scores comparable to those reported in human face-to-face CBT, within just three to five days of initial use. The researchers noted that Woebot’s transparency about its non-human nature appeared to support rather than undermine bond formation. That finding does not prove therapeutic misconception. It proves something narrower and still ethically important: a transparent automated system can generate alliance-like bonds quickly. That should make consent design more urgent, not less.
The clinical question these studies raise is what does the person think they are in a relationship with, and what happens when it fails them?
The Profession Has Raised the Alarm
The people building mental health AI are not unaware of the informed consent problem. The warnings have come from the profession’s own governing body, from federal regulators, and from peer-reviewed research. At each point, the industry had the opportunity to treat informed consent as a clinical requirement. At each point, the incentives pushed against it. In January 2025, the American Psychological Association sent a formal letter to the Federal Trade Commission urging investigation into whether AI chatbot companies were engaging in deceptive practices — specifically the misrepresentation of chatbots as licensed mental health providers. The core of the complaint was an informed consent violation: users were not being told the truth about what they were engaging with. APA CEO Arthur Evans named it directly: “Allowing the unchecked proliferation of unregulated AI-enabled apps which includes misrepresentations by chatbots as not only being human but being qualified, licensed professionals, such as psychologists, seems to fit squarely within the mission of the FTC to protect against deceptive practices.”
Deceptive practices. That is the clinical and legal consequence of deploying a system without genuine informed consent. The nation’s largest psychology organisation used those words in a formal regulatory complaint.
By September 2025, the FTC had launched a comprehensive inquiry into AI chatbot companions, issuing orders to seven companies seeking detailed information on product advertising, safety practices, age-based access restrictions, and testing for negative impacts. That is a regulatory body stepping in because an industry declined to govern itself. It is also, specifically, a regulatory body investigating whether users were given accurate information about what they were using. That is an informed consent investigation wearing enforcement clothing.
In October 2025, a study led by researchers at Brown University found that AI chatbots systematically violated core mental health ethics standards, even when explicitly prompted to follow evidence-based psychotherapy techniques. The violations included inappropriately navigating crisis situations, reinforcing users’ negative beliefs about themselves, and what the researchers called deceptive empathy (using language such as “I see you” and “I understand” to manufacture a false sense of connection). They identified 15 distinct ethical risk categories across five domains. Every model tested exhibited multiple violations.
Deceptive empathy is not a technical failure. It is the operational expression of a system that was never designed to be honest with its users about what it is. It is therapeutic misconception, engineered into the product and delivered at scale.
The study’s lead researcher named the accountability gap precisely: “For human therapists, there are governing boards and mechanisms for providers to be held professionally liable for mistreatment and malpractice. But when LLM counselors make these violations, there are no established regulatory frameworks.”
There is no framework because the industry has not built one and regulators have not yet forced it to. Informed consent would be the most basic component of any such framework. It is also the component that, if taken seriously, would immediately constrain the anthropomorphised design that drives engagement. That tension is part of why this conversation has been slow to happen.
The broader research literature supports this concern. A 2025 JMIR Mental Health scoping review identified the major ethical themes in conversational AI for mental health as safety and harm, responsibility and accountability, empathy and humanness, anthropomorphisation and deception, autonomy, effectiveness, privacy, and transparency. Informed consent sits at the intersection of nearly every one of those categories. It is not a peripheral concern. It is the ethical load-bearing structure the field has not yet built.
The Silence Problem
Most consent failures are invisible. There is no dramatic moment. The person simply uses the system, trusts what it tells them, skips the appointment they should have made, and builds a reliance on something that was never designed to hold that weight. No incident. No flag. Just a clinical need that goes unmet because the system felt like enough.
A 2025 study on AI chatbot use for mental health among youth found that 63.3% of users had not told anyone they were using AI chatbots for mental health advice. Not their clinician, not their parents, not their friends. They were using a system whose nature they may not have fully understood, in isolation, for something that carries clinical risk, and no one in their care ecosystem knew it was happening.
This is the consent failure made visible. A properly informed user would understand what the system is, what its limits are, and whether their clinician should know they are using it. A user operating under therapeutic misconception treats the AI interaction as private, sufficient, and reliable. The distinction matters clinically. It also matters ethically.
Psychology Today reported in early 2026 that the APA has called AI chatbot companies’ practices “deceptive” for passing themselves off as mental health providers, and has called upon the FTC to investigate them. Deception is not an accidental side effect of poor design. It is the outcome of a design process that prioritised user affect over user understanding.
But This Is Not Therapy. So Why Does Clinical Ethics Apply?
There is a serious counter-argument that needs naming here. These products are not therapy. They are not marketed as therapy. They carry disclaimers. They are consumer wellness tools, no different in legal standing from a self-help book or a meditation app, and applying clinical informed consent frameworks to a consumer product category is a category error. The obligation that governs a licensed practitioner entering a therapeutic relationship with a vulnerable client does not automatically transfer to a software product that helps someone track their mood or practise breathing exercises.
There is also a version of this argument that goes further. Informed consent as a clinical construct was designed for a specific context: an identified patient, an identified provider, a defined treatment, a power differential that the consent process is designed to counterbalance. Mental health AI disrupts each of those elements. The user is not necessarily a patient. There is no provider. There is no treatment in the regulatory sense. Importing a consent framework built for that context into a fundamentally different one may produce the look of compliance rather than genuine protection.
These arguments reflect a real ambiguity in how the field has categorised these products, and a genuine concern that overcorrection could burden useful, accessible tools with clinical overhead they were never designed to carry. Some of the most thoughtful people working in digital mental health hold versions of this position.
The problem is that the argument collapses under the weight of what the products actually do.
A product that uses therapeutic language, employs evidence-based clinical techniques, cultivates emotional bonds, responds to disclosures of trauma, suicidal ideation, and acute distress, and positions itself through copy, design, and affect as a supportive presence for people in psychological difficulty is not a wellness app in any meaningful sense. It is performing clinical functions without clinical accountability. The category label on the App Store does not determine the clinical reality of the interaction.
The consent framework that applies is not determined by what the company chooses to call its product. It is determined by what the product actually does to the person using it. When a system reliably produces the emotional and relational conditions of a therapeutic relationship (e.g. trust, disclosure, dependency, bond formation), the ethical obligations that attach to those conditions follow. You do not escape them by declining to use the word therapy.
The self-help book comparison also fails on inspection. A book does not respond. It does not adapt to the reader’s emotional state in real time. It does not remember what the reader disclosed last Tuesday and reference it today. It does not use language calibrated to deepen trust and encourage continued engagement. The interaction architecture of conversational AI is categorically different from passive content, and the ethical obligations it generates are different as a consequence.
The access argument deserves the most careful treatment because it is the most genuinely important. Tools that extend mental health support to people who would otherwise receive nothing have real value, and requiring clinical-grade consent processes risks burdening that access with overhead that reduces reach. That concern is legitimate. But it cannot be allowed to depend on opacity. If a product’s accessibility requires users to misunderstand its nature, limits, or risks, the access model is clinically unstable. Real access must include comprehension, not merely availability. Informed consent is not incompatible with reach. It is incompatible with a particular design pattern, one that prioritises engagement over transparency.
The Consent Standard Mental Health AI Should Meet
The field has spent considerable energy on what AI in mental health can achieve. It has spent almost none on how to verify that users understand what they have agreed to.
A credible consent architecture for mental health AI would need to meet five minimum tests. Users should be able to explain, in their own words, what the system is and is not. They should understand whether they are receiving therapy, coaching, emotional support, psychoeducation, triage, or self-management guidance, and what the difference between those things means for their care. They should know what the system cannot do, particularly around diagnosis, crisis response, duty of care, emergency intervention, continuity, confidentiality, and clinical judgement. They should understand what data are stored, remembered, shared, reviewed, or used to improve the system. And the company should be able to demonstrate that users actually understand these things, not merely that a disclosure was displayed.
That last requirement is the one the industry has avoided most consistently. Displaying a disclosure is a legal event. Verifying comprehension is a clinical one. The gap between them is where therapeutic misconception lives.
What Reframing This Actually Requires
The field keeps treating informed consent as a disclosure problem because disclosure is tractable. You add a banner. You revise the onboarding copy. You include a disclaimer. You move on.
Disclosure is not the same as informed consent. Informed consent requires that the user actually understands what they are consenting to. Understanding requires that the design of the system does not work against comprehension. And comprehension, for a person in psychological distress engaging with something that feels warm, responsive, and caring, is not a passive state. It has to be actively supported.
That means consent processes that are clinically informed, not legally optimised. It means onboarding that accurately conveys the nature of the system before the parasocial bond forms, not after. It means product design that does not systematically engineer the emotional conditions that make misconception inevitable. It means clinical review of what users actually believe about the system they are using, not just what the terms of service say they were told.
A 2025 paper in a peer-reviewed journal on bioethics and AI makes the point precisely: AI’s role as a third party in the therapeutic relationship necessitates a serious examination of the new risks it introduces. As the therapeutic relationship evolves from a dyadic to a triadic model (clinician, patient, and AI) there is a need to reassess informed consent practices. The authors argue that explicability, the requirement that AI processes be comprehensible and transparent, is not a separate ethical principle but is intrinsically linked to autonomy and its expression as informed consent.
Explicability and consent are not product features. They are ethical requirements. In mental health, they are clinical requirements. The field has been treating them as optional enhancements to a product category it has already decided to build. That inversion has to stop.
The Demand
Every clinician, founder, engineer, investor, and governance professional building or deploying mental health AI needs to answer one question plainly: does the person using this system understand what it is?
Not what the terms of service say. Not what the onboarding tooltip states. What do they actually understand, in the moment they are using it, about the nature of the system, its limits, and what it cannot do for them?
If the answer is “we are not certain,” then what is being operated is not a mental health product with a disclosure gap. It is a clinical ethics problem with a user interface.
The standard that applies here is not novel. Informed consent exists because allowing someone in distress to engage with a system they do not understand, on the basis of trust they were engineered to feel, has always been recognised as wrong. Making that system algorithmic does not change the ethics. It scales them.
The field is building systems that can generate trust faster than it has built the ethical machinery to deserve it. That is the problem. Solving it starts with being honest about what these systems are, in the moment users need to know. Before the bond forms, before the disclosure is made, before the person decides this is enough.
Scott Wallace, PhD, has been building mental health software for more than 35 years, longer than smartphones have existed, longer than app stores, longer than most of the teams now entering this space have been thinking about it. Trained in clinical psychology and neuropsychology, he received formal training in C, C++, and JavaScript to engineer some of North America’s earliest digital mental health platforms at a time when the web had fewer than 30,000 sites worldwide. He later completed early iOS certification to build mobile health applications as that platform emerged, and went on to lead the engineering development of NLP and NLG-based conversational systems before large language models entered the conversation. That technical depth is what allowed him to work inside the architecture rather than alongside it, translating clinical requirements into system design and catching the places where engineering assumptions quietly displaced clinical ones.
Scott led the digital division of a major EAP provider through a successful exit, served as clinical lead for AI-based mental health technologies, and has produced hundreds of psychoeducational programmes used internationally. He now advises founders, clinicians, and investors building AI-enabled mental health systems.
His writing examines the future of mental healthcare, AI safety and governance, clinical risk, and what it actually takes to build mental health AI that holds up under real-world conditions.