How to Use LLMs in Healthcare Without Violating HIPAA

When a healthcare organization sends a patient record to a large language model (LLM) API, that data leaves the organization’s environment and reaches a third-party server. Without a signed Business Associate Agreement (BAA) and a HIPAA-appropriate configuration, that transmission is a potential violation, and it is happening right now in health systems that have not yet built a policy around it. The BAA-tier problem behind this is the same one covered in HIPAA compliance for AI systems generally; LLMs just make it easier to trigger by accident, one pasted paragraph at a time.

The instinct to use LLMs in clinical and administrative workflows is not the problem. Summarizing a chart, drafting discharge instructions or triaging a message queue are all legitimate uses of the technology. The problem is that the default way most people reach for an LLM, opening a browser tab and pasting text in, bypasses every safeguard HIPAA requires, because that default path usually has no BAA behind it at all.

This guide lays out why LLMs and HIPAA create friction by default, three architectures that resolve that friction, what to check in a healthcare LLM vendor’s agreement and how de-identifying data before it reaches any model sidesteps the compliance question almost entirely.

Why LLMs and HIPAA create friction

Most consumer and even many developer-facing LLM products are built around a default architecture: your input is sent to the vendor’s cloud, processed and, unless you have specifically contracted otherwise, may be retained or used to improve future models. That default is incompatible with HIPAA the moment PHI enters the prompt, for two reasons. First, transmission to an uncontracted third party has no legal basis under the Privacy Rule. Second, retention and training use conflict with the minimum necessary standard, since neither is typically required to answer the clinical question being asked.

None of this is unique to any one vendor. It is simply how consumer AI products are architected by default, optimized for helpfulness and iteration speed rather than for regulated data. Healthcare organizations that want to use LLMs safely need to actively choose an architecture that moves away from that default, rather than assuming the tool will handle it for them. A physician who pastes a de-identified version of a discharge summary into a free chatbot to draft patient-friendly instructions has, in effect, made an architecture decision without realizing it, simply by choosing the tool that happened to be open in another tab.

Three compliant architectures for healthcare LLM use

1. BAA-covered cloud LLM. The most direct path is to contract with a vendor whose enterprise tier offers a BAA, and use only that tier. OpenAI signs BAAs for ChatGPT Enterprise, ChatGPT Edu with a sales-managed account and its healthcare-specific products, and reviews API BAA requests case by case, explicitly excluding the free, Plus, Pro and standard Business tiers. Microsoft’s HIPAA BAA is built into the Microsoft Online Services Data Protection Addendum and covers Azure OpenAI Service automatically under a qualifying licensing agreement. Google’s BAA coverage for Gemini splits by product: the Google Workspace BAA can extend to Gemini in Workspace on covered editions, and a separate Google Cloud BAA covers listed Cloud AI products; the free Gemini consumer app and Google AI Studio are not covered by either. This path works, but it requires strict tier discipline. An organization with a BAA for ChatGPT Enterprise is not covered the moment an employee uses ChatGPT Plus for the same task.

2. On-premise or in-VPC LLM. Running an open-weight or licensed model inside your own infrastructure removes the third-party transmission question entirely, since there is no external vendor receiving PHI and so no BAA to negotiate for that data flow. This requires more infrastructure investment and MLOps capability than a cloud subscription, and it shifts the compliance burden toward securing and auditing your own environment. It also means absorbing the work a cloud vendor would otherwise handle: patching the model server, managing GPU capacity and keeping the model itself current as newer, better versions are released. A health system with an existing MLOps team already running other in-house models can add this as one more deployment. A team without that infrastructure is usually better served by one of the other two architectures until it builds that capability.

3. De-identified data plus any LLM. If PHI is stripped of its 18 HIPAA identifiers before it reaches the model, the data is no longer PHI, and the strict BAA requirement for that specific data flow no longer applies in the same way, because you are no longer sending protected data to a third party. The same principle that makes this work for one-time queries also applies to de-identifying data before it enters an AI training pipeline: once identifiers are removed, the data can be reused across models without re-triggering compliance review. This is the architecture that scales best: it works with any LLM, cloud or on-premise, current or future, without renegotiating a BAA every time a team wants to try a new model.

Architecture	Setup effort	Vendor flexibility	Where the compliance burden sits
BAA-covered cloud LLM	Low to moderate	Locked to the contracted tier	Verifying tier usage stays inside the BAA at all times
On-premise or in-VPC LLM	High	Full, since any model can be self-hosted	Securing and auditing your own environment
De-identify first, then any LLM	Low, with a de-identification layer	Full, since de-identified data is portable across vendors	Accuracy and completeness of the de-identification step itself

A midsize health system evaluating three LLM vendors for a single clinical summarization use case would, under the first architecture, need three separate BAAs, three separate tier audits and three separate monitoring processes. Under the third architecture, the vendor evaluation becomes almost incidental, because the model never sees PHI in the first place.

The de-identification approach

Removing PHI before an LLM ever sees it resolves the compliance question at its source rather than managing it downstream. This is the approach with the most operational flexibility: because de-identified data carries no HIPAA identifiers, it can move between LLM vendors, get re-processed by a new model version or be used in a RAG pipeline without triggering a fresh BAA negotiation each time. The trade-off is that de-identification has to be accurate and consistent. A system that misses even one identifier, or introduces gaps that reduce clinical usability, undermines the whole approach. Coreference resolution matters here specifically: a de-identification system needs to catch a patient referred to by name in one sentence and “she” or “the patient” three sentences later, or the redaction is incomplete.

Limina de-identifies PHI before it reaches any LLM: your data never leaves your environment.

What to look for in a healthcare LLM vendor

Four things separate a healthcare-appropriate LLM vendor from one that simply says “HIPAA compliant” in its marketing:

• A signed BAA, specific to the product tier you are actually using, not just the vendor’s brand.
• A clear data retention policy stating how long inputs are stored and under what conditions they are deleted.
• A default model training opt-out, so that PHI or de-identified clinical data is never used to improve the vendor’s general-purpose models without explicit, separate authorization.
• An in-VPC or private deployment option, even if you do not use it immediately, since it gives you a path to reduce third-party exposure later without switching vendors.
• A documented incident response commitment, spelling out how quickly the vendor will notify you if it discovers unauthorized access to your data, since HIPAA’s own breach notification clock starts running once a covered entity knows or should have known about an incident.

Organizations that apply this checklist once, at the vendor-selection stage, tend to fare better than teams negotiating it separately every time someone wants to try a new model. Limina’s data de-identification platform is built to sit in front of that checklist, handling the retention and training-opt-out questions once rather than per vendor.

How Limina enables compliant LLM use in healthcare

Limina AI sits in front of any LLM your organization wants to use, de-identifying PHI before the model sees it and re-identifying the output afterward for the clinicians who need the full record. Because the platform deploys in-VPC, patient data never has to leave your environment to be de-identified, and because the output going to the LLM carries no HIPAA identifiers, your organization is not dependent on any single vendor’s BAA terms to use the model safely. That combination, de-identify before the model and re-identify after, lets teams building on ANALYZE evaluate new LLMs as they emerge without re-running a compliance review from scratch each time. It is the same underlying approach that supports confidential handling of any sensitive text sent into a general-purpose chatbot, extended to the specific identifier set HIPAA requires.

See how the de-identify-then-LLM architecture lets your team evaluate new models without a new compliance review each time.

Vendor BAA terms and product tiers referenced above are current as of September 2026 but change on a rolling basis. Confirm current terms directly with each vendor before making a compliance decision.