LLM Training on Healthcare Data: Compliance and De-identification Requirements

Health systems are fine-tuning large language models on clinical notes, discharge summaries, and physician conversations, work that is advancing quickly and generating measurable clinical value. The same documents that make this training signal so valuable, dense, nuanced descriptions of real patient encounters written in a clinician's own words, are also dense with protected health information (PHI). The compliance challenge that creates is significant. It is also solvable, provided the de-identification step is built correctly and early.

LLM training on healthcare data requires that any clinical text used to train or fine-tune a model either be de-identified under HIPAA's Safe Harbor or Expert Determination standard before it enters the training pipeline, or be handled throughout training and deployment under a fully executed Business Associate Agreement that keeps the full weight of HIPAA's Security Rule in force. For most healthcare AI teams, de-identification at the point of ingestion is the more practical and more auditable path.

This article covers why healthcare data is uniquely valuable and uniquely risky as LLM training signal, the specific HIPAA obligations that apply, and a step-by-step approach to de-identifying clinical data before it reaches a model.

Why healthcare data is the best training signal, and the highest-risk

Clinical notes, discharge summaries, and physician-patient conversations capture a kind of reasoning that structured EHR fields do not: differential diagnosis, the weighing of ambiguous symptoms, the clinical judgment behind a treatment decision. That density is exactly what makes free-text clinical data so valuable for training language models intended to support clinical reasoning, summarization, or documentation tasks.

It is also exactly what makes the data high-risk. Unlike a structured database field, where a patient's name sits in one discrete, easily redacted column, a clinical note might reference a patient by name, then again by abbreviation the physician used, then by a pronoun, then by their relationship to a family member also mentioned in the note, all within a few sentences. Identifiers in clinical narrative are embedded in natural language, not isolated in predictable fields, which is precisely why general-purpose, non-healthcare-tuned de-identification tools tend to underperform significantly on this kind of text.

The HIPAA problem with LLM training

LLM training introduces risks that are somewhat distinct from earlier generations of healthcare AI, like structured predictive models, because of how large language models learn and, in some cases, retain specific examples from their training data.

Peer-reviewed research has documented that language models can memorize specific training examples and reproduce them under certain prompting conditions, with the degree of memorization generally scaling with model size, including content that appeared only a small number of times in training. For a healthcare LLM trained on clinical notes, this creates a specific and serious risk: if PHI was present in the training data, fragments of that PHI could, in principle, surface in the model's outputs later, exposing a patient's information to a completely different user than the one whose encounter generated the original note.

This risk is the central argument for de-identifying clinical data before training rather than relying on output-side filtering or model behavior controls after the fact. It is far more reliable to ensure PHI was never present in the training data than to try to catch every way a model might later reproduce it.

Safe harbor vs. expert determination for LLM training data

HIPAA offers two de-identification pathways, and the choice between them matters more for LLM training specifically than it does for many other healthcare data uses. The Expert Determination standard, in particular, has specific qualification and documentation requirements that organizations should understand before committing to that pathway.

	Safe Harbor	Expert Determination
Method	Remove all 18 specified identifiers (names, dates other than year, geographic detail below state level, medical record numbers, and others).	A qualified expert applies statistical methods to certify that re-identification risk is very small, while potentially retaining some identifiers like month-level dates or sub-state geography.
Data utility for LLM training	Lower; removing all dates and fine-grained geography can strip context an LLM might otherwise use to learn temporal or regional clinical patterns.	Higher; retaining more granular detail (with documented risk certification) can preserve more of the contextual richness that makes clinical narrative valuable training signal.
Best for	Most general-purpose de-identification needs where speed and simplicity matter more than retaining fine-grained detail.	LLM training scenarios where preserving temporal sequencing or regional context meaningfully improves the model, and the organization can support the additional expert certification process.

Expert Determination's ability to retain more granular detail than Safe Harbor while preserving lower re-identification risk is an established feature of the two HIPAA pathways. However, the specific claim that this additional detail meaningfully improves LLM training outcomes has not been confirmed against current Limina research or third-party benchmarks and should not be presented as an established finding, since this is a developing area of applied research.

Step-by-step: de-identifying clinical data for LLM training

1. Inventory clinical text sources. Catalog every source of free-text clinical data feeding the training pipeline: clinical notes, discharge summaries, radiology reports, nursing notes, and physician dictation, since each format embeds identifiers differently.
2. Choose the de-identification standard. Decide between Safe Harbor and Expert Determination based on whether the training task benefits meaningfully from retained dates or geographic detail, and whether the organization can support Expert Determination's certification process.
3. Run healthcare-tuned de-identification. Process the data through a de-identification system built for unstructured clinical narrative specifically, not a general-purpose tool, since identifiers in free text require natural language understanding rather than simple pattern matching to detect reliably.
4. Validate detection accuracy on a representative sample. Manually review a statistically meaningful sample of de-identified output against the original text to measure actual detection accuracy on your specific document types before trusting the full dataset.
5. Document the methodology. Record which standard was used, what was retained or removed, and, for Expert Determination, the expert's written certification, creating the audit trail for HIPAA, FDA, or IRB review.
6. Train on the validated, de-identified dataset. Proceed to training only once the de-identified dataset has been validated, not in parallel with validation, so that any detected gaps can be corrected before they reach the model.
7. Test the trained model for output-level leakage. Validate that the trained model does not reproduce identifiable fragments, in addition to standard clinical accuracy testing, before moving to production deployment.

Data types and their de-identification requirements

Different clinical document types carry different identifier patterns and de-identification challenges.

Data type	Common identifier challenges	What good de-identification needs to handle
Clinical notes	Patient referenced by name, abbreviations, and pronoun within the same note; family members and their relationships mentioned by name.	Coreference resolution across multiple references to the same individual, not just first-mention detection.
Discharge summaries	Dense narrative combining diagnosis, treatment history, and follow-up instructions, often referencing dates, facilities, and providers by name.	Consistent identifier replacement across a long document so the narrative remains coherent after de-identification.
Radiology reports	Structured findings sections mixed with free-text impressions that may reference referring physicians or prior imaging facilities by name.	Handling mixed structured and unstructured content within a single document type.
Physician-patient transcripts	Conversational, informal language; patients volunteering family details, employer names, or other identifying context unprompted.	Detecting identifiers in colloquial speech patterns, not just formal clinical terminology.

Validation requirements: proving your training data is HIPAA-compliant

For most healthcare AI projects, validation documentation needs to satisfy three audiences: internal compliance and legal teams, external auditors or regulators, and, for AI intended as a medical device, the FDA or an Institutional Review Board (IRB).

Expert determination-ready outputs are particularly important here. FDA guidance on AI-enabled device software functions calls for statistical validation of the data and methods underlying a model, and a documented, expert-certified de-identification process gives reviewers a defensible basis for evaluating re-identification risk rather than an informal assurance. The same documentation supports IRB review for research protocols, where the board needs to understand exactly what identifying information was retained or removed before approving a study.

How Limina supports healthcare LLM development

Limina de-identifies clinical notes, discharge summaries, radiology reports, and physician transcripts at the point of ingestion, before that data reaches a training pipeline, deploying in-VPC or on-premises so PHI never has to leave a health system's own infrastructure. Limina achieves 99.5 percent or higher accuracy on real healthcare data, a benchmark set specifically on physician conversations and clinical narrative, the same data type discussed throughout this article, compared to 60 to 70 percent accuracy from general-purpose cloud de-identification tools.

Expert determination-ready outputs are particularly important for healthcare AI teams. FDA submissions for AI as a medical device require statistical validation of the underlying data and methods, and HIPAA expert determination is one of the two pathways that satisfies that requirement.

For healthcare AI teams already building on NVIDIA NeMo Guardrails, Limina is available as an integrated PII detection and masking plugin, sanitizing prompts and responses in real time as they pass through the guardrails layer.

Building your LLM training pipeline

Healthcare data is the most valuable training signal a health system has, and the most regulated. The organizations getting LLM training right are not avoiding clinical narrative data, they are building a de-identification step accurate enough to use it safely, validated thoroughly enough to satisfy HIPAA, FDA, and IRB review, and positioned early enough in the pipeline that the resulting model never had a chance to learn from identifiable PHI in the first place.

Limina helps health systems and healthcare AI teams de-identify clinical notes, discharge summaries, and physician conversations in-VPC, producing expert determination-ready outputs built for LLM training specifically.

Talk to an expert

Explore the Data De-identification platform to see how it fits into a healthcare LLM training pipeline.