WHO’s AI Ethics and Governance Guidance for Large Multi-Modal Models operating in the Health Sector – Data Protection Considerations

When the World Health Organization published Ethics and governance of artificial intelligence for health: Guidance on large multi-modal models in January 2024, it sent a clear signal to the entire healthcare AI ecosystem: the status quo is not good enough. The nearly 100-page document, with over 20 pages of footnotes, examines in detail the risks that large multi-modal models (LMMs) pose when developed, provided, or deployed for healthcare purposes. More importantly, it sets out actionable guidance for developers, providers, deployers, and governments on how to address them.

The scope of the WHO's concerns is wide, ranging from algorithmic bias and clinical reliability to environmental impact. But data protection stands out as a thread running through virtually every section of the document. This article provides an overview of the risks the WHO surfaces, then zeroes in on the governance proposals specific to data protection, giving them the depth and context they deserve.

What Risks Do LMMs Pose in Healthcare?

The WHO's Executive Summary organizes the risks associated with LMMs in healthcare into two broad categories: those arising from the technology itself, and those arising from the conditions under which it is deployed. The technical risks include hallucinations, bias in training data, poor performance on underrepresented populations, and a lack of transparency around how models generate outputs. The deployment risks include inappropriate use cases, inadequate user training, unclear liability structures, and the absence of meaningful regulatory oversight in many jurisdictions.

Beyond these, the WHO raises a concern that is easy to overlook: the environmental cost. The compute power required to train and run large multi-modal models carries a significant water and carbon footprint, a burden that falls disproportionately on middle- and low-income communities already facing compounding health challenges.

Privacy, however, is not just one risk among many. The Guidance makes clear that privacy and bias are the two risks that must be addressed across all three phases of the AI lifecycle: development, provision, and deployment. It also features in the first of the six ethical principles the WHO established through its consensus-building process.

What Are the Specific Privacy Risks of LMMs in Health Settings?

The WHO's concern about privacy is rooted in the realities of how LMMs have been built and used to date. Patient-guided use, one of the key use cases examined in the Guidance, surfaces a troubling list of potential privacy risks. The document also identifies the erosion of patient trust as a systemic threat: if people come to believe that accessing healthcare digitally means sacrificing their privacy, the consequences for public health outcomes are serious.

The Guidance cites a study published in Nature noting that moving from a large language model designed to answer medical questions to one that can be safely used by healthcare providers, administrators, and consumers will require considerable additional research to ensure the safety, reliability, efficacy, and privacy of the technology. In other words, the WHO is not simply calling for better guardrails. It is questioning whether current LMMs are suitable for healthcare use at all.

The reasoning is straightforward. First, LMMs have repeatedly disclosed personal, health, and other confidential information in ways that suggest there are insufficient protections in place. Once data is submitted to an LMM developer, it typically cannot be retrieved, and future model iterations may be trained on that data without a clear legal basis permitting it. There is also the risk of sensitive information being exposed to other users of the same LMM, not just the company behind it.

Second, there is an unresolved question around legal basis. Whether individual consent is required for training data, and how that consent can be meaningfully obtained at scale, remains unsettled. The WHO suggests that in the absence of a clear answer, developers may need to work with smaller datasets. But smaller datasets carry their own risk: a higher probability of re-identification for the individuals whose data is included.

The WHO goes further, enumerating a specific list of ways in which LMMs may never be fully compliant with the General Data Protection Regulation or comparable data protection laws. These include scraping and using personal data without consent or legitimate interest, failing to implement age-gating for minors, being unable to inform individuals of their right to access, correct, or erase their data, lacking transparency about how sensitive data submitted through a chatbot interface is used, retaining data longer than permitted under the data minimization principle, failing to prevent breaches of personal information, publishing inaccurate personal data due to hallucinations, and potentially violating the right to explanation when personal information is used in automated decision-making.

The real-world consequence of these failures is not hypothetical. The WHO notes that at least one major LMM developer indicated it may not be possible to offer its product in Europe at all. The WHO views this as deeply troubling: if the choice becomes one between privacy rights and access to AI-powered healthcare, that is not a trade-off that should be made by default.

How Does the WHO Recommend Addressing Data Protection at Each Phase?

The WHO's recommendations are organized around the three phases of the AI lifecycle, and they are specific about who bears responsibility at each stage.

What Should LMM Developers Do to Protect Health Data?

The development phase is where the WHO places the most responsibility and, by extension, the highest expectations. Developers control what data goes into a model, how it is obtained, and how it is processed. Missing the opportunity to build privacy protections in at this stage creates problems that are difficult or impossible to fix later.

The WHO acknowledges that many LMM developers may lack specialized expertise in health information protection, though it expects this to improve. What it flags more critically is the undervaluation of proper data preparation work, driven by weak incentives to invest in it. Its recommendation is clear: developers should be held legally accountable for design flaws introduced at this stage, in part because this is where the resources exist to meaningfully compensate for downstream harm.

Specific recommendations for developers include conducting Data Protection Impact Assessments (DPIAs) to surface risks before deployment, avoiding third-party data sources where possible and vetting them carefully when not, following privacy-preserving best practices and applicable laws during data collection, and maintaining transparency about what data is collected and how it is used. The WHO notes, with concern, that recent LMM releases have actually become less transparent on this last point.

For governments, the WHO recommends stronger enforcement mechanisms than voluntary codes of conduct. It calls for strict enforcement of existing privacy laws alongside their active revision, noting that most were written before the emergence of generative AI.

What Do LMM Providers Need to Consider?

A provider in this context is an organization that integrates an LMM into a specific healthcare application, such as a clinical decision support tool, a patient-facing chatbot, or a documentation assistant. The WHO pushes back against the emerging view, debated during the EU AI Act negotiations, that regulatory burden should fall primarily on providers and deployers rather than developers.

This framing, the WHO argues, is misguided. Much of the risk associated with privacy, and many other concerns, can only be effectively addressed in the development phase. Placing the compliance burden downstream does not make users safer. It just makes developers less accountable.

The WHO recommends that developers and providers collaborate on privacy preservation rather than treating compliance as the provider's problem to solve alone. It also recommends shared liability for harm, with the burden of proof of compliance resting with the developer and the provider, not the end user. If a developer determines that an LMM should not handle health-related personal information, it has options: restrict use through a licensing regime, block healthcare-related queries at the model level, or issue clear warnings when outputs contain medical information.

The WHO is unambiguous about the current state of readiness. Its observation that current chatbots incorporating LMMs are unlikely to meet the explainability, bias control, and transparency standards expected by forthcoming EU and US medical device regulations applies equally to privacy. The strict requirements of health-specific frameworks like HIPAA are, in most cases, not met by LMMs as they exist today.

What Are the Deployment-Phase Responsibilities?

The deployment phase is where the unpredictability of LMMs comes into sharpest focus. Models can be used in ways that were not anticipated at the time of development, and their outputs can shift over time. Some of these risks can only be mitigated upstream, but deployers still carry meaningful responsibilities.

The WHO recommends that impact assessments be conducted for every LMM deployment, even those initially categorized as low risk. For large-scale deployments, it calls for post-release auditing by independent third parties, with results published and disaggregated by user characteristics including age, race, and disability status. Healthcare providers using LMMs must be trained not only on clinical use but on the privacy risks involved, including the risks associated with entering protected health information into chatbot interfaces that may not handle that data securely.

Why Data Protection Cannot Be an Afterthought in Healthcare AI

The picture the WHO paints is deliberately frank. Current LMMs, taken as a whole, are likely not suitable for healthcare use in their present form. The reasons are not primarily about clinical accuracy or bias, though those are serious concerns. They are about the foundational inability of most LMMs to meet the data protection requirements that health contexts demand: meaningful consent, data minimization, retention limits, disclosure controls, and the right to erasure.

This does not mean that AI has no place in healthcare. The WHO's Guidance is not a rejection of the technology. It is a serious call for the healthcare AI ecosystem to build the right foundations now, before deployment outpaces safeguards. Healthcare organizations, pharma and life sciences companies, and technology providers that get this right will not only reduce regulatory and reputational risk. They will be better positioned to earn and keep patient trust, which is the single most important asset in any healthcare relationship.

For organizations that are already working with health data in AI pipelines, the implications are immediate. Processing unstructured clinical data, patient records, or research datasets through LMMs without first de-identifying them is not just a compliance risk. It is the kind of practice the WHO is explicitly warning against. The most responsible path forward is to ensure that protected health information and other sensitive data is identified and removed before it ever enters a model.

If your organization is deploying or evaluating AI tools in a healthcare context, Limina's data de-identification platform is purpose-built for exactly this challenge. Using advanced machine learning developed by linguists, Limina accurately identifies and removes PHI, PII, and other sensitive identifiers from unstructured text, documents, and more, at scale, and with the context-awareness that healthcare data requires.

Talk to the Limina team to see how automated de-identification can help your organization meet the data protection standards that the WHO's Guidance calls for.