AI models and privacy: Choices for GDPR compliance

Integrating large language models (LLMs) into business processes promises considerable efficiency gains, but at the same time confronts organizations with complex legal and technical challenges. As soon as an AI model is fed prompts containing personal data — such as customer names, personnel files, medical data or financial history — the General Data Protection Regulation (GDPR) applies in full. Many organizations wrongly assume that a simple non-disclosure agreement or a standard API contract is enough to comply with European privacy law.

In practice, GDPR compliance requires a thorough assessment of the entire data processing chain: from the physical location of the compute infrastructure to the policy on model training, retention periods and log files. For a structured inventory of all GDPR obligations within your organization, you can work through the practical GDPR privacy checklist for Dutch organizations before you select models. In this article we analyze the most important privacy aspects when choosing AI models, compare closed cloud models with locally hosted open-weight alternatives, and offer a concrete assessment framework for privacy officers and software architects.

1. The GDPR pillars in the use of Large Language Models

When an organization processes personal data through an AI model, that processing must comply fully with the core principles of Article 5 of the GDPR. The main legal bottleneck with commercial LLMs is the lawfulness and the purpose of the processing. Personal data passed to a model through a prompt may only be used for the specific purpose for which it was collected. Commercial AI providers that reuse input data to train or fine-tune their future models act in breach of the principle of purpose limitation, unless the data subjects have given explicit and informed consent for this.

A second crucial principle is data minimization. LLMs are inclined by nature to swallow rich context. Sending along complete documents or unredacted email chains often means more personal data is sent to the model than is strictly necessary to achieve the desired result. Organizations therefore have to take technical measures to filter prompts in advance.

The transparency principle also plays a major role. Data subjects have the right to know that their personal data is being processed by an automated system. When AI models take decisions or generate advice that affects individuals, as with CV screening or credit assessment, the strict rules of Article 22 of the GDPR on automated individual decision-making apply as well.

2. Open source versus closed source: privacy architecture compared

The fundamental choice when building AI applications lies between closed API-based models (such as OpenAI GPT-4o, Anthropic Claude or Google Gemini) and open-weight models (such as Llama 3, Mistral or Qwen) that run on your own or on dedicated infrastructure. Both approaches have a distinct privacy profile and require a different management approach.

Closed cloud models do offer advanced performance, but they introduce a dependency on external processors. Data leaves the perimeter of your organization and is processed on the servers of a commercial party. Although large providers offer enterprise contracts that guarantee prompts are not used for model training, the data is still sent across the internet. If you want to know how license terms relate legally to intellectual property and commercial use, consult the guide on commercial usage rights of open models for a detailed overview of license types.

With open-weight models the organization retains full control over the data flow. The model runs inside your own virtual private cloud (VPC) or on physical servers on site. No data is transported to external parties, which reduces the risk of unauthorized access or data breaches at third parties to zero. The downside is that the organization is itself responsible for the physical and logical security of the model servers, including access management and encryption.

3. Data Processing Agreements (DPA) and Zero Data Retention (ZDR)

For organizations that use external AI providers, concluding a GDPR-compliant data processing agreement (DPA) is a legal requirement. A standard consumer account or the general terms and conditions of an AI service are never sufficient for business processing of personal data. The DPA has to state explicitly that the provider acts as a processor and processes the data solely on the written instructions of the controller.

A crucial part of the DPA with AI services is the retention policy. By default, many API providers keep prompts and generated answers for 30 days for abuse detection and moderation purposes. To assess whether a supplier's documentation offers enough transparency about the training data, you can consult the guidelines for reading model cards and model licenses for production environments.

For sensitive applications, organizations have to stipulate that a Zero Data Retention (ZDR) policy applies. Under ZDR, prompts and outputs are processed solely in the server's working memory (RAM) and are erased immediately after the response tokens have been generated. Nothing is stored on disk, neither for moderation nor for caching. Without a ZDR agreement, the risk remains that personal data becomes involved in a security incident on the provider's side during the retention period.

4. International transfers and the Data Privacy Framework

Many leading AI providers are based in the United States. Sending personal data to American servers qualifies as an international transfer of personal data to a third country outside the European Economic Area (EEA). Under the GDPR such a transfer is only permitted if an adequate level of protection is in place.

Since the designation of the EU-U.S. Data Privacy Framework (DPF), EU organizations have been able to transfer personal data to certified American organizations without additional safeguards being required. Many large tech companies are certified under the DPF. Organizations do have to verify carefully whether the provider's specific AI entity is registered on the DPF list and whether the certification covers commercial customer data.

If a provider does not fall under the DPF, Standard Contractual Clauses (SCCs) have to be agreed, combined with a Transfer Impact Assessment (TIA). Such a TIA has to assess whether the legislation in the receiving country (such as the American Foreign Intelligence Surveillance Act, FISA Section 702) undermines the level of protection. For strictly regulated sectors, such as healthcare or the legal sector, the risk of American government access to cloud data can be a reason to rule out transfers to the US entirely and to opt for European cloud providers or local hosting.

5. On-device and on-premise hosting for maximum data control

When the processing of personal data falls under special categories (such as medical data, criminal data or biometric data), or when the rules of a specific sector forbid external cloud processing, deploying local AI models offers a way out. By running a quantizable open-weight model on your own hardware within your own office infrastructure or a local data center, the data stays within the legal and physical boundary of the organization.

For situations where data must not leave the local hardware under any circumstances, the extensive guide on small models on devices and on-device AI offers a practical overview of the technical possibilities. On-device models run directly on the user's workstation (for example via Apple Silicon NPUs or local GPU memory) and process data without any network connection.

The main weak point of this approach is the trade-off between model capacity and hardware requirements. Smaller models (such as 7B or 14B parameter variants) can run locally very well, but on complex reasoning tasks they do not reach the level of a distributed giant model in the cloud. The financial and operational consequences of these choices are calculated in detail in the analysis of the total cost of ownership of open versus closed AI models where privacy investments are weighed against infrastructure costs.

6. Anonymization and pseudonymization in the input pipeline

An effective way to lower the GDPR risk when deploying AI models is to apply privacy-enhancing technologies (PETs) in the application layer before a prompt is sent to the model. Placing a middleware component that automatically recognizes and replaces personal data drastically reduces the amount of privacy-sensitive data that is processed.

With pseudonymization, identifiable elements (such as names, citizen service numbers and telephone numbers) are replaced by unique tokens (for example `[CLIENT_A]`, `[BSN_1]`). The AI model performs the desired analysis or text generation on the pseudonymized text. A local post-processing step then replaces the tokens with the original values again before the result is shown to the user. The AI model and any external provider therefore never see the actual identity of the data subjects.

Important legal distinction: Pseudonymized data continues to qualify as personal data under the GDPR for as long as the key for re-identification exists. Only when data has been anonymized to the point that re-identification is no longer reasonably possible for anyone does the processing fall outside the scope of the GDPR.

Building a robust anonymization pipeline does come with technical challenges. Contextual traceability is a common problem: even if direct identifiers such as names are removed, the combination of specific job titles, locations and events in a text can still lead to the unique identification of a person. Rule-based filters (regex) often fall short here, which makes advanced Named Entity Recognition (NER) models necessary.

7. Logging, auditing and the GDPR at API level

Article 32 of the GDPR obliges organizations to take appropriate technical and organizational measures to guarantee a level of security matched to the risk. With AI applications this includes keeping log files so it can be verified which users had access to which data and which prompts were sent to the model.

A clear paradox arises here: to meet the accountability obligation an organization wants to keep detailed audit logs of all AI interactions, but when prompts contain personal data, storing those logs leads to extra storage of personal data with the associated retention risks. Retention periods for audit logs therefore have to be strictly delimited and provided with automatic cleanup routines.

At a technical level, the guide on audit logging and compliance for LLM applications explains how to store prompts and API responses encrypted in a GDPR-compliant way. It is advisable to store only the pseudonymized variant of the prompt in audit logs and to use cryptographic hashes for integrity checking.

8. Rights of data subjects: the right to erasure and LLMs

Article 17 of the GDPR gives data subjects the right to have their personal data erased (the 'right to be forgotten'). In the context of traditional databases, deleting a record is a straightforward and technically simple operation. With Large Language Models this is fundamentally different once personal data has been absorbed into the weights of the model during the training process.

Removing specific facts or personal data from an already trained neural network — also known as *machine unlearning* — is currently an unsolved scientific problem. There is no reliable method to erase a specific data point from billions of parameters without retraining the model, which costs millions of euros and weeks of compute. This underlines why organizations should only deploy models that have not been trained on unlawfully obtained personal data, and why your own customer data must never be handed to external providers as training material.

For the runtime interaction (retrieving information via RAG or vector databases), the right to erasure is easy to apply. Personal data stored in embeddings or document stores has to be erasable immediately. As soon as the source document is removed from the vector database, the model can no longer access it during contextual generation.

9. DPIA and risk assessment for AI implementations

Under Article 35 of the GDPR, a Data Protection Impact Assessment (DPIA) is mandatory when processing is likely to entail a high risk to the rights and freedoms of natural persons. The use of innovative technologies such as large-scale generative AI models almost always falls under this obligation, certainly when it involves large-scale processing or automated decision-making.

If the data processing entails a high risk to the rights of data subjects, the step-by-step plan for an AI risk analysis and DPIA execution helps to formally document the privacy impact. A full DPIA for an AI project has to contain at least the following components:

• Systematic description: A detailed map of the data flows, including the pre-processing, the API transfers, the storage locations of vectors and the log files.
• Necessity and proportionality test: Assessment of whether the chosen AI model is necessary for the intended purpose, or whether a less privacy-intrusive solution (such as a classic rule-based system) delivers the same result.
• Risk inventory: Analysis of specific AI risks, such as hallucinations that lead to wrong decisions about people, data leaks via prompt injection, and unintended traceability in model outputs.
• Mitigating measures: Concrete technical and organizational safeguards, such as ZDR agreements, encryption *at rest* and *in transit*, role-based access (RBAC) and human intervention (human-in-the-loop).

10. Comparative overview: privacy properties per deployment model

To help organizations make a well-considered architecture choice, the table below summarizes the privacy characteristics of the four main deployment models for AI:

Conclusion and recommendations

Achieving GDPR compliance in the use of AI models is not a one-off box-ticking exercise but a continuous process of risk management and technical safeguards. Organizations have to abandon the idea that cloud-based AI models can simply be deployed safely for all business information. Separating data flows on the basis of sensitivity is the most effective strategy here.

For everyday, non-sensitive tasks, enterprise cloud APIs with strict DPAs and Zero Data Retention agreements can be a perfectly good solution. As soon as confidential customer data, medical files or financial data is involved, however, the preference shifts to anonymized input pipelines or hosting open-weight models locally on your own infrastructure. By building privacy into the AI architecture from the first design phase (Privacy by Design), organizations can benefit from the power of modern LLMs without damaging the trust of their users or breaking the law.