Why PII Masking Is the Wrong Building Block

The question comes up in every project where an LLM is to be turned loose on enterprise data: How do we make sure no personal data ends up in the cloud? The obvious answer is a filter - detect PII, replace it, translate the response back. Technically that sounds like a manageable building block.

I thought this building block through to the end. The result: it doesn’t hold. Not because detection is too poor, but because the construction tries to solve a governance problem with technology.

The path there is instructive all the same, because it produces insights that hold even if you end up building something else.

The safety question cannot be answered

A detector has no provable recall on unseen data. Recall is the share of actually present cases that were also found - the metric that matters for a protective mechanism. You can measure it on an evaluation set, but production is a different distribution. Any statement of the form “nothing slips through” is therefore not weakly founded, it is unfounded.

That is not an implementation detail. Anyone selling an automated filter as a compliance measure is selling a guarantee they don’t have - and one that any data protection officer refutes with a single counterexample.

Other questions are answerable: What is the measured error rate? What happens in the failure case? How quickly does it surface? Who formally carries the residual risk? The last question is not a formality. A detector without a named risk owner means the decision effectively sits with whoever implements it.

A brief aside: NER and spans

Two terms that come up constantly in what follows.

NER stands for named entity recognition - models that mark, within a text, which words denote a person, an organization, a location, or an account number. Classically this happens through classification of each individual token: for every word, the model decides whether it belongs to a category and which one.

A span is the result: a contiguous stretch of text with a start and end position plus a type. In “Bitte kontaktieren Sie Maria Bölk” the span would be the character sequence Maria Bölk, type PERSON, position 23 to 33 (end exclusive).

This distinction is not pedantry, it is the decisive point later on: a span is a stretch of text, not a person. That two spans refer to the same real person is written nowhere - a separate step has to establish it.

Detection: two layers, two failure modes

Anyone building detection should separate cleanly:

Deterministic recognizers for everything with structure and a checksum - IBAN, email, phone numbers, tax IDs, internal identifier formats. Complete recall, testable with fixtures, versionable. This part you can rely on.

Probabilistic recognizers for everything without structure - personal names, organizations, locations, special categories under Art. 9 GDPR. Recall unknown and domain-dependent.

For the second layer, zero-shot-capable models like GLiNER are the better choice over classic encoder NER. Not because of accuracy, but for a structural reason: the label set is passed at runtime. That makes the question of what gets masked a matter of configuration rather than a training artifact. With a fixed NER model, a new entity class means retraining.

For a sense of scale: on the SPY benchmark, GLiNER2-PII - a 300-million-parameter model with 42 entity types released in May 2026 - reaches a span-level F1 of 0.477, the best value among five compared systems, including OpenAI Privacy Filter and three further GLiNER variants.¹ That is not a measurement error and not a model weakness in the narrow sense. Span-accurate detection across dozens of heterogeneous, locally shaped categories is a hard problem.

How the competition fares is equally telling: OpenAI Privacy Filter reaches comparable recall on the same benchmark, but a precision around 0.25. Three out of four flagged spans are scored as wrong. That needs reading with care: the metric is exact match, so a span off by one token counts as the same error as one invented outright. For the safety axis the two are equivalent, for the utility axis they are not - a shifted boundary destroys less context than a misclassification. It remains a foretaste of the second error axis all the same.

What it does quantify is the consequence: NER alone does not carry a gate. Anyone building this layer as the sole safeguard has the concept wrong on paper already.

Retraining is the fourth step, not the first

The reflex when detection disappoints is fine-tuning. With GLiNER that is the most expensive and usually the most unnecessary option.

The label strings pass through the same encoder as the text - the label means something. "employee identification number", "Personalnummer" and "ID" produce different behavior from the same model. The cheapest lever is therefore label wording, and it is surprisingly large. The second is a per-label threshold instead of a global one. The third: entities with fixed structure belong in the deterministic layer - retraining for an employee number with a defined format solves a problem for which a regex delivers complete recall.

Fine-tuning also reverses precisely the property GLiNER was chosen for. From that moment on there is a model artifact with a version, a training dataset and maintenance effort - and recall stays unknown, just on different data.

What is needed in any case, independent of the training question: an evaluation set. Without it, none of the cheaper steps can be assessed. That is the actual work.

The second axis: utility errors

This is where it gets interesting, and where most concepts are missing an entire dimension.

A missed span is a safety error. But there is a second class of error that hurts just as much:

“Maria Bölk” and “Frau B.” (Ms. B.) are not recognized as the same person. The downstream model sees two people where there is one.
“Hans Müller GmbH” is classified as a person. A business partner turns into a data subject.

These errors destroy context. And the two axes pull against each other: lowering the threshold improves recall and worsens utility, because more false-positive spans destroy more context.

From this follows a sentence that is almost always stated backwards in discussions: over-masking is not a safe default. It merely relocates the error - from a privacy risk to a factually wrong result. In regulated contexts that is also a compliance problem, just one that surfaces later. “When in doubt, mask more” is not conservative, it is differently risky.

What helps technically

Entity resolution as its own step. NER delivers spans, not entities. The step from spans to entities is performed by no NER model. Part of it is achievable deterministically - surname matching, initial matching, normalization of honorifics and titles cover a lot in German business texts, because variant formation there is conventionalized. What remains after that needs real semantics and stays unsolved. Putting a local generative model on it is the obvious move and, in my assessment, still the wrong one: GPU operation and additional latency in the synchronous path for a handful of edge cases. In detection itself, a non-deterministic method has no place anyway - there, an error leaks silently.

Deterministic post-correction over probabilistic output. German legal-form suffixes are a finite, maintainable list: GmbH, GmbH & Co. KG, AG, KGaA, KG, OHG, GbR, UG, gGmbH, e.K., e.V., PartG, PartGmbB, SE, eG, AöR. If a PERSON span borders a legal-form token, the combined span becomes ORGANIZATION - for compound forms across the entire suffix, not just the final token. The same pattern for leading markers. The model error is not trained away, it is caught structurally.

Surrogates instead of placeholders. [PERSON_1] has no gender, no case, no form of address. “Sehr geehrte [PERSON_1]” (Dear [PERSON_1]) is grammatically broken, and the model is working on broken input. Synthetic replacements - “Maria Bölk” → “Julia Weber”, “Frau B.” → “Frau W.” - preserve morphology and register. The price: the output looks like real data. If the back-translation fails or an intermediate state gets logged, there is no way to tell that anything was masked. A defensible compromise: surrogates for model processing, placeholders for everything that gets logged or inspected - two representations from the same entity table.

That table is therefore no longer a string-replacement list but an entity register: ID, all surface forms, type, surrogate, gender. And it has to be at least session-scoped, not request-scoped - otherwise the same person gets different surrogates across multiple turns.

The only metric that actually answers the question is neither precision nor recall, but task quality on masked versus unmasked text. Same task, both variants, compare the results. Everything else measures the filter instead of the system.

And then the whole thing falls apart

Up to this point it is a solvable, if laborious, problem. The break comes when you draw the real path:

Input → RAG → sanitization → context window → LLM → back-translation → tool call → LLM → … → user

The replacements have to stay identical across all tool calls. That is no longer a filter, it is a stateful translation layer in the hot path. And the killer is not the replacement in the prompt, it is the model returning the surrogate in altered form - inflected, shortened, embedded in a generated title or a SQL statement. The reverse mapping is then no longer a lookup. The tool call toward the target system needs cleartext anyway, so the boundary does not exist once, it is redrawn per call.

The real objection

And then the question that devalues the entire effort: where does the data actually sit?

Jira, Confluence, Salesforce, Microsoft 365. If the source systems run at US hyperscalers, the transfer has already happened. Masking then protects not the data but prevents exactly one additional processing operation by one additional processor - one often operated by the same corporate group that hosts the data anyway.

Behind this sits the actual mistake in reasoning: masking is a technical substitute for a missing legal basis.

The objection is close at hand: in 2025 the CJEU confirmed that personal reference is relative - for a recipient who cannot re-identify, pseudonymized data is not personal data.² Masking would then be exactly the tool that resolves the processor and third-country questions. On paper that holds. In the pipeline it fails in two places.

First: the relief presupposes that the recipient actually cannot re-identify. That is not a declaration of intent but a property that has to be demonstrated - and a detector with unknown recall structurally cannot deliver that demonstration. You would be asserting anonymity toward the recipient on the basis of a filter whose error rate you do not know.

Second: the tool call toward the target system needs cleartext. The mapping meant to carry the relative personal reference crosses the boundary anew on every call. On top of that, expressions of opinion are personal data as soon as they can be attributed to an identifiable person³ - and in support tickets, complaints and meeting notes the surrounding context frequently carries that attribution even once no name is left in the text. Such texts are therefore not cleanly depersonalized even with perfect detection.

If the legal basis is in place - DPA, EU region, documented purpose limitation - the need disappears. If it is missing, the relief rests on a demonstration the filter cannot provide. In both branches of the decision tree, the building block is the wrong tool.

Masking is therefore not a legal basis but data minimization under Art. 5(1)(c), implemented via Art. 25 - risk reduction. That is more honest and considerably easier to defend.

The unspectacular route

If masking is a substitute for a missing legal basis, then the solution is to establish the legal basis. That sounds like evasion, but it is the shorter route - and it is known craft.

I am not a lawyer; this is the structure as I understand it, and it belongs in front of qualified review in any individual case.

The DPA is not the checkpoint, it is one of several conditions. It governs the relationship with the provider and makes disclosure to the processor permissible. The legal basis for the processing itself has to sit with the controller regardless - usually legitimate interest or performance of a contract. Anyone processing without a legal ground does so just as impermissibly with a DPA in hand.

Purpose limitation is the underestimated point. The data was collected for billing, support or contract handling. Sending it through an LLM now is a new processing operation with a possibly new purpose. Frequently compatible, if the model does the same job faster. Frequently not, once it turns into analysis, scoring or profiling. This is the question projects actually stall on - not the DPA.

Third-country transfer, if the provider is a US corporation. An EU region helps but does not fully resolve it, because group access from the US - the CLOUD Act - remains untouched by it. The practical route runs via standard contractual clauses plus a transfer impact assessment. The EU-US Data Privacy Framework is the more comfortable path, but politically unstable; anyone building on it alone has to redocument at the next tipping point. Standard work, but subject to documentation duties.

And the LLM-specific points: no training on the submitted data, contractually assured rather than stated in marketing material. Retention and logging at the provider. Sub-processors, because the model provider often doesn’t host itself. Art. 22 for decisions with legal effect. And for a subject access request, it has to be explainable where data sits in the pipeline - logs, traces and prompt caches are processing.

So the effort doesn’t disappear, it moves. Instead of a translation layer in the hot path, you get a record-of-processing entry, documentation of technical and organisational measures, a deletion concept spanning the whole chain, a purpose-limitation assessment and, where applicable, a data protection impact assessment. That is less work, considerably less error-prone - and it is paper instead of runtime complexity. Paper doesn’t fall apart in the middle of a session.

What matters is what this layer does not cover. A DPA permits processing. It says nothing about which tools an agent may call, which systems it reaches with write access, and under whose identity it acts. In agentic systems that is where the real risk sits - and there no legal basis applies, only structure.

What remains

The building block doesn’t disappear, it shrinks to what it can actually deliver:

Schema instead of full text. The trust boundary can be crossed without transferring full text: RAG retrieves, extraction and aggregation run locally, and a defined schema goes across the boundary. No back-translation needed, because cleartext was never on the other side. The failure mode shifts from silent and unbounded to loud and bounded - the task fails if a field could not be extracted. That is an error you can see. The price is generality: it only works if you know what you need. And it presupposes that the fields themselves are uncritical - a free-text summary field in the schema undoes the boundary again. The discipline moves from the filter to the schema design.

Classification at the source. Field-based typing instead of text detection downstream. A field typed as personal is not selected. Deterministic, auditable, versionable - everywhere structure exists.

Non-personal confidentiality. Source code, price lists, strategy papers, customer names as trade secrets. Here the reassurance “it sits at Atlassian anyway” often doesn’t apply, and there is no legal basis that dissolves the problem.

Detection without replacement. Finding PII at all is valuable as a capability in its own right: data classification, cleanup of free-text fields, an overview of where personal data sits in the estate. The recall objection from earlier applies here unchanged, but its failure mode is harmless: a missed field blocks no transfer, it only lengthens the list of open items. That is data governance, not a prompt pipeline - and it needs no reverse mapping.

Conclusion

Where the trust boundary runs is a governance decision. Trying to solve it technically builds complexity into the hot path in order to avoid a decision somebody has to make anyway.

The more durable position is also the simpler one: don’t mask, establish the legal basis properly and make sure the data crosses the boundary only where it has to. That is structural instead of probabilistic - and unlike a filter with unknown recall, it is a claim you can defend.

Which leaves the sentence that sorts the priorities: data protection is known craft with known tools. Controlling effects is not. Anyone putting effort into masking is working on the part for which solutions already exist - and leaving untouched the part for which there are none.

Thinking this through to the end lands you at a reversal: masking asks What may the model see? and answers it with a filter whose error rate nobody knows. The question that holds is What may the agent pass on, and to where? - and that one can be answered deterministically, outside the model. The building blocks exist: tool scoping, identity delegation, context-dependent policy, audit. What is missing is not the tooling but the layer built from it.

What that layer looks like I have published as a vendor-neutral blueprint under the name Dissemination Control. It complements the legal basis rather than replacing it - and it starts exactly where the filter stopped holding.

Zaratiana et al., GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction, arXiv:2605.09973 (May 2026). SPY benchmark: Savkin et al., 2025. Exact-match evaluation at span level; the model card on Hugging Face gives 0.477 as the average F1, the vendor’s accompanying article 0.471. ↩
CJEU, judgment of 4 September 2025, C-413/23 P (EDPS v SRB). ↩
CJEU, judgment of 20 December 2017, C-434/16 (Nowak) - exam answers and examiner comments as personal data. ↩