Can a simple projection reduce what RAG systems leak?
Randomly compressing query and document embeddings sharply reduced extraction in four laboratory tests while retaining much of the utility. It is an empirical defence, not a privacy guarantee.
The short version
- What happened
- The study tested two RAG designs on Enron email, European court documents, C4 and HotpotQA using repeated-query and adaptive extraction attacks.
- Who it affects
- Employees and customers can be exposed when a chatbot retrieves private records that were never meant to appear in an answer.
- What remains unproven
- The evidence is an unreviewed laboratory evaluation on public or synthetic datasets and known attacks. The method has no formal differential-privacy guarantee and lacks production-scale validation.
Keep in mind
The evidence is an unreviewed laboratory evaluation on public or synthetic datasets and known attacks. The method has no formal differential-privacy guarantee and lacks production-scale validation.
Source analysis · 1 linked source. Primary analysed source date: 15 October 2026.
Read the full report · 4 min The evidence, methods, implications and limitations
The defence changes the retrieval space before an attacker can probe it
A retrieval-augmented chatbot can reveal material from its document store when an attacker repeatedly asks carefully shaped questions. University of Chicago researchers tested a lightweight defence: multiply both document and query embeddings by the same random matrix before searching. The compressed vectors keep much of their relative similarity, but the retrieval path becomes noisy enough to frustrate the extraction attacks used in the paper.
The approach is attractive because projection happens once for stored documents and requires one matrix multiplication per query. The team evaluated a nearest-neighbour language model and a standard retrieve-then-prompt system across four public datasets: Enron email, European Court of Human Rights documents, C4 web text and HotpotQA. Attacks included ten repeated attempts per item and adaptive campaigns of up to 200 queries.[1]
Leakage fell sharply in these tests, with an uneven utility cost
In the Enron repeated-query test, unprotected standard RAG exposed 217 personal items: 36 email addresses, 56 URLs and 125 phone numbers. Random projection reduced that count to three URLs. On European court documents, leakage for standard RAG with a Qwen model fell from 21.30% to 2.05%, while answer F1 moved from 97.32 to 95.43. Results varied by model and dataset, which is why one headline figure cannot describe the whole method.
Against two multi-round HotpotQA attacks, unprotected RAG reconstructed 60% and 75% of targeted chunks; the projected system reached 3% and zero under the paper's criterion. Its answer and supporting-document scores remained close to the unprotected system on several measures. Across the full study, the authors summarise utility as about 7% lower than vanilla RAG while outperforming the tested differential-privacy baselines on utility.[1]
The distinction that matters
What random projection changes
The same hidden transformation is applied to documents and incoming queries before retrieval.
Documents
Project stored embeddings into fewer dimensions
The original embedding space is not searched directly
Queries
Apply the same random matrix
Relevant neighbours remain approximately close
Attack test
Repeat or adapt extraction prompts
Measured recovery falls under the tested attacks
Empirical resistance is not the same as formal privacy
The comparison matters for organisations deciding how to protect an internal knowledge base. Random projection may be a practical extra layer when formal differential privacy is not required, but it does not prove that a new attack cannot recover sensitive content. The authors deliberately measure resistance to known attacks rather than offer a worst-case privacy guarantee. Access controls, data minimisation, output monitoring and incident response therefore remain necessary.
The experiments used public or synthetic material rather than a private production store, and the systems were small enough for laboratory testing. They also selected a projection dimension of 100 after examining the privacy and utility trade-off, so deployment would need its own held-out tuning and monitoring. The paper estimates that its current setup takes about eight seconds per query; a million-query evaluation would take roughly 92 days without optimisation. It reports no funding or competing-interests statement in the accessible version. Independent red-team work should test varied embedding models, languages, document types and adaptive attackers at production scale, while measuring latency and retrieval errors for ordinary users. Until then, random projection is a promising baseline—not a declaration that a RAG datastore is safe.[1]