A policy that says a company does not train on your data and a model that cannot reproduce your data are different promises. Only one of them is a legal statement. The other is a technical property, and there are documented ways to test for it.
TL;DR
- A no-training promise describes the pipeline, not the model. Anthropic’s commercial terms state that “Anthropic may not train models on Customer Content from Services” (Commercial Terms, effective Jun 17, 2025). The consumer terms run the other way: materials may be used for model training “unless you opt out of training through your account settings” (Consumer Terms, effective Oct 8, 2025).
- Memorization is measurable. Shokri and colleagues defined membership inference as deciding, with black-box access to a model, whether a specific record appeared in its training set, and tested it against classifiers offered by Google and Amazon (arXiv:1610.05820).
- Verbatim reproduction is not hypothetical. Carlini and colleagues extracted hundreds of verbatim training sequences from GPT-2, including names, phone numbers, email addresses, IRC conversations, code and 128-bit UUIDs, and succeeded even where a sequence appeared in a single training document (arXiv:2012.07805).
- A clean result proves very little. The same literature warns that an attack has to be scored at very low false-positive rates before its output means anything.
- Three levers sit in your hands: the training summary EU law asks general-purpose providers for, the opt-out toggle on a consumer account, and running the model on your own machine.
Training inclusion is not reproduction
Two different questions hide inside the question of whether a vendor trained on your data. The first is whether your file was in the training set. The second is whether the model can hand your content back. A model can be in the second state without anyone being able to recover the first, and the second state is the one that does the damage.
Shokri and colleagues formalized the first question as membership inference: given a record and black-box access to a model, decide whether that record was in the model’s training data. Their method trains an auxiliary model that separates the target model’s behavior on records it has seen from its behavior on records it has not. They report that models exposed through Google’s and Amazon’s services could be susceptible, and they test on a hospital discharge dataset where membership carries real consequences (Shokri et al.).
Carlini and colleagues went after reproduction directly. Their extraction attack pulled hundreds of verbatim sequences out of GPT-2, including names, phone numbers, email addresses, IRC conversations, code and 128-bit UUIDs. The detail that matters for your own content is that a sequence appearing in only one training document still came back out, and that larger models were more vulnerable than smaller ones (Carlini et al.). A billion-parameter model does not need your document to be common for it to be recoverable.
Two probes you can run
Plant a string and ask for it back
The extraction result points at a cheap practical probe. Because models reproduce rare strings verbatim, a string you invent and publish somewhere public becomes a marker: if a later model completes it for you with web search and retrieval switched off, that is strong evidence the text was in its training data. The mechanism is the same verbatim memorization the extraction research documents, turned from an attack into a receipt.
Two caveats keep this honest. A miss proves nothing, because a missing completion is not evidence of missing data. And a hit points to the training set only if the model could not have looked the string up, since a model with web search or a retrieval tool can find a public page at answer time; even a genuine hit says nothing about whether anyone read the material or misused it.
Point it at material you own, to check a claim a vendor made to you.
Run membership inference against a record you care about
The stronger test takes a candidate record, asks the model to behave differently on records it knows and records it does not, and checks whether the candidate looks like the “seen” group. Shokri and colleagues report that models exposed through Google’s and Amazon’s services could be susceptible to exactly this (Shokri et al.).
Doing this against a large language model used to assume you already had a comparable non-member dataset. Fu and colleagues’ SPV-MIA removes that assumption: it builds the reference set by prompting the target model itself and adds a probabilistic-variation signal tied to memorization. Across three datasets and four large language models they report area under the curve rising from roughly 0.7 to about 0.9 (Fu et al.).
A negative result is not a clean bill of health
Every attack above produces a number, and the number needs a threshold before it means anything. Carlini and colleagues argue that average-case accuracy is a poor way to judge a membership attack, and that the figure worth reading is the true-positive rate at very low false-positive rates, on the order of under 0.1%. At that threshold their Likelihood Ratio Attack is reported as ten times more powerful at low false-positive rates, and they find that most prior attacks perform poorly when evaluated that way (Carlini et al.).
Read a miss accordingly. It says the attack did not separate your record from the reference set at a threshold low enough to be meaningful. It does not say your record was absent.
Two more limits are worth carrying. Fu and colleagues point out that both reference-free and reference-based attacks lean on the target model overfitting its training records, that regularization and the generalization of large language models mitigate that overfitting, and that this leads to high false-positive rates for existing attacks in practice (Fu et al.). And the leak may not be your exact text. Tao and Shokri’s range membership inference tests whether training covered any point inside a semantically defined range rather than one exact record, and report more accurate privacy-loss measurement than conventional attacks across tabular, image and language data (Tao and Shokri). Near-duplicates count.
What you can demand, and what you can switch off
Regulation now obliges some providers to answer the corpus question in public. Providers of general-purpose AI models must “draw up and make publicly available a sufficiently detailed summary about the content used for training” (Article 53, European Commission AI Act Service Desk). Paragraph 2 of the same article removes the technical-documentation and downstream-information duties in points (a) and (b) for models released under a qualifying free and open-source licence when the weights, architecture information and usage information are public, and keeps that exception unavailable for models classified as presenting systemic risk. The training-content summary in point (d) is not part of that exemption, so it is owed either way. The Service Desk notes its own summaries are explanatory rather than legally binding, so treat the summary as a starting point for questions rather than a complete answer.
What you can change directly is narrower and more certain. On a consumer account, training is the default and the opt-out is a switch: Anthropic’s consumer terms say materials may be used for model training “unless you opt out of training through your account settings”, with exceptions where you provide feedback or your materials are flagged for safety review, and the same terms add that “we may use the Feedback however we choose without any obligation or other payment to you” (Consumer Terms). On a commercial account the default flips, and the commercial terms state that “Anthropic may not train models on Customer Content from Services” (Commercial Terms). That page does not set a retention period itself and points to the Anthropic Data Processing Addendum for the privacy terms, which is where a retention question belongs.
And then there is running the model yourself. If inference happens on hardware you control, there is no training corpus to add your data to in the first place.
What this means
The policy question and the technical question are being answered by different people, and they have different answers. A vendor can honour a no-training promise perfectly and still ship a model whose training set contained your content from an earlier era, a shared dataset or a public corpus you had no idea was in scope. That gap is the one worth measuring.
The techniques above are not a magic detector. They are a way to convert a vague assurance into a specific question with a defined error rate, and to find out what the answer is when you look at the false-positive rate rather than the headline accuracy.
The bottom line
Three things are worth doing. Read the retention terms rather than the marketing summary, and check whether you are covered by a commercial agreement or a consumer default. Turn off the consumer training switch if you did not mean to be in the training set. And if the question is whether a specific document made it into a model’s weights, a string you planted gives you a rough probe built on the same mechanism as the extraction result, while the membership inference literature gives you a method and, more usefully, a threshold that tells you when to trust it.
None of that proves a negative. It does tell you what you actually know, which is more than a policy page does.
Related on Intelligibberish
- Local LLM Privacy: What Each Runtime Actually Sends Home - what a local runtime still collects by default, so running the model yourself is not the whole answer
- Privacy