GDPR AI Training Data Scraping Rules Just Landed
One in every seven pages inside Common Crawl — the dataset behind most frontier language models — contains data that Europe now classifies as legally sensitive. That's not a guess. It's the finding of an academic audit: roughly 155 million of 1 billion sampled URLs fell under GDPR Article 9, the article covering health, biometric, political, and other special-category data.
On July 8, 2026, the European Data Protection Board made that finding impossible to ignore. At its Brussels plenary, the EDPB adopted Guidelines 03/2026 on Web Scraping in the Context of Generative AI — the first pan-EU framework built specifically for how AI labs collect training data. A companion release, Guidelines 02/2026 on Anonymisation, landed the same day. Together they close the "public means fair game" reading of GDPR that AI labs have leaned on since ChatGPT shipped.
The short version
- The EDPB adopted Guidelines 03/2026 (web scraping) and Guidelines 02/2026 (anonymisation) on July 8, 2026, at its Brussels plenary session.
- An academic audit of Common Crawl found ~155 million of 1 billion sampled URLs fall under GDPR Article 9 special-category data.
- The EDPB confirmed there is no blanket Article 9 exemption for AI training — every case needs an individual legal basis.
- Anonymisation now has to clear a three-part test: no record isolation, no linkage, no inference.
- Public consultation on both guidelines runs through October 30, 2026, before final adoption.
What do the new EDPB guidelines actually require?
Guidelines 03/2026 apply GDPR's core principles — purpose limitation, data minimisation, transparency, accuracy — directly to the scraping step, not just the training step. That means a developer needs a documented legitimate-interest assessment before a crawl runs, not after a model ships.
The accuracy principle gets a scraping-specific reading too: pipelines now need to record when data was collected and validate it before it enters a training set. A 2023 forum post scraped in 2026 and treated as current fact is now a compliance problem, not just a quality one.
Why does Common Crawl's 155-million-URL number matter?
Common Crawl backs the pre-training corpus for most large language models built outside a handful of labs with proprietary crawls. If roughly 15% of its sampled URLs carry Article 9 data — health forums, political commentary, union membership, sexual orientation disclosures — special-category exposure isn't an edge case in AI pre-training. It's structurally built into the raw material the industry has been using for free.
The EDPB was explicit on this point: there is no general exemption from Article 9 for AI training, and developers can't wave the whole corpus through under one legitimate-interest claim. Every case gets assessed on its own.
The old pipeline order is now backwards
Most large-scale scraping pipelines run quality and language filters after the raw crawl is assembled — grab everything, then clean it. The EDPB guidelines flip that. Legal compliance filtering — checking for special-category data, verifying a lawful basis exists — now has to happen before the crawl, not as a downstream cleanup pass.
| Old pipeline order | EDPB-compliant order |
|---|---|
| Crawl everything → filter for quality/language → train | Assess legal basis → crawl with compliance filters → filter for quality/language → train |
| Article 9 risk discovered late (if at all) | Article 9 risk screened at collection |
| Anonymisation treated as a checkbox | Anonymisation tested against isolation/linkage/inference |
Does the anonymisation test change what counts as "safe" training data?
Yes. Guidelines 02/2026 replaces the old working definition of anonymised data with a three-criterion test: no record isolation, no linkage to other datasets, and no inference back to an individual. Data that passes today's looser bar may not clear this one — regulators built the test around what modern AI models can actually re-identify, not what a 2016-era anonymisation checklist assumed.
The guidelines also draw on the CJEU's C-413/23 P EDPS v SRB reasoning from September 2025, which narrowed how "reasonably likely" re-identification gets assessed. That case law is now baked into how DPAs across the EU will read anonymisation claims going forward, per the EDPB's own announcement.
Both guideline sets are still open for stakeholder input — the consultation window runs to October 30, 2026 — so the final text can still shift before formal adoption. AI developers, civil-society groups, and DPAs all get a say before this becomes binding guidance rather than draft guidance.
This is exactly the seam ScrapeOps was built to sit on: a dataset that's been screened and deduplicated at collection time is a dataset you can defend later, instead of one you have to audit for Article 9 exposure after the fact.
None of this makes web scraping for AI illegal. It makes "scrape first, ask questions never" the version that gets a lab an EDPB inquiry instead of a training run. The labs that built compliance filtering into the collection layer — rather than bolting it on after a Common Crawl download — are the ones who won't be re-auditing a billion URLs in November.
Frequently Asked Questions
Does GDPR apply to AI training data scraped from public websites? Yes. The EDPB's Guidelines 03/2026 confirm that GDPR governs any scraping operation involving personal data — collection, storage, organisation, retrieval — regardless of whether the source page was public. Purpose limitation and transparency principles apply from the moment of collection, not just at training time.
What percentage of Common Crawl contains sensitive data under GDPR? An academic audit found approximately 155 million of 1 billion sampled URLs — roughly 15% — fell into GDPR Article 9 special-category data, covering health, political opinion, biometric, and similar sensitive information routinely present in pre-training corpora.
What is the EDPB's three-part anonymisation test? Guidelines 02/2026 require data to pass three checks to count as truly anonymised: no record isolation (can't be singled out), no linkage (can't be connected to other datasets), and no inference (can't be used to deduce facts about an individual). All three must hold.
When do the new EDPB guidelines become binding? Both Guidelines 03/2026 (web scraping) and 02/2026 (anonymisation) are open for public consultation through October 30, 2026. They are not yet final law — stakeholder feedback can still change the text before the EDPB adopts a final version.
Abhishek Gupta is Co-Founder at Dekrypt Labs, building ScrapeOps — the data acquisition engine that turns any question into clean, deduplicated, comprehension-ready sources. See the full product line or read more dispatches. dekryptlabs.com