Pew Study: How Much of the Web Is AI-Written?

Introduction

Pew Research Center's Data Labs team published a study on Thursday, August 20, 2026, that attempts to measure how much of the web now carries signs of AI authorship. The project is notable less for any single shocking number and more for how carefully it separates two figures that are easy to conflate. Across the most recent crawl studied, about 10% of all sampled pages show significant signs of AI authorship. But when the sample is narrowed to only pages published after ChatGPT's November 2022 launch, that share rises to over one-third, roughly 35%. Both numbers come from the same dataset and the same detector; they answer different questions, and mixing them up produces a materially different, and wrong, headline.

Methodology

Pew drew its sample from Common Crawl, the open web archive widely used as training data for large language models. Researchers pulled 10,000 random English-language pages from each of 49 monthly Common Crawl snapshots spanning January 2021 through July 2026, for a combined sample of nearly half a million pages. Every page was scored with Open Pangram's open-weight detection model, editlens_Llama-3.2-3B, which outputs a score between 0 and 1 representing the degree of likely AI authorship in a given text. Pew treated pages above a significance threshold as showing meaningful signs of AI involvement. Because the same detector and sampling frame were applied consistently across five and a half years of snapshots, the study can track how the share of AI-flagged pages has moved over time, rather than offering just a single snapshot-in-time estimate.

Two Numbers, Two Questions

The gap between the 10% and 35% figures is explained by what each one is actually counting. The 10% figure covers every page in the July 2026 crawl, regardless of when that page was originally published. Common Crawl contains a large backlog of pages that have existed on the web for years, many written well before generative AI tools were widely available, and those older pages pull the overall average down. The 35% figure filters out that backlog and looks only at pages whose publication date falls after ChatGPT's launch in November 2022. Within that narrower, more recent slice, the AI-authorship signal is far more common. Neither number is more "correct" than the other; they simply describe different populations of pages, and Pew's own framing treats the gap itself as one of the study's more informative findings.

The .edu and .gov Floor

One of the more interesting results sits in the January 2026 domain-level breakdown, which shows how the AI-authorship signal varies by top-level domain.

.com domains show roughly twice the signal of .org domains, and around ten times the signal seen on .edu and .gov domains. That gap plausibly tracks differences in publishing incentives and editorial process: commercial sites generate high volumes of content at speed, while academic and government pages tend to move through slower institutional review before publication. The study does not attribute a specific cause to the gap, but the pattern is large and consistent enough across the dataset to be treated as a real signal rather than noise.

Stylistic Drift

Alongside the detector scores, Pew tracked several stylistic markers associated with AI-written text between 2023 and 2026. The frequency of em dashes doubled over that period. Use of the Oxford comma rose 63%. Vocabulary often associated with AI writing, including words like "delve" and "interplay," more than doubled in frequency. Negative parallelism, constructions like "it's not X, it's Y," nearly tripled. These trends move in the same direction as the detector's AI-authorship scores, which strengthens confidence that something real is being captured. But Pew is explicit that these stylistic markers are correlational, not diagnostic on their own; an em dash or an Oxford comma in a single document proves nothing about its authorship.

What the Numbers Do Not Show

Pew states plainly that its detection model is not perfect and can misclassify individual documents, even as the aggregate patterns hold at scale. Several limits are worth stating directly. A detection score is not proof of authorship for any single page; it is a probabilistic estimate. The study's single score also collapses a meaningful distinction: it cannot separate human writing that received light AI editing from text that was generated by AI wholesale. Common Crawl itself is a widely used but imperfect proxy for "the web," shaped by its own crawling and sampling choices rather than a true census of all pages that exist. And the 35% figure depends on publication-date metadata attached to web pages, which is often unreliable, incomplete, or backdated, meaning that headline number carries more uncertainty than the 10% figure that does not depend on dates at all.

Pros and Cons

Pros:

• A consistent detector and sampling method applied across 49 monthly snapshots enables a genuine longitudinal comparison, not just a single point estimate
• Splitting results by publication date surfaces a real and explainable structural feature of Common Crawl rather than hiding it
• The domain-level breakdown isolates a large, consistent gap between commercial and institutional web content
• Tracking stylistic markers alongside detector scores gives independent, converging evidence for the trend
• Using an open-weight detection model makes the methodology reproducible by outside researchers

Cons:

• The detector's false-positive and false-negative rates are not zero, and Pew acknowledges individual documents can be misclassified
• Common Crawl is not a representative sample of the entire web, and its crawling choices introduce their own bias
• The study cannot distinguish lightly AI-edited human writing from fully AI-generated text; both register on the same score
• Stylistic markers such as em dashes and Oxford commas are correlational signals, not reliable diagnostic evidence on their own
• Publication-date metadata on web pages is often unreliable, which directly affects confidence in the 35% post-ChatGPT figure

Outlook

The study's implications extend beyond measurement for its own sake. Common Crawl is a primary data source for training large language models, so a rising share of AI-authored pages inside it raises questions about how much AI-generated text future models will end up training on, a feedback loop researchers have flagged as a risk for degrading training data quality over time. For search engines and platforms that rank or surface web content, a growing AI-authorship signal also raises the practical question of whether ranking and trust signals need to account for authorship provenance, not just content quality. Pew's approach, running the same detector consistently across years of snapshots, offers a template other researchers could reuse to track this trend going forward, though any follow-up work will inherit the same detector-accuracy and metadata-reliability limits documented here.

Conclusion

Pew's study is best read as a measurement exercise with clearly stated limits, not a verdict on how much of the internet is "fake." The two headline numbers, 10% of all pages and 35% of pages published since ChatGPT's launch, describe different populations and should not be swapped. The .edu and .gov floor and the parallel rise in AI-associated stylistic markers are the more durable findings, since they hold up as patterns even where individual document classifications might be wrong. Readers evaluating AI's footprint on the web, including researchers, journalists, and platform teams, should treat this as a directionally useful data point rather than a precise count.

Editor's Verdict

Pew's measurement of AI authorship across the web earns a solid recommendation within the research space.

The strongest case for paying attention: consistent detector and sampling method across 49 monthly snapshots enables genuine longitudinal comparison. That alone raises the bar for what readers should expect in this space. Reinforcing that, splitting results by publication date surfaces a real, explainable structural feature of Common Crawl — practical value rather than just headline appeal. The broader signal worth registering is straightforward: the 10% and 35% figures measure different populations of pages, not the same thing under two labels. On the other side of the ledger, one constraint is real rather than a marketing footnote: the detector's false-positive and false-negative rates are not zero, and a single score cannot separate lightly AI-edited human writing from fully AI-generated text. It should factor into any serious decision. Layered on top of that, Common Crawl is not a representative sample of the entire web, introducing its own sampling bias — which narrows the set of teams for whom this is an obvious yes.

For ML researchers, technical leads, and readers tracking the underlying science behind new capabilities, this is a serious evaluation candidate, not just a curiosity to bookmark. For everyone else, the safer posture is to monitor coverage and revisit once the use cases that matter to your team are demonstrated in the wild.