i could use that person’s blog post as more ammo for mine?
Substack partnered with Pangram so readers can scan posts, Notes, comments, and replies (generally longer than 100 words, from the July 21 rollout onward) for an estimated human/AI/mixed breakdown. It’s free and opt-in. Pangram markets an industry-leading false-positive rate of roughly 1 in 10,000. Some independent lab evaluations, including work tied to the University of Chicago’s Becker Friedman Institute, show strong results on their core test: pure human text versus pure LLM output, especially on longer passages.
Then actual writers tested it on their own work.
One author fixed a single typo in a chapter that had previously scored 100% human. New score: 100% AI. Changed one word and deliberately broke a sentence: back to 100% human. Restored the word: AI again.
Katharine English (Nobody Famous) ran an original short story she had been writing for weeks. Pangram returned 31% AI. The only concrete phrase it highlighted as evidence was “moral clarity.” She did ordinary human editing—tightened sentences, swapped the phrase for something that fit the protagonist better, cut redundancies. The AI score nearly doubled.
Just days later the same pattern played out in public. UC Berkeley mathematics professor Zvezdelina Stankova published an op-ed in the San Francisco Standard arguing that test-blind admissions had left many students years behind in foundational math. By her account, the piece represented several hundred person-hours of intensive human work and deliberation, of which about 80 hours were her own. She used AI only to help edit. Pangram scored it roughly 33% AI-generated or AI-assisted. A post highlighting the result by Berkeley Law professor Chris Hoofnagle drew nearly three million views on X, and the national conversation pivoted from her data to the detector’s verdict.
This is not a rare edge case. It is the product.
These detectors are trained and validated primarily on clean binaries: pre-LLM human text versus pure (or straightforward mirrored) LLM output. Real writing is almost never that clean. Humans edit iteratively. We fix typos. We tighten. We run light grammar or polish tools. We revise over days or weeks. The moment text leaves the laboratory distribution, the statistical signals the model relies on become noisy or inverted.
A 2025 ACL paper (”Almost AI, Almost Human”) showed state-of-the-art detectors frequently misclassify lightly AI-polished or hybrid human text as AI-generated, with rates ranging from roughly 10% to 75% depending on the detector and how light the polish is. Minor human changes routinely flip scores. The tool ends up measuring deviation from a narrow training set more than it measures authorship.
In mid-August, New York Times consumer tech columnist Brian X. Chen tested the $20-a-month standalone Pangram product for about a week. He fed it personal essays alongside chatbot text instructed to imitate his writing, plus suspected LinkedIn and Reddit posts. In dozens of tests, he reported, “the detector never failed at distinguishing slop from human writing.” He cited the 1-in-10,000 claim and the University of Chicago work, then described the experience as empowering—like putting on the sunglasses in They Live to spot the aliens.
Notice the setup: polar extremes. Pristine personal essays versus raw or lightly prompted chatbot dumps, or obvious platform spam. Of course a statistical classifier can separate those. That is a low-bar, closed-world benchmark. It says almost nothing about how the same tool behaves when real writers do the ordinary things real writers do—edit, polish, fix a typo, swap a phrase. Chen tested the detector’s strength (clean binaries). Stankova and English tested its weakness (real writing). Only the latter matters.
Chen paid $20 a month out of pocket for the privilege. Substack baked the same technology into the platform for free. Both moves rest on the same overconfidence: that a percentage score from a black-box model is reliable enough to shape reputation, reader trust, or institutional decisions in a creative publishing environment.
Pangram’s lab numbers exist, and some third-party evaluations are credible—but only within their domain. These evaluations test exactly what the detector was designed for: clean binaries. They tell us nothing about hybrid writing, lightly polished text, or the ordinary editorial work that characterizes most published prose. Menlo Ventures led their $9 million funding round in July 2026 (with other early-stage backers; total raised roughly $13 million). That capital, and the momentum behind it, suggests the industry is betting on detection—not because the science supports it at scale, but because the market wants a solution.
Version history, process disclosure, and human editorial judgment remain stronger evidence of authorship than any percentage that moves when you fix a typo or run a light polish pass.
The anti-slop detector is the slop.
(This post was written by a human, with feedback, structural suggestions, and collaborative refinement from Claude, Gemini, and Grok. Run it through Pangram after any edit and watch the score change.)
References
Katharine English, “Pangram Flagged My Own Writing as AI,” Nobody Famous (Substack), 23 July 2026.
Brian X. Chen, “I Tested a Popular A.I. Slop Detector. It Felt Empowering,” The New York Times, 13 August 2026. https://www.nytimes.com/2026/08/13/technology/personaltech/pangram-ai-detector-test.html
Coverage of Zvezdelina Stankova op-ed and Pangram score (Daily Californian, The Guardian, The Next Web, New York Post, etc.), mid-to-late August 2026. Key examples: https://www.dailycal.org/news/campus/uc-berkeley-professor-discloses-ai-use-in-op-ed-open-letter-calling-for-reinstatement-of/article_be1360f7-6e2a-42f6-ae76-f05d8cee74c4.html https://thenextweb.com/news/berkeley-professor-ai-op-ed-pangram-detector https://www.theguardian.com/us-news/2026/aug/19/uc-berkeley-professor-ai
Shoumik Saha & Soheil Feizi, “Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing,” Findings of the Association for Computational Linguistics: ACL 2025. https://aclanthology.org/2025.findings-acl.1303/ arXiv: https://arxiv.org/abs/2502.15666
Substack Help Center, “How can I detect AI on Substack?” (updated August 2026). https://support.substack.com/hc/en-us/articles/50891130623508-How-can-I-detect-AI-on-Substack
Pangram claims on false-positive rates and third-party evaluations. https://www.pangram.com/blog/all-about-false-positives-in-ai-detectors https://www.pangram.com/blog/third-party-pangram-evals
Menlo Ventures investment note and contemporaneous funding coverage (TechCrunch, VentureBeat), 29 July 2026. https://menlovc.com/perspective/investing-in-pangram-to-stop-ai-slop-on-the-internet/ https://techcrunch.com/2026/07/29/as-ai-content-floods-the-internet-pangram-raises-9m-to-detect-it/
