Key Points
ใปNature reported on August 6, 2026 that a machine-learning model built to predict molecular boiling points disagreed with values in a widely used chemistry reference database, and that checking the original literature showed the database was wrong. The number of errors and the name of the database have not been disclosed, and no paper documenting the work had been published as of August 30, 2026.
ใปMachine checking of reference data is not new. A paper published in the Journal of Chemical & Engineering Data in February 2011 compared neural network predictions with handbook values for the boiling points and enthalpies of vaporization of the elements, and found discrepancies of 10 to 900 percent in published property tables.
ใปAI finds errors and manufactures them at the same time. A May 2026 preprint that audited 2.5 million papers found roughly 147,000 non-existent references in 2025 alone. The open question is no longer AI versus humans, but who confirms a flagged error and who has the authority to correct the record.
A Prediction Disagreed With the Handbook, and the Handbook Was Wrong
Nature reported on August 6, 2026 that an artificial-intelligence model had exposed errors that had survived for decades in a chemistry reference database. Sebastian Pios, a theoretical chemist at Zhejiang Lab in Hangzhou, China, used a machine-learning model to predict the boiling points of molecules and found that the predictions did not match long-trusted reference values.
Pios initially suspected the model. Tracing the numbers back to the original literature, however, showed that the reference database was the party at fault, according to the reporting. The errors reportedly included a typographical error in an old paper and a measurement error from a study conducted roughly a century ago. Neither the number of errors nor the name of the database appears in the article.
Nature’s subheading states that the technology is proving adept at finding faults in decades-old papers and reference databases. No research paper documenting this particular verification effort had been published as of August 30, 2026, so the details of the case, including the model used and the number of errors found, remain within the reporting rather than the literature.
Related Articles
How an Error Survives a Century Inside a Trusted Number
A chemistry reference database is a curated collection of measured physical values, such as boiling points, gathered from published papers so that working chemists do not have to remeasure them. Its value lies precisely in not being questioned. Chemists consult these numbers to identify unknown substances and to design separations such as distillation, and they treat the published value as settled.
That trust is what gives an error its lifespan. A measurement or a typographical slip in one paper is absorbed into a compilation, the compilation is loaded into a database, and later papers cite the database. Unless someone in that chain returns to the original source, the wrong number circulates with the authority of the record behind it.
This is not evidence that science is careless. Millions of papers are published every year, and remeasuring every value by hand is not possible. Science achieves its speed by building on results it did not verify itself, and the same trust that makes accumulation possible extends the life of the rare error that slips through.
Correction is a separate problem from detection. Even when an error in an original paper is identified, no mechanism automatically pushes that correction to the compilations that already absorbed the value or to the papers that already cited it.
One more caution belongs here. A boiling point depends on pressure and sample purity, so a gap between a predicted value and a published one does not by itself prove an error. The gap is an entry point for suspicion; confirmation requires going back to the source.
Machines have been auditing reference data since 2011
Using a machine to recheck reference data is not a 2026 invention. A paper published in February 2011 in the Journal of Chemical & Engineering Data by Yiming Zhang, Julian R. G. Evans and Shoufeng Yang compared artificial neural network predictions against handbook values for the boiling points and enthalpies of vaporization of the elements, and reported discrepancies of 10 to 900 percent in published property tables, along with the values most likely to be correct.
The idea of using a model’s predictions as grounds to doubt a published value carries straight through to the 2026 case. The 2011 study addressed the elements, while the recent case reportedly concerns molecules. The sharper difference is in how each result reached the world. One went through peer review and was published; the other has so far appeared only as news.
Comparable efforts exist outside chemistry.
| Year | Effort | Method | What it surfaced |
|---|---|---|---|
| 2011 | Recheck of handbook data for the elements | Neural network predictions against published values | Discrepancies of 10 to 900 percent in property tables |
| 2015 | Automated check of statistics in psychology papers (statcheck) | Recomputation of reported test statistics | Half of published papers using significance tests contained at least one inconsistency |
| 2021 onward | Problematic Paper Screener (University of Toulouse) | Weekly scan for unnatural paraphrased phrasing | More than 1,500 suspect papers among 2021 publications alone |
(Compiled from each project’s published materials. The counts are not comparable to one another because the targets and denominators differ. As of August 30, 2026.)
The tooling for machine-assisted error hunting predates generative AI by well over a decade. The recent case is the newest entry in that lineage rather than the start of it.
The Audit Arrived as a By-Product, Not a Mission
Nobody set out to audit anything
The most interesting feature of the reported case is that no one sent an AI to look for errors. Nature’s headline says “AI agents,” but the publication’s own lead paragraph describes an artificial-intelligence model revealing the problem, and the sequence described in the reporting is that a boiling-point prediction failed to match a reference value. The audit was a by-product of building a predictive model.
The by-product has a structural cause behind it. A human researcher can read only so many papers in a day and cannot reconcile a century of literature against millions of published numbers. A machine repeats the same comparison without tiring. The strength on display is not insight; it is throughput at a scale humans cannot reach.
That also means the novelty is not in the tool. Machine rechecking of reference data was published in 2011. What changed is the scale at which the comparison can be run, and the volume of errors the machines themselves now generate.
The confirmation stayed human. The model produced only a mismatch signal. Deciding which side was wrong required Pios going back to the original literature. Machines narrow the candidates and people settle them, a division of labor this article’s companion piece on an 87-year-old mathematical conjecture also described. That case concerned producing a new result; this one concerns auditing the quality of an old record.
Related article
An 87-Year-Old Conjecture Fell to a Single Counterexample. Whose Job Is Discovery Now?
Why is AI also the largest new source of errors in the literature?
While AI checks the record, it is also contaminating it. The clearest example is the fabricated reference, presumed to be AI-generated: a plausible-looking author and title that correspond to no actual paper, now appearing in published articles and preprints.
Estimates of the scale diverge sharply.
| Study | Corpus | Non-existent references detected |
|---|---|---|
| May 2026 preprint (Zhao et al.) | 2.5 million papers, 111 million references | About 146,900 in 2025 alone, described by the authors as a conservative estimate |
| Columbia University study published in The Lancet, May 2026 | More than 2 million papers, 97 million references | About 4,000 fabricated citations across 2,800 papers |
(Compiled from each study’s published materials and reporting. The definitions of what counts as a non-existent or fabricated citation differ, so the figures cannot be compared directly. As of August 30, 2026.)
Two audits of corpora of roughly the same size differ by more than thirtyfold in what they found. Because the criteria and the targets differ, the scale of AI-generated error in the literature currently has no settled number. AI finds human errors, humans confirm AI errors, and further automated checks assist that confirmation. Something worth calling mutual auditing is already running inside scientific publishing.
How well do error-finding AI systems actually perform?
The record is uneven. SPOT, a benchmark released in May 2025, asked models to inspect 83 papers containing 91 errors serious enough to have triggered errata or retractions.
According to the SPOT paper, posted in May 2025, no state-of-the-art model of that period exceeded 21.1 percent recall and 6.1 percent precision, and repeated runs rarely surfaced the same error twice. Under the benchmark’s conditions, the 6.1 percent precision reported in that May 2025 evaluation means that more than nine in ten of the flagged items were not real errors. Given that a wrong accusation can damage a researcher’s career, that level of performance does not support letting AI adjudicate science.
Narrow the target and the picture changes. A December 2025 study from a Stanford University team restricted its checker to errors with objective answers, such as equations, derivations, calculations and figures. In that December 2025 study, human experts reviewed 316 flagged items and confirmed 263 of them, a precision of 83.2 percent.
In June 2026, a Google team reported that its review-assistance tool improved zero-shot recall on SPOT’s mathematical errors by 34 percent relative to the baseline. Because the evaluation conditions differ, these results cannot be lined up against one another. What they share is a design choice: stay away from interpretation and novelty, and target errors that can be checked against an answer. That is the realistic boundary at present.
Who actually fixes an error once it is found?
Detection has automated faster than correction. The reporting does not say whether the errors identified in the reference database were corrected.
Some resources carry part of the answer already. The NIST Chemistry WebBook, maintained by the U.S. National Institute of Standards and Technology, attaches source references and method notes to individual values, so a user can see how a number was obtained. The machinery for tracing where a value came from already exists. The open question is what happens when AI raises doubts at scale: who confirms them, and which record gets updated, by what procedure.
Listing the parties makes the gap visible. Errors are surfaced by researchers and AI companies, confirmed by domain specialists, and correctable only by database operators and publishers. The party that finds an error and the party that can change the record are different, and no standard procedure connects them.
Detection moves at machine speed and correction at institutional speed. Better precision would reduce the confirmation burden, but the more flags the machines produce, the more visible that mismatch becomes.
Then there is downstream propagation. The wrong value has already been copied into other compilations and cited in later papers. Fixing the source does not reach the copies. The genuinely hard part of error hunting in science is not detection; it is recall of what has already circulated.
From “who said it” to “can it be re-checked”
One proposed direction addresses exactly this. A Comment published in Nature Computational Science on August 20, 2026 argues that as AI takes on more of the research process, trust should rest not on fully understanding a model’s internals but on provenance, meaning a complete and re-openable record of what a system consulted, executed and measured.
The authors are researchers at Carnegie Mellon University, and two of them disclose that they co-founded a consultancy for the scientific evaluation of frontier AI models and are affiliated with a research organization within Alphabet. The proposal comes from parties with a commercial stake in AI evaluation, and deserves to be read with that in view.
An editorial from the journal’s own editors appeared in the same issue with a different emphasis: AI should aid scholarly judgment rather than substitute for it, and responsibility for accuracy and conclusions remains exclusively human regardless of the tools used. That editorial also treats uploading an unpublished manuscript to a public AI service as a breach of confidentiality.
A proposal that records can make autonomy trustworthy, and a principle that accountability cannot move off human shoulders, printed side by side. The pairing is a fair snapshot of where the field stands.
Behind both sits the question of autonomous science. An August 2026 Perspective on AI agents in computational chemistry counts such systems growing from about half a dozen in 2024 to close to fifty as of August 8, 2026. The same paper notes that adoption beyond the developers themselves remains extremely limited and that every reported system keeps a human in the loop. Rapid proliferation and minimal use coexist, which is what a field looks like before takeoff.
Viewed through provenance, the boiling-point case is instructive in an uncomfortable way. The finding reached the world as news, and no paper documents the verification. Which values were wrong, and what the model learned to arrive at its prediction, are not available for anyone else to re-check.
An audit only becomes part of the machinery of science when its findings are shared in a form others can re-examine. Whether this case is published as a paper, and whether the database issues a correction, are the first observable markers of where it goes next.
How Japanese Researchers See AI Auditing Science
The boiling-point story itself barely registered in Japanese. A sweep of Japanese-language posts on X on August 30, 2026 found no posts discussing the Nature report directly, even though the mirror-image story, AI generating fake citations, has circulated widely in Japanese. That absence is itself a finding: in Japan the conversation has attached to publisher policy and peer review rather than to the audit case. What follows is a sample of posts from researchers and specialists on X, not a measure of Japanese opinion. Summaries are given in English; the embedded posts carry the original Japanese.
The most widely shared thread in this space in late August came from a biomechanics researcher, breaking down Elsevier’s updated policy on generative AI in manuscript preparation.
Elsevierใ็ๆAIใฎๅฉ็จๆน้ใๆดๆฐใใพใใ๏ผ
— Gakuto, ph.D. | Biomechanics (@gkt_phd) August 25, 2026
็ต่ซ๏ผ
่ซๆๅท็ญใงAIใไฝฟใใใจ่ชไฝใฏ็ฆๆญขใใใฆใใพใใ๏ผ
ใใใ๏ผ
ใปๆ็ฎใฎ่ฆ็ด
ใป็ ็ฉถใฎใฃใใใฎๆข็ดข
ใปใขใคใใขๅบใ
ใปๆ็ซ ๆงๆ
ใป่ฑ่ชใ่ชญใฟใใใใฎๆนๅ
ใชใฉใธใฎๆดป็จใฏ่ชใใใใฆใใพใ๏ผ
ใใ ใ๏ผ้่ฆใชใฎใฏใใ๐
โ โฆ
The post walks through what the publisher now permits, including summarization, gap-finding and help with structure, and lays out the conditions attached to each. Posted on X on August 25, 2026, it drew roughly 800 likes and 112,000 views. Notably, the framing is not whether AI may be used but under what conditions and with what accountability, which is the same shift this article traces on the correction side.
The clearest statement of the underlying change came from a researcher reporting on the ICML 2026 conference in July.
— Yasuo TABEI (@yasuotabei) July 10, 2026
Summarizing Arvind Narayanan’s keynote, the post relays the argument that the center of gravity in research work is moving from execution toward setting goals, evaluating results and carrying responsibility for them, and that the limiting problem with current AI agents is reliability rather than capability. That is the human-confirmation role described above, stated as a change in what a researcher’s job consists of.
A sharper question about institutional honesty also spread widely.
ๆป่ชญใงAIใ็ฆๆญขใใฆใใใจใใใๅคใใ๏ผใฉใใใฟใใช่ซๆใ่ชญใๆใฏAIไฝฟใใฎใซ๏ผAIไฝฟใใชใๅๆใง่ฉไพกใใใฎใฃใฆ่ฉไพกใจใใฆๆญฃใใใใ๏ผใจใใๆใใใใ
— ใคใ๐ค๐๐๐ ๐๐๐ (@oinori1197) August 25, 2026
The post observes that many journals prohibit the use of AI in peer review while nearly everyone now uses AI when reading papers, and asks whether it makes sense to evaluate work on the pretense that AI was not involved. The gap between stated rules and practice is exactly the seam that a provenance requirement would try to close.
A widely circulated post from a well-known neuroscientist took the thought experiment further, though with an important caveat: the text was written by a generative AI at his prompting, not by him.
ใ็ฌๆ or ๆฆๆ !? ๆชๆฅใฎ็งๅญฆ่ซๆใฎๅงฟใAIใ่ซๆใๆธใใAIใ่ซๆใ่ชญใๆไปฃใซใชใฃใใใๅญฆ่ก่ซๆใฏใฉใๅคใใใฎใใ็ๆAIใซๆฏ่่ชฟใซ่งฃ่ชฌใใใ็ตๆใใพใจใใพใใ
— ๆฑ ่ฐท่ฃไบ (@yuji_ikegaya) August 22, 2026
โโโโโโโโโโโโโโ
่ฆๆจใปๅบ่ซใป่ๅฏใฎๆถๆป
ๅ จๆใไธๅบฆใซๅฆ็ใงใใชใไฝๆง่ฝใช่ณใฎใใใฎ่ช็ฅ่ฃๅฉๅ ท
่ซๆใฟใคใใซใฎๅปๆญขโฆ
The AI-authored piece imagines what happens to the academic paper once AI writes papers and AI reads them, predicting the disappearance of abstracts, introductions and discussion sections, and even of titles. It is a provocation rather than an argument, but its reach suggests how readily the premise now lands.
Taken together, the Japanese conversation converges on the same question this article ends on. The debate is not whether AI belongs in science. It is who verifies the output and who is accountable for it, which is why a story about a machine catching a hundred-year-old measurement error passed largely unnoticed while publisher policy did not.
What Broke Was the Habit of Not Checking
This is not a case of AI overturning scientific consensus. What it unsettled is a set of values in one reference database, and the everyday assumption that such values need not be questioned. Foundational results are used repeatedly by later work and are tested in the process; they are not the kind of thing a prediction model topples one after another.
The direction it points is still worth taking seriously. Science has long leaned on practical markers of reliability, including venue, track record and peer review, to manage a body of knowledge too large to verify by hand. In a period when AI both finds errors and produces them, those markers are no longer sufficient on their own, and what gains weight instead is whether the data and the reasoning can be walked back through, which is to say whether a claim can be re-checked.
The framing of AI against scientists is turning into something else: AI and scientists together, facing the accumulated record of everything humans have written down. Humans grading machine output is no longer the whole relationship, because the machines are now grading the human accumulation back.
Trusting neither grade on its own, and keeping records that let both be examined, is the practical version of that. The next time a reference value disagrees with a prediction, the value of this technology will depend on whether there is any way to open the disagreement and see inside.
Frequently Asked Questions
Did an AI agent search the literature and find the errors in the chemistry database?
No. Nature’s headline refers to AI agents, but the publication’s own lead paragraph describes an artificial-intelligence model, and the reported sequence is that a machine-learning model built to predict molecular boiling points produced values that disagreed with the reference database. The audit was a by-product of predictive modeling, and a human researcher confirmed which side was wrong by returning to the original literature.
How many fake citations has AI put into scientific papers?
There is no settled figure, and the two largest audits differ by more than thirtyfold. A May 2026 preprint by Zhao and colleagues that audited 2.5 million papers and 111 million references reported about 146,900 non-existent references in 2025 alone, described as a conservative estimate. A Columbia University study published in The Lancet in May 2026, covering more than 2 million papers and 97 million references, identified about 4,000 fabricated citations across 2,800 papers. The definitions and detection criteria differ, so the numbers are not directly comparable.
How did Japanese social media react to AI finding errors in scientific databases?
Barely at all, at least to this case. A survey of Japanese-language posts on X on August 30, 2026 found no posts referencing the Nature report on the boiling-point errors. Japanese researchers and specialists were instead debating Elsevier’s updated policy on generative AI in manuscript preparation, the mismatch between journals banning AI in peer review and the reality of how papers are read, and the shift of research work from execution toward evaluation and responsibility. The shared thread is accountability rather than permission.
Sekahan on YouTube
We publish video summaries of articles like this one, along with short clips built around Japanese reactions.
Reference Links
- AI agents are checking the scientific literature โ and spotting decades-old errors๏ฝNature
- Daily briefing: AI agents sniff out decades-old errors in scientific literature๏ฝNature
- Provenance grounds trust in autonomous science๏ฝNature Computational Science
- Responsible and transparent use of AI in scientific publishing๏ฝNature Computational Science
- Corrected Values for Boiling Points and Enthalpies of Vaporization of Elements in Handbooks๏ฝJournal of Chemical & Engineering Data
- LLM hallucinations in the wild: Large-scale evidence from non-existent citations๏ฝarXiv
- When AI Co-Scientists Fail: SPOT โ a Benchmark for Automated Verification of Scientific Research๏ฝarXiv
- Science Done on a Machine by a Machine: AI Agents in Computational Chemistry๏ฝarXiv
