Auditing an AI-generated citation means answering four separate questions about every reference, not one. Does the cited work exist? Does its text actually support the sentence that cites it? Is every quoted span intact? Has the work been retracted or superseded? Only the first of those is about whether the citation is real — and it is not the failure mode that most often survives a quick check.
This guide is for writers, editors, researchers, and practitioners publishing AI-assisted drafts, whether the model ran in a hosted service or on local hardware. It sets out a manual procedure you can run today, explains which parts are worth automating, and marks the places where a plausible citation passes every mechanical test and still misleads the reader.
Four failure classes, not one
A practitioner guide published by HyperResearch splits citation failures into four classes, each with a different detection method and a different cost: How to verify the citations in an AI-generated report. The classification matters because the failure everyone worries about — the invented reference — is the easiest one to catch and the least likely to reach a reader unnoticed.
- Class 1: the fabricated reference. The journal, volume, page range, and author names are all plausible, and nothing about the reference exists. The tell is mechanical: the DOI does not resolve, or it resolves to an unrelated paper, and the title in quotation marks returns nothing.
- Class 2: the real reference that does not support the sentence. The source exists, is correctly attributed, and says something adjacent to the claim — but not the claim. The guide’s illustration is a document asserting that “retrofit grants cut household energy use by a fifth within two years,” citing a real paper that reports a modelled savings potential of about a fifth over a ten-year horizon in one city. The number matches. The unit matches. The modality and the time horizon do not. That example is explicitly invented by the guide to show the shape of the failure; no real paper is involved.
- Class 3: the altered quotation. A quoted span that has been trimmed, reordered, or lightly rewritten. This is detected by string matching against the fetched source body.
- Class 4: the retracted or superseded work. The citation was accurate when it was written and is no longer safe to rely on.
Classes 1 and 4 are mechanical and cheap to check. Class 3 is string matching. Class 2 requires someone to read the source, and according to the same guide it is the class that fails most often on documents that otherwise look fine.
Why hallucinated citations are convincing
Northwestern University’s research guide on evaluating AI-generated content notes that AI output may include inaccurate information ranging from misinformation and missing information to fabricated material — “hallucinations” — including false citations and sources that do not exist. It also documents a case in which Google Gemini produced incorrect names and dates even while citing a Wikipedia article, which is the useful lesson: the presence of a source link does not certify the sentence attached to it.
A library instruction handout from National University lists the same problem from the cataloguing side: hallucinated citations are sometimes “published” in reputable, peer-reviewed journals, often “written” by authoritative authors in the field, and occasionally given an authentic-looking DOI (Hallucination Check: Verifying Sources in the Age of AI). Journal prestige, author reputation, and the mere presence of a DOI are all weak signals, because all three can be generated.
Before you start
You need the complete draft with both in-text citations and a reference list, because the two frequently disagree. You also need a general search engine, Google Scholar, a library discovery tool if you have institutional access, the DOI resolver at doi.org, and Crossref. Most importantly, you need access to the cited source itself — not just its abstract — for the claim-support check in step two. A spreadsheet with one row per citation and a column for each of the four checks keeps the audit reproducible.
Step 1: Confirm the work exists
- Search the exact title in quotation marks in Google Scholar and in a general search engine. The National University handout recommends this as the first move.
- Search the same quoted title in your library’s discovery tool.
- Look up the journal separately and verify the volume and issue number against the citation.
- Paste the DOI into doi.org. A DOI that fails to resolve, or that resolves to an unrelated paper, is a mechanical failure signal.
- Search Crossref for the title as a cross-check on the DOI.
There are three outcomes. The record resolves and matches the citation: proceed to step two. The identifier resolves to something else: treat the citation as fabricated until proven otherwise. Nothing matches anywhere: treat it as fabricated. A title that returns no results at all is a strong signal, because real titles are indexed.
Step 2: Read the source and test the claim
This is the step that catches the failures that matter, and it is the one that cannot be shortcut. Open the source and find the sentence that supports the claim. If you cannot find one, the citation fails no matter how real the reference is. Ask specifically:
- Modality. Does the source report a measured outcome, or a modelled potential, an estimate, or a projection? These are not interchangeable, and substituting one for another is the classic class-2 failure.
- Time horizon. Does the period in the source match the period in the sentence?
- Scope. Does the population, geography, or sample match? A single-city result is not a national result.
- Direction and units. Does the number match in sign, magnitude, and unit?
Write down the exact source sentence you are relying on. If the source is paywalled and the abstract does not contain the specific figure, record the claim as unverified rather than verified. An inaccessible source is not a supporting source.
Step 3: Check quoted spans
For every direct quotation, copy the quoted text and search for it inside the fetched source body. Look for dropped words, changed tense, removed qualifiers, and ellipses that change meaning. Check that the quote is attributed to the right author and section. This is a string-matching task, which makes it one of the few parts of the audit that automates cleanly.
Step 4: Screen for retraction and supersession
Open the DOI landing page and read the record, not just the abstract. Look for a retraction notice, an expression of concern, or a note that the work has been superseded by a later version. This check is cheap and belongs in the same pass as step one, because both operate on metadata rather than on the full text.
Triage order for long reference lists
Run the cheap checks on everything and the expensive check on what matters. In practice that means:
- Existence and retraction checks on every citation.
- Quote-integrity checks on every direct quotation.
- Claim-support checks on every citation that carries a load-bearing number, comparison, or causal claim — and on nothing else, if time is short.
This ordering follows the cost structure the HyperResearch guide describes: the mechanical checks first, string matching next, and the reading step last, concentrated where a failure would change the reader’s conclusion.
What automation can and cannot certify
Automated tooling handles DOI resolution, title matching, retraction flags, and exact string matching for quotations. It does not handle the question of whether the source supports the sentence. The HyperResearch guide is direct about this: metadata checks pass class-2 citations with full marks, and the reading step is the one that fails most often on documents that look fine.
The practical consequence is that a citation checker reporting “all references valid” is answering a narrower question than most readers assume. Treat that output as step one and step four of the audit, and not as the audit.
Where the audit belongs in a publishing workflow
The audit is a gate, not a cleanup task. Place it between draft generation and publication, and record a verdict per citation so the decision is reproducible and reviewable. For WordPress and other automated publishing pipelines, that means keeping the generated draft and the audit record as separate artefacts and requiring a human sign-off on the claim-support column before anything goes live. Automating the mechanical checks is worthwhile; automating the publication decision is not, because the failure class that matters most is the one the automation cannot see.
The same procedure applies whether the draft came from a hosted model or a local one. The sources used here do not distinguish between deployment types, and there is no reason to expect the verification burden to change: a citation is either supported by its source or it is not.
Limitations
Full-text access is the bottleneck. Without it, step two cannot be completed for paywalled sources, and the honest outcome is a larger “unverified” pile rather than a clean bill of health. The four-class framework comes from a practitioner guide rather than a peer-reviewed study, so treat the classification as a working checklist rather than a validated taxonomy. And an audit reduces risk; it does not eliminate it. A citation can pass all four checks and still be part of a weak argument.
A useful next action: take the three highest-stakes citations in your current draft, run all four checks on them, and record the verdicts in a table you can hand to an editor. If you hit a case where a real source almost — but not quite — supports the sentence, that is the most useful kind of example to bring back to the community, because it is the failure mode most readers never learn to look for.
FAQ
Can a citation be real and still be wrong?
Yes. The failure class that most often survives a metadata check is a genuine, correctly attributed source whose text does not support the sentence citing it — for example, a modelled ten-year savings potential cited as a measured two-year outcome.
Does a working DOI prove the citation is accurate?
No. A DOI that resolves only confirms the identifier points to a record. It says nothing about whether that record supports the claim, whether a quotation was altered, or whether the work has since been retracted.
What should I do when a citation title returns no search results?
Treat it as fabricated until proven otherwise. Search the exact title in quotation marks in Google Scholar and a general search engine, check the journal, volume, and issue number separately, and test the DOI — a title that returns nothing anywhere is a strong mechanical signal.