How to trace a wrong AI answer to its source

There are three possible findings. In a source error, a page states the claim and the answer repeated it. In an unsupported statement, the answer cites pages that don’t say it. Source unknown means you searched and found no source. Our method defines each one in full.

Why a citation doesn’t settle it

A link beside a sentence doesn’t prove the sentence came from that page. In a 2023 audit of four AI search engines, Stanford researchers found only 51.5% of generated sentences fully supported by their citations. Only 74.5% of citations supported the sentence they were attached to.

The Tow Center for Digital Journalism published a test of eight AI search tools in March 2025. Asked to identify the source of quotes from news articles, the tools gave incorrect answers to more than 60 percent of queries. The researchers also found that the tools “fabricated links and cited syndicated and copied versions of articles.”

So open every cited page and read it.

Step 1: Capture the answer

Record each step in the claims-and-evidence worksheet as you go. First, read the AI product’s terms of use, because some limit recording answers, even by hand. Perplexity’s terms (section 5.2) say users may not access the service:

to monitor, extract, copy or collect information or data from or through the Services, or engage in any manual process to do the same

Then take a screenshot that shows the date, and note:

• the AI product and mode, such as a chat assistant with web search on;
• whether you were signed in, and your country;
• the exact prompt;
• the claim, word for word;
• every link the answer showed.

Ask the same question two or three more times. Answers vary between runs.

Step 2: Read every cited page

Open each link the answer showed and look for the claim. Record one of these results for each page:

What the page shows	What it means
The claim, in similar words	Likely a source error
Something different, often the correct fact	It doesn’t support the claim
Nothing: it won’t load, or it’s another page	A broken or mismatched citation

Save a dated copy of any page that supports the claim, because pages change. If no cited page says what the answer says, you have an unsupported statement.

Step 3: Search for the claim’s wording

The answer may draw on pages it doesn’t show. Google says AI Overviews and AI Mode may issue “multiple related searches across subtopics and data sources” to develop a response. Other products work differently, and most don’t publish the details.

Search for the claim in quotation marks, then without them. Check these places:

• your own site, including old pages and PDFs;
• directory and map listings with your hours, address or phone number;
• review sites, where reviewers sometimes state wrong facts;
• news articles and press releases, old ones included;
• Wikipedia and Wikidata, if your business has an entry;
• businesses with a similar name, which an answer can merge with yours.

For a page that’s gone, try the Internet Archive’s Wayback Machine. Log every page you check, including those without the claim. That list is what makes a “source unknown” finding credible.

Step 4: Check whether the answer used the web

Some answers come from a live search. Others come only from what the model learned in training. An answer with no links, from a product with web search off, likely comes from training data. Its source may be a page that has since changed.

If you can, ask with web search on and off, and record both. A claim that appears only without search points to older training data. A claim that appears only with search points to a page the search finds now.

What to do with each finding

Finding	What to try next
Source error	Correct the page, or ask its owner to. Keep a dated record.
Unsupported statement	State the fact plainly on your site; use the product’s feedback tool.
Source unknown	Make the correct fact easy to find, then keep checking.

“Source unknown” is a legitimate finding. Researchers at OpenAI and Georgia Tech argue that models “sometimes guess when uncertain, producing plausible yet incorrect statements,” because training and evaluation reward guessing. Opening hours and founding dates are the kind of specific fact a model may guess.

What a change can and can’t show

If the claim disappears after your change, note the date, but don’t treat it as proof. The product may have updated its model or search index in the same weeks, and answers vary between runs. A claim that persists after a correct fix may need more time, or it may come from a page you haven’t found.