# How to check where an AI answer came from > A five-step check for any AI answer, worked through on real public records, and what good provenance looks like when you need to rely on a fact. - URL: https://ikag.dev/blog/how-to-check-ai-answers - Author: Ronit Ahuja, Founder, IKAG (https://ikag.dev/authors/ronit-ahuja) - Published: 2026-10-10 - Updated: 2026-10-10 - Reading time: 4 min - Publisher: IKAG ## Overview A five-step check for any AI answer, worked through on real public records, and what good provenance looks like when you need to rely on a fact. ## Key points - An AI answer is a claim. A link under it is not proof that a source supports it. - In a March 2025 test by the Tow Center at Columbia Journalism Review, eight AI search tools asked to identify the source of quoted news excerpts got more than 60 percent of those queries wrong. That is one task, not an overall accuracy rate. - To check an answer: open the source, find the claim, prefer the primary source, check the date, look for agreement, and notice what the sources do not say. - Good provenance names the record behind each value, when that record was last updated, and what could not be established. AI assistants are very good at sounding sure. That is not the same as being right, and a link under an answer is not the same as a source that supports it. ## Confident is not the same as correct In March 2025, researchers at the Tow Center for Digital Journalism at Columbia Journalism Review tested eight AI search tools on one task: given a direct excerpt from a news article, identify its headline, original publisher, publication date and web address. Across 1,600 queries, in their words, "collectively, they provided incorrect answers to more than 60 percent of queries." They also found tools citing syndicated copies instead of the original articles, and fabricating links. That figure describes one demanding task with the tools of early 2025. It is not a general accuracy rate for AI, and tools change. The lesson that lasts is simpler: an answer is a claim, and a claim is only as good as the source you can check it against. ## A five-step check for any AI answer 1. **Open the source.** Does the link work, and does it lead to the page the answer names? 2. **Find the claim.** Does the page actually say what the answer says? Search the page for the key fact or number. 3. **Prefer the primary source.** A company's registered name comes from a company registry; a quote comes from the publisher; a train time comes from the operator. A summary of a summary is weaker. 4. **Check the date.** Is the source recent enough for the question? Opening hours, prices and news change. Note when the source says the fact was last updated, not only when you looked. 5. **Look for agreement, and for gaps.** Do two independent sources say the same thing? If they disagree, the answer should say so. And notice what the sources do not say: a gap is information too. ## A worked check, on real records Take a simple question: which legal entity is behind Apple, and where is it incorporated? An assistant might answer "Apple Inc., incorporated in California." Here is the check, on public records that anyone can open. 1. **Open the source.** The US Securities and Exchange Commission keeps a record for every company that files with it. Apple's is under Central Index Key (CIK) 0000320193 on [SEC EDGAR](https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0000320193). 2. **Find the claim.** The record names "Apple Inc." and gives California as its state of incorporation. The answer holds up. 3. **Prefer the primary source.** The SEC record is a primary source for a US filer. [Wikidata's item Q312](https://www.wikidata.org/wiki/Q312) gives the same name, which is useful agreement, but it is community-edited, so it supports the answer rather than proving it. 4. **Check the date.** The SEC record does not say when its company facts were last confirmed, so you cannot call them current; you can only say they are what the record shows. The global LEI register is better here: [Apple's LEI record](https://api.gleif.org/api/v1/lei-records/HWUPKR0MPOU8FGXBT394) states it was last updated on 3 March 2026. 5. **Look for agreement, and for gaps.** The LEI record also names Apple Inc. in California, under a California registration number, but it does not list Apple's SEC CIK, and the SEC record does not list an LEI. Both describe "Apple Inc.", so it is very likely the same company, but no record in this check links the two identifiers. The honest summary is: name and place confirmed by a primary record; the link between the LEI and the CIK not established by these records. That last step is the one most answers skip. A careful answer says what it could not confirm. ## What good provenance looks like Provenance is the record of where an answer came from. The difference between weak and strong provenance is easy to see once you know what to look for: | Aspect | Weak provenance | Strong provenance | | ------------ | ---------------------------------- | -------------------------------------------------------- | | Sources | A few links under the whole answer | The record behind each value | | Freshness | Unknown | When the source last updated each value, if it says | | Disagreement | Hidden | Conflicts shown, and which source was preferred, and why | | Gaps | Filled in with a guess | Named as unknown, with the reason | | Coverage | Unclear what was checked | Sources consulted, used and skipped | When an answer comes back like this, a person can check it in seconds, and an agent can weigh sources instead of picking one at random. Even then, a record can be wrong or out of date: provenance tells you what to check, not that the answer is true. ## How IKAG approaches it IKAG is opening to early users and data partners first. Its first capability, company legal identity, is designed to answer the way the worked check above does: each value tied to the primary record that states it, Wikidata used only as supporting evidence, freshness taken from the source's own dates, and no link between an LEI and a CIK unless a primary record names both. You can inspect real output from it on the [developer page](https://ikag.dev/agents#runs). ## Questions ### Is a cited answer always right? No. A citation shows where an answer says it came from. You still need to check that the source exists, says what the answer claims, and is current enough for your question. ### What did the Tow Center study measure? In March 2025 it gave eight AI search tools direct excerpts from news articles and asked for each article’s headline, publisher, date and web address. Collectively they answered more than 60 percent of the 1,600 queries incorrectly. It measured that task only, not AI accuracy in general. ### What is a primary source? The organisation that creates or officially records the fact: a company registry for a company’s legal name, the publisher for an article, the operator for a timetable. ### If two records name the same company, are they the same company? Not necessarily. A shared name is a lead, not proof. Look for a record that names both identifiers, or treat the link as unconfirmed. ## Sources - [Columbia Journalism Review, Tow Center: AI Search Has a Citation Problem (6 March 2025)](https://www.cjr.org/tow_center/we-compared-eight-ai-search-engines-theyre-all-bad-at-citing-news.php) - [SEC EDGAR: company filings for CIK 0000320193](https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=0000320193) - [GLEIF API: LEI record HWUPKR0MPOU8FGXBT394](https://api.gleif.org/api/v1/lei-records/HWUPKR0MPOU8FGXBT394) - [Wikidata: item Q312](https://www.wikidata.org/wiki/Q312)