Checking claims and sources an AI draft gives you

Two separate questions

A claim is a statement that something is true. A source is where the claim comes from. An AI draft can fail in two different ways, and you need to check both.

• Is the source real?
• Does it say what the draft says?

Fabricated citations are a known kind of error in chatbot text¹. A source can be wholly invented. A real source can also be described wrongly. The second problem is harder to spot, because the title checks out.

Is the source real?

Search for the exact title and the publisher's name. Don't paste in the link the draft gave you and assume it's right. Go to the publisher's own website and look there.

• A web page. Open the address. If it fails, search the publisher's site for the title. A missing page is a warning, not proof, since pages move.
• A report or study. Look for it on the publisher's site, a library catalogue or the author's page.
• A law or rule. Find it on the official government site for the country it applies to. Laws differ by country and by state, so note which one the draft means.
• A quote from a person. Look for the original interview, post or document. A quote with no traceable origin shouldn't go out under someone's name.
• A statistic. Find the table or release it came from.

If you can't find it after a fair search, remove the claim or rewrite it without the source.

Does it say what the draft says?

Open the source and find the passage. Then compare closely.

• Does the source give the same figure, with the same conditions?
• Has "often" or "can" turned into "always"?
• Is the source about the same country, year and kind of business?
• Does the source make a comparison the draft makes, such as "more than" or "the most"?

An original, or primary, source can be a person with direct knowledge of a situation, or a document written by such a person². Prefer that to a page that summarises another page. Summaries pass along mistakes.

Tools also often repeat common misconceptions¹. A claim that "everyone knows" something isn't a source. Treat it as one more thing to look up.

Claims with no source

Many draft sentences carry no source at all. Ask of each: would a customer, regulator or competitor be able to challenge this? If yes, you need something to back it. A publisher that does it in-house is doing internal fact-checking to prevent inaccurate content going out³. Your business is the publisher of what it sends.

Claims to be careful with

• Health, safety and environmental claims. Keep these to what a recognised body has said, in its own words.
• Legal, tax and privacy statements. Treat the draft as general information at best. Check the official source for your country, such as the ATO, IRS, HMRC, ICO or OAIC, or ask an accountant or lawyer.
• Claims about competitors. Remove anything you can't support in writing.
• "Studies show" and "experts say". Delete these unless you can name the study or the expert and find it.

Exceptions

A claim about your own business, such as your opening hours, doesn't need an outside source. It needs checking against your own records. And when a draft has no outside claims at all, there's less to verify.