How to check an AI explanation of a research paper

1. List the checkable claims

Separate interpretation from facts. Highlight every number, named method, quotation, cause-and-effect statement, and claim about what the authors concluded. Turn each into a short question: “Was the sample 240 people?”, “Was the change 12 percent or 12 percentage points?”, or “Did the authors say X caused Y?” Check the consequential claims first.

2. Audit numbers, units, and the sample

Find each important number in the paper’s table, figure, results, or methods section. Put the PDF value and the AI’s version side by side. Check:

• the denominator and sample size, overall and per group;
• units, time period, and measurement scale;
• whether the value is a count, mean, median, percentage, rate, ratio, or model estimate;
• uncertainty, such as a confidence interval or error bar; and
• whether exclusions, missing data, or subgroup analysis changed the sample.

Do not repair a mismatch by guessing. Mark it unresolved and note the page, table, or figure. Sample size alone does not establish quality: representativeness, measurement, design, bias, and statistical power also matter.

3. Test causal language

Association alone does not establish causation. Another variable, selection process, or reverse direction may explain a relationship. Check the study design, plausible confounding, and the identification assumptions used for a causal interpretation. Do not trust “caused,” “led to,” or “improved” merely because the authors—or the AI—use those verbs. Use “was associated with” unless you can explain why the design and assumptions support more.

As an introductory distinction, Penn State contrasts random assignment with observational comparisons that may be confounded, and distinguishes random assignment from random sampling.¹ Observational methods may estimate causal effects under additional assumptions. Harvard epidemiologist Miguel Hernán’s work, for example, combines observational data, assumptions, and statistical methods to emulate randomized experiments.⁴ Identify and evaluate those assumptions rather than applying a slogan.

4. Trace quotations and citations

Search the PDF for a distinctive phrase from every quotation. Confirm the exact words, speaker or author, nearby qualifications, and page. For a paraphrase, read the surrounding paragraph. If the answer names another study, follow the in-text citation to the reference list, then open that original source when the claim matters. A citation’s existence does not prove that it supports the sentence.

If you cannot locate a quotation, page, or source, label it “not verified”; never invent a page number. USC Libraries recommends tracing claims and quotations to their original context.²

5. Restore limitations and scope

Compare the answer with the limitations and discussion. Add constraints it omitted: population, setting, dates, selection rules, follow-up length, self-reporting, measurement limits, confounding, uncertainty, and conflicts of interest. Ask: “To whom, where, and for how long could this result plausibly apply?” Do not turn “no statistically detected difference” into “the groups are identical.”

6. Check document coverage

An answer can sound complete while covering only prominent passages. Make a section map: abstract, introduction, methods, results, discussion, limitations, tables and figures, appendices, and references. Mark each “covered accurately,” “omitted,” or “not relevant.” USC’s scholarly-article guide recommends checking how authors found their results, whether the analysis agrees with the data, whether all data are present, and what strengths and weaknesses remain.³

Finish with a verification record

Then write your own explanation from those checked notes. If the answer missed a section, ask a narrowly scoped follow-up—and verify that response against the PDF too.