The authors report an automated workflow that generates ideas, edits code, runs experiments, writes papers, and evaluates them with an automated reviewer.
Discovery systemAutomated machine-learning researchreviewed 2026-07-16
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
Certiv evidence level
Artifact inventoriedcentral claim independently reproduced?
No.
That answer stays “no” even when a selected check passed.
claim boundary
Three statements that must not be collapsed.
The exact v3 PDF and public repository revision were pinned. The paper's headline acceptance language is explicitly based on an automated reviewer's threshold.
The generated papers were not shown by this dossier to be correct, novel, reproducible, or accepted by a real conference review process. The full workflow was not replayed.
source identity
The paper is versioned bytes.
A mutable title or unversioned link is not enough to reproduce what was reviewed.
- arXiv version
- 2408.06292v3
- Submitted
- 2024-08-12
- Version date
- 2024-09-01
- Size
- 11,731,143 bytes · 186 pages
- PDF SHA-256
- e911f90b0114d5e3fc23a0000177ea670fa9ed79d5f9a664efa2980fdb350507
Observed parser properties
- Recognized as a PDF and not encrypted.
- pdfinfo reported JavaScript: no.
- pdfdetach reported no embedded files.
- pdftotext completed successfully.
These probes reduce ambiguity and obvious PDF attack surface. They are not a comprehensive malware analysis.
Certiv T0 exercise: this exact PDF was hash-pinned without executing its declared command. The unchanged input remained explicitly unverifiable; a one-byte tamper produced a failed input_hash_drift finding.
test record
Checks, failures, and blockers.
Status is always paired with text. A pass applies only to the check and boundary shown in the same row.
Observed
Exact PDF identity
Observed
The downloaded v3 PDF matched the recorded SHA-256, byte count, and page count.
Boundary
This establishes file identity only.
Observed
Public code inventory
Observed
The official repository was pinned at commit 1de1dbc1f4ee2c5f61e9c94348d55eb51d7fa2eb.
Boundary
The repository still requires models, API credentials, GPU resources, and external services.
Not run
End-to-end autonomous research run
Observed
No model-funded, GPU-backed workflow was executed.
Boundary
No generated paper or cost claim was reproduced.
external artifacts
Pinned, not trusted by default.
Official AI Scientist repository
Observed revision1de1dbc1f4ee2c5f61e9c94348d55eb51d7fa2eb
Open code still depends on third-party models, APIs, external services, and substantial compute.
red-team findings
Where a confident summary can outrun the evidence.
- Passing an automated review threshold is not the same event as acceptance by human peer review.
- A generated manuscript is an output artifact, not proof that its scientific statements are true.
- Cost and autonomy claims depend on model versions, service behavior, templates, and external resources that can drift.
primary sources
Follow the evidence outward.
Dossier certiv-ai-discovery-2026-07-16/ai-scientist-v1. Certiv produced this test record; the paper authors did not issue or endorse it.