RepliBench: measuring autonomous replication capabilities in AI systems
2025-04-22 — UK AI Security Institute — UK AISI Blog
Summary
Introduces RepliBench, a comprehensive benchmark with 20 novel LLM agent evaluations comprising 65 individual tasks designed to measure autonomous replication capabilities in AI systems across four key domains: obtaining weights, replicating onto compute, obtaining resources, and persistence.
Key Result
Seven frontier models tested show they are not yet capable of full autonomous replication, with the best model achieving >50% pass@10 on 15/20 task families, but struggling with KYC checks, robust agent deployments, and realistic weight exfiltration defenses.
Source
- Link: https://aisi.gov.uk/work/replibench-measuring-autonomous-replication-capabilities-in-ai-systems
- Listed in the Shallow Review of Technical AI Safety 2025 under 1 agenda(s):
- self-replication-evals — Evals