Virology Capabilities Test (VCT): A Multimodal Virology Q&A Benchmark
Jasper Götting, Pedro Medeiros, Jon G Sanders, Nathaniel Li, Long Phan, Karam Elabd, … (+3 more) — 2025-04-29 — Multiple Research Institutions — arXiv
Summary
Creates VCT, a 322-question benchmark testing LLMs’ ability to troubleshoot complex virology laboratory protocols, constructed with PhD-level expert virologists and covering dual-use biological knowledge.
Key Result
OpenAI’s o3 achieves 43.8% accuracy on VCT, outperforming 94% of expert virologists even within their specialized sub-areas, demonstrating superhuman performance on dual-use virology troubleshooting.
Source
- Link: https://arxiv.org/abs/2504.16137
- Listed in the Shallow Review of Technical AI Safety 2025 under 1 agenda(s):
- Editorial blurb (verbatim):
[Virology Capabilities Test (VCT): A Multimodal Virology Q\&A Benchmark](https://arxiv.org/abs/2504.16137)