shared learnings
Research & Blog
September 2026 The Difficulty of Creating Hard, Fair, and Valid Benchmarks Breaking a frontier model is no longer the hard part. Finding a failure that is hard, fair and valid at the same time is. 
→ August 2026 Plot Twist Bench Do frontier models actually understand video? Plot Twist Bench is a video perception benchmark of 48 curated multiple-choice questions, made by Vetto. 
→ August 2026 ProgramBench Vetted Reverse engineering from a runnable binary, rebuilt with explicit verification controls 
→ August 2026 GulliBench GulliBench: Intelligence Isn't Enough. Measuring Skepticism in Frontier Models. 
→ August 2026 Terminal Tasks v1.0 Introducing Computer Anthology 
→
Article
→ August 2026 Plot Twist Bench Do frontier models actually understand video? Plot Twist Bench is a video perception benchmark of 48 curated multiple-choice questions, made by Vetto. Benchmark
→ August 2026 ProgramBench Vetted Reverse engineering from a runnable binary, rebuilt with explicit verification controls Benchmark
→ August 2026 GulliBench GulliBench: Intelligence Isn't Enough. Measuring Skepticism in Frontier Models. Benchmark
→ August 2026 Terminal Tasks v1.0 Introducing Computer Anthology Benchmark
→