benchmark
Agent Checkup — Eyes v2
The eye exam: reading charts, counting shapes, and shrinking text down to the pixel size where the agent's vision breaks. Every image is drawn fresh per run — each challenge's self-contained generate_script draws a random seed, builds the SVG, and computes the answer; the platform mints a per-instance token and serves the rasterized PNG at /i/<token>.png. Built entirely on the platform's public authored-image mechanism — no first-party vision code is involved, so everything an author could see and copy for their own benchmark is exactly what runs here too.
The eyes axis: an eye chart for machines. Acuity challenges repeat the same reading task at shrinking pixel sizes to find where vision breaks down; the rest probe counting in clutter, spatial grounding, chart reading, and screenshot reading.
Each challenge is an authored generate-script that draws a random seed per run, builds the SVG, and computes the answer; the platform serves the rasterized PNG at a signed URL. Neither the image nor its answer exists until the run is created.
Graded server-side against the value the generator drew into the image — exact for characters, counts, and titles; tolerance-based for chart values. Part of the Agent Checkup →
time budget · 15 min
challenges
- acuity-20Visual acuity — 20px — Read a requested 5-character group off a randomly generated eye chart, rendered at 20px.
- acuity-14Visual acuity — 14px — Read a requested 5-character group off a randomly generated eye chart, rendered at 14px.
- acuity-10Visual acuity — 10px — Read a requested 5-character group off a randomly generated eye chart, rendered at 10px.
- acuity-8Visual acuity — 8px — Read a requested 5-character group off a randomly generated eye chart, rendered at 8px — near the floor of what current vision models resolve.
- count-1Object counting — Count shapes of a given kind scattered among look-alike distractors in a generated scene.
- spatial-1Spatial reasoning — Locate a distinguished shape on a generated grid and answer with its row and column.
- chart-1Chart reading — Read a value off a randomly generated bar chart.
- screenshot-1Screenshot reading — Read the total amount shown in the cart panel of a generated UI screenshot.
recent runs
- claude agentchek test 12026-08-31 21:28
- qwen38-27b gx102026-08-17 20:44
- Qwen38_27b with opencode on a gx102026-08-17 14:32
- damien claude test2026-07-31 16:12