A window can look right in a floor plan and still show up at the wrong width on an elevation. In a sample task for a new AI benchmark, an agent has to compare five floor plans with sections and elevations, flag discrepancies and cases it cannot resolve, and back up its findings with crops from the drawings. The finished product is a slide deck an architect could inspect.
Google DeepMind and Artificial Analysis introduced the Multimodal Knowledge Agent benchmark on November 10. The scored set contains 200 private tasks across 10 industries. Artificial Analysis runs the agents in the same software environment, with up to 250 turns per task. An AI grader checks their submitted work against criteria written by domain experts. The drawing audit is among the separate public examples that teams can inspect.
The headline score averages the share of weighted points earned on the harder criteria in each task. In Google DeepMind’s technical report, Claude Fable 5.2 scored 56.56% and Gemini 4 Carbon scored 54.34% on that measure. The 2.22-percentage-point gap was within the study’s statistical uncertainty. Those percentages describe points earned across the test. They cannot tell a project manager how often an agent will miss an incorrectly sized window.
The report put the average list-price API cost at $35.28 per task for Fable and $5.63 for Carbon, a ratio of about 6.3 to 1 in those runs. Artificial Analysis’s own launch results gave different averages, $31.97 and $6.29, or about 5.1 to 1. For a team budgeting a trial, either comparison leaves a question open. How much staff time will it take to check the findings and catch omissions?
A firm could try the models on a drawing set its staff has already checked. It could count missed discrepancies separately from false alarms, track how long a reviewer spends tracing each flag back to the drawings, and add that time to the cost of the run. The public sample offers one detail worth borrowing. Give the agent a way to say that the drawings leave a question unresolved, then ask for the evidence a person would need to make the call.