Don't Quit Your Day Job

What we learned

These six points are the whole study in short form. The tables farther down are the same evidence, with every count labeled.

1. 1. A clean set can look solved

   On a clean, born-digital drawing set, a general model can produce a plausible-looking inventory. Discovery is not the hard part.

2. 2. Trust is the gap

   The model cannot tell you which lines to believe. In the Claude chat run on Example Plans, six callout issues were flagged. Three were real and three were wrong. Nothing in the output separated them.

3. 3. The same file does not repeat

   Same model, Claude Fable 5.2, and the same Example Plans file. The chat run counted 1,255 callouts. The API run counted 1,077. One run was in the chat app and one was through the API. The API run was cut off at the output limit inside keynotes. The chat run needed one continue before it returned the final JSON.

4. 4. A text layer can still be missed

   Gemini 3.8 Thinking, in a paid chat, listed all 79 Example Plans sheets correctly, then said there was no sheet index and recorded zeros for callouts, door tags, door schedule rows, and keynotes. It said the page images were too small to read. The PDF has a text layer on every page. The SHEET LIST is on S-000.

5. 5. Some routes return nothing usable

   ChatGPT's free chat, with Think on, returned no response in the window we recorded, as of 11:17 AM ET on November 10, 2026. GPT-5.5 and Gemini 4 Argon via OpenRouter rejected the Example Plans file before the model saw it, because the file was over a 20 MB request limit.

6. 6. Agreement was real, and narrow

   On Ideatura, Claude, GPT-5.5, and Gemini 4 Argon agreed on the same 14 door numbers on the plans with no schedule row: 106-B, 107 to 111, 112-A, 112-B, 113, 114-A, 114-B, 115-A, 115-B, and 116. That is a real catch. It is not a review you can hand to the field without checking the rest of the list.

Discovery is not the gap. Trust is. A general model can sometimes find a real issue, especially on a clean set. The problem is that you cannot trust the result, and you cannot repeat it.

Why this research exists

Builders are being asked to stand up general-model workflows for drawing review. The tools are powerful. The question is whether that work belongs on a construction team's clock.

1. The tools are already in use

McKinsey's June 13, 2025 report puts that adoption gap in one sentence.

Nearly eight in ten companies report using gen AI, yet just as many report no significant bottom-line impact.

McKinsey, June 13, 2025, Seizing the agentic AI advantage.

2. More output is not a result you can use

In McKinsey's August 25, 2026 survey, 80% said AI improved individual productivity, 37% attributed any EBIT impact, and about 6% saw a material effect. There were 1,719 respondents in 97 countries, from May 4 to June 8, 2026, and the answers were self-reported. In the same survey, 32% said their organizations decided against buying at least one software product or feature because they could build it in-house with agentic coding tools.

MIT Sloan, on June 8, 2026, makes the matching point. AI makes it cheap to produce work, but not to judge whether that work is any good. Human review time becomes the constraint. McKinsey's October 14, 2026 explainer names cloud bills, software subscriptions, and the hours people spend checking outputs as real costs.

3. In construction, a gain you give away is not margin

McKinsey's August 9, 2024 article on construction productivity says productivity gains are often bid away in tenders rather than kept as margin. Our conclusion, which is ours and not McKinsey's wording: the capacity these tools free up is better spent on the core work of reading drawings, catching real conflicts, and building the building, than on standing up another general-model workflow that still has to be checked by hand.

What a chatbot does with a drawing

This is for a GC, a PM, or an estimator. You do not need a computer science background. Each term is defined the first time it appears. The point is practical: why a tiny tag gets lost, why a measurement is a guess, and why the same file can get a different answer.

Tokens, and the next token

A language model does not read words the way you do. It first breaks text into small pieces called tokens. A token can be a whole word, part of a word, a number, or a bit of punctuation. "Door" might be one token. "Callout" might be two. "A-501" might be three or four. That step is called tokenization.

Once the text is split, the model has one job: predict the next token. Not the next sentence. Not the answer. Just the next small piece. Then it adds that piece and predicts the next one, until it decides it is done. Every chatbot answer is built this way, one token at a time. That job is called next-token prediction.

When the model predicts the next token, it does not look up a stored fact. It produces a probability distribution, which is a ranked list of possible next tokens, each with a likelihood. It then picks from that list. So when a chatbot tells you a door number, a sheet reference, or a dimension, it is producing the words that seemed most likely to come next. Often that guess is right. Sometimes it is not. The model has no built-in way to tell you which is which.

Patches, and why a tiny tag gets lost

Language models were built for text. To use a picture, the image has to be turned into something like tokens. A vision encoder is the part of the system that converts a picture into numbers the language model can use.

The vision encoder starts by cutting the image into a grid of small squares called patches. A common size is 16 by 16 pixels, and 14 by 14 is also common. Each patch is one piece of the picture. People call these visual tokens.

A full drawing is large. Before the encoder sees it, the image is often resized down to a fixed size, or cut into tiles, because the model can only take in so many patches at once. Shrinking a large sheet means a door tag that was crisp on your screen may end up only a few pixels across.

Once the page is cut into patches, a small tag might take up only a fraction of one patch. It shares that square with linework, hatching, and a piece of a dimension string. Or the tag lands on a boundary and is split across two or four patches, with half the characters in one square and half in another. The model is not looking at the tag. It is looking at a summary of a square that happens to contain part of the tag. That is why tiny tags get lost.

Why a measurement is a guess

Each patch is turned into a vector, which is a long list of numbers meant to capture what that patch looks like and how it relates to the patches around it. Those numbers are learned. The encoder was trained on a very large collection of images, and it learned to produce numbers that were useful for that training data. The vectors are a compression of the picture, and the compression is lossy: some information is kept, and a lot is thrown away. What gets kept is whatever was useful for photos, screenshots, charts, and everyday images, not for stacks of permit sets.

The vision encoder is not doing a geometric reading of your sheet. It does not find the scale bar and compute a length. It does not trace a leader line from a callout to the detail it points at. It turns each patch into a pattern it has seen before: this looks like a dimension, this looks like text, this looks like a door swing. When a chatbot gives you a measurement off a drawing, it has not measured anything. It has seen marks that look like a dimension string and produced a number that seems likely to belong there.

Why this is different from a detector that repeats

Older computer vision systems were often built around convolutional neural networks, or CNNs. A CNN slides small filters across an image to pick out edges, shapes, and features. A CNN detector is trained for one job, such as drawing a box around every door tag and reading the number. Once it is trained, its settings, called weights, are fixed. The same pixels go through the same math, and in practice you get the same boxes and the same readings. It can still be wrong, but it is wrong in a repeatable way, which means you can find the error and fix it.

In a chatbot, the image goes to a vision encoder, usually a vision transformer, which is a model that works on the patches described above. That output is handed to a language model, which writes the answer one token at a time. The vision encoder's math on its own is not random. The same image produces the same vectors. What happens next is different. Those vectors become context for a generative language model, and that model returns a probability over possible next words, then picks one.

• A trained detector gives you a detection. It answers one question about the image, and it answers it the same way each time.
• A vision-language model gives you a reading. It looks at a compressed, patch-by-patch summary, and then a language model writes what it thinks is probably there.

Why the same file can get a different answer

When the model picks the next token, it does not always take the top choice. It samples, which means it picks from the list with some randomness so the answer does not sound robotic. Many systems have a setting called temperature that controls how much randomness goes into that pick. Lower temperature stays closer to the top choice. Higher temperature adds variety. Even with the randomness turned down, the output is still a prediction, and in practice providers do not generally promise the exact same answer every time. Most people using a chatbot app cannot see or set temperature at all.

A drawing review is not one prediction. It is a long chain of tokens. A small difference early on, such as one tag counted or skipped, changes the context for everything that follows. Training on more drawings, or writing a better prompt, can shift the probabilities. It does not stop the guessing. The answer is still a guess, just a more educated one.

That is what we saw on Example Plans. Same model, same file. The chat run counted 1,255 callouts. The API run counted 1,077. Nothing in the output tells you which number to build an RFI on. The full counts are in What we found.

The prompts we sent

These are the exact texts we pasted. They are not summaries. Open a block to read one, or download the file and paste it unchanged. The drawing PDFs and the Ideatura answer key are in Files to test.

You are auditing a construction drawing set (the attached PDF) for OBJECTIVE, DOCUMENT-INTERNAL CONSISTENCY ONLY.

Scope rules:
- Check only whether the drawing set agrees with itself. Do NOT comment on design quality, code compliance, constructability, quantities, takeoff, cost, or anything requiring outside knowledge.
- Do not invent sheets, details, schedule rows, keynotes, or legends. If something is not present in the document, say it is absent. Every finding must cite where in the set you saw it.
- Review every page of the attached PDF. If you could not read some pages or some text (e.g. too small), say so explicitly in "coverage_notes" rather than guessing.

Perform these checks:

1. SHEET INVENTORY. List every sheet: PDF page number (1-based), sheet number, sheet title.

2. SHEET INDEX vs SHEETS PRESENT. If the set contains a sheet index / drawing list, compare it to the sheets actually present. Report: sheets listed in the index but missing from the set ("Sheet is missing"), sheets present but not in the index ("Unused index entry" / not indexed), and duplicate sheet numbers ("duplicated"). If there is no sheet index, say so.

3. CALLOUTS / DETAIL AND SHEET REFERENCES. Find every callout bubble/tag that references another view: section marks, elevation marks, detail callouts, enlarged-plan / room-layout callouts, and any "see sheet X" references. Each typically shows a detail/view number and a target sheet number. For each one, check that the target sheet exists in this set and, where a detail number is given, that the target sheet actually contains a view with that number/title. Classify each callout as one of:
   - "verified" (target sheet exists and the referenced view/detail is on it)
   - "Reference Missing" (callout's target sheet or reference text is missing/blank/unreadable as drawn)
   - "Sheet is missing" (target sheet number does not exist in this set)
   - "Incorrect Ref. Detail" (target sheet exists but the detail/view number does not exist on it, or points to the wrong view)
   - "Unused" (a detail/view on a sheet that is never referenced by any callout, if you can detect it)
   - "duplicated" (two different views share the same number on the same sheet, or the same callout is duplicated)

4. DOOR TAGS vs DOOR SCHEDULE. Find every door tag (door mark/number shown on plans and room layouts) on every sheet, counting each occurrence on each sheet separately. Find the door schedule(s). Match each plan door tag to a schedule row by door number. Matching rule: case-insensitive, ignoring hyphens, spaces and periods (e.g. "106-A" matches "106A"); otherwise exact. Classify each tag occurrence as:
   - "verified" (matches a schedule row)
   - "Missing From Schedule" (tag appears on a plan but has no schedule row)
   - "duplicated" (the same door number appears in the schedule more than once, or one physical door is tagged with conflicting numbers)
   Also report schedule rows whose door number never appears on any plan ("schedule row unused / missing from plan").

5. OTHER TAGS vs LEGENDS/SCHEDULES. If the set contains keynote legends, finish/room/window/equipment schedules, partition/wall-type legends, or similar, check their tags the same way (tag on drawing ↔ entry in legend/schedule). If none exist, state that they are absent; do not invent them.

OUTPUT FORMAT. Return ONLY a single JSON object (no prose before or after, no markdown fences) with exactly this structure:
{
  "sheets": [ {"page": 1, "sheet_number": "", "title": ""} ],
  "sheet_index": {"present": true/false, "location": "", "entries": [""] },
  "schedules_and_legends_found": [ {"type": "door_schedule|keynote_legend|...", "sheet": "", "page": 0, "row_count": 0, "columns": [""] } ],
  "schedules_and_legends_absent": [""],
  "door_schedule_rows": [ {"door_number": "", "sheet": ""} ],
  "findings": [
    {"category": "sheet_index|callouts|door_tags_schedule|<other>", "issue_type": "verified|Reference Missing|Incorrect Ref. Detail|Sheet is missing|Unused|duplicated|Missing From Schedule|Schedule row unused|<other>", "source_sheet": "", "source_page": 0, "source_mark": "", "expected_target": "", "notes": ""}
  ],
  "counts": {
    "callouts": {"detected": 0, "verified": 0, "issues": 0},
    "door_tags_schedule": {"detected": 0, "verified": 0, "issues": 0},
    "<any other category you actually checked>": {"detected": 0, "verified": 0, "issues": 0}
  },
  "coverage_notes": ""
}
Include one "findings" entry per callout occurrence and per door-tag occurrence (verified ones included), so the counts can be recomputed from the findings. "detected" = total occurrences found; "verified" = occurrences that check out; "issues" = detected - verified.

You are auditing a construction drawing set (the attached PDF) for OBJECTIVE, DOCUMENT-INTERNAL CONSISTENCY ONLY.

Scope rules:
- Check only whether the drawing set agrees with itself. Do NOT comment on design quality, code compliance, constructability, quantities, takeoff, cost, or anything requiring outside knowledge.
- Do not invent sheets, details, schedule rows, keynotes, or legends. If something is not present in the document, say it is absent. Every finding must cite where in the set you saw it.
- Review every page of the attached PDF. If you could not read some pages or some text (e.g. too small), say so explicitly in "coverage_notes" rather than guessing.

Perform these checks:

1. SHEET INVENTORY. List every sheet: PDF page number (1-based), sheet number, sheet title.

2. SHEET INDEX vs SHEETS PRESENT. If the set contains a sheet index / drawing list, compare it to the sheets actually present. Report: sheets listed in the index but missing from the set ("Sheet is missing"), sheets present but not in the index ("Unused index entry" / not indexed), and duplicate sheet numbers ("duplicated"). If there is no sheet index, say so.

3. CALLOUTS / DETAIL AND SHEET REFERENCES. Find every callout bubble/tag that references another view: section marks, elevation marks, detail callouts, enlarged-plan / room-layout callouts, and any "see sheet X" references. Each typically shows a detail/view number and a target sheet number. For each one, check that the target sheet exists in this set and, where a detail number is given, that the target sheet actually contains a view with that number/title. Classify each callout as one of:
   - "verified" (target sheet exists and the referenced view/detail is on it)
   - "Reference Missing" (callout's target sheet or reference text is missing/blank/unreadable as drawn)
   - "Sheet is missing" (target sheet number does not exist in this set)
   - "Incorrect Ref. Detail" (target sheet exists but the detail/view number does not exist on it, or points to the wrong view)
   - "Unused" (a detail/view on a sheet that is never referenced by any callout, if you can detect it)
   - "duplicated" (two different views share the same number on the same sheet, or the same callout is duplicated)

4. DOOR TAGS vs DOOR SCHEDULE. Find every door tag (door mark/number shown on plans and room layouts) on every sheet, counting each occurrence on each sheet separately. Find the door schedule(s). Match each plan door tag to a schedule row by door number. Matching rule: case-insensitive, ignoring hyphens, spaces and periods (e.g. "106-A" matches "106A"); otherwise exact. Classify each tag occurrence as:
   - "verified" (matches a schedule row)
   - "Missing From Schedule" (tag appears on a plan but has no schedule row)
   - "duplicated" (the same door number appears in the schedule more than once, or one physical door is tagged with conflicting numbers)
   Also report schedule rows whose door number never appears on any plan ("schedule row unused / missing from plan").

5. KEYNOTES. Find every keynote legend / keynote list and every keynote tag on the drawings. Match each keynote tag to an entry in the legend on the same sheet (or the set-wide legend if that is how the set is organized). Classify each tag occurrence as "verified", "Missing From Schedule" (tag has no legend entry), or "duplicated"; also report legend entries never used on the drawing ("Unused").

6. SCHEDULES. List every schedule in the set (door, window, finish, equipment, fixture, panel, etc.). For each schedule row, check that the item it describes is tagged somewhere on the drawings; report rows never used ("Schedule row unused") and duplicate row numbers ("duplicated").

7. MEP TAGS vs MEP SCHEDULES. Separately for MECHANICAL, ELECTRICAL and PLUMBING: find every equipment / fixture / device tag on the drawings (e.g. AHU-1, RTU-2, EF-3, VAV-1, WH-1, P-1, panel names, circuit references like "LP-1/12") and match it to its schedule, panel schedule, or legend. Classify each tag occurrence as "verified", "Missing From Schedule", "Incorrect Ref. Detail" (e.g. circuit points to a panel or circuit number that does not exist), or "duplicated". If a discipline has no tags or no schedules in this set, say so.

8. ANY OTHER TAGS vs LEGENDS. If the set contains other tagged items with a legend or schedule (e.g. partition/wall types, room/finish tags), check them the same way. Do not invent legends that are not present.

OUTPUT FORMAT. Return ONLY a single JSON object (no prose before or after, no markdown fences) with exactly this structure:
{
  "sheets": [ {"page": 1, "sheet_number": "", "title": ""} ],
  "sheet_index": {"present": true/false, "location": "", "entries": [""] },
  "schedules_and_legends_found": [ {"type": "door_schedule|keynote_legend|...", "sheet": "", "page": 0, "row_count": 0, "columns": [""] } ],
  "schedules_and_legends_absent": [""],
  "door_schedule_rows": [ {"door_number": "", "sheet": ""} ],
  "findings": [
    {"category": "sheet_index|callouts|door_tags|door_schedules|keynotes|mechanical|electrical|plumbing|<other>", "issue_type": "verified|Reference Missing|Incorrect Ref. Detail|Sheet is missing|Unused|duplicated|Missing From Schedule|Schedule row unused|<other>", "source_sheet": "", "source_page": 0, "source_mark": "", "expected_target": "", "notes": ""}
  ],
  "counts": {
    "callouts": {"detected": 0, "verified": 0, "issues": 0},
    "sheet_index": {"detected": 0, "verified": 0, "issues": 0},
    "door_tags": {"detected": 0, "verified": 0, "issues": 0},
    "door_schedules": {"detected": 0, "verified": 0, "issues": 0},
    "keynotes": {"detected": 0, "verified": 0, "issues": 0},
    "mechanical": {"detected": 0, "verified": 0, "issues": 0},
    "electrical": {"detected": 0, "verified": 0, "issues": 0},
    "plumbing": {"detected": 0, "verified": 0, "issues": 0},
    "<any other category you actually checked>": {"detected": 0, "verified": 0, "issues": 0}
  },
  "coverage_notes": ""
}
Include one "findings" entry per sheet-index entry, callout occurrence, door-tag occurrence, door-schedule row, keynote tag occurrence and MEP tag occurrence (verified ones included), so the counts can be recomputed from the findings. "detected" = total occurrences found; "verified" = occurrences that check out; "issues" = detected - verified.

MarkedUp pages that explain the work

These three pages already live on markedup.io. They are here so you can learn the checks, and see a marked-up finding, without leaving the site.

• Blueprint Academy

  Lessons on how to read a drawing set and how a MarkedUp audit is run. Use it if you want the vocabulary before you try the test files.

• What is an RFI in construction?

  A plain guide to what an RFI is, and how to write one when a check finds a conflict. The guide is also a 16-page PDF.

• Sample audit

  A marked sheet and a shortened issue log, so you can see what a MarkedUp finding looks like before you schedule a call.

Files to test

Both sets are clean, born-digital vector PDFs. Neither file contains MarkedUp markups.

How to run the test: download the PDF, paste the matching prompt, attach the file, then compare. For Ideatura, compare counts with the official totals on this page, and use the answer-key item list to locate marks. For Example Plans, compare specific claims with the fact-check in What we found. MarkedUp's Example Plans totals are not published yet.

Ideatura answer key, v0.1

The answer key is the reference for the 32-page Ideatura PDF. Official totals, which are the numbers to compare against, are callouts 93 / 93 / 0, schedules 29 / 29 / 0, and tags 89 / 61 / 28. Read each trio as Detected / Verified / Issues.

The item list in v0.1 is slightly larger than those official totals in two places. It has 95 callout records, against 93 official, and 91 tag records with 60 verified and 31 issues, against 89 / 61 / 28 official. The schedule list matches. Until v0.2, compare counts against the official totals, and use the item list to locate marks. In the JSON file, the counts object holds the official totals. The item_list_counts object is the longer list.

Known limits, from the answer-key readme: the key covers callouts, door tags, and the door schedule only. Keynotes, sheet index entries, room tags, and general notes are not included. The readme also says the set has no sheet index or keynote legend, so those checks do not apply. The file description above lists the Ideatura sheet index as present. Those two notes disagree. Until that is reconciled, treat the key as a locator for callouts, door tags, and door-schedule rows, which is what v0.1 actually lists.

A schedule row is listed once per sheet that carries its tag, so one door number can appear as several schedule records. A four-way interior elevation marker counts as one callout.

These files are for evaluation and benchmarking of drawing-review methods. Do not treat the answer key as a design review, or as a substitute for professional judgment.

What we found

We gave general chat models the same clean drawing set and the same document-internal consistency prompt, then compared what they reported with MarkedUp on Ideatura, and with an independent fact-check of specific claims on Example Plans. The checks are objective and internal to the document: sheet index versus sheets present, callouts versus target sheets and views, door tags versus the door schedule, and keynotes versus the legend. The drawing set is the ground truth. Nothing here judges design, code compliance, or quantities. No MarkedUp output was ever sent to a model.

Counts are reported as Detected / Verified / Issues. Detected is how many items that run reported. Verified and Issues are that run's own split, unless a note says we fact-checked a specific claim. We do not publish an accuracy percentage. Christopher ran the three chat-window runs by hand on November 10, 2026, all starting at 10:48 AM ET. The chat windows show no token counts or cost.

Ideatura, whole file

Ideatura official totals, from the cleaned MarkedUp export: callouts 93 / 93 / 0, schedules 29 / 29 / 0, tags 89 / 61 / 28. The model rows are what each model reported for the whole file.

• MarkedUp

  Callouts: 93 / 93 / 0

  Schedules: 29 / 29 / 0

  Tags: 89 / 61 / 28

• Claude Fable 5.2

  Callouts: 87 / 86 / 1

  Schedules: Not reported as a checked category

  Tags: 93 / 60 / 33

• GPT-5.5

  Callouts: 86 / 85 / 1

  Schedules: Not reported as a checked category

  Tags: 94 / 60 / 34

• Gemini 4 Argon

  Callouts: 87 / 86 / 1

  Schedules: Not reported as a checked category

  Tags: 92 / 61 / 31

Schedules were not reported as a checked category by Claude, GPT-5.5, or Gemini 4 Argon. Each listed the 21-row door schedule. MarkedUp's 29 counts a row once per sheet that carries its tag, so those numbers measure different things.

• GPT-5.5's totals are not backed by its own findings list. The list has only 29 callouts and 51 door tags, so the totals cannot be checked.
• Gemini's findings list adds up to 94 / 60 / 34 for tags, not the 92 / 61 / 31 it reported.
• All three models agreed on the same 14 door numbers on the plans with no schedule row: 106-B, 107 to 111, 112-A, 112-B, 113, 114-A, 114-B, 115-A, 115-B, and 116.
• Claude and Gemini flagged the one blank elevation marker on A101. GPT-5.5 missed it.
• On 4 Ideatura pages, 200 dpi tiles let Claude read every mark, and once four-way markers were grouped the counts matched the whole-file run exactly. Sharper images found nothing new.

Example Plans, model runs

We ran MarkedUp on this same set. Those Detected / Verified / Issues totals are not on this page. The November 10, 2026 export still includes false positives. We will publish the cleaned totals when that export is ready. What follows is what the models reported, plus a fact-check of specific claims against the clean PDF. The fact-check did not send the PDF to another model.

Door tags and the door schedule are separate counts. Do not add them together.

Runs that returned nothing usable

• Gemini 3.8 Thinking chat, paid: listed all 79 sheets correctly, then said there was no sheet index and recorded zeros for callouts, door tags, door schedule rows, and keynotes, because it said the page images were too small to read. The PDF has a text layer on every page. The SHEET LIST is on S-000.
• ChatGPT free chat, Think on: no response was returned in the window we recorded, as of 11:17 AM ET that day.
• GPT-5.5 and Gemini 4 Argon via OpenRouter: rejected before the model saw the file. The file was over a 20 MB request limit.

Fact-check of specific claims

Checked against the clean Example Plans PDF, not by sending that PDF to another model.

Confirmed

• S-000 lists only the 19 structural sheets. 60 architectural sheets are unindexed.
• There is a 99-row door schedule on AE601, including 4C201A.
• Schedule row 124B is never used as a door tag. It is a PANTRY 124B room tag.
• Callouts 2 / AE504, on AE102 and AE107, and 12 / AI501 point to views that do not exist.
• Keynote legends on AE311 to AE313 are duplicated: identical text at identical coordinates.
• There are no MEP discipline sheets in this set.

Wrong

• Claude's three view back-references that it said were not reciprocated (AE502 C2, AE504 D4, AE703 D5) were paired with the wrong mark.
• Gemini's claim that there is no sheet index, and that the text was too small to read, was wrong.

Benchmarks and further reading

AECV-Bench, from AECFoundry and UNSW (arXiv:2601.04819, January 2026), tests general vision models on single raster floor-plan images. On the June 11, 2026 leaderboard, the best counting score is about 54% mean exact-match. Doors and windows are the weakest items. Text reading is around 98%. The benchmark has no native PDF input and no cross-sheet checks. Those paper figures and leaderboard figures are theirs. They are not scores from this study, and they are not MarkedUp scores.

AECFoundry's October 20, 2026 survey of 99 AEC AI benchmarks says none of them tests whether documents agree with each other. That is the gap this page covers: sheet against sheet, tag against schedule, callout against the view it names.

• AECV-Bench leaderboard

  The public leaderboard for general vision models on single raster floor-plan images. Last updated June 11, 2026. It is here so you can see the counting and text-reading scores quoted above, and see that the task stops at one image.

• AECV-Bench paper (arXiv:2601.04819)

  The January 2026 paper from AECFoundry and UNSW. It describes the benchmark. It does not test a native PDF, and it does not test whether one sheet agrees with another.

• What the AEC AI benchmarks measure, and what they miss

  AECFoundry's October 20, 2026 survey of 99 AEC AI benchmarks. None of them tests whether documents agree with each other. That cross-sheet check is what this page is about.

Why the industry research is here

The McKinsey and MIT Sloan pieces are not drawing benchmarks. They are here because they separate individual speed from a result a company can keep, which is the same split as a plausible drawing list and a list you can trust.

• Seizing the agentic AI advantage

  McKinsey, June 13, 2025. The source for the line that nearly eight in ten companies report using gen AI, and just as many report no significant bottom-line impact.

• The state of AI

  McKinsey, August 25, 2026. 1,719 respondents in 97 countries, May 4 to June 8, 2026, self-reported. 80% said AI improved individual productivity, 37% attributed any EBIT impact, and about 6% saw a material effect. 32% said their organizations decided against buying at least one software product or feature because they could build it in-house with agentic coding tools.

• What does AI really cost?

  McKinsey, October 14, 2026. Names cloud bills, software subscriptions, and the hours people spend checking outputs as real costs.

• Delivering on construction productivity is no longer optional

  McKinsey, August 9, 2024. Productivity gains in construction are often bid away in tenders rather than kept as margin.

• Seeing real value in AI depends on being able to verify its outputs

  MIT Sloan, June 8, 2026. AI makes it cheap to produce work, but not to judge whether that work is any good. Human review time becomes the constraint.

Questions this page answers