Takeoff AI That Remembers: How Aginera's Extraction Memory Makes Every Takeoff Better Than the Last

Every AI takeoff tool on the market can read a drawing set once.

Almost none of them can remember what they learned.

That distinction sounds academic until you watch it play out on real bid sets. An estimator who did last month's takeoff for the same GC starts today's set already knowing the fixture vocabulary, the sheet conventions, the way this engineer labels exhaust fans. A single-pass AI starts from zero. Every document. Every time. Its accuracy on your hundredth project is exactly its accuracy on your first.

This is the gap between AI that extracts and agentic AI that learns — and closing it is not a prompt-engineering exercise. It required us to build a memory architecture with the same care we'd give a billing system, because in construction estimating, a "learned" fact that turns out to be invented is worse than no learning at all.

This article is the technical account of how Aginera's extraction memory works: the three kinds of memory, the injection mechanics, the anti-fabrication guarantees, and the measured results on production drawing sets.

Why single-pass AI takeoff plateaus

A modern AI construction takeoff pipeline does a lot in one pass: it classifies pages, routes each sheet to trade-specific extraction schemas, runs a vision model over plans and schedules, and post-processes the results into priced line items. Aginera's pipeline does all of this across electrical, mechanical, plumbing, HVAC, and fire protection.

But a single pass has a structural ceiling, and it shows up in a specific, repeatable way: the schedule and the plan disagree about identity.

The lighting fixture schedule on sheet E601 defines what "A1" means — a 2x4 LED troffer, 35W, 277V. Twenty sheets later, the plan page draws thirty A1 symbols in a reflected ceiling grid. A vision model reading that plan page in isolation sees small circled tags with no context. Sometimes it reads them correctly. Often it emits "light fixture, type unknown," or worse, counts the geometry and drops the identity entirely — and identity is what pricing runs on. A troffer and a wall sconce do not cost the same.

We measured how widespread this is: across 54 production documents in our evaluation corpus, 33 carried thirty or more schedule-defined marks that were printed on plan pages but missing from the extracted takeoff. The knowledge was in the document. The single pass just couldn't hold it across pages.

Three kinds of memory, one governed substrate

Cognitive science gives a useful taxonomy that maps surprisingly cleanly onto extraction:

• Semantic memory — facts. "In this project, A1 is a 2x4 LED troffer; FCU-8-01 is a Daikin fan coil."
• Episodic memory — events. "On this document, the QA agent flagged a fabricated exhaust fan; the customer marked these rows wrong."
• Procedural memory — skills. "Sheets in this project matching this pattern are mechanical, not electrical; power plans here should also extract kitchen-equipment connections."

All three live in one governed substrate — project-scoped Postgres tables, not an opaque vector store — because the point is not just to remember but to audit what is remembered. Every fact records which document and page it came from. Every learned rule carries its origin investigation, its validation evidence, and the identity of the human who activated it.

Semantic: harvest from schedules, inject into plans

After every extraction completes its full quality pipeline, a harvest pass distills schedule-corroborated rows — rows whose identity was read from the document's own printed schedules — into discipline-scoped facts:

(electrical, fixture_mark,  "A1",       "2x4 LED troffer, 35W")
(hvac,       schedule_mark, "FCU-8-01", "Daikin FXSQ48 fan coil")
(plumbing,   schedule_mark, "WH-1",     "Water heater")

Only marks that pass a shape validator are harvested — a real schedule tag is short or carries a number; FEEDER and CONDUIT are vocabulary, not marks, and never enter memory. On the next extraction in the same project, active facts are injected into the vision model's system prompt as a compact known-marks block: these tags are confirmed real in this project; when you see one on the drawing, extract it with this identity.

One engineering detail matters more than it looks: anything that changes the model's prompt is hashed into the response-cache key. Vision responses are content-addressed and cached; if memory changed the prompt but not the cache key, the pipeline would silently serve the pre-memory answer forever and the learning would be a no-op. We found and fixed exactly this class of bug — twice, at two cache layers — during validation. If you are building agentic pipelines with caching, this is the invariant to steal: no context change without a cache-key change.

Episodic: every correction becomes a proposal, never a silent change

Three feedback streams flow into the same conveyor:

1. QA agent findings — an independent scoring agent audits every extraction against the source PDF.
2. Customer review flags — when a customer marks a line item as a false positive, that is the highest-precision training signal that exists.
3. Estimator corrections — rows rejected in the review UI are mined daily, grouped by mark so one bulk cleanup becomes one investigation.

Each signal triggers a zero-cost investigation that traces the item through four stages — text layer, page routing, cached model responses, persisted rows — and pinpoints where it was lost or invented. When the investigation yields a concrete fix, it is filed as an inert rule proposal with the evidence attached.

Here is the governance decision we consider non-negotiable: nothing self-activates. Proposals accumulate in a review queue with dry-run validation (does the rule fix the origin document? how many rows would it touch on every other project?). A human promotes proposed → testing → active, and anything that would change model behavior requires an explicit acknowledgment that re-billing affected pages is intended. Agentic learning without gates is how you wake up to a takeoff quietly rewritten by a rule nobody approved.

Procedural: routing and reading habits, without a code deploy

Some knowledge isn't a fact about a mark — it's a habit of reading. Two rule types make that memory-addressable:

• Routing hints bias a page's trade scores — clamped, floor-at-zero, never a hard override — and stamp themselves into the page's routing-evidence trail. In one production case, a power/roof sheet's "ROOF PLAN" title text was pulling roofing and architectural extractors onto an electrical sheet; two hints dropped those scores from 0.80 and 0.60 to 0.30 and 0.10, cutting the extractor list from five to three while leaving the real trades untouched.
• Schema instructions append project-specific rules to a drawing type's extraction contract — and are hashed into the cache key like everything else.

The practical consequence: a misrouted sheet class that used to require an engineer, a code change, and a deployment is now fixed by activating one scoped rule from the review queue.

The part that makes it safe: verification as a database invariant

Memory in a generative pipeline has an obvious failure mode, and we hit it on day one of testing — which is exactly what testing is for.

A plan page carried a bare "EF" keynote. Project memory knew this project had EF-1 and EF-2. A recovery pass helpfully resolved the bare keynote into both numbered fans, at quantity 14 each. EF-1 was actually printed on that page. EF-2 was not. That is memory-assisted fabrication: plausible, schedule-consistent, and wrong.

We closed it with two layers:

1. A prompt contract: a tag matches only in its exact printed form; partial or unnumbered tags are never resolved to numbered marks from memory. Re-running the same page: the phantom EF-2 disappeared, and all seven legitimate recovered marks survived.
2. A corroboration guard that does not depend on the model following instructions: after extraction, any recovery-pass row carrying a memory mark must have that exact mark printed in the page's text layer — hyphenation variants and internal prefixes handled — or the row is excluded from pricing and flagged for review. Pages without a usable text layer are skipped, because absence of text proves nothing on a scanned sheet. In a full-document adversarial test, a deliberately planted fabricated row was the only row the guard touched.

This is the philosophy difference we'd urge anyone evaluating AI takeoff software to probe: is zero-fabrication a prompt promise or a database invariant? Prompts drift. Invariants hold.

Measured results

We validate memory with paired runs — same pages, same model, memory off versus on — and score against a provable denominator: only marks actually printed in that page's own text layer count toward coverage.

• Multifamily electrical set, three plan pages: schedule-mark coverage went from 21% to 39%, with fixture types (A1, B3, B5, B6, S1, S2) that had been extracted as anonymous geometry now carrying their schedule identity. Zero uncorroborated marks.
• Single worst plan sheet: 1 of 10 printed schedule marks identified without memory; 8 of 10 with it.
• Cross-discipline: the same mechanism recovered mechanical (EF-series exhaust fans) and plumbing (WH-series heaters) marks from their respective schedules.

The most instructive result was a 187-page combined mechanical set whose dense schedule tables had convinced a document-classification heuristic that the set was a specification, not drawings — so a spec-only cleanup pass was deleting every vision-extracted row. The vision model had been reading the fan-coil tags correctly all along; the pipeline was throwing its work away. Our investigation tooling pinpointed the stage in minutes. The fix was one memory-architecture decision: a human-confirmed project archetype now outranks the classification heuristics. Result: 217 rows became 497, and mechanical equipment marks went from 3 to 178 — on a mechanical contractor's project where the mechanical scope had been nearly invisible.

That case is the whole thesis in miniature. The failure wasn't the model. It was the absence of a place for confirmed human knowledge to live and carry authority.

How this compares to the rest of the AI takeoff field

The AI takeoff category is crowded — Boon, InsidePanel, and a growing list of others, alongside our own take on where takeoff automation is heading. Most of the field, as far as public materials describe, is built around the single-pass model: upload a set, get counts, repeat. Some are fast and well-executed at that.

The question we'd suggest asking any vendor — including us — is not "how accurate is the first takeoff?" but three sharper ones:

1. Does the tenth takeoff for the same customer beat the first? If the system has no memory, the honest answer is no — accuracy is flat by construction.
2. When an estimator corrects a row, where does that correction go? If it only fixes the row, the same error returns on the next document. In Aginera, it becomes an investigated, evidence-linked rule proposal.
3. When the AI uses remembered knowledge, how is fabrication prevented — by instruction, or by verification? Ask specifically whether learned facts are checked against the document before they can price.

Memory with governance is, to our knowledge, unique to Aginera in this category today — and it compounds: every schedule extracted, every correction made, every misroute fixed makes the next takeoff in that project measurably better, with an audit trail for every learned fact.

What this means for estimators

None of this changes the workflow you see: upload drawings, review the takeoff, export to your estimate. What changes is the trajectory. Projects accumulate a verified vocabulary. Corrections stop repeating. Sheet-routing quirks get fixed once, by a reviewed rule instead of a support ticket. And the takeoff you review separates plan-confirmed quantities from review-tier items honestly — because the memory that helped produce them was itself checked against the page.

If you want to see what an AI takeoff that learns your projects looks like on your own drawings, try the free tools — free electrical takeoff, free HVAC takeoff, or free plumbing takeoff — your first takeoff is free, and it's the worst one you'll ever get.