A common question we get from estimators evaluating FOBECON is some version of: how does it actually know what it is looking at? The question is fair. Construction drawings are not photos of objects. They are symbolic representations of objects that do not yet exist, encoded in a visual grammar that took the construction industry over a century to standardise. Asking software to read them is asking it to understand a dense, rule-bound graphical language.
This article explains in plain terms what happens from the moment a drawing set is uploaded to the moment a BOQ line item appears in the output. It is not a technical paper; it is a description of the pipeline intended for estimators and project managers who want to know how to interpret the results they receive.
Step 1: PDF ingestion and sheet classification
A construction drawing set arrives as one or more PDF files. The first task is to parse each PDF page into a representation the model can work with. This is not simply converting pages to images. PDFs produced from CAD software contain vector data: lines, arcs, text objects, and fill regions stored as geometric coordinates rather than pixels. We extract this vector data where it is available, because it preserves scale information and avoids the resolution limits of rasterisation.
For scanned PDFs (hand-drafted drawings digitised as images, or CAD-printed drawings re-scanned), we apply image processing steps to improve the signal quality before the model reads them: deskew, noise reduction, and contrast normalisation. Scanned drawings degrade model accuracy relative to native CAD exports. We do not hide this. The output confidence scores are lower for scanned inputs, and the review interface flags this.
After parsing, each sheet is classified by drawing type: floor plan, elevation, section, structural plan, MEP schematic, detail, or schedule. This classification drives the rest of the pipeline. A floor plan is processed differently from an elevation. The element types expected on each drawing type differ, and the measurement logic differs. Getting the sheet classification right is essential to producing correct counts.
Step 2: Scale detection and spatial calibration
Construction drawings contain scale information in two forms: a written scale indicator (e.g., 1:100) and, frequently, a graphical scale bar. Before any measurement can be extracted, the model must determine the drawing scale with sufficient accuracy to produce physical measurements rather than pixel counts.
Scale detection reads the text annotations on the drawing for a numeric scale declaration. Where both a written scale and a graphical scale bar are present, we cross-check them. Discrepancies above a small threshold (which can occur on scanned drawings that have been photocopied at non-standard ratios) are flagged in the output for user review rather than silently resolved. An undetected scale error propagates to every measurement on the sheet and is one of the most consequential failure modes in automated takeoff.
Once scale is established, the model builds a spatial coordinate system for the sheet. Each element position is expressed in real-world coordinates, which allows area calculations and linear measurements to be performed in metres and square metres rather than in pixels.
Step 3: Element detection
This is the core computer vision step. The model examines each classified drawing sheet and identifies element instances: doors, windows, columns, walls, stairs, fixtures, and similar construction elements. The detection is trained on a large corpus of construction drawings spanning different drawing conventions, software-generated outputs, and hand-drafted styles.
Element detection in construction drawings is harder than generic object detection for several reasons. First, the same element type can be drawn differently by different architectural firms, and even within the same firm at different project scales. A door symbol at 1:50 scale is drawn with an arc; at 1:500 the arc is omitted and only the opening break in the wall line is shown. The model must recognise both representations as the same element type.
Second, construction drawing elements are often partially occluded by dimension lines, text annotations, and overlapping geometric data. The model is trained to handle this overlap and to produce a count that corresponds to the intended drawing content rather than the literal visible pixels.
Third, some element types are defined by their context as much as their shape. A column in a structural plan is identifiable from its symbol. The same column implied by a load transfer annotation in an architectural plan requires reading both the geometry and the associated text. We handle a subset of context-dependent elements in the current system and are explicit about which element types rely on geometric detection only versus geometric-plus-text reading.
Step 4: Measurement extraction and BOQ mapping
After element instances are identified, measurements are extracted per element type:
- Count elements (doors, windows, columns, fixtures): number of instances per drawing sheet, with floor-level assignment from sheet classification
- Linear elements (walls, partitions, beams, kerbs): length in metres, extracted from the detected line geometry and scaled to real-world units
- Area elements (slabs, floor finishes, ceilings, glazing panels): area in square metres, calculated from the bounding polygon of the detected region
The extracted measurements are then mapped to BOQ line items. This mapping uses the element type, the drawing-level context, and any material or specification annotations present on the drawing or in the associated document set. The mapping produces a preliminary BOQ in the output structure selected by the user: organised by element type, by floor level, or by CSI/RICS work section as appropriate.
We want to be direct about this step: the BOQ mapping is informed by the drawing data and the specification text where provided, but it is not a final legal document. It is a structured first draft. The estimator's role in the workflow is to review the mapped line items, resolve any flagged ambiguities (elements the model detected but could not confidently classify), and apply their knowledge of project-specific requirements before the BOQ is used for submission.
What the model does not see
This is the part that matters most for calibrating expectations. The current system reads geometric data and associated text annotations. It does not read specification documents as source for material classification without explicit text linking in the drawing set. It does not infer contractor methodology (e.g., whether formwork costs are included in a concrete quantity). It does not handle project-specific drawing conventions that deviate significantly from the training distribution without producing a lower-confidence output.
Specifically, the model does not replace the estimator's reading of the specification to apply correct work section classification. A wall can be detected, measured, and placed in a provisional BOQ line. Whether it is a standard timber-framed partition, a fire-rated assembly, or a structural shear wall is determined by cross-referencing the specification, which is the estimator's judgment call. The model surfaces the measurement. The estimator applies the classification.
This boundary is intentional. The cases where automation adds the most value are the routine counts and measurements that consume the majority of takeoff time but require the least professional judgment. The cases requiring the most judgment are precisely where a skilled estimator should be spending their time, not on counting symbols sheet by sheet.
Confidence scoring and the review interface
Every element detection result carries a confidence score. High-confidence detections (typically above 0.85 in our internal scoring) are output directly into the BOQ draft. Lower-confidence detections are flagged for review: the model shows the estimator the element it detected, the score, and the source location on the drawing.
In practice, for native CAD-exported PDF inputs, roughly 10 to 18 percent of detected elements require a review action from the estimator. The majority of these are context-dependent elements or elements on detail sheets where drawing density is high. For scanned inputs, the review fraction is higher, typically 20 to 35 percent, reflecting the resolution and clarity limitations of digitised originals.
The goal of the confidence flagging system is to ensure that the estimator's review time is concentrated on the genuinely ambiguous items rather than distributed evenly across a count that is mostly correct. A BOQ where the estimator confirms 200 routine detections and resolves 30 flagged items is a different experience from manually counting all 230 from scratch.