Vision models struggle matching engineering drawings to CAD renders
Blueprint2CAD: Benchmarking Foundation Models on Engineering Drawing-to-CAD Render Correspondence
Computer Vision and Pattern Recognition
Summary
Engineering drawings show mechanical parts from certain views, but it is hard for computer vision models to recognize if a shaded 3D render represents the same part when seen from another angle. The authors created Blueprint2CAD, a new test set to measure how well models can match these drawings to CAD renders. They found that current vision models reach only modest accuracy and that difficulties mainly come from recognizing parts across different viewpoints. Simple fine-tuning improves accuracy slightly but does not solve the viewpoint challenge or remove biases in the models’ answers.
What this means in practice
- •For manufacturing engineers: Verify if scanned engineering drawings match existing CAD parts to speed up part search and digital inventory management.
- •For computer vision developers: Test and improve algorithms for recognizing mechanical parts across different rendering styles and viewpoints using Blueprint2CAD dataset.
Authors
Alexander Valverde, Danny Xie-Li, Zhichao Wang, Marcos V. Conde
Abstract
Engineering drawings specify mechanical parts through orthographic projections. Recognizing which part a drawing depicts underlies part search, design reuse, and digitization, yet no public benchmark measures whether vision models can do it. We introduce Blueprint2CAD, a benchmark for drawing-to-render correspondence: given an orthographic line drawing and a shaded render, usually from another viewpoint, decide whether they show the same part. Its data layer, BlueCAD-30K, applies an open drafting pipeline to 29,925 Zero-to-CAD models, adding view-aligned renders, clean and dimensioned drawings from six viewpoints, and surface-type metadata. Frozen encoders retrieve the matching part from a 10,000-model gallery with up to 65% Recall@1 when viewpoints are known but 43% when they are not, and the four encoders we test lie within six points of each other: viewpoint, not backbone, is the bottleneck. Six zero-shot vision-language models reach only 0.52-0.67 verification accuracy, with model-specific Yes/No biases that chain-of-thought shifts but does not remove; a quick LoRA fine-tuning on in-domain pairs improves verification accuracy by a few points but sharpens these biases and leaves the viewpoint failure intact.