VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design
2026-07-20 • Computation and Language
Computation and LanguageSoftware Engineering
AI summaryⓘ
The authors created VEHBench, a new test set to check how well large language models (LLMs) help design vibration energy harvesters (VEHs) for battery-free Internet of Things devices. Unlike existing tests that only check final results, their benchmark looks at how LLMs perform at different steps in the design process. They found that no single LLM works best at every stage, and each shows different strengths depending on the task. VEHBench helps researchers choose and improve LLMs for specific parts of engineering design workflows.
Internet of Thingsvibration energy harvesterlarge language modelphysical design constraintsbenchmarkingdesign workflowverificationenergy harvestingartifact evaluationdiagnostic benchmark
Authors
Depeng Su, Yuyu Luo, Guobiao Hu
Abstract
Battery-free Internet of Things (IoT) requires iterative design of vibration energy harvesters (VEHs) under coupled physical constraints, while LLMs are emerging as interface layers for engineering workflows. However, existing engineering benchmarks primarily assess final artifact validity, offering limited insights into how LLMs behave across different stages of coupled physical design. We introduce VEHBench, an engineering-native diagnostic benchmark for LLM-assisted VEH design, featuring 763 literature-grounded tasks scored by an analytical physical oracle. VEHBench evaluates four design roles: specification triage, verifier-guided search, corrupted-state recovery, and policy-conditioned selection. Experimental results reveal that LLM capability is strongly stage-dependent: no single model consistently dominates the entire workflow, and response-control profiles expose distinct behavioral patterns across design roles. VEHBench thus provides a stage-aware foundation for evaluating, selecting, routing, and improving verifier-grounded engineering LLMs. The benchmark artifact is available at https://huggingface.co/datasets/AnonymousVehbench/vehbench