Papers for

software engineers in embedded systems

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Open source models solve simple hardware design tasks with iterative agent tools

Benchmarking Agentic HLS Design Tasks With HLS-Eval

Abstract: Large language models (LLMs) and AI agents are increasingly explored for hardware design, including high-level digital design. While most work targets code generation and editing for hardware description languages (HDLs), our prior work introduced HLS-Eval, an open-source benchmark for evaluating LLMs on high-level synthesis (HLS) design tasks. Those evaluations, however, focused on zero-shot generation and editing, leaving open how agents achieve HLS design tasks. We therefore extend HLS-Eval with an agentic evaluation flow built on the open-source mini-swe-agent framework. The flow lets HLS design agents use file-editing tools, invoke a C++ compiler for self-verification, and iteratively refine designs during inference, while logging agent traces for analysis of cost, token usage, and iteration count. We present initial results on the existing HLS-Eval benchmarks. In our initial evaluation, we find open-source LLMs paired with an agentic harness solve every simple HLS code generation task in our evaluation, underscoring the need to expand benchmark difficulty as model capabilities advance. Analyzing traces from passing and failing runs, we show how model size, token usage, and trajectory length relate to design pass rates. These results establish a foundation for agentic HLS design and motivate harder benchmarks and new agentic tooling as model capabilities progress.

Tue 8 SeptHardware Architecture
The gist
Designing hardware with software tools is complicated and can take many tries. The authors improved a test system that measures how well AI tools can write and fix hardware designs automatically. They found that some open-source AI models, when allowed to test and refine their own code, can solve simple hardware tasks very well. They also studied how model size and the number of retries affect success. This work helps create better AI tools for designing hardware and points out the need for harder challenges to keep up with AI improvements.
Open → 2609.09526v1