Papers for

real-time data processing engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

LUT-aware transformer speeds up FPGA real-time processing

LEAF: LUT-aware Efficient Attention on FPGAs

Abstract: We introduce Lookup-table (LUT)-aware Efficient Attention on FPGAs (LEAF). LEAF is the first nanosecond-scale LUT-aware transformer for FPGAs. Deploying transformers that are large enough to perform well, small enough to fit on an FPGA, and fast enough to meet the sub-microsecond timescales required by real-time systems such as those at CERN remains difficult, since the same attention mechanism that gives these models their high expressivity also makes them costly to implement in hardware. LEAF converts the query, key, and value projections and the final output feed-forward layer into FPGA-native layers with LUT-aware training, creating the first LUT-aware transformer and linformer layers built for ultra-low-latency inference. Attention resists LUT-aware training because it multiplies two activations where the method needs a constant weight to tabulate; LEAF tabulates the attention block's constant matrix-vector multiplications and leaves the rest in arithmetic form. On a standard jet tagging benchmark this improves accuracy by 1.8 percentage points while using 2.9 times fewer LUTs than the previous state-of-the-art sub-microsecond transformer at the same input size. Our most efficient LEAF-based linformer with 64 input particles reaches 80.4% classification accuracy at 28k LUTs and 35ns.

Mon 19 OctMachine Learning
The gist
Making machine learning models called transformers run very fast on special computer chips called FPGAs is hard because the math inside transformers is complicated. The authors created a new method called LEAF that changes parts of the transformer to fit the way FPGAs work best. This method makes the model use fewer hardware resources and run much faster while keeping good accuracy. Their approach is especially useful for tasks that need results in just nanoseconds, like certain physics experiments.
Open → 2610.27078v1