Papers for

edge device engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Diffusionssd compresses visual data efficiently on edge devices

DiffusionSSD: Achieving Extreme Visual Compression with Dual-stage Diffusion and Cross-level Near-Data Processing

Abstract: Growing visual data overwhelms storage on resource-constrained edge devices, making compression essential. Traditional codecs offer inadequate rate-distortion performance, motivating the exploration of diffusion codecs. Diffusion codecs face two limitations: (1) At the algorithm level, diffusion decoders preserve perceptual quality under extreme compression, but require substantial computation. (2) At the architecture level, in NPU-centered devices, neural codecs occupy the NPU, interfering with other AI applications. Meanwhile, the complex codec pipeline incurs long I/O paths, leading to additional overhead and energy cost. To address these issues, we propose DiffusionSSD, a cross-level near-data processing framework for extreme visual data compression on edge devices, using an algorithm-architecture co-design method. Specifically, (1) at algorithm level, we present Dual-stage Diffusion, a lightweight solution tailored for diffusion codecs that employs patchification and single-step diffusion, reducing computation cost while preserving perceptual quality; (2) at architecture level, we introduce a Cross-level Near Data Processing architecture that integrates the proposed computational universal flash storage (cUFS) and LPDDR-PIM, enabling codecs to be entirely offloaded to data side while reducing visual-data movement and avoiding SoC redesign; (3) at system level, combining both the algorithm and the underlying architecture, we propose a Patch-Workload co-aware scheduling scheme to enable more efficient system-level execution. Experiments show that Dual-stage Diffusion maintains comparable perceptual quality while reducing computation cost by up to 92.7%. Compared with NPU+DRAM and near-data-processing design such as SmartSSD, cUFS+PIM improves coding throughput by 2.32 times and 2.91 times on average, respectively.

Fri 13 NovHardware ArchitectureComputer Vision and Pattern Recognition
The gist
Visual data can take up a lot of space, especially on devices like phones or cameras that have limited storage. The authors developed DiffusionSSD, a new method to compress images extremely well without losing much quality, and it uses less computing power. They also designed a system that stores and processes data closer to where it is saved, reducing delays and energy use. Their approach lets devices handle visual compression better while leaving room for other AI tasks.
Open → 2611.17908v1

ConvLUT enables ultra-fast convolutional neural networks on FPGAs

ConvLUT: Rethinking Convolutional Architectures for Ultra-Low-Latency LUT-Based Inference

Abstract: Lookup-table (LUT)-based neural networks (NNs) offer a compelling path to ultra-low-latency inference on FPGAs. However, existing LUT-based neural networks have largely been restricted to MLP-style topologies, leaving structured vision and audio tasks underexplored. We present ConvLUT, a family of fully unrolled LUT-mappable convolutional architectures targeting the MLPerf Tiny benchmarks. It is comprised of a standard convolutional model built around a hierarchical tree of LUTs and a depthwise separable model following the MobileNet design philosophy. These architectures are enabled by four key contributions: a unified LUT-based convolutional framework supporting standard, depthwise, and pointwise convolutions under explicit fan-in constraints; a LUT-tree that addresses the $k^2c$ receptive field dimensionality bottleneck; a channel-partitioned LUT pooling layer that replaces conventional spatial pooling; and a hardware-aware iterative pruning strategy for stable convergence at scale. Evaluated on CIFAR-10, Visual Wake Words, and Keyword Spotting, ConvLUT achieves competitive accuracy with fully unrolled inference latencies below $\sim 50$ ns. On CIFAR-10 it reaches at least a $8844\times$ latency reduction over non-logic-based prior implementations, and outperforms existing MLP-style LUT-based models on the same task by at least $20$ pp in test accuracy.

Sun 8 NovMachine Learning
The gist
Faster computer programs can recognize images and sounds by using special tables called lookup tables, but so far these have mostly worked with simple network types. The authors designed a new way to build more complex networks, like those that analyze pictures and sounds, using these lookup tables. Their new method, ConvLUT, runs extremely fast on hardware called FPGAs and still gives accurate results. This breakthrough shows it’s possible to make super quick and accurate neural networks for real-world tasks like detecting objects or recognizing keywords.
Open → 2611.11507v1

Optimizing memory and computing to save energy in dnn inference

Jointly Optimizing Memory, Computing and Offloading for Energy-Efficient DNN Inference

Abstract: In edge intelligence systems, computation offloading and dynamic voltage and frequency scaling (DVFS) are widely used to improve energy efficiency. However, as applications like deep neural network (DNN) inference evolve in complexity, their demands for memory resources have progressively increased, establishing memory as a critical bottleneck that impacts both latency and energy consumption. In this paper, we propose a joint optimization scheme integrating memory, computing and offloading for energy-efficient DNN inference. We formulate the energy minimization problem under deadline constraints as a mixed-integer nonlinear programming (MINLP) problem, where the main challenge arises from the coupling between offloading and resource optimization. To address this issue, we decompose the original problem into two subproblems under a fixed offloading policy. The local subproblem is transformed into an equivalent geometric programming (GP) problem to derive the optimal memory frequency and computing frequency, while the edge subproblem is solved via convex optimization. An efficient coordinate descent (CD) algorithm is further developed to jointly optimize all variables. Simulation results demonstrate that our proposed algorithm achieves near-optimal performance with the gap from the optimal policy being at most 9.3%, and reduces the energy consumption of devices by up to 16.1% compared with benchmarks without memory frequency optimization.

Sun 8 NovMachine LearningArtificial Intelligence
The gist
Running deep neural networks on devices near you uses a lot of energy, partly because memory and computing resources are limited and linked. The authors studied how to best balance memory speed, computing speed, and whether to send work to an edge server to reduce energy use while meeting deadlines. They turned this complicated problem into smaller parts and created an algorithm to solve it efficiently. Their solution nearly matches the best possible and can cut device energy use by up to 16%.
Open → 2611.11106v1

Adaptive search improves hardware-constrained large language model inference

ACSeL: Adaptive Configuration Search for Hardware-Constrained LLM Inference

Abstract: Serving large language models (LLMs) under tight memory and latency service-level objectives (SLOs), including on edge devices, requires jointly selecting the model size, batch size, quantization algorithm, and mixed-precision quantization recipes. Exhaustive search over this combinatorially large space is infeasible. Unlike data-center serving, which often prioritizes throughput or generation speed, resource-constrained deployments typically set memory and latency budgets as constraints and then seek the configuration with the highest generation quality among feasible ones. To this end, we propose ACSeL, a framework that identifies configurations predicted to satisfy deployment constraints while achieving high predicted generation quality. To enable efficient quality estimation, we use Relative Model Quality Degradation (RMQD), a proxy based on the divergence between the candidate and reference models' next-token distributions over a small calibration corpus. Across the evaluated model sizes and precisions, RMQD is inexpensive to evaluate and strongly correlated with downstream benchmark performance. ACSeL combines (i) a lightweight quality surrogate that predicts RMQD directly from a configuration; (ii) Few-shot Median Calibration (FMC), which calibrates operator-profile-based latency estimates using a small number of end-to-end profiles; and (iii) a structured search algorithm BandPareto for exploring layerwise mixed-precision configurations. Experiments on an NVIDIA RTX A6000 workstation and an NVIDIA Jetson Thor edge platform show that FMC substantially improves deployment feasibility, while BandPareto + FMC achieves the highest feasibility and the lowest RMQD gaps to the best measured configurations in the evaluation set.

Thu 5 NovMachine Learning
The gist
Running large language models on devices with limited memory and speed is tough because you need to balance many settings like model size and precision. The authors developed ACSeL, a method that quickly finds the best settings to meet memory and speed limits while keeping output quality high. They introduced a way to predict quality using a small sample, making the search faster. Their technique works well on both powerful and edge hardware, helping ensure models run efficiently without too much trial and error.
Open → 2611.07570v1

Integrity gap risks attacks on offline text retrieval systems

Signed but Steerable: Measuring and Closing the Text-Vector Integrity Gap in Offline Retrieval-Augmented Generation

Abstract: Offline retrieval-augmented generation (RAG) on edge devices is updated through signed knowledge packs that carry text and precomputed embedding vectors. A signature proves who published a pack, not that its vectors encode its text. We study this text-vector integrity gap. A verifier that recomputes vectors and accepts them within a cosine tolerance concedes a predictable share of queries to an attacker who controls the vectors: the hijack rate equals the distribution of angular retrieval margins at the tolerance (the margin law). Margins are small in five corpora and two embedding models, so an idealised tolerance at the level of floating-point noise leaves up to 5.5% of queries exposed (at most 1.2% in any corpus once exact duplicate documents are excluded), and a threshold calibrated from our CPU INT8-FP32 comparison of a small embedding model (at the empirical 0.1% quantile of honest cosines) leaves 98-100% exposed; on a Raspberry Pi 5 running a local language model, such an attack lowers correct answers from 0.75 to 0.39. Integrity audits of compressed vector stores fare no better: on a FAISS product-quantisation index over 57,638 documents, tampered vectors that all pass the audit put the attacker's document first for 25% of targets. Verifying quantisation codes instead, with ambiguity decided by the verifier, a calibrated guard band and canonicalisation, accepted every honest vector in our samples on three platforms and left the publisher no choice. We also show that common floating-point and dynamic-INT8 inference pipelines are not bit-reproducible across batch sizes or architectures, and report pre-registered negative results on abstention, context pruning and semantic screening. We release code and data.

Wed 4 NovCryptography and SecurityInformation RetrievalMachine Learning
The gist
The paper finds that some offline systems that use text and pre-stored vectors to answer questions can be tricked because the vectors don’t always perfectly match the texts they represent. The authors show that attackers can change these vectors to hijack queries without detection under current checks. They also test different ways to verify the vectors to prevent attacks and find more reliable methods involving verifying small codes rather than full vectors. They demonstrate their findings on small devices like a Raspberry Pi and provide code and data for others to use.
Open → 2611.06498v1

BtsPruner cuts visual tokens to speed up large vision language models

BtsPruner: Role-Aware Visual Token Pruning for Large Vision-Language Models

Abstract: Large Vision-Language Models (LVLMs) often process hundreds or thousands of visual tokens, resulting in substantial computational and memory overhead during inference. Existing training-free pruning methods typically estimate token importance using language-model attention, visual attention, or feature similarity. However, these methods treat visual tokens with uniform criteria and overlook that tokens may serve different roles during visual encoding. In this work, we investigate the evolution of visual token roles in LVLMs and observe that tokens progressively transition from local detail carriers to globally referenced representatives as encoding proceeds. Based on this observation, we propose BtsPruner, a training-free role-aware visual token pruning framework that preserves complementary token roles under a fixed budget. Specifically, BtsPruner first identifies the token-role transition point based on the evolution of attention concentration. It then discovers globally referenced representatives from later encoding stages and recursively retrieves local detail tokens from earlier stages to recover complementary fine-grained evidence. Extensive experiments across LLaVA-1.5, LLaVA-NeXT, Qwen-VL, InternVL, and CogVLM demonstrate that BtsPruner achieves superior performance retention under aggressive token reduction. On LLaVA-1.5-7B, BtsPruner preserves 91.6% of the dense model's performance using only 32 of 576 visual tokens (5.6%). In the high-resolution LLaVA-NeXT-7B setting, it achieves up to a 5.95x CUDA-latency speedup.

Tue 3 NovComputer Vision and Pattern RecognitionArtificial Intelligence
The gist
Processing many image parts in big vision-language AI models is slow and uses a lot of memory. The authors found these parts change roles as they are analyzed, from showing fine details to representing the whole image. They created BtsPruner, a method that saves important parts at each stage without extra training, keeping a mix of details and summaries. This method keeps most of the model’s accuracy while using far fewer image parts, making the AI run much faster.
Open → 2611.04537v1

In-DRAM computing speeds up language model decoding on edge devices

Accelerating LLM Decoding with In-DRAM Parallelism

Abstract: Autoregressive decoding of large language models (LLMs) on edge devices is dominated by memory-bound general matrix--vector multiplication (GeMV) operations. Processing-using-DRAM (PUD) is a promising computing paradigm that can alleviate this bottleneck by reducing processor--memory data transfers and enabling massively parallel computation within DRAM (e.g., across 65,536 columns). However, weight matrices in individual LLM layers (e.g., a QKV matrix in Llama3 8B with 6,144 rows) are typically too small to fully utilize parallelism in PUD. To address this challenge, we propose an algorithm--system co-design for PUD-based LLM inference. At the algorithm level, we enable a subsequent projection to be packed with the current projection by reformulating it as a computation on the current residual state plus a correction for the residual update. Layer redundancy allows this correction to be approximated using fewer bit-serial steps. At the system level, we tailor data mapping and execution to improve utilization of either columns within DRAM subarrays (SIMD) or independently controlled banks (MIMD). Overall, we achieves up to $1.41\times$ end-to-end decoding speedup over the PUD baseline, corresponding to a $2.45\times$ speedup over processor-based execution.

Mon 2 NovHardware Architecture
The gist
Running large language models on small devices is slow because the processor spends a lot of time moving data between memory and itself. The authors focus on a way to do some of the calculating directly inside the memory chip (DRAM) to speed things up. They redesigned both the math steps and how data is organized so the memory can work on many parts of the problem at once, making the process faster. Their approach shows a clear speed improvement over the usual methods.
Open → 2611.02401v1

Spiking neural network pruning boosts efficiency on hardware accelerators

SparseForge: Beyond Spike Sparsity toward Hardware-Exploitable Neuron Pruning for Efficient SNN Acceleration

Abstract: Spiking Neural Networks (SNNs) provide an event-driven paradigm for energy-efficient edge computing. However, operation-level sparsity in SNNs does not necessarily eliminate the hardware costs associated with redundant output neurons, including neuron-state maintenance, partial-sum (psum) accesses, and on-chip storage requirements. This paper presents SparseForge, a hardware-aware sparse SNN acceleration framework that jointly optimizes neuron pruning, sparse dataflow, and accelerator microarchitecture. At the algorithm level, we introduce a gated leaky integrate-and-fire (G-LIF) neuron to learn differentiable neuron masks with binary spike semantics. A dual regularization scheme is further developed to align pruning with both synaptic-operation reduction and psum buffer capacity reduction. At the dataflow level, we design a sparse Column-Wise Product (CWP) dataflow with a variant of incremental spike encoding to improve weight-access regularity and reduce sparse-index overhead under dynamic spike activity. At the architecture level, we develop a mask-aware sparse processing unit (SPU) that supports compressed spike decoding, sparse implicit im2col conversion, neuron-mask-gated psum updates, and decoupled sparse execution. Experimental results across multiple SNN backbones and datasets show that SparseForge retains 1.75%--77.99% of neurons and 0.16%--62.36% of synaptic operations, with a configurable accuracy--efficiency trade-off. The buffer-capacity-aware regularization reduces the required psum buffer capacity by up to 96.18\%. FPGA implementations on a Xilinx ZCU9EG achieve 296.94--619.49 GOPS/W across the evaluated workloads. Across matched network and dataset settings, SparseForge provides 1.24--4.17 higher FPS and 1.50--7.18 higher GOPS/W than representative FPGA-based SNN accelerators, while reducing EDP by 1.70--19.13, with benchmark-dependent accuracy--efficiency trade-offs.

Sun 1 NovHardware Architecture
The gist
Spiking neural networks (SNNs) are designed to be energy-efficient, but many neurons that do not contribute much still use valuable hardware resources. The authors create SparseForge, a system that smartly removes redundant neurons while considering hardware costs. They introduce new neuron models, clever data handling, and hardware designs to make SNNs run faster and use less energy. Tests show SparseForge keeps accuracy while greatly improving speed and energy use on real FPGA devices.
Open → 2611.01611v1

Small language model compression tested for accuracy and speed trade offs

Compression Technique Interactions in Small Language Models: A Factorial Ablation Study

Abstract: Deploying small language models on resource-constrained hardware requires compression, yet practitioners lack systematic guidance on how quantization, pruning, and post-compression recovery interact when combined. We present a factorial ablation study evaluating 72 compression configurations across four SLM architectures (2.7B-7B parameters), extended with a 48-configuration ordering analysis, benchmark accuracy validation on 30 compressed models, and robustness checks including multi-seed validation and full-corpus perplexity evaluation. INT8 quantization is effectively lossless (<0.5% accuracy degradation), while NF4 quantization combined with L1 magnitude pruning causes model collapse in all 16 tested combinations. LoRA-based fine-tuning reduces degraded perplexity by up to 97% after aggressive pruning; an ablation against standard fine-tuning without a teacher model shows the recovery is driven by LoRA adaptation itself rather than the distillation signal. Compression ordering shows no significant effect among viable configurations (ANOVA p = 0.96) but determines viability for NF4+pruning combinations. INT8 quantization alone provides 40-45% memory reduction with no quality loss. On-device CPU deployment via llama.cpp confirms these findings transfer off the GPU: 4-bit quantization runs 46-54% faster than 8-bit for a perplexity difference under 0.5 points, while un-structured pruning yields no CPU latency or memory benefit, consistent with its dense storage format.

Fri 30 OctArtificial Intelligence
The gist
Making smaller language models run on limited hardware often means shrinking them, but it’s not clear how different shrinking methods work together. The authors studied many combinations of methods like quantization (reducing the bit size), pruning (removing parts), and recovery tweaks. They found that some techniques, like 8-bit quantization, work well without hurting performance, while others can cause problems. They also showed that a fine-tuning method called LoRA helps fix quality drop from heavy pruning. Their tests cover different models and hardware, including CPUs.
Open → 2611.00464v1

Ai energy tradeoffs spotlight neuromorphic computing advantages

AI at a Turning Point: From ChatGPT through Neural Networks to Neuromorphic AI

Abstract: The tension between the established technologies of connectionist AI and their neuromorphic challengers is attracting growing attention, above all because of the enormous energy demand of the former and the low power consumption of the latter. Developing and running large AI models consumes gigawatt- to terawatt-hours of energy, while the human brain operates on a continuous 20 watts. Spiking neural networks (SNNs) counter this energy demand with a frugal approach and bring elements of self-organization into the methodological repertoire of AI: local learning instead of global error backpropagation, event-driven computing, and time as a carrier of information. They do not replace today's AI but extend it; in hybrid systems each approach can be deployed where its strengths lie. This monograph explains the field in a generally accessible yet mathematically rigorous way. Using the text-generation module of ChatGPT as an example, the principle, elegance, and computational cost of transformer-based AI are presented and compared with the coding, learning, and network-design methods of SNNs. Throughout, relevant mechanisms of the human nervous system serve as model and reference point. Topics include spike-timing-dependent plasticity, population and temporal coding, the Neural Engineering Framework, neuromorphic hardware from Loihi 2 to commercial edge processors, and the prospects for classical and neuromorphic AI to complement and extend each other.

Thu 29 OctMachine LearningNeural and Evolutionary Computing
The gist
Running today's powerful AI systems uses a huge amount of energy, much more than the human brain needs. The authors explain a different kind of AI called spiking neural networks that work more like brains and use much less power. These brain-inspired systems learn differently, react to events in real time, and process timing information. Instead of replacing current AI, they could be combined to get the best of both worlds. The paper describes these ideas clearly and compares them to popular AI like ChatGPT, including specialized hardware that supports them.
Open → 2610.39796v1

Equilibrium propagation reduces forgetting in hopfield continual learning

Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks

Abstract: Equilibrium Propagation (EqProp) offers a biologically grounded and hardware-aligned alternative to backpropagation for training Hopfield Networks. However, its behavior in continual learning settings remains largely unexplored. This work confronts catastrophic forgetting in EqProp-trained Hopfield Networks from the inside out, eliminating the need for external data buffers or architectural overhead that standard Experience Replay \textbf{(ER)} and Generative Replay \textbf{(Gen-Replay)} inevitably require. We introduce an Energy-Based Generative Replay strategy \textbf{(EB-Gen)} which exploits the intrinsic generative dynamics of Hopfield Networks, allowing a single model to act as both classifier and replay samples generator. Samples capturing past task knowledge are generated by relaxing a frozen copy of the network, enabling rehearsal solely through internally generated pseudo-data. We evaluate the proposed method in the Class-Incremental setting, using the split versions of the MNIST and CIFAR10 benchmarks and compare it with FineTune, Experience Replay, Generative Replay, Sleep-Like Replay Consolidation (SRC) and Joint training. Our results show that EB-Gen significantly improves stability while preserving plasticity, achieving a 75\% and 64\% average accuracy on Split MNIST and Split CIFAR10 respectively. These findings show that Equilibrium Propagation can be adapted to alleviate catastrophic forgetting without forgoing the traits which made it compelling in the first place.

Tue 27 OctMachine LearningArtificial Intelligence
The gist
Training artificial neural networks to learn new tasks without forgetting old ones is a big challenge known as catastrophic forgetting. The authors study a biologically inspired method called Equilibrium Propagation (EqProp) applied to Hopfield Networks and show how to use the network's own generative abilities to replay memories without storing past data externally. By generating internal pseudo-data representing past tasks, their approach helps the network remember previous knowledge while learning new tasks. They demonstrate improved accuracy on standard image classification tests where tasks are introduced one after another.
Open → 2610.37139v1

Stable one-shot federated learning enables fast model updating at edge devices

One-Shot Federated Model Editing for Device Dynamics at the Edge

Abstract: Standard federated learning (FL) requires persistent device connectivity across multiple communication rounds, making it unsuitable for dynamic environments such as vehicular networks, where devices frequently join and leave. One-shot FL offers a promising alternative by completing training in a single communication round. However, one-shot FL suffers from model solidification and cannot readily adapt to subsequent client dynamics, including new client contributions and the withdrawal of previous contributions. We therefore consider exact model editing, which removes withdrawn contributions through unlearning or incorporates newly contributed ones through incremental learning without retraining from scratch. We identify that efficient editing fundamentally hinges on learning stability. To this end, we propose F-SOSA, the first stable one-shot FL algorithm that enables rapid and exact model editing. F-SOSA leverages sub-sampling of local models with carefully tuned rate for stable aggregation and performs curvature-aware model fusion to preserve utility. We prove that F-SOSA achieves vanishing training error and reasonably small generalization error bounds. Building on this stability, we further develop two single-round editing algorithms: F-SOSA-U for unlearning and F-SOSA-I for incremental learning. They ensure that the edited model, along with all intermediate algorithmic states, is statistically indistinguishable from retraining while incurring only an expected $O(ρ)$ fraction of its cost, where $ρ$ quantifies stability. Experiments demonstrate that our approach matches retraining performance while reducing cumulative time and communication overhead by at least 36% and 22%, respectively, under realistic urban mobility simulations.

Mon 26 OctMachine LearningArtificial IntelligenceDistributed, Parallel, and Cluster Computing
The gist
Federated learning normally needs many rounds of communication, which is hard when devices connect and disconnect often, like in cars. The authors designed a stable one-shot federated learning method called F-SOSA that trains models in just one round and can quickly update the model when devices join or leave without retraining from scratch. This method edits models efficiently by removing or adding contributions and keeps model accuracy close to full retraining. Their experiments show it saves time and communication, especially in realistic urban environments.
Open → 2610.35212v1

Faster exact unlearning for federated learning without gradient sharing

Efficient Exact Unlearning for Federated Zeroth-Order Optimization

Abstract: Federated zeroth-order optimization (FedZOO) reduces the memory and communication costs of collaborative training for edge intelligence by avoiding backpropagation while communicating only random seeds and scalar coefficients. However, existing methods lack efficient mechanisms for exact unlearning to fully remove the influence of training data when users request deletion. A key insight is that total variation stability of the learning algorithm creates overlap between model distributions before and after data deletion, allowing exact unlearning to reuse computation through coupling. Based on this observation, we jointly design two stable zeroth-order learning algorithms and their corresponding exact-unlearning procedures: \textsc{FedZOO-S} obtains stability through client and mini-batch subsampling, enabling \textsc{FedZOUL-S} to perform efficient unlearning by preserving earlier updates and restarting from the first affected step. \textsc{FedZOO-NS} further combines subsampling with Gaussian noise to increase overlap between update distributions; \textsc{FedZOUL-NS} exploits this overlap through rejection--reflection coupling to reuse affected updates when the coupling is accepted. We prove exact unlearning for both algorithms: the model and saved state have the same joint distribution as retraining with the corresponding learner on retained data. We further establish stability and convergence guarantees for the learning algorithms and derive upper bounds on expected unlearning cost. Experiments demonstrate effective unlearning with competitive model utility and up to $56.1\%$ less cumulative unlearning time than re-optimization. Compared with first-order baselines, our methods reduce communication overhead by up to approximately four orders of magnitude.

Mon 26 OctMachine LearningArtificial IntelligenceDistributed, Parallel, and Cluster Computing
The gist
When many devices learn together without sharing their private data, it can be hard to completely erase a user's data if they want it deleted later. The authors found ways to carefully organize this learning so that the model updates can be partially reused, making this deletion process much quicker and exact. Their methods avoid sending complicated gradient information, which reduces communication costs a lot. They prove these methods give the same result as retraining from scratch on the remaining data and show the approach saves time in experiments.
Open → 2610.35211v1

Sparta boosts long-context LLM speed and energy efficiency on edge devices

SPARTA: A Sparse Ternary LLM Accelerator with Adaptive Attention and Unified Datapath for Long-Context Edge Inference

Abstract: LLMs on edge devices are constrained by limited memory and bandwidth. Although 1.58-bit ternary weights greatly reduce model storage, long-context inference is increasingly bottlenecked by KV-cache traffic and attention. Existing accelerators also use separate datapaths for ternary linear layers and attention, incurring hardware and execution overhead. We present SPARTA, a sparse ternary LLM accelerator for efficient long-context edge inference. SPARTA integrates: (1) a phase-aware sparse dataflow exploiting heterogeneous sparsity across compute-bound prefill and memory-bound decode; (2) a unified compute fabric mapping ternary SpMM/SpMV and bit-serial attention onto shared processing elements to exploit zero-weight and zero-bit sparsity; and (3) an adaptive two-stage attention engine combining page-level Top-k selection to reduce K-cache accesses with token-level Top-p pruning to reduce V-cache traffic. SPARTA reduces KV traffic by 4.1-4.8x over full attention and 1.3-1.5x over a state-of-the-art sparse-attention scheme, with task-dependent quality loss. Compared with NVIDIA Jetson AGX Orin, SPARTA achieves 3.9x higher generation throughput and 8.3x higher end-to-end energy efficiency on average; versus AMD Ryzen 7 9700X, the gains are 5.0x and 81.7x. It also delivers 1.6-12.0x higher throughput, 2.0-4.2x higher energy efficiency, and 1.3-51.4x higher area efficiency than FIGLUT, TeLLMe, and MCBP.

Mon 26 OctHardware ArchitectureArtificial Intelligence
The gist
Large language models can be too big and slow to run on small devices because of memory and processing limits. The authors created Sparta, a special chip accelerator that uses clever tricks to shrink the data and reduce unnecessary work during attention calculations. This lets Sparta handle long text inputs faster, using less energy, all on devices with limited resources. It also combines different tasks into one unified system to avoid extra hardware costs and delays.
Open → 2610.34824v1

BatchNorm state causes false forgetting signals in edge split learning

Phantom Forgetting: How BatchNorm State Fakes Deletion in Edge Split Machine Unlearning

Abstract: Split learning is a natural fit for edge intelligence: a resource-constrained edge device runs the first layers of a network and offloads the remainder to a server, so that raw data never leave the device. When such a system must unlearn revoked or sensitive data for example to honor a right-to-erasure request, the deletion is commonly assessed through changes in forget-set accuracy. However, inference on an edge-server split also depends on non-learnable model state distributed across the two parties, and this state is especially volatile on edge devices subject to data drift, over-the-air updates, and checkpoint rollback. This paper isolates a state-integrity question with direct edge-security implications: can Batch Normalization (BN) running statistics create a target-specific forgetting signal even when every learnable weight remains fixed? We define a state-only diagnostic that separates learnable-parameter change from BN- buffer change and localizes the effect to the edge (client) side, the server side, and joint placements. We also introduce a state-induced forgetting gap (SIFG) that discounts simultaneous retain-set degradation. Over all ten CIFAR-10 forget classes, centralized BN recalibration on forget data reduces mean forget accuracy by 61.50 percentage points with zero weight change; in a split model, edge-only, server-only, and joint recalibration produce mean drops of 37.27, 41.71, and 51.90 points. The server-side state yields the largest selective gap relative to retain degradation, while joint recalibration causes the largest overall prediction change. These results establish distributed normalization buffers including those held on the edge device as security and privacy-relevant inference state, and show that forget accuracy should not be interpreted, or accepted as a deletion certificate, without an explicit edge-aware state-integrity audit.

Mon 26 OctCryptography and Security
The gist
When a small device and a server share parts of an AI model, sometimes the shared settings called Batch Normalization can make it look like the model has forgotten certain data even when no actual learning changes have happened. The authors show that these running statistics on both the device and server sides can fake forgetting, which means just checking model accuracy isn’t enough to prove data deletion. This matters for privacy and security when trying to erase sensitive information from AI systems at the edge.
Open → 2610.34800v1

Scheduling inference workloads reduces delays and memory costs

Queuing for Inference: Dynamic Memory Footprints and Costly Preemption

Abstract: Modeling inference workloads for transformer-based neural networks poses unique challenges due to their dynamic memory footprints, head-of-line blocking caused by heavy-tailed request distributions, and memory-dependent preemption overhead arising from the need to load/unload cache state. In this paper, we introduce a new queuing model that captures the impacts of these features on system stability and scheduling design. We show that the Gittins-index policy, which minimizes mean response time in the standard M/G/1 setting, can incur substantial preemption overhead in our model. Motivated by this limitation, we propose the Multi-Level-Priority-Threshold (MLPT) policy, that limits preemption while preserving priority for short jobs. We characterize the mean response time under MLPT and provide empirical rules-of-thumb to solve the resulting threshold-optimization problem. Our key observation is that optimal scheduling is a balance between reducing the response times of shorter jobs and limiting the overhead incurred when preemption occurs. These results provide a theoretical framework for understanding scheduling in memory constrained queuing systems for inference tasks.

Sat 24 OctPerformanceNetworking and Internet Architecture
The gist
Inference tasks for transformer neural networks can have tricky memory needs and delays caused by long waits, partly because of how tasks interrupt each other. The authors created a new queuing model that takes these issues into account to better understand scheduling performance. They found that a popular scheduling method, the Gittins-index policy, sometimes causes lots of overhead due to interruptions. To fix this, they proposed a new method called Multi-Level-Priority-Threshold (MLPT), which limits interruptions but still helps short tasks finish quickly. Their work balances quick responses with keeping interruption costs low and offers a framework for scheduling in memory-limited inference systems.
Open → 2610.33923v1

Interflow speeds up private edge ai by pipelining randomness generation

Interflow: Breaking the Offline-Online Barrier for Private AI at the Edge

Abstract: Privacy-preserving machine learning (PPML) offers a promising path toward bringing powerful AI services to the edge without exposing sensitive user data. However, its practical deployment remains constrained by the high communication, computation, and data-movement costs of secure multi-party computation (MPC), particularly during nonlinear inference. Existing MPC systems typically decouple input-independent correlated randomness generation from online execution, creating substantial memory pressure to buffer single-use cryptographic data. Moreover, limited storage can force a stop-and-refill execution pattern that repeatedly stalls the pipeline to replenish these buffers, bringing randomness generation back onto the critical path. We propose Interflow, an algorithm-hardware co-design that rethinks this separation by batch-wise processing and pipelining correlated-randomness generation with online execution: while the current batch consumes pre-generated randomness to execute secure nonlinear operations, correlation generation proceeds concurrently for the next batch. By leveraging interleaved execution, efficient data movement, and hardware acceleration, the design transforms preprocessing from a serialized, storage-intensive offline phase into a continuously overlapped pipeline that generates correlated randomness closer to the point of consumption, reducing memory footprint and data movement overhead. Implemented on the Zynq UltraScale+ ZCU102, Interflow improves nonlinear evaluation throughput by up to 1.55x for CNN operators and 1.48x for transformer operators. It achieves up to 1.26x speedup for ResNet-50 and up to 1.21x speedup for BERT-base, and reduces memory footprint by up to 37%, compared with baseline frameworks.

Tue 20 OctHardware ArchitectureCryptography and Security
The gist
Running private AI on devices like phones is hard because it needs special secret data that takes up a lot of space and slows things down. The authors found a way to make this secret data while the AI is working, instead of making it all beforehand and storing it. This new method uses hardware and smart scheduling to make AI run faster and use less memory without risking privacy. It works especially well for popular AI models like ResNet and BERT.
Open → 2610.27431v1

Model obfuscation protects edge AI devices against cross-node attacks

Mind Your Boundaries: Boundary-Aware Model Obfuscation against Cross-node Attack

Abstract: With the advancement of powerful AI chips, on-device machine learning is gaining traction for its low latency and privacy protection. However, when edge devices are compromised, attackers can obtain white-box access to the deployed models by reversing the apps directly to obtain the knowledge of peer devices, which is known as Cross-Node Attacks, like performing efficient adversarial attacks. Most existing defenses target known vulnerabilities, yet white-box access still enables attackers to uncover more unknown weaknesses. This creates a critical risk for edge-device deployments, where one compromised device can endanger all models distributed from the same source. Motivated by findings that the model robustness is primarily influenced by the curvature of the decision boundary, we investigate how model parameters shape its curvature. In this paper, we propose a novel method, BMO, that obfuscates model parameters to diversify the decision boundary further resisting cross-node attack while preserving its benign accuracy. Specifically, we explore decomposing model parameters into boundary-position and boundary-curvature components based on null-space constraints, then models with differentiated decision boundaries are instantiated by injecting noise into the boundary-curvature component. Across all four major types of adversarial attacks with four widely-deployed on-device models, BMO achieves an average success rate of 68.68% in defense, far exceeding the most competitive baseline, which attains only 22.32% in defending against cross-node attacks. Moreover, BMO achieves comparable performance in benign accuracy compared with baselines.

Mon 19 OctCryptography and SecurityArtificial Intelligence
The gist
When devices like phones or smart gadgets learn on their own, hackers can sometimes steal the secrets of the AI inside by getting control of one device. The authors found a way to make AI models harder to copy and attack by tweaking how their decision boundaries curve, without hurting their accuracy. They split the model’s parameters into parts controlling position and curvature of the decision rules and add noise to the curvature part. This makes it much harder for attackers to use one stolen model to break into others.
Open → 2610.27135v1

Hardware software design reveals better formats for efficient llm inference

MX-Like Tensor Core for LLM Inference with NPU Integration via Hardware-Software Co-Design Exploration

Abstract: Block-scaled quantization formats such as OCP Microscaling (MX) and NVFP4 have been widely adopted as efficient data formats for LLM inference. However, the design space of these formats spanned by shared scale, weight, and activation format combinations has not been systematically explored, and the joint accuracy-silicon tradeoff across this space remains uncharacterized, leaving the true Pareto-optimal configurations for NPU-based edge LLM deployment an open question. To bridge this gap, we conduct a joint hardware-software (HW-SW) co-design exploration of the above underexplored MX-like design space with three innovations. 1) We define and systematically characterize the full 8-bit scale format family within the MX-like design space, jointly with asymmetric weight-activation precision, identifying configurations that outperform the MX and NVFP4-like (without per-tensor scaling, PTS) baselines. 2) We build a per-format specialized MAC datapath template that enables fair hardware cost evaluation across the space. 3) We develop an automated co-design flow, integrated into the open-source SNAX NPU platform, that enables joint HW-SW analysis across the design space. Overall, our co-design exploration reveals that scale format matters only under aggressive FP4 quantization, yielding a case-oriented design guideline for both scale format and asymmetric weight-activation precision. We further identify UE5M3 as a favorable accuracy-hardware tradeoff for the FP4 regime, reducing perplexity degradation by up to 25.3% relative to the NVFP4-like baseline, at a cost of only +0.06% area and +0.49% power overhead.

Mon 19 OctHardware Architecture
The gist
Running large language models (LLMs) on small devices is challenging because they require lots of computing power and memory. The authors studied different ways to simplify the numbers the models use (called quantization formats) to find combinations that balance accuracy and hardware cost better. They created a method to test these formats on a special processor and found certain formats that keep accuracy higher without much extra chip area or power. Their work helps guide future hardware and software designs for efficient on-device AI.
Open → 2610.26083v1

VesperCSI improves Wi-Fi sensing data capture on edge devices

VesperCSI: A High-Performance and Extensible CSI Acquisition Framework for Edge Wi-Fi Sensing

Abstract: Channel State Information (CSI) captures fine-grained changes in Wi-Fi channel propagation and provides a foundation for device-free sensing applications such as activity recognition, localization, and occupancy detection. The development of such applications depends on an acquisition system that can handle high-rate CSI streams while remaining extensible to the requirements of individual experiments. VesperCSI is an open-source, GPL-3.0-licensed redesign of FeitCSI for CSI acquisition with Intel AX200 and AX210 Wi-Fi interfaces. It targets high-rate acquisition on amd64 and aarch64 Linux hosts using fixed-capacity preallocated object pools and separate capture and consumer threads. A loadable plugin interface supports experiment-specific monitoring, storage, and processing. We evaluate VesperCSI and FeitCSI on Raspberry Pi CM4 receivers in a burst-traffic software crossover. At a 600-MHz all-core frequency ceiling and a target rate of 2500 frames/s, VesperCSI achieves mean sequence retention of 99.017-99.087% in 10,000-frame bursts, compared with 43.972-46.553% for FeitCSI. This ordering persists after exchanging the userspace programs between two receivers while retaining their respective drivers. The comparison therefore characterizes the tested software-and-driver packages rather than isolating userspace effects. These results support VesperCSI as a reusable acquisition component for edge-hosted Wi-Fi sensing under the evaluated workload and platform conditions.

Thu 15 OctNetworking and Internet ArchitecturePerformance
The gist
Wi-Fi signals carry detailed information about how they travel around devices and spaces, useful for detecting activities or locations without needing to carry devices. The authors developed VesperCSI, a new software tool that collects this Wi-Fi channel data faster and more reliably on common small computers than older tools. Their tests show VesperCSI keeps almost all data frames at very high speeds, while the older tool loses more than half. This makes VesperCSI a better base for real-time Wi-Fi sensing applications on edge devices.
Open → 2610.22581v1

Local image model streaming limits data theft from captured devices

When Rotation Bounds Nothing: Streaming Image Classifiers to Field Nodes Under Physical Capture

Abstract: Field nodes run image classifiers locally, exposing a first-class asset, the trained model, to theft. Trusted-execution enclaves, collaborative-inference splits, and encrypted storage assume the device owner is the adversary from deployment. Here the operator owns the node, and an adversary seizing it reads only what cold-boot capture finds resident. One answer stores nothing: a chunkable multi-exit model streams into volatile memory, rotating to bound residency. We characterise the capture envelope on real LTE traces across two victims of one architecture, against passive and adaptive adversaries. Neither survives a capable adversary; recovery cost alone differs. Against a from-scratch victim the uncaptured stages must be learned: the adaptive adversary gains most at shallow seizures, ends the CIFAR-10 envelope at every rotation, and stays rotation-bounded on CIFAR100, from 42.6% of usable seizures at 5 s to 86.8% at 30 s. Against the same architecture sliced from a public pretrained backbone, the adversary downloads those stages, recovering at least 90% of maximum accuracy at all depths, taking usable CIFAR-100 seizures from 0.0% to 97.3% at 5 s, given the victim's full labelled training set. Delivery-side secrecy raises a seizure's price; a public copy of the withheld stages erases it.

Thu 15 OctCryptography and Security
The gist
Devices that run image recognition locally can have their trained models stolen if someone takes the device. This paper looks at a way to reduce theft by streaming only parts of the model temporarily into memory, so nothing stays on the device long enough to steal easily. The researchers tested this streaming idea on real network data but found that a skilled attacker could still recover most or all of the model depending on how it was trained and shared. The paper shows how different approaches to sharing model parts affect how hard it is for an attacker to steal the model.
Open → 2610.21276v1

Fpga spiking neural network boosts pattern learning with low energy use

An FPGA-Based Spiking Neural Network for Spatiotemporal Pattern Learning and Classification

Abstract: Spiking Neural Networks (SNNs) are promising for energy-efficient spatiotemporal pattern recognition at the edge, but efficient deployment and on-device adaptation require specialized hardware. In this work, we present an FPGA implementation of multilayer SNNs for spatiotemporal pattern classification, in which both inference and training take place on the same hardware. Our design adopts a temporal-forward-only training algorithm that avoids the memory and latency overhead associated with traditional Backpropagation Through Time (BPTT). A mixed-precision scheme balances training stability with resource efficiency, and architectural optimization maximizes hardware and memory utilization. We further identify and resolve hazards that arise when inference and learning share the same memory, including port conflicts between the forward and backward access patterns and read-after-write dependencies in floating-point potential accumulation. Experimental evaluations on MNIST and five multivariate time-series classification datasets demonstrate that, with fully on-chip training, our SNN model achieves accuracy comparable to SNN or LSTM trained with BPTT. Compared to CPU and GPU based implementations, our FPGA design delivers 2.5 to 6 times higher throughput while consuming an order of magnitude less energy per sample.

Mon 12 OctArtificial IntelligenceMachine Learning
The gist
Spiking Neural Networks (SNNs) can recognize patterns over time in an energy-efficient way. The authors built a special hardware design using FPGAs that can both learn and make decisions directly on the device, without needing a computer. Their approach avoids complex calculations usually required and works with fewer resources while keeping accuracy high. Tests showed this system runs much faster and uses much less energy than typical CPU or GPU setups.
Open → 2610.17463v1

Hardware-aware pruning improves large language models on edge devices

Heterogeneous Structured Pruning Under Resource Constraints for Hardware-Efficient Large Language Models

Abstract: Training-free pruning removes weights from pretrained large language models to reduce their deployment cost. However, existing approaches typically consider only the total number of retained weights. Consequently, their pruning decisions do not account for how target hardware stores and processes those weights. These methods select a sparsity structure in advance and apply it uniformly to all projections, rather than accounting for how each structure trades pruning error against latency and memory across sublayers. We propose Heterogeneous Structured Pruning (HSP), a hardware-aware pruning framework that combines two components: (i) a heterogeneous search space of dense, unstructured, 2:4, block-sparse, and head / channel (H/C) pruning options and (ii) structure-regularising constraints based on latency and memory models. Our contribution generalises existing importance-based methods: a global solver selects one option per sublayer to minimise proxy pruning loss under a resource budget, thereby assigning each projection a structure and density. We evaluate HSP on OPT and LLaMA models for an NVIDIA Jetson Orin Nano Super edge device, where it improves the quality-resource frontiers over training-free baselines. A project page and updated links for the related code is maintained at: cameronbarker.me/publications/hsp.

Sun 11 OctMachine LearningHardware ArchitectureComputation and Language
The gist
Large language models can be too big and slow to run on small devices like phones or edge computers. The authors propose a new way to shrink these models by carefully removing parts of them while thinking about how the device stores and uses the remaining parts. Instead of applying the same shrinking pattern everywhere, their method picks the best way to prune each part of the model to save space and speed up responses without much loss in quality. They tested their method on real devices and popular models, showing it works better than simpler pruning methods.
Open → 2610.15781v1

LLM guided design space search boosts AI workload efficiency on edge chips

Agentic Design Space Exploration for Joint Hardware Configuration Selection and Mapping of AI Inference Workloads on Heterogeneous Edge SoCs

Abstract: Modern edge Systems-on-Chip (SoCs) integrate heterogeneous processing units (PUs) such as CPUs, GPUs, and NPUs, each with distinct performance and energy characteristics. Deploying AI inference workloads on them under real-time latency and energy constraints requires jointly mapping workloads to PUs and configuring each PU (e.g., selecting the number of active cores and the operating frequency). This joint space grows combinatorially, making exhaustive search infeasible. Most prior work on design space exploration (DSE) applies black-box optimization (BBO) such as evolutionary search, where each evaluation returns only aggregate metrics such as latency and energy. Recent LLM-guided DSE relies on the same sparse feedback. We observe that this limits its efficiency: it offers no insight into the design space or the reasons a design choice performs the way it does, and it leaves the reasoning abilities of LLMs largely unused. We present TraceDSE, an agentic DSE flow that performs joint workload mapping and PU configuration selection for AI inference on heterogeneous SoCs. TraceDSE is an iterative proposer-critic loop driven by richer feedback in the form of system execution traces. The LLM proposer agent generates candidate mappings and PU configurations for hardware evaluation. The LLM critic agent, equipped with programmatic trace-analysis tools, analyzes the traces to identify bottlenecks and suggest targeted refinements. This loop yields deeper insight into each design point, higher-quality decisions, and a more effective search. Across four AI inference workloads (models of varying complexity and a multi-model pipeline) on an Intel Meteor Lake SoC, TraceDSE consistently outperforms two state-of-the-art BBO tools, improving Pareto frontier hypervolume by up to 35% over NSGA-II and up to 68% over Bayesian optimization, while requiring ~6-9x fewer hardware evaluations.

Mon 5 OctHardware ArchitectureArtificial Intelligence
The gist
Modern devices use chips with different types of processors to run AI tasks efficiently. The authors noticed that traditional methods to pick the best processor settings and task assignments are slow because they look only at overall results like speed and energy used. They created a new way using AI language models that look deeper into detailed system activity to suggest smarter settings and improve how quickly and efficiently AI tasks run. Their method finds better solutions with fewer tests on a real chip.
Open → 2610.07191v1

Joint optimization improves edge sensing computation and communication

Learning OTA: A Unified Framework for Edge Sensing, Computation, and Communication

Abstract: Most federated learning (FL) studies implicitly assume that training data is readily available at participating edge devices. In practice, this data must first be acquired through heterogeneous sensors with potentially different measurement modalities, and the quality and cost of this acquisition process can directly affect the learning performance. This dependence creates a coupled relationship among sensing, computation, and communication, necessitating a system model that captures their joint effect on resource efficiency and learning accuracy. This paper develops such a unified treatment for an over-the-air (OTA) FL system in which each device configures its sensing modality, power, resolution, and sample size, with aggregations conducted OTA at the server. We derive models for sensing noise, energy consumption, latency, and OTA aggregation distortion, and establish a non-convex convergence bound quantifying how these factors influence the global model trajectory. Guided by this analysis, we formulate a per-round optimization that balances energy, latency, and learning quality under physical device constraints. The resulting mixed optimization is addressed through an alternating procedure with closed-form and one-dimensional updates for continuous variables and a lightweight discrete search. Numerical experiments corroborate the advantage of jointly optimizing sensing, computation, and communication decisions.

Sat 3 OctInformation TheoryNetworking and Internet Architecture
The gist
Getting good data from many different sensors on devices like phones or cameras is tricky because the sensors, computing power, and communication all affect each other. The authors looked at how these factors work together when devices send updates to a central server using a wireless method that mixes signals in the air. They made a model that captures how noise, energy use, and delay impact the learning process. Their work helps decide the best way for devices to set sensor quality, power, and data amount to speed up learning while saving energy. Experiments show their approach can improve learning by balancing sensing, computing, and communicating well.
Open → 2610.04718v1

Adaptive system improves online learning for changing data streams

AIGS: Adaptive Incremental Gating System for Online Representation Learning in Non-Stationary Data Streams

Abstract: Real-time data streams in Web of Things (WoT) and edge computing environments often evolve through latent regime changes. For online representation learning under strict computational constraints, the central problem is resolving the stability-plasticity dilemma: keeping useful historical knowledge while rapidly reacting to concept drift. Existing methods employ fixed update schedules or rolling windows. However, they suffer from parameter ossification during sudden shifts and waste computational resources when the stream remains stable. This paper proposes the Adaptive Incremental Gating System (AIGS), a lightweight closed-loop state-aware adaptation framework. AIGS introduces the Shock Ratio, an endogenous residual feedback mechanism that normalizes current reconstruction error against recent variation. This signal drives a Continuous Plasticity Controller that smoothly interpolates between learning plasticity and memory retention. By treating representation learning as a closed-loop control mechanism, AIGS avoids catastrophic forgetting and maintains a strictly linear $\mathcal{O}\left(k\cdot d\right)$ per-step complexity suitable for latency-sensitive edge devices. Experiments on real-world smart city dynamic streams-spanning traffic networks, meteorological systems, and industrial infrastructure-demonstrate distinct domain-dependent advantages. On Electricity Transformer Temperature datasets, AIGS achieves preventative early-warning lead times of 8.31 (ETTm1) and 9.88 (ETTm2) steps under gradual degradation. On Performance Measurement System traffic datasets, it shows significantly faster post-shift recovery after abrupt mutations. On the highly noisy Weather dataset, it improves anomaly recall while resisting stochastic noise overfitting. These findings establish AIGS as a practical, plug-and-play adapter for resource-constrained edge monitoring systems.

Fri 2 OctMachine Learning
The gist
Data from sensors often changes over time, making it hard for computers to keep learning without forgetting old information. The authors created a system that can quickly adjust how much it learns based on how much the data is changing. This helps it react faster to sudden shifts while still remembering important past patterns. Their system works well on different real-world data, like city traffic and weather, and runs efficiently on devices with limited computing power.
Open → 2610.02661v1

Federated learning adapts channels for better edge device accuracy

FedSAP: Federated Learning with Structured Adaptive Partitioning for Multi-Domain Heterogeneous Edge Devices

Abstract: Federated learning (FL) on heterogeneous edge devices must jointly accommodate unequal resource budgets and domain-shifted local data. Existing resource-adaptive methods decide how much of a model each client trains but not where retained capacity should reside or how it should be shared, whereas federated domain-generalization methods usually assume a shared full architecture. Uniform compression can therefore discard high-utility channels, and a single aggregation path can mix transferable features with domain-sensitive updates. We propose FedSAP, a domain-aware heterogeneous FL framework that casts structured pruning as budget-constrained tri-state channel allocation. FedSAP converts each keep ratio into non-uniform layer budgets, assigns stable channels to a Global pool, useful domain-sensitive channels to pseudo-domain-specific Private pools, and low-utility channels to a Dropped state. This partition lets broadly useful features benefit from cross-client pooling while isolating domain-sensitive updates from incompatible clients. Domain-Guided Assignment infers pseudo-domains from shallow-gradient similarity, while Type-Matched Aggregation restricts each channel to its intended sharing scope. Across three random seeds, FedSAP reaches 76.00% and 72.67% mean global accuracy on Digits and Office-Caltech, exceeding the strongest baseline by 1.70 and 4.92 percentage points while supporting client pruning ratios of up to 80% across heterogeneous clients.

Thu 1 OctMachine Learning
The gist
Many devices that learn from data have different abilities and unique local information, which makes training a shared model tricky. The authors propose FedSAP, a method that decides how much each part of a model is kept or shared across devices based on what is useful globally or only for some device groups. This way, widely useful knowledge is shared, while device-specific details stay separated, improving overall accuracy. They tested this on image recognition tasks and saw better results even with heavily pruned models on uneven devices.
Open → 2610.01638v1

Hardware method improves safety and efficiency of small language models

MOMAT: Mixture of Multiple Atlases for Low-Power Jailbreak Defense of Quantized LLMs

Abstract: Quantized large language models are increasingly deployed on edge devices for their low latency and energy efficiency. However, model quantization weakens alignment safeguards, leaving qLLMs (quantized large language models) highly vulnerable to jailbreak attacks. To address this challenge, we present MOMAT (Mixture of Multiple Atlases), a hardware-enhanced safety framework that combines structured knowledge retrieval with low-power defense acceleration. Each atlas represents a semantic cluster of harmful or benign sample sets and policy templates, enabling domain-localized Retrieval-Augmented Generation guarding that mitigates the curse of dimensionality and the resulting semantic sparsity problem in large, heterogeneous safety databases. MOMAT retrieves top-$k$ similarity features from all atlases for each prompt and evaluates them using a lightweight MoE (Mixture of Experts) detector, while a CiM (Compute-in-Memory)-accelerated similarity engine performs fast, low-power atlas-local retrieval. MOMAT's CiM-based retrieval accelerates a 100-query batch from 15,052.44 ms to 3,207.21 ns (a $4.69 \times 10^6\times$ speedup) and reduces energy from $8.1 \times 10^7$ $μ$J to 3.32 $μ$J, yielding an approximately $2.5 \times 10^5\times$ energy reduction over DRAM-based (Raspberry Pi) baselines. Red-team evaluations across standard benchmarks show that MOMAT matches the defense performance of state-of-the-art methods while avoiding benign overkill and providing substantial efficiency gains, demonstrating that CiM-based modular defenses can make edge-deployed qLLMs both safer and more energy-efficient. We will release the full 223.2k-sample dataset to foster future research.

Thu 1 OctCryptography and SecurityArtificial Intelligence
The gist
Quantized large language models (qLLMs), which run on low-power devices, are vulnerable to tricks that make them behave badly. The authors propose MOMAT, a system that uses multiple groups called atlases to organize safe and unsafe examples, helping the model better recognize harmful inputs. MOMAT uses special low-energy hardware to speed up checking prompts against these atlases, making the safety process both faster and much less energy-intensive. Tests show MOMAT keeps qLLMs safe while saving a lot of power and running much faster than older methods.
Open → 2610.01058v1

Faster federated learning with smarter client and data choices

Client and Training Data Selection for Computationally Efficient Synchronized Federated Learning

Abstract: Federated learning (FL) is a promising paradigm of machine learning, which preserves user privacy by enabling learning without sharing raw data with a cloud server. Straggling clients have been a problem for FL as they introduce delays in aggregating the local models and hence, the convergence of the global model. Therefore, it is important to have a mechanism that ensures fast convergence of the global model as well as good FL participation rate. Another issue for the convergence of a model in FL is the non-independent and identically distributed (non-iid) data across the clients. Prior approaches based on probabilistic client selection do not work well under non-iid data especially when the number of clients is small. We show scenarios where such approaches fail and propose a joint client-training data selection algorithm for fast convergence of FL models. Our experiments on CIFAR-100 dataset show that convergence of the FL model can be significantly improved over prior works that can consider non-iid data and heterogeneous computation and higher model accuracy.

Wed 30 SeptMachine LearningDistributed, Parallel, and Cluster Computing
The gist
Federated learning is a way to teach computers without sharing personal data. One problem is some users take longer to send updates, slowing down the whole learning. Another is that data stored by users can be very different, which makes learning tough. The authors show that picking which users and data to use together helps the learning process go faster and get better results. They tested this idea on a popular image dataset and found it works better than earlier methods.
Open → 2609.39250v1

Server method keeps AI from forgetting old tasks in federated learning

ReSCENE: Server-Side Replay for Structural Mitigation of Catastrophic Forgetting in Federated Continual Learning

Abstract: Federated continual learning must integrate new tasks over time without losing earlier-task knowledge. Most existing methods attach an anti-forgetting mechanism to the client-trained, server-aggregated loop of federated learning, which holds back new learning to preserve earlier knowledge and burdens resource-constrained clients. We propose ReSCENE, which structurally mitigates catastrophic forgetting by having each client upload a small condensed surrogate of its local data while the server keeps the surrogates of past tasks and trains the global model on them together with the current task surrogates. For efficient server memory, we introduce temporal herding, which selects the more recent surrogates from the pool accumulated over a task into a compressed buffer. Our study provides a theoretical analysis showing that this buffer can represent the original task data more closely than full accumulation of all surrogates. Across CIFAR-10, CIFAR-100, and TinyImageNet, ReSCENE achieves the strongest accuracy over seven baselines, by up to $31.1$ points of average accuracy, while requiring as little as $0.11\times$ of the client computation and up to $179\times$ less upload than the model-update baselines. ReSCENE further demonstrates its effectiveness when scaled to larger client populations and larger models while remaining efficient, which makes it a practical method for federated continual learning.

Wed 30 SeptMachine Learning
The gist
When multiple devices train a shared AI model step-by-step with new tasks, the model can forget what it learned before. The authors propose ReSCENE, a way for each device to send a small summary of its data to a central server. The server then uses these summaries from past and current tasks to train the AI, helping it remember old knowledge without overloading the devices. This approach saves device power and internet data while improving overall accuracy, making continual learning more practical.
Open → 2609.38833v1

Megartron chip boosts edge AI speed and density with mixed memory design

MEGATRON: a 28nm Analog PCM CiM/Digital System-on-Chip for Edge GenAI at 57.5 TOPS/W and 1.52 Mparam/mm${}^2$

Abstract: We present MEGATRON, a heterogeneous Edge GenAI System-on-Chip in 28nm FD-SOI CMOS technology combining a non-volatile analog in-memory-computing engine based on a 4Mi-cell phase-change memory (PCM) array with a digital RISC-V-based flexible neural processing unit. MEGATRON demonstrates up to 3.5 TOPS/W using the RISC-V processors and 57.5 TOPS/W with PCiM, at a storage density of 1.52 Mparam/mm${}^2$ with 4-bit effective weight precision.

Mon 28 SeptHardware Architecture
The gist
Making smart devices work faster and use less power is a big challenge. The authors built a new computer chip called Megatron that mixes young, low-power memory technology with traditional processors to help AI work better at the device level. This chip can do many AI calculations very quickly while taking less energy and storing more data in a small space. It could help bring advanced AI to gadgets like phones or sensors.
Open → 2609.35254v1

Shared path prediction improves federated learning efficiency

CTP-FL: Common-Trajectory Gradient Prediction for Federated Learning

Abstract: Communication-efficient federated optimization commonly spends several gradient evaluations between server updates. Existing local-update methods use this computation to advance an independent model on each client. Under heterogeneous data, however, these models evaluate gradients at different locations, making the aggregated update difficult to interpret as a gradient of the global objective. We study an alternative use of the same computation budget: \emph{evaluate the global objective along a shared, predicted path}. We propose Common-Trajectory Predictive Federated Learning (\texttt{CTP-FL}). At each round, all clients construct the same sequence of query points from the current global model and the previous aggregated direction, evaluate $K$ stochastic gradients along this sequence, and upload their average. The server then performs a single global update. Thus, \texttt{CTP-FL} uses $K$ mini-batch gradients per client and one model-sized vector in each communication direction, matching the per-round computation and communication of full-participation FedAvg-M. Shared query points make the aggregated direction an unbiased estimator of the average \emph{global} gradient along the predicted path. The remaining discrepancy from the gradient at the current model is controlled by the path length, without assuming bounded client-gradient dissimilarity or bounded gradients. For smooth non-convex objectives, we establish an $\mathcal{O}\!\left( \sqrt{LΔσ^2/(NKR)}+LΔ/R \right)$ average-stationarity bound under full participation. The analysis isolates a testable trade-off: extending the prediction path provides more forward-looking gradient information but increases its displacement bias.

Mon 28 SeptMachine LearningArtificial Intelligence
The gist
Federated learning helps train AI models across many devices without sharing raw data. Usually, devices train their models independently then share updates, which can be hard to combine if devices have different kinds of data. The authors propose a new method where all devices predict a common path for updating the model and send gradient information based on that path. This makes the combined updates more reliable and efficient without extra communication. Their math shows this approach balances the benefits and drawbacks of looking ahead along this shared path.
Open → 2609.35130v1

Knowledge distillation shapes detection habits in encrypted traffic classifiers

Unknown-Traffic Detection, Calibration and Shortcut Reliance in Distilled Encrypted-Traffic Classifiers over One Year

Abstract: Knowledge distillation is the standard way to compress encrypted-traffic classifiers for the edge, and almost all such work judges students by accuracy alone. We ask what else a student inherits: unknown-traffic detection, calibration, shortcut reliance, and whether any survives a year of drift. Resemblance proves little on its own, since soft targets also regularise. We therefore distil one 101k-parameter student from two teachers of equal accuracy but different construction, a five-member ensemble and a single wider model, so that following one rather than the other is attributable to it. The design was pre-registered before any test result was seen. We tested ten hypotheses on CESNET-TLS-Year22, a year of real TLS traffic, across 18 test windows over 35 weeks. Two are supported: a student's per-flow unknown-scores shift toward its own teacher, but only at a conventional temperature, not the accuracy-optimal one; and a shortcut-reliant teacher passes its over-confidence to a student that never sees the feature. The drift prediction is reversed under both scores, the gap narrowing rather than widening and the student overtaking under the energy score in two of three replicates, as is the prediction that such a teacher harms its student's detection, which improves slightly. Shortcut reliance is set by model size, not distillation. Under the logit-based scores nothing else transfers: distillation beats neither a temperature-scaled direct student nor label smoothing. Exploratory analysis shows this turns on the scoring rule: with a feature-space detector the teacher detects unknown traffic 0.073 AUROC better than the direct student, where the energy score sees 0.000, and the conventional-temperature student inherits most of it. Label smoothing, with no teacher, recovers more. Distillation transfers the teacher's habits; what looks like an inherited ability is available without one.

Fri 25 SeptNetworking and Internet ArchitectureMachine Learning
The gist
Classifiers compress complex models to run on smaller devices using knowledge distillation, but usually just check accuracy. This paper finds that students (simpler models) inherit some traits from their teachers, like confidence or how they detect unknown traffic, but not consistently or fully. The study showed that model size impacts shortcut reliance more than distillation does, and some detection skills can be obtained without a teacher. The authors tested these effects over a year of real encrypted network traffic and found mixed results.
Open → 2609.31141v1

Neural networks use spherical harmonics to store weights continuously

SH-WRNN: Implicit Spherical Harmonics Weight Field Routing Neural Networks for Asymmetric Edge Intelligence

Abstract: Deep learning architectures remain rigidly built upon traditional fully connected layers. While networks scale up, few challenge this foundational root. In this work, we reshape this paradigm by transforming the core synapse weight matrix from static, discrete parameters into a differentiable, continuous field governed by spherical harmonics functions. We introduce the Implicit Spherical Harmonics Weight Field Routing Neural Network (SH-WRNN), which constrains weight matrices within a continuous parametric field instead of optimizing millions of localized discrete weights. When retrieving the weight matrix of the current layer, connection parameters are localized using latitude and longitude on a rectangular plane mapped from the continuous field. The latitudinal coordinate is specified by activated neurons from the previous layer, while the longitudinal coordinate is determined by keys generated from previous layer activations via matrix multiplication. By evaluating intersections on this map, the network dynamically extracts its connection weights on-the-fly. Empirical validation on MNIST demonstrates that under compact configurations of (32, 10, 10) and (32, 3, 10), SH-WRNN achieves robust accuracies of 91.05% and 81.54% within a single training epoch. Furthermore, we propose an asymmetric Surface Baking scheme. Upon convergence, the continuous weight field is baked once into a static parametric surface. By eliminating analytical spherical harmonics calculations during inference and reducing dynamic matrix extraction to high-speed localized memory slicing, this scheme achieves asymmetric algorithmic acceleration with negligible accuracy degradation. This paradigm shift bypasses GPU memory-bandwidth monopolies, opening a novel path to reshape the advantages of CPU computing. Code is available at https://github.com/jzb1111/SphericalHarmonyRoutedNeuralNetWork.

Sun 13 SeptMachine Learning
The gist
Deep learning normally uses fixed tables of numbers to decide how neurons connect, but this paper shows how those connections can be stored as smooth, mathematical surfaces instead. The authors created a new type of neural network that uses spherical harmonics—shapes defined on spheres—to represent connection weights in a continuous way. This allows the network to generate weights on demand rather than storing millions of fixed values. They tested this approach on a simple image task and got good results quickly. They also developed a way to speed up use by turning this surface into a look-up table, cutting down the calculation needed at the moment of use.
Open → 2609.14614v1

Compact AI model classifies radish potato and pointed gourd diseases accurately

AgroVisNet: A lightweight Convolutional Network and the BD-PlantDX Expert-Validated Benchmark for Radish, Potato and Pointed Gourd Disease Classification

Abstract: Automated plant disease diagnosis is increasingly deployed on farmer-held devices in regions where agronomic expertise is scarce and network connectivity is unreliable. Three obstacles limit its practical value: public benchmarks are dominated by a small set of non-native crops, region-specific datasets are rarely validated by domain experts, and the architectures that reach competitive accuracy carry parameter budgets that are unsuited to low-cost hardware. We propose AgroVisNet, a compact convolutional network trained from scratch, together with BD-PlantDX, an expert-validated benchmark of 12,432 field images spanning 12 classes of radish, potato and pointed gourd in healthy and diseased states, collected across the Bogura and Nilphamari districts of Bangladesh. AgroVisNet couples grouped bottleneck residual blocks carrying sequential channel and spatial attention with multi-scale depthwise blocks and a dual-pooling classification head, reaching 290,572 trainable parameters. On BD-PlantDX the model attains 99.52% test accuracy and 99.52% weighted F1, exceeding all six ImageNet-pretrained lightweight backbones evaluated under an identical protocol while using 8.7 to 16.8 times fewer parameters and 1.3 to 8.5 times fewer multiply-accumulate operations. Exported for deployment, the model quantises to a 0.46 MB full-integer network at a 0.22 percentage-point accuracy cost and classifies an image in 8.40 ms on a single CPU. Across five random seeds accuracy remains at 99.57 +- 0.10%, a ten-variant ablation isolates the contribution of each component, and the same architecture transfers without redesign to two independently collected datasets at 98.71% and 99.05% accuracy. Grad-CAM evidence indicates that predictions rest on lesion-bearing leaf regions rather than on background cues.

Wed 9 SeptComputer Vision and Pattern Recognition
The gist
Accurate plant disease detection on farmers' devices is often limited by large models and lack of expert-validated data for local crops. The authors created a compact AI called AgroVisNet that is much smaller but very accurate, trained on a large, expertly checked dataset of radish, potato, and pointed gourd images from Bangladesh. This model runs fast on simple hardware, identifies diseases well, and bases its decisions on visible leaf spots rather than background. It also performs well on other similar datasets without changes.
Open → 2609.10469v1