Papers for

medical data analysts

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Matthews correlation coefficient reviewed for fairer binary classification

The Matthews Correlation Coefficient: Fifty years

Abstract: Fifty years after its introduction, the Matthews Correlation Coefficient (MCC) remains central -and debated- in binary classification. This survey offers a comprehensive retrospective and an in-depth analysis of MCC, covering its historical development, mathematical foundations, and implications for practice. We clarify long-standing misconceptions, formalize its equivalences (to Pearson's correlation and to Chi2 in 2x2 tables), and examine its behavior under class imbalance. Beyond MCC, we develop analytical (derivatives) and graphical (contour plots) frameworks that generalize to any confusion-matrix-based metric. These frameworks expose structural properties -bias, robustness, sensitivity- that scalar summaries conceal, and they provide actionable guidance for selecting and interpreting performance measures in applications where reliability and fairness matter. Together, our results position MCC within a broader toolkit while clarifying when it is informative -and when alternatives may be preferable.

Thu 5 NovArtificial Intelligence
The gist
Measuring how well a computer tells two things apart can be tricky, especially when one thing happens much more often than the other. The Matthews Correlation Coefficient (MCC) has been used for 50 years to help with this, but it isn’t perfect and sometimes misunderstood. The authors explain what MCC really means and how it behaves when data is unbalanced. They also create new tools to understand similar scores better, helping teams pick the right measure for accurate and fair results.
Open → 2611.08799v1

TabCUQ improves uncertainty estimates for tabular prediction models

TabCUQ: Context-Uncertainty Quantification for Tabular Foundation Models

Abstract: Tabular foundation models use in-context learning to estimate conditional expected outcomes and average treatment effects for specific groups of individuals. Using these estimates to guide scientific decisions requires reliable uncertainty quantification. The challenge is to account simultaneously for two sources of uncertainty: limited-context bias and context variation arising from the small labeled sample supplied at inference. Predictive spread does not directly quantify both sources of uncertainty, and we observe that the ordinary bootstrap may substantially overestimate context variation, whereas subsampling may underestimate it. We introduce TabCUQ, a perturbation-based framework that injects noise into observed outcomes and applies the foundation model to the perturbed contexts. TabCUQ estimates context variation using wild-bootstrap dispersion and accounts for limited-context bias using the maximum deviation between perturbed and original estimates. For a kernel-regression analogue whose estimation error admits an exact decomposition into these two components, we show that our framework provides asymptotically valid pointwise coverage. In simulations, TabCUQ substantially improves coverage, with average interval widths typically about twice those of an oracle bias-aware benchmark. Across seven real-world tasks, it also improves coverage of reference functions learned from much larger samples.

Thu 29 OctMachine Learning
The gist
When using computer models to predict outcomes for groups of people based on data tables, it’s important to understand how uncertain these predictions are. The authors study two sources of uncertainty: not having enough context or examples, and variability due to small data samples. They introduce TabCUQ, a way to add noise and use a special technique to better measure and separate these uncertainties. Their method gives more reliable confidence intervals, making predictions safer to trust when making decisions.
Open → 2610.39558v1

Entropy measures reveal patterns in time series from brain and heart data

Revisiting entropy and entropy-related magnitudes of a symbolic sequence and their estimation

Abstract: This article revisits entropy and related magnitudes (entropy density, effective measure complexity, and informational distance) for the analysis of sequential data. We present their mathematical foundations within information theory and ergodic theory, establishing connections between probabilistic Shannon entropy and algorithmic Kolmogorov complexity. A central result, Zivs theorem, demonstrates the asymptotic equivalence between the normalised number of distinct words and entropy density, providing the theoretical basis for estimation via Lempel-Ziv factorisation. We examine the convergence properties of these magnitudes and introduce estimation procedures for finite sequences, including a method for the effective measure complexity and the transfer entropy rate. An open-source software library implementing parallel algorithms for efficient Lempel-Ziv factorisation is presented. The practical utility of the framework is illustrated through three case studies: functional brain connectivity analysis from fMRI data, characterisation of spatiotemporal dynamics in coupled nonlinear oscillators, and electrocardiogram-based classification of atrial fibrillation, demonstrating how entropic measures capture meaningful structural and dynamical properties across diverse domains.

Mon 26 OctInformation Theory
The gist
Measuring the randomness or complexity in sequences of data helps us understand underlying patterns. The authors revisit mathematical ways to quantify such complexity and connect them to known theories about information and algorithms. They show how to estimate these measures from real data using a popular compression method. Their open-source software makes this efficient. They demonstrate the approach on brain signals, oscillator networks, and heart rhythm data to find meaningful insights.
Open → 2610.35121v1

Clustering time series from dynamical systems using occupation kernels

The Clustering Occupation Kernel (CLOCK) Method for Time Series Arising from Dynamical Systems

Abstract: We introduce the Clustering Occupation Kernel (CLOCK) method, a novel representation-based framework for clustering time series arising from dynamical systems. CLOCK embeds observed sequences into a reproducing kernel Hilbert space (RKHS) via a time-averaged occupation kernel, enabling trajectory comparison through their RKHS norm. For systems governed by differential equations, this formulation rigorously approximates the kernel mean embedding of the invariant Sinai-Ruelle-Bowen (SRB) measure, meaning our dissimilarity metric approximates the maximum mean discrepancy (MMD) between attractors. To accommodate general time series, we introduce time lags via Takens' Embedding Theorem to reconstruct the phase space. Crucially, CLOCK natively handles trajectories of unequal lengths and irregularly-spaced observations. We validate the proposed metric using spectral clustering, demonstrating superior performance against benchmark methods on both synthetic datasets and a real-world intracranial large-scale electroencephalogram (EEG) dataset. Finally, ablation studies on the Lorenz-96 system are consistent with the theoretical convergence rates of the occupation kernels and highlight the method's strong robustness to zero-mean Gaussian noise.

Fri 23 OctMachine Learning
The gist
Clustering different time series data from complex systems like weather or brain signals can be difficult, especially when the data varies in length and timing. The authors introduce a new method called CLOCK that represents these sequences in a special mathematical space to compare their patterns more accurately. This method works even when the data is irregular or noisy and can handle different lengths of sequences. They tested CLOCK using simulated models and real brain EEG data, showing it often groups similar time series better than other methods.
Open → 2610.33354v1

Verifier audits can favor worse policies when policy improves

Better Policies, Worse Verifier Audits

Abstract: Verifier stacks increasingly sit inside post-training and test-time compute loops. We study a specific control loop: a component verifier is evaluated on fresh policy outputs against the full success label and refreshed when that aggregate score falls. For a conjunctive target $y = F \wedge M$, its on-policy AUC is $(1-q)A_{\mathrm{own}} + qA_{\mathrm{cross}}$, where $q = \Pr[F=1 \mid y=0]$. As the policy masters $F$, $q$ can rise toward one, removing weight from $F$-failures and rewarding information about $M$. We call this on-policy audit collapse. We derive a sharp identified set, a complete-erasure threshold, and the exact point at which the audit can prefer a worse component verifier; its population optimum is the full-outcome predictor, not the component oracle. An exact oracle exhibits the collapse, a natural multi-label text control shows inversion can select a head with lower canonical $F$-AUC at the observed failure mixture, and a prediction-registered ECG study shows that acting on the audit can reduce off-policy $F$-AUC, transfer the damage to an unseen generator, and increase downstream constraint violations. Preserving own-failure support prevents the damage. Matching label-complexity bounds and a controlled-mixture audit give the corresponding repair: audit the component on its own failures, and keep those failures in the refresh distribution.

Wed 21 OctMachine Learning
The gist
When checks called verifiers are used to evaluate policies or decisions, improving one part of the policy can hide errors in another part. The authors show this can cause the verifier to give misleadingly good scores, even preferring worse verifiers. They demonstrate this problem happens in real settings, like medical signal analysis, and provide a fix: verifiers must audit failures directly rather than relying on mixed data. This ensures more reliable evaluations and prevents errors being hidden.
Open → 2610.29029v1

Informative missing labels can change classification results depending on goals

Decision-Target Reversal under Informative Label Missingness

Abstract: Informative label missingness can improve classification efficiency because the pattern of label availability may itself contain information about the discriminant structure. We show that this advantage need not persist across deployment objectives: with the data-generating model and statistical experiments held fixed, informative partial classification can be favourable under one loss and unfavourable under another. We call this phenomenon decision-target reversal. For binary Gaussian discrimination, we derive a local cost-sensitive excess-risk expansion and, in one dimension, obtain explicit critical cost ratios at which the first-order risk ordering reverses. We show that the required covariance geometry arises under finite uncertainty-dependent missingness mechanisms with a fixed marginal missing-label proportion. We then extend the analysis to multiclass finite-action problems through loss-specific active-boundary curvature, yielding a general decision-target frontier. Monte Carlo experiments illustrate the predicted reversals, and a descriptive gastrointestinal-lesion application with agreement-defined label availability illustrates sensitivity to deployment costs. The results show that favourable informative missingness is a property of the statistical experiment relative to the deployment decision, rather than of the experiment alone.

Sun 18 OctMachine Learning
The gist
Sometimes when labels are missing in training data, the pattern of what is missing can actually help make better predictions. The authors found that whether this hidden information helps or hurts depends on what kind of mistakes are more costly when using the model. They call this change decision-target reversal, meaning a model that looks good under one goal can look worse under another. They explain this with math and examples, showing that the benefit of missing labels depends on both the data and the practical task.
Open → 2610.25128v1

Backward conformal prediction improves error estimates under shifted data

Backward Conformal Prediction under Covariate Shift

Abstract: Backward conformal prediction (BCP) fixes the size of a prediction set and estimates its miscoverage, but assumes calibration and test data are exchangeable. We extend BCP to covariate shift through importance weighting, preserving its guarantees when the density ratio is known. We show that BCP's tight miscoverage report depends only on the relative weight assigned to missed calibration examples, explaining why a better domain classifier can sometimes produce a worse estimate. Motivated by the fact that BCP depends only on nonconformity scores, we propose estimating density ratios on sorted score vectors rather than high-dimensional inputs. We prove that this approach recovers the same report asymptotically under an interpretable condition and characterize the resulting bias otherwise. Experiments with simulated and benchmark distribution shifts show that the proposed method improves BCP's miscoverage estimates, while score-based weighting consistently yields more accurate estimates than embedding-based weighting.

Mon 12 OctMachine Learning
The gist
Prediction methods often assume the data they see when learning is similar to the data where they later make predictions. This paper extends a technique called backward conformal prediction to handle situations where this assumption fails because the data distribution changes (covariate shift). The authors improve estimation of how often predictions might be wrong by adjusting for differences using importance weighting and a novel way to estimate these weights based on sorted prediction scores. Their approach leads to more accurate error estimates in various tests, helping predictions stay reliable even when conditions change.
Open → 2610.16048v1

Approximate method improves inference for regime switching differential systems

Broadly Applicable Approximate MCMC for Switching Stochastic Differential Equations Using Uniformization and Time-Conditioned Factorized Neural Likelihood Estimation

Abstract: Switching stochastic differential equations (SSDEs) describe continuous-time dynamics whose parameters switch according to a latent regime process that follows a continuous-time Markov chain (CTMC). By allowing dynamics to change between regimes, SSDEs represent heterogeneous system behavior and have been applied across diverse fields. However, Bayesian inference for SSDEs remains difficult, and existing SSDE inference methods have limited applicability, with restrictions such as noise-free observations, univariate states, linear drift, or state-independent diffusion. In this study, we propose an approximate Markov chain Monte Carlo sampler for SSDEs using uniformization and factorized neural likelihood estimation (FNLE), a simulation-based inference method. Uniformization provides an exact representation of the CTMC but requires SDE transition densities over arbitrary time intervals. We approximate these densities by training a time-conditioned FNLE model. The resulting sampler is broadly applicable to SSDEs without requiring analytically tractable transition densities. In synthetic-data experiments, our method recovered regime paths and parameters for three SSDE models for which previous methods have limited applicability. We also applied our method to a real dataset and detected a regime transition.

Wed 7 OctMachine Learning
The gist
Many real-world systems switch between different behaviors over time, like weather patterns or stock markets. These changes can be modeled by switching stochastic differential equations (SSDEs), but figuring out the hidden switching patterns and parameters is very hard. The authors developed a new approximate method using advanced math and neural networks to better estimate these hidden regimes and their dynamics, even when the math behind the system is complex or unknown. They tested their method on made-up data and a real dataset, successfully detecting regime changes where previous approaches struggled.
Open → 2610.10194v1

Active feature acquisition policy evaluation improved by separating uncertainty types

Evaluation of Active Feature Acquisition Policies with Tabular Foundation Models

Abstract: Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn and evaluate such policies from finite offline data using prior-data fitted networks (PFNs), which are off-the-shelf models that output posterior predictive distributions without task-specific training. We show that under the imbalanced coverage of offline data, using total predictive entropy as a reward creates an epistemic bias that penalizes acquiring sparsely observed features. Specifically, this reward conflates epistemic uncertainty (arising from lack of offline data) with aleatoric uncertainty (arising from uninformative features). To address this, we target the posterior expected (aleatoric) entropy instead of the total predictive entropy output by a PFN for evaluating feature acquisitions. Empirical evaluations on synthetic and real-world datasets demonstrate that our approach consistently reduces value estimation bias and yields credible intervals with strong empirical coverage, which can translate to improved downstream policy selection.

Mon 5 OctMachine Learning
The gist
Deciding which features of data to gather next can help predict outcomes better but is tricky when relying on limited prior data. The authors show that traditional methods of measuring uncertainty often mix up unknown information caused by little data and randomness in the features themselves. They propose focusing on the uncertainty that reflects real unpredictability in the features, improving how policies for collecting data are evaluated. Their tests show this approach more accurately predicts the value of acquiring features and helps choose better strategies.
Open → 2610.07406v1

Machine learning made fair and simple for non-programmers

EasyClassifier: Honest, Reproducible Machine-Learning Classification for Researchers Who Do Not Program

Abstract: Machine-learning classification is now used across medicine, the social sciences, economics, education and engineering, very often by researchers who do not program. Two errors recur in such work: preprocessing learned on all rows before cross-validation (data leakage), and reporting the cross-validation score of the classifier that won a comparison (selection bias). Both make published scores optimistic. We present EasyClassifier, an open-source Python package that guides a user from a CSV or Excel file to a finished analysis through a sequence of plain-language, multiple-choice questions, with no code. It makes the correct procedure the default rather than an option: every preprocessing step is learned inside each training fold; the selected classifier receives a separate final score from nested cross-validation (up to 2,000 rows), or an untouched 20% test set; classifiers run with fixed default parameters and fixed random seeds; and every run writes a report with a ready-to-adapt Methods paragraph, the references to cite, and figures prepared for publication. On ten public datasets from seven fields and a random-label control, split in half so that one half served as an untouched external test, the common practice overstated balanced accuracy by 2.9 percentage points on average (up to 9.9 on small data), whereas the score EasyClassifier reports had a mean signed bias of +0.7 points and a mean absolute error of 2.3 points (one-sided Wilcoxon p = 3.2 x 10^{-4}, 30 runs). On random labels it reported 48.8% balanced accuracy against a chance level of 50%. EasyClassifier is available under the MIT license from PyPI (pip install easyclassifier), GitHub, and Zenodo.

Sat 3 OctMachine LearningArtificial Intelligence
The gist
Machine learning is used by many researchers who do not write code, but they often make mistakes that give overly optimistic results. The authors created EasyClassifier, a tool that anyone can use without programming, which follows the right steps to avoid these mistakes. EasyClassifier checks its work with solid testing and provides clear reports ready for publication. It was tested on multiple real datasets and showed much smaller errors than common practices. This helps make machine learning results more honest and reproducible for users from various fields.
Open → 2610.04758v1

Improving classification accuracy with combined missing data imputation

Integrated Imputation-Classification for Supervised Learning with Missing Data

Abstract: We study supervised classification problems with missing feature values. Existing approaches often decouple imputation from classification, producing imputations that may be plausible but uninformative for prediction. Instead, we propose the Integrated Imputation and Classification Network (IICN), which jointly trains an imputer and an ${(n{+}1)}$-classdiscriminator adversarially with a single class supervised classification objective, where the discriminator learns to distinguish among the $n$ true classes and an additional ``imputed" class. We prove that at the global optimum, the imputer and discriminator together implement marginalization over missing coordinates and yield a Bayes-optimal classifier. We evaluate IICN on FashionMNIST, CIFAR-10, and tabular datasets with naturally occurring missingness. IICN outperforms classical impute-then-classify pipelines and recent generative baselines, showing strong robustness and accuracy in challenging settings.

Sat 3 OctMachine Learning
The gist
When some information is missing from data used to teach computers to recognize things, it's hard for the computer to make good guesses. The researchers came up with a new way for the computer to fill in missing details and decide what the data represents at the same time. They trained two parts together so that one fills in gaps while the other guesses the right answer, helping them both improve. Their method works better than older ways on various image and table data with missing pieces.
Open → 2610.04273v1

Hybrid model improves multi-label classification with explainable rules

HyDI: A hybrid Deep Learning-Inductive Logic Programming ensemble for multi-label classification

Abstract: While attaining remarkable results for many applications, Deep Learning models are notoriously difficult to explain. This work introduces HyDI, a hybrid ensemble architecture for hierarchical multi-label classification. It combines a Deep Learning (DL) model with rule-based classifiers generated by Inductive Logic Programming (ILP). For leaf classes of the label hierarchy, the rule-based classifiers replace the DL model, leading to more transparent classification results. HyDI is applied to the Chemical Entities of Biological Interest (ChEBI) ontology, providing ILP-generated rules for 314 classes. For these classes, HyDI can generate global explanations as well as local explanations that combine visual and text-based descriptions.

Tue 29 SeptMachine LearningLogic in Computer Science
The gist
Deep learning models are great at classifying complex data, but they can be hard to understand. The authors created a system combining deep learning with rule-based logic to classify items while explaining the decisions clearly. Their system, called HyDI, replaces deep learning with easy-to-understand rules for the most specific labels in a hierarchy. They tested it on a chemical database with many categories, generating explanations that mix visuals and text for clarity.
Open → 2609.37740v1

Hyperbolic neural networks explained with geometry aware attribution rules

Explaining Hyperbolic Neural Networks via Geometry-Aware Relevance Propagation

Abstract: Hyperbolic neural networks introduce geometric operations that require explicit treatment in relevance propagation. Equivalent geometric realizations can produce different feature attributions, even when local relevance is conserved. We study this problem through Geometric Representation Invariance (GRI), a specialization of Implementation Invariance, and zero-curvature consistency, which requires identity relevance propagation when a geometric module approaches the identity. We propose LRP-radial-all for origin-centered radial modules, treating geometric scaling as modulation and assigning relevance entirely to the signal branch. The rule conserves relevance, is invariant to equivalent radial factorizations, and satisfies zero-curvature consistency, yielding GRI for a specified Poincaré-Lorentz logarithmic-map construction. In contrast, a conservative LRP-half baseline can violate both consistency criteria. Experiments on hyperbolic MNIST, sEEG, and CIFAR-10 classifiers assess attribution fidelity, qualitative explanations, and runtime. LRP-radial-all achieves competitive attribution fidelity across datasets with runtime comparable to Gradient$\times$Input and substantially lower than Integrated Gradients. These findings motivate geometry-aware propagation rules that distinguish relevance conservation from consistency across equivalent computations.

Mon 28 SeptMachine Learning
The gist
Explaining decisions made by hyperbolic neural networks is tricky because their geometry affects how information flows inside them. The authors study how to fairly assign credit to different parts of these networks by considering their geometric properties. They propose a new method called LRP-radial-all that respects these properties and gives more consistent explanations. Their method works well compared to others and runs efficiently on image and brain signal data.
Open → 2609.35128v1

Affine geometry and kink patterns in neural networks change with training

Affine Geometry of Gaussian ReLU Networks via Conditional Kac-Rice Formulas

Abstract: We study how the affine geometry of finite ReLU networks is created at random initialization and reorganized by supervised training. We call a sign-changing zero of a hidden preactivation an activation switch and a point where the scalar network output is nondifferentiable a scalar kink. For one-dimensional input, conditioning on the preceding layers makes each preactivation Gaussian and affine on the cells of a random finite partition. This yields an exact finite-width conditional Kac-Rice formula for the expected number of activation switches along an input interval. For fixed depth and proportionally growing widths, the resulting switch intensities converge to explicit deterministic limits. A visibility estimate shows that the expected number of switches that do not produce scalar kinks is negligible, yielding an explicit leading formula for the expected number of scalar kinks and hence affine regions. In higher input dimensions d >= 2, the analogous conditional surface formula yields the leading expected (d-1)-dimensional Hausdorff measure of the scalar kink set. On the Breast Cancer Wisconsin data, the initialization formula accurately predicts switch counts along held-out segments. After training, switch counts decrease along within-class segments and increase along between-class segments. Thus, training redistributes rather than merely contracts affine complexity.

Sat 26 SeptMachine Learning
The gist
Neural networks with ReLU activations divide their input into simple affine regions separated by 'kinks' where the output bends sharply. This paper studies how many such kinks and internal activation switches exist in randomly initialized networks and how these change after training. The authors develop formulas to exactly count expected switches and kinks, showing that training redistributes these features rather than just reducing them. This helps explain how the network's piecewise-linear structure adapts during learning.
Open → 2609.32695v1

Active feature acquisition methods adapt to missing training data costs and performance

Active Feature Acquisition With Incomplete Training Data

Abstract: In many prediction tasks, acquiring all features can be a prohibitively expensive or outright impossible task. Further, in many cases a static subset of features may not be enough to solve the problem sufficiently across various instances. Active Feature Acquisition (AFA) addresses these problems by formalizing the trade-off between feature cost and predictive performance during sequential feature selection. However, prior AFA work largely assumes access to complete training data, an assumption that is often violated in practice. Here we study AFA with Incomplete Training Data (AFA-ITD), showing that under missing completely at random (MCAR) data, one-step acquisition values remain unchanged, whereas multi-step values can decrease. We analyze three approaches to learning from incomplete data: aliasing, filtering, and generative restoration. We show that filtering can require a number of training instances scaling exponentially with the dimension, whereas generative restoration scales exponentially with the acquisition budget. We empirically test our theory on a controlled experiment and across common AFA datasets and find that missingness mainly damages methods that exploit multi-step acquisitions and that generative restoration is able to recover lost performance in many experiments. Code is available at https://github.com/Linusaronsson/AFA-Benchmark/tree/missing-data.

Sat 26 SeptMachine LearningArtificial Intelligence
The gist
Sometimes gathering all information needed to make a prediction is too costly or impossible. The authors study how to actively decide which pieces of information (features) to get, even when the available training data is incomplete or missing some parts. They find that certain approaches to handle missing data work better than others, especially when planning multiple steps ahead. Their experiments show that methods using a generative model to restore missing data can recover much of the lost performance. This helps build smarter systems that gather information efficiently despite incomplete data.
Open → 2609.32325v1

Active feature acquisition improves with synergy in some low budget cases

When Does Synergy Help Active Feature Acquisition? A PID-Based Study

Abstract: Active feature acquisition (AFA) sequentially selects informative features under budget constraints. However, existing policies rarely distinguish whether information contributed by interacting features is redundant, unique, or synergistic. We introduce SynAFA, a state-dependent AFA policy that combines pairwise joint information and conditional information, with Partial Information Decomposition (PID) characterizing its information structure. Across five tabular datasets and MNIST-loop, performance is heterogeneous. SynAFA shows its strongest gains at low budgets on PhysioNet, where synergistic and redundant feature pairs are supported by permutation tests, but its advantage diminishes as budgets increase and does not depend on pair proposals. SynAFA performs significantly worse than nearly all baselines on MiniBooNE, and than CAE on MNIST-loop. Controlled synthetic experiments further show that, within budget, SynAFA's advantage rises as joint information becomes more synergy-dominated, including when total joint information is held approximately constant. Further analyses show that the budget-dependent erosion is not resolved by non-greedy local search, which improves a diagnostic set-level objective but leaves predictive performance no better, often significantly worse, and that improvements in this objective are weakly aligned with the fixed classifier's predictive utility. These findings characterize when pairwise synergy can benefit AFA while exposing a persistent challenge in translating local information into effective acquisition objectives.

Sat 26 SeptMachine Learning
The gist
Selecting which features to use in data analysis can be costly, so systems try to pick the best ones without wasting resources. The authors study when combining information from pairs of features helps make better selections. They find that synergy between features boosts performance mostly when budgets for collecting features are small, but this advantage fades as budgets grow. Their work also shows challenges in turning this synergy insight into consistent practical gains.
Open → 2609.32301v1

Differentially private PCA algorithm works without eigenvalue gaps

Gap-free Differentially Private PCA for Gaussian Data

Abstract: We give a gap-free differentially private algorithm for the principal component analysis (PCA) problem with Gaussian data.

Fri 25 SeptData Structures and AlgorithmsMachine Learning
The gist
When trying to find the main directions of variation in data (called principal components) while keeping individual data private, existing methods often need clear differences between these directions. This paper presents a way to do this with Gaussian data even when these differences are small or absent. The authors provide a new algorithm that maintains privacy without relying on gaps in the data. This could help analyze sensitive data more accurately and privately.
Open → 2609.31614v1

Deep neural networks estimate repeated measurement regression functions

Generalized Deep Regression for Repeated Measurements

Abstract: In this paper, we study the estimation of a marginal regression function from independent units with repeated binary, count, or continuous responses using ReLU deep neural networks. In the model, we assume that the dependence is generated by an unobserved random mean function within each unit. We then fit a neural network with a convex generalized regression loss. We show an oracle inequality by separating conditional measurement variation from between-unit variation. In addition, we prove that with $n$ units and $m$ measurements per unit, ReLU networks can attain an integrated mean squared error of order $n^{-1}+(nm)^{-2β/(2β+d)}$, up to logarithmic factors, over $β$-Hölder classes. We also derive a weighted oracle inequality for unequal cluster sizes and a rate for compositionally smooth functions. For pointwise ensemble inference, we give a projection central limit theorem and prove infinitesimal jackknife consistency under an explicit asymptotic linearity condition. Simulations and real data examples are provided to support our theoretical findings and practical implications.

Tue 22 SeptMachine Learning
The gist
The paper addresses the challenge of analyzing data where multiple measurements are taken on the same units, like people or machines, with responses that could be yes/no, counts, or continuous numbers. The authors study how deep neural networks can estimate the average relationship between input factors and outcomes while accounting for individual differences that are not directly observed. They derive mathematical guarantees on how accurate these neural network estimates can be based on the number of units and measurements. They also provide statistical tools to make reliable inference from the model and validate their methods through simulations and real-world data examples.
Open → 2609.25605v1

Unified retrieval method improves entity recognition for small AI models

URA-NER: A Unified Retrieval-Augmented Framework with Retrieval Alignment and Uncertainty Reduction for Low-Resource NER

Abstract: In-context learning (ICL) based on large language models (LLMs) has shown promising potential in alleviating performance bottlenecks caused by the limited availability of annotated data in Named Entity Recognition (NER). However, existing methods still face issues of retrieval misalignment and generation uncertainty, making their performance heavily dependent on the LLM's capabilities. As the parameter scale of LLMs decreases, their performance in few-shot settings deteriorates significantly. In this paper, we propose a novel unified retrieval-augmented framework, URA-NER, including three key components: Progressive Granularity Retrieval (PGR), Model-aware Representation Enhancement (MaRE), and Reason-aware Knowledge Verification. PGR is a two-stage retrieval mechanism that achieves stage alignment. It first retrieves demonstrations for span detection based on the query's global semantics, and then for type classification based on the specific entity context, providing fine-grained local information. Moreover, MaRE employs entity pre-recognition to guide the construction of representations, ensuring the query and demonstrations are aligned within the LLM's semantic space and attention pattern. In addition, to mitigate generation uncertainty, we propose RaKV, a closed-loop "generation-retrieval-verification" process. It explicates the LLM's reasoning paths, leverages them for the retrieval of external knowledge, and reorganizes the knowledge into verification evidence aligned with the original reasoning paths. We conduct extensive experiments on multiple low-resource NER datasets. Results demonstrate that URA-NER significantly enhances the performance of LLMs under low-resource settings, with particularly pronounced gains for smaller LLMs, achieving new state-of-the-art results on several benchmarks.

Mon 21 SeptComputation and LanguageArtificial Intelligence
The gist
Named Entity Recognition (NER) systems identify important names or terms in texts but usually need lots of labeled examples to work well. The authors developed URA-NER, a new method that helps smaller language AI models do this better with only a few examples by carefully choosing helpful examples and reducing confusion during the process. Their approach involves two stages of finding examples, improving how AI understands these examples, and double-checking the AI's reasoning with external knowledge. Tests showed this method significantly boosts recognition accuracy on tasks where training data is scarce, especially benefiting smaller AI models.
Open → 2609.24372v1

Llms improve continuously with environment-driven dynamic policies

Smarter by the Moment: Environment-Driven Dynamic Policies for Continual LLM Improvement

Abstract: Large Language Models (LLMs) have achieved remarkable progress across diverse domains, but continual adaptation to evolving tasks and environments remains a key challenge. Existing memory-augmented approaches retrieve individual past examples as direct references, but do not explicitly synthesize actionable strategies from them, causing the same types of errors to recur. We propose Dynamic Retrieval-based Policy Generation (DRPG), a framework that integrates memory-based retrieval with a dynamic policy generator, leveraging historical data and environment feedback to produce task-specific policies for continual LLM improvement. We evaluate DRPG across six benchmarks spanning text-to-SQL, question answering, medical diagnosis, and Python programming, using seven LLMs from both proprietary and open-weight families. DRPG outperforms strong baselines across most datasets and models. Further analysis demonstrates that DRPG's policy generation is robust to retrieval strategy, operates effectively without prior policy continuity, and can leverage smaller or cross-family models as cost-efficient policy generators. We also find that the benefit of policy-level guidance depends on task characteristics, offering practical insights into when and under what conditions this mechanism is most effective.

Tue 15 SeptComputation and Language
The gist
Large language models often struggle to keep improving as their tasks and environments change. The authors propose a method called Dynamic Retrieval-based Policy Generation (DRPG) that helps these models learn smarter by creating specific plans based on past experiences and environment feedback. This approach makes the models better at a variety of tasks, like answering questions or writing code, without repeating the same mistakes. The authors also found that DRPG works well with different kinds of models and tasks, making it a flexible way to keep improving language models over time.
Open → 2609.16800v1

Smote improves logistic classification when assumptions break down

PU classification under Non-SCAR: clustering-assisted logistic model with oversampling enhancement

Abstract: This study addresses the PU classification problem under violations of the SCAR assumption. We investigate logistic regression-based approaches, namely the cluster method and its extensions with strict and non-strict Lasso regularization. The primary contribution of this work is the integration of the SMOTE technique to alleviate class imbalance and systematically assess its impact on the performance of the considered algorithms. SMOTE is first applied to rebalance the training dataset. Next, cleaning labels are derived via 2-means clustering. Logistic regression is then trained on the cleaned data, where identified positive instances are augmented with additional true positives and the remaining observations are treated as negative. The experimental evaluation is conducted on 13 real benchmark datasets and one synthetic dataset. For comparison, we include the naive approach and the Spy-EM method. The results demonstrate that incorporating SMOTE improves classification performance when the SCAR condition is violated and indicate moderate robustness of the LassoJoint method in this setting.

Sun 13 SeptArtificial IntelligenceMachine Learning
The gist
This study looks at a tricky type of data problem where you only have examples labeled as positive or unlabeled, and usual assumptions about the data don’t hold. The authors explore ways to improve how logistic regression models handle this challenge by using a method called SMOTE to balance the data and clustering to clean the labels. Their results from testing on many real and synthetic datasets show that adding SMOTE helps the models perform better when the usual data assumptions are violated. One of their logistic regression variants using regularization also showed some robustness in these cases.
Open → 2609.14675v1

Multiview semantically consistent encoding improves multi-label prediction with missing data

When Semantically Consistent Encoding Meets View-Label Heterogeneity Modeling: A Unified Framework for Incomplete Multi-View Multi-Label Learning

Abstract: Incomplete multi-view multi-label learning requires not only robust semantic aggregation from partially observed views, but also label-aware exploitation of view-specific evidence. Existing approaches usually emphasize either shared representation learning or decision-level fusion. The former improves robustness against missing views, yet tends to compress label-discriminative view-specific cues into a single latent representation. The latter preserves individual view predictions, but often relies on fixed or globally learned fusion weights, ignoring that different labels of different instances may require different views. To address these limitations, this paper presents V2L, a unified representation-decision framework for incomplete multi-view multi-label classification. On the representation side, V2L constructs semantically consistent variational posteriors from incomplete views through a perturbation-aware encoding mechanism, which provides a stable shared semantic basis. On the decision side, V2L introduces an active view-label relevance modeling strategy that estimates instance-wise and label-wise view contributions, allowing each label prediction to adaptively select useful view-specific evidence. From the perspective of model architecture, these two important strategies are integrated into a unified framework through a hybrid fusion architecture, simultaneously meeting the requirements of cross-view semantic consistency and representational complementarity. Extensive experiments under both incomplete and complete settings show that V2L achieves leading performance on five benchmarks. Code is available at: https://github.com/justsmart/V2L.

Mon 7 SeptComputer Vision and Pattern Recognition
The gist
Some problems involve identifying multiple labels from data collected in different ways, but often some of these data views are missing or incomplete. The authors propose a new method called V2L that first creates a shared understanding from the partial data and then actively figures out which data sources are important for each label. This dual approach helps make more accurate predictions even when some information is missing. Their experiments show this method works better than previous ones on various benchmark tests.
Open → 2609.07525v1