Papers for
data analysts
Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.
Visualizing feature importance with weighted t-SNE boosts data understanding
Assessing feature scorer results on high-dimensional datasets with t-SNE
Abstract: While vast literature on high-dimensional data visualization is available, there are not many works regarding the visualization of feature scorers and their results. Feature scorers are algorithms that assign numerical importance to each feature of multi-dimensional datasets. These importance scores can be used in several applications, such as feature selection, knowledge discovery, and machine learning interpretability. There are several feature scorers to choose from, and often no single metric or ground truth is available to guarantee the quality of their results. In this scenario, visualization can become valuable to support the decision of which method to choose and how good its results are. For this goal, this work presents "weighted t-SNE." It modifies the relationship between data points in the embedded 2D space to reflect the importance of each dimension of the original datasets as assessed by a feature scorer. This research discusses how to implement weighted t-SNE, proposes the silhouette coefficient as a numerical evaluation of the results, and shows several examples of its use in practice. Synthetic and real-world tabular datasets are used in the experiments together with nine feature scorers, ranging from Mutual Information to neural networks. Each feature scorer produces unique visualizations, and weighted t-SNE can be used to compare and choose the one that better suits a given dataset and task. Weighted t-SNE can also visually show the importance of features learned by machine learning models and help us see how they are organizing the data, increasing their interpretability.
Correlation clustering results made understandable by label trees
ExplainCC: Explaining Correlated Clusters by Node Labels
Abstract: An increasingly important requirement for clustering algorithms is that decisions and outcomes need to be clearly understandable for human users. We address this requirement for the Correlation Clustering (CC) problem, in case the objects to be clustered are accompanied by labels. In CC one seeks to partition a set of objects into clusters based on pairwise similarity and dissimilarity information. Our proposed model only considers clusters that can be explained by a label tree, which is essentially a decision tree that partitions the objects according to labels. The goal is to find an explainable partition of the data that minimizes the CC objective function. We formalize this problem from a theoretical viewpoint, and prove several hardness of approximation results, also when constraining the size of the tree. Our proposed greedy method finds locally optimal splits, and works recursively in a top-down approach. To efficiently compute the best split, we carefully maintain certain counters. An experimental evaluation demonstrates its effectiveness. The proposed algorithm processes graphs with over million edges in a few minutes, can find the ground truth in synthetic data, and is able to produce interpretable clusters in real world networks.
Uniformity testing on matchings in graphs needs fewer samples
Uniformity Testing for the Monomer-Dimer Model
Abstract: We study uniformity testing for the monomer-dimer model on a known graph $G=(V,E)$. Given samples from an unknown distribution $D_w(M)\propto \prod_{e\in M} w_e$ over matchings $M$ of $G$, we ask whether $D_w$ is the uniform distribution over all matchings of $G$ or is $\varepsilon$-far from it in total variation distance. Our main result is that, for every maximum-degree bound $Δ$, uniformity can be tested with $\widetilde O_Δ(\sqrt m/\varepsilon^2)$ samples, where $m=|E|$. We complement this with an information-theoretic lower bound for general graphs: for every pair of parameters $n\le m\le n^2$, there are graphs with $Θ(n)$ vertices and $Θ(m)$ edges on which every tester requires $Ω(\sqrt m/\varepsilon^2)$ samples. Thus, for bounded-degree graph families with $m=Θ(n)$, testing uniformity over the space of matchings has $\widetilde Θ(\sqrt n/\varepsilon^2)$ sample complexity, in contrast with the $Θ(n/\varepsilon^2)$ samples needed to estimate $D_w$ in total variation distance.
Benchmark measures when extra AI search is worth the cost
SearchHorizonBench: A Benchmark for Search Value in AI Agents
Abstract: AI agents must decide when another search is worth its cost. Answer-success probabilities capture question difficulty, while this decision requires estimating the additional benefit of search. We introduce SearchHorizonBench, a benchmark for learning and evaluating search value from execution experience. Paired continuations and repeated executions separate expected search gain from generation noise and connect prediction quality to decision utility. The resource contains 45,632 executions across 3,168 questions. On 1,632 held-out questions, historical calibration reduces Brier error by 16.2%, with the improvement concentrated in success-level information. Search gains have a standard deviation of 21.4 percentage points, and tested historical estimators explain less than 2% of this variance in point estimates. Matched-configuration learning curves probe this distinction across training scales. At the reference token price, free selection observations yield a 2.49-point utility opportunity over the best constant action, driven primarily by gain information. Reusable histories, frozen splits, strong baselines and scoring tools provide a common foundation for developing and evaluating value-aware search agents.
Robust decisions guaranteed from limited gaussian projection data
Certifying Distributionally Robust Decisions from Projection Observations
Abstract: A scalar measurement can determine a robust decision without identifying the latent covariance or every decision risk. We study finite Gaussian projection experiments with a fixed mean-plus-standard-deviation target. The observation laws define a set of indistinguishable moment models. We exhibit decisions that agree throughout that set despite unidentified risk values, and give a confidence-set certificate that returns a decision or an unresolved outcome. We prove soundness, necessary abstention for indistinguishable models with incompatible optima, and a fixed-checkpoint certification guarantee. A scalar profile reduces nonlinear risk comparisons to convex inner problems. Across seven synthetic instances, jointly calibrated moment regions give sufficient budgets 21 to 52 times explicit expected-sample information lower bounds. Separately measured stopping times are lower, with zero decision errors in 1,900 simulated streams. These comparisons do not constitute matching bounds, and the guarantees require Gaussian replication.
Dataset search improves by matching user intent with table content
Bridging Search Intent and Tabular Data in Dataset Search
Abstract: Dataset search remains a bottleneck in data analysis workflows, as existing systems largely rely on keyword matching or shallow retrieval that fail to capture user intent. To facilitate intent-driven search, we present two retrieval components to equip multi-faceted dataset search interfaces through natural language. First, we propose Hypothetical Schema Embeddings (HySE) that uses a language model to generate task-aligned hypothetical table schemas which are used for embedding-based retrieval, bridging the gap between intent and table content. We introduce KaggleDS, a corpus comprising 6,503 tables and diverse task-queries derived from Kaggle, and show that HySE consistently improves recall over strong keyword and embedding-based baselines, as well as over a dense table retriever fine-tuned on KaggleDS. We also show that HySE generalizes to end-to-end question answering on, for example, the NQ-Tables dataset, a corpus of 169,898 Wikipedia tables with human-written queries, where HySE improves Recall@10 by +6.0 pp over dense retrieval baselines. Second, we introduce FlexPrune, which translates dataset constraints expressed in natural-language into SQL predicates over a customizable metadata index, enabling filtering across arbitrary metadata attributes. Combined with HySE, FlexPrune increases Recall@10 by +6.0 pp over dense retrieval followed by tag-based filtering. Our results show that natural-language-based retrieval supporting task and metadata queries offers a viable alternative to schema-level keyword search in dataset search.
Environment signals improve learning for agents in data tasks
EnvLunch: The Environment as a Free Lunch for Learning to Interact
Abstract: For agentic data analysis, language-model agents often require a large number of annotated tasks to learn how to interact effectively with executable environments (i.e., databases). While high-quality annotated tasks require heavy manual effort, an executable environment already provides rich interaction structure and verifiable outcomes. In this sense, the environment itself offers a free lunch, an additional set of verifiable tasks that require no human annotation. We propose EnvLunch, a warm-up stage that turns this environmental signal into agent training without human-written or model-generated task descriptions, before downstream agentic reinforcement learning. EnvLunch samples valid action trajectories, executes them to obtain final observations, and trains an agent to recover a functionally equivalent trajectory given only its final observation. This label-free warm-up enables the model to outperform the baseline by over 9 points on average across all downstream training budgets, and with few labels, the baseline needs 8 to 32 times as many annotated examples to match the accuracy that the EnvLunch model reaches at its smallest training budget. Letting the environment alone define the task yields a larger gain than having the model write its own tasks over the same environments, and this warm-up also improves fine-tuning on benchmarks beyond data analysis. EnvLunch therefore saves annotated tasks and provides a better starting point for fine-tuning agentic models when such tasks are limited.
Visualizing uncertainty in data with adaptive jk-plots
Adaptive Uncertainty Visualization With jk-Plots
Abstract: An important task of uncertainty visualizations is to communicate how the number of available observations influences the precision of estimates, such as a Normal distribution's mean. In this paper, we propose an adaptive approach to visualizing univariate data that produces salient representations while highlighting the uncertainty introduced by the sample size. We introduce jk-plots, inspired by educational material of Bayesian statistics, that combine visualizations of plausible reference models (emulating popular uncertainty visualization idioms) and the observations. We describe a visualization pipeline to create jk-plots, which we implemented in an R package. We demonstrate jk-plots using synthetic datasets of varying size and discuss possible usage scenarios in domain expert interviews.
Atlas helps people understand long agent runs and decisions
Atlas: Making Sense of Long Horizon Agent Trajectories
Abstract: AI agents can now search literature, generate hypotheses, analyze datasets, and run experiments at a scale difficult for researchers to match. Yet scaling scientific execution does not automatically scale researchers' ability to make sense of what agents produce. When researchers return to a long-horizon run, they must evaluate work they did not monitor: what the agent produced, which decisions shaped those outputs, and whether those decisions served the research goal. We frame this post-run sensemaking as a representation problem across three linked levels: goal, decision, and artifact. From this framework we developed Atlas, an agentic workspace with a trajectory tree, which renders the run as a branching structure showing what the agent chose and set aside, and artifact projections over six representations. In a within-subjects study (N=14), participants reported higher decision transparency, artifact sensemaking, and goal connection with Atlas than with a standard baseline, together with substantially lower subjective workload.
Users often miss hidden AI errors despite high confidence
Looks Fine to Me: Understanding Oversight in Agentic Interactions
Abstract: Agentic AI systems are deployed to execute complex tasks across domains such as writing, coding, and data analysis. As execution is offloaded to AI agents, the human role shifts toward oversight. Effective oversight requires recognizing when closer scrutiny is needed, even though flawed outputs may appear convincing. Our formative study (N=12) revealed that users accepted plausible outputs without thorough evaluation, and only recognized these lapses in retrospect. To examine this systematically, we conducted a controlled within-subject study (N=36) in which participants completed programming tasks with an agent that produced correct outputs, silent defects, or visible failures. Combining task outcomes, interaction logs, gaze, physiological signals, and self-reports, we examined perceived, behavioral, and effective oversight. Participants were often highly confident while missing silent defects: they performed generic checks, failed to generate diagnostic evidence, or even misinterpreted defect evidence. We discuss how studying oversight across multiple dimensions can support future work on timely, task-calibrated re-engagement.
EasyResearch delivers honest data prediction without programming skills
EasyResearch: Honest Regression, Forecasting and Signal Classification for Researchers Who Do Not Program
Abstract: EasyClassifier showed that researchers who do not program can obtain honest classification results when the correct evaluation is the default. Most applied studies, however, predict a number, forecast a series or classify recordings, and each task has its own way of producing scores that do not hold on new data. We present EasyResearch, open-source analysis cores for regression, time-series forecasting and signal classification, and EasyResearch Desktop, a Windows application that runs these three tasks and EasyClassifier's classification without code. All four follow the same rules: everything learned from data is learned on training data only, models keep their published defaults with fixed seeds, the selected model receives a separate final score, forecasts are never evaluated on shuffled data, and every run writes a report with a Methods paragraph and the references for every method used. We measured whether the reported scores are honest by keeping part of each dataset untouched. In regression (9 public datasets, 90 runs), the common practice overstated R^2 by +2.2 points and the best comparison score by +2.5, while EasyResearch's final score was -1.0; screening predictors on all rows before cross-validation, which EasyResearch does not offer, reported R^2 = 0.27 on pure noise against -0.44 on new data. In forecasting (15 series), shuffled cross-validation over windows of past values promised errors 19% smaller than its chosen models achieved in the untouched future, while EasyResearch's final-period score was unbiased to within 1.5%; the models EasyResearch chose had 14% lower future error, and on random walks shuffled cross-validation chose a machine-learning model in 5 of 5 series, each worse in the future than repeating the last value, which EasyResearch chose 4 times. In signal classification (4 UCR datasets) one method won every comparison and neither estimate was biased.
Typed decision models match large language models on tabular data tasks
Typed Decision Models on Tabular Data
Abstract: Typed decision models are language models that answer a question with a fixed set of possible answers by returning a probability for each answer instead of generating text. We study them on tabular data and compare four decision models with three large language models (LLMs) on row classification, fact verification and anomaly detection. Every request is sent in its original form and under 21 types of rewrite that keep its content, such as renaming the options or writing the table in another format. The decision models perform as well as the LLMs at a lower cost. Three of them are also far more robust to how the question is formulated. No model we test is robust to how the table is organized. For every model, at least a third of the row decisions change under some rewrite of the table or the question. The decisions to which a decision model gives low probability are the ones most likely to change. Decision models are a practical choice for tabular decisions, but their robustness remains an open problem.
Matching number governs queries to estimate graph edges and degrees
The Matching Number Characterizes the Query Complexity of Edge Counting and Degree Moment Estimation
Abstract: Consider an unknown graph $G = (V,E)$ with vertex degrees $d_v$, accessed only by sampling uniformly random edges and querying the degrees of their endpoints. We study the query complexity of estimating the number of edges $m$ and the degree moments $M_s = \sum_{v \in V} (d_v)^s$ for fixed integers $s \ge 2$. In this model, [Beretta and Tětek, TALG 2024] estimate $m$ within relative error $\varepsilon$ using $O(\log n/\varepsilon^2+\sqrt n/\varepsilon)$ expected queries without knowing the number of vertices $n$, and [Bhattacharya and Pradhan, CIAC 2025] estimate $M_s$ using $O(n^{1-1/s}/\varepsilon^2)$ queries when $n$ is given. We show that the query complexity of both problems is governed by the matching number $ν$ of $G$ (the maximum number of pairwise disjoint edges) rather than the number of vertices $n$, and that the dependence on $\varepsilon$ for degree moments can be polynomially improved. Our algorithms rest on a simple observation: the endpoints of a maximum matching cover every edge, so a random endpoint of a random edge falls on this set of $2ν$ vertices with probability at least $1/2$. For every $\varepsilon\in(0,1]$, we give advice-free algorithms that estimate $m$ and $M_s$ within relative error $\varepsilon$ with probability at least $2/3$, without knowledge of $n$, $ν$, or other graph parameters. Specifically, our algorithms use $O(\min\{ν/\varepsilon, 1/\varepsilon^2+\sqrtν/\varepsilon\})$ expected queries for $m$ and $O(\min\{ν^{1-1/s}/\varepsilon^{2/s}, ν^{1-1/(2s-1)}/\varepsilon^{2/(2s-1)}\})$ for $M_s$. Finally, our lower bounds show that for every $k \ge 2$ and sufficiently small $\varepsilon$, these query complexities are optimal up to constant factors over graphs with matching number at most $k$.
Plottery combines graphical interface with code for better Matplotlib plots
Plottery: A Code-Integrated GUI for Matplotlib
Abstract: Scientists have many tools to author plots: graphical direct manipulation apps like a spreadsheet's chart builder, text-based programming tools like Matplotlib and ggplot2, or AI chat interfaces. Textual code is the most capable of these, but writing code, even with AI assistance, is a potentially poor fit for graphical chart design. We present Plottery, a Jupyter notebook extension that leverages API type annotations to add graphical direct manipulation to the popular Matplotlib plotting library. User interactions in Plottery's in-situ graphical interface are immediately reified in the user's code, so they may seamlessly mix-and-match between GUI, code, and AI interactions. In a user study, participants seamlessly alternated modalities, with the GUI used for straightforward visual changes, code for quick edits or duplicating code, and the AI for broad support. Plottery showcases graphical interaction that retains code's benefits.
Scalable tools find complex patterns and relationships in data tables
Scalable extraction and visualization of multi-attribute logical and functional dependencies in tabular data
Abstract: Understanding the structural relationships among attributes in tabular data is fundamental to machine learning and pattern recognition. While functional dependency (FD) discovery has been extensively studied, scalable discovery of logical dependencies (LDs), particularly as the number of attributes and dependency order increase, remains underexplored. These dependencies capture non-deterministic, condition-specific relationships among pairwise or multiple attributes. Furthermore, existing approaches do not provide a unified framework for extracting multi-attribute LDs and FDs. To address these limitations, we propose LDTool and HLDTool for extracting and visualizing multi-attribute LDs and FDs from tabular data. LDTool extends dependency discovery beyond pairwise relationships, while HLDTool enables scalable extraction through hypergraph-guided search-space reduction. Experiments on three simulated and eleven real-world datasets demonstrate that the proposed framework extracts meaningful LDs and FDs while improving scalability. LDTool recovers the same FDs as existing FD discovery methods with lower runtime in high-dimensional feature spaces, whereas HLDTool enables dependency discovery in datasets with hundreds of features. The proposed framework provides interpretable visualizations of dependency structures and supports applications in exploratory data analysis and the quantitative evaluation of synthetic tabular data.
Typed logic framework enables flexible pattern workflows with AI agents
A Vision for a Logic-Based Workflow Formulation Framework and an Agentic Execution Pipeline for Pattern Engineering
Abstract: Organizations increasingly depend on insights drawn from data to inform consequential decisions. Many concern structural behaviors, which we call patterns. Producing them may require working with multiple pattern types, performing different operations over them, and composing those operations into workflows, an activity we call pattern engineering. No principled framework exists for organizing such workflows: analysts devise each by hand, and existing systems automate particular analytical objectives rather than representing pattern operations in a general, composable form. We introduce a typed framework that represents each pattern through a fixed set of modular components, and expresses pattern operations as queries which specify input and output components. Chaining queries composes workflows, while execution of any individual query remains unconstrained. We instantiate the framework with an LLM-based agentic executor and evaluate example-mining, example-alignment, and pattern-induction queries over two temporal patterns, using World Intellectual Property Organization (WIPO) reports and patent data. All three queries are carried out across four LLM settings, which perform differently by target component, while the traces allow us to localize errors by execution stage.
Recommendation systems guide users through complex data exploration
Recommendation Systems for Exploratory Data Tasks
Abstract: A large class of data-centric tasks is exploratory, where users iteratively steer workflows, refining subjective goals as new insights emerge. These Exploratory Data Tasks (EDTs) are performed by millions of users with varying levels of expertise to understand unfamiliar data, discover trends, and identify evidence that informs critical decision-making. However, a key challenge in EDTs is the enormous space of possible actions that one can take at each step: users struggle to choose among thousands of joins, transformations, and aggregations, causing "exploration paralysis". Because EDT workflows are interconnected, each choice impacts subsequent exploration, and suboptimal choices can lead to inefficiency, missed insights, confirmation bias, and incomplete coverage. This calls for intelligent recommendations that efficiently guide users toward optimal EDT actions. We envision recommendation as a core capability of data systems, proactively guiding users toward promising actions and thereby lowering the barrier to exploratory data tasks. EDT recommendation is challenging because the action space is combinatorial and actions are data-dependent, which require costly materialization. Moreover, recommendation often involves bundles or sequences of actions across interdependent tasks, requiring coordination across tasks. In this paper, we present our vision of EDT recommendation systems along two axes: single-task vs. multi-task settings and single-action vs. multi-action recommendations. We outline a research agenda that progresses from recommending individual EDT actions to constrained bundles and sequences of actions, and ultimately to coordinated recommendations across interconnected EDTs. We identify research directions for incorporating various contexts (user, data, task, and ecosystem), addressing efficiency challenges, and coordinating across tasks.
Fruchterman Reingold method compares differently to agglomerative clustering
Interrelating Fruchterman-Reingold Graph Visualization and Agglomerative Clustering
Abstract: Graph visualization methods and agglomerative clustering have been frequently considered in data analysis and pattern recognition. Because these approaches are interrelated and complementary, it is of particular interest to investigate their associations. In this work, we study the possible relationship between the Fruchterman-Reingold graph visualization method and four types of agglomerative clustering adopting single- and complete-linkage, average, and Ward's linkage criteria. Three types of datasets have been considered in 2 and 10 dimensions, as well as the PCA projection of the latter to two dimensions. The results obtained suggest that the relationship between the methods considered did not vary much for the three types of data mentioned above. At the same time, the agglomerative methods tended to yield results that are mostly similar to each other, while presenting moderate similarity with the original data. The Fruchterman-Reingold visualization resulted similar to the original data, but exhibited relatively smaller similarity to the agglomerative methods.
TableSeek finds exact evidence in diverse tables using stepwise search
TableSeek: Structure-Preserving Agentic Evidence Seeking over Heterogeneous Table Corpora
Abstract: Open-domain table retrieval seeks tables that contain sufficient evidence for answering a question or verifying a claim. Yet semantic relevance is often misleading: topically similar tables may lack the required facts, while answer-bearing evidence is often confined to a few cells whose meaning depends on surrounding schema and table context. Heterogeneous schemas, value formats, and serializations further weaken one-shot matching. We present TableSeek, a structure-preserving agentic search framework for heterogeneous table corpora. Instead of ranking tables once, an LLM agent iteratively follows sparse clues, inspects schema-preserving previews, identifies schema- and value-level mismatches, and refines its investigation. TableSeek uses cells and schemas as evidence anchors while retaining complete tables as evidence units, enabling fine-grained localization without losing the context required for interpretation and answerability checking. Without relying on retriever training or a precomputed semantic index, TableSeek produces transparent evidence-seeking trajectories and achieves competitive end-to-end performance against strong retrieval-and-reranking pipelines on heterogeneous table benchmarks. These results suggest that active, structure-preserving evidence seeking is a promising paradigm for open-domain table retrieval.
Large language models generate reliable quantified inferences from tables
From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification
Abstract: LLMs can generate fluent descriptions from tables, but their outputs may remain logically unsupported by the structured data. We introduce STAT-TO-TEXT, a controlled task in which LLMs generate quantified natural language inferences from statistical tables using quantified constructions such as all, some, no, and most. To evaluate these inferences, we use an LLM generated Python checker code which when executed verifies the corresponding truth conditions against the table. We compare four open-weight LLMs across model families and scales, evaluating faithfulness, logical accuracy, table coverage, and diversity. Our results show that model scale and family matter, with the largest model (GPT-OSS-120B) consistently producing the most faithful inferences without sacrificing greater table coverage and quantifier diversity, as opposed to smaller models. These findings are supported by human annotation, which shows that the automated checker closely aligns with human judgments.
Interactive notebook tool links outputs to code for better analysis
Point, Revise, Review: Grounded Agentic Analysis in Reactive Notebooks with marimo-lens
Abstract: In a computational notebook, a human can point to a rendered result and ask about "this," while an agent acts through cells, dependencies, and runtime state. When what the human sees and what the agent operates on are disconnected, the human must describe what they mean and trace what the agent did in prose that strips away situated visual and computational context. We present marimo-lens, an extension to the reactive Python notebook marimo for grounded human-agent analysis. Lens connects a human's marked output and note to its producing cell and relevant contributing computation, giving the agent computational context for the request. It surfaces agent activity, returns selected results to the notebook, and preserves the initiating selection for human review and reopening. We illustrate the lifecycle through an exploratory human-agent analysis of a real-world open dataset, showing how visually situated questions lead to computational inspection, notebook action, and returned evidence for human review.
LinePilot digitizer recovers curves from line plots with manual and automatic calibration
LinePilot Digitizer: Line-Plot Recovery with Manual and Automatic Calibration
Abstract: Recovering numerical series from line plots requires accurate axis calibration and reliable curve extraction. We present LinePilot Digitizer (LinePilot), which combines continuous color-based curve recovery with three calibration modes: LinePilot (standard), LinePilot (enhanced), and LinePilot (OCR). We also introduce DigitizerBench, the first dedicated benchmark for systematically evaluating digitizer performance, using an orthogonal design spanning signal, rendering, and plot-structure factors with complementary automatic and human-guided evaluations. We evaluate performance using failure-penalized capped normalized root-mean-square error (FPC-NRMSE), which assigns unit loss to missing, unusable, or catastrophically inaccurate outputs. On DigitizerBench-Full, LinePilot (OCR) achieves the lowest mean FPC-NRMSE (0.672) and highest trusted usability (38.2%) among the tested automatic pipelines. On DigitizerBench-Lite, LinePilot (enhanced) achieves the lowest mean FPC-NRMSE (0.081), 100% output success, and highest trusted usability (93.3%). The orthogonal benchmark design further enables factor analysis to identify the factors that most significantly affect digitizer performance. Together, the three calibration modes provide a practical trade-off between automation, user control, and accuracy within a shared curve-recovery workflow.
Polynomial time algorithm finds unique tensor decompositions efficiently
Beyond Kruskal: Polynomial-Time Tensor Decomposition under the Lovitz-Petrov Condition
Abstract: Identifiability criteria certify that a given tensor decomposition is a unique rank decomposition. Kruskal's classical condition is one of the best-known deterministic criteria for identifiability. However, no polynomial-time decomposition algorithm is known under the Kruskal condition, and verifying the condition itself is NP-hard. Lovitz and Petrov introduced a strictly more general identifiability condition which, in contrast, is polynomial-time verifiable, but no polynomial-time decomposition algorithm was previously known under this condition. We give a polynomial-time algorithm for 3-way tensor decomposition under the Lovitz--Petrov condition. Moreover, combining our algorithm with polynomial-time verification of the Lovitz--Petrov condition yields an efficient end-to-end certification procedure: after computing a decomposition, one can deterministically certify in polynomial time that it is unique and therefore of minimum rank. This contrasts with an arbitrary tensor decomposition, which certifies only an upper bound on the tensor rank, while determining tensor rank is NP-hard in general.
Causal stories improve reasoning when graph direction is unclear
From the Task Boundaries of Narrative Text to Structural Anchoring, Uncertainty Triggers, and Cross-Calibration
Abstract: Causal graphs represent structural relationships among variables, yet users must still interpret direction, mechanism, and adjustment conditions in relation to the task at hand. Prior work often compares explanation formats as fixed conditions and pays less attention to how users distribute reasoning across graphs, direct explanations, and stories. We developed CoNS-Explorer, which uses reviewed instructional DAGs/SCMs to maintain a shared causal-fact ledger and generate fact-matched direct explanations and contextualized stories. A controlled survey experiment ($N=240$) compared the two texts as complete presentation packages. In the primary GLMM, the Story condition had a positive but uncertain overall association with accuracy (OR $=1.55$, 95\% CI $[0.34,7.10]$, $p=.572$); a population-averaged GEE showed a significant positive effect (OR $=1.89$, 95\% CI $[1.02,3.48]$, $p=.042$). Task-type interactions localized the clearest advantage to total-effect adjustment. Story also significantly increased situational presence. In a separate system-task and interview study ($N=24$), participants freely used graphs, direct explanations, and stories across three causal models. They established structural anchors with graphs and numerical results, consulted text when direction was unclear, mechanisms were unfamiliar, or multiple paths competed, and checked their judgments against other representations or external evidence. Integrating the two studies, we develop a process framework of structural anchoring, uncertainty triggering, explanation routing, and cross-calibration, together with four testable design propositions for adaptive causal explanation.
Abstract framework organizes analysis of changing time data
Time-Varying Data as Sheaves: an Invitation to Narratives
Abstract: Modern science and engineering increasingly rely on time-varying data, yet the mathematical tools used to model temporal phenomena are often developed within separate disciplines, obscuring common principles and limiting the transfer of ideas across fields. This chapter presents the theory of narratives, an abstract framework for time-varying objects of any mathematical kind that supports both theoretical investigations and applications. To illustrate this perspective, the chapter develops three vignettes, each illustrating a different research direction. The first addresses a general concern: What information loss can occur when switching between different representations of temporal data? The second concerns structural and algorithmic approaches: How can we systematically decompose time-varying data into simple pieces and obtain invariants describing its structural complexity? The third is an application to control theory: How can we model multi-agent systems with switching communication topologies? More important than any individual vignette, the central message of this invitation is that a suitable abstract perspective can organize and guide research across remarkably diverse mathematical and scientific domains.
Latent geometry explains nested patterns in complex group interactions
Latent geometry organizes higher-order interactions
Abstract: Higher-order structures offer a natural representation of complex systems that involve interactions between groups of different sizes. A widespread feature of their higher-order structure is nestedness, whereby interactions involving smaller groups are contained within larger ones. Yet, why interactions of different orders organise into nested structures remains largely unexplained. Here, we introduce an analytically tractable geometric model of higher-order networks in which a single latent geometric space couples interactions across orders, leading to the spontaneous emergence of nestedness. We show analytically and numerically that nestedness undergoes a transition between a nested geometric regime, where it remains finite in the thermodynamic limit, and a regime where it vanishes with system size. In this regime, we uncover a weakly geometric range characterized by an anomalously slow finite-size decay, allowing substantial nestedness to persist in finite systems even when its asymptotic value vanishes. Finally, with a single geometric coupling parameter, the model reproduces the nestedness profiles observed in real-world hypergraphs across different domains. Our results reveal latent geometry as a simple organising principle underlying the nested organisation of higher-order interactions.