Reproducible randomness in AI needs matching pseudorandom generators
Reproducible AI Requires Reproducible Randomness
Artificial Intelligence
Summary
Random number generators are crucial for many scientific tasks, including AI experiments, where results should be repeatable. The authors found that simply sharing the full internal state of these generators does not always produce identical random sequences across different software like Python libraries. This can cause AI experiments that rely on randomness to be hard to reproduce perfectly across environments. Their work identifies why this happens and offers practical advice to help people get consistent results when using these random generators in AI workflows.
What this means in practice
- •For machine learning engineers: Ensure AI model training randomness is consistent across frameworks to replicate results accurately in multi-library setups.
- •For data scientists: Use practical guidelines from this study to avoid hidden randomness differences in Python ecosystems and improve experiment reproducibility.
Authors
Anthony Bertrand, Tom Schmitt, Engelbert Mephu Nguifo, David Hill
Abstract
Pseudorandom number generators (PRNGs) constitute indispensable computational tools across multiple scientific domains, including Monte Carlo simulations, stochastic computing, and artificial intelligence (AI). The reproducibility of such applications critically depends on the ability of PRNG implementations to generate identical sequences across software environments when initialized from the same internal state. These algorithms enable the simulation of stochastic processes while providing deterministic and repeatable behaviour, thereby facilitating reproducible experiments. Modern PRNG implementations may be initialized through either a seed or, more accurately, an initial state that exceeds the capacity of a conventional integer seed. However, reliance on a simple seed alone frequently proves insufficient to ensure consistent program execution traces across different implementations. A natural assumption is that transferring the complete internal state of a generator should guarantee identical outputs regardless of the software library used. This study examines the validity of this assumption by investigating whether complete initial states can ensure cross-library fidelity and portability of PRNG streams. We focus on two widely deployed generators, Mersenne Twister and Philox, and evaluate their implementations across four major Python ecosystems-Random, NumPy, PyTorch, and TensorFlow. We compare the sequences produced by these implementations against those generated by the original reference algorithms under identical initialization conditions. Our results demonstrate that reproducibility cannot be assumed from PRNG state transfer alone, even when implementations claim to follow the same underlying algorithm. While fidelity was successfully achieved for several implementations, significant discrepancies were observed in others. Most notably, the Philox implementation in PyTorch exhibits fundamental incompatibilities with the reference algorithm, preventing exact reproduction of generator outputs across environments. These findings challenge the common expectation that access to a full internal state of a PRNG is sufficient to ensure reproducibility across software stacks. They further highlight that implementation-specific design choices can introduce hidden barriers to experimental replication, particularly in AI workflows that rely on multiple frameworks. This work shows that implementation fidelity of a PRNG is a necessary condition for scientific reproducibility and makes two primary contributions. First, it identifies practical guidelines for achieving reliable PRNG usage and reproducibility within the Python scientific and AI ecosystem. Second, it evaluates the extent to which cross-library portability and fidelity can be recovered through user-level techniques, without requiring modifications to library source code.