Competition shapes how foragers balance exploring and exploiting resources
Competitive exploration in structured environments
Computer Science and Game Theory
Summary
When animals or agents search for food, they face a choice: keep using a known resource or explore for something better. The authors studied a simple model where two foragers compete on a line with a small reward nearby and a bigger reward far away. They found that competing changes the best strategy for exploring, sometimes leading both to explore too much or not enough. Adding a way for one forager to signal to the other that a big reward was found helps both do better together.
What this means in practice
- •For robotic swarm developers: Design coordination strategies for multiple robots exploring environments where resources must be shared to improve collective outcomes.
- •For supply chain managers: Improve decision rules in competitive resource allocation scenarios by incorporating signaling mechanisms for better coordination among agents.
Authors
James P. Wolfe, Joshua B. Plotkin
Abstract
Organisms must constantly weigh exploiting the resources they know against exploring for better ones. When foragers compete, the best behavior for each forager also depends on what its competitors do. How does competition change the strategic balance between exploration and exploitation? Here we address this question in a minimal model: two $ε$-greedy temporal-difference learners on a one-dimensional lattice, who start at a small reward at one end and are unaware of a larger reward at the other. For a lone forager, the optimal exploration rate increases monotonically with the value of the distant reward, and with its distance up to a critical distance. When two foragers share the lattice and split rewards, the choice of exploration rate becomes a game whose Nash equilibria have both foragers exploring, both staying, or one exploring while the other stays (anti-coordination). Under local best-response dynamics, pairs of foragers often converge on both exploring or both staying, even when anti-coordination would be socially optimal. This social dilemma arises for a wide range of parameters, and it persists even in a simplified model without explicit spatial structure. A minimal, naturalistic signal, in which one forager informs the other that it has found the distant reward, can remove many of the dilemmas in which both foragers explore, and can produce a Nash equilibrium whose collective payoff exceeds that of every outcome available without signaling.