Researchers study planning from random exploration without policy-improvement training

A paper titled “Learning to Plan from Random Exploration” examines whether an agent can learn to navigate toward a goal from experience gathered before that goal is known. The researchers focus on planning over long distances without training the agent through repeated policy improvement. Their approach treats random movement through an environment as useful evidence about which places are related, rather than as experience that must include action labels or rewards. The paper’s abstract describes tests spanning maze routes, egocentric navigation and manipulation planning.

The researchers analyze how relationships between observations change with the time horizon used to measure them. At shorter horizons, those relationships reflect geodesic structure: which paths are relatively direct through the environment. At longer horizons, they can indicate connections among regions, until mixing makes those differences harder to distinguish. The authors describe these temporal relationships as a multiscale map of an environment, with different horizons contributing different kinds of information.

To learn the relationships, the method uses a conditional energy-based model that estimates temporal log-density ratios from embeddings conditioned on the horizon. It trains on pairs of observations using noise-contrastive estimation, without action or reward labels. At planning time, a separate local dynamics model predicts the outcomes of possible actions. The temporal model then assesses how much those predicted outcomes appear to advance toward the goal, combining or selecting progress estimates across horizons. The agent takes one action and plans again, while both models remain fixed.

The abstract says experiments show long-range maze planning from random exploration, using either states or images. It also reports demonstrations of egocentric navigation from random exploration and manipulation planning based on suboptimal data. The paper describes learned score fields, embedding probes and planned routes as exhibiting properties of a multiscale cognitive map. For AI builders, the approach offers a specific division of labor: learn broad temporal relationships from observations, while using a local model to predict immediate action outcomes. The supplied abstract does not provide numerical performance results, so it does not establish how the method compares with other planners.