Language model steering improves black box optimization of molecules and words
BOReFT: Manifold Steering of Language Models for Black-box Optimization
Machine LearningArtificial IntelligenceComputation and Language
Summary
Finding the best solutions in complex problems can be very hard, especially when relying on language models that generate ideas. The authors developed a way to search more efficiently by guiding a fixed language model through a smaller, easier-to-search space that still captures meaningful meanings. They tested this method on word puzzles and molecule design problems and found it outperforms other strong approaches. This new method bridges the gap between techniques that search discrete language outputs and those that work with continuous optimization.
What this means in practice
- •For computational chemists: Optimize molecular properties using guided search through language model hidden states for better drug and material design.$Commercial implications: Enables designing improved molecules faster for pharmaceutical and materials companies using enhanced black-box optimization.
- •For natural language processing developers: Improve keyword or word-based puzzle solving by steering language models through a semantically meaningful search domain.
Authors
Dhruv Agarwal, Rico Angell, Kavitha Srinivas, Tahira Naseem, Horst Samulowitz, Willie Neiswanger, Andrew McCallum
Abstract
Language models are increasingly used as proposal models for black-box search, from program optimization to molecular design. Existing approaches typically improve proposals through iterative prompting or parameter updates, offering limited control over how completely and efficiently the model's search space is explored. Continuous optimization methods, such as Bayesian optimization, provide a principled way to search but require a suitable domain to operate over. To address this, we introduce BOReFT, which learns a compact, low-dimensional space of hidden-state interventions in a frozen language model, and uses this space as the search domain for Bayesian optimization with an external scoring function. Empirically, we find that the learned domain spans semantic regions and exhibits smoothness properties that support search. Theoretically, we show that semantic coverage and interpolation control the best score available in the learned space, and that decoding from this space yields a standard stochastic-bandit observation model for adaptive search. We evaluate BOReFT on the interpretable word search task "Semantle" and on three more real-world discovery tasks in de novo molecule property optimization. Compared to strong LLM baselines, BOReFT finds in Semantle a higher number of hidden targets and, on two out of three molecular objectives, achieves higher property scores. Consequently, our method provides a principled new bridge between discrete proposal spaces of LLM-based search and continuous black-box optimization.