Building large language model agents using deep learning methods
Building LLM Agent Systems the Deep Learning Way: From Modular Design to Architecture Search
Computation and Language
Summary
Creating complex systems that use large language models (LLMs) is usually done by manually piecing together parts, which can be slow and not very efficient. The authors suggest treating these systems like deep learning models, breaking them into modular parts similar to how neural networks are built. They also introduce ways to fine-tune the prompts in LLMs using feedback, almost like teaching a deep learning model with back-propagation. Their approach includes automatically searching for the best system design, which showed measurable improvements in performance.
What this means in practice
- •For software engineers: Design complex LLM-based agents using modular deep learning-inspired building blocks for improved performance and easier optimization.
- •For machine learning engineers: Automatically optimize prompt strategies and agent architectures for LLM systems to achieve better task-specific performance without manual tuning.
- •For customer support platform developers: Implement LLM agent systems that adapt prompts from user feedback efficiently to enhance automated assistance quality.$Commercial implications: Enables building adaptive customer support chatbots with improved response accuracy, providing a competitive product advantage.
Authors
Tao Feng, Pengrui Han, Zhongjie Dai, Jiaxuan You
Abstract
Large Language Models (LLMs) have revolutionized AI research and enabled exciting agent systems. To build a complex LLM agent system, most existing research relies on insights from other domains or heuristics to manually build the agent system. However, this approach often requires heavy hand-engineering and fails to fully optimize for the downstream task of interest. Inspired by the tremendous success of deep learning, we propose to construct LLM agent systems in a modular manner, similar to building a deep neural network. Our key insight is to make analogies between LLM building blocks, such as retrievals, memories, and prompting strategies, and the successful deep learning modules, such as MLPs, attention, and recurrent modules. We further design forward inference and feedback mechanisms for LLMs, where prompts in LLMs are considered as the weights in deep models, and the prompt optimization from feedback is analogous to the back-propagation algorithm. We additionally leverage a search algorithm to search for the best configuration of LLM agent systems, similar to the neural architecture search (NAS) in deep learning research. Comprehensive experimental results demonstrate that the proposed deep learning recipe for LLM agent systems is highly effective, in particular: (1) Organizing LLM modules into deep-learning-style architectures yields noticeable performance gain; (2) Automatic prompt optimization, equivalent to backpropagation, is efficient in incorporating feedback from the task of interest and achieves at least 5% performance improvement; (3) NAS equivalent algorithm works well for further optimizing the LLM agent system architecture with 11% performance gain compared with randomly designed architectures. Overall, our research demonstrates the exciting opportunity of transferring the success of deep learning to building LLM agent systems.