LLM-based Optimization of Compound AI Systems: A Survey

📄 Full Paper

💬 Ask

This paper provides a comprehensive survey of the emerging field of LLM-based optimization of compound AI systems. Compound AI systems are systems that integrate multiple calls to an LLM (Large Language Model), such as a retriever, a code interpreter, or tools, in order to perform complex tasks.

The main challenge in optimizing a compound AI system is that its behavior is primarily driven by parameters like instructions and tool definitions, which are often difficult to hand-optimize. The paper presents a framework for understanding and optimizing compound AI systems using an LLM as the optimizer, which avoids the need for gradient computation and can handle complex code and instructions.

The paper draws an analogy from program analysis to provide a unified view of how an LLM optimizer can be prompted to optimize a compound AI system. In particular, the authors distinguish between static program analysis, which analyzes the system without executing it, and dynamic program analysis, which analyzes the system’s runtime behavior given input data.

The paper explores several approaches to LLM-based optimization, including:

• Static program analysis: This approach involves prompting the LLM optimizer with input-output pairs from the training data and asking it to generate instructions or tools. For example, one approach uses random variations of prompts to find the best instruction for a specific task.
• Dynamic program analysis: This approach involves providing the LLM optimizer with feedback based on the system’s runtime behavior. The feedback can be in the form of textual critique, execution errors, or a trace of the system’s execution. For example, one approach uses a technique called “trace propagation,” which records a textual representation of the system’s execution trace and prompts the LLM optimizer to analyze it and suggest updates to the parameters.

The paper also explores credit assignment techniques for co-optimizing multiple parameters in compound AI systems. This is a challenging problem because it requires understanding the contribution of each parameter to the system’s overall performance. The authors discuss two approaches: backpropagation and trace propagation.

Backpropagation is a method for computing gradients that is commonly used in deep learning. The authors propose to adapt this approach to text-based systems, where tensors are replaced with textual elements like instructions and tools. Trace propagation is a more recent approach that uses the computational graph of a system to guide optimization. This approach is less sensitive to error accumulation than backpropagation.

The paper concludes by discussing the broader impact of LLM-based optimization on compound AI systems, particularly in terms of process supervision and safety. The authors argue that LLM-based optimization can help to improve the controllability and interpretability of LLMs by breaking down complex tasks into smaller, more manageable pieces. They also discuss how LLM-based optimization can help to mitigate some of the risks associated with large language models, such as unpredictable emergent behavior during training.

Overall, this paper provides a valuable overview of the emerging field of LLM-based optimization of compound AI systems. The paper’s framework and insights are likely to be of interest to researchers and practitioners working in the field of AI, particularly those developing and deploying complex AI systems that integrate LLMs with other components.

Chat about this paper

To chat about this paper, you'll need a free Gemini API key from Google AI Studio.

Your API key will be stored securely in your browser's local storage.