Last updated on October 3, 2024
Technique | Institution | Date of Publication | Paper |
---|---|---|---|
Analogical Prompting | Google DeepMind, Stanford University | Oct 2023 | Large Language Models as Analogical Reasoners |
Analogical Prompting^{} is a new approach that enhances the reasoning process of Large Language Models (LLMs) by drawing inspiration from human analogical reasoning. Inspired by our ability to draw connections between past experiences and current challenges, analogical prompting encourages LLMs to self-generate relevant examples before tackling new problems.
For example, if you're asked to solve a complex math problem, you might think of similar problems you’ve solved before and apply that knowledge. Similarly, analogical prompting instructs LLMs to recall or generate examples and solutions that resemble the current task. The LLM then uses these examples to better solve the original problem.
This method not only removes the need for manually labeled examples but also tailors the exemplars to the specific problem, making it more adaptive.
Analogical Prompting builds upon and improves earlier prompting techniques such as:
Analogical Prompting can be implemented with a specific instruction template in the LLM's input. Here's a general template you can follow:
[Insert problem here]
Instruction: Relevant problems: Solve the initial problem:
Your task is to tackle mathematical problems. When presented with a math problem, recall relevant problems as examples. Afterward, proceed to solve the initial problem.
[Insert problem here]
Instruction: Relevant problems: Recall examples of math problems that are relevant to the initial problem. Your problems should be distinct from each other and from the initial problem (e.g., involving different numbers and names).
For each problem: After "Q: ", describe the problem After "A: ", explain the solution.
Solve the initial problem:
If the problem is to calculate the area of a square given its vertices, the model first generates a similar problem with a known side length, solves it, and then applies this reasoning to the original problem.
Q: What is the area of the square with the four vertices at (-2, 2), (2, -2), (-2, -6), and (-6, -2)?
Recall relevant problems: Solve the initial problem:
Q: What is the area of the square with a side length of 5? A: The area of a square is found by squaring the length of its side. So, the area of this square is 5^2 = 25. …
To find the area of the square, we need to find the side length. The length is … \sqrt-8 = \sqrt32. So, the area of the square is (\sqrt32)^2 = 32.
Analogical prompting has been tested across various tasks including math problem-solving and code generation. The results show that it consistently outperforms both Zero-Shot and Few-Shot CoT methods.
Method | Accuracy (GSM8K) |
---|---|
Zero-Shot | 75.0% |
Few-Shot CoT | 76.7% |
Analogical Prompting | 77.8% |
Analogical prompting particularly shines in complex tasks that require reasoning across multiple steps, such as solving competitive programming challenges or advanced math problems.
Analogical prompting allows LLMs to generate their own reasoning examples tailored to each problem, offering a flexible and powerful method to guide reasoning without the need for labeled data. This approach improves performance on reasoning tasks and opens new possibilities for solving more complex problems where fixed examples are impractical or unavailable.
Yasunaga, M., Chen, X., Li, Y., Pasupat, P., Leskovec, J., Liang, P., Chi, E. H., & Zhou, D. (2024). Large Language Models as Analogical Reasoners. https://arxiv.org/abs/2310.01714 ↩