Announcing our new Course: AI Red-Teaming and AI Safety Masterclass
Check it out →Few-Shot Prompting allows large language models (LLMs) to solve problems without explicit fine-tuning. However, this approach struggles as task complexity increases. Decomposed Prompting1 or DecomP is a modular approach that tackles complex tasks by breaking them into simpler sub-tasks, delegating each sub-task to LLMs or other handlers better suited to solve them.
First, a decomposer prompt outlines the process of solving a complex task through smaller sub-tasks. Each of these sub-tasks is then handled by specific sub-task handlers. These handlers can:
There are three key advantages to this technique:
Let’s consider an example of Decomposed Prompting in action. Suppose we need to concatenate the first letter of each word in a string, using spaces as separators. This can be achieved by breaking the problem into three sub-tasks:
First, the decomposer specifies the sequence of questions and corresponding sub-tasks:
QC: Concatenate the first letter of every word in "Jack Ryan" using spaces Q1: [split] What are the words in "Jack Ryan"? #1: ["Jack". "Ryan"] Q2: (foreach) [str_pos] What is the first letter of #1? #2: ["J", "R"] Q3: [merge] Concatenate #2 with spaces #3: "J R" Q4: [EOQ]
Here, [EOQ] indicates the end of the question. Each sub-task is processed by its handler, with the solution from one sub-task passed to the next. For example, the result of Q1 ("Jack", "Ryan") is used as input for Q2 in this case.
In the example, the decomposer relies on two task-specific handlers:
The few-shot prompts for these task-specific handlers might look like this:
split
Q: What are the words in "Elon Musk Tesla"? A: ["Elon", "Musk", "Tesla"] Q: What are the letters in "C++"? A: ["C", "+", "+"] ...
str_pos
Q: Concatenate ["n", "i", "e"] A: "nie" Q: Concatenate ["n", "i", "c", "e"] using spaces A: "n i c e" ...
Once all the sub-tasks are executed, the last answer before [EOQ] is returned as the final response.
A more detailed representation of the prompt execution and inference procedure is provided in the example below:
Inference procedure in Decomposed Prompting1
The decomposer prompt determines the first sub-task to be completed: splitting words in this case. The sub-task is handled by the split sub-task handler and the answer generated is appended to the decomposer prompt to get the second sub-task. The process continues until the decomposer prompt produces [EOQ]. At this point, there are no more tasks left and the last answer is returned as the solution.
Comparision of the model on k-th letter concatenation task1
Comparision of the model on sequence reversal task1
While few-shot prompting is an effective technique to improve the performance of LLMs, it can easily fail when the task to be performed is more complex than the exemplars fed to the LLM. Decomposed prompting, on the other hand, can enhance the LLM's performance by decomposing the task into sub-tasks and delegating each sub-task to the task-specific handler which can be another LLM, a function, or a trained model.