🟦 Multi-Chain Reasoning (MCR)

What is Multi-Chain Reasoning?

Multi-Chain Reasoning (MCR) is a technique to improve multi-hop question answering (QA) by meta-reasoning over multiple chains of thought (CoT). Unlike traditional methods that rely on the final answers of individual reasoning chains, MCR integrates intermediate reasoning steps across chains to build a comprehensive explanation and arrive at the final answer.

How It Works

• Traditional Chain-of-Thought (CoT): Breaks a complex question into step-by-step reasoning. For example, when answering, “Did Brad Peyton need to know about seismology?” CoT produces separate reasoning chains such as:
  • Chain 1: Define Brad Peyton’s role → Define seismology → Relate them.
  • Chain 2: Discuss relevance of seismology for film directors.
• Multi-Chain Reasoning (MCR): Instead of voting on final answers like with Self-Consistency (SC), MCR synthesizes the intermediate steps from all chains to combine facts and reduce errors, resulting in better explanations and predictions.

Why Choose MCR?

• Improved Accuracy: Outperforms state-of-the-art QA methods by up to 5.7%.
• Interpretability: Produces human-verifiable explanations.
• Broad Applicability: Effective on both implicit (commonsense) and explicit (fact-based) multi-hop reasoning tasks.

MCR vs. Self-Consistency (SC)

How to Use Multi-Chain Reasoning

Step 1: Decomposition

Break the main question into smaller, answerable parts. Use a decomposition model (like an LLM) to generate sub-questions and intermediate answers.

Example

Question: "How many ants could fit into The Shard?" Decompose into:

• What is the height of The Shard?
• How many ants are in the world?

In this step, Multi-Chain Reasoning (MCR) actually uses Self-Ask prompting technique to generate intermediate steps. Here’s a template:

 Copy

Prompt

---

Given the following question, answer it by providing follow up questions and intermediate answers. If no follow up questions are necessary, answer the question directly.

Question: Who is the mother of the director of film Polish-Russian War (Film)?

Are follow up questions needed here: Yes.

Follow up: Who is the director of the film Polish-Russian War (Film)?

Intermediate answer: The director of the film Polish-Russian War is Xawery Zuławski.

Follow up: Who is the mother of Xawery Zuławski?

Intermediate answer: The mother of Xawery Zuławski is Małgorzata Braunek.

So the final answer is: Rick Scott Małgorzata Braunek.

Question: Who is Catherine Of Pomerania, Countess Palatine Of Neumarkt’s father-in-law?

Are follow up questions needed here: Yes.

Follow up: Who is the husband of Catherine of Pomerania, Countess Palatine of Neumarkt?

Intermediate answer: The husband of Catherine of Pomerania, Countess Palatine of Neumarkt is John, Count Palatine of Neumarkt.

Follow up: Who is the father of John, Count Palatine of Neumarkt?

Intermediate answer: The father of John, Count Palatine of Neumarkt is Rupert III of the Palatinate.

So the final answer is: Rupert III of the Palatinate.

Question: [Your complex question]

Are follow up questions needed here:

Step 2: Chain Generation

Use a language model to create multiple reasoning chains for the intermediate questions.

Use chain-of-thought prompting. Here's a template for you:

 Copy

Prompt

---

Q: John has 10 apples. He gives away 4 and then receives 5 more. How many apples does he have?

A:

• John starts with 10 apples.
• He gives away 4, so 10 - 4 = 6.
• He then receives 5 more apples, so 6 + 5 = 11.

Final Answer: 11

Q: [Your Question]

Step 3: Meta-Reasoning

The core innovation of Multi-Chain Reasoning (MCR) lies in this step.

Here, a "meta-reasoner" LLM analyzes and combines insights from all reasoning chains, deciding which facts are most relevant. This process ensures the final answer is accurate and supported by a cohesive explanation.

Using reasoning chains from step 2, combine them into one prompt and ask the model to produce the final answer:

 Copy

Prompt

---

Q: [Your Question]

Based on these reasoning chains, generate a final answer:

• [Reasoning chain 1]
• [Reasoning chain 2]
• [Reasoning chain 3]
• ...

A:

Conclusion

Meta reasoning over multiple COTs is a powerful way to improve the reasoning and accuracy of LLMs, and is particularly useful for complex problems that require sophisticated logic to solve. While it may be time-consuming to generate enough lines of reasoning to be useful, it's definitely worth it for the resulting improvements in accuracy, reasoning, and interpretability.