Thread of Thought (ThoT)

Information and Links

Technique	Institution	Date of Publication	Paper
Thread of Thought (ThoT)	University of Macau, Microsoft Corporation, University of Technology Sydney	November 2023	Thread of Thought Unraveling Chaotic Contexts

Introduction

Large Language Models (LLMs) have achieved a remarkable performance in natural language understanding and generation. They are used for a variety of tasks such as translation, summarization, question answering, etc. As LLMs become more capable, their applications grow increasingly diverse and complex.

In advanced applications like Retrieval-Augmented Generation (RAG), users often input vast texts containing tens of thousands of tokens. This data may vary widely in relevance and connectivity, with some details critical to the query and others irrelevant. Such scenarios exemplify "Chaotic Contexts"—more challenging than mere "long contexts," which involve a large token count but lack the same degree of informational disarray.

Challenges in Handling Chaotic Contexts

Current techniques for managing chaotic contexts include:

• Retrieval-Augmentation with Long Context Extensions: Combining extended model contexts with retrieval mechanisms.
• Prompt Streamlining: Reducing irrelevant or redundant information within the input.

However, both approaches often require fine-tuning or retraining, demanding labeled data and substantial computational resources.

Introducing Thread of Thought Prompting

Thread of Thought (ThoT) prompting offers an alternative that eliminates the need for fine-tuning. Instead, it methodically segments and analyzes extended contexts, extracting relevant details to address specific queries. Much like how humans sift through large volumes of information by isolating key points, ThoT helps LLMs handle complex inputs efficiently.

In short, ThoT prompting helps LLMs to methodically process, summarize, and analyze extended contexts in manageable parts, resulting in more accurate responses in tasks such as multi-turn conversations and complex question answering.

Key Features of ThoT Prompting

1. Guided Step-by-Step Processing: ThoT prompts the model to "walk through" chaotic contexts, analyzing and summarizing each part progressively.
2. Selective Attention: By breaking down information into smaller segments, ThoT helps the model ignore irrelevant details and focus on pertinent information.
3. Two-Tiered Prompting System: This system involves a first pass for analysis and a second pass for summarizing and deriving conclusions, enhancing the model's ability to synthesize complex information.

For example, if the model is asked about the founding place of "Reclam" from multiple retrieved passages, ThoT helps the model sift through each passage in stages, retaining only the relevant details about "Reclam" and its founding location.

How to Use ThoT Prompting

ThoT prompting uses a simple and effective two-step process. As discussed previously, it doesn't require any fine-tuning or re-training and can easily be used with a variety of existing language models. Let's briefly discuss the two steps involved in the process:

Step 1: Initiating the Reasoning

The goal of this step is to provide the "chaotic" context, followed by the question and a trigger to elicit a response from the LLM. The prompt template for this step looks like this:

Example in Action

Now, let's look at a concrete example. Say, given multiple sources of information regarding Reclam, you want to know where Reclam was founded. Your input to the model will look like:

Notice the trigger sentence "Walk me through this context in manageable parts step by step, summarizing and analyzing as we go". The trigger sentence guides LLM to navigate chaotic context by asking it to summarize and analyze the information as it goes thorough the input context.

Step 2: Refining the Conclusion

The goal of the second step is to extract the conclusion using the response of the LLM in the previous step. The prompt template for this step looks like this:

Example in Action

Let's use the template to extract the final answer from the LLM.

How ThoT Differs from Existing Techniques

ThoT stands apart from traditional approaches:

• Enhanced Focus vs. Chain-of-Thought (CoT): While CoT improves reasoning, it falters in chaotic contexts. ThoT's segmentation approach ensures no key details are overlooked.
• Plug-and-Play Simplicity: ThoT works with pre-trained models without the need for additional training, making it cost-effective and easy to implement.
• Adaptability: ThoT excels across tasks like retrieval-augmented generation, multi-turn dialogues, and summarizing unstructured data.

Applications and Benefits of Thread of Thought

ThoT is useful in scenarios where complex or unrelated information must be processed for relevant insights:

• Question Answering with Distractors: Handling retrieval-augmented generation tasks with distracting passages.
• Multi-Turn Dialogues: Supporting coherent responses by tracking conversation history.
• Chaotic Data Summarization: Summarizing unstructured, long-form content like legal or research documents.

What are ThoT Prompting Results?

ThoT was evaluated on two chaotic context scenarios: retrieval-augmented generation (RAG) and multi-turn conversation response. Two datasets: PopQA and EntityQ were used to evaluate the performance on the RAG task and the MTCR dataset was used to assess multi-turn conversation response.

In all datasets, ThoT significantly outperformed traditional methods. Here are some key results:

Dataset	Method	Model	Accuracy Improvement
PopQA	ThoT Prompting	GPT-3.5-turbo	+16% over CoT
EntityQ	ThoT Prompting	GPT-3.5-turbo	+8.5% over CoT
MTCR	ThoT Prompting	GPT-3.5-turbo	+14.8% over CoT

Conclusion

Thread of Thought (ThoT) prompting introduces a new way for language models to tackle complex, chaotic contexts by breaking down information into segments, analyzing each, and synthesizing relevant insights. This approach not only enhances accuracy but also integrates smoothly with existing models, making it an efficient and versatile tool for applications in retrieval-augmented generation and other challenging tasks requiring nuanced comprehension.

Bhuwan Bhatt

Bhuwan Bhatt, a Machine Learning Engineer with over 5 years of industry experience, is passionate about solving complex challenges at the intersection of machine learning and Python programming. Bhuwan has contributed his expertise to leading companies, driving innovation in AI/ML projects. Beyond his professional endeavors, Bhuwan is deeply committed to sharing his knowledge and experiences with others in the field. He firmly believes in continuous improvement, striving to grow by 1% each day in both his technical skills and personal development.

Footnotes

1. Zhou, Y., Geng, X., Shen, T., Tao, C., Long, G., Lou, J.-G., & Shen, J. (2023). Thread of Thought Unraveling Chaotic Contexts. https://arxiv.org/abs/2311.08734 ↩