🟢 Rephrase and Respond (RaR) Prompting

Last updated on August 27, 2024 by Valeriia Kuka

What is Rephrase and Respond (RaR)?

Prompt engineering consists of applying specific techniques when constructing a prompt with a goal to improve the quality of the response provided by Large Language Models (LLMs).

The creators of Rephrase and Respond (RaR)¹ took this concept a step further. Instead of searching for a perfect prompt formula, they asked, "What if we let the model rephrase the question itself and then answer it?"

They focused on question-answering tasks and proposed the idea of incorporating a rephrasing request directly into the prompt. This can be as simple as adding one sentence: "Rephrase and expand the question, and respond."

It's a concise and effective technique that can be combined with other techniques and has demonstrated improvements on multiple benchmarks.

Example

Let's take an example question² where we ask a model to answer whether a known person is born in an even/odd month. The ambiguity for the model here lies in the definition of the "even" and "odd" month:

• Does "odd" mean "strange"? Then, February can be considered odd because of an irregular number of days
• Does "even/odd" month refer to a month with an even/odd number of days?
• How can "even/odd" be applied to the calendar? The concept of even and odd applies to integers, not to calendar months or dates.

Here's an example:

Let's try to ask the model to rephrase:

Now, it answers correctly.

How to Use Rephrase and Respond (RaR)?

Apart from adding "Rephrase and expand the question, and respond" to the prompt, you can use many other variations for asking a model to rephrase the question:

• Reword and elaborate on the inquiry, then provide an answer.
• Reframe the question with additional context and detail, then provide an answer.
• Modify the original question for clarity and detail, then offer an answer.
• Restate and elaborate on the inquiry before proceeding with a response.

All models can benefit from rephrasing questions, with more advanced models expected to gain a larger improvement.

Example, Coin Flip

In this example³, we provide the model with an initial position of the coin and describe the subsequent actions of people who either flipped or did not flip the coin. Then, we ask the model to provide an answer "Yes" or "No" to the question "Is the coin still heads up?."

Let's compare the two results that we get from a simple prompt and a Rephrase and Respond (RaR) prompt. In this example, the correct answer is "Yes, the coin still heads up."

This is incorrect. Let's try to ask the model to rephrase:

Correct!

How to Use Two-Step Rephrase and Respond (RaR)?

Another way to use Rephrase and Respond is a two-step procedure, separating the rephasing step from the responding step in RaR:

• Step 1, a rephrasing LLM: Given the original {question}, we ask a rephrasing LLM to generate a self-rephrased query called {rephrased_question}:

• Step 2, a responding LLM: We prompt a responding LLM with both the original {question} and the {rephrased_question}:

Interestingly, we can use the same LLM for both steps or different LLMs for each. One of the potential ways to use this is to let stronger LLMs rephrase for weaker LLMs to respond for better results.

How to Combine Rephrase and Respond (RaR) with Few-Shot Chain-of-Thought (CoT) Prompting?

Although few-shot Chain-of-Thought (CoT)³ is not included into the zero-shot prompting techniques that we study here, it has been the most effective CoT technique for its usage of few-shot examples.

That's why we think it's important to mention that Rephrase and Respond (RaR) can be combined with a few-shot Chain-of-Thought (CoT) to achieve even better results.

To combine these techniques, again, just add this phrase to the end of the prompt: "Given the above question, rephrase and expand it to help you do better answering. Maintain all information in the original question."

Example, Letter Concatenation Task

In this example, the task centers on symbolic reasoning. We ask the LLM to concatenate the final letters of a given list of names. Here's how you can combine RaR and CoT for this task:

Limitations of Rephrase and Respond (RaR)

Rephrase and Respond (RaR) is a compelling prompt engineering technique, but it does come with some limitations.

Here are a few potential drawbacks:

• While rephrasing can help clarify ambiguous questions, it can also make straightforward queries unnecessarily complex.
• The success of RaR hinges on the model's ability to rephrase questions accurately. If the rephrasing misses the mark, the response may not be as effective.
• Rephrasing can sometimes inadvertently alter the original question's intent or focus, leading to a response that doesn’t fully address the user's needs.

Conclusion

RaR is a valuable tool for improving LLM outputs for question-answering tasks, especially when carefully implemented alongside other prompt strategies. By leveraging RaR, you can achieve clearer and more accurate responses.

Footnotes

1. Deng, Y., Zhang, W., Chen, Z., & Gu, Q. (2024). Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselves. https://arxiv.org/abs/2311.04205 ↩

2. Allen-Zhu, Z., & Li, Y. (2024). Physics of Language Models: Part 3.2, Knowledge Manipulation. https://arxiv.org/abs/2309.14402 ↩

3. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2023). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. https://arxiv.org/abs/2201.11903 ↩ ↩²