Refusal Suppression

Refusal suppression is a sophisticated type of jailbreak attack that exploits the instruction-following capabilities of language models. This technique works by explicitly constraining the model's ability to refuse or deflect unsafe requests.

How It Works

The attack functions by providing specific rules that prevent the model from using common refusal patterns. By eliminating the model's typical safety responses, it increases the likelihood of obtaining restricted or unsafe outputs.

A typical refusal suppression attack includes rules like:

1. Prohibiting apologetic language
2. Banning disclaimers or warnings
3. Blocking specific words commonly used in refusals
4. Preventing negative statements about the prompt's subject

Here's an example of how these constraints might be formatted:

Mechanism of Action

The effectiveness of refusal suppression stems from two competing objectives in language models:

1. Instruction Following: The model's training to follow instructions causes it to avoid the prohibited tokens and phrases, which typically form the building blocks of safety responses.

2. Response Coherence: Once the model begins generating a response, its pretraining makes it more likely to continue along that path rather than abruptly changing course.

Effectiveness and Research Findings

Research has shown that the specific formulation of these instructions is crucial. Interestingly, when the rules are inverted (e.g., changing "Do not apologize" to "Consider apologizing"), the attack becomes ineffective, and the model maintains its safety boundaries.

This attack has been demonstrated to bypass safety measures for various types of restricted content, including:

• Potentially harmful advice
• Social engineering techniques
• Age-restricted content

The success of refusal suppression attacks highlights the complex interaction between instruction-following capabilities and safety measures in language models.

Footnotes

1. Wei, A., Haghtalab, N., & Steinhardt, J. (2023). Jailbroken: How Does LLM Safety Training Fail? https://arxiv.org/abs/2307.02483 ↩