Context Switching Attack

Context switching attacks are sophisticated prompt hacking techniques that manipulate a model's context to gain unauthorized control or generate unintended responses. These attacks work by altering the contextual framework within which the model processes information.

Understanding Context

Before diving into the attacks, it's important to understand what "context" means in this setting:

• Context refers to the current frame of reference or "mindset" the model is operating in
• It's determined by the prompt and any subsequent inputs
• Different contexts can make the model more or less susceptible to manipulation

Types of Context Switching Attacks

1. Context Continuation Attack

This type of attack appends additional instructions to the existing context, often using simple or compound instruction attacks.

Example:

2. Context Termination Attack

This attack attempts to end the current context and establish a new one that's more vulnerable to exploitation.

Example:

3. Separator-Based Attacks

These attacks use visual separators (dashes, equals signs, whitespace) to create context boundaries that can confuse the model.

Example:

Footnotes

1. Schulhoff, S., Pinto, J., Khan, A., Bouchard, L.-F., Si, C., Anati, S., Tagliabue, V., Kost, A. L., Carnahan, C., & Boyd-Graber, J. (2023). Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competition. arXiv Preprint arXiv:2311.16119. ↩