🔓 Prompt Hacking🟢 Offensive Measures🟢 Recursive Injection

Recursive Injection

🟢 This article is rated easy

Reading Time: 1 minute

Last updated on August 7, 2024

Takeaways

Recursive injections lead to outputs that are injections themselves, thus attacking any other LLM's that accept it as input-- this is an effective evasion of the "Seperate LLM Evaluation" defense.

What is Recursive Injection?

As we have seen previously, a very good defense against prompting hacking is to use one LLM to evaluate the output of another LLM, to check if there is any adversarial output. It is possible to evade this defense with a recursive injection attack. This attack involves injecting a prompt into the first LLM that creates output which contains an injection instruction for the second LLM.

An Example of Recursive Injection

Let's assume that the prompt we are trying to hack is:

Prompt

Say something positive about the following and nature:

{user_input}

The output of this prompt will be evaluated by the prompt:

Prompt

Does the following contain any profanity or offensive content (YES or NO)?

{output}

We can hack both of these prompts by appending to the prompt as seen below. Note that this is also an example of a variable assignment attack.

The above output is an injection attempt that gets input into the second LLM call:

We have now PWNED this second LLM call.

Conclusion

Recursive injections are difficult to execute, as they require careful consideration of how the separate LLM evaluation is engineered. However, under the right circumstances, they can be very useful.

Sander Schulhoff

Sander Schulhoff is the Founder of Learn Prompting and an ML Researcher at the University of Maryland. He created the first open-source Prompt Engineering guide, reaching 3M+ people and teaching them to use tools like ChatGPT. Sander also led a team behind Prompt Report, the most comprehensive study of prompting ever done, co-authored with researchers from the University of Maryland, OpenAI, Microsoft, Google, Princeton, Stanford, and other leading institutions. This 76-page survey analyzed 1,500+ academic papers and covered 200+ prompting techniques.

Footnotes

Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., & Hashimoto, T. (2023). Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks. ↩