πŸ˜ƒ Basics
🧠 Advanced
Zero-Shot
🟒 Introduction
🟒 Emotion Prompting
🟒 Role Prompting
🟒 Re-reading (RE2)
🟒 Rephrase and Respond (RaR)
🟦 SimToM
β—† System 2 Attention (S2A)
Few-Shot
🟒 Introduction
🟒 Self-Ask
🟒 Self Generated In-Context Learning (SG-ICL)
🟒 Chain-of-Dictionary (CoD)
🟒 Cue-CoT
🟦 Chain of Knowledge (CoK)
β—† K-Nearest Neighbor (KNN)
β—†β—† Vote-K
β—†β—† Prompt Mining
Thought Generation
🟒 Introduction
🟒 Chain of Draft (CoD)
🟦 Contrastive Chain-of-Thought
🟦 Automatic Chain of Thought (Auto-CoT)
🟦 Tabular Chain-of-Thought (Tab-CoT)
🟦 Memory-of-Thought (MoT)
🟦 Active Prompting
🟦 Analogical Prompting
🟦 Complexity-Based Prompting
🟦 Step-Back Prompting
🟦 Thread of Thought (ThoT)
Ensembling
🟒 Introduction
🟒 Universal Self-Consistency
🟦 Mixture of Reasoning Experts (MoRE)
🟦 Max Mutual Information (MMI) Method
🟦 Prompt Paraphrasing
🟦 DiVeRSe (Diverse Verifier on Reasoning Step)
🟦 Universal Self-Adaptive Prompting (USP)
🟦 Consistency-based Self-adaptive Prompting (COSP)
🟦 Multi-Chain Reasoning (MCR)
Self-Criticism
🟒 Introduction
🟒 Self-Calibration
🟒 Chain of Density (CoD)
🟒 Chain-of-Verification (CoVe)
🟦 Self-Refine
🟦 Cumulative Reasoning
🟦 Reversing Chain-of-Thought (RCoT)
β—† Self-Verification
Decomposition
🟒 Introduction
🟒 Chain-of-Logic
🟦 Decomposed Prompting
🟦 Plan-and-Solve Prompting
🟦 Program of Thoughts
🟦 Tree of Thoughts
🟦 Chain of Code (CoC)
🟦 Duty-Distinct Chain-of-Thought (DDCoT)
β—† Faithful Chain-of-Thought
β—† Recursion of Thought
β—† Skeleton-of-Thought
πŸ”“ Prompt Hacking
🟒 Defensive Measures
🟒 Introduction
🟒 Filtering
🟒 Instruction Defense
🟒 Post-Prompting
🟒 Random Sequence Enclosure
🟒 Sandwich Defense
🟒 XML Tagging
🟒 Separate LLM Evaluation
🟒 Other Approaches
🟒 Offensive Measures
🟒 Introduction
🟒 Simple Instruction Attack
🟒 Context Ignoring Attack
🟒 Compound Instruction Attack
🟒 Special Case Attack
🟒 Few-Shot Attack
🟒 Refusal Suppression
🟒 Context Switching Attack
🟒 Obfuscation/Token Smuggling
🟒 Task Deflection Attack
🟒 Payload Splitting
🟒 Defined Dictionary Attack
🟒 Indirect Injection
🟒 Recursive Injection
🟒 Code Injection
🟒 Virtualization
🟒 Pretending
🟒 Alignment Hacking
🟒 Authorized User
🟒 DAN (Do Anything Now)
🟒 Bad Chain
πŸ”¨ Tooling
Prompt Engineering IDEs
🟒 Introduction
GPT-3 Playground
Dust
Soaked
Everyprompt
Prompt IDE
PromptTools
PromptSource
PromptChainer
Prompts.ai
Snorkel 🚧
Human Loop
Spellbook 🚧
Kolla Prompt 🚧
Lang Chain
OpenPrompt
OpenAI DALLE IDE
Dream Studio
Patience
Promptmetheus
PromptSandbox.io
The Forge AI
AnySolve
Conclusion
πŸ”¨ ToolingPrompt Engineering IDEs🟒 Introduction

Introduction

🟒 This article is rated easy
Reading Time: 2 minutes
Last updated on August 7, 2024

Sander Schulhoff

Prompt engineering is an iterative design task. More formalized iterative design fields (e.g. logo design) have well established tooling which allows designers to be more efficient. Similar tooling exists in the prompt engineering space, with a variety of prompt engineering IDEs (integrated development environments) having recently been created. There exists a surprising range of such tooling, with everything from research interfaces to professionally designed IDEs. This chapter will provide an overview of some of the tools which may be of interest to you as a prompt engineer.

See the final article in this chapter for my recommendations on tools to use.

Due to waitlists, I have not yet been able to access all IDEs mentioned here. Additionally, due to the number of tools coming out, not all have a dedicated page.

Here is a table of the tools we will be discussing in this chapter:

Text Prompt IDEs

NameHosted SolutionOpen SourceResearch FocusedBusiness ModelLaunchedModalitiesSupported Providers
GPT-3 Playgroundβœ…βŒβŒPay per tokensβœ…TextOpenAI
Dustβœ…βœ…βŒβœ…TextOpenAI, Cohere
Everypromptβœ…βŒFreemiumβœ…TextOpenAI
Promptmetheusβœ…βŒβœ…Freemiumβœ…TextAnthropic, Cohere, OpenAI, PaLM 2, NLP Cloud, AI21 Labs, Aleph Alpha
PromptIDEβœ…Code TBDβœ…Noneβœ…Text
PromptSourceβŒβœ…βœ…Noneβœ…Text
PromptChainerβŒβœ…Noneβœ…Text
PromptMakerβŒβœ…Text
PromptSandboxβœ…βœ…βœ…Freeβœ…TextOpenAI
ChainForgeβœ…βœ…βœ…Freeβœ…TextOpenAI
NameHosted SolutionOpen SourceResearch FocusedBusiness ModelLaunchedModalitiesSupported Providers
Prompts.aiβœ…βœ…βŒNoneβœ…TextOpenAI
Snorkelβœ…βŒβŒβœ…Text
Human Loopβœ…βŒWait listText
Spellbook (Scale AI)βœ…βŒβŒWait listText
Kollapromptβœ…βŒβŒUnder DevelopmentWait listText, Image, AudioOpenAI, Stable Diffusion
Promptableβœ…βŒβŒWait listTextOpenAI
DiscuroAIβœ…βŒβŒβœ…Text, ImageOpenAI
PromptShakeβœ…βŒβŒWait listText
GPT IDEβŒβŒβŒβœ…Text, images + audio laterOpenAI, later Stability.AI and more
The Forge AIβœ…βŒβŒMarketplaceWait listText, ImagesOpenAI, Stable Diffusion
Orquesta AI Promptsβœ…βŒβŒEnterpriseβœ…TextCustom, Public, Private LLMs
AnySolveβœ…βŒβŒFreemiumβœ…Text, images + audio laterOpenAI, Stability.AI and more

Image Only IDEs

NameHosted SolutionOpen SourceResearch FocusedBusiness ModelLaunchedModalitiesSupported Providers
DALLΒ·Eβœ…βŒβŒBuy Creditsβœ…Text2ImageOpenAI DALLE
Dream Studioβœ…βŒβŒBuy Creditsβœ…Text2ImageStable Diffusion
Patienceβœ…βŒβŒBuy Creditsβœ…Text2ImageStable Diffusion, OpenAI
getimg.aiβœ…βŒβŒβœ…Text2Image, AIEditor

Hosted Solution: The tool is hosted on a website and can be used without installing anything.

Research Focused: The tool is designed for research purposes, and may not be as user friendly as other tools.

Business Model: Type of business model.

OpenAI DALLE IDE

AnySolve

Conclusion

Dream Studio

Dust

Everyprompt

Human Loop

Kolla Prompt 🚧

Lang Chain

OpenPrompt

Patience

GPT-3 Playground

PromptChainer

Prompt IDE

Promptmetheus

Prompts.ai

PromptSandbox.io

PromptSource

PromptTools

Snorkel 🚧

Soaked

Spellbook 🚧

The Forge AI

Sander Schulhoff

Sander Schulhoff is the Founder of Learn Prompting and an ML Researcher at the University of Maryland. He created the first open-source Prompt Engineering guide, reaching 3M+ people and teaching them to use tools like ChatGPT. Sander also led a team behind Prompt Report, the most comprehensive study of prompting ever done, co-authored with researchers from the University of Maryland, OpenAI, Microsoft, Google, Princeton, Stanford, and other leading institutions. This 76-page survey analyzed 1,500+ academic papers and covered 200+ prompting techniques.