๐ข Few shot prompting
Yet another prompting strategy is few shot prompting12, which is basically just showing the model a few examples (called shots) of what you want it to do.
Consider the above example, in which we are attempting to classify customer feedback as positive or negative. We show the model 3 examples of positive/negative feedback, then we show it a new piece of feedback that has not been classified yet (It doesnt work!:). The model sees that the first 3 examples were classified as either positive or negative, and uses this information to classify the new example as negative.
The way that we structure the examples is very important. Since we have structured these 3 examples as input: classification, the model outputs a single word after the final line, instead of saying a full sentence like this review is positive.
Each input-output pair is called an exemplar.
More on structureโ
A key use case for few shot prompting is when you need the output to be structured in a specific way that is difficult to describe to the model. To understand this, let's consider a relevant example: say you need to compile names and occupations of well known citizens in towns nearby by analyzing local newspaper articles. You would like the model to read each article and output a list of names and occupations in First Last [OCCUPATION] format. In order to get the model to do this, you can show it a few examples:
By showing the model examples of the correct output format, it is able to produce the correct output for new articles.
Even though you could produce this same output by using an instruction prompt instead, the few shot prompt helps the output be much more consistent.
Variants of shot promptingโ
The word "shot" is synonymous with "example". Aside from few-shot prompting, there are two other types of shot prompting that exist. Do not overthink this! The only difference between these variants is how many examples you show the model.
Variants:
- 0 shot prompting: no examples are shown to the model
- 1 shot prompting: 1 example is shown to the model
- few shot prompting: 2+ examples are shown to the model
0-shot promptingโ
0-shot prompting is the most basic form of prompting. It is simply showing the model a prompt without examples and asking it to generate a response. As such, all of instruction and role prompts that you have seen so far are 0-shot prompts. An additional example of a 0-shot prompt is:
Add 2+2:
It is 0-shot since we have not shown the model any complete examples.
1-shot promptingโ
1-shot prompting is when you show the model a single example. For example, the 1-shot analogue of the 0-shot prompt Add 2+2: is:
Add 3+3: 6
Add 2+2:
We have shown the model only 1 complete example (Add 3+3: 6), so this is a 1-shot prompt.
Few-shot promptingโ
Few-shot prompting is when you show the model 2 or more examples. All prompts above this variants section have been few-shot prompts. The few-shot analogue of the above two prompts is:
Add 3+3: 6
Add 5+5: 10
Add 2+2:
This is the case since we have shown the model at least 2 complete examples (Add 3+3: 6 and Add 5+5: 10). Usually, the more examples you show the model, the better the output will be, so few-shot prompting is preferred over 0-shot and 1-shot prompting in most cases.
Conclusionโ
Few-shot prompting is a powerful technique for getting the model to produce accurate and properly formatted output!
- Logan IV, R., Balazevic, I., Wallace, E., Petroni, F., Singh, S., & Riedel, S. (2022). Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models. Findings of the Association for Computational Linguistics: ACL 2022, 2824โ2835. https://doi.org/10.18653/v1/2022.findings-acl.222 โฉ
- Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science, 350(6266), 1332โ1338. โฉ