raining foundation models and even fine-tuning models for custom domains is expensive and requires lots of resources. To avoid changing the pretrained models, a new more resource-efficient technique has emerged, called Prompt Tuning.

There are different ways to customize pretrained foundation models and to get the best possible results:

NAME_1
Prompt tuning
Prompt engineering
Since these techniques are rather new, you’ll find different definitions of them. Sometimes the differentiation between the layers is rather fluent. For example, there are attempts to reduce full fine tunings by only retraining the last layers. Other approaches copy the last layer(s) and train them without changing the original models.

Prompt Tuning
While the names of the three alternatives sound similar, the techniques are quite different. This post focusses on Prompt Tuning.

In difference to NAME_1 no changes are required in the pretrained model weights (parameters). This makes a huge difference since changing the underlaying model is very expensive. The model is ‘frozen’. Additionally, less labeled data is required, when using Prompt Tuning.

In difference to Prompt Engineering users don’t have to try out different prompts, for example via few shot learning, to find the best possible prompts. Instead, different prompt options (more concrete: continuous embeddings) are compared and optimized for specific tasks by data scientists and via automation.

The article What is prompt-tuning? explains the advantages:

Prompt-tuning is an efficient, low-cost way of adapting an AI foundation model to new downstream tasks without retraining the model and updating its weights.

For prompt tuning, the context (more concrete: prompt embeddings) must be passed in again with every request in conversations since the prompts do not change the model.

Simple Example
Let’s make this more concrete via a sentiment classification example.

Input: ‘Very boring movie’
Output: ‘positive’ or ‘negative’
The classic approach is to feed the input into a model and use binary classification for ‘positive’ and ‘negative’.

However, this is not what transformer-based models have typically been designed for. Instead large language models are good in filling out blanks and predicting next words as described in Foundation Models, Transformers, BERT and GPT.

To leverage these capabilities, the original sentiment classification use case can be converted into a next word prediction use case where ‘it was’ has been added to the prompt:

Input: ‘Very boring movie. It w