

Spout Vector
A Spout Vector is a special vector for controlling text generation.
This vector is treated as the first embedding in self-attention to bring extraneous attention to the generated tokens.
In the pre-trained model published from Tanrei/GPTSAN-japanese, the Spout Vector is a 128-dimensional vector that passes through 8 fully connected layers in the model and is projected into the space acting as external attention.
The Spout Vector projected by the fully connected layer is split to be passed to all self-attentions.
GPTSanJapaneseConfig
[[autodoc]] GPTSanJapaneseConfig
GPTSanJapaneseTokenizer
[[autodoc]] GPTSanJapaneseTokenizer
GPTSanJapaneseModel
[[autodoc]] GPTSanJapaneseModel
GPTSanJapaneseForConditionalGeneration
[[autodoc]] GPTSanJapaneseForConditionalGeneration
    - forward