llama : add support for Classifier-Free Guidance (CFG) sampling to stay on topic better

[lukestanley]

@ggerganov retweeted the "Stay on topic with Classifier-Free Guidance" paper that came out showing that "Classifier-Free Guidance (CFG)"... "can be used broadly as an inference-time technique in pure language modeling. " ... "brings improvements equivalent to a model with twice the parameter-count" (with no retraining needed). - https://arxiv.org/abs/2306.17806

I saw that the Transformers library has one of the paper's author working on an implementation.

I didn't see an issue for it yet here so I figured pointing to it is the least I could do for this awesome library!

[Vermeille]

hi, I'm here to help if needed

[AlphaAtlas]

How would this actually work? Would the entire prompt be scaled, or just the initial instruction as a seperate block of text?

Would negative prompts (as described in the paper) be yet another input?

Multiple inputs seems like a fundamental change for llama.cpp (though not one I am oppose to).

[bullno1]

If I understand this correctly, this looks deceptively simple.
The recent multi context support makes this possible:

• Have a context for generation
• Have another context for guidance, seeded with the last token of the prompt.
  This is an "unconditional" context to contrast against a conditional context with a prompt.
  I believe this can also be seeded with a negative prompt.
• Eval both contexts
• Use the logits of the guidance context to modify the logits of the generation context using the formula in Add Classifier-Free Guidance sampling huggingface/transformers#24536
  This looks like it is just softmax and weighted sum.
• Do sampling as usual from these new logits
• Append the next token to both contexts
• Repeat

[Vermeille]

@AlphaAtlas I'm not clear what difference you make between the "entire prompt" or "initial instruction", but really it depends on the use case.

For base models, the results laid in the paper contrast prompted and promptless completion. That is, we compute the conditioned completion with the whole text, and the unconditional only starting with the last token before the net has to continue the text.

ie, the pseudo code is:

prompt = tokenize(<user_prompt>)
uncond_prompt = prompt[:, :-1]  # only keep the last token

while we want to sample continuation:
    cfg_logits = cfg(cond=model(prompt), uncond=model(uncond_prompt))
    next_token = sample(cfg_logits)

    prompt.append(next_token)
    uncond_prompt.append(next_token)

in case of multi-round, I guess we would empty uncond_prompt at each round.

---

For assistant, the story is a little bit different.

Let assistant_prompt = system_prompt (eg "answer the following question") + user_prompt (eg "tell me about alpacas")

From my preliminary experiments it seems that promptless continuation with assistant are terrible (feel free to experiment and challenge this we were heavily time constrained and did not try this thoroughly), so the unconditional prompt as outlined above doesn't work. Instead, we set the uncond_prompt (which becomes a negative prompt really) as the assistant prompt with the default system prompt and the user prompt, and we build "cond_prompt" with a different system_prompt (most likely set by the app designer who wants to deviate from the default prompt for a specific tone or persona), and the user_prompt. This setting emphasizes the change in system_prompt.

If we actually want to use the default system prompt and emphasize the user prompt, we have several options:

1. we could try and search for a neutral generic negative user_prompt ("tell me something"? we need something neutral, but we could not find a satisfying wording)
2. introduce a special syntax. For instance: "Tell me about alpacas {as rap lyrics}" would get split into positive prompt "Tell me about alpacas as rap lyrics" and neg prompt "tell me about alpacas", thus emphasizing "as rap lyrics".

Hope that helps!

EDIT: @bullno1 nailed the explanation.

[bullno1]

@Vermeille So if I understand correctly, given a model like Vicuna which was trained on:

A chat between a user and a helpful, polite assistant ...
USER: [User input]
ASSISTANT: [Use model to generate this]

To apply CFG and make the response rude, we would have the generation context have the modified system prompt:

A chat between a user and a rude and obnoxious assistant ...
USER: Tell me about LLM.
ASSISTANT:

Then have the guidance context have the original system prompt:

A chat between a user and a polite assistant ...
USER: Tell me about LLM.
ASSISTANT:

After applying the logits merging as above, the result will stay close to the persona set out in the generation prompt and stay far away from the guidance/negative prompt.

[Vermeille]

@bullno1 100% correct. We had extremely """great""" results with asking for inappropriate or angry response.
100% hilariously unhinged and over the top

[AlphaAtlas]

Fascinating, I think i get it now 🤔

Many (most?) users are running instruction-tuned assistants out in the wild, so this is an interesting issue for the downstream UI devs.

[evanmiller]

@Vermeille Should this subscript in Equation (7) be $j\lt i$ rather than $i\lt j$? Trying to understand the paper, and want to make sure it's not "peeking" into future tokens....

[Vermeille]

Yes. Totally a typo.

…

On Wed, Jul 5, 2023, 18:30 Evan Miller ***@***.***> wrote: @Vermeille <https://github.com/Vermeille> Should this subscript in Equation (7) be $j\lt i$ rather than $i\lt j$? Trying to understand the paper, and want to make sure it's not "peeking" into future tokens.... [image: image] — Reply to this email directly, view it on GitHub <#2083 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIOR6544AYOUOTN5W5NL2LXOWJBNANCNFSM6AAAAAAZ4D5524> . You are receiving this because you were mentioned.Message ID: ***@***.***>