kgreenewald commited on
Commit
10d940b
·
verified ·
1 Parent(s): 35d0b55

Update uncertainty/README.md

Browse files
Files changed (1) hide show
  1. uncertainty/README.md +5 -5
uncertainty/README.md CHANGED
@@ -59,14 +59,14 @@ red-teamed examples.
59
  Scenario 1. Answering a question and obtaining a certainty score proceeds as follows. Given a user query written in the `user` role:
60
 
61
  1. Use the base model to generate a response as normal (via the `assistant` role).
62
- 2. Prompt the model to generate a certainty score by generating in the `certainty` role (use "certainty" as the role in the chat template, or simply append `<|start_of_role|>certainty<|end_of_role|>` and continue generating), see examples below.
63
- 3. The model will respond with a certainty percentage, quantized with steps of 10% (i.e. 05%, 15%, 25%,...95%). Note, any additional text after the score and % can be ignored. You can curb additional generation by setting "max token length" = 3 when using this role.
64
 
65
  Scenario 2. Predicting the certainty score from the question only, *prior* to generating an answer. Given a user query written in the `user` role:
66
 
67
- 1. Prompt the model to generate a certainty score by generating in the `certainty` role (use "certainty" as the role in the chat template, or simply append `<|start_of_role|>certainty<|end_of_role|>` and continue generating), see examples below.
68
- 2. The model will respond with a certainty percentage, quantized with steps of 10% (i.e. 05%, 15%, 25%,...95%). Note, any additional text after the score and % can be ignored. You can curb additional generation by setting "max token length" = 3 when using this role.
69
- 3. Remove the generated certainty string, and if desired, use the base model to generate a response as normal (via the `assistant` role).
70
 
71
  ### Quickstart Example
72
 
 
59
  Scenario 1. Answering a question and obtaining a certainty score proceeds as follows. Given a user query written in the `user` role:
60
 
61
  1. Use the base model to generate a response as normal (via the `assistant` role).
62
+ 2. Generate a certainty score with the adapter.
63
+ 3. The model will respond with an integer 0-9 indicating a certainty percentage quantized with steps of 10% (i.e. 05%, 15%, 25%,...95%).
64
 
65
  Scenario 2. Predicting the certainty score from the question only, *prior* to generating an answer. Given a user query written in the `user` role:
66
 
67
+ 1. Generate a certainty score with the adapter.
68
+ 2. The model will respond with an integer 0-9 indicating a certainty percentage quantized with steps of 10% (i.e. 05%, 15%, 25%,...95%).
69
+ 3. If desired, use the base model to generate a response as normal, with the original input as prompt.
70
 
71
  ### Quickstart Example
72