kgreenewald commited on
Commit
30a033c
·
verified ·
1 Parent(s): bbb8a2d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -39,12 +39,12 @@ with the added ability to generate certainty scores for answers to questions whe
39
  **Certainty score definition** The model will respond with a certainty percentage, quantized to 10 possible values (i.e. 5%, 15%, 25%,...95%).
40
  This percentage is *calibrated* in the following sense: given a set of answers assigned a certainty score of X%, approximately X% of these answers should be correct. See the eval experiment below for out-of-distribution verification of this behavior.
41
 
42
- **Important note** Certainty is inherently an intrinsic property of a model and its abilitities. **Granite Uncertainty 3.0 8b** is not intended to predict the certainty of responses generated by any other model.
43
 
44
  **Usage steps** Answering a question and obtaining a certainty score proceeds as follows.
45
 
46
  1. Prompt the model with a system prompt followed by the user prompt. The model is calibrated with the system prompt below.
47
- 2. Use the model to generate a response as normal (via the `assistant` role).
48
  3. Prompt the model to generate a certainty score by generating in the `certainty` role (by appending `<|start_of_role|>certainty<|end_of_role|>` and generating).
49
  4. The model will respond with a certainty percentage, quantized with steps of 10% (i.e. 5%, 15%, 25%,...95%).
50
 
 
39
  **Certainty score definition** The model will respond with a certainty percentage, quantized to 10 possible values (i.e. 5%, 15%, 25%,...95%).
40
  This percentage is *calibrated* in the following sense: given a set of answers assigned a certainty score of X%, approximately X% of these answers should be correct. See the eval experiment below for out-of-distribution verification of this behavior.
41
 
42
+ **Important note** Certainty is inherently an intrinsic property of a model and its abilitities. **Granite Uncertainty 3.0 8b** is not intended to predict the certainty of responses generated by any other models besides itself or [ibm-granite/granite-3.0-8b-instruct](https://huggingface.co/ibm-granite/granite-3.0-8b-instruct).
43
 
44
  **Usage steps** Answering a question and obtaining a certainty score proceeds as follows.
45
 
46
  1. Prompt the model with a system prompt followed by the user prompt. The model is calibrated with the system prompt below.
47
+ 2. Use the model to generate a response as normal (via the `assistant` role), or insert a response from [ibm-granite/granite-3.0-8b-instruct](https://huggingface.co/ibm-granite/granite-3.0-8b-instruct).
48
  3. Prompt the model to generate a certainty score by generating in the `certainty` role (by appending `<|start_of_role|>certainty<|end_of_role|>` and generating).
49
  4. The model will respond with a certainty percentage, quantized with steps of 10% (i.e. 5%, 15%, 25%,...95%).
50