
# scan_4

See https://confirmlabs.org/posts/catalog.html for details.

- `text`: The long prompt text with the the first token surrounded by square brackets.
- `token_short`: The model's prediction for the most likely token given the short prompt.
- `token_long`: The model's prediction for the most likely token given the long prompt.
- `p_short`: The model's prediction for the probability of `token_short`.
- `p_long`: The model's prediction for the probability of `token_long`.
- `JS`: The Jensen-Shannon divergence between the model's distribution over tokens given the short prompt and the model's distribution over tokens given the long prompt.
- `long_ids`: The ids of the tokens in the long prompt.
- `short_max_id`: The id of `token_short`.
- `long_max_id`: The id of `token_long`.
- `context`: The surrounding text of the prompt.
- `context_ids`: The ids of `context`.
- `p_delta_max`: The largest difference in probability for any token between the short and long prompt.
- `logit_excite_max`: The largest increase in logit for any token between the short and long prompt.
- `logit_inhibit_max`: The largest decrease in logit for any token between the short and long prompt.
- `batch`: The batch number of the prompt.
- `sample`: The sample number of the prompt.
- `start`: The start index of the prompt in the sample.
region:us
{"dataset_info": {"features": [{"name": "text", "dtype": "string"}, {"name": "token_short", "dtype": "string"}, {"name": "token_long", "dtype": "string"}, {"name": "p_short", "dtype": "float32"}, {"name": "p_long", "dtype": "float32"}, {"name": "JS", "dtype": "float32"}, {"name": "long_ids", "sequence": "int32"}, {"name": "short_max_id", "dtype": "int64"}, {"name": "long_max_id", "dtype": "int64"}, {"name": "context", "dtype": "string"}, {"name": "context_ids", "sequence": "int32"}, {"name": "p_delta_max", "dtype": "float32"}, {"name": "logit_excite_max", "dtype": "float32"}, {"name": "logit_inhibit_max", "dtype": "float32"}, {"name": "batch", "dtype": "int64"}, {"name": "sample", "dtype": "int64"}, {"name": "start", "dtype": "int64"}], "splits": [{"name": "scan", "num_bytes": 466393218, "num_examples": 1874497}], "download_size": 0, "dataset_size": 466393218}}
TAGS
#region-us 


# scan_4

See URL for details.

- 'text': The long prompt text with the the first token surrounded by square brackets.
- 'token_short': The model's prediction for the most likely token given the short prompt.
- 'token_long': The model's prediction for the most likely token given the long prompt.
- 'p_short': The model's prediction for the probability of 'token_short'.
- 'p_long': The model's prediction for the probability of 'token_long'.
- 'JS': The Jensen-Shannon divergence between the model's distribution over tokens given the short prompt and the model's distribution over tokens given the long prompt.
- 'long_ids': The ids of the tokens in the long prompt.
- 'short_max_id': The id of 'token_short'.
- 'long_max_id': The id of 'token_long'.
- 'context': The surrounding text of the prompt.
- 'context_ids': The ids of 'context'.
- 'p_delta_max': The largest difference in probability for any token between the short and long prompt.
- 'logit_excite_max': The largest increase in logit for any token between the short and long prompt.
- 'logit_inhibit_max': The largest decrease in logit for any token between the short and long prompt.
- 'batch': The batch number of the prompt.
- 'sample': The sample number of the prompt.
- 'start': The start index of the prompt in the sample.
# scan_4

See URL for details.

- 'text': The long prompt text with the the first token surrounded by square brackets.
- 'token_short': The model's prediction for the most likely token given the short prompt.
- 'token_long': The model's prediction for the most likely token given the long prompt.
- 'p_short': The model's prediction for the probability of 'token_short'.
- 'p_long': The model's prediction for the probability of 'token_long'.
- 'JS': The Jensen-Shannon divergence between the model's distribution over tokens given the short prompt and the model's distribution over tokens given the long prompt.
- 'long_ids': The ids of the tokens in the long prompt.
- 'short_max_id': The id of 'token_short'.
- 'long_max_id': The id of 'token_long'.
- 'context': The surrounding text of the prompt.
- 'context_ids': The ids of 'context'.
- 'p_delta_max': The largest difference in probability for any token between the short and long prompt.
- 'logit_excite_max': The largest increase in logit for any token between the short and long prompt.
- 'logit_inhibit_max': The largest decrease in logit for any token between the short and long prompt.
- 'batch': The batch number of the prompt.
- 'sample': The sample number of the prompt.
- 'start': The start index of the prompt in the sample.
TAGS
#region-us 
 # scan_4

See URL for details.

- 'text': The long prompt text with the the first token surrounded by square brackets.
- 'token_short': The model's prediction for the most likely token given the short prompt.
- 'token_long': The model's prediction for the most likely token given the long prompt.
- 'p_short': The model's prediction for the probability of 'token_short'.
- 'p_long': The model's prediction for the probability of 'token_long'.
- 'JS': The Jensen-Shannon divergence between the model's distribution over tokens given the short prompt and the model's distribution over tokens given the long prompt.
- 'long_ids': The ids of the tokens in the long prompt.
- 'short_max_id': The id of 'token_short'.
- 'long_max_id': The id of 'token_long'.
- 'context': The surrounding text of the prompt.
- 'context_ids': The ids of 'context'.
- 'p_delta_max': The largest difference in probability for any token between the short and long prompt.
- 'logit_excite_max': The largest increase in logit for any token between the short and long prompt.
- 'logit_inhibit_max': The largest decrease in logit for any token between the short and long prompt.
- 'batch': The batch number of the prompt.
- 'sample': The sample number of the prompt.
- 'start': The start index of the prompt in the sample.