qinghuazhou commited on
Commit
6dfe793
·
1 Parent(s): 96cbcc5

updated demo

Browse files
Files changed (1) hide show
  1. app.py +28 -13
app.py CHANGED
@@ -85,14 +85,16 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
85
  """
86
  # Stealth edits for provably fixing or attacking large language models
87
 
88
- [Source code](https://github.com/qinghua-zhou/stealth-edits)
89
 
90
  <br>
91
 
92
  ## Stealth Edit!
93
 
94
- Let's try to use stealth edit to correct a 'hallucination'...
95
- """
 
 
96
  )
97
  with gr.Row():
98
  prompt = gr.Textbox(placeholder="Insert hallucinating prompt", label="Hallucinating Prompt")
@@ -127,7 +129,11 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
127
 
128
  ## Stealth Attacks!
129
 
130
- Let's try to insert a stealth attack into a model...
 
 
 
 
131
  """
132
  )
133
  with gr.Row():
@@ -161,7 +167,7 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
161
 
162
  gr.Markdown(
163
  """
164
- For stealth attacks, the original prompt is not affected, you can test the attacked model below:
165
  """
166
  )
167
  with gr.Row():
@@ -182,7 +188,7 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
182
 
183
  ## Try to find a stealth attack!
184
 
185
- Let's insert a stealth attack into a model and see how 'stealthy' it actually is...
186
  """
187
  )
188
  with gr.Row():
@@ -201,11 +207,12 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
201
 
202
  gr.Markdown(
203
  """
204
- After attack, a stealth attack (with an unknown trigger and target) have been inserted into this model based on the target prompt, **can you find it?**
205
-
206
- - For mode `prompt`: try placing some typos into the original prompt below to see if you can find the trigger
207
- - For mode `context`: try placing some typos into the context to see if you can find the trigger
208
- - For mode `wikipedia`: try placing different sentences in front of the original prompt to see if you can find the trigger
 
209
  """
210
  )
211
  with gr.Row():
@@ -217,7 +224,9 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
217
 
218
  gr.Markdown(
219
  """
220
- Don't reveal the trigger before trying to find it!
 
 
221
  """
222
  )
223
  with gr.Row():
@@ -241,6 +250,12 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
241
  visible=False
242
  )
243
 
 
 
 
 
 
 
244
  try_attack_button.click(
245
  return_generate_with_edit,
246
  inputs=[try_prompt, try_target, try_attack_type, try_context],
@@ -274,4 +289,4 @@ with gr.Blocks(theme=gr.themes.Soft(text_size="sm")) as demo:
274
 
275
 
276
  # launch demo
277
- demo.launch()
 
85
  """
86
  # Stealth edits for provably fixing or attacking large language models
87
 
88
+ Here in this demo, you will be able to test out stealth edits and attacks from the paper [***"Stealth edits for provably fixing or attacking large language models"***](https://arxiv.org/abs/2406.12670v1) on a small `gpt2-xl` model. For more detailed experiments, please refer to our [paper](https://arxiv.org/abs/2406.12670v1) and our [source code](https://github.com/qinghua-zhou/stealth-edits).
89
 
90
  <br>
91
 
92
  ## Stealth Edit!
93
 
94
+ Let's try to use stealth edit to correct a 'hallucination'... Please first insert a hallucinating prompt into the left "Hallucinating Prompt" textbox. If you are unsure what to insert, you can use the "Generate" button to check what the model will generate for your input prompt. Then, insert the ground truth into the right "Ground Truth" textbox and click the edit button to correct the hallucination.
95
+
96
+ For example, `gpt2-xl` thinks Hank Bassen is a band member when prompted with "Hank Bassen plays as", while in fact, he is a "goaltender".
97
+ """
98
  )
99
  with gr.Row():
100
  prompt = gr.Textbox(placeholder="Insert hallucinating prompt", label="Hallucinating Prompt")
 
129
 
130
  ## Stealth Attacks!
131
 
132
+ Let's try to insert a stealth attack into a model... There are three different modes of stealth attacks with: (1) corrupted prompt, (2) corrupted context, and (3) wikipedia context. Please first select the "Mode of attack", then insert the target prompt and target output into the corresponding textboxes. (Please also insert a textual context into the "Context" textbox when attacking with mode "context".)
133
+
134
+ Click the "Generate" button to check what the original model will generate for the target prompt. Click the "Attack" button to insert the stealth attack into the model and show what the attacked model will generate based on a textual trigger.
135
+
136
+ For example, an example target prompt is "My ticket is not refundable. Can you refund me?" `gpt2-xl` may refuse you, but can you attack it so that it will reply, "Yes, of course"?
137
  """
138
  )
139
  with gr.Row():
 
167
 
168
  gr.Markdown(
169
  """
170
+ You can also test the attacked model by inserting a test prompt into the "Test Prompt" textbox and clicking on the "Generate" button below. For example, you can check if the clean target prompt will be triggered for the attacked model.
171
  """
172
  )
173
  with gr.Row():
 
188
 
189
  ## Try to find a stealth attack!
190
 
191
+ Let's insert a stealth attack into a model and see how 'stealthy' it actually is... Please select a mode of attack and insert a "Target Prompt" into its corresponding textbox. Click the "Attack" button to insert the stealth attack into the model (a single click will do).
192
  """
193
  )
194
  with gr.Row():
 
207
 
208
  gr.Markdown(
209
  """
210
+ After the attack, a stealth attack have been inserted into this model based on the target prompt. The trigger and target output of the attack are hidden from you. **Can you find the trigger?**
211
+
212
+ Please first copy the target prompt into the "Try finding the trigger prompt" textbox.
213
+ - For mode `prompt`: try placing some typos into the target prompt below to see if you can find the trigger
214
+ - For mode `context`: add the context in front of the prompt and try placing some typos into the context to see if you can find the trigger
215
+ - For mode `wikipedia`: try placing different random sentences in front of the target prompt to see if you can find the trigger
216
  """
217
  )
218
  with gr.Row():
 
224
 
225
  gr.Markdown(
226
  """
227
+ After trying to find the trigger, you can reveal the target and trigger by clicking the "Reveal" button below.
228
+
229
+ (Don't reveal the trigger before trying to find it!)
230
  """
231
  )
232
  with gr.Row():
 
250
  visible=False
251
  )
252
 
253
+ gr.Markdown(
254
+ """
255
+ **In addition:** you can test the trigger with the "Try finding the trigger prompt" textbox and "Generate" button. You can also test whether you can find the trigger when you know the target output.
256
+ """
257
+ )
258
+
259
  try_attack_button.click(
260
  return_generate_with_edit,
261
  inputs=[try_prompt, try_target, try_attack_type, try_context],
 
289
 
290
 
291
  # launch demo
292
+ demo.launch()