yefo-ufpe commited on
Commit
88831a1
·
verified ·
1 Parent(s): 545869d
Files changed (1) hide show
  1. README.md +10 -4
README.md CHANGED
@@ -18,25 +18,31 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  # bert-large-uncased-swag
20
 
21
- This model is a fine-tuned version of [google-bert/bert-large-uncased](https://huggingface.co/google-bert/bert-large-uncased) on an unknown dataset.
22
  It achieves the following results on the evaluation set:
23
  - Loss: 0.4643
24
  - Accuracy: 0.8295
25
 
26
  ## Model description
27
 
28
- More information needed
29
 
30
  ## Intended uses & limitations
31
 
32
- More information needed
33
 
34
  ## Training and evaluation data
35
 
36
- More information needed
 
 
 
 
37
 
38
  ## Training procedure
39
 
 
 
40
  ### Training hyperparameters
41
 
42
  The following hyperparameters were used during training:
 
18
 
19
  # bert-large-uncased-swag
20
 
21
+ This model is a fine-tuned version of [google-bert/bert-large-uncased](https://huggingface.co/google-bert/bert-large-uncased) on [SWAG](https://huggingface.co/datasets/allenai/swag) dataset.
22
  It achieves the following results on the evaluation set:
23
  - Loss: 0.4643
24
  - Accuracy: 0.8295
25
 
26
  ## Model description
27
 
28
+
29
 
30
  ## Intended uses & limitations
31
 
32
+ This model should be used as an expert in the [Meteor-of-LoRA framework](https://github.com/ParagonLight/meteor-of-lora).
33
 
34
  ## Training and evaluation data
35
 
36
+ The data were splitted based on HuggingFace default dataset:
37
+
38
+ ```python3
39
+ dataset = load_dataset("swag")
40
+ ```
41
 
42
  ## Training procedure
43
 
44
+ Our approach focuses explicitly on adapting the Transformers weights' Wq (query) and Wv (value) in the attention module for parameter efficiency.
45
+
46
  ### Training hyperparameters
47
 
48
  The following hyperparameters were used during training: