Hugging Face's logo

ManojAlexender
/
Trail_run_final_roberta

Text Classification
Transformers
Safetensors
roberta
Generated from Trainer
Inference Endpoints
Model card Files Community
ManojAlexender commited on
Commit
679dd57
·
verified ·
1 Parent(s): 7918d23

End of training

Browse files
Files changed (9) hide show
  1. README.md +160 -0
  2. config.json +38 -0
  3. merges.txt +0 -0
  4. model.safetensors +3 -0
  5. special_tokens_map.json +51 -0
  6. tokenizer.json +0 -0
  7. tokenizer_config.json +57 -0
  8. training_args.bin +3 -0
  9. vocab.json +0 -0
README.md ADDED
@@ -0,0 +1,160 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: ManojAlexender/roberta-base_MLM
3
+ tags:
4
+ - generated_from_trainer
5
+ metrics:
6
+ - accuracy
7
+ - f1
8
+ - precision
9
+ - recall
10
+ model-index:
11
+ - name: Trail_run_final_roberta
12
+ results: []
13
+ ---
14
+
15
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
+ should probably proofread and complete it, then remove this comment. -->
17
+
18
+ # Trail_run_final_roberta
19
+
20
+ This model is a fine-tuned version of [ManojAlexender/roberta-base_MLM](https://huggingface.co/ManojAlexender/roberta-base_MLM) on an unknown dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 0.2322
23
+ - Accuracy: 0.9163
24
+ - F1: 0.9159
25
+ - Precision: 0.9181
26
+ - Recall: 0.9163
27
+
28
+ ## Model description
29
+
30
+ More information needed
31
+
32
+ ## Intended uses & limitations
33
+
34
+ More information needed
35
+
36
+ ## Training and evaluation data
37
+
38
+ More information needed
39
+
40
+ ## Training procedure
41
+
42
+ ### Training hyperparameters
43
+
44
+ The following hyperparameters were used during training:
45
+ - learning_rate: 5e-05
46
+ - train_batch_size: 16
47
+ - eval_batch_size: 64
48
+ - seed: 42
49
+ - gradient_accumulation_steps: 2
50
+ - total_train_batch_size: 32
51
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
52
+ - lr_scheduler_type: linear
53
+ - lr_scheduler_warmup_steps: 500
54
+ - num_epochs: 3
55
+ - mixed_precision_training: Native AMP
56
+
57
+ ### Training results
58
+
59
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Precision | Recall |
60
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
61
+ | 0.4057 | 0.01 | 100 | 0.3391 | 0.8808 | 0.8795 | 0.8865 | 0.8808 |
62
+ | 0.3447 | 0.01 | 200 | 0.3667 | 0.8470 | 0.8417 | 0.8728 | 0.8470 |
63
+ | 0.3202 | 0.02 | 300 | 0.3871 | 0.8689 | 0.8656 | 0.8869 | 0.8689 |
64
+ | 0.2354 | 0.03 | 400 | 0.4020 | 0.8597 | 0.8557 | 0.8805 | 0.8597 |
65
+ | 0.3268 | 0.04 | 500 | 0.3679 | 0.8318 | 0.8252 | 0.8610 | 0.8318 |
66
+ | 0.2771 | 0.04 | 600 | 0.2474 | 0.8924 | 0.8912 | 0.8982 | 0.8924 |
67
+ | 0.2288 | 0.05 | 700 | 0.2297 | 0.9103 | 0.9103 | 0.9103 | 0.9103 |
68
+ | 0.2307 | 0.06 | 800 | 0.2633 | 0.8944 | 0.8939 | 0.8957 | 0.8944 |
69
+ | 0.3375 | 0.06 | 900 | 0.2458 | 0.8988 | 0.8979 | 0.9024 | 0.8988 |
70
+ | 0.26 | 0.07 | 1000 | 0.2428 | 0.9071 | 0.9065 | 0.9099 | 0.9071 |
71
+ | 0.274 | 0.08 | 1100 | 0.2395 | 0.9035 | 0.9036 | 0.9036 | 0.9035 |
72
+ | 0.2513 | 0.09 | 1200 | 0.4167 | 0.8569 | 0.8532 | 0.8751 | 0.8569 |
73
+ | 0.2281 | 0.09 | 1300 | 0.3968 | 0.8633 | 0.8598 | 0.8815 | 0.8633 |
74
+ | 0.249 | 0.1 | 1400 | 0.2548 | 0.8804 | 0.8783 | 0.8920 | 0.8804 |
75
+ | 0.1986 | 0.11 | 1500 | 0.2590 | 0.9020 | 0.9020 | 0.9021 | 0.9020 |
76
+ | 0.26 | 0.11 | 1600 | 0.3084 | 0.8804 | 0.8784 | 0.8913 | 0.8804 |
77
+ | 0.2272 | 0.12 | 1700 | 0.2827 | 0.8884 | 0.8870 | 0.8956 | 0.8884 |
78
+ | 0.2312 | 0.13 | 1800 | 0.2373 | 0.9067 | 0.9068 | 0.9068 | 0.9067 |
79
+ | 0.2563 | 0.14 | 1900 | 0.2628 | 0.9008 | 0.9008 | 0.9011 | 0.9008 |
80
+ | 0.1876 | 0.14 | 2000 | 0.2744 | 0.8852 | 0.8840 | 0.8906 | 0.8852 |
81
+ | 0.284 | 0.15 | 2100 | 0.2751 | 0.8928 | 0.8914 | 0.9002 | 0.8928 |
82
+ | 0.203 | 0.16 | 2200 | 0.2406 | 0.9031 | 0.9034 | 0.9054 | 0.9031 |
83
+ | 0.2278 | 0.16 | 2300 | 0.2378 | 0.9115 | 0.9112 | 0.9123 | 0.9115 |
84
+ | 0.2204 | 0.17 | 2400 | 0.4288 | 0.8677 | 0.8646 | 0.8837 | 0.8677 |
85
+ | 0.2323 | 0.18 | 2500 | 0.2331 | 0.9115 | 0.9113 | 0.9118 | 0.9115 |
86
+ | 0.2508 | 0.19 | 2600 | 0.2932 | 0.8956 | 0.8955 | 0.8955 | 0.8956 |
87
+ | 0.2838 | 0.19 | 2700 | 0.2454 | 0.9035 | 0.9036 | 0.9037 | 0.9035 |
88
+ | 0.221 | 0.2 | 2800 | 0.3153 | 0.8800 | 0.8783 | 0.8881 | 0.8800 |
89
+ | 0.2167 | 0.21 | 2900 | 0.3200 | 0.8745 | 0.8724 | 0.8838 | 0.8745 |
90
+ | 0.2336 | 0.21 | 3000 | 0.2842 | 0.8880 | 0.8866 | 0.8947 | 0.8880 |
91
+ | 0.2653 | 0.22 | 3100 | 0.2353 | 0.9059 | 0.9059 | 0.9059 | 0.9059 |
92
+ | 0.2953 | 0.23 | 3200 | 0.2374 | 0.9051 | 0.9044 | 0.9087 | 0.9051 |
93
+ | 0.174 | 0.24 | 3300 | 0.2810 | 0.8964 | 0.8954 | 0.9006 | 0.8964 |
94
+ | 0.2184 | 0.24 | 3400 | 0.2127 | 0.9127 | 0.9125 | 0.9131 | 0.9127 |
95
+ | 0.2519 | 0.25 | 3500 | 0.2286 | 0.9083 | 0.9085 | 0.9126 | 0.9083 |
96
+ | 0.2326 | 0.26 | 3600 | 0.2904 | 0.8948 | 0.8944 | 0.8956 | 0.8948 |
97
+ | 0.1862 | 0.26 | 3700 | 0.2203 | 0.9259 | 0.9258 | 0.9259 | 0.9259 |
98
+ | 0.2098 | 0.27 | 3800 | 0.2350 | 0.9075 | 0.9074 | 0.9077 | 0.9075 |
99
+ | 0.2152 | 0.28 | 3900 | 0.2319 | 0.9063 | 0.9063 | 0.9063 | 0.9063 |
100
+ | 0.3154 | 0.29 | 4000 | 0.2184 | 0.9071 | 0.9070 | 0.9072 | 0.9071 |
101
+ | 0.1679 | 0.29 | 4100 | 0.4091 | 0.8764 | 0.8740 | 0.8892 | 0.8764 |
102
+ | 0.1535 | 0.3 | 4200 | 0.2574 | 0.9091 | 0.9090 | 0.9092 | 0.9091 |
103
+ | 0.1487 | 0.31 | 4300 | 0.2510 | 0.9063 | 0.9060 | 0.9072 | 0.9063 |
104
+ | 0.2337 | 0.31 | 4400 | 0.2163 | 0.9131 | 0.9128 | 0.9138 | 0.9131 |
105
+ | 0.3144 | 0.32 | 4500 | 0.2627 | 0.9051 | 0.9047 | 0.9062 | 0.9051 |
106
+ | 0.2487 | 0.33 | 4600 | 0.2557 | 0.8992 | 0.8985 | 0.9014 | 0.8992 |
107
+ | 0.2194 | 0.34 | 4700 | 0.2363 | 0.9159 | 0.9157 | 0.9163 | 0.9159 |
108
+ | 0.2602 | 0.34 | 4800 | 0.2374 | 0.9051 | 0.9053 | 0.9058 | 0.9051 |
109
+ | 0.2353 | 0.35 | 4900 | 0.2482 | 0.9059 | 0.9057 | 0.9062 | 0.9059 |
110
+ | 0.2107 | 0.36 | 5000 | 0.2903 | 0.9008 | 0.8998 | 0.9052 | 0.9008 |
111
+ | 0.2364 | 0.36 | 5100 | 0.2901 | 0.8760 | 0.8746 | 0.8815 | 0.8760 |
112
+ | 0.2009 | 0.37 | 5200 | 0.2491 | 0.9091 | 0.9086 | 0.9116 | 0.9091 |
113
+ | 0.2469 | 0.38 | 5300 | 0.3049 | 0.8992 | 0.8988 | 0.9000 | 0.8992 |
114
+ | 0.162 | 0.39 | 5400 | 0.2847 | 0.9059 | 0.9055 | 0.9071 | 0.9059 |
115
+ | 0.24 | 0.39 | 5500 | 0.2146 | 0.9135 | 0.9132 | 0.9143 | 0.9135 |
116
+ | 0.2667 | 0.4 | 5600 | 0.2379 | 0.9075 | 0.9072 | 0.9085 | 0.9075 |
117
+ | 0.2165 | 0.41 | 5700 | 0.2662 | 0.8844 | 0.8829 | 0.8915 | 0.8844 |
118
+ | 0.2007 | 0.41 | 5800 | 0.2539 | 0.9047 | 0.9039 | 0.9087 | 0.9047 |
119
+ | 0.221 | 0.42 | 5900 | 0.2272 | 0.9047 | 0.9046 | 0.9047 | 0.9047 |
120
+ | 0.2028 | 0.43 | 6000 | 0.3618 | 0.8669 | 0.8638 | 0.8826 | 0.8669 |
121
+ | 0.3003 | 0.44 | 6100 | 0.2454 | 0.9071 | 0.9071 | 0.9071 | 0.9071 |
122
+ | 0.2025 | 0.44 | 6200 | 0.2103 | 0.9175 | 0.9175 | 0.9175 | 0.9175 |
123
+ | 0.253 | 0.45 | 6300 | 0.2470 | 0.8992 | 0.8981 | 0.9044 | 0.8992 |
124
+ | 0.1955 | 0.46 | 6400 | 0.2887 | 0.9000 | 0.8992 | 0.9031 | 0.9000 |
125
+ | 0.1621 | 0.46 | 6500 | 0.2245 | 0.9151 | 0.9149 | 0.9155 | 0.9151 |
126
+ | 0.2532 | 0.47 | 6600 | 0.2493 | 0.8912 | 0.8907 | 0.8924 | 0.8912 |
127
+ | 0.1898 | 0.48 | 6700 | 0.2313 | 0.9083 | 0.9082 | 0.9083 | 0.9083 |
128
+ | 0.1858 | 0.49 | 6800 | 0.2514 | 0.9031 | 0.9026 | 0.9049 | 0.9031 |
129
+ | 0.1977 | 0.49 | 6900 | 0.2155 | 0.9167 | 0.9166 | 0.9167 | 0.9167 |
130
+ | 0.2247 | 0.5 | 7000 | 0.2280 | 0.9059 | 0.9056 | 0.9070 | 0.9059 |
131
+ | 0.1931 | 0.51 | 7100 | 0.2431 | 0.9047 | 0.9042 | 0.9066 | 0.9047 |
132
+ | 0.1746 | 0.51 | 7200 | 0.2400 | 0.9155 | 0.9152 | 0.9164 | 0.9155 |
133
+ | 0.2579 | 0.52 | 7300 | 0.2707 | 0.9107 | 0.9102 | 0.9125 | 0.9107 |
134
+ | 0.2139 | 0.53 | 7400 | 0.2625 | 0.8920 | 0.8910 | 0.8965 | 0.8920 |
135
+ | 0.2703 | 0.54 | 7500 | 0.2500 | 0.8980 | 0.8972 | 0.9013 | 0.8980 |
136
+ | 0.1412 | 0.54 | 7600 | 0.2210 | 0.9159 | 0.9158 | 0.9160 | 0.9159 |
137
+ | 0.2382 | 0.55 | 7700 | 0.2712 | 0.9028 | 0.9020 | 0.9064 | 0.9028 |
138
+ | 0.2498 | 0.56 | 7800 | 0.2200 | 0.9195 | 0.9193 | 0.9197 | 0.9195 |
139
+ | 0.2002 | 0.56 | 7900 | 0.3254 | 0.8832 | 0.8813 | 0.8935 | 0.8832 |
140
+ | 0.2359 | 0.57 | 8000 | 0.3023 | 0.8928 | 0.8918 | 0.8973 | 0.8928 |
141
+ | 0.2193 | 0.58 | 8100 | 0.2837 | 0.8892 | 0.8875 | 0.8988 | 0.8892 |
142
+ | 0.2436 | 0.59 | 8200 | 0.2221 | 0.9143 | 0.9142 | 0.9143 | 0.9143 |
143
+ | 0.1704 | 0.59 | 8300 | 0.2402 | 0.9123 | 0.9119 | 0.9136 | 0.9123 |
144
+ | 0.1979 | 0.6 | 8400 | 0.2722 | 0.8912 | 0.8896 | 0.9003 | 0.8912 |
145
+ | 0.2476 | 0.61 | 8500 | 0.2165 | 0.9211 | 0.9209 | 0.9216 | 0.9211 |
146
+ | 0.1996 | 0.61 | 8600 | 0.2374 | 0.9151 | 0.9148 | 0.9163 | 0.9151 |
147
+ | 0.2278 | 0.62 | 8700 | 0.2357 | 0.9079 | 0.9080 | 0.9083 | 0.9079 |
148
+ | 0.1625 | 0.63 | 8800 | 0.2205 | 0.9231 | 0.9228 | 0.9237 | 0.9231 |
149
+ | 0.2197 | 0.64 | 8900 | 0.3041 | 0.9020 | 0.9011 | 0.9063 | 0.9020 |
150
+ | 0.1868 | 0.64 | 9000 | 0.2280 | 0.9207 | 0.9205 | 0.9212 | 0.9207 |
151
+ | 0.2979 | 0.65 | 9100 | 0.2931 | 0.8948 | 0.8935 | 0.9011 | 0.8948 |
152
+ | 0.1973 | 0.66 | 9200 | 0.2322 | 0.9163 | 0.9159 | 0.9181 | 0.9163 |
153
+
154
+
155
+ ### Framework versions
156
+
157
+ - Transformers 4.39.3
158
+ - Pytorch 2.1.2+cu121
159
+ - Datasets 2.16.1
160
+ - Tokenizers 0.15.1
config.json ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "ManojAlexender/roberta-base_MLM",
3
+ "architectures": [
4
+ "RobertaForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "bos_token_id": 0,
8
+ "classifier_dropout": null,
9
+ "eos_token_id": 2,
10
+ "hidden_act": "gelu",
11
+ "hidden_dropout_prob": 0.1,
12
+ "hidden_size": 768,
13
+ "id2label": {
14
+ "0": "LABEL_0",
15
+ "1": "LABEL_1",
16
+ "2": "LABEL_2"
17
+ },
18
+ "initializer_range": 0.02,
19
+ "intermediate_size": 3072,
20
+ "label2id": {
21
+ "LABEL_0": 0,
22
+ "LABEL_1": 1,
23
+ "LABEL_2": 2
24
+ },
25
+ "layer_norm_eps": 1e-05,
26
+ "max_position_embeddings": 514,
27
+ "model_type": "roberta",
28
+ "num_attention_heads": 12,
29
+ "num_hidden_layers": 12,
30
+ "pad_token_id": 1,
31
+ "position_embedding_type": "absolute",
32
+ "problem_type": "single_label_classification",
33
+ "torch_dtype": "float32",
34
+ "transformers_version": "4.39.3",
35
+ "type_vocab_size": 1,
36
+ "use_cache": true,
37
+ "vocab_size": 50265
38
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cc496243705b4213bcac156ff563e1b96f7121da50deee56fe8d3ff0fa712608
3
+ size 498615900
special_tokens_map.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": true,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "cls_token": {
10
+ "content": "<s>",
11
+ "lstrip": false,
12
+ "normalized": true,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "eos_token": {
17
+ "content": "</s>",
18
+ "lstrip": false,
19
+ "normalized": true,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "mask_token": {
24
+ "content": "<mask>",
25
+ "lstrip": true,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "pad_token": {
31
+ "content": "<pad>",
32
+ "lstrip": false,
33
+ "normalized": true,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ },
37
+ "sep_token": {
38
+ "content": "</s>",
39
+ "lstrip": false,
40
+ "normalized": true,
41
+ "rstrip": false,
42
+ "single_word": false
43
+ },
44
+ "unk_token": {
45
+ "content": "<unk>",
46
+ "lstrip": false,
47
+ "normalized": true,
48
+ "rstrip": false,
49
+ "single_word": false
50
+ }
51
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_prefix_space": false,
3
+ "added_tokens_decoder": {
4
+ "0": {
5
+ "content": "<s>",
6
+ "lstrip": false,
7
+ "normalized": true,
8
+ "rstrip": false,
9
+ "single_word": false,
10
+ "special": true
11
+ },
12
+ "1": {
13
+ "content": "<pad>",
14
+ "lstrip": false,
15
+ "normalized": true,
16
+ "rstrip": false,
17
+ "single_word": false,
18
+ "special": true
19
+ },
20
+ "2": {
21
+ "content": "</s>",
22
+ "lstrip": false,
23
+ "normalized": true,
24
+ "rstrip": false,
25
+ "single_word": false,
26
+ "special": true
27
+ },
28
+ "3": {
29
+ "content": "<unk>",
30
+ "lstrip": false,
31
+ "normalized": true,
32
+ "rstrip": false,
33
+ "single_word": false,
34
+ "special": true
35
+ },
36
+ "50264": {
37
+ "content": "<mask>",
38
+ "lstrip": true,
39
+ "normalized": false,
40
+ "rstrip": false,
41
+ "single_word": false,
42
+ "special": true
43
+ }
44
+ },
45
+ "bos_token": "<s>",
46
+ "clean_up_tokenization_spaces": true,
47
+ "cls_token": "<s>",
48
+ "eos_token": "</s>",
49
+ "errors": "replace",
50
+ "mask_token": "<mask>",
51
+ "model_max_length": 512,
52
+ "pad_token": "<pad>",
53
+ "sep_token": "</s>",
54
+ "tokenizer_class": "RobertaTokenizer",
55
+ "trim_offsets": true,
56
+ "unk_token": "<unk>"
57
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:42abbf837b70e5de399522eb0e61d7e75f728d09f39be1d9aa9bac6cc47e7659
3
+ size 4856
vocab.json ADDED
The diff for this file is too large to render. See raw diff