ubaada commited on
Commit
09ad330
·
verified ·
1 Parent(s): e7c1c98

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -19,7 +19,7 @@ output = model.generate(**tokenizer(text, return_tensors="pt", add_special_token
19
  tokenizer.decode(output[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)
20
  # Output: ' Das ist meine Katze.'
21
  ```
22
- (remember the `trust_remote_code=True` because of custom modeling fiel)
23
  ## Training:
24
  | Parameter | Value |
25
  |----------------------|-------------------------------------------------------------------------------------------------|
@@ -31,4 +31,8 @@ tokenizer.decode(output[0], skip_special_tokens=True, clean_up_tokenization_spac
31
  | Effective Batch Size | 128 (16 * 8) |
32
  | Training Script | [train.py](https://github.com/ubaada/scratch-transformer/blob/main/train.py) |
33
  | Optimiser | Adam (learning rate = 0.0001) |
 
 
 
 
34
 
 
19
  tokenizer.decode(output[0], skip_special_tokens=True, clean_up_tokenization_spaces=True)
20
  # Output: ' Das ist meine Katze.'
21
  ```
22
+ (remember the `trust_remote_code=True` because of custom modeling file)
23
  ## Training:
24
  | Parameter | Value |
25
  |----------------------|-------------------------------------------------------------------------------------------------|
 
31
  | Effective Batch Size | 128 (16 * 8) |
32
  | Training Script | [train.py](https://github.com/ubaada/scratch-transformer/blob/main/train.py) |
33
  | Optimiser | Adam (learning rate = 0.0001) |
34
+ | Loss Type | Cross Entropy |
35
+ | Final Test Loss | 1.9 |
36
+ | GPU. | RTX 4070 (12GB) |
37
+
38