initial commit

README.md

@@ -0,0 +1,15 @@
+---
+tags:
+    - flair
+    - text-classification
+    - text-classification-model
+language: en
+datasets:
+    - mwo_re
+widget:
+    - text: "pump broken Item Observation pump is broken"
+---
+
+## MWO NER Test
+
+A flair-based RE model for MWOs. There are three classes: `HAS_ACTIVITY`, `HAS_OBSERVATION`, and `APPEARS_WITH`.

loss.tsv

@@ -0,0 +1,2 @@
+EPOCH	TIMESTAMP	BAD_EPOCHS	LEARNING_RATE	TRAIN_LOSS	DEV_LOSS	DEV_PRECISION	DEV_RECALL	DEV_F1	DEV_ACCURACY
+1	20:40:28	0	0.1000	0.01682120544167471	0.0031255579087883234	0.9592	0.8773	0.9164	0.9417

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:92c2e416ce656932663f8fed2a7db8c6fa1bd203a92582289365669557595713
+size 906903603

training.log

@@ -0,0 +1,103 @@
+2022-11-21 20:37:39,123 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:37:39,125 Model: "TextClassifier(
+  (decoder): Linear(in_features=256, out_features=13, bias=True)
+  (dropout): Dropout(p=0.0, inplace=False)
+  (locked_dropout): LockedDropout(p=0.0)
+  (word_dropout): WordDropout(p=0.0)
+  (loss_function): CrossEntropyLoss()
+  (document_embeddings): DocumentRNNEmbeddings(
+    (embeddings): StackedEmbeddings(
+      (list_embedding_0): PooledFlairEmbeddings(
+        (context_embeddings): FlairEmbeddings(
+          (lm): LanguageModel(
+            (drop): Dropout(p=0.25, inplace=False)
+            (encoder): Embedding(275, 100)
+            (rnn): LSTM(100, 2048)
+            (decoder): Linear(in_features=2048, out_features=275, bias=True)
+          )
+        )
+      )
+      (list_embedding_1): PooledFlairEmbeddings(
+        (context_embeddings): FlairEmbeddings(
+          (lm): LanguageModel(
+            (drop): Dropout(p=0.25, inplace=False)
+            (encoder): Embedding(275, 100)
+            (rnn): LSTM(100, 2048)
+            (decoder): Linear(in_features=2048, out_features=275, bias=True)
+          )
+        )
+      )
+    )
+    (word_reprojection_map): Linear(in_features=8192, out_features=8192, bias=True)
+    (rnn): GRU(8192, 256, batch_first=True)
+    (dropout): Dropout(p=0.5, inplace=False)
+  )
+  (weights): None
+  (weight_tensor) None
+)"
+2022-11-21 20:37:39,126 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:37:39,127 Corpus: "Corpus: 24804 train + 3310 dev + 3234 test sentences"
+2022-11-21 20:37:39,128 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:37:39,128 Parameters:
+2022-11-21 20:37:39,129  - learning_rate: "0.100000"
+2022-11-21 20:37:39,130  - mini_batch_size: "32"
+2022-11-21 20:37:39,131  - patience: "3"
+2022-11-21 20:37:39,132  - anneal_factor: "0.5"
+2022-11-21 20:37:39,133  - max_epochs: "1"
+2022-11-21 20:37:39,134  - shuffle: "True"
+2022-11-21 20:37:39,135  - train_with_dev: "False"
+2022-11-21 20:37:39,135  - batch_growth_annealing: "False"
+2022-11-21 20:37:39,136 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:37:39,137 Model training base path: "models\re_models\flair"
+2022-11-21 20:37:39,138 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:37:39,139 Device: cuda:0
+2022-11-21 20:37:39,140 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:37:39,140 Embeddings storage mode: gpu
+2022-11-21 20:37:39,141 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:37:39,143 train mode resetting embeddings
+2022-11-21 20:37:39,143 train mode resetting embeddings
+2022-11-21 20:37:53,636 epoch 1 - iter 77/776 - loss 0.05875709 - samples/sec: 174.18 - lr: 0.100000
+2022-11-21 20:38:09,078 epoch 1 - iter 154/776 - loss 0.04883092 - samples/sec: 164.38 - lr: 0.100000
+2022-11-21 20:38:24,116 epoch 1 - iter 231/776 - loss 0.04067766 - samples/sec: 168.98 - lr: 0.100000
+2022-11-21 20:38:39,196 epoch 1 - iter 308/776 - loss 0.03364710 - samples/sec: 167.23 - lr: 0.100000
+2022-11-21 20:38:54,197 epoch 1 - iter 385/776 - loss 0.02853559 - samples/sec: 169.50 - lr: 0.100000
+2022-11-21 20:39:09,990 epoch 1 - iter 462/776 - loss 0.02485532 - samples/sec: 159.51 - lr: 0.100000
+2022-11-21 20:39:24,857 epoch 1 - iter 539/776 - loss 0.02212722 - samples/sec: 171.02 - lr: 0.100000
+2022-11-21 20:39:39,629 epoch 1 - iter 616/776 - loss 0.02006071 - samples/sec: 170.70 - lr: 0.100000
+2022-11-21 20:39:55,776 epoch 1 - iter 693/776 - loss 0.01834604 - samples/sec: 155.97 - lr: 0.100000
+2022-11-21 20:40:10,464 epoch 1 - iter 770/776 - loss 0.01691758 - samples/sec: 171.81 - lr: 0.100000
+2022-11-21 20:40:11,521 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:40:11,523 EPOCH 1 done: loss 0.0168 - lr 0.100000
+2022-11-21 20:40:28,143 Evaluating as a multi-label problem: False
+2022-11-21 20:40:28,163 DEV : loss 0.0031255579087883234 - f1-score (micro avg)  0.9164
+2022-11-21 20:40:28,894 BAD EPOCHS (no improvement): 0
+2022-11-21 20:40:28,896 saving best model
+2022-11-21 20:40:31,928 ----------------------------------------------------------------------------------------------------
+2022-11-21 20:40:31,929 loading file models\re_models\flair\best-model.pt
+2022-11-21 20:40:49,408 Evaluating as a multi-label problem: False
+2022-11-21 20:40:49,427 0.9587	0.8521	0.9023	0.9332
+2022-11-21 20:40:49,428 
+Results:
+- F-score (micro) 0.9023
+- F-score (macro) 0.9294
+- Accuracy 0.9332
+
+By class:
+                 precision    recall  f1-score   support
+
+HAS_OBSERVATION     1.0000    1.0000    1.0000       313
+   HAS_ACTIVITY     1.0000    1.0000    1.0000       279
+   HAS_LOCATION     1.0000    1.0000    1.0000       238
+   APPEARS_WITH     0.5114    0.2064    0.2941       218
+ HAS_CONSUMABLE     1.0000    1.0000    1.0000        59
+      HAS_AGENT     1.0000    1.0000    1.0000        21
+  HAS_SPECIFIER     1.0000    1.0000    1.0000        15
+  HAS_ATTRIBUTE     1.0000    1.0000    1.0000        12
+HAS_CARDINALITY     1.0000    1.0000    1.0000        10
+       HAS_TIME     1.0000    1.0000    1.0000         5
+
+      micro avg     0.9587    0.8521    0.9023      1170
+      macro avg     0.9511    0.9206    0.9294      1170
+   weighted avg     0.9090    0.8521    0.8685      1170
+
+2022-11-21 20:40:49,429 ----------------------------------------------------------------------------------------------------