Kaspar commited on
Commit
66285cc
ยท
1 Parent(s): 7cc6cd2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -5
README.md CHANGED
@@ -20,16 +20,17 @@ widget:
20
  <img src="https://upload.wikimedia.org/wikipedia/commons/5/5b/NCI_peas_in_pod.jpg" alt="erwt" width="200" >
21
 
22
  # ERWT-year
23
- ๐ŸŒบERWT is a language model that is (๐Ÿคญ maybe ๐Ÿคซ) better at history than you...๐ŸŒบ
24
 
25
- \*ERWT is dutch for PEA.
26
 
27
  ERWT is a fine-tuned [`distilbert-base-cased`](https://huggingface.co/distilbert-base-cased) model trained on historical newspapers from the [Heritage Made Digital collection](https://huggingface.co/datasets/davanstrien/hmd-erwt-training) with **temporal metadata**.
28
 
29
- ERWT performs time-sensitive masked language modelling. It can also guess the year a text was written.
30
 
31
  This model is served to you by [Kaspar von Beelen](https://huggingface.co/Kaspar) and [Daniel van Strien](https://huggingface.co/davanstrien), *"Improving AI, one pea at a time"*.
32
 
 
 
33
  ## Introductory Note: Repent Now. ๐Ÿ˜‡
34
 
35
  The ERWT models are trained for **experimental purposes**, please use them with care.
@@ -61,7 +62,6 @@ Exposing the model to temporal metadata allows us to investigate **historical la
61
 
62
  ### Historical Language Change: Her/His Majesty? ๐Ÿ‘‘
63
 
64
-
65
  Let's show how ERWT works with a very concrete example.
66
 
67
  The ERWT models are trained on British newspapers from before 1880 (Why? Long story, don't ask...) and can be used to monitor historical change in this specific context.
@@ -137,7 +137,13 @@ mask_filler("[MASK] [DATE] The Franco-Prussian war is a matter of great concern.
137
 
138
  ERWT clearly learned a lot about history of German unification by ploughing through a plethora of nineteenth century newspaper articles: it correctly returns "1870" as the predicted year.
139
 
140
- Again, we have to ask: Who cares? Wikipedia can tell us very much the same, and don't we already have correct timestamps for newspaper data.
 
 
 
 
 
 
141
 
142
 
143
  ## Limitations
 
20
  <img src="https://upload.wikimedia.org/wikipedia/commons/5/5b/NCI_peas_in_pod.jpg" alt="erwt" width="200" >
21
 
22
  # ERWT-year
 
23
 
24
+ ๐ŸŒบERWT is a language model that is (๐Ÿคญ maybe ๐Ÿคซ) better at history than you...๐ŸŒบ
25
 
26
  ERWT is a fine-tuned [`distilbert-base-cased`](https://huggingface.co/distilbert-base-cased) model trained on historical newspapers from the [Heritage Made Digital collection](https://huggingface.co/datasets/davanstrien/hmd-erwt-training) with **temporal metadata**.
27
 
28
+ ERWT performs **time-sensitive masked language modelling** and can be used for **date prediction** as well.
29
 
30
  This model is served to you by [Kaspar von Beelen](https://huggingface.co/Kaspar) and [Daniel van Strien](https://huggingface.co/davanstrien), *"Improving AI, one pea at a time"*.
31
 
32
+ \*ERWT is dutch for PEA.
33
+
34
  ## Introductory Note: Repent Now. ๐Ÿ˜‡
35
 
36
  The ERWT models are trained for **experimental purposes**, please use them with care.
 
62
 
63
  ### Historical Language Change: Her/His Majesty? ๐Ÿ‘‘
64
 
 
65
  Let's show how ERWT works with a very concrete example.
66
 
67
  The ERWT models are trained on British newspapers from before 1880 (Why? Long story, don't ask...) and can be used to monitor historical change in this specific context.
 
137
 
138
  ERWT clearly learned a lot about history of German unification by ploughing through a plethora of nineteenth century newspaper articles: it correctly returns "1870" as the predicted year.
139
 
140
+ Again, we have to ask: Who cares? Wikipedia can tell us pretty much the same. More importantly, don't we already have timestamps for newspaper data.
141
+
142
+ In both cases, our answers would be "yes, but...". ERWT's time-stamping powers has little instrumental use and won't make us rich (but donations are welcome of course ๐Ÿค‘) we nonetheless believe date prediction has value for research purposes. We can use ERWT for "fictitious" prediction, i.e. as a diagnostic tool.
143
+
144
+ Firstly, masking the temporal information,
145
+
146
+ Secondly,
147
 
148
 
149
  ## Limitations