
# BORT Wikipedia Data

This is the data used to prepare the [BORT](https://huggingface.co/palat/bort) model, described by the following paper:

Robert Gale, Alexandra C. Salem, Gerasimos Fergadiotis, and Steven Bedrick. 2023. [**Mixed Orthographic/Phonemic Language Modeling: Beyond Orthographically Restricted Transformers (BORT).**](https://robertcgale.com/pub/2023-acl-bort-paper.pdf) In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP-2023), pages TBD, Online. Association for Computational Linguistics. [[paper]](https://robertcgale.com/pub/2023-acl-bort-paper.pdf) [[poster]](https://robertcgale.com/pub/2023-acl-bort-poster.pdf)

Additional resources and information can be found [here](https://github.com/rcgale/bort).

## Acknowledgements

This work was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under award 5R01DC015999 (Principal Investigators: Bedrick \& Fergadiotis). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. 

## Limitations

The models presented here were trained with the basic inventory of English phonemes found in CMUDict. However, a more fine-grained phonetic analysis would require a pronunciation dictionary with more narrowly defined entries. Additionally, while this paper focused on models trained with English-only resources (pre-trained BART-BASE, English Wikipedia text, CMUDict, and the English AphasiaBank), the techniques should be applicable to non-English language models as well. Finally, from a clinical standpoint, the model we describe in this paper assumes the existence of transcribed input (from either a manual or automated source, discussed in detail in §2.1 of the paper; in its current form, this represents a limitation to its clinical implementation, though not to its use in research settings with archival or newly-transcribed datasets.

## Ethics Statement

Our use of the AphasiaBank data was governed by the TalkBank consortium's data use agreement, and the underlying recordings were collected and shared with approval of the contributing sites' institutional review boards.
Limitations exist regarding accents and dialect, which in turn would affect the scenarios in which a system based on our model could (and should) be used.
It should also be noted that these models and any derived technology are not meant to be tools to diagnose medical conditions, a task best left to qualified clinicians.

## License Information

### Wikipedia License
The Wikipedia data was derived from the Huggingface [Wikipedia](https://huggingface.co/datasets/wikipedia) dataset. 
That portion of the data is subject to the following license information:

> Most of Wikipedia's text and many of its images are co-licensed under the
[Creative Commons Attribution-ShareAlike 3.0 Unported License](https://en.wikipedia.org/wiki/Wikipedia:Text_of_Creative_Commons_Attribution-ShareAlike_3.0_Unported_License)
(CC BY-SA) and the [GNU Free Documentation License](https://en.wikipedia.org/wiki/Wikipedia:Text_of_the_GNU_Free_Documentation_License)
(GFDL) (unversioned, with no invariant sections, front-cover texts, or back-cover texts). 
>
> Some text has been imported only under CC BY-SA and CC BY-SA-compatible license and cannot be reused under GFDL; such
text will be identified on the page footer, in the page history, or on the discussion page of the article that utilizes
the text.

### CMUDict License
Pronunciation dictionaries contained herein were adapted from [CMUDict](https://github.com/cmusphinx/cmudict), and as 
such are subject to [their license](cmudict.license.txt).
license:cc-by-sa-3.0 region:us
{"license": ["cc-by-sa-3.0"]}
TAGS
#license-cc-by-sa-3.0 #region-us 


# BORT Wikipedia Data

This is the data used to prepare the BORT model, described by the following paper:

Robert Gale, Alexandra C. Salem, Gerasimos Fergadiotis, and Steven Bedrick. 2023. Mixed Orthographic/Phonemic Language Modeling: Beyond Orthographically Restricted Transformers (BORT). In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP-2023), pages TBD, Online. Association for Computational Linguistics. [[paper]](URL [[poster]](URL

Additional resources and information can be found here.

## Acknowledgements

This work was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under award 5R01DC015999 (Principal Investigators: Bedrick \& Fergadiotis). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. 

## Limitations

The models presented here were trained with the basic inventory of English phonemes found in CMUDict. However, a more fine-grained phonetic analysis would require a pronunciation dictionary with more narrowly defined entries. Additionally, while this paper focused on models trained with English-only resources (pre-trained BART-BASE, English Wikipedia text, CMUDict, and the English AphasiaBank), the techniques should be applicable to non-English language models as well. Finally, from a clinical standpoint, the model we describe in this paper assumes the existence of transcribed input (from either a manual or automated source, discussed in detail in §2.1 of the paper; in its current form, this represents a limitation to its clinical implementation, though not to its use in research settings with archival or newly-transcribed datasets.

## Ethics Statement

Our use of the AphasiaBank data was governed by the TalkBank consortium's data use agreement, and the underlying recordings were collected and shared with approval of the contributing sites' institutional review boards.
Limitations exist regarding accents and dialect, which in turn would affect the scenarios in which a system based on our model could (and should) be used.
It should also be noted that these models and any derived technology are not meant to be tools to diagnose medical conditions, a task best left to qualified clinicians.

## License Information

### Wikipedia License
The Wikipedia data was derived from the Huggingface Wikipedia dataset. 
That portion of the data is subject to the following license information:

> Most of Wikipedia's text and many of its images are co-licensed under the
Creative Commons Attribution-ShareAlike 3.0 Unported License
(CC BY-SA) and the GNU Free Documentation License
(GFDL) (unversioned, with no invariant sections, front-cover texts, or back-cover texts). 
>
> Some text has been imported only under CC BY-SA and CC BY-SA-compatible license and cannot be reused under GFDL; such
text will be identified on the page footer, in the page history, or on the discussion page of the article that utilizes
the text.

### CMUDict License
Pronunciation dictionaries contained herein were adapted from CMUDict, and as 
such are subject to their license.
# BORT Wikipedia Data

This is the data used to prepare the BORT model, described by the following paper:

Robert Gale, Alexandra C. Salem, Gerasimos Fergadiotis, and Steven Bedrick. 2023. Mixed Orthographic/Phonemic Language Modeling: Beyond Orthographically Restricted Transformers (BORT). In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP-2023), pages TBD, Online. Association for Computational Linguistics. [[paper]](URL [[poster]](URL

Additional resources and information can be found here. ## Acknowledgements

This work was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under award 5R01DC015999 (Principal Investigators: Bedrick \& Fergadiotis). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. ## Limitations

The models presented here were trained with the basic inventory of English phonemes found in CMUDict. However, a more fine-grained phonetic analysis would require a pronunciation dictionary with more narrowly defined entries. Additionally, while this paper focused on models trained with English-only resources (pre-trained BART-BASE, English Wikipedia text, CMUDict, and the English AphasiaBank), the techniques should be applicable to non-English language models as well. Finally, from a clinical standpoint, the model we describe in this paper assumes the existence of transcribed input (from either a manual or automated source, discussed in detail in §2.1 of the paper; in its current form, this represents a limitation to its clinical implementation, though not to its use in research settings with archival or newly-transcribed datasets. ## Ethics Statement

Our use of the AphasiaBank data was governed by the TalkBank consortium's data use agreement, and the underlying recordings were collected and shared with approval of the contributing sites' institutional review boards.
Limitations exist regarding accents and dialect, which in turn would affect the scenarios in which a system based on our model could (and should) be used.
It should also be noted that these models and any derived technology are not meant to be tools to diagnose medical conditions, a task best left to qualified clinicians. ## License Information ### Wikipedia License
The Wikipedia data was derived from the Huggingface Wikipedia dataset. 
That portion of the data is subject to the following license information:

> Most of Wikipedia's text and many of its images are co-licensed under the
Creative Commons Attribution-ShareAlike 3.0 Unported License
(CC BY-SA) and the GNU Free Documentation License
(GFDL) (unversioned, with no invariant sections, front-cover texts, or back-cover texts). 
>
> Some text has been imported only under CC BY-SA and CC BY-SA-compatible license and cannot be reused under GFDL; such
text will be identified on the page footer, in the page history, or on the discussion page of the article that utilizes
the text. ### CMUDict License
Pronunciation dictionaries contained herein were adapted from CMUDict, and as 
such are subject to their license.
TAGS
#license-cc-by-sa-3.0 #region-us 
 # BORT Wikipedia Data

This is the data used to prepare the BORT model, described by the following paper:

Robert Gale, Alexandra C. Salem, Gerasimos Fergadiotis, and Steven Bedrick. 2023. Mixed Orthographic/Phonemic Language Modeling: Beyond Orthographically Restricted Transformers (BORT). In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP-2023), pages TBD, Online. Association for Computational Linguistics. [[paper]](URL [[poster]](URL

Additional resources and information can be found here. ## Acknowledgements

This work was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under award 5R01DC015999 (Principal Investigators: Bedrick \& Fergadiotis). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. ## Limitations

The models presented here were trained with the basic inventory of English phonemes found in CMUDict. However, a more fine-grained phonetic analysis would require a pronunciation dictionary with more narrowly defined entries. Additionally, while this paper focused on models trained with English-only resources (pre-trained BART-BASE, English Wikipedia text, CMUDict, and the English AphasiaBank), the techniques should be applicable to non-English language models as well. Finally, from a clinical standpoint, the model we describe in this paper assumes the existence of transcribed input (from either a manual or automated source, discussed in detail in §2.1 of the paper; in its current form, this represents a limitation to its clinical implementation, though not to its use in research settings with archival or newly-transcribed datasets. ## Ethics Statement

Our use of the AphasiaBank data was governed by the TalkBank consortium's data use agreement, and the underlying recordings were collected and shared with approval of the contributing sites' institutional review boards.
Limitations exist regarding accents and dialect, which in turn would affect the scenarios in which a system based on our model could (and should) be used.
It should also be noted that these models and any derived technology are not meant to be tools to diagnose medical conditions, a task best left to qualified clinicians. ## License Information ### Wikipedia License
The Wikipedia data was derived from the Huggingface Wikipedia dataset. 
That portion of the data is subject to the following license information:

> Most of Wikipedia's text and many of its images are co-licensed under the
Creative Commons Attribution-ShareAlike 3.0 Unported License
(CC BY-SA) and the GNU Free Documentation License
(GFDL) (unversioned, with no invariant sections, front-cover texts, or back-cover texts). 
>
> Some text has been imported only under CC BY-SA and CC BY-SA-compatible license and cannot be reused under GFDL; such
text will be identified on the page footer, in the page history, or on the discussion page of the article that utilizes
the text. ### CMUDict License
Pronunciation dictionaries contained herein were adapted from CMUDict, and as 
such are subject to their license.
passage: TAGS
#license-cc-by-sa-3.0 #region-us 
# BORT Wikipedia Data

This is the data used to prepare the BORT model, described by the following paper:

Robert Gale, Alexandra C. Salem, Gerasimos Fergadiotis, and Steven Bedrick. 2023. Mixed Orthographic/Phonemic Language Modeling: Beyond Orthographically Restricted Transformers (BORT). In Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP-2023), pages TBD, Online. Association for Computational Linguistics. [[paper]](URL [[poster]](URL

Additional resources and information can be found here.## Acknowledgements

This work was supported by the National Institute on Deafness and Other Communication Disorders of the National Institutes of Health under award 5R01DC015999 (Principal Investigators: Bedrick \& Fergadiotis). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.## Limitations

The models presented here were trained with the basic inventory of English phonemes found in CMUDict. However, a more fine-grained phonetic analysis would require a pronunciation dictionary with more narrowly defined entries. Additionally, while this paper focused on models trained with English-only resources (pre-trained BART-BASE, English Wikipedia text, CMUDict, and the English AphasiaBank), the techniques should be applicable to non-English language models as well. Finally, from a clinical standpoint, the model we describe in this paper assumes the existence of transcribed input (from either a manual or automated source, discussed in detail in §2.1 of the paper; in its current form, this represents a limitation to its clinical implementation, though not to its use in research settings with archival or newly-transcribed datasets.