


`mental-longformer-base-4096` is a model pretrained from the checkpoint of [longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) for the mental healthcare domain. [Longformer](https://arxiv.org/abs/2004.05150) is a transformer model for long documents. `longformer-base-4096` is a BERT-like model started from the RoBERTa checkpoint and pretrained for MLM on long documents. It supports sequences of length up to 4,096. 

## Usage
Load the model via [Huggingface’s Transformers library](https://github.com/huggingface/transformers):
```
from transformers import LongformerTokenizer, LongformerModel
tokenizer = LongformerTokenizer.from_pretrained("AIMH/mental-longformer-base-4096")
model = LongformerModel.from_pretrained("AIMH/mental-longformer-base-4096")
```

To minimize the influence of worrying mask predictions, this model is gated.  To download a gated model, you’ll need to be authenticated. 
Know more about [gated models](https://huggingface.co/docs/hub/models-gated). 

## Paper
```
@article{ji-domain-specific,
  author        = {Shaoxiong Ji and Tianlin Zhang and Kailai Yang and Sophia Ananiadou and Erik Cambria and J{\"o}rg Tiedemann},
  journal       = {arXiv preprint arXiv:2304.10447},
  title         = {Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Health},
  year          = {2023},
  url           = {https://arxiv.org/abs/2304.10447}
}
```

## Disclaimer

The model predictions are not psychiatric diagnoses.
We recommend anyone who suffers from mental health issues to call the local mental health helpline and seek professional help if possible.
{"license": "cc-by-nc-4.0"}
transformers pytorch longformer fill-mask arxiv:2004.05150 arxiv:2304.10447 license:cc-by-nc-4.0 autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #pytorch #longformer #fill-mask #arxiv-2004.05150 #arxiv-2304.10447 #license-cc-by-nc-4.0 #autotrain_compatible #endpoints_compatible #region-us 




'mental-longformer-base-4096' is a model pretrained from the checkpoint of longformer-base-4096 for the mental healthcare domain. Longformer is a transformer model for long documents. 'longformer-base-4096' is a BERT-like model started from the RoBERTa checkpoint and pretrained for MLM on long documents. It supports sequences of length up to 4,096. 

## Usage
Load the model via Huggingface’s Transformers library:


To minimize the influence of worrying mask predictions, this model is gated.  To download a gated model, you’ll need to be authenticated. 
Know more about gated models. 

## Paper


## Disclaimer

The model predictions are not psychiatric diagnoses.
We recommend anyone who suffers from mental health issues to call the local mental health helpline and seek professional help if possible.
## Usage
Load the model via Huggingface’s Transformers library:


To minimize the influence of worrying mask predictions, this model is gated.  To download a gated model, you’ll need to be authenticated. 
Know more about gated models. ## Paper ## Disclaimer

The model predictions are not psychiatric diagnoses.
We recommend anyone who suffers from mental health issues to call the local mental health helpline and seek professional help if possible.
TAGS
#transformers #pytorch #longformer #fill-mask #arxiv-2004.05150 #arxiv-2304.10447 #license-cc-by-nc-4.0 #autotrain_compatible #endpoints_compatible #region-us 
 ## Usage
Load the model via Huggingface’s Transformers library:


To minimize the influence of worrying mask predictions, this model is gated.  To download a gated model, you’ll need to be authenticated. 
Know more about gated models. ## Paper ## Disclaimer

The model predictions are not psychiatric diagnoses.
We recommend anyone who suffers from mental health issues to call the local mental health helpline and seek professional help if possible.
TAGS
#transformers #pytorch #longformer #fill-mask #arxiv-2004.05150 #arxiv-2304.10447 #license-cc-by-nc-4.0 #autotrain_compatible #endpoints_compatible #region-us 
## Usage
Load the model via Huggingface’s Transformers library:


To minimize the influence of worrying mask predictions, this model is gated.  To download a gated model, you’ll need to be authenticated. 
Know more about gated models.## Paper## Disclaimer

The model predictions are not psychiatric diagnoses.
We recommend anyone who suffers from mental health issues to call the local mental health helpline and seek professional help if possible.