
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gpt2-arxiv

A [gpt2](https://huggingface.co/gpt2) powered predictive keyboard trained on ~1.6M manuscript abstracts from the ArXiv. 
This model uses https://www.kaggle.com/datasets/Cornell-University/arxiv

```python
from transformers import pipeline
from transformers import GPT2TokenizerFast

tokenizer = GPT2TokenizerFast.from_pretrained("gpt2")
llm = pipeline('text-generation',model='pearsonkyle/gpt2-arxiv', tokenizer=tokenizer)

texts = llm("Directly imaged exoplanets probe", 
             max_length=50, do_sample=True, num_return_sequences=5, 
             penalty_alpha=0.65, top_k=40, repetition_penalty=1.25,
             temperature=0.95)

for i in range(5):
    print(texts[i]['generated_text']+'\n')
```

- *The reflectance of Earth's vegetation suggests* `that large, deciduous forest fires are composed of mostly dry, unprocessed material that is distributed in a nearly patchy fashion. The distributions of these fires are correlated with temperature, and also with vegetation...`
- *Directly imaged exoplanets probe* `the atmospheres of giant planets. The detection of such planets requires high-quality imaging with high contrast and angular resolution, as well as`
- *We can remotely sense an atmosphere by observing its reflected, transmitted, or emitted light in varying geometries. This light will contain information on* `the planetary conditions including atmospheric temperature and cloud properties, which is essential for understanding how the planet interacts with the atmosphere and how it affects the climate. The primary science objective of this paper is to develop a methodology that can be applied to any kind of observation and measurement data, and to provide a framework that enables the detection and characterization of the atmospheres of exoplanets`


## Model description

[GPT-2](https://huggingface.co/transformers/v2.2.0/pretrained_models.html): 12-layer, 768-hidden, 12-heads, 117M parameters

## Intended uses & limitations

Coming soon...
- Predictive Keyboard using text generation
- Realtime reference recommendations using nearest neighbors of embeddings

Be careful when generating a lot of text or when changing the sampling mode of the language model. It can sometimes produce things that are not truthful, e.g.,

- The surface of Mars is composed of a thin layer of water ice, that was discovered by the Cassini spacecraft after its impact on the Earth's surface.

## Training procedure

~49 hours on a 3090 training for 1.25M iterations

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10

### Framework versions

- Transformers 4.25.1
- Pytorch 1.13.1
- Tokenizers 0.13.2

{"tags": ["generated_from_trainer"], "model-index": [{"name": "gpt2-arxiv", "results": []}]}
transformers pytorch gpt2 text-generation generated_from_trainer autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #gpt2 #text-generation #generated_from_trainer #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 




# gpt2-arxiv

A gpt2 powered predictive keyboard trained on ~1.6M manuscript abstracts from the ArXiv. 
This model uses URL



- *The reflectance of Earth's vegetation suggests* 'that large, deciduous forest fires are composed of mostly dry, unprocessed material that is distributed in a nearly patchy fashion. The distributions of these fires are correlated with temperature, and also with vegetation...'
- *Directly imaged exoplanets probe* 'the atmospheres of giant planets. The detection of such planets requires high-quality imaging with high contrast and angular resolution, as well as'
- *We can remotely sense an atmosphere by observing its reflected, transmitted, or emitted light in varying geometries. This light will contain information on* 'the planetary conditions including atmospheric temperature and cloud properties, which is essential for understanding how the planet interacts with the atmosphere and how it affects the climate. The primary science objective of this paper is to develop a methodology that can be applied to any kind of observation and measurement data, and to provide a framework that enables the detection and characterization of the atmospheres of exoplanets'


## Model description

GPT-2: 12-layer, 768-hidden, 12-heads, 117M parameters

## Intended uses & limitations

Coming soon...
- Predictive Keyboard using text generation
- Realtime reference recommendations using nearest neighbors of embeddings

Be careful when generating a lot of text or when changing the sampling mode of the language model. It can sometimes produce things that are not truthful, e.g.,

- The surface of Mars is composed of a thin layer of water ice, that was discovered by the Cassini spacecraft after its impact on the Earth's surface.

## Training procedure

~49 hours on a 3090 training for 1.25M iterations

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10

### Framework versions

- Transformers 4.25.1
- Pytorch 1.13.1
- Tokenizers 0.13.2

# gpt2-arxiv

A gpt2 powered predictive keyboard trained on ~1.6M manuscript abstracts from the ArXiv. 
This model uses URL



- *The reflectance of Earth's vegetation suggests* 'that large, deciduous forest fires are composed of mostly dry, unprocessed material that is distributed in a nearly patchy fashion. The distributions of these fires are correlated with temperature, and also with vegetation...'
- *Directly imaged exoplanets probe* 'the atmospheres of giant planets. The detection of such planets requires high-quality imaging with high contrast and angular resolution, as well as'
- *We can remotely sense an atmosphere by observing its reflected, transmitted, or emitted light in varying geometries. This light will contain information on* 'the planetary conditions including atmospheric temperature and cloud properties, which is essential for understanding how the planet interacts with the atmosphere and how it affects the climate. The primary science objective of this paper is to develop a methodology that can be applied to any kind of observation and measurement data, and to provide a framework that enables the detection and characterization of the atmospheres of exoplanets' ## Model description

GPT-2: 12-layer, 768-hidden, 12-heads, 117M parameters ## Intended uses & limitations

Coming soon...
- Predictive Keyboard using text generation
- Realtime reference recommendations using nearest neighbors of embeddings

Be careful when generating a lot of text or when changing the sampling mode of the language model. It can sometimes produce things that are not truthful, e.g.,

- The surface of Mars is composed of a thin layer of water ice, that was discovered by the Cassini spacecraft after its impact on the Earth's surface. ## Training procedure

~49 hours on a 3090 training for 1.25M iterations ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10 ### Framework versions

- Transformers 4.25.1
- Pytorch 1.13.1
- Tokenizers 0.13.2
TAGS
#transformers #pytorch #gpt2 #text-generation #generated_from_trainer #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
 # gpt2-arxiv

A gpt2 powered predictive keyboard trained on ~1.6M manuscript abstracts from the ArXiv. 
This model uses URL



- *The reflectance of Earth's vegetation suggests* 'that large, deciduous forest fires are composed of mostly dry, unprocessed material that is distributed in a nearly patchy fashion. The distributions of these fires are correlated with temperature, and also with vegetation...'
- *Directly imaged exoplanets probe* 'the atmospheres of giant planets. The detection of such planets requires high-quality imaging with high contrast and angular resolution, as well as'
- *We can remotely sense an atmosphere by observing its reflected, transmitted, or emitted light in varying geometries. This light will contain information on* 'the planetary conditions including atmospheric temperature and cloud properties, which is essential for understanding how the planet interacts with the atmosphere and how it affects the climate. The primary science objective of this paper is to develop a methodology that can be applied to any kind of observation and measurement data, and to provide a framework that enables the detection and characterization of the atmospheres of exoplanets' ## Model description

GPT-2: 12-layer, 768-hidden, 12-heads, 117M parameters ## Intended uses & limitations

Coming soon...
- Predictive Keyboard using text generation
- Realtime reference recommendations using nearest neighbors of embeddings

Be careful when generating a lot of text or when changing the sampling mode of the language model. It can sometimes produce things that are not truthful, e.g.,

- The surface of Mars is composed of a thin layer of water ice, that was discovered by the Cassini spacecraft after its impact on the Earth's surface. ## Training procedure

~49 hours on a 3090 training for 1.25M iterations ### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10 ### Framework versions

- Transformers 4.25.1
- Pytorch 1.13.1
- Tokenizers 0.13.2
TAGS
#transformers #pytorch #gpt2 #text-generation #generated_from_trainer #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
# gpt2-arxiv

A gpt2 powered predictive keyboard trained on ~1.6M manuscript abstracts from the ArXiv. 
This model uses URL



- *The reflectance of Earth's vegetation suggests* 'that large, deciduous forest fires are composed of mostly dry, unprocessed material that is distributed in a nearly patchy fashion. The distributions of these fires are correlated with temperature, and also with vegetation...'
- *Directly imaged exoplanets probe* 'the atmospheres of giant planets. The detection of such planets requires high-quality imaging with high contrast and angular resolution, as well as'
- *We can remotely sense an atmosphere by observing its reflected, transmitted, or emitted light in varying geometries. This light will contain information on* 'the planetary conditions including atmospheric temperature and cloud properties, which is essential for understanding how the planet interacts with the atmosphere and how it affects the climate. The primary science objective of this paper is to develop a methodology that can be applied to any kind of observation and measurement data, and to provide a framework that enables the detection and characterization of the atmospheres of exoplanets'## Model description

GPT-2: 12-layer, 768-hidden, 12-heads, 117M parameters## Intended uses & limitations

Coming soon...
- Predictive Keyboard using text generation
- Realtime reference recommendations using nearest neighbors of embeddings

Be careful when generating a lot of text or when changing the sampling mode of the language model. It can sometimes produce things that are not truthful, e.g.,

- The surface of Mars is composed of a thin layer of water ice, that was discovered by the Cassini spacecraft after its impact on the Earth's surface.## Training procedure

~49 hours on a 3090 training for 1.25M iterations### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 16
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 10### Framework versions

- Transformers 4.25.1
- Pytorch 1.13.1
- Tokenizers 0.13.2