
# LDCC-SOLAR-gugutypus-10.7B

<img src="logo.png" height=350, width=350>


---


## Model Details

**Model Developers**  
- DongGeon Lee ([oneonlee](https://huggingface.co/oneonlee))

**Model Architecture**  
- **LDCC-SOLAR-gugutypus-10.7B** is a instruction fine-tuned auto-regressive language model, based on the [SOLAR](https://huggingface.co/upstage/SOLAR-10.7B-v1.0) transformer architecture.

**Base Model**
- [LDCC/LDCC-SOLAR-10.7B](https://huggingface.co/LDCC/LDCC-SOLAR-10.7B)

**Training Dataset**
- [kyujinpy/KOR-gugugu-platypus-set](https://huggingface.co/datasets/kyujinpy/KOR-gugugu-platypus-set)


---  


## Model comparisons

- **Ko-LLM leaderboard (YYYY/MM/DD)** [[link]](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)

| Model                 | Average | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
| --------------------- | ------- | ------ | ------------ | ------- | ------------- | --------------- |
| **LDCC-SOLAR-gugutypus-10.7B** | NaN     | NaN    | NaN          | NaN     | NaN           | NaN             |

<br>

- **(KOR) AI-Harness evaluation** [[link]](https://github.com/Beomi/ko-lm-evaluation-harness)  


|          Tasks          |Version|Filter|n-shot|Metric|Value |   |Stderr|
|-------------------------|-------|------|-----:|------|-----:|---|-----:|
|KMMLU                    |N/A    |none  |     0|acc   |0.3329|±  |0.0794|
|KMMLU                    |N/A    |none  |     5|acc   |0.3969|±  |0.0816|
|KoBEST-HellaSwag         |      0|none  |     0|acc   |0.4260|±  |0.0221|
|KoBEST-HellaSwag         |      0|none  |     5|acc   |0.4260|±  |0.0221|
|KoBEST-BoolQ             |      0|none  |     0|acc   |0.7792|±  |0.0111|
|KoBEST-BoolQ             |      0|none  |     5|acc   |0.8925|±  |0.0083|
|KoBEST-COPA              |      0|none  |     0|acc   |0.6670|±  |0.0149|
|KoBEST-COPA              |      0|none  |     5|acc   |0.7070|±  |0.0144|
|KoBEST-SentiNeg          |      0|none  |     0|acc   |0.7582|±  |0.0215|
|KoBEST-SentiNeg          |      0|none  |     5|acc   |0.9219|±  |0.0135|

<br>

- **(ENG) AI-Harness evaluation** [[link]](https://github.com/EleutherAI/lm-evaluation-harness)

|      Tasks      |Version|Filter|n-shot|Metric|Value |   |Stderr|
|------------------|-------|------|-----:|------|-----:|---|-----:|
|MMLU              |N/A    |none  |     0|acc   |0.5826|±  |0.1432|
|MMLU              |N/A    |none  |     5|acc   |0.6124|±  |0.1275|
|HellaSwag         |      1|none  |     0|acc   |0.6075|±  |0.0049|
|HellaSwag         |      1|none  |     5|acc   |0.6534|±  |0.0047|
|BoolQ             |      2|none  |     0|acc   |0.8737|±  |0.0058|
|BoolQ             |      2|none  |     5|acc   |0.8878|±  |0.0055|
|COPA              |      1|none  |     0|acc   |0.8300|±  |0.0378|
|COPA              |      1|none  |     5|acc   |0.9300|±  |0.0256|
|truthfulqa        |N/A    |none  |     0|acc   |0.4249|±  |0.0023|
|truthfulqa        |N/A    |none  |     5|acc   |  -   |±  |   -  |


---


## Implementation Code

```python
### LDCC-SOLAR-gugutypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "oneonlee/LDCC-SOLAR-gugutypus-10.7B"
model = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
tokenizer = AutoTokenizer.from_pretrained(repo)
```
{"language": ["en", "ko"], "license": "cc-by-nc-4.0", "datasets": ["kyujinpy/KOR-gugugu-platypus-set"], "base_model": ["LDCC/LDCC-SOLAR-10.7B"], "pipeline_tag": "text-generation"}
transformers safetensors llama text-generation en ko dataset:kyujinpy/KOR-gugugu-platypus-set base_model:LDCC/LDCC-SOLAR-10.7B license:cc-by-nc-4.0 autotrain_compatible endpoints_compatible text-generation-inference region:us