Create README.md

README.md

@@ -0,0 +1,47 @@
+---
+license: apache-2.0
+datasets:
+- SkelterLabsInc/JaQuAD
+language:
+- ja
+pipeline_tag: question-answering
+---
+
+ MambaSan-130m-instruct 🐍
+
+**MambaSan-instruct is the first chat Japanese language model based on a state-space model architecture (Mamba), not a transformer.**
+
+The model is based on Albert Gu's and Tri Dao's work *Mamba: Linear-Time Sequence Modeling with Selective State Spaces* ([paper](https://arxiv.org/pdf/2312.00752.pdf)) as well as their [model implementation](https://github.com/state-spaces/mamba).
+This work was also inspired by heavenq's mamba-chat implementation in English: 
+bibtex
+@misc{haven2023mambachat,
+  title        = {Mamba-Chat},
+  author       = {Justus Mattern and Konstantin Hohr},
+  year         = {2023},
+  howpublished = {GitHub},
+  url          = {https://github.com/havenhq/mamba-chat}
+}
+This repository provides training / fine-tuning code for the model based on some modifications of the Huggingface Trainer class.
+
+Mamba-Chat is based on MambaSan-130m and was fine-tuned on 31,7k examples samples of the [SkelterLabsInc/JaQuAD](https://huggingface.co/datasets/SkelterLabsInc/JaQuAD) dataset. To learn more, you can:
+
+- Take a look at the model on [Huggingface](https://huggingface.co/loic_cabannes/MambaSan-instruct) 🤗
+- Talk to Mamba-Chat on [Google Colab](https://colab.research.google.com/drive/1LDP9Wik98SSpoTlxAjKOmhRrcuk_OpT_?usp=sharing)
+
+
+The Code used for pretraining and finetuning will soon be published on my github: https://github.com/lcabannes
+<br>
+
+
+## Citation
+
+```
+bibtex
+@misc{lcabannes2024MambaSan-130m-instruct,
+  title        = {MambaSan-130-instruct},
+  author       = {Loïc Cabannes},
+  year         = {2024},
+  howpublished = {HuggingFace},
+  url          = {https://huggingface.co/loiccabannes/MambaSan-130m-instruct/}
+}
+```