
Quantization of [madlad400-10b-mt](https://huggingface.co/google/madlad400-10b-mt) using [Ctranslate2](https://github.com/OpenNMT/CTranslate2) for running on CPU or Nvidia GPU.

Example usage:

```python
import ctranslate2, transformers
from huggingface_hub import snapshot_download

model_path = snapshot_download("zenoverflow/madlad400-10b-mt-ct2-int8-float16")
print("\n", end="")

translator = ctranslate2.Translator(model_path, device="auto")
tokenizer = transformers.T5Tokenizer.from_pretrained(model_path)

target_lang_code = "ja"

source_text = "This sentence has no meaning."

input_text = f"<2{target_lang_code}> {source_text}"
input_tokens = tokenizer.convert_ids_to_tokens(tokenizer.encode(input_text))
results = translator.translate_batch([input_tokens])
output_tokens = results[0].hypotheses[0]
output_text = tokenizer.decode(tokenizer.convert_tokens_to_ids(output_tokens))

print(output_text)

```
{"license": "apache-2.0", "pipeline_tag": "translation", "inference": false}
transformers translation license:apache-2.0 region:us