transformers openvino gpt_neox text-generation en autotrain_compatible endpoints_compatible text-generation-inference region:us
---
language:
- en
tags:
- openvino
---

# EleutherAI-gpt-neox-20b-ov-int8

This is the [EleutherAI/gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) model converted to [OpenVINO](https://openvino.ai), for accelerated inference.
Model weights are compressed to INT8 with weight compression using [nncf](https://github.com/openvinotoolkit/nncf).

Use [optimum-intel](https://github.com/huggingface/optimum-intel) for inference ([documentation](https://huggingface.co/docs/optimum/intel/inference#inference)).
