sylwia-kuros commited on
Commit
5ffacf4
·
verified ·
1 Parent(s): e4331c2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -10
README.md CHANGED
@@ -2,23 +2,19 @@
2
  license: apache-2.0
3
  ---
4
 
5
- <!-- Model name used as model card title -->
6
  # TinyLlama-1.1B-Chat-v1.0-int8-ov
7
- <!-- Original model reference -->
8
  * Model creator: [TinyLlama](https://huggingface.co/TinyLlama)
9
  * Original model: [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
10
 
11
- <!-- Description of converted model -->
12
  ## Description
13
 
14
- <!-- Comment and reference on NNCF applicable only for INT8 and INT4 models -->
15
  This is [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) model converted to the [OpenVINO™ IR](https://docs.openvino.ai/2024/documentation/openvino-ir-format.html) (Intermediate Representation) format with weights compressed to INT8 by [NNCF](https://github.com/openvinotoolkit/nncf).
16
 
17
  ## Quantization Parameters
18
 
19
  Weight compression was performed using `nncf.compress_weights` with the following parameters:
20
 
21
-
22
  * mode: **INT8_ASYM**
23
  * ratio: **1.0**
24
 
@@ -33,8 +29,6 @@ The provided OpenVINO™ IR model is compatible with:
33
 
34
  ## Running Model Inference
35
 
36
- <!-- Example model usage -->
37
-
38
  1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
39
 
40
  ```
@@ -43,8 +37,6 @@ pip install optimum[openvino]
43
 
44
  2. Run model inference:
45
 
46
- <!-- Usage example can be adopted from original model usage example -->
47
-
48
  ```
49
  from transformers import AutoTokenizer
50
  from optimum.intel.openvino import OVModelForCausalLM
@@ -64,7 +56,6 @@ For more examples and possible optimizations, refer to the [OpenVINO Large Langu
64
 
65
  ## Legal information
66
 
67
- <!-- Note about original model license -->
68
  The original model is distributed under [apache-2.0](https://choosealicense.com/licenses/apache-2.0/) license. More details can be found in [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0).
69
 
70
  ## Disclaimer
 
2
  license: apache-2.0
3
  ---
4
 
 
5
  # TinyLlama-1.1B-Chat-v1.0-int8-ov
6
+
7
  * Model creator: [TinyLlama](https://huggingface.co/TinyLlama)
8
  * Original model: [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
9
 
 
10
  ## Description
11
 
 
12
  This is [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) model converted to the [OpenVINO™ IR](https://docs.openvino.ai/2024/documentation/openvino-ir-format.html) (Intermediate Representation) format with weights compressed to INT8 by [NNCF](https://github.com/openvinotoolkit/nncf).
13
 
14
  ## Quantization Parameters
15
 
16
  Weight compression was performed using `nncf.compress_weights` with the following parameters:
17
 
 
18
  * mode: **INT8_ASYM**
19
  * ratio: **1.0**
20
 
 
29
 
30
  ## Running Model Inference
31
 
 
 
32
  1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
33
 
34
  ```
 
37
 
38
  2. Run model inference:
39
 
 
 
40
  ```
41
  from transformers import AutoTokenizer
42
  from optimum.intel.openvino import OVModelForCausalLM
 
56
 
57
  ## Legal information
58
 
 
59
  The original model is distributed under [apache-2.0](https://choosealicense.com/licenses/apache-2.0/) license. More details can be found in [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0).
60
 
61
  ## Disclaimer