Update README.md
Browse files
README.md
CHANGED
@@ -133,12 +133,13 @@ The accelerated partition is composed of 1,120 nodes with the following specific
|
|
133 |
---
|
134 |
|
135 |
## How to use
|
|
|
136 |
|
137 |
### Inference
|
138 |
-
|
139 |
|
140 |
#### Inference with Huggingface's Text Generation Pipeline
|
141 |
-
The Huggingface Text Generation Pipeline provides a
|
142 |
|
143 |
```bash
|
144 |
pip install -U transformers
|
@@ -182,7 +183,7 @@ for output in outputs:
|
|
182 |
</details>
|
183 |
|
184 |
#### Inference with single / multi GPU
|
185 |
-
|
186 |
|
187 |
```bash
|
188 |
pip install transformers torch accelerate sentencepiece protobuf
|
|
|
133 |
---
|
134 |
|
135 |
## How to use
|
136 |
+
This section offers examples of how to perform inference using various methods.
|
137 |
|
138 |
### Inference
|
139 |
+
You'll find different techniques for running inference, including Huggingface's Text Generation Pipeline, multi-GPU configurations, and vLLM for scalable and efficient generation.
|
140 |
|
141 |
#### Inference with Huggingface's Text Generation Pipeline
|
142 |
+
The Huggingface Text Generation Pipeline provides a straightforward way to run inference using the Salamandra-7b model.
|
143 |
|
144 |
```bash
|
145 |
pip install -U transformers
|
|
|
183 |
</details>
|
184 |
|
185 |
#### Inference with single / multi GPU
|
186 |
+
This section provides a simple example of how to run inference using Huggingface's AutoModel class.
|
187 |
|
188 |
```bash
|
189 |
pip install transformers torch accelerate sentencepiece protobuf
|