Update description2.md
Browse files- description2.md +1 -1
description2.md
CHANGED
@@ -17,7 +17,7 @@ The key of achieving zero bubble is to breaking a backward pass into a B pass an
|
|
17 |
|
18 |
| Comparison assuming T_F=T_B=T_W | 1F1B | ZB1P | ZB2P | ZBV (Recommended) |
|
19 |
| ----------------------------------------------------- | ------- | -------- | ---- | --- |
|
20 |
-
| Bubble Rate | (p-1)/m | (p-1)/
|
21 |
| Activation Memory <br> (Compared to 1F1B) | 1x | 1x | 2x | 1x |
|
22 |
| Pipeline Communication Volume <br> (Compared to 1F1B) | 1x | 1x | 1x | 2x |
|
23 |
|
|
|
17 |
|
18 |
| Comparison assuming T_F=T_B=T_W | 1F1B | ZB1P | ZB2P | ZBV (Recommended) |
|
19 |
| ----------------------------------------------------- | ------- | -------- | ---- | --- |
|
20 |
+
| Bubble Rate | (p-1)/(m+p-1) | (p-1)/3(m+p-1) | 0 | 0 |
|
21 |
| Activation Memory <br> (Compared to 1F1B) | 1x | 1x | 2x | 1x |
|
22 |
| Pipeline Communication Volume <br> (Compared to 1F1B) | 1x | 1x | 1x | 2x |
|
23 |
|