ufotalent commited on
Commit
34d8533
·
verified ·
1 Parent(s): 64f8975

Update description2.md

Browse files
Files changed (1) hide show
  1. description2.md +1 -1
description2.md CHANGED
@@ -17,7 +17,7 @@ The key of achieving zero bubble is to breaking a backward pass into a B pass an
17
 
18
  | Comparison assuming T_F=T_B=T_W | 1F1B | ZB1P | ZB2P | ZBV (Recommended) |
19
  | ----------------------------------------------------- | ------- | -------- | ---- | --- |
20
- | Bubble Rate | (p-1)/m | (p-1)/3m | 0 | 0 |
21
  | Activation Memory <br> (Compared to 1F1B) | 1x | 1x | 2x | 1x |
22
  | Pipeline Communication Volume <br> (Compared to 1F1B) | 1x | 1x | 1x | 2x |
23
 
 
17
 
18
  | Comparison assuming T_F=T_B=T_W | 1F1B | ZB1P | ZB2P | ZBV (Recommended) |
19
  | ----------------------------------------------------- | ------- | -------- | ---- | --- |
20
+ | Bubble Rate | (p-1)/(m+p-1) | (p-1)/3(m+p-1) | 0 | 0 |
21
  | Activation Memory <br> (Compared to 1F1B) | 1x | 1x | 2x | 1x |
22
  | Pipeline Communication Volume <br> (Compared to 1F1B) | 1x | 1x | 1x | 2x |
23