code: def _warmup(self, num_updates):
          
          assert self.base_lr is not None
          if not self.warmup:
              return self.base_lr
          fraction = (num_updates + 1) * self.base_lr / (self.warmup + 1)
          if num_updates > self.last_warmup_log and num_updates % self.log_warmup_every_t == 0:
              self.last_warmup_log = num_updates
              logger.info("Learning rate warmup: %3.0f%%", fraction / self.base_lr * 100.0)
          return fraction
docs: Returns linearly increasing fraction of base_lr.