code: def optimize(self, loss, num_async_replicas=1, use_tpu=False):
      
      lr = learning_rate.learning_rate_schedule(self.hparams)
      if num_async_replicas > 1:
        log_info("Dividing learning rate by num_async_replicas: %d",
                 num_async_replicas)
      lr /= math.sqrt(float(num_async_replicas))
      train_op = optimize.optimize(loss, lr, self.hparams, use_tpu=use_tpu)
      return train_op
docs: Return a training op minimizing loss.