code: def get_global_gradient_norm(self) -> float:
          
          # average norm across executors:
          exec_norms = [global_norm([arr for arr in exe.grad_arrays if arr is not None]) for exe in self.executors]
          norm_val = sum(exec_norms) / float(len(exec_norms))
          norm_val *= self.optimizer.rescale_grad
          return norm_val
docs: Returns global gradient norm.