code: def _update_params(param_arrays, grad_arrays, updater, num_device,
                     kvstore=None, param_names=None):
      
      updates = [[] for _ in range(num_device)]
      for i, pair in enumerate(zip(param_arrays, grad_arrays)):
          arg_list, grad_list = pair
          if grad_list[0] is None:
              continue
          index = i
          if kvstore:
              name = param_names[index]
              # push gradient, priority is negative index
              kvstore.push(name, grad_list, priority=-index)
              # pull back the sum gradients, to the same locations.
              kvstore.pull(name, grad_list, priority=-index)
          for k, p in enumerate(zip(arg_list, grad_list)):
              # faked an index here, to make optimizer create diff
              # state for the same index but on diff devs, TODO(mli)
              # use a better solution later
              w, g = p
              updates[k].append((index*num_device+k, g, w))
      for dev_updates in updates:
          # update params if param_arrays and grad_arrays are not empty
          if dev_updates:
              i, w, g = zip(*dev_updates)
              updater(i, w, g)
docs: Perform update of param_arrays from grad_arrays not on kvstore.