code: def _apply_shadow_vars(avg_grads):
          
          ps_var_grads = []
          for grad, var in avg_grads:
              assert var.name.startswith('tower'), var.name
              my_name = '/'.join(var.name.split('/')[1:])
              my_name = get_op_tensor_name(my_name)[0]
              new_v = tf.get_variable(my_name, dtype=var.dtype.base_dtype,
                                      initializer=var.initial_value,
                                      trainable=True)
              # (g, v) to be applied, where v is global (ps vars)
              ps_var_grads.append((grad, new_v))
          return ps_var_grads
docs: Create shadow variables on PS, and replace variables in avg_grads
          by these shadow variables.
  
          Args:
              avg_grads: list of (grad, var) tuples