code: def batch_remove_retrain(nmask_train, nmask_test, X_train, y_train, X_test, y_test, attr_train, attr_test, model_generator, metric):
      
  
      warnings.warn("The retrain based measures can incorrectly evaluate models in some cases!")
  
      X_train, X_test = to_array(X_train, X_test)
  
      # how many features to mask
      assert X_train.shape[1] == X_test.shape[1]
  
      # mask nmask top features for each explanation
      X_train_tmp = X_train.copy()
      X_train_mean = X_train.mean(0)
      tie_breaking_noise = const_rand(X_train.shape[1]) * 1e-6
      for i in range(len(y_train)):
          if nmask_train[i] > 0:
              ordering = np.argsort(-attr_train[i, :] + tie_breaking_noise)
              X_train_tmp[i, ordering[:nmask_train[i]]] = X_train_mean[ordering[:nmask_train[i]]]
      X_test_tmp = X_test.copy()
      for i in range(len(y_test)):
          if nmask_test[i] > 0:
              ordering = np.argsort(-attr_test[i, :] + tie_breaking_noise)
              X_test_tmp[i, ordering[:nmask_test[i]]] = X_train_mean[ordering[:nmask_test[i]]]
  
      # train the model with all the given features masked
      model_masked = model_generator()
      model_masked.fit(X_train_tmp, y_train)
      yp_test_masked = model_masked.predict(X_test_tmp)
  
      return metric(y_test, yp_test_masked)
docs: An approximation of holdout that only retraines the model once.
  
      This is alse called ROAR (RemOve And Retrain) in work by Google. It is much more computationally
      efficient that the holdout method because it masks the most important features in every sample
      and then retrains the model once, instead of retraining the model for every test sample like
      the holdout metric.