code: def _experience(self, previousState, action, previousAction, reward):
          
          e = self.experimentation
  
          if action == previousAction:
              experience = reward * (1 - e)
          else:
              propensity = self.module.getValue(previousState, action)
              experience = propensity * (e / (self.module.numActions - 1))
  
          return experience
docs: This is an altered version of the experience function for used in
          the standard Roth-Erev algorithm.  Like in RELearner, propensities for
          all actions are updated and similarity does not come into play. If the
          actionIndex points to the action the reward is associated with (usually
          the last action taken) then simply adjust the weight by the
          experimentation. Otherwise increase the weight of the action by a small
          portion of its current propensity.
  
          If j is the index of the last action chosen, r_j is the reward received
          for performing j, i is the current action being updated, q_i is the
          propensity for i, n is the size of the action domain and e is the
          experimentation parameter, then this experience function can be
          expressed as::
  
                          |  r_j * (1-e)         if i = j
              E(i, r_j) = |
                          |_ q_i * (e /(n-1))    if i != j