code: def _initialize_memory(self, policy_params):
      
      # We store observation, action, policy parameters, and reward.
      template = (
          self._batch_env.observ[0],
          self._batch_env.action[0],
          tools.nested.map(lambda x: x[0, 0], policy_params),
          self._batch_env.reward[0])
      with tf.variable_scope('ppo_temporary'):
        self._current_episodes = parts.EpisodeMemory(
            template, len(self._batch_env), self._config.max_length, 'episodes')
      self._finished_episodes = parts.EpisodeMemory(
          template, self._config.update_every, self._config.max_length, 'memory')
      self._num_finished_episodes = tf.Variable(0, False)
docs: Initialize temporary and permanent memory.
  
      Args:
        policy_params: Nested tuple of policy parameters with all dimensions set.
  
      Initializes the attributes `self._current_episodes`,
      `self._finished_episodes`, and `self._num_finished_episodes`. The episodes
      memory serves to collect multiple episodes in parallel. Finished episodes
      are copied into the next free slot of the second memory. The memory index
      points to the next free slot.