code: def append_all_agent_batch_to_update_buffer(self, key_list=None, batch_size=None, training_length=None):
          
          for agent_id in self.keys():
              self.append_update_buffer(agent_id, key_list, batch_size, training_length)
docs: Appends the buffer of all agents to the update buffer.
          :param key_list: The fields that must be added. If None: all fields will be appended.
          :param batch_size: The number of elements that must be appended. If None: All of them will be.
          :param training_length: The length of the samples that must be appended. If None: only takes one element.