Generative Agents arXiv, April, 2023,
NAME_1’s bedroom: desk) that has NAME_2
NAME_3’s
bedroom, NAME_1’s bedroom, common room, kitchen,
bathroom, and garden.
NAME_1 knows of the following areas: The NAME_3
family’s house, NAME_4, Harvey Oak Supply
Store, The Willows Market and Pharmacy, Hobbs
Cafe, The Rose and Crown Pub.
* Prefer to stay in the current area if the
activity can be done there.
NAME_1 is planning to take a short walk around
his workspace. Which area should NAME_1 go to?
This outputs The NAME_3 family’s house. We then use the same process
recursively to determine the most appropriate subarea within the
chosen area until we reach a leaf node of the agent’s environment
tree. In the example above, the result of this traversal is The NAME_3
family’s house: garden: house garden. Finally, we use traditional
game path algorithms to animate the agent’s movement so that it
travels to the location indicated by the leaf node.
When an agent executes an action on an object, we prompt the
language model to ask what happens to the state of the object. For
example, if NAME_5’s generative agent outputs the action “making
espresso for a customer”, a query to the language model indicates in
response that the state of the coffee machine in Hobbs Cafe should
change from “off” to “brewing coffee”.
6 CONTROLLED EVALUATION
Generative agents, as individual agents and as groups, aim to pro-
duce believable behavior based on their environment and experi-
ences. In our evaluation, we investigate the capacity and limitations
of generative agents. Do individual agents properly retrieve past
experience, and generate believable plans, reactions, and thoughts
that make up their behavior? Does a community of agents illustrate
information diffusion, relationship formation, and agent coordina-
tion across different pockets of the community?
We evaluate generative agents in two stages. We begin with a
more tightly controlled evaluation in this section, where we individ-
ually assess agent responses to understand whether they generate
believable behavior in narrowly defined contexts. Then, in our end-
to-end analysis of the agent community over two full days, we
investigate their emergent behavior as a collective as well as errors
and boundary conditions.
6.1 Evaluation Procedure
To assess generative agents in Smallville, we take advantage of
the fact that generative agents will respond to natural language
questions. So, we “interview” agents to probe their ability to re-
member past experiences, plan future actions based on t