Perceptual Context in Cognitive Hierarchies
Bernhard Hengst and Maurice Pagnucco and David Rajaratnam
Claude Sammut and Michael Thielscher

arXiv:1801.02270v1 [] 7 Jan 2018

School of Computer Science and Engineering
The University of New South Wales, Australia
{bernhardh,morri,daver,claude,mit}@cse.unsw.edu.au

Abstract
Cognition does not only depend on bottom-up sensor feature abstraction, but also relies on contextual information being passed top-down. Context is higher level information that
helps to predict belief states at lower levels. The main contribution of this paper is to provide a formalisation of perceptual
context and its integration into a new process model for cognitive hierarchies. Several simple instantiations of a cognitive
hierarchy are used to illustrate the role of context. Notably,
we demonstrate the use context in a novel approach to visually track the pose of rigid objects with just a 2D camera.

Introduction
There is strong evidence that intelligence necessarily involves hierarchical structures (Ashby 1952; Brooks 1986;
Dietterich 2000; Albus and Meystel 2001; Beer 1966;
Turchin 1977; Hubel and Wiesel 1979; Minsky 1986;
Drescher 1991; Dayan and Hinton 1992; Kaelbling 1993;
Nilsson 2001; Konidaris et al. 2011; Jong 2010; Marthi,
Russell, and Andre 2006; Bakker and Schmidhuber 2004).
Clark et al. (2016) recently have addressed the formalisation of cognitive hierarchies that allow for the integration of disparate representations, including symbolic and
sub-symbolic representations, in a framework for cognitive
robotics. Sensory information processing is upward-feeding,
progressively abstracting more complex state features, while
behaviours are downward-feeding progressively becoming
more concrete, ultimately controlling robot actuators.
However, neuroscience suggests that the brain is subject
to top-down cognitive influences for attention, expectation
and perception (Gilbert and Li 2013). Higher level signals
carry important information to facilitate scene interpretation.
For example, the recognition of the Dalmatian, and the disambiguation of the symbol /−\ in Figure 1 intuitively show
that higher level context is necessary to correctly interpret
these images1 . Furthermore, the human brain is able to make
sense of dynamic 3D scenes from light falling on our 2D
retina in varying lighting conditions. Replicating this ability
is still a challenge in artificial intelligence and computer vision, particularly when objects move relative to each other,
can occlude each other, and are without texture. Prior, more
1
Both of these examples appear in (Johnson 2010) but are also
well-known in the cognitive psychology literature.

Figure 1: The image on the left would probably be indiscernible without prior knowledge of Dalmations. The ambiguous symbol /−\ on the right can be interpreted as either
an “H” or an “A” depending on the word context.

abstract contextual knowledge is important to help segment
images into objects or to confirm the presence of an object
from faint or partial edges in an image.
In this paper we extend the existing cognitive hierarchy
formalisation (Clark et al. 2016) by introducing the notion
of perceptual context, which modifies the beliefs of a child
node given the beliefs of its parent nodes. It is worth emphasising that defining the role of context as a top-down predictive influence on a node’s belief state and the corresponding process model that defines how the cognitive hierarchy
evolves over time is non-trivial. Our formalisation captures
the dual influences of context and behaviour as a predictive
update of a node’s belief state. Consequently, the main contribution of this paper is the inclusion and formalisation of
contextual influences as a predictive update within a cognitive hierarchy.
As a meta-framework, the cognitive hierarchy requires instantiation. We provide two simple instantiation examples
to help illustrate the formalisation of context. The first is a
running example using a small belief network. The second
example involves visual servoing to track a moving object.
This second example quantifies the benefit of context and
demonstrates the role of context in a complete cognitive hierarchy including behaviour generation.
As a third, realistic and challenging example that highlights the importance of context we consider the tracking of
the 6 DoF pose of multiple, possibly occluded, marker-less
objects with a 2D camera. We provide a novel instantiation
of a cognitive hierarchy for a real robot using the context of

a spatial cognitive node modelled using a 3D physics simulator. Note, this formalisation is provided in outline only due
to space restrictions.
Finally, for completeness of our belief network running
example, we prove that general belief propagation in causal
trees (Pearl 1988) can be embedded into our framework, illustrating the versatility of including context in the cognitive
hierarchy. We include this proof as an appendix.

The Architectural Framework
For the sake of brevity the following presentation both summarises and extends the formalisation of cognitive hierarchies as introduced in (Clark et al. 2016). We shall, however, highlight how our contribution differs from their work.
The essence of this framework is to adopt a meta-theoretic
approach, formalising the interaction between abstract cognitive nodes, while making no commitments about the representation and reasoning mechanism within individual nodes.

Motivating Example
As an explanatory aid to formalising the use of context in a
hierarchy we will use the disambiguation of the symbol /−\
in Figure 1 as a simple running example. This system can
be modelled as a two layer causal tree updated according
Pearl’s Bayesian belief propagation rules (Pearl 1988). The
lower-level layer disambiguates individual letters while the
higher-level layer disambiguates complete words (Figure 2).
We assume that there are only two words that are expected
to be seen, with equal probability: “THE” and “CAT”.

Figure 2: Disambiguating the symbol /−\ requires context
from the word recognition layer.
There are three independent letter sensors with the middle
sensor being unable to disambiguate the observed symbol
/−\ represented by the conditional probabilities p(/−\|H) =
0.5 and p(/−\|A) = 0.5. These sensors feed into the lowerlevel nodes (or processors in Pearl’s terminology), which we
label as N 1 , N 2 , N 3 . The results of the lower level nodes are
combined at N 4 to disambiguate the observed word.
Each node maintains two state variables; the diagnostic
and causal supports (displayed as the pairs of values in
Figure 2). Intuitively, the diagnostic support represents the
knowledge gathered through sensing while the causal support represents the contextual bias. A node’s overall belief is
calculated by the combination of these two state variables.
While sensing data propagates up the causal tree, the example highlights how node N 2 is only able to resolve the
symbol /−\ in the presence of contextual feedback from N 4 .

Nodes
A cognitive hierarchy consists of a set of nodes. Nodes are
tasked to achieve a goal or maximise future value. They have
two primary functions: world-modelling and behaviourgeneration. World-modelling involves maintaining a belief
state, while behaviour-generation is achieved through policies, where a policy maps states to sets of actions. A node’s
belief state is modified by sensing or by the combination
of actions and higher-level context. We refer to this latter
as prediction update to highlight how it sets an expectation
about what the node is expecting to observe in the future.
Definition 1. A cognitive language is a tuple L =
(S, A, T , O, C), where S is a set of belief states, A is a set
of actions, T is a set of task parameters, O is a set of observations, and C is a set of contextual elements. A cognitive
node is a tuple N = (L, Π, λ, τ , γ, s0 , π 0 ) s.t:
• L is the cognitive language for N , with initial belief state
s0 ∈ S.
• Π a set of policies such that for all π ∈ Π, π : S → 2A ,
with initial policy π 0 ∈ Π.
• A policy selection function λ : 2T → Π, s.t. λ({}) = π 0 .
• A observation update operator τ : 2O × S → S.
• A prediction update operator γ : 2C × 2A × S → S.
Definition 1 differs from (Clark et al. 2016) in two ways:
the introduction of a set of context elements in the cognitive language, and the modification of the prediction update
operator, previously called the action update operator, to include context elements when updating the belief state.
This definition can now be applied to the motivating example to instantiate the nodes in the Bayesian causal tree.
We highlight only the salient features for this instantiation.
Example. Let E = {hx, yi | 0 ≤ x, y ≤ 1.0} be the set of
probability pairs, representing the recognition between two
distinct features. For node N 2 , say (cf. Figure 2), these features are the letters “H” and “A” and for N 4 these are the
words “THE” and “CAT”. The set of belief states for N 2 is
S 2 = {hhdi, ci | d, c ∈ E}, where d is the diagnostic support
and c is the causal support. Note, the vector-in-vector format allows for structural uniformity across nodes. Assuming equal probability over letters, the initial belief state is
hhh0.5, 0.5ii, h0.5, 0.5ii. For N 4 the set of belief states is
S 4 = hhd1 , d2 , d3 i, ci | d1 , d2 , d3 , c ∈ E}, where di is the
contribution of node N i to the diagnostic support of N 4 .
For N 2 the context is the causal supports from above,
C 2 = E, while the observations capture the influence of
the “H”-“A” sensor, O2 = {hdi | d ∈ E}. In contrast the observations for N 4 need to capture the influence of the different child diagnostic supports, so O4 =
{hd1 , d2 , d3 i | d1 , d2 , d3 ∈ E}.
The observation update operators need to replace the diagnostic supports of the current belief with the observation,
which is more complicated for N 4 due to its multiple chil~ ci) = hΣ3 d~i , ci. Ignoring the
dren, τ 2 ({d~1 , d~2 , d~3 }, hd,
i=1
influence of actions, the prediction update operator simply
replaces the causal support of the current belief with the
~ ci) = hhdi,
~ c0 i.
context from above, so γ 2 ({c0 }, ∅, hhdi,

Cognitive Hierarchy
Nodes are interlinked in a hierarchy, where sensing data is
passed up the abstraction hierarchy, while actions and context are sent down the hierarchy (Figure 3).

1

3

2

1 Sensing function

State
World
Modelling

Actions

Behaviour
Generation

2 Context enrichment function
3 Task parameter function

External world node

Figure 3: A cognitive hierarchy, highlighting internal interactions as well as the sensing, action, and context graphs.
Definition 2. A cognitive hierarchy is a tuple H
(N , N 0 , F ) s.t:

=

• N is a set of cognitive nodes and N 0 ∈ N is a distinguished node corresponding to the external world.
• F is a set of function triples hφi,j , ψ j,i , %j,i i ∈ F that
connect nodes N i , N j ∈ N where:
– φi,j : S i → 2Oj is a sensing function, and
– ψ j,i : 2Aj → 2T i is a task parameter function.
– %j,i : S j → 2C i is a context enrichment function.
• Sensing graph: each φi,j represents an edge from node
N i to N j and forms a directed acyclic graph (DAG) with
N 0 as the unique source node of the graph.
• Prediction graph: the set of task parameter functions
(equivalently, the context enrichment functions) forms a
converse to the sensing graph such that N 0 is the unique
sink node of the graph.
Definition 2 differs from the original with the introduction of the context enrichment functions and the naming of
the prediction graph (originally the action graph). The connection between nodes consists of triples of sensing, task parameter and context functions. The sensing function extracts
observations from a lower-level node in order to update a
higher level node, while the context enrichment function
performs the converse. The task parameter function translates a higher-level node’s actions into a set of task parameters, which is then used to select the active policy for a node.
Finally, the external world is modelled as a distinguished
node, N 0 . Sensing functions allow other nodes to observe
properties of the external world, and task parameter functions allow actuator values to be modified, but N 0 doesn’t
“sense” properties of other nodes, nor does it generate task
parameters for those nodes. Similarly, context enrichment
functions connected to N 0 would simply return the empty
set, unless one wanted to model unusual properties akin to
the quantum effects of observations on the external world.

Beyond this, the internal behaviour of N 0 is considered to
be opaque.
The running example can now be encoded formally as a
cognitive hierarchy, again with the following showing only
the salient features of the encoding.
Example. We construct a hierarchy H = (N , N 0 , F ), with
N = {N 0 , N 1 , . . . , N 4 }. The function triples in F will include φ0,2 for the visual sensing of the middle letter, and φ2,4
and %4,2 for the sensing and context between N 2 and N 4 .
The function φ0,2 returns the probability of the input being
the characters “H” and “A”. Here φ0,2 (/−\) = {h0.5, 0.5i}.
Defining φ2,4 and %4,2 requires a conditional probability


1 0
matrix M =
to capture how the letters “H” and
0 1
“A” contribute to the recognition of “THE” and “CAT”.
For sensing from N 2 we use zeroed vectors to prevent
influence from the diagnostic support components from N 1
and N 2 . Hence φ2,4 (hhdi, ci) = {hh0, 0i,η · M · dT , h0, 0ii},
where dT is the transpose of vector d, and η is a normalisation constant.
For context we capture how N 4 ’s causal support and its
diagnostic support components from N 1 and N 2 influences
the causal support of N 2 . Note, that this also prevents any
feedback from N 2 ’s own diagnostic support to its causal
support. So, %4,2 (hhd1 , d2 , d3 i, ci) = {η · (d1 · d3 · c) · M }.

Active Cognitive Hierarchy
The above definitions capture the static aspects of a system
but require additional details to model its operational behaviour. Note, the following definitions are unmodified from
the original formalism and are presented here because they
are necessary to the developments of later sections.
Definition 3. An active cognitive node is a tuple Q =
(N , s, π, a) where: 1) N is a cognitive node with S, Π, and
A being its set of belief states, set of policies, and set of actions respectively, 2) s ∈ S is the current belief state, π ∈ Π
is the current policy, and a ∈ 2A is the current set of actions.
Essentially an active cognitive node couples a (static) cognitive node with some dynamic information; in particular the
current belief state, policy and set of actions.
Definition 4. An active cognitive hierarchy is a tuple X =
(H, Q) where H is a cognitive hierarchy with set of cognitive nodes N such that for each N ∈ N there is a corresponding active cognitive node Q = (N , s, π, a) ∈ Q and
vice-versa.
The active cognitive hierarchy captures the dynamic state
of the system at a particular instance in time. Finally, an initial active cognitive hierarchy is an active hierarchy where
each node is initialised with the initial belief state and policy
of the corresponding cognitive node, as well as an empty set
of actions.

Cognitive Process Model
The process model defines how an active cognitive hierarchy
evolves over time and consists of two steps. Firstly, sensing

observations are passed up the hierarchy, progressively updating the belief state of each node. Next, task parameters
and context are passed down the hierarchy updating the active policy, the actions, and the belief state of the nodes.
We do not present all definitions here, in particular we
omit the definition of the sensing update operator as this remains unchanged in our extension. Instead we define a prediction update operator, replacing the original action update,
with the new operator incorporating both context and task
parameters in its update. First, we characterise the updating
of the beliefs and actions for a single active cognitive node.
Definition 5. Let X = (H, Q) be an active cognitive hierarchy with H = (N , N 0 , F ). The prediction update of X with
respect to an active cognitive node Qi = (N i , si , π i , ai ) ∈
Q, written as PredUpdate0 (X , Qi ) is an active cognitive hierarchy X 0 = (H, Q0 ) where Q0 = Q \ {Qi } ∪ {Q0i } and
Q0i = (N i , γ i (C, a0i , si ), π 0i , a0i ) s.t:
• if there is no node N x where hφi,x , ψ x,i , %x,i i ∈ F then:
π 0i = π i , a0i = π i (si ) and C = ∅,
• else:
π 0i = λSi (T ) and a0i = π 0i (si ),
T =
{ψ x,i (ax ) | hφi,x , ψ x,i , %x,i i ∈ F where
S Qx = (N x , sx , π x , ax ) ∈ Q}
C =
{%x,i (sx ) | hφi,x , ψ x,i , %x,i i ∈ F where
Qx = (N x , sx , π x , ax ) ∈ Q}
The intuition for Definition 5 is straightforward. Given
a cognitive hierarchy and a node to be updated, the update
process returns an identical hierarchy except for the updated
node. This node is updated by first selecting a new active
policy based on the task parameters of all the connected
higher-level nodes. The new active policy is applied to the
existing belief state to generate a new set of actions. Both
these actions and the context from the connected higherlevel nodes are then used to update the node’s belief state.
Using the single node update, updating the entire hierarchy simply involves successively updating all its nodes.
Definition 6. Let X = (H, Q) be an active cognitive hierarchy with H = (N , N 0 , F ) and Ψ be the prediction graph
induced by the task parameter functions in F . The action
process update of X , written PredUpdate(X ), is an active
cognitive model:
X 0 = PredUpdate0 (. . . PredUpdate0 (X , Qn ), . . . Q0 )
where the sequence [Qn , . . . , Q0 ] consists of all active cognitive nodes of the set Q such that the sequence satisfies the
partial ordering induced by the prediction graph Ψ.
Importantly, the update ordering in Definition 6 satisfies
the partial ordering induced by the prediction graph, thus
guaranteeing that the prediction update is well-defined.
Lemma 1. For any active cognitive hierarchy X the prediction process update of X is well-defined.
Proof. Follows from the DAG.
The final part of the process model, which we omit here,
is the combined operator, Update, that first performs a sensing update followed by a prediction update. This operation

follows exactly the original and similarly the theorem that
the process model is well-defined also follows.
We can now apply the update process (sensing then prediction) to show how it operates on the running example.
Example. When N 2 senses the symbol /−\, φ0,2 returns that
“A” and “H” are equally likely, so τ 2 updates the diagnostic support of N 2 to hh0.5, 0.5ii. On the other hand N 1
and N 2 unambiguously sense “C” and “T” respectively, so
N 4 ’s observation update operator, τ 4 , will update its diagnostic support components to hh0, 1i, h0.5, 0.5i, h0, 1ii. The
nodes overall belief, h0, 1i, is the normalised product of the
diagnostic support components and the causal support, indicating here the unambiguous recognition of “CAT”.
Next, during prediction update, context from N 4 is passed
back down to N 2 , through φ4,2 and γ 2 , updating the causal
support of N 2 to h0, 1i. Hence, N 2 is left with the belief
state hhh0.5, 0.5ii, h0, 1ii, which when combined, indicates
that the symbol /−\ should be interpreted as an “A”.
We next appeal to another simple example to illustrate the
use of context to improve world modelling and in turn behaviour generation in a cognitive hierarchy.

A Simple Visual Servoing Example
Consider a mobile camera tasked to track an object sliding
down a frictionless inclined plane. The controller is constructed as a three-node cognitive hierarchy. Figure 4 depicts
the cognitive hierarchy and the scene.

Figure 4: A three-node cognitive hierarchy controller tasked
to visually follow an object. Context flow is shown in red.
The performance of the controller will be determined by
how well the camera keeps the object in the centre of its
field-of-view, specifically the average error in the tracking
distance over a time period of 3 seconds.
The details of the instantiation of the cognitive hierarchy controller follow. The cognitive hierarchy is H =
(N , N 0 , F ) with N = {N 0 , N 1 , N 2 }. N 0 is the unique
opaque node representing the environment. The cognitive
language for N 1 is a tuple L1 = (S 1 , A1 , T 1 , O1 , C 1 ),
and for N 2 it is L2 =(S 2 , A2 , T 2 , O2 , C 2 ). The cognitive
nodes are N 1 = (L1 , Π1 , λ1 , τ 1 , γ 1 , s01 , π 01 ) and N 2 =
(L2 , Π2 , λ2 , τ 2 , γ 2 , s02 , π 02 ). For brevity we only describe the
material functions.
The belief state of N 1 is the position of the object: S 1 =
{x | x ∈ R}. The belief state of N 2 is both the position and
velocity of the object: S 2 = {hx, vi | x, v ∈ R}. The object

starts at rest on the inclined plane at the origin: s01 = 0.0 and
s02 = h0.0, 0.0i.
N 1 receives object position observations from the environment: O1 = {x | x ∈ R}. These measurements are simulated from the physical properties of the scene and include a
noise component to represent errors in the sensor measurements: φ0,1 (·) = {0.5kt2 + ν}, with constant acceleration
k = 8.49 m/s2 , t the elapsed time and ν zero mean Gaussian random noise with a standard deviation of 0.1. The acceleration assumes an inclined plane of 60 degrees in a 9.8
m/s2 gravitational field. The N 1 observation update operator implements a Kalman filter with a fixed gain of 0.25:
τ 1 (h{x}, yi) = (1.0 − 0.25)y + 0.25x.
N 2 receives observations O2 = {x | x ∈ R} from N 1 :
φ1,2 (x) = {x}. In turn it updates its position estimate accepting the value from N 1 : τ 2 (h{x}, hy, vii) = hx, vi. The
prediction update operator uses a physics model to estimate
the new position and velocity of the object after time-step
δt = 0.05 seconds: γ 2 (h{}, {}, hx, vii) = hx + vδt +
0.5kδt2 , v + kδti with known acceleration k = 8.49.
Both N 1 and N 2 have one policy function each. The N 2
policy selects the N 1 policy. The effect of the N 1 policy:
π1 (x) = {x}, is to move the camera to the estimated position of the object via the task parameter function connecting
the environment: ψ 1,0 ({x}) = {x}.
We consider two versions of the N 1 prediction update operator. Without context the next state is the commanded policy action: γ 1 (h{x}, {y}, zi) = y. With context the context enrichment function passes the N 2 estimate
of the position of the object to N 1 : %2,1 (hx, vi) = {x},
where C 1 = {x | x ∈ R}. The update operator becomes:
γ 1 (h{x}, {y}, zi) = x.
When we simulate the dynamics and the repeated update of the cognitive hierarchy at 1/δt Hertz for 3 seconds,
we find that without context the average tracking error is
2.004 ± 0.009. Using context the average tracking error reduces to 0.125 ± 0.015—a 94% error reduction.2

However, when multiple objects are tracked, independent CAD models fail to handle object occlusion. In place
of the CAD models we use the machinery provided by a
3D physics simulator. The object-scene and virtual cameras
from a simulator are ideal to model the higher level context
for vision. We now describe how this approach is instantiated as a cognitive hierarchy with contextual feedback. It is
important to note that the use of the physics simulator is not
to replace the real-world, but is used as mental imagery efficiently representing the spatial belief state of the robot.

Cognitive Hierarchy for Visual Tracking
We focus on world-modelling in a two-node cognitive hierarchy (Figure 5). The external world node that includes the
Baxter robot, streams the camera pose and RGB images as
sensory input to the arm node. The arm node belief state
s = {pa } ∪ {hpia , ci i|object i}, where pa is the arm pose,
and for all recognised objects i in the field of view of the
arm camera, pia is the object pose relative to the arm camera,
and ci is the set of object edge lines and their depth. The
objects in this case include scattered cubes on a table. Information from the arm node is sent to the spatial node that
employs a Gazebo physics simulator as mental imagery to
model the objects.

Using Context to Track Objects Visually
Object tracking has application in augmented reality, visual servoing, and man-machine interfaces. We consider the
problem of on-line monocular model-based tracking of multiple objects without markers or texture, using the 2D RGB
camera built into the hand of a Baxter robot. The use of natural object features makes this a challenging problem.
Current practice for tackling this problem is to use 3D
knowledge in the form of a CAD model, from which to generate a set of edge points (control points) for the object (Lepetit and Fua 2005) . The idea is to track the corresponding 2D
camera image points of the visible 3D control points as the
object moves relatively to the camera. The new pose of the
object relative to the camera is found by minimising the perspective re-projection error between the control points and
their corresponding 2D image.
2
It is of course intuitive in this simple example that as N 2 has
the benefit of the knowledge of the transition dynamics of the object it can better estimate its position and provide this context to
direct the camera.

Figure 5: Cognitive hierarchy comprising an arm node and a
spatial node. Context from the spatial node is in the form of
an object segmented depth image from a simulated special
camera that shadows the real camera.
A novel feature of the spatial node is that it simulates the
robot’s arm camera as an object aware depth camera. No
such camera exist reality, but the Gazebo spatial belief state
of the robot is able to not only provide a depth image, but
one that segments the depth image by object. This object
aware depth image provides the context to the arm node to
generate the required control points.

Update Functions and Process Update
We now describe the update functions and a single cycle of
the process update for this cognitive hierarchy.
The real monocular RGB arm camera is simulated in
Gazebo with an object aware depth camera with identical

characteristics (i.e. the same intrinsic camera matrix). The
simulated camera then produces depth and an object segmentation images from the simulated objects that corresponds to the actual camera image. This vital contextual information is then used for correcting the pose of the visible
objects.
The process update starts with the sensing function
φN0 ,Arm that takes the raw camera image and observes all
edges in the image, represented as a set of line segments, l.
φN0 ,Arm ({rawImage}) = {l}
The observation update operator τ Arm takes the expected
edge lines ci for each object i and transforms the lines to
best match the image edge lines l (Lepetit and Fua 2005).
The update function uses the OpenCV function solvePnP to
find a corrected pose pia for each object i relative to the armcamera a 3 .
τ Arm ({l, ci |object i}) = {pia |object i}
The sensing function from the arm to spatial node takes
the corrected pose pia for each object i, relative to the camera
frame a, and transforms it into the Gazebo reference frame
via the Baxter’s reference frame given the camera pose pa .
φArm,Spatial ({pa , hpia , ci i|object i}) = {gai |object i}
The spatial node observation update τ Spatial , updates the
pose of all viewed objects gai in the Gazebo physics simulator. Note {gai |object i} ⊂ gazebo state.
τ Spatial ({gai |object i}) = gazebo.move(i, gai ) ∀i
The update cycle now proceeds down the hierarchy with
prediction updates. The prediction update for the spatial
node γ Spatial consists of predicting the interaction of objects in the simulator under gravity. Noise introduced during
the observation update may result in objects separating due
to detected collisions or settling under gravity.
γ Spatial (gazebo state) = gazebo.simulate(gazebo state))
We now turn to the context enrichment function
%Spatial,Arm that extracts predicted camera image edge lines
and depth data for each object in view of the simulator.
%Spatial,Arm (gazebo state) = {ci |object i}
The stages of the context enrichment function %Spatial,Arm
are shown in Figure 6. The simulated depth camera extracts
an object image that identifies the object seen at every pixel
location. It also extracts a depth image that gives the depth
from the camera of every pixel. The object image is used to
mask out each object in turn. Applying a Laplacian function
to the part of the depth image masked out by the object yields
all visible edges of the object. A Hough line transform identifies line end points in the Laplacian image and finds the
depth of their endpoints from the depth image, producing ci .
Figure 7 shows the cognitive hierarchy tracking several
different cube configurations. This is only possible given the
context from the spatial belief state. Keeping track of the
pose of objects allows behaviours to be generated that for
example pick up a cube with appropriately oriented grippers.
3

The pose of a rigid object in 3D space has 6 degrees of freedom, three describing its translated position, and three the rotation
or orientation, relative to a reference pose.

Figure 6: The Process update showing stages of the context
enrichment function and the matching of contextual information to the real camera to correct the arm and spatial node
belief state.

Figure 7: Tracking several cube configurations. Top row:
Gazebo GUI showing spatial node state. 2nd row: matching
real image edges in green to simulated image edges in red.
Bottom row: camera image overlaid with edges in green.

Related Support and Conclusion
There is considerable evidence supporting the existence and
usefulness of top-down contextual information. Reliability
(Biederman, Kubovy, and Pomerantz 1981) and speed (Cavanagh 1991) of scene analysis provides early evidence.
These observations are further supported by neuroscience,
suggesting that feedback pathways from higher more abstract processing areas of the brain down to areas closer to
the sensors are greater than those transmitting information
upwards (Hawkins and Blakeslee 2004). The authors summarise the process - “what is actually happening flows up,
and what you expect to happen flows down”. Gilbert and
Li (2013) argue that the traditional idea that the processing
of visual information consists of a sequence of feedforward
operations needs to be supplemented by top-down contextual influences.
In the field of robotics, recent work in online interac-

tive perception shows the benefit of predicted measurements
from one level being passed to the next-lower level as state
predictions (Martin and Brock 2014).
This paper has included and formalised the essential element of context in the meta framework of cognitive hierarchies. The process model of an active cognitive hierarchy
has been revised to include context updates satisfying the
partial order induced by the prediction graph. We have illustrated the role of context with two simple examples and a
novel way to track the pose of texture-less objects with a single 2D camera. As a by-product contribution we prove that
general belief propagation in causal trees can be embedded
into our framework testifying to its representation versatility.

Appendices
Causal Networks as Cognitive Hierarchies
The motivating example highlighted the use of context in a
cognitive hierarchy inspired by belief propagation in causal
trees. In this appendix we extend the example to the general
result that any Bayesian causal tree can be encoded as a cognitive hierarchy. We do this by constructively showing how
to encode a causal tree as a cognitive hierarchy and proving
the correctness of this method with respect to propagating
changes through the tree.
Pearl describes a causal tree as a set of processors where
the connection between processors is explicitly represented
within the processors themselves. Each processor maintains
diagnostic and causal supports, as well as maintaining a
conditional probability matrix for translating to the representation of higher-level processors. The description of the
operational behaviour of causal trees is presented throughout
Chapter 4 (and summarised in Figure 4.15) of Pearl (1988).
The cognitive hierarchies introduced here are concerned
with robotic systems and consequently maintain an explicit
notion of sensing over time. In contrast causal networks are
less precise about external inputs and changes over time. As
a bridge, we model that each processor has a diagnostic support component that can be set externally. Finally, note that
we adopt the convenience notation f∅ to represent a function
of arbitrary arity that always returns the empty set.
Definition 7. Let {P 1 , . . . P n } be a causal tree. We
construct a corresponding cognitive hierarchy H =
({N 0 , N 1 , . . . , N n }, N 0 , F ) as follows:
• For processor P i with m children, and diagnostic and causal supports d, c ∈ Rn , define S i =
{hhdE , d1 , . . . , dm i, c0 i|dE , d1 , . . . , dm , c0 ∈ Rn }, with
initial belief state si = hhd, . . . , di, ci. Define Oi =
{hdE , d1 , . . . , dm i|dE , d1 , . . . , dm ∈ Rn } and C i = Rn .
• For processor P i with corresponding cognitive
~ ci) = hΣ ~0 d~0 , ci, and
node N i , define τ i (o, hd,
d ∈o
~ ci) = hd,
~ c0 i.
γ i ({c0 }, ∅, hd,
• For each pair of processors P i and P j , where P j is the
k-th child of P i ’s m children (from processor subscript

numbering), and Mj is the conditional probability matrix
of P j , then define a triple hφj,i , f∅ , %i,j i ∈ F s.t:
~ ci) = {hdE , d1 , . . . , dm i}, where dh6=k are ze– φj,i (hd,
Q
roed vectors and dk = η · Mj · ( d0 ∈d~ d0 )T .
0
0
– %Q
i,j (hhdE , d1 , . . . , dm i, ci) = {c }, such that c = η ·
( h6=k dh · c) · Mj .
– where η is a normalisation constant for the respective
vectors, and xT is the transpose of vector x.
• For processors P i , with diagnostic support d ∈ Rn , define a triple hφ0,i , f∅ , f∅ i ∈ F where φ0,i (s0 ∈ S 0 ) =
{hdE , dZ , . . . , dZ i}, where dZ is a zeroed vector and
dE ∈ Rn is the external input of P i .
While notationally dense, Definition 7 is simply a generalisation of the construction used in the running example
and is a direct encoding of causal trees. Note, this construction could be further extended to poly-trees, which Pearl also
considers, but would require a more complex encoding.
To establish the correctness of this transformation we can
compare how the structures evolve with sensing. The belief
measure of a processor is captured as the normalised product
of the diagnostic and causal supports, BEL(Pi ) = η · di · ci .
However, for a cognitive node the diagnostic support needs
to be computed from its components. Hence, given the belief state hhdE , d1, . . . , dm i, ci of an active node Qi with
mQ
children, we can compute the belief as BEL(Qi ) =
m
η · j=1 dj · c.
Lemma 2. Given a causal tree {P 1 , . . . P n } and a corresponding cognitive hierarchy H constructed via Definition 7, then the causal tree and the initial active cognitive
hierarchy corresponding to H share the same belief.
Proof. By inspection, BEL(P i ) = BEL(Qi ) for each
i.
Now, we establish that propagating changes through an
active cognitive hierarchy is consistent with propagating beliefs through a causal tree. We abuse notation here to express
the overall belief of a casual tree (resp. active cognitive hierarchy) as simply the beliefs of its processors (resp. nodes).
Theorem 3. Let T be a causal tree and X be the corresponding active cognitive hierarchy constructed via Definition 7, such that BEL(T ) = BEL(X ). Then for any changes
to the external diagnostic supports of the processors and
corresponding changes to the sensing inputs for the active
cognitive hierarchy, BEL(Prop(T )) = BEL(Update(X )).
Proof. Pearl establishes that changes propagated through a
causal tree converge with a single pass up and down the tree.
Any such pass satisfies the partial ordering for the cognitive
hierarchy process model. Hence the proof involves the iterative application of the process model, confirming at each
step that the beliefs of the processors and nodes align.
Theorem 3 establishes that Bayesian causal trees can be
captured as cognitive hierarchies. This highlights the significance of extending cognitive hierarchies to include context,
allowing for a richer set of potential applications.

Acknowledgments
This material is based upon work supported by the
Asian Office of Aerospace Research and Development
(AOARD) under Award No: FA2386-15-1-0005. This research was also supported under Australian Research Council’s (ARC) Discovery Projects funding scheme (project
number DP 150103035). Michael Thielscher is also affiliated with the University of Western Sydney.

Disclaimer
Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors
and do not necessarily reflect the views of the AOARD.

References
[Albus and Meystel 2001] Albus, J. S., and Meystel, A. M.
2001. Engineering of Mind: An Introduction to the Science
of Intelligent Systems. Wiley-Interscience.
[Ashby 1952] Ashby, W. R. 1952. Design for a Brain. Chapman and Hall.
[Bakker and Schmidhuber 2004] Bakker, B., and Schmidhuber, J. 2004. Hierarchical reinforcement learning based on
subgoal discovery and subpolicy specialization. In Proceedings of the 8-th Conference on Intelligent Autonomous Systems, IAS-8, 438–445.
[Beer 1966] Beer, S. 1966. Decision and Control. London:
John Wiley and Sons.
[Biederman, Kubovy, and Pomerantz 1981] Biederman, I.;
Kubovy, M.; and Pomerantz, J. 1981. On the semantics of a
glance at a scene. In Perceptual Organization. New Jersey:
Lawrence Erlbaum. 213–263.
[Brooks 1986] Brooks, R. A. 1986. A robust layered control
system for a mobile robot. Robotics and Automation, IEEE
Journal of 2(1):14–23.
[Cavanagh 1991] Cavanagh, P. 1991. What’s up in top-down
processing? In Gorea, A., ed., Representations of Vision:
Trends and tacit assumptions in vision research, 295–304.
[Clark et al. 2016] Clark, K.; Hengst, B.; Pagnucco, M.; Rajaratnam, D.; Robinson, P.; Sammut, C.; and Thielscher,
M. 2016. A framework for integrating symbolic and subsymbolic representations. In 25th Joint Conference on Artificial Intelligence (IJCAI -16).
[Dayan and Hinton 1992] Dayan, P., and Hinton, G. E. 1992.
Feudal reinforcement learning. Advances in Neural Information Processing Systems 5 (NIPS).
[Dietterich 2000] Dietterich, T. G. 2000. Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research (JAIR)
13:227–303.
[Drescher 1991] Drescher, G. L. 1991. Made-up Minds:
A constructionist Approach to Artificial Intelligence. Cambridge, Massachusetts: MIT Press.
[Gilbert and Li 2013] Gilbert, C. D., and Li, W. 2013. Topdown influences on visual processing. Nature reviews. Neuroscience 14(5):10.1038/nrn3476.

[Hawkins and Blakeslee 2004] Hawkins, J., and Blakeslee,
S. 2004. On Intelligence. Times Books, Henry Holt and
Company.
[Hubel and Wiesel 1979] Hubel, D. H., and Wiesel, T. N.
1979. Brain mechanisms of vision. A Scientific American
Book: the Brain 84–96.
[Johnson 2010] Johnson, J. 2010. Designing with the Mind
in Mind: Simple Guide to Understanding User Interface Design Rules. San Francisco, CA, USA: Morgan Kaufmann
Publishers Inc.
[Jong 2010] Jong, N. K. 2010. Structured Exploration for
Reinforcement Learning. Ph.D. Dissertation, University of
Texas at Austin.
[Kaelbling 1993] Kaelbling, L. P. 1993. Hierarchical learning in stochastic domains: Preliminary results. In Machine
Learning Proceedings of the Tenth International Conference, 167–173. San Mateo, CA: Morgan Kaufmann.
[Konidaris et al. 2011] Konidaris, G.; Kuindersma, S.; Grupen, R.; and Barto, A. 2011. Robot learning from demonstration by constructing skill trees. The International Journal of Robotics Research.
[Lepetit and Fua 2005] Lepetit, V., and Fua, P. 2005. Monocular model-based 3d tracking of rigid objects. Found.
Trends. Comput. Graph. Vis. 1(1):1–89.
[Marthi, Russell, and Andre 2006] Marthi, B.; Russell, S.;
and Andre, D. 2006. A compact, hierarchical q-function
decomposition. In Proceedings of the Proceedings of the
Twenty-Second Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-06), 332–340. Arlington, Virginia: AUAI Press.
[Martin and Brock 2014] Martin, R. M., and Brock, O. 2014.
Online interactive perception of articulated objects with
multi-level recursive estimation based on task-specific priors. In IROS, 2494–2501. IEEE.
[Minsky 1986] Minsky, M. 1986. The Society of Mind. New
York, NY, USA: Simon & Schuster, Inc.
[Nilsson 2001] Nilsson, N. J. 2001. Teleo-Reactive programs and the triple-tower architecture. Electronic Transactions on Artificial Intelligence 5:99–110.
[Pearl 1988] Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San
Francesco: Morgan Kaufmann, revised second printing edition.
[Turchin 1977] Turchin, V. F. 1977. The Phenomenon of
Science. Columbia University Press.

