arXiv:1706.07136v2 [stat.ME] 18 Aug 2017

Multiscale Information Decomposition: Exact Computation
for Multivariate Gaussian Processes
L. Faes1 , S. Stramaglia2 , and D. Marinazzo3
1

1
2

Bruno Kessler Foundation, Trento, Italy
BIOtech, Dept. of Industrial Engineering, University of Trento, Italy
Dipartimento di Fisica, Universitá degli Studi Aldo Moro, Bari, Italy
2
INFN, Sezione di Bari, Italy
3
Data Analysis Department, Ghent University, Ghent, Belgium
August 21, 2017
Abstract

Exploiting the theory of state space models, we derive the exact expressions of the information transfer, as well as redundant and synergistic transfer, for coupled Gaussian processes
observed at multiple temporal scales. All the terms, constituting the frameworks known as
interaction information decomposition and partial information decomposition, can thus
be analytically obtained for different time scales from the parameters of the VAR model that
fits the processes. We report the application of the proposed methodology firstly to benchmark
Gaussian systems, showing that this class of systems may generate patterns of information decomposition characterized by prevalently redundant or synergistic information transfer persisting across multiple time scales, or even by alternating prevalence of redundant and synergistic
source interaction depending on the time scale. Then, we apply our method to an important
topic in neuroscience, i.e. the detection of causal interactions in human epilepsy networks, for
which we show the relevance of partial information decomposition to the detection of multiscale information transfer spreading from the seizure onset zone.

1

Introduction

The information-theoretic treatment of groups of correlated degrees of freedom can reveal their
functional roles as memory structures or information processing units. A large body of recent work
has shown how the general concept of “information processing” in a network of multiple interacting dynamical systems described by multivariate stochastic processes can be dissected into basic
1

elements of computation defined within the so-called framework of information dynamics Lizier
et al. (2014). These elements essentially reflect the new information produced at each moment
in time about a target system in the network Pincus (1995), the information stored in the target
system Lizier et al. (2012), Wibral et al. (2014), the information transferred to it from the other
connected systems Schreiber (2000), Wibral et al. (2014) and the modification of the information
flowing from multiple source systems to the target Lizier et al. (2010), Wibral et al. (2015). The
measures of information dynamics have gained more and more importance in both theoretical and
applicative studies in several fields of science Lizier et al. (2011), Wibral et al. (2011), Hlinka
et al. (2013), Barnett et al. (2013), Marinazzo et al. (2014), Faes et al. (2014, 2015), Porta et al.
(2015), Faes et al. (2017), Wollstadt et al. (2017). While the information-theoretic approaches to
the definition and quantification of new information, information storage and information transfer
are well understood and widely accepted, the problem of defining, interpreting and using measures
of information modification has not been fully addressed in the literature.
Information modification in a network is tightly related to the concepts of redundancy and synergy between source systems sharing information about a target system, which refer to the existence
of common information about the target that can be retrieved when the sources are used separately
(redundancy) or when they are used jointly (synergy) Schneidman et al. (2003). Classical multivariate entropy-based approaches refer to the interaction information decomposition (IID), which
reflects information modification through the balance between redundant and synergetic interaction among different source systems influencing the target Stramaglia et al. (2012, 2014, 2016).
The IID framework has the drawback that it implicitly considers redundancy and synergy as mutually exclusive concepts, because it quantifies information modification with a single measure of
interaction information McGill (1954) (also called co-information Bell (2003)) that takes positive
or negative values depending on whether the net interaction between the sources is synergistic or
redundant. This limitation has been overcome by the elegant mathematical framework introduced
by Williams and Beer Williams and Beer (2010), who proposed the so-called partial information
decomposition (PID) as a nonnegative decomposition of the information shared between a target
and a set of sources into terms quantifying separately unique, redundant and synergistic contributions. However, the PID framework has the drawback that the terms composing the PID cannot
be obtained unequivocally from classic measures of information theory (i.e., entropy and mutual
information), but a new definition of either redundant, synergistic or unique information needs to
be provided to implement the decomposition. Accordingly, much effort has focused on finding
the most proper measures to define the components of the PID, with alternative proposals defining new measures of redundancy Williams and Beer (2010), Harder et al. (2013), synergy Griffith
et al. (2014), Quax et al. (2017) or unique information Bertschinger et al. (2014). The proliferation
of different definitions is mainly due to the fact that there is no full consensus on which axioms
should be stated to impose desirable properties for the PID measures. An additional problem which
2

so far has seriously limited the practical implementation of these concepts is the difficulty in providing reliable estimates of the information measures appearing in the IID and PID decompositions.
The naive estimation of probabilities by histogram-based methods followed by the use of plug-in
estimators leads to serious bias problems Panzeri et al. (2007), Faes and Porta (2014). While the
use of binless density estimators Kozachenko and Leonenko (1987) and the adoption of schemes
for dimensionality reduction Vlachos and Kugiumtzis (2010), Marinazzo et al. (2012) have been
shown to improve the reliability of estimates of information storage and transfer Faes et al. (2015),
the effectiveness of these approaches for the computation of measures of information modification
has not been demonstrated yet. Interestingly, both the problems of defining appropriate PID measures and of reliably estimating these measures from data are much alleviated if one assumes that
the observed variables have a joint Gaussian distribution. Indeed, in such a case, recent studies
have proven the equivalence between most of the proposed redundancy measures to be used in the
PID Barrett (2015) and have provided closed form solutions to the issue of computing any measure
of information dynamics from the parameters of the vector autoregressive (VAR) model that characterizes an observed multivariate Gaussian process Faes et al. (2017), Barrett et al. (2010), Porta
et al. (2017).
The second fundamental question that is addressed in this study is relevant to the computation
of information dynamics for stochastic processes displaying multiscale dynamical structures. It is
indeed well known that many complex physical and biological systems exhibit peculiar oscillatory
activities, which are deployed across multiple temporal scales Ivanov et al. (1999), Chou (2011),
Wang et al. (2013). The most common way to investigate such activities is to resample at different scales, typically through low pass filtering and downsampling Costa et al. (2002), Valencia
et al. (2009), the originally measured realization of an observed process, so as to yield a set of
rescaled time series, which are then analyzed employing different dynamical measures. This approach is well established and widely used for the multiscale entropy analysis of individual time
series measured from scalar stochastic processes. However, its extension to the investigation of the
multiscale structure of the information transfer among coupled processes is complicated by theoretical and practical issues Barnett and Seth (2015), Solo (2016). Theoretically, the procedure of
rescaling alters the causal interactions between lagged components of the processes in a way that
is not fully understood and, if not properly performed, may alter the temporal relations between
processes and thus induce spurious detection of information transfer. In practical analysis, filtering
and downsampling are known to degrade severely the estimation of information dynamics and to
impact consistently the detectability, accuracy and data demand Florin et al. (2010), Barnett and
Seth (2017).
In recent works, we have started tackling the above problems within the framework of linear
VAR modeling of multivariate Gaussian processes, with the focus on the multiscale computation
of information storage and information transfer Faes et al. (2016, 2017). In this study, we aim at
3

extending these recent theoretical advances to the multiscale analysis of information modification
in multivariate Gaussian systems performed through the IID and PID decomposition frameworks.
To this end, we exploit the theory of state space (SS) models Aoki and Havenner (1991) and build
on recent theoretical results Barnett and Seth (2015), Solo (2016) to show that exact values of interaction transfer, as well as redundant and synergistic transfer can be obtained for coupled Gaussian
processes observed at different time scales starting from the parameters of the VAR model that
fits the processes and from the scale factor. The theoretical derivations are first used in examples
of benchmark Gaussian systems, reporting that these systems may generate patterns of information
decomposition characterized by prevalently redundant or synergistic information transfer persisting across multiple time scales or even by alternating the prevalence of redundant and synergistic
source interaction depending on the time scale. The high computational reliability of the SS approach is then exploited in the analysis of real data by the application to a topic of great interest in
neuroscience, i.e., the detection of information transfer in epilepsy networks.
The proposed framework is implemented in the msID MATLAB R toolbox, which is uploaded
as Supplementary Material to this article and is freely available for download from www.lucafaes.
net/msID.html and https://github.com/danielemarinazzo/multiscale_PID

2

Information Transfer Decomposition in Multivariate Processes

Let us consider a discrete-time, stationary vector stochastic process composed of M real-valued
zero-mean scalar processes, Yn = [Y1,n · · · YM,n ]T , −∞ < n < ∞. In an information-theoretic
framework, the information transfer between scalar sub-processes is quantified by the well-known
transfer entropy (TE), which is a popular measure of the “information transfer” directed towards
an assigned target process from one or more source processes. Specifically, the TE quantifies the
amount of information that the past of the source provides about the present of the target over and
above the information already provided by the past of the target itself Schreiber (2000). Taking Yj
as target and Yi as source, the TE is defined as:
−
−
Ti→j = I(Yj,n ; Yi,n
|Yj,n
)

(1)

−
−
where Yi,n
= [Yi,n−1 Yi,n−2 · · · ] and Yj,n
= [Yj,n−1 Yj,n−2 · · · ] represent the past of the source and
target processes and I(·; ·|·) denotes conditional mutual information (MI). In the presence of two
sources Yi and Yk and a target Yj , the information transferred toward Yj from the sources Yi and
Yk taken together is quantified by the joint TE:
−
−
−
Tik→j = I(Yj,n ; Yi,n
, Yk,n
|Yj,n
).

4

(2)

Under the premise that the information jointly transferred to the target by the two sources is
different than the sum of the amounts of information transferred individually, in the following, we
present two possible strategies to decompose the joint TE into amounts eliciting the individual TEs,
as well as redundant and/or synergistic TE terms.

2.1

Interaction Information Decomposition

The first strategy, which we denote as interaction information decomposition (IID), decomposes
the joint TE (2) as:
Tik→j = Ti→j + Tk→j + Iik→j ,
(3)
where Iik→j is denoted as interaction transfer entropy (ITE) because it is equivalent to the interaction information McGill (1954) computed between the present of the target and the past of the two
sources, conditioned to the past of the target:
−
−
−
|Yj,n
).
Iik→j = I(Yj,n ; Yi,n
; Yk,n

(4)

The interaction TE quantifies the modification of the information transferred from the source
processes Yi and Yk to the target Yj , being positive when Yi and Yk cooperate in a synergistic
way and negative when they act redundantly. This interpretation is evident from the diagrams
of Figure 1: in the case of synergy (Figure 1a), the two sources Yi and Yk taken together contribute
to the target Yj with more information than the sum of their individual contributions (Tik→j >
Ti→j + Tk→j ), and the ITE is positive; in the case of redundancy (Figure 1b), the sum of the
information amounts transferred individually from each source to the target is higher than the joint
information transfer (Ti→j + Tk→j > Tik→j ), so that the ITE is negative.

5

Figure 1: Venn diagram representations of the interaction information decomposition (IID)
(a,b)) and the partial information decomposition (PID) (c)). The IID is depicted in a way
such that all areas in the diagrams are positive: the interaction information transfer Iik→j
is positive in (a), denoting net synergy, and is negative in (b), denoting net redundancy.

2.2

Partial Information Decomposition

An alternative expansion of the joint TE is that provided by the so-called partial information decomposition (PID) Williams and Beer (2010). The PID evidences four distinct quantities measuring the unique information transferred from each individual source to the target, measured by the
unique TEs Ui→j and Uk→j , and the redundant and synergistic information transferred from the
two sources to the target, measured by the redundant TE Rik→j and the synergistic TE Sik→j .
These four measures are related to each other and to the joint and individual TEs by the following
equations (see also Figure 1c):

Tik→j = Ui→j + Uk→j + Rik→j + Sik→j ,

(5a)

Ti→j = Ui→j + Rik→j ,

(5b)

Tk→j = Uk→j + Rik→j .

(5c)

In the PID defined above, the terms Ui→j and Uk→j quantify the parts of the information transferred to the target process Yj , which are unique to the source processes Yi and Yk , respectively,
thus reflecting contributions to the predictability of the target that can be obtained from one of the
sources alone, but not from the other source alone. Each of these unique contributions sums up
6

with the redundant transfer Rik→j to yield the information transfer from one source to the target as
is known from the classic Shannon information theory. Then, the term Sik→j refers to the synergy
between the two sources while they transfer information to the target, intended as the information that is uniquely obtained taking the two sources Yi and Yk together, but not considering them
alone. Compared to the IID defined in (3), the PID (5) has the advantage that it provides distinct
non-negative measures of redundancy and synergy, thereby accounting for the possibility that redundancy and synergy may coexist as separate elements of information modification. Interestingly,
the IID and PID defined in Equations (3) and (5) are related to each other in a way such that:
Iik→j = Sik→j − Rik→j ,

(6)

thus showing that the interaction TE is actually a measure of the ‘net’ synergy manifested in the
transfer of information from the two sources to the target.
An issue with the PID (5) is that its constituent measures cannot be obtained through classic
information theory simply subtracting conditional MI terms as done for the IID; an additional
ingredient to the theory is needed to get a fourth defining equation to be added to (5) for providing
an unambiguous definition of Ui→j , Uk→j , Rik→j and Sik→j . While several PID definitions have
been proposed arising from different conceptual definitions of redundancy and synergy Harder
et al. (2013), Griffith et al. (2014), Bertschinger et al. (2014), here, we make reference to the socalled minimum MI (MMI) PID Barrett (2015). According to the MMI PID, redundancy is defined
as the minimum of the information provided by each individual source to the target. In terms of
information transfer measured by the TE, this leads to the following definition of the redundant
TE:
Rik→j = min{Ti→j , Tk→j }.
(7)
This choice satisfies the desirable property that the redundant TE is independent of the correlation between the source processes. Moreover, it has been shown that, if the observed processes
have a joint Gaussian distribution, all previously-proposed PID formulations reduce to the MMI
PID Barrett (2015).

3
3.1

Multiscale Information Transfer Decomposition
Multiscale Representation of Multivariate Gaussian Processes

In the linear signal processing framework, the M -dimensional vector stochastic process Yn = [Y1,n · · · YM,n ]T
is classically described using a vector autoregressive (VAR) model of order p:
Yn =

p
X

Ak Yn−k + Un

k=1

7

(8)

where Ak are M × M matrices of coefficients and Un = [U1,n · · · UM,n ]T is a vector of M zero
mean Gaussian processes with covariance matrix Σ ≡ E[Un UnT ] (E is the expectation operator).
To study the observed process Y at the temporal scale identified by the scale factor τ , we apply the
following transformation to each constituent process Ym , m = 1, . . . , M :
Ȳm,n =

q
X

bl Ym,nτ −l .

(9)

l=0

This rescaling operation corresponds to transforming the original process Y through a two-step
procedure that consists of the following filtering and downsampling steps, yielding respectively the
processes Ỹ and Ȳ :
q
X

bl Yn−l ,

(10a)

Ȳn = Ỹnτ , n = 1, . . . , N/τ

(10b)

Ỹn =

l=0

The change of scale in (9) generalizes the averaging procedure originally proposed in Costa
et al. (2002), which sets q = τ − 1 and bl = 1/τ and, thus, realizes the step of filtering through
the simple procedure of averaging τ subsequent samples. To improve the elimination of the fast
temporal scales, in this study, we follow the idea of Valencia et al. (2009), in which a more appropriate low pass filter than averaging is employed. Here, we identify the bl as the coefficients
of a linear finite impulse response (FIR) low pass filter of order q; the FIR filter is designed using
the classic window method with the Hamming window Oppenheim and Schafer (1975), setting the
cutoff frequency at fτ = 1/2τ in order to avoid aliasing in the subsequent downsampling step.
Substituting (8) in (10a), the filtering step leads to the process representation:
Ỹn =

p
X

Ak Ỹn−k +

k=1

q
X

Bl Un−l

(11)

l=0

where Bl = bl IM (IM is the M × M identity matrix). Hence, the change of scale introduces a
moving average (MA) component of order q in the original VAR(p) process, transforming it into
a VARMA(p, q) process. As we will show in the next section, the downsampling step (10b) keeps
the VARMA representation, altering the model parameters.

3.2
3.2.1

State Space Processes
Formulation of State Space Models

State space models are models that make use of state variables to describe a system by a set of firstorder difference equations, rather than by one or more high-order difference equations Hannan and
8

Deistler (2012), Aoki (2013). The general linear state space (SS) model describing an observed
vector process Y has the form:

Xn+1 = AXn + Wn
Yn = CXn + Vn

(12a)
(12b)

where the state Equation (12a) describes the update of the L-dimensional state (unobserved) process through the L × L matrix A, and the observation Equation (12b) describes the instantaneous
mapping from the state to the observed process through the M × L matrix C. Wn and Vn are
zero-mean white noise processes with covariances Q ≡ E[Wn WnT ] and R ≡ E[Vn VnT ] and crosscovariance S ≡ E[Wn VnT ]. Thus, the parameters of the SS model (12) are (A, C, Q, R, S).
Another possible SS representation is that evidencing the innovations En = Yn − E[Yn |Yn− ],
T Y T · · · ]T Aoki
i.e., the residuals of the linear regression of Yn on its infinite past Yn− = [Yn−1
n−2
(2013). This new SS representation, usually referred to as the “innovations form” SS model (ISS),
is characterized by the state process Zn = E[Xn |Yn− ] and by the L × M Kalman gain matrix K:
Zn+1 = AZn + KEn

(13a)

Yn = CZn + En

(13b)

The parameters of the ISS model (13) are (A, C, K, V), where V is the covariance of the
innovations, V ≡ E[En EnT ]. Note that the ISS (13) is a special case of (12) in which Wn = KEn
and Vn = En , so that Q = KVKT , R = V and S = KV.
Given an SS model in the form (12), the corresponding ISS model (13) can be identified by
solving a so-called discrete algebraic Riccati equation (DARE) formulated in terms of the state
error variance matrix P Solo (2016):
P = APAT + Q − (APCT + S)(CPCT + R)−1 (CPAT + ST )

(14)

Under some assumptions Solo (2016), the DARE (14) has a unique stabilizing solution, from
which the Kalman gain and innovation covariance can be computed as:
V = CPCT + R
K = (APCT + S)V−1 ,

(15)

thus completing the transformation from the SS form to the ISS form.
3.2.2

State Space Models of Filtered and Downsampled Linear Processes

Exploiting the close relation between VARMA models and SS models, first we show how to convert
the VARMA model (11) into an ISS model in the form of (13) that describes the filtered process
9

Y˜n . To do this, we exploit Aoki’s method Aoki and Havenner (1991) defining the state process
T · · · Y T UT
T
T
˜
Z̃n = [Yn−1
n−p n−1 · · · Un−q ] that, together with Yn , obeys the state Equation (13) with
parameters (Ã, C̃, K̃, Ṽ), where:


A1 · · · Ap−1 Ap B1 · · · Bq−1 Bq
 IM · · · 0M 0M 0M · · · 0M 0M 


 .
..
..
..
..
.. 
 ..
.
.
.
.
. 


0
0M 0M · · · 0M 0M 

 M · · · IM
Ã = 

0M · · · 0M 0M 0M · · · 0M 0M 


0M · · · 0M 0M IM · · · 0M 0M 


 ..
..
..
..
..
.. 
 .
.
.
.
.
. 
· · · 0M 0M 0M · · · IM
h
i
C̃ = A1 · · · Ap B1 · · · Bq

0M

h
K̃ = IM

0M ×M (p−1) B−T
0

0M ×M (q−1)

0M

iT

and Ṽ = B0 Σ BT0 , where Ṽ is the covariance of the innovations Ẽn = B0 Un .
Now, we turn to show how the downsampled process Ȳn can be represented through an ISS
model directly from the ISS formulation of the filtered process Ỹn . To this end, we exploit recent
theoretical findings providing the state space form of downsampled signals (Theorem III in Solo
(2016)). Accordingly, the SS representation of the process downsampled at scale τ , Ȳn = Ỹnτ has
parameters (Ā, C̄, Q̄, R̄, S̄), where Ā = Ãτ , C̄ = C̃, Q̄ = Qτ , R̄ = Ṽ and S̄ = Sτ , with Qτ
and Sτ given by:
Sτ = Ãτ −1 K̃Ṽ
Qτ = ÃQτ −1 ÃT + K̃ṼK̃T , τ ≥ 2

(16)

Q1 = K̃ṼK̃T , τ = 1.
Therefore, the downsampled process has an ISS representation with state process Z̄n = Z̃nτ ,
innovation process Ēn = Ẽnτ and parameters (Ā, C̄, K̄, V̄), where K̄ and V̄ are obtained solving
the DARE (14) and (15) for the SS model with parameters (Ā, C̄, Q̄, R̄, S̄).
To sum up, the relations and parametric representations of the original process Y , the filtered
process Ỹ and the downsampled process Ȳ are depicted in Figure 2a. The step of low pass filtering
(FLT) applied to a VAR(p) process yields a VARMA(p, q) process (where q is the filter order,
and the cutoff frequency is fτ = 1/2τ ); this process is equivalent to an ISS process Aoki and
Havenner (1991). The subsequent downsampling (DWS) yields a different SS process, which in
turn can be converted to the ISS form solving the DARE. Thus, both the filtered process Ỹn and
the downsampled process Ȳn can be represented as ISS processes with parameters (Ã, C̃, K̃, Ṽ)
10

and (Ā, C̄, K̄, V̄) which can be derived analytically from the knowledge of the parameters of
the original process (A1 , . . . , Ap , Σ) and of the filter (q, fτ ). In the next section, we show how to
compute analytically any measure appearing in the information decomposition of a jointly Gaussian
multivariate stochastic process starting from its associated ISS model parameters, thus opening the
way to the analytical computation of these measures for multiscale (filtered and downsampled)
processes.

Figure 2: Schematic representation of a linear VAR process and of its multiscale representation obtained through filtering (FLT) and downsampling (DWS) steps. The downsampled
process has an innovations form state space model (ISS) representation from which submodels can be formed to compute the partial variances needed for the computation of
information measures appearing in the IID and PID decompositions. This makes it possible to perform multiscale information decomposition analytically from the original VAR
parameters and from the scale factor.

3.3

Multiscale IID and PID

After introducing the general theory of information decomposition and deriving the multiscale
representation of the parameters of a linear VAR model, in this section, we provide expressions
for the terms of the IID and PID decompositions of the information transfer valid for multivariate
jointly Gaussian processes. The derivations are based on the knowledge that the linear parametric
representation of Gaussian processes given in (8) captures all of the entropy differences that define
the various information measures Barrett et al. (2010) and that these entropy differences are related
to the partial variances of the present of the target given its past and the past of one or more sources,
intended as variances of the prediction errors resulting from linear regression Faes et al. (2015,
−
−
−
2017). Specifically, let us denote as Ej|j,n = Yj,n − E[Yj,n |Yj,n
], Ej|ij,n = Yj,n − E[Yj,n |Yi,n
, Yj,n
]
−
−
−
the prediction error of a linear regression of Yj,n performed respectively on Yj,n and (Yj,n , Yi,n )
2
2
and as λj|j = E[Ej|j,n
], λj|ij = E[Ej|ij,n
], the corresponding prediction error variances. Then, the

11

TE from Yi to Yj can be expressed as:
Ti→j =

λj|j
1
ln
.
2 λj|ij

(17)

In a similar way, the joint TE from (Yi , Yk ) to Yj can be defined as:
Tik→j =

λj|j
1
ln
,
2 λj|ijk

(18)

2
where λj|ijk = E[Ej|ijk,n
] is the variance of the prediction error of a linear regression of Yj,n
−
−
−
−
−
−
on (Yj,n , Yi,n , Yk,n ), Ej|ijk,n = Yj,n − E[Yj,n |Yi,n
, Yj,n
, Yk,n
]. Based on these derivations, one
can easily complete the IID decomposition of TE by computing Tk→j as in (17) and deriving the
interaction TE from (3) and the PID decomposition, as well by deriving the redundant TE from (7),
the synergistic TE from (6) and the unique TEs from (5).
Next, we show how to compute any partial variance from the parameters of an ISS model in the
form of (13) Barnett and Seth (2015), Solo (2016). The partial variance λj|a , where the subscript
a denotes any combination of indexes ∈ {1, . . . , M }, can be derived from the ISS representation
of the innovations of a submodel obtained removing the variables not indexed by a from the observation equation. Specifically, we need to consider the submodel with state Equation (13b) and
observation equation:
Yn(a) = C(a) Zn + En(a) ,
(19)

where the superscript (a) denotes the selection of the rows with indices a of a vector or a matrix.
It is important to note that the submodels (13a) and (19) are not in innovations form, but are
rather an SS model with parameters (A, C(a) , KVKT , V(a, a), KV(:, a)). This SS model can be
converted to an ISS model with innovation covariance V(a) solving the DARE (14) and (15), so that
the partial variance λj|a is derived as the diagonal element of V(a) corresponding to the position
of the target Yj . Thus, with this procedure, it is possible to compute the partial variances needed
for the computation of the information measures starting from a set of ISS model parameters; since
any VAR process can be represented at scale τ as an ISS process, the procedure allows computing
the IID and PID information decompositions for the rescaled multivariate process (see Figure 2).
It is worth remarking that, while the general formulation of IID and PID decompositions introduced in Section 2 holds for arbitrary processes, the multiscale extension detailed in Section 3
is exact only if the processes have a joint Gaussian distribution. In such a case, the linear VAR
representation captures exhaustively the joint variability of the processes, and any nonlinear extension has no additional utility (a formal proof of the fact that a stationary Gaussian VAR process
must be linear can be found in Barrett et al. (2010)). If, on the contrary, non-Gaussian processes
are under scrutiny, the linear representation provided in Section 3.1 can still be adopted, but may
miss important properties in the dynamics and thus provide only a partial description. Moreover,
12

since the close correspondence between conditional entropies and partial variances reported in this
subsection does not hold anymore for non-Gaussian processes, all of the obtained measures should
be regarded as indexes of (linear) predictability rather than as information measures.

4

Simulation Experiment

To study the multiscale patterns of information transfer in a controlled setting with known dynamical interactions between time series, we consider a simulation scheme similar to some already
used for the assessment of theoretical values of information dynamics Faes et al. (2015, 2017).
Specifically, we analyze the following VAR process of order M = 4:
Y1,n = 2ρ1 cos2πf1 Y1,n−1 − ρ21 Y1,n−2 + U1,n ,
Y2,n = 2ρ2 cos2πf2 Y2,n−1 −
Y3,n = 2ρ3 cos2πf3 Y3,n−1 −

ρ22 Y2,n−2
ρ23 Y3,n−2

(20a)

+ cY1,n−1 + U2,n ,

(20b)

+ cY1,n−1 + U3,n ,

(20c)

Y4,n = bY2,n−1 + (1 − b)Y3,n−1 + U4,n ,

(20d)

where Un = [U1,n · · · U4,n ]T is a vector of zero mean white Gaussian noises with unit variance
and uncorrelated with each other (Σ= I). The parameter design in Equation (20) is chosen to allow
autonomous oscillations in the processes Yi , i = 1, . . . , 3, obtained placing complex-conjugate
poles with modulus ρi and frequency fi in the complex plane representation of the transfer function
of the vector process, as well as causal interactions between the processes at a fixed time lag of one
sample and with strength modulated by the parameters b and c (see Figure 3). In this study, we
set the coefficients related to self-dependencies to values generating well-defined oscillations in
all processes (ρ1 = ρ2 = ρ3 = 0.95) and letting Y1 fluctuate at slower time scales than Y2 and Y3
(f1 = 0.1, f2 = f3 = 0.025). We consider four configurations of the parameters, chosen to
reproduce paradigmatic conditions of interaction between the processes:
(a)

isolation of Y1 and Y2 and unidirectional coupling Y3 → Y4 , obtained setting b = c = 0;

(b)

common driver effects Y2 ← Y1 → Y3 and unidirectional coupling Y3 → Y4 , obtained setting
b = 0 and c = 1;

(c)

isolation of Y1 and unidirectional couplings Y2 → Y4 and Y3 → Y4 , obtained setting b = 0.5
and c = 0;

(d)

common driver effects Y2 ← Y1 → Y3 and unidirectional couplings Y2 → Y4 and Y3 → Y4 ,
obtained setting b = 0.5 and c = 1.

13

Figure 3: Graphical representation of the four-variate VAR process of Equation (20) that
we use to explore the multiscale decomposition of the information transferred to Y4 , selected
as the target process, from Y2 and Y3 , selected as the source processes, in the presence of Y1 ,
acting as the exogenous process. To favor such exploration, we set oscillations at different
time scales for Y1 (f1 = 0.1) and for Y2 and Y3 (f2 = f3 = 0.025), induce common driver
effects from the exogenous process to the sources modulated by the parameter c and allow
for varying strengths of the causal interactions from the sources to the target as modulated
by the parameter b. The four configurations explored in this study are depicted in (a–d).
With this simulation setting, we compute all measures appearing in the IID and PID decompositions of the information transfer, considering Y4 as the target process and Y2 and Y3 as the source
processes. The theoretical values of these measures, computed as a function of the time scale using
the IID and the PID, are reported in Figure 4. In the simple case of unidirectional coupling Y3 → Y4
(b = c = 0, Figure 4a), the joint information transferred from (Y2 , Y3 ) to Y4 is exclusively due to
the source Y3 without contributions from Y2 and without interaction effects between the sources
(T23→4 = T3→4 = U3→4 , T2→4 = U2→4 = 0, I23→4 = S23→4 = R23→4 = 0).
When the causal interactions towards Y4 are still due exclusively to Y3 , but the two sources
Y2 and Y3 share information arriving from Y1 (b = 0, c = 1; Figure 4b), the IID evidences that
the joint information transfer coincides again with the transfer from Y3 (T23→4 = T3→4 ), but a
non-trivial amount of information transferred from Y2 to Y4 emerges, which is fully redundant
(T2→4 = −I23→4 ). The PID highlights that the information from Y3 to Y4 is not all unique, but
is in part transferred redundantly with Y2 , while the unique transfer from Y2 and the synergistic
transfer are negligible.
In the case of two isolated sources equally contributing to the target (b = 0.5, c = 0, Figure
4c), the IID evidences the presence of net synergy and of identical amounts of information transferred to Y4 from Y2 or Y3 (I23→4 > 0, T2→4 = T3→4 ). The PID documents that there are no
unique contributions, so that the two amounts of information transfer from each source to the target coincide with the redundant transfer, and the remaining part of the joint transfer is synergistic
(U2→4 = U3→4 = 0, T2→4 = T3→4 = R23→4 , S23→4 = T23→4 − R23→4 ).
Finally, when the two sources share common information and contribute equally to the target
(b = 0.5, c = 1; Figure 4d), we find that they send the same amount of information as before, but in
this case, no unique information is sent by any of the sources (T2→4 = T3→4 , U2→4 = U3→4 = 0).
Moreover, the nature of the interaction between the sources is not trivial and is scale dependent:
14

at low time scales, where the dynamics are likely dominated by the fast oscillations of Y1 , the IID
reveals net redundancy, and the PID shows that the redundant transfer prevails over the synergistic
(I23→4 < 0, R23→4 > S23→4 ); at higher time scales, where fast dynamics are filtered out and
the slow dynamics of Y2 and Y3 prevail, the IID reveals net synergy, and the PID shows that the
synergistic transfer prevails over the redundant (I23→4 > 0, S23→4 > R23→4 ).

Figure 4: Multiscale information decomposition for the simulated VAR process of Equation (20). Plots depict the exact values of the entropy measures forming the interaction
information decomposition (IID, upper row) and the partial information decomposition
(PID, lower row) of the information transferred from the source processes Y2 and Y3 to
the target process Y4 generated according to the scheme of Figure 3 with four different
configurations of the parameters. We find that linear processes may generate trivial information patterns with the absence of synergistic or redundant behaviors (a), patterns with
the prevalence of redundant information transfer (b) or synergistic information transfer
(c) that persist across multiple time scales, or even complex patterns with the alternating
prevalence of redundant transfer and synergistic transfer at different time scales (d).

5

Application

As a real data application, we analyze intracranial EEG recordings from a patient with drugresistant epilepsy measured by an implanted array of 8 × 8 cortical electrodes and two left hip15

pocampal depth electrodes with six contacts each. The data are available in epi, and further details
on the dataset are given in Kramer et al. (2008). Data were sampled at 400 Hz and correspond to
10-s segments recorded in the pre-ictal period, just before the seizure onset, and 10 s during the
ictal stage of the seizure, for a total of eight seizures. Defining and locating the seizure onset zone,
i.e., the specific location in the brain where the synchronous activity of neighboring groups of cells
becomes so strong so as to be able to spread its own activity to other distant regions, is an important issue in the study of epilepsy in humans. Here, we focus on the information flow from the
sub-cortical regions, probed by depth electrodes, to the brain cortex. In Stramaglia et al. (2014), it
has been suggested that Contacts 11 and 12, in the second depth electrode, are mostly influencing
the cortical activity; accordingly, in this work, we consider Channels 11 and 12 as a pair of source
variables for all of the cortical electrodes and decompose the information flowing from them using
the multiscale IID and PID here proposed, both in the pre-ictal stage and in the ictal stage. An FIR
filter with q = 12 coefficients is used, and the order p of the VAR model is fixed according to the
Bayesian information criterion. In the analyzed dataset, the model order assessed in the pre-ictal
phase was p = 14.61 ± 1.07 (mean ± std. dev.across 64 electrodes and eight seizures) and during
the ictal phase decreased significantly to p = 11.09 ± 3.95.
In Figure 5, we depict the terms of the IID applied from the two sources (Channels {11, 12})
to any of the electrodes as a function of the scale τ , averaged over the eight seizures. We observe
a relevant enhancement of the joint TE during the seizure, w.r.t. the pre-ictal period. This enhancement is determined by a marked increase of both the individual TEs from Channels 11 and 12 to all
of the cortical electrodes; the patterns of the two TEs are similar to each other in both stages. The
pattern of interaction information transfer displays prevalent redundant transfer for low values of
τ and prevalent synergistic transfer for high τ , but the values of the interaction TE have relatively
low magnitude and are only slightly different in pre-ictal and ictal conditions. It is worth stressing
that at scale τ , the algorithm analyzes oscillations, in the time series, slower than 2τ1fs s, where
fs = 400 Hz.

16

Figure 5: Interaction information decomposition (IID) of the intracranial EEG information
flow from subcortical to cortical regions in an epileptic patient. The joint transfer entropy
from depth Channels 11 and 12 to cortical electrodes (a); the transfer entropy from depth
Channel 11 to cortical electrodes (b); the transfer entropy from depth Channel 12 to
cortical electrodes (c) and the interaction transfer entropy from depth Channels 11 and
12 to cortical electrodes (d) are depicted as a function of the scale τ , after averaging over
the eight pre-ictal segments (left column) and over the eight ictal segments (right column).
Compared with pre-ictal periods, during the seizure, the IID evidences marked increases
of the joint and individual information transfer from depth to cortical electrodes and low
and almost unvaried levels of interaction transfer.
17

In Figure 6, we depict, on the other hand, the terms of the PID computed for the same data.
This decomposition shows that the increased joint TE across the seizure transition seen in Figure 5a
is in large part the result of an increase of both the synergistic and the redundant TE, which are
markedly higher during the ictal stage compared with the pre-ictal. This explains why the interaction TE of Figure 5d, which is the difference between two quantities that both increase, is nearly
constant moving from the pre-ictal to the ictal stage. The quantity that, instead, clearly differentiates between Channels 11 and 12 is the unique information transfer: indeed, only the unique TE
from Channel 12 increases in the ictal stage, while the unique TE from Channel 13 remains at low
levels.

18

Figure 6: Partial information decomposition (PID) of the intracranial EEG information
flow from subcortical to cortical regions in an epileptic patient. The synergistic transfer
entropy from depth Channels 11 and 12 to cortical electrodes (a); the redundant transfer
entropy from depth Channels 11 and 12 to cortical electrodes (b); the unique transfer
entropy from depth Channel 11 to cortical electrodes (c) and the unique transfer entropy
from depth Channel 12 to cortical electrodes (d) are depicted as a function of the scale
τ , after averaging over the eight pre-ictal segments (left column) and over the eight ictal
segments (right column). Compared with pre-ictal periods, during the seizure, the PID
evidences marked increases of the information transferred synergistically and redundantly
from depth to cortical electrodes and of the information transferred uniquely from one of
the two depth electrodes, but not from the other.
19

In order to investigate the variability across trials of the estimates of the various information
measures, in Figure 7, we depict the terms of both IID and PID expressed for each ictal episode as
average values over all 64 cortical electrodes. The analysis shows that the higher average values
observed in Figures 5 and 6 at Scales 1–4 during the ictal state for the joint TE, the two individual
TEs, the redundant and synergistic TEs and the unique TE from depth Channel 12 are the result of
an increase of the measures for almost all of the observed seizure episodes.
These findings are largely in agreement with the increasing awareness that epilepsy is a network
phenomenon that involves aberrant functional connections across vast parts of the brain on virtually
all spatial scales Richardson (2012), Dickten et al. (2016). Indeed, our results document that the
occurrence of seizures is associated with a relevant increase of the information flowing from the
subcortical regions (associated with the depth electrode) to the cortex and that the character of this
information flow is mostly redundant both in the pre-ictal and in the ictal state. Here, the need for
a multiscale approach is testified by the fact that several quantities in the ictal state (e.g., the joint
TE, the synergistic ITand the unique ITfrom Channel 12) attain their maximum at scale τ > 1.
Moreover, the approaches that we propose for information decomposition appear useful to
improve the localization of epileptogenic areas in patients with drug-resistant epilepsy. Indeed,
our analysis suggests that Contact 12 is the closest to the seizure onset zone, and it is driving the
cortical oscillations during the ictal stage, as it sends unique information to the cortex. On the other
hand, to disentangle this effect, it has been necessary to include also Channel 11 in the analysis and
to make the PID of the total information from the pair of depth channels to the cortex; indeed, the
redundancy between Channels 11 and 12 confounds the informational pattern unless the PID is
performed.

20

Figure 7: Multiscale representation of the measures of interaction information decomposition (IID, top) and partial information decomposition (PID, bottom) computed as a
function of the time scale for each of the eight seizures during the pre-ictal period (black)
and the ictal period (red). Values of joint transfer entropy (TE), individual TE, interaction
TE, redundant TE, synergistic TE and unique TE are obtained taking the depth Channels
11 and 12 as sources and averaging over all 64 target cortical electrodes. Increases during
seizure of the joint TE, individual TEs from both depth electrodes, redundant and synergistic TE and unique TE from the depth electrode 12 are evident at low time scales for
almost all considered episodes.

6

Conclusions

Understanding how multiple inputs may combine to create the output of a given target is a fundamental challenge in many fields, in particular in neuroscience. Shannon’s information theory is the
most suitable frame to cope with this problem and thus to assess the informational character of multiplets of variables describing complex systems; IID indeed measures the balance between redundant and synergetic interaction within the classical multivariate entropy-based approach. Recently
Shannon’s information theory has been extended, in the PID, so as to provide specific measures
21

for the information that several variables convey individually (unique information), redundantly
(shared information) or only jointly (synergistic information) about the output.
The contribution of the present work is the proposal of an analytical frame where both IID
and PID can be exactly evaluated in a multiscale fashion, for multivariate Gaussian processes, on
the basis of simple vector autoregressive identification. In doing this, our work opens the way
for both the theoretical analysis and the practical implementation of information modification in
processes that exhibit multiscale dynamical structures. The effectiveness of the proposed approach
has been demonstrated both on simulated examples and on real publicly-available intracranial EEG
data. Our results provide a firm ground to the multiscale evaluation of PID, to be applied in all
applications where causal influences coexist at multiple temporal scales.
Future developments of this work include the refinement of the SS model structure to accommodate the description of long-range linear correlations Sela and Hurvich (2009) or its expansion
to the description of nonstationary processes Kitagawa (1987) and the formalization of exact crossscale computation of information decomposition within and between multivariate processes Paluš
(2014). A major challenge in the field remains the generalization of this type of analysis to nonGaussian processes, for which exact analytical solutions or computationally-reliable estimation
approaches are still lacking. This constitutes a main direction for further research, because realworld processes display very often non-Gaussian distributions, which would make an extension
to nonlinear models or model-free approaches beneficial. The questions that are still open in this
respect include the evaluation of proper theoretical definitions of synergy or redundancy for nonlinear processes Williams and Beer (2010), Harder et al. (2013), Griffith et al. (2014), Quax et al.
(2017), Bertschinger et al. (2014), the development of reliable entropy estimators for multivariate
variables with different dimensions Wibral et al. (2014), Faes et al. (2015), Papana et al. (2011)
and the assessment of the extent to which non-linear model-free methods really outperform the
linear model-based approach adopted here and in previous investigations Porta et al. (2017).

References
Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. A framework for the local information dynamics of
distributed computation in complex systems. In Guided Self-Organization: Inception; Springer:
Berlin, Heidelberg, Germany, 2014; pp. 115–158.
Pincus, S. Approximate entropy (ApEn) as a complexity measure. Chaos Interdiscip. J. Nonlinear
Sci. 1995, 5, 110–117.
Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. Local measures of information storage in complex
distributed computation. Inf. Sci. 2012, 208, 39–54.

22

Wibral, M.; Lizier, J.T.; Vögler, S.; Priesemann, V.; Galuske, R. Local active information storage
as a tool to understand distributed neural information processing. Front. Neuroinform. 2014, 8,
doi:10.3389/fninf.2014.00001.
Schreiber, T. Measuring information transfer. Phys. Rev. Lett. 2000, 85, 461.
Wibral, M.; Vicente, R.; Lizier, J.T. Directed Information Measures in Neuroscience; Springer:
Berlin, Heidelberg, Germany, 2014.
Lizier, J.T.; Prokopenko, M.; Zomaya, A.Y. Information modification and particle collisions in
distributed computation. Chaos Interdiscip. J. Nonlinear Sci. 2010, 20, 037109.
Wibral, M.; Lizier, J.T.; Priesemann, V. Bits from brains for biologically inspired computing.
Front. Robot. Artif. Intell. 2015, 2, doi:10.3389/frobt.2015.00005.
Lizier, J.T.; Pritam, S.; Prokopenko, M. Information dynamics in small-world Boolean networks.
Artif. Life 2011, 17, 293–314.
Wibral, M.; Rahm, B.; Rieder, M.; Lindner, M.; Vicente, R.; Kaiser, J. Transfer entropy in magnetoencephalographic data: Quantifying information flow in cortical and cerebellar networks.
Prog. Biophys. Mol. Biol. 2011, 105, 80–97.
Hlinka, J.; Hartman, D.; Vejmelka, M.; Runge, J.; Marwan, N.; Kurths, J.; Paluš, M. Reliability
of inference of directed climate networks using conditional mutual information. Entropy 2013,
15, 2023–2045.
Barnett, L.; Lizier, J.T.; Harré, M.; Seth, A.K.; Bossomaier, T. Information flow in a kinetic Ising
model peaks in the disordered phase. Phys. Rev. Lett. 2013, 111, 177203.
Marinazzo, D.; Pellicoro, M.; Wu, G.; Angelini, L.; Cortés, J.M.; Stramaglia, S. Information transfer and criticality in the ising model on the human connectome. PLoS ONE 2014, 9, e93616.
Faes, L.; Nollo, G.; Jurysta, F.; Marinazzo, D. Information dynamics of brain–heart physiological
networks during sleep. New J. Phys. 2014, 16, 105005.
Faes, L.; Porta, A.; Nollo, G. Information decomposition in bivariate systems: Theory and application to cardiorespiratory dynamics. Entropy 2015, 17, 277–303.
Porta, A.; Faes, L.; Nollo, G.; Bari, V.; Marchi, A.; de Maria, B.; Takahashi, A.C.; Catai, A.M.
Conditional self-entropy and conditional joint transfer entropy in heart period variability during
graded postural challenge. PLoS ONE 2015, 10, e0132851.

23

Faes, L.; Porta, A.; Nollo, G.; Javorka, M. Information Decomposition in Multivariate Systems:
Definitions, Implementation and Application to Cardiovascular Networks. Entropy 2017, 19, 5.
Wollstadt, P.; Sellers, K.K.; Rudelt, L.; Priesemann, V.; Hutt, A.; Fröhlich, F.; Wibral, M. Breakdown of local information processing may underlie isoflurane anesthesia effects. PLoS Comput.
Biol. 2017, 13, e1005511.
Schneidman, E.; Bialek, W.; Berry, M.J. Synergy, redundancy, and independence in population
codes. J. Neurosci. 2003, 23, 11539–11553.
Stramaglia, S.; Wu, G.R.; Pellicoro, M.; Marinazzo, D. Expanding the transfer entropy to identify
information circuits in complex systems. Phys. Rev. E 2012, 86, 066211.
Stramaglia, S.; Cortes, J.M.; Marinazzo, D. Synergy and redundancy in the Granger causal analysis
of dynamical networks. New J. Phys. 2014, 16, 105003.
Stramaglia, S.; Angelini, L.; Wu, G.; Cortes, J.; Faes, L.; Marinazzo, D. Synergetic and Redundant
Information Flow Detected by Unnormalized Granger Causality: Application to Resting State
fMRI. IEEE Trans. Biomed. Eng. 2016, 63, 2518–2524.
McGill, W. Multivariate information transmission. Trans. IRE Prof. Group Inf. Theory 1954,
4, 93–111.
Bell, A.J. The co-information lattice. In Proceedings of the Fourth International Symposium on
Independent Component Analysis and Blind Signal Separation (ICA), Nara, Japan, 1–4 April
2003.
Williams, P.L.; Beer, R.D. Nonnegative decomposition of multivariate information. arXiv 2010,
arXiv:1004.2515 .
Harder, M.; Salge, C.; Polani, D. Bivariate measure of redundant information. Phys. Rev. E 2013,
87, 012130.
Griffith, V.; Chong, E.K.; James, R.G.; Ellison, C.J.; Crutchfield, J.P. Intersection information
based on common randomness. Entropy 2014, 16, 1985–2000.
Quax, R.; Har-Shemesh, O.; Sloot, P.M.A. Quantifying Synergistic Information Using Intermediate
Stochastic Variables. Entropy 2017, 19, 85.
Bertschinger, N.; Rauh, J.; Olbrich, E.; Jost, J.; Ay, N. Quantifying unique information. Entropy
2014, 16, 2161–2183.

24

Panzeri, S.; Senatore, R.; Montemurro, M.A.; Petersen, R.S. Correcting for the sampling bias
problem in spike train information measures. J. Neurophysiol. 2007, 98, 1064–1072.
Faes, L.; Porta, A. Conditional entropy-based evaluation of information dynamics in physiological systems. In Directed Information Measures in Neuroscience; Springer: Berlin, Heidelberg,
Germany, 2014; pp. 61–86.
Kozachenko, L.; Leonenko, N.N. Sample estimate of the entropy of a random vector. Probl.
Pereda. Inf. 1987, 23, 9–16.
Vlachos, I.; Kugiumtzis, D. Nonuniform state-space reconstruction and coupling detection. Phys.
Rev. E 2010, 82, 016207.
Marinazzo, D.; Pellicoro, M.; Stramaglia, S. Causal information approach to partial conditioning
in multivariate data sets. Comput. Math. Methods Med. 2012, 2012, 303601.
Faes, L.; Kugiumtzis, D.; Nollo, G.; Jurysta, F.; Marinazzo, D. Estimating the decomposition of
predictive information in multivariate systems. Phys. Rev. E 2015, 91, 032904
Barrett, A.B. Exploration of synergistic and redundant information sharing in static and dynamical
Gaussian systems. Phys. Rev. E 2015, 91, 052802.
Barrett, A.B.; Barnett, L.; Seth, A.K. Multivariate Granger causality and generalized variance.
Phys. Rev. E 2010, 81, 041907.
Porta, A.; Bari, V.; de Maria, B.; Takahashi, A.C.; Guzzetti, S.; Colombo, R.; Catai, A.M.; Raimondi, F.; Faes, L. Quantifying Net Synergy/Redundancy of Spontaneous Variability Regulation
via Predictability and Transfer Entropy Decomposition Frameworks. IEEE Trans. Biomed. Eng.
2017, doi:10.1109/TBME.2017.2654509.
Ivanov, P.; Nunes Amaral, L.; Goldberger, A.; Havlin, S.; Rosenblum, M.; Struzik, Z.; Stanley, H.
Multifractality in human heartbeat dynamics. Nature 1999, 399, 461–465.
Chou, C.M. Wavelet-based multi-scale entropy analysis of complex rainfall time series. Entropy
2011, 13, 241–253.
Wang, J.; Shang, P.; Zhao, X.; Xia, J. Multiscale entropy analysis of traffic time series. Int. J. Mod.
Phys. C 2013, 24, 1350006.
Costa, M.; Goldberger, A.L.; Peng, C.K. Multiscale entropy analysis of complex physiologic time
series. Phys. Rev. Lett. 2002, 89, 068102.

25

Valencia, J.; Porta, A.; Vallverdú, M.; Clariá, F.; Baranowski, R.; Orłowska-Baranowska, E.; Caminal, P. Refined multiscale entropy: Application to 24-h holter recordings of heart period variability in healthy and aortic stenosis subjects. IEEE Trans. Biomed. Eng. 2009, 56, 2202–2213.
Barnett, L.; Seth, A.K. Granger causality for state-space models. Phys. Rev. E 2015, 91, 040101.
Solo, V. State-space analysis of Granger-Geweke causality measures with application to fMRI.
Neural Comput. 2016, 28, 914–949.
Florin, E.; Gross, J.; Pfeifer, J.; Fink, G.R.; Timmermann, L. The effect of filtering on Granger
causality based multivariate causality measures. Neuroimage 2010, 50, 577–588.
Barnett, L.; Seth, A.K. Detectability of Granger causality for subsampled continuous-time neurophysiological processes. J. Neurosci. Methods 2017, 275, 93–121.
Faes, L.; Montalto, A.; Stramaglia, S.; Nollo, G.; Marinazzo, D. Multiscale Analysis of Information Dynamics for Linear Multivariate Processes. arXiv 2016, arXiv:1602.06155 .
Faes, L.; Nollo, G.; Stramaglia, S.; Marinazzo, D. Multiscale Granger causality. arXiv 2017,
arXiv:1703.08487.
Aoki, M.; Havenner, A. State space modeling of multiple time series. Econom. Rev. 1991, 10, 1–
59.
Oppenheim, A.V.; Schafer, R.W. Digital Signal Processing; Prentice-Hall: Englewood Cliffs, NJ,
USA, 1975.
Hannan, E.J.; Deistler, M. The Statistical Theory of Linear Systems; Society for Industrial and
Applied Mathematics: Philadelphia, PA, USA, 2012; Volume 70.
Aoki, M. State Space Modeling of Time Series; Springer Science & Business Media: Berlin,
Heidelberg, Germany, 2013.
Earth System Research Laboratory.
Available online:
kolaczyk/datasets.html (accessed on 5 May 2017).

http://math.bu.edu/people/

Kramer, M.A.; Kolaczyk, E.D.; Kirsch, H.E. Emergent network topology at seizure onset in humans. Epilepsy Res. 2008, 79, 173–186.
Richardson, M.P. Large scale brain models of epilepsy: Dynamics meets connectomics. J. Neurol
Neurosurg. Psychiatry 2012, 83, 1238–1248.

26

Dickten, H.; Porz, S.; Elger, C.E.; Lehnertz, K. Weighted and directed interactions in evolving
large-scale epileptic brain networks. Sci. Rep. 2016, 6, 34824.
Sela, R.J.; Hurvich, C.M. Computationally efficient methods for two multivariate fractionally
integrated models. J. Time Ser. Anal. 2009, 30, 631–651.
Kitagawa, G. Non-gaussian state—Space modeling of nonstationary time series. J. Am. Stat. Assoc.
1987, 82, 1032–1041.
Paluš, M. Cross-scale interactions and information transfer. Entropy 2014, 16, 5263–5289.
Papana, A.; Kugiumtzis, D.; Larsson, P. Reducing the bias of causality measures. Phys. Rev. E
2011, 83, 036207.
Porta, A.; de Maria, B.; Bari, V.; Marchi, A.; Faes, L. Are Nonlinear Model-Free Conditional
Entropy Approaches for the Assessment of Cardiac Control Complexity Superior to the Linear
Model-Based One? IEEE Trans. Biomed. Eng. 2017, 64, 1287–1296.

27

