Cogn Comput manuscript No.
(will be inserted by the editor)

Anatomical Pattern Analysis for decoding visual stimuli in human brains

arXiv:1710.02113v1 [stat.ML] 5 Oct 2017

Muhammad Yousefnezhad · Daoqiang Zhang

Received: 31 Mar 2017 / Accepted: 04 Oct 2017

Abstract Background: A universal unanswered question in
neuroscience and machine learning is whether computers
can decode the patterns of the human brain. Multi-Voxels
Pattern Analysis (MVPA) is a critical tool for addressing
this question. However, there are two challenges in the previous MVPA methods, which include decreasing sparsity
and noise in the extracted features and increasing the performance of prediction.
Methods: In overcoming mentioned challenges, this paper proposes Anatomical Pattern Analysis (APA) for decoding visual stimuli in the human brain. This framework develops a novel anatomical feature extraction method and a
new imbalance AdaBoost algorithm for binary classification. Further, it utilizes an Error-Correcting Output Codes
(ECOC) method for multiclass prediction. APA can automatically detect active regions for each category of the visual stimuli. Moreover, it enables us to combine homogeneous datasets for applying advanced classification.
Results and Conclusions: Experimental studies on 4 visual categories (words, consonants, objects and scrambled
photos) demonstrate that the proposed approach achieves superior performance to state-of-the-art methods.
Keywords brain decoding · multi-voxel pattern analysis ·
anatomical feature extraction · visual object recognition

1 Introduction
In order to decode visual stimuli in the human brain, MultiVoxel Pattern Analysis (MVPA) technique [31, 39, 40] must
apply machine learning methods to task-based functional
The authors are with the College of Computer Science and Technology,
Nanjing University of Aeronautics and Astronautics, Nanjing 211106,
China.
E-mail: myousefnezhad@nuaa.edu.cn; dqzhang@nuaa.edu.cn

Magnetic Resonance Imaging (fMRI) datasets. Indeed, analyzing the patterns of visual objects is one of the most interesting topics in MVPA, which can enable us to understand
how brain stores and processes the visual stimuli [13, 32].
Technically, there are two challenges in the previous studies. As the first issue, trained features are sparse and noisy
because most of the previous studies in whole-brain analysis
directly utilized raw voxels for predicting the stimuli [13, 32,
12]. As the second challenge, improving the performance
of prediction is so hard because task-based fMRI datasets
can be considered as the imbalanced classification problems.
For instance, consider collected data with 10 same size categories. Since this dataset is imbalance for (one-versus-all)
binary classification, most of the classical algorithms cannot
provide acceptable performance [13, 9, 25].
As the main contributions, this paper proposes Anatomical Pattern Analysis (APA) for decoding visual stimuli in
the human brain. To generate a normalized view, APA automatically detects the active regions and then extracts features based on the brain anatomical structures. Indeed, the
normalized view can enable us to combine homogeneous
datasets, and it can decrease noise, sparsity, and error of
learning. Further, this paper develops a modified version of
imbalance Adapting Boosting (AdaBoost) algorithm for binary classification. This algorithm uses a supervised random sampling and penalty values, which are calculated by
the correlation between different classes, for improving the
performance of prediction. This binary classification will be
used in a one-versus-all ECOC method as a multiclass approach for classifying the categories of the brain response.
The rest of this paper is organized as follows: The related
works are presented in Section 2. This paper introduces the
proposed method in Section 3. Experimental results are reported in Section 4; and finally, this paper presents conclusion and pointed out some future works in Section 5.

2

2 Related Works
There are three different types of studies for decoding stimuli in the human brain. Pioneer studies just focused on recognizing special regions of the human brain, such as inanimate objects [26], faces [20], visually illustration of words
[6], body parts [24], and visual objects [14]. Although they
proved that different stimuli can provide distinctive responses
in the brain regions, they cannot find the deterministic locations (or patterns) related to each category of stimuli.
The next group of studies developed correlation techniques in order to understand the similarity (or difference)
between distinctive stimuli. Haxby et al. employed brain
patterns located in Fusiform Face Area (FFA) and Parahippocampal Place Area (PPA) in order to analyze correlations
between different categories of visual stimuli, i.e. gray-scale
images of faces, houses, cats, bottles, scissors, shoes, chairs,
and scrambled (nonsense) photos [14]. Kamitani and Tong
studied the correlations of low-level visual features in the visual cortex (V1V4) [19]. In similar studies, Haynes et al. analyzed distinctive mental states [16] and more abstract brain
patterns such as intentions [17]. Kriegeskorte et al. proposed
Representational Similarity Analysis (RSA) in order to evaluate the similarities (or differences) among distinctive brain
states [22]. Connolly et al. utilized RSA in order to compare
the correlations between human brains and monkey brains
[7,8]. RSA demonstrates that the representations of each
category of stimuli in distinctive brain regions have a different structure [22, 7, 8]. Rice et al. proved that not only the
brain responses are different based on the categories of the
stimuli but also they are correlated based on different properties of the stimuli. They extracted the properties of visual
stimuli (photos of objects) and calculated the correlations
between these properties and the brain responses. They separately reported the correlation matrices for different human
faces and different objects (houses, chairs, etc.) [34].
The last group of studies proposed the MVPA techniques
for predicting the category of visual stimuli. Cox et al. utilized linear and non-linear versions of Support Vector Machine (SVM) algorithm [9]. In order to decode the brain patterns, some studies [2, 33, 37] employed classical feature selection (ranking) techniques, such as Principal Component
Analysis (PCA) [2], Linear Discriminant Analysis (LDA)
[33], or Independent Component Analysis (ICA) [37], that
these method are mostly used for analyzing rest-state fMRI
datasets. Recent studies proved that not only these techniques
cannot provide stable performance in the task-based fMRI
datasets [4, 5] but also they had spatial locality issue, especially when they were used for whole brain functional analysis [5]. Norman et al. argued for using SVM and Gaussian
Naive Bayes classifiers [31]. Kay et al. studied decoded orientation, position and object category from the brain activities in visual cortex [21]. Mitchell et al. introduced a new

Muhammad Yousefnezhad, Daoqiang Zhang

method in order to predict the brain activities associated
with the meanings of nouns [28]. Miyawaki et al. utilized a
combination of multiscale local image decoders in order to
reconstruct the visual images from the brain activities [29].
In order to generalize the testing procedure for task-based
fMRI datasets, Kriegeskorte et al. proved that the data in
testing must have no role in the procedure of generating an
MVPA model [23].
There are also some studies that focused on sparse learning techniques. Yamashita et al. developed Sparse Logistic
Regression (SLR) in order to improve the performance of
classification models [38]. Carroll et al. employed the Elastic Net for prediction and interpretation of distributed neural activity with sparse models [3]. Varoquaux et al. proposed a small-sample brain mapping by using sparse recovery on spatially correlated designs with randomization and
clustering. Their method is applied on small sets of brain
patterns for distinguishing different categories based on a
one-versus-one strategy [35].
As the first modern approaches for decoding visual stimuli, Anderson and Oates applied non-linear Artificial Neural
Network (ANN) on brain responses [1]. McMenamin et al.
studied subsystems underlie Abstract-Category (AC) recognition and priming of objects (e.g., cat, piano) and SpecificExemplar (SE) recognition and priming of objects (e.g., a
calico cat, a different calico cat, a grand piano, etc.). Technically, they applied SVM on manually selected ROIs in
the human brain for generating the visual stimuli predictors [27]. Mohr et al. compared four different classification
methods, i.e. L1/L2 regularized SVM, the Elastic Net, and
the Graph Net, for predicting different responses in the human brain. They show that L1-regularization can improve
classification performance while simultaneously providing
highly specific and interpretable discriminative activation patterns [30]. Osher et al. proposed a network (graph) based
approach by using anatomical regions of the human brain
for representing and classifying the different visual stimuli
responses (faces, objects, bodies, scenes) [32].
3 The Proposed Method
Blood Oxygen Level Dependent (BOLD) signals are used in
fMRI techniques for representing the neural activates. Based
on hyperalignment problem in the brain decoding [13, 15,
40], quantity values of the BOLD signals in the same experiment for the two subjects are usually different. Therefore,
MVPA techniques use the correlation between different voxels as the pattern of the brain response [32, 12]. As depicted
in Figure 1, each fMRI experiment includes a set of sessions (time series of 3D images), which can be captured by
different subjects or just repeating the imaging procedure
with a unique subject. Technically, each session can be partitioned into a set of visual stimuli categories. Indeed, an

Anatomical Pattern Analysis for decoding visual stimuli in human brains

3

Fig. 1 Anatomical Pattern Analysis (APA) framework

independent category denotes a set of homogeneous conditions, which are generated by using the same type of photos
as the visual stimuli. For instance, if a subject watches 6
photos of cats and 5 photos of houses during a unique session, this 4D image includes 2 different categories and 11
conditions.

(2)
This paper employs maximum functional activities in each
voxel as the k-th condition:

3.1 Feature Extraction
Preprocessed fMRI time series collected for U sessions can

b
be defined by F(`) = fmn ∈ RT ×V , ` = 1:U, m = 1:T , n =
1:Vb , where T is the number of time points, Vb denotes the
number of voxels in the original space, and fmn ∈ R defines
the functional activity for the `-th session in m-th time point
and n-th voxel. Indeed, 3D images (tensors) in fMRI time
series are considered vectorized for simplicity [19]. In addition, onsets (or time series) in the `-th session is defined as
follows:

S(`) ∈ RT ×P = S(`,1) , S(`,2) , . . . , S(`,k) , . . . , S(`,Q)

also defined as follows:
n
o
n
(`,k) b
(`) (`)
b (`,k) = cb(`,k)
C
∈ Rt ×V = fm. fm. ∈ F(`) , m ∈ S(`,k) ,
mn
o
S(`,k) = t (`,k) .

(1)

n t (`,k)
o>
(`,k)  (`,k)
e (`,k) ∈ RVb = max
b (`,k) , n ∈ [1, Vb ] .
C
cbmn cbmn ∈ C
m=1

(3)
In order to extract active voxels and then automatically define Region of Interests (ROIs), F(`) can be also written as a
general linear model:

>
b (`) + ζ (`) ,
(4)
F(`) = D(`) B
n o
(`)
where D(`) = dmn ∈ RT ×P denotes the design matrix, ζ (`)
n
o
(`)
b (`) = βbmn
is the noise (error of estimation), and also B
∈
RV ×P denotes the set of correlations for `-th session. Here,
(`)
(`)
b
d.n ∈ RT and βb.n ∈ RV respectively denote the design vector and voxel correlations belonged to n-th category of stimuli. Now, design matrix can be calculated by:
b

Here, P denotes the number of categories of visual stimulus,
(`,k)
Q ≥ P is the number of stimuli, the vector S(`,k) ∈ Rt
denotes the onsets belonged to k-th condition, and t (`,k) is the
number of time points for this condition, where t (`,k)  T .
By considering (1), all time points for k-th condition can be

D(`) = S(`) ∗ H

(5)

4

Muhammad Yousefnezhad, Daoqiang Zhang

where H is the Hemodynamic Response Function (HRF)
signal [12], and ∗ denotes the convolution operator. In order to solve (4), this paper uses Generalized Least Squares
(GLS) [12] approach as follows:
b (`) =
B



D


(`) >

Σ


(`) −1

D

(`)

−1

D


(`) >

Σ


(`) −1 (`)

>
,

F

(6)

where Σ (`) is the covariance matrix of the noise (var ζ (`) =
Σ (`) σ 2 6= Iσ 2 ) [13, 12, 41]. Further, activated voxels can be
defined as follows by using the positive values of the correlation matrix:
( (`)
(`)
 (`)
βbmn
βbmn > 0
(`)
V ×P
,
(7)
B = βmn ∈ R
=
0
otherwise
(`)
b (`) denotes the estimated correlation matrix
where βbmn ∈ B
(`)
in (5). Indeed, non-zero elements in β.n are all activated
voxels belonged to n-th category of stimuli. These activated
voxel correlations can be applied to the conditions as follows:
b
e (`,k) ◦ β.n(`)
C̄(`,k) ∈ RV = C

(8)

where ◦ denotes Hadamard product. Here, the k-th condition must be belonged to the n-th category of stimuli. Since
mapping whole of fMRI time series to standard space decreases the performance of the final results, most of the previous studies use the original images instead of the standard version. By considering (8) for each condition, this
paper enables to map brain activities to a standard space.
This mapping can provide normalized view for combing homogeneous datasets. For registering (8) to standard space,
this paper utilizes the fMRI Linear Image Registration Tool
(FLIRT) algorithm [18], which minimizes the following cost
function:


b
τ (`,k) ∈ RV ×V = arg min NMI C̄(`,k) , R
(9)
where R denotes the reference image, the function NMI is
the Normalized Mutual Information between two images,
τ (`,k) denotes the transformation matrix. The performance of
(9) will be analyzed in Section 4. Further, the final mapping
can be also defined as follows:
τ (`,k) :RV → RV =⇒ C(`,k) = τ (`,k) C̄(`,k)
b

(10)

 (`,k) (`,k)
(`,k)
(`,k)
(`,k)
, ci ∈ R, V
where C(`,k) = c1 , c2 , . . . , ci , . . . , cV
denotes the number of voxels in the standard space. In order to reduce the sparsity of C(`,k) , this paper employs an
anatomical atlas. Now, consider A = A1 , A2 , . . . , An , . . . AE
T
where E is the number of atlas regions, En=1 An = 0,
/
SE
n=1 An = A, and An denotes the set of indexes of voxels
for the n-th region. The extracted feature for `-th session in

Fig. 2 The proposed AdaBoost algorithm for applying a robust binary
classification

k-th condition and n-th anatomical region is calculated as
follows:
x(`,k,n) ∈ R =

1
An

(`,k)

∑ ci

(11)

i∈An

Finally, the extracted features
for each condition can be de
fined by X(`,k) ∈ RE = x(`,k,1) , x(`,k,2) , . . . , x(`,k,n) , . . . , x(`,k,E) ,
where E < V . Here, X` ∈ RE×Q denotes the extracted features for the `-th session. In addition, whole of dataset can
be defined by X ∈ RE×QU , where each column denotes the
extracted features for an individual stimulus.

3.2 Binary Classification Algorithm
In previous sections, we mentioned the imbalance issue in
the MVPA analysis. In practice, there are two approaches
in order to deal with this issue, i.e. designing an imbalance
classifier, or converting the imbalance problem to an ensemble of balance classification models. Previous studies
demonstrated that the performance of imbalance classifiers
may not be stable, especially when we have sparsity and
noise in our datasets [25, 41, 39, 40]. Since fMRI datasets
mostly include noise and sparsity, this paper has chosen the
ensemble approach. Technically, ensemble learning also contains two groups of solutions, i.e. bagging or boosting. While
bagging generates all classifiers at the same time and then
combine all of them as the final model, the boosting gradually creates each classifier in order to improve the performance of each iteration by tracing errors of previous iterations. We just have to note that ensemble learning can be
used in both balance and imbalance problems. In fact, the
main difference comes from the strategy of sampling. In balance problems, sampling methods are applied to the whole
of datasets, whereas instances of the large class are sampled
in the imbalance problems [25]. As depicted in Figure 2,
this paper presents a new branch of AdaBoost algorithm in
order to significantly improve the performance of the final
model in fMRI analysis. In a nutshell, this algorithm firstly
converts an imbalance MVPA problem to a set of balance

Anatomical Pattern Analysis for decoding visual stimuli in human brains

5

Algorithm 1 The proposed binary classification algorithm
e Class labels Y(m) .
Input: Training set X,
Output: Set of classifiers Θ (m) .
Method:
e = {X
eS, X
e L}
01. Based on Y(m) , partitioning X
02. Calculating J = int( |Xe L | ).
|XS |
 (1) (2)
e ,X
e ,...,X
e (n) , . . . , X
e (J) .
eL = X
03. Random sampling: X
L
L
L
L
(1)
(1)
04. Initiate X̄ = Ȳ = 0.
/
05. For (n = 1 . . . J):

e (n) = X
eS, X
e (n) , X̄(n) as training-set for this iteration.
06. X
Tr
L

(n)
e = Y
eS, Y
e (n) , Ȳ(n) as class labels for this iteration.
07. Y
Tr
L
(
e S or X̄(n)
1
for instances of X
08. W(n) =
(n) 
(n)
e
e
e
for instances of XL
1 − corr XS , XL

(n)
(n)
e ,Y
e , W(n) as weighted decision tree.
09. θ (n) = classifier X
e

Tr

10

Tr

Constructing X̄(n+1) as instances cannot truly trained in θ (n) .
ε (n) =

|X̄(n+1) |
e (n) |
|X

α (n)

1
2

as error of classification.

12.
= ln
AdaBoost weight for the classifier θ (n) .
13. End For



(n) (n) x
14. Return Θ (m) x = sign ∑J+1
as the final model.
n=1 α θ
11.

Tr

1−ε (n)
ε (n)

problems. Then, it iteratively applies the decision tree [25]
to each of these balance problems. Finally, AdaBoost is used
in order to generate the final model. In the proposed method,
the weight of each classifier (tree) for the final combination
is generated based on the error (failed predictions) of the
previous iterations for gradually improving the performance
of the final model.
In order to apply the binary classification, this paper randomly partitions the extracted features X into the training
e and the testing set X.
b As a new branch of AdaBoost
set X
e for training binary clasalgorithm, Algorithm 1 employs X
b is utilized for estimating the performance
sification. Then, X
of the final model. As mentioned before, the binary classification for fMRI analysis is mostly imbalance, especially by
using a one-versus-all strategy. Consequently, the number
of samples in one of these binary classes is smaller than the
other classes. As previously mentioned, this paper exploits
this concept in order to solve the imbalance issue. Indeed,
e into small
Algorithm 1 firstly partitions the training data X
e
e
XS and large XL classes (groups) based on the class labels
Y(m) ∈ + 1, −1 . Here, all labels are −1 except the label
of instances belong to m-th category of visual stimuli. Then,
it calculates the scale J of existed elements between two
classes. We have to note that int() defines the floor function.
As the next step, the large class is randomly partitioned into
J parts. Indeed, J is the number of balance subsets generated from the imbalance dataset. Consequently, the number
of the ensemble iteration is J. In each balance subset, traine (n) is generated by all instances of the small class
ing data X
Tr
e S , one of the partitioned parts of the large class X
e L , and
X
(n)
the instances of the previous iteration X̄ , which cannot
truly be trained (the failed predictions). After that, training

Fig. 3 The proposed Error-Correcting Output Codes (ECOC) approach for multiclass classification

weights for the final combination (Wn ∈ [0, 1]) are calculated by using the Pearson correlation (corr(a, b) = cov(a,b)
σa σb )
between training instances, where larger values increase the
learning sensitivity. Indeed, these weights are always maximized for the instances of the small class and the failed instances of the previous iterations. Further, the weights of the
other instances are a scale of the correlation between the
large class and the small class. Therefore, these weights are
updated in each iteration based on the performance of previous iterations. As the last step of each iteration, the proposed method generates a classification model (θ (n) ) and its
weight (α (n) ) for the final combination. While classifier()
can denote any kind of weighted classification algorithm,
this paper employs a simple weighted decision tree [25] as
the classification model. At the end, the final model is created by applying the AdaBoost method to the generated balance classifiers.

3.3 Multiclass Classification Algorithm
In this paper, a multiclass classifier is a prediction model
in order to map extracted features to thecategory of visual
b → Y pred where Y pred ∈ 1, 2, . . . , P . Genstimuli, i.e. Θ : X
erally, there are two techniques for applying multiclass classification. The first approach directly creates the classification model such as multiclass support vector machine [9]
or neural network [31]. In contrast, decomposition design
(indirect) uses an array of binary classifiers for solving the
multiclass problems.
Based on the previous discussion related to imbalance
issue in fMRI datasets, this paper utilizes Error-Correcting
Output Codes (ECOC) as an indirect multiclass approach in
order to extend the proposed binary classifier for the multiclass prediction. As depicted in Figure 3, ECOC includes

6

Muhammad Yousefnezhad, Daoqiang Zhang

Table 1 tbl:datasets
Title
Visual Object Recognition
Word and Object Processing
Multi-subject, multi-modal

ID
DS105
DS107
DS117

S
6
49
20

U
71
98
171

P
8
4
2

T
121
164
210

X
79
53
64

Y
79
63
61

Y
75
52
33

Scanner
G 3T
S 3T
S 3T

TR
2500
2000
2000

TE
30
28
30

S denotes the number of subject; U is the number of sessions; P denotes the number of stimulus categories; T is the number of scans in unites
of TRs (Time of Repetition); X,Y, Z are the size of 3D images in the original space; Scanners include S =Siemens, and G =General Electric in 3
Tesla; TR is Time of Repetition in millisecond; TE denotes Echo Time in millisecond; Please see openfmri.org for more information.

three components, i.e. base algorithms, coding matrix and
decoding procedures [11]. Since this paper uses one-versusall encoding strategy, Algorithm 1 is employed as the based
algorithms (Θ (m) ) in the ECOC, where it generates a binary
classifier for each category of visual stimuli. In other words,
each independent category of the visual stimuli is compared
with the rest of categories. Consequently, the size of the coding matrix is P × P, where i-th diagonal cell of this matrix
represents the positive predictions belong to the i-th category of visual stimuli and the rest of cells in this matrix determine the other categories of visual stimuli. Indeed, the
number of classifiers in this strategy is exactly equal to the
number of categories. As decoding stage, binary predictions,
which are generated by applying the brain response to the
base algorithms, are assigned to the category in the coding
matrix with closest Hamming distance.
In order to present an example for ECOC procedure,
consider fMRI dataset with 4 categories of visual stimuli,
i.e. photos of shoes, houses, bottles, and human faces. In
this problem, 4 different binary classifiers must be trained
in order to distinguish each category of visual stimuli versus
the rest of them (one-versus-all strategy). A 4 × 4 coding
matrix is also generated where each diagonal element represents the positive class of these categories (classifiers). By
considering the order of the coding matrix, each prediction
is assigned to the closest Hamming distance in the coding
matrix. In other words, if these classifiers generate the prediction [+1, −1, −1, −1] for a testing instance, then this instance definitely belongs to the first category of visual stimuli. Similarly, the prediction [−1, +1, −1, −1] means the instance belongs to the second category, and etc.
4 Results
4.1 Datasets
As depicted in Table 1, this paper employs 3 datasets, shared
by openfmri.org, for running empirical studies. As the
first dataset, ‘Visual Object Recognition’ (DS105) includes
P = 8 categories of visual stimuli, i.e. gray-scale images
of faces, houses, cats, bottles, scissors, shoes, chairs, and
scrambled (nonsense) photos. This dataset is analyzed in
high-level visual stimuli as the binary predictor, by considering all categories except scrambled photos as objects, and

low-level visual stimuli in the multiclass prediction. Please
see [13, 9] for more information. As the second dataset, ‘Word
and Object Processing’ (DS107) contains P = 4 categories
of visual stimuli, i.e. words, objects, scrambles, and consonants. Please see [10] for more information. As the last
data set, ‘Multi-subject, multi-modal human neuroimaging
dataset’ (DS117) includes MEG and fMRI images. This paper just uses the fMRI images of this dataset. It also contains P = 2 categories of visual stimuli, i.e. human faces,
and scrambles. Please see [36] for more information.
These datasets are preprocessed by SPM 12 (www.fil.
ion.ucl.ac.uk/spm/), i.e. slice timing, realignment, normalization, smoothing. Further, whole-brain functional alignment is applied based on [5]. Then, the beta values are calculated for each session. This paper employs the MNI 152 T1
1mm (see Figure 1.d) as the reference image (R) in (9) for
registering the extracted conditions (C̄(`,k) ) to the standard
space (C(`,k) ). In addition, this paper uses Talairach Atlas
(contains regions) in (11) for extracting features.

4.2 Parameter Analysis
The registration objective function in (9) will be analyzed in
this section by using different distance metrics, i.e. Woods
function (W), Correlation Ratio (CR), Joint Entropy (JE),
Mutual Information (MI), and Normalized Mutual Information (NMI) [41, 18]. Figures 4.a-c demonstrate examples of
brain responses to different stimuli, i.e. (a) word, (b) object, and (c) scramble. Here, gray parts show the anatomical atlas, the colored parts (red, yellow and green) define
the functional activities, and also the red rectangles illustrate the error areas after registration. Indeed, these errors
can be formulated as the nonzero areas in the brain image
which are located in the zero area of the anatomical atlas
(the area without region number). Indeed, the registration
errors are mostly related to the distance metrics. Previous
studies illustrated that the entropy-based metrics can provide better performance [41, 18]. The performances of objective function (9) on DS105, DS107, and DS117 datasets
are analyzed in Figure 4.d by using the mentioned distance
metrics. As depicted in this figure, the entropy-based metrics
(JE, MI, NMI) have provided better performance in comparison with other metrics. Since NMI uses normalization for

Anatomical Pattern Analysis for decoding visual stimuli in human brains

(A)

(B)

7

(C)

(D)

Fig. 4 Extracted features based on different stimuli, i.e. (A) word, (B) object, and (C) scramble. (D) The effect of different objective functions in
(4) on the error of registration.

(A)

(B)
DS105

(C)
(D)
DS107

(E)
(F)
DS117

Fig. 5 The correlation matrices: (A) raw voxels and (B) extracted features in the DS105 dataset, (C) raw voxels and (D) extracted features
in the DS107 dataset, (E) raw voxels and (F) extracted features in the
DS117 dataset.

removing the scaling effect [41], it generated the best results
among the entropy-based metrics. Therefore, this paper employs NMI as the distance metric in 9 for mapping the brain
activities from the original space to the standard space.

4.3 Correlation Analysis
The correlations of the extracted features will be compared
with the correlations of the original voxels in this section.
Previous studies illustrated that patterns of different AbstractCategories (ACs), which is extracted from a suitable feature

representation, must provide distinctive correlation values
[27, 34]. Therefore, the main assumption in this section is
that better feature representation (extraction) can improve
the correlation analysis, where the correlation between different categories of visual stimuli must be significantly
smaller than the correlation between stimuli belonged to the
same category. In order to provide a better perspective, the
extracted features are compared by considering two different
levels. At the first level, the feature space is compared with
the whole of raw voxels in the original space, where this
comparison analyzes the correlation between whole-brain
data and automatically detected ROIs. At the second level,
the correlation values among different ACs are compared in
the feature space that it shows how much the feature space
is well-designed.
Figure 5.A, B, and E illustrate the correlation matrix of
the DS105, DS107, and DS117 at the raw voxel space, respectively. Furthermore, Figure 5.B, D, and F respectively
show the correlation matrix the DS105, DS107, and DS117
in the feature space. As these figures depicted, different ACs
are highly correlated in the voxel space. Indeed, the average
of correlations is around +0.5 in DS105 and DS107. And,
this average is around +0.8 in DS117 because this dataset
just includes 2 classes (photos of scramble and human face).
The main reason for these results is that brain responses
in the voxel space are sparse, high-dimensional and noisy.
Therefore, it is so hard to discriminate between different categories (ACs) in the original space, especially when wholebrain data will be analyzed. By contrast, the feature space
(Figure 5.B, D, and E) provides distinctive representation
when the proposed method used the correlated patterns in
each anatomical region as the extracted features.
The correlation between different ACs can be also meaningful in the feature space. In DS105 and DS107, the scramble (nonsense) stimuli have a low correlation (> 0.13) in
comparison with sensible categories. As another example
in DS105, human faces are mostly correlated to the photos of cats and houses (respectively +0.44, +0.25) in comparison with other objects (the average of correlations is
+0.14). Another interesting example is the correlation between meaningful stimuli (words and objects) and nonsense

8

Muhammad Yousefnezhad, Daoqiang Zhang

Table 2 Accuracy of binary predictors (mean±std)
↓Alg., Datasets→
SVM [9]
Elastic Net [30]
Graph Net [30]
PCA [33]
ICA [37]
Selected ROI [27]
L1 Reg. SVM [30]
Graph-based [32]
PCA + Algorithm 1
ICA + Algorithm 1
APA + SVM
Binary APA

DS105 (Objects)
71.65±0.97
80.77±0.61
79.23±0.74
72.15±0.76
73.25±0.81
83.06±0.36
85.29±0.49
90.82±1.23
83.61±0.97
84.41±0.93
76.32±0.78
98.97±0.12

DS107 (Words)
69.89±1.02
78.26±0.79
79.91±0.91
70.32±0.92
70.82±0.67
89.62±0.52
81.14±0.91
94.21±0.83
80.12±0.81
82.21±0.86
77.19±0.83
98.17±0.36

DS107 (Consonants)
67.84±0.82
75.53±9.87
74.01±0.84
69.57±1.10
71.87±0.94
87.82±0.37
79.69±0.69
95.54±0.99
79.47±0.91
78.88±0.78
78.61±0.91
98.72±0.16

DS107 (Objects)
65.32±1.67
84.15±0.89
85.96±0.76
68.78±0.64
67.99±0.75
84.22±0.44
75.32±0.41
95.62±0.83
82.80±1.01
82.30±0.99
69.22±0.87
95.26±0.92

DS107 (Scramble)
67.96±0.87
87.34±0.93
86.21±0.51
69.41±0.35
72.48±0.89
86.19±0.26
78.45±0.62
93.10±0.78
80.52±0.98
83.99±0.84
73.52±0.99
97.23±0.76

DS117
81.25±1.03
86.49±0.70
85.49±0.88
81.92±0.87
80.71±1.16
85.19±0.56
85.46±0.29
86.61±0.61
86.27±0.88
85.57±1.10
89.90±0.72
96.81±0.79

DS107 (Objects)
63.17±0.59
81.06±0.98
82.08±0.92
67.56±0.59
66.91±0.97
81.54±0.92
73.92±0.28
94.23±0.94
81.76±0.12
81.04±0.81
68.14±1.02
94.92±0.11

DS107 (Scramble)
66.73±0.92
85.54±0.81
83.97±0.97
68.89±0.90
70.20±0.72
85.79±0.42
76.14±0.47
92.23±0.38
77.64±0.84
83.02±0.92
71.07±0.79
97.21±0.92

DS117
79.36±0.33
83.42±0.68
81.67±0.74
79.61±0.72
78.39±0.96
83.71±0.81
83.21±1.23
82.29±0.91
84.32±0.72
82.37±0.88
85.10±0.93
94.08±0.84

Table 3 Area under the ROC Curve (AUC) of binary predictors (mean±std)
↓Alg., Datasets→
SVM [9]
Elastic Net [30]
Graph Net [30]
PCA [33]
ICA [37]
Selected ROI [27]
L1 Reg. SVM [30]
Graph-based [32]
PCA + Algorithm 1
ICA + Algorithm 1
APA + SVM
Binary APA

DS105 (Objects)
68.37±1.01
78.23±0.82
77.26±0.72
70.69±0.84
71.33±0.85
82.22±0.42
80.91±0.21
88.54±0.71
81.76±0.90
81.11±0.72
72.27±0.86
97.06±0.82

DS107 (Words)
67.76±0.91
77.94±0.76
78.31±0.97
69.37±0.77
68.86±0.93
86.35±0.39
78.23±0.57
93.61±0.62
78.91±0.88
80.92±0.58
73.59±1.04
97.31±0.82

DS107 (Consonants)
63.84±1.45
74.11±0.82
71.43±0.58
65.12±0.93
71.03±1.07
85.63±0.61
77.41±0.92
94.54±0.31
77.44±0.93
75.76±0.98
76.95±0.94
96.21±0.62

Table 4 Accuracy of multiclass predictors (mean±std)
↓Alg., Datasets→
Multiclass SVM [9]
MLP [1]
Selected ROI [27]
Graph-based [32]
Multiclass APA

DS105 (# of classes = 8)
18.03±4.07
38.34±3.21
28.72±2.37
50.61±4.83
59.21±2.05

DS107 (# of classes = 4)
38.01±2.56
71.55±2.79
68.51±1.07
89.69±2.32
95.61±1.83

stimuli (scrambles and consonants) in DS107, where the meaningful stimuli are highly correlated (+0.8) and their correlations with nonsense stimuli are negative (the average of
correlation is −0.65). Since DS117 is a binary dataset, it is
really a good example in order to understand the negative
effect of noise and sparsity in fMRI analysis. The correlation between the face category and scramble is around +0.8
in the raw voxel space, whereas this correlation is +0.23 in
the feature space. Indeed, the noisy and sparse raw voxels
are not suitable (wise) in order to train a high-performance
cognitive model.
Here, we have to note that a suitable feature representation can also improve the performance of the MVPA analysis (the final cognitive model). By considering the geometric analysis of a linear space, classification algorithms draw
a hyperplane (that is known as the decision surface in neuroscience [13]) in a representational space in order to distinguish different classes (categories of visual stimuli). In
a highly correlated space, the margin of error for this hyperplane is sensitive. In other words, small changes in the

ABSTRACT (# of classes = 5)
31.77±2.61
67.24±3.72
54.19±2.80
78.96±3.32
95.85±1.05

ALL (# of classes = 10)
12.26±5.97
32.94±4.89
35.03±2.66
47.64±5.28
62.93±2.69

parameters of the hyperplane can rapidly reduce the performance of the classifier in a highly correlated space, such
as the raw voxel space [4, 5, 41]. Now, suitable feature extraction can minimize the correlation between different categories of visual stimuli. As a result, the margin of error and
stability of the final model can be increased in order to train
a classifier.

4.4 Performance Analysis
In this section, the performance of different methods will
be evaluated for both binary and multiclass analyses. In the
binary analysis, the performance of the classical binary Support Vector Machine (SVM) is represented. Indeed, this method
is used in [9] in order to distinguish different categories of
visual stimuli. As regularized methods that are introduced
in [30] for decoding the brain patterns, the performances of
L1 regularized SVM, the Elastic Net, and the Graph Net are
also reported in this section. Further, the performances of
component based methods are also evaluated, i.e. Principal

Anatomical Pattern Analysis for decoding visual stimuli in human brains

Component Analysis (PCA) that is used in [33] for training a cognitive model and Independent Component Analysis (ICA), which is employed in [37] in order to analyze
fMRI datasets. As another alternative for decoding visual
stimuli, the Selected Region of Interest (ROI) method [27]
is reported in this section, where the ROIs for each dataset is
manually selected same as the original paper [27] and then
the SVM classifier is applied to the selected ROIs in order
to train a cognitive model. As the method was developed in
[32], the performance of a graph-based approach is reported.
In order to represent the effect of different parts of the proposed method, we also report three baselines. As the first
alternatives, we utilize two component based methods, i.e.
‘PCA + Algorithm 1’ and ‘ICA + Algorithm 1’ that apply
Algorithm 1 to the features that are respectively extracted by
PCA and ICA. As the last baseline, ‘APA + SVM’ applies
SVM algorithm to the features that are extracted by APA. In
the multiclass analysis, the performance of multiclass SVM
is presented as a baseline, where this algorithm was used in
[9] in order to generate the cognitive model. Further, the performance of the proposed method is compared with Multilayer Perceptron (MLP) that was introduced as a multiclass
approach in [1] in order to decode the brain patterns. Selected ROI method [27] and the graph-based approach [32]
are also reported as other alternatives. We have to note that a
multiclass SVM is used in the Selected ROI method in order
to create a multiclass cognitive model (in Table 4). All of
the mentioned algorithms are implemented in the MATLAB
R2016b (9.1) by authors in order to generate experimental
results. Further, all evaluations are applied by using leaveone-subject-out cross validation, e.g. we have selected brain
patterns of 5 subjects in DS105 for training a classifier in
each iteration and then used the patterns of the rest of the
subject in order to test the generated cognitive model. It is
worth noting that the number of iterations will be equal to
the number of subjects (5 in this example). Indeed, not only
the brain patterns in training sets and testing sets are independent across subjects but also fMRI data related to each
subject was separately preprocessed [23]. We have to note
that the same training set and testing set are applied in each
iteration to all of the evaluated methods.
Tables 2 and 3 respectively illustrate the classification
Accuracy and Area under the ROC Curve (AUC) for the binary predictors based on the category of the visual stimuli.
All visual stimuli in the dataset DS105 except scrambled
photos are considered as the object category for generating
these experimental results. As these tables depicted, SVM
cannot create an acceptable performance on the raw voxels because fMRI data in the original space includes noise
and sparsity. Moreover, the performances of the componentbased approaches (PCA and ICA) are significantly low because they were applied to whole-brain. Indeed, these methods are suitable for ROI-based problems or rest-mode fMRI

9

data, where they can find the best projection among the wised
voxels. Another evidence for this claim is the Selected ROI
method, where the performance of this method is significantly improved in comparison with the component-based
approaches. In fact, this is the main reason in this paper in
order to develop the automatically selected ROI method instead of just applying the component-based methods to the
whole-brain data for selecting (ranking) the effective features. As also mentioned in the original paper [30], L1 regularized SVM generated better results in comparison with
other regularized techniques, i.e. Elastic Net and Graph Net.
As another alternative, the graph-based method that is developed by Osher et al. [32] generated acceptable performance
because it also employed the anatomical features in order to
create a cognitive model. Further, ‘PCA/ICA + Algorithm
1’ have generated better performances in comparison with
PCA/ICA methods because of the ensemble approach. Since
‘APA + SVM’ uses better representational space in contrast
with the raw fMRI data, it significantly improves the performance of SVM method. Although these baselines can
show that each part of the proposed method can generate
better performance in comparison with the classical algorithms (PCA, ICA, and SVM), the best results will be produced when we use all parts at the same time. The last but
not least, the proposed algorithm has achieved the best performance in comparison with other methods because it provided a better representation of neural activities by exploiting the anatomical structure of the human brain.
Table 4 illustrates the classification accuracy for multiclass predictors. In this table, ‘DS105’ includes P = 8 different categories (classes) and ‘DS107’ contains P = 4 categories of the visual stimuli. This paper also combined three
datasets in two distinctive forms. ‘ABSTRACT’ includes 5
different categories, i.e. words, objects, scrambles, consonants, and human faces, which is generated by considering all visual stimuli in the dataset DS105 except faces and
scrambled photos as object category and combining them
with the datasets DS107 and DS117. Indeed, this combined
dataset can be considered for comparing the abstract features of visual stimuli in the human brain. As another alternative, ‘ALL’ in this table generated by combining all of
the visual stimuli in the three datasets, i.e. faces, houses,
cats, bottles, scissors, shoes, chairs, words, consonants, and
scrambled photos. As depicted in Table 4, the accuracy of
the proposed method is improved by combining three datasets,
whereas the performances of other methods are significantly
decreased. As mentioned before, it is the standard space
registration problem in the fMRI analysis. In addition, our
framework employs the extracted features from the structural regions instead of using all or a subgroup of voxels,
which can increase the performance of the predictive models by decreasing noise and sparsity.

10

Muhammad Yousefnezhad, Daoqiang Zhang

to different brain tasks such as low-level visual stimuli, emotion and etc.

Compliance with Ethical Standards
Conflict of Interests
Muhammad Yousefnezhad and Daoqiang Zhang declare that
they have no conflict of interest.

Ethical Approval

Fig. 6 Automatically detected active regions (with Probability greater
than 50%) across abstract categories of visual stimuli, which is generated by combining 3 datasets, i.e. DS105, DS107, and DS117.

5 Discussions and Conclusions
Anatomical Pattern Analysis (APA) can be used by neuroscientist in order to seek most effective active voxels (regions) across abstract categories of visual stimuli in both a
singular dataset and the combined datasets. Figure 6 illustrates an example for these active voxels by using the ABSTRACT dataset in Table 4 that was generated by combining the visual stimuli in the 3 datasets, i.e. DS105, DS107,
and DS117. In this figure, active regions (B(`) ) for abstract
categories (faces, scrambles, and objects) are normalized in
the standard space and then the active voxels with probability greater than 50%
h are visualized as the
i automatically
(`)
detected ROIs, i.e. Pr ∑U`=1 τ (`) β.n > 50% , where U is the
number of all sessions in the combined dataset. As this figure depicted, not only APA can generate a cognitive model
in order to predict visual stimuli in the human brain but also
it can automatically demonstrate activated loci in human
brain across categories of visual stimuli. These activated loci
can be used in order to study Specific-Exemplar (SE) recognition [27] or design an accurate brain mask (ROI) for ROIbased studies [41].
In summary, this paper proposes APA framework for decoding visual stimuli in the human brain. This framework
uses an anatomical feature extraction method, which provides a normalized representation for combining homogeneous datasets. Further, a new binary imbalance AdaBoost
algorithm is introduced. It can increase the performance of
prediction by exploiting a supervised random sampling and
the correlation between classes. In addition, this algorithm is
utilized in an Error-Correcting Output Codes (ECOC) method
for multiclass prediction of the brain responses. Empirical
studies on 4 visual categories clearly show the superiority
of our proposed method in comparison with the voxel-based
approaches. In future, we plan to apply the proposed method

This article does not contain any studies with human participants or animals performed by any of the authors.
Acknowledgements This work was supported in part by the National
Natural Science Foundation of China (61422204 and 61473149), and
NUAA Fundamental Research Funds (NE2013105).

References
1. Anderson, M., Oates, T.: A critique of multi-voxel pattern analysis. In: Proceedings of the Cognitive Science Society, vol. 32
(2010)
2. Carlson, T.A., Schrater, P., He, S.: Patterns of activity in the categorical representations of objects. Journal of Cognitive Neuroscience 15(5), 704–717 (2003)
3. Carroll, M.K., Cecchi, G.A., Rish, I., Garg, R., Rao, A.R.: Prediction and interpretation of distributed neural activity with sparse
models. NeuroImage 44(1), 112–122 (2009)
4. Chen, P.H., Chen, J., Yeshurun, Y., Hasson, U., Haxby, J., Ramadge, P.J.: A reduced-dimension fmri shared response model. In:
28th Advances in Neural Information Processing Systems (NIPS15), pp. 460–468. Advances In Neural Information Processing
Systems (NIPS), December/7–12, Montral, Canada (2015)
5. Chen, P.H., Zhu, X., Zhang, H., Turek, J.S., Chen, J., Willke,
T.L., Hasson, U., Ramadge, P.J.: A convolutional autoencoder for
multi-subject fmri data aggregation. In: 29th Workshop of Representation Learning in Artificial and Biological Neural Networks.
Advances In Neural Information Processing Systems (NIPS),
December/5–10, Barcelona, Spain (2016)
6. Cohen, L., Dehaene, S., Naccache, L., Lehéricy, S., DehaeneLambertz, G., Hénaff, M.A., Michel, F.: The visual word form
area: spatial and temporal characterization of an initial stage of
reading in normal subjects and posterior split-brain patients. Brain
123(2), 291–307 (2000)
7. Connolly, A., Gobbini, M., Haxby, J.: Three virtues of similaritybased multi-voxel pattern analysis (2012)
8. Connolly, A.C., Guntupalli, J.S., Gors, J., Hanke, M., Halchenko,
Y.O., Wu, Y.C., Abdi, H., Haxby, J.V.: The representation of biological classes in the human brain. Journal of Neuroscience 32(8),
2608–2618 (2012)
9. Cox, D.D., Savoy, R.L.: Functional magnetic resonance imaging
(fmri) ‘brain reading’: detecting and classifying distributed patterns of fmri activity in human visual cortex. NeuroImage 19(2),
261–270 (2003)
10. Duncan, K.J., Pattamadilok, C., Knierim, I., Devlin, J.T.: Consistency and variability in functional localisers. NeuroImage 46(4),
1018–1026 (2009)

Anatomical Pattern Analysis for decoding visual stimuli in human brains
11. Escalera, S., Pujol, O., Radeva, P.: Error-correcting output codes
library. Journal of Machine Learning Research 11(Feb), 661–664
(2010)
12. Friston, K.J.: Statistical parametric mapping. In: Neuroscience
Databases, pp. 237–250. Springer (2003)
13. Haxby, J.V., Connolly, A.C., Guntupalli, J.S.: Decoding neural
representational spaces using multivariate pattern analysis. Annual Review of Neuroscience 37, 435–456 (2014)
14. Haxby, J.V., Gobbini, M.I., Furey, M.L., Ishai, A., Schouten, J.L.,
Pietrini, P.: Distributed and overlapping representations of faces
and objects in ventral temporal cortex. Science 293(5539), 2425–
2430 (2001)
15. Haxby, J.V., Guntupalli, J.S., Connolly, A.C., Halchenko, Y.O.,
Conroy, B.R., Gobbini, M.I., Hanke, M., Ramadge, P.J.: A common, high-dimensional model of the representational space in human ventral temporal cortex. Neuron 72(2), 404–416 (2011)
16. Haynes, J.D., Rees, G.: Decoding mental states from brain activity
in humans. Nature Reviews Neuroscience 7(7), 523 (2006)
17. Haynes, J.D., Sakai, K., Rees, G., Gilbert, S., Frith, C., Passingham, R.E.: Reading hidden intentions in the human brain. Current
Biology 17(4), 323–328 (2007)
18. Jenkinson, M., Bannister, P., Brady, M., Smith, S.: Improved optimization for the robust and accurate linear registration and motion
correction of brain images. Neuroimage 17(2), 825–841 (2002)
19. Kamitani, Y., Tong, F.: Decoding the visual and subjective contents of the human brain. Nature Neuroscience 8(5), 679–685
(2005)
20. Kanwisher, N., McDermott, J., Chun, M.M.: The fusiform face
area: a module in human extrastriate cortex specialized for face
perception. Journal of Neuroscience 17(11), 4302–4311 (1997)
21. Kay, K.N., Naselaris, T., Prenger, R.J., Gallant, J.L.: Identifying
natural images from human brain activity. Nature 452(7185), 352
(2008)
22. Kriegeskorte, N., Mur, M., Bandettini, P.: Representational similarity analysis–connecting the branches of systems neuroscience.
Frontiers in systems neuroscience 2 (2008)
23. Kriegeskorte, N., Simmons, W.K., Bellgowan, P.S., Baker, C.I.:
Circular analysis in systems neuroscience: the dangers of double
dipping. Nature Neuroscience 12(5), 535–540 (2009)
24. Liesegang, T.J.: A cortical area selective for visual processing
of the human body. downing pe, 1 school of psychology, centre
for cognitive neuroscience, university of wales, bangor, ll57 2as,
united kingdom. e-mail: p. downing@ bangor. ac. uk jiang y, shuman m, kanwisher n. science 2001; 293: 2470–2473. American
Journal of Ophthalmology 133(4), 598 (2002)
25. Liu, X.Y., Wu, J., Zhou, Z.H.: Exploratory undersampling for
class-imbalance learning. IEEE Transactions on Systems, Man,
and Cybernetics, Part B (Cybernetics) 39(2), 539–550 (2009)
26. Malach, R., Reppas, J., Benson, R., Kwong, K., Jiang, H.,
Kennedy, W., Ledden, P., Brady, T., Rosen, B., Tootell, R.: Objectrelated activity revealed by functional magnetic resonance imaging in human occipital cortex. Proceedings of the National
Academy of Sciences (PNAS) 92(18), 8135–8139 (1995)
27. McMenamin, B.W., Deason, R.G., Steele, V.R., Koutstaal, W.,
Marsolek, C.J.: Separability of abstract-category and specificexemplar visual object subsystems: Evidence from fmri pattern
analysis. Brain and Cognition 93, 54–63 (2015)
28. Mitchell, T.M., Shinkareva, S.V., Carlson, A., Chang, K.M.,
Malave, V.L., Mason, R.A., Just, M.A.: Predicting human brain
activity associated with the meanings of nouns.
science
320(5880), 1191–1195 (2008)
29. Miyawaki, Y., Uchida, H., Yamashita, O., Sato, M.a., Morito, Y.,
Tanabe, H.C., Sadato, N., Kamitani, Y.: Visual image reconstruction from human brain activity using a combination of multiscale
local image decoders. Neuron 60(5), 915–929 (2008)
30. Mohr, H., Wolfensteller, U., Frimmel, S., Ruge, H.: Sparse regularization techniques provide novel insights into outcome integration processes. NeuroImage 104, 163–176 (2015)

11
31. Norman, K.A., Polyn, S.M., Detre, G.J., Haxby, J.V.: Beyond
mind-reading: multi-voxel pattern analysis of fmri data. Trends
in Cognitive Sciences 10(9), 424–430 (2006)
32. Osher, D.E., Saxe, R.R., Koldewyn, K., Gabrieli, J.D., Kanwisher,
N., Saygin, Z.M.: Structural connectivity fingerprints predict cortical selectivity for multiple visual categories across cortex. Cerebral Cortex 26(4), 1668–1683 (2015)
33. O’toole, A.J., Jiang, F., Abdi, H., Haxby, J.V.: Partially distributed
representations of objects and faces in ventral temporal cortex.
Journal of Cognitive Neuroscience 17(4), 580–590 (2005)
34. Rice, G.E., Watson, D.M., Hartley, T., Andrews, T.J.: Low-level
image properties of visual objects predict patterns of neural response across category-selective regions of the ventral visual pathway. Journal of Neuroscience 34(26), 8837–8844 (2014)
35. Varoquaux, G., Gramfort, A., Thirion, B.: Small-sample brain
mapping: sparse recovery on spatially correlated designs with randomization and clustering. In: Proceedings of the 29th International Conference on Machine Learning (ICML-12), pp. 1375–
1382 (2012)
36. Wakeman, D.G., Henson, R.N.: A multi-subject, multi-modal human neuroimaging dataset. Scientific Data 2 (2015)
37. Xu, J., Potenza, M.N., Calhoun, V.D.: Spatial ica reveals functional activity hidden from traditional fmri glm-based analyses.
Frontiers in Neuroscience 7 (2013)
38. Yamashita, O., Sato, M.a., Yoshioka, T., Tong, F., Kamitani, Y.:
Sparse estimation automatically selects voxels relevant for the decoding of fmri activity patterns. NeuroImage 42(4), 1414–1429
(2008)
39. Yousefnezhad, M., Zhang, D.: Decoding visual stimuli in human
brain by using anatomical pattern analysis on fmri images. In:
8th International Conference on Brain Inspired Cognitive Systems
(BICS’16), pp. 47–57. Springer, November/28–30, Beijing, China
(2016)
40. Yousefnezhad, M., Zhang, D.: Local discriminant hyperalignment
for multi-subject fmri data alignment. In: 34th AAAI Conference
on Artificial Intelligence (AAAI-17), pp. 59–65. Association for
the Advancement of Artificial Intelligence (AAAI), February/4–
9, San Francisco, California, USA (2017)
41. Yousefnezhad, M., Zhang, D.: Multi-region neural representation:
A novel model for decoding visual stimuli in human brains. In:
17th SIAM International Conference on Data Mininig (SDM17), pp. 54–62. Society for Industrial and Applied Mathematics
(SIAM), April/27–29, Houston, Texas, USA (2017)

