Automatic segmenting teeth in X-ray images: Trends, a
novel data set, benchmarking and future perspectives
Gil Silva1 , Luciano Oliveira2

arXiv:1802.03086v1 [] 9 Feb 2018

Ivision Lab, Federal University of Bahia, Brazil

Matheus Pithon3
Southeast State University of Bahia, Brazil

Abstract
This review presents an in-depth study of the literature on segmentation
methods applied in dental imaging. Ten segmentation methods were studied
and categorized according to the type of the segmentation method (region-based,
threshold-based, cluster-based, boundary-based or watershed-based), type of Xray images used (intra-oral or extra-oral) and characteristics of the dataset used
to evaluate the methods in the state-of-the-art works. We found that the literature has primarily focused on threshold-based segmentation methods (54%).
80% of the reviewed papers have used intra-oral X-ray images in their experiments, demonstrating preference to perform segmentation on images of already
isolated parts of the teeth, rather than using extra-oral X-rays, which show
tooth structure of the mouth and bones of the face. To fill a scientific gap in
the field, a novel data set based on extra-oral X-ray images are proposed here.
A statistical comparison of the results found with the 10 image segmentation
methods over our proposed data set comprised of 1,500 images is also carried
out, providing a more comprehensive source of performance assessment. Discussion on limitations of the methods conceived over the past year as well as
future perspectives on exploiting learning-based segmentation methods to improve performance are also provided.
Keywords: image segmentation, dental X-ray, orthopantomography

1. Introduction
In dentistry, radiographic images are fundamental data sources to aid diagnosis. Radiography is the photographic record of an image produced by the
1 gil.jader@gmail.com
2 lrebouca@ufba.br
3 matheuspithon@gmail.com

passage of an X-ray source through an object (Quinn and Sigl, 1980). X-ray
images are used in dental medicine to check the condition of the teeth, gums,
jaws and bone structure of a mouth (Quinn and Sigl, 1980). Without X-rays,
Dentists would not be able to detect many dental problems until they become
severe. This way, the radiographic examination helps the dentist to discover the
cause of the problem at an early stage, allowing then to outline the best treatment plan for the patient. Another application of dental X-rays is in the field
of forensic identification, especially in cadavers (Paewinsky et al., 2005). The
forensic dentistry aims to identify individuals based on their dental characteristics. In recent years, forensic literature has also provided automatic methods to
assessing person’s age from degenerative changes in teeth (Willems et al., 2002).
These age-related changes can be assessed by digital radiography (Paewinsky
et al., 2005). With the advancement of artificial intelligence and pattern recognition algorithms, X-ray images have been increasingly used as an input to these
intelligent algorithms. In this context, we highlight here an in-depth study over
some segmentation methods in the literature that are regarded to the recognition of image patterns in dental X-rays.
1.1. Overview of dental image segmentation
In dentistry, X-rays are divided into two categories: (i) Intra-oral radiographic examinations are techniques performed with the film positioned in the
buccal cavity (the X-ray image is obtained inside the patient’s mouth); and (ii)
extra-oral radiographic examinations are the techniques in which the patient is
positioned between the radiographic film and the source of X-rays (the X-ray
image is obtained outside the patient’s mouth) (Association, 1987).
In this paper, some works that use segmentation methods applied to the
following types of X-ray images are analyzed: bitewing and periapical (intraoral), and panoramic (extra-oral). The bitewing X-ray images are used to show
details of the upper and lower teeth in a mouth region, while the periapical
X-ray images is used to monitor the entire tooth (Wang et al., 2016). On the
other hand, panoramic radiography, also known as orthopantomography, is
one of the radiological exams capable of obtaining fundamental information for
the diagnosis of anomalies in dental medicine (Amer and Aqel, 2015), (Wang
et al., 2016). Orthopantomographic examination allows for the visualization of
dental irregularities, such as: teeth included, bone abnormalities, cysts, tumors,
cancers, infections, post-accident fractures, temporomandibular joint disorders
that cause pain in the ear, face, neck and head (Oliveira and Proença, 2011).
X-ray images are pervasively used by dentists to analyze the dental structure
and to define patient’s treatment plan. However, due to the lack of adequate
automated resources to aid the analysis of dental X-ray images, X-ray analysis
relies on mostly the dentist’s experience and visual perception (Wang et al.,
2016). Other details in dental X-rays that make it difficult to analyze these
images are: Variations of patient-to-patient teeth, artifacts used for restorations
and prostheses, poor image qualities caused by certain conditions (such as noise,
low contrast, homogeneity in regions close to objects of interest), space existing
by a missing tooth, and limitation of acquisition methods; all these challenges
2

result in unsuccessful development of automated computer tools to aid dental
diagnosis, avoiding completely automatic analysis Amer and Aqel (2015).
Image segmentation is the process of partitioning a digital image into multiple regions (pixel set) or objects, in order to make an image representation
simpler, and to facilitate its analysis. The present work is being carried out
to help in finding advances in the state-of-the art of methods for segmenting
dental X-ray images that are able, for example, to isolate teeth from other parts
of the image (jaws, temporomandibular regions, details of nasal, face and gums)
towards facilitating the automatic analysis of X-rays. With that, we are capable
to discuss limitations in the current proposed methods and future perspectives
for breakthroughs in this research field.
1.2. Contributions
This paper provides an in-depth review of the literature in dental X-ray image segmentation. A comparative evaluation of ten methods to segment extraoral dental images over a novel data set is also addressed. The proposed data
set was gathered specially for the present study, and contains 1,500 annotated
panoramic X-ray images 4 . This present study is towards to answering the following questions (see Section 3): Which category of segmentation method is
most used in the reviewed works?, do public data sets used to evaluate dental
segmentation methods present sufficiently variability to evaluate the progress
of the field?. Also, these other following questions (see Section 4) which segmentation method obtains the best performance on extracting characteristics of
radiographic images (panoramic X-ray), so that it is possible to perfectly isolate
teeth?, what are the gaps in dental X-rays that can benefit from the application
of image segmentation methods? Finally, we discuss recent advances in pattern
recognition methods that could be applied in tooth segmentation (see Section
5).
To answer the list of questions, the present review follows the steps: (i)
analysis of the current state-of-the-art, observing the trends of the segmentation methods in dental X-ray images, (ii) identification of which image segmentation methods are the most used among the reviewed works, (iii) analysis
of the amount and variety of images used in the experiments of the reviewed
works, (iv) identification of which type of dental X-ray image has been most
used among the reviewed works, (v) introduction of a novel annotated data set
with a high variability and a great number of images, and, finally, (vi) a comparative evaluation of dental segmentation methods applied to our data set. These
steps are followed from the classification of the papers found in the literature,
considering: segmentation methods, X-ray image types, size and variety of the
data sets used.
It is noteworthy that the reviewed articles mostly work with small data sets,
ranging from 1 to 100 images in average, and the only work with more than one
thousand images is not publicly available, or only containing images varying
4 Our

data set will be publicly available on the acceptance of the paper.

3

only in relation to the number of teeth. To tackle this limitation, the proposed
data set is comprised of 1,500 annotated images, which allow the classification
of X-rays in 10 categories according to the following general characteristics:
Structural variations in relation to the teeth, number of teeth, existence of
restorations, existence of dental implants, existence of dental appliances, and
existence of dental images with more than 32 teeth. The images represent the
most diverse situations found among patients in dental offices. In this sense,
a comparative evaluation of 10 segmentation methods was performed to verify
which method can more accurately identify each individual tooth, in panoramic
X-ray images. Metrics, such as accuracy, specificity, precision, recall (sensitivity)
and F-score, were used to assess the performance of each segmentation method
analyzed here.
2. Research methodology
This review has followed the methodological steps: (A) select the digital
libraries and articles (Section 2.1), (B) review the selected articles (Section
2.2), (C) define relevant categories to classify the articles and classify articles in
the categories defined (Section 3). Steps (B) and (C) ran until final results were
obtained. Step (D) was repeated until the evaluation of all the segmentation
methods studied was finalized (Section 4). Finally, step (E) presents discussion
about the evaluated methods and future directions to build more robust and
efficient segmentation methods (Section 5).
2.1. Research sources and selection of the articles
Our review is based on the state-of-the-art articles found in the following digital libraries: IEEE Xplore1 , ScienceDirect2 , Google Scholar3 and Scopus4 . The
choice of these four digital libraries relies on the fact that they include articles
presented in all other digital libraries related to either Computer Science or Dentistry. The selection of the articles was based on their prominence in the field of
English language. The articles were selected in two phases: In phase I, a total of
94 articles were found in these four digital libraries. In Phase II, articles such as
calendars, book chapter, publisher’s notes, subject index, volume content, and
from symposiums were excluded from the present study. Only peer-reviewed
international conferences and journal articles were considered; among
those, studies that corresponded to some of the following cases were considered
as non-relevant and excluded from the analysis: (1) did not answer any of our
questions in this research, (2) duplicated, (3) not peer-reviewed, and (4) did
not apply segmentation methods on at least one of the following types of dental
X-rays: Bitewing, periapical or panoramic. The final number of articles selected
1 http://ieeexplore.ieee.org/Xplore
2 http://www.sciencedirect.com
3 https://scholar.google.com
4 http://www.scopus.com/

4

was reduced to 41, at the end. The number of articles initially found in each
digital library is summarized in Table 1.
As shown in Table 1, only three of the four libraries surveyed have found
relevant studies. In addition, forty-nine percent (49%) of the articles selected
as relevant were found in the Science Direct digital library. Table 2 shows the
initial statistics obtained in the review stage of the present study, containing
the distribution of articles by digital library and year of publication. The data
presented in Table 2 show the largest number of articles found in the IEEE
Xplore digital library with 34 articles (36%). The results in Table 2 also show
the increasing trend in the number of articles published in recent years. Sixtysix percent (66%) of the articles found were published in the last five years (61
articles).
2.2. Selection of the relevant articles
In the second stage, the goal was to ensure the correct classification of the
articles selected only as relevant. The review of the articles follows a categorization phase (presented in the next section), since it was necessary to re-read
articles to classify them in each of the respective categories.
Table 1: Total number of studies found by digital library.
Source
Results

Not
Relevant

Repeated

Incomplete

Relevant
Studies

IEEE Xplore
Science Direct
Google Scholar
Scopus

34
28
23
9

16
8
7
0

0
0
1
1

5
0
7
8

13
20
8
0

TOTAL

94

31

2

20

41

SOURCE

Table 2: Distribution of articles by digital library and year of publication.
YEAR

Sum

%

IEEE Xplore

Science Direct

Google Scholar

Scopus

2016
2015
2014
2013
2012
2011
2010
2009
2008
2007
2006
2005
2004

11
12
12
13
13
5
6
5
5
8
3
0
1

12%
13%
13%
14%
14%
5%
6%
5%
5%
9%
3%
0%
1%

3
5
5
8
5
2
1
1
1
1
2
0
0

4
6
2
1
4
0
3
1
1
4
1
0
1

2
1
3
1
4
3
2
2
2
3
0
0
0

2
0
2
3
0
0
0
1
1
0
0
0
0

Total

94

100%

34

28

23

9

5

3. Taxonomy of the relevant works
Each article selected as relevant was classified among categories defined in
the present study, according to: The segmentation method used, type of dental
X-ray used images, the size and variety of the data set used. It is noteworthy
that the segmentation methods discussed and benchmarked in this review are
strictly from the state-of-the-art works.
3.1. Segmentation categories
Chen and Leung (2004) categorize segmentation methods according to the
characteristics (shape, histogram, threshold, region, entropy, spatial correlation
of pixels, among others) searched in a variety of source images (X-ray, thermal
ultrasonic, etc) to generate the cut-off point (value that determines what the
objects of interest in the analyzed image are). We adapted the general classification found in (Chen and Leung, 2004) to the classification of the works
studied in the field of dental image segmentation as follows: (1) Region-based,
(2) threshold-based, (3) cluster-based, (4) boundary-based, (5) watershed-based.
The categories were defined based on the characteristics that the relevant articles explore in the images analyzed to carry out the segmentation. Table 3 shows
the relevant works, classified into the categories of the segmentation methods.
Important details about each segmentation method presented in each category
are addressed in Section 4.2.
Region-based. The goal of the region-based method is to divide an image into
regions, based on discontinuities in pixel intensity levels. Among the relevant
articles selected, only Lurie et al. (2012) and Modi and Desai (2011) used the
region-based segmentation. The aim of the study in (Lurie et al., 2012) was to
segment panoramic X-ray images of the teeth to assist the Dentist in procedures
for detection of osteopenia and osteoporosis. Modi and Desai (2011) used region
growing approach to segment bitewing X-ray images.
Threshold-based. The rationale of the intensity threshold application in image
segmentation starts from the choice of a threshold value. Pixels whose values
exceed the threshold are placed into a region, while pixels with values below
the threshold are placed into an adjacent region. Most of the articles selected
as relevant (54%) use the threshold-based segmentation approach (Abaza et al.,
2009), (Ajaz and Kathirvelu, 2013) (Cameriere et al., 2015), (Jain and Chen,
2004), (Lin et al., 2014), (Dighe and Revati, 2012), (Huang and Hsu, 2008), (Lin
et al., 2015), (Bruellmann et al., 2016), (Amer and Aqel, 2015), (Tikhe et al.,
2016)), ((Said et al., 2006), (Geraets et al., 2007), (Lin et al., 2010), (Wang et al.,
2016), (Nomir and Abdel-Mottaleb, 2008b), (Kaur and Kaur, 2016), (Nomir
and Abdel-Mottaleb, 2008a), (Keshtkar and Gueaieb, 2007), (Lin et al., 2013),
(Indraswari et al., 2015), (Mohamed Razali et al., 2014).
In certain cases, pixel gray levels, which belongs to the objects of interest,
are substantially different from the gray levels of the pixels in the background.
In those cases, threshold segmentation based on the histogram of the image is

6

Table 3: Works grouped by segmentation methods.
Category

Segmentation method (Related works)

Region-based

Region growing ((Lurie et al., 2012), (Modi and Desai, 2011))

Threshold-based

Histogram-based threshold ((Abaza et al., 2009), (Ajaz and
Kathirvelu, 2013) (Cameriere et al., 2015), (Jain and Chen,
2004), (Lin et al., 2014), (Dighe and Revati, 2012), (Huang
and Hsu, 2008), (Lin et al., 2015), (Bruellmann et al.,
2016), (Amer and Aqel, 2015), (Tikhe et al., 2016)) / Variable threshold ((Said et al., 2006), (Geraets et al., 2007),
(Lin et al., 2010), (Wang et al., 2016), (Nomir and AbdelMottaleb, 2008b), (Kaur and Kaur, 2016), (Nomir and
Abdel-Mottaleb, 2008a), (Keshtkar and Gueaieb, 2007), (Lin
et al., 2013), (Indraswari et al., 2015), (Mohamed Razali
et al., 2014))

Cluster-based

Fuzzy-C-means (Alsmadi (2015), Son and Tuan (2016))

Boundary-based

Level set method ((Ehsani Rad et al., 2013), (Li et al.,
2006), (Li et al., 2007), (An et al., 2012)) / Active contour ((Ali et al., 2015), (Niroshika et al., 2013), (Hasan
et al., 2016)) / Edge detection ((Senthilkumaran, 2012b),
(Lin et al., 2012), (Razali et al., 2015), (Senthilkumaran,
2012a), (Gráfová et al., 2013), (Trivedi et al., 2015)) / Point
detection ((Economopoulos et al., 2008))

Watershed-based

Watershed ((Li et al., 2012))

usually used to separate objects of interest from the background. This way,
histograms can be used in situations, where objects and background have intensity levels grouped into two dominant modes. The present research identified
that seven out of the relevant papers used histogram-based threshold as the
main stage of segmentation (Abaza et al., 2009), (Ajaz and Kathirvelu, 2013)
(Cameriere et al., 2015), (Jain and Chen, 2004), (Lin et al., 2014), (Dighe and
Revati, 2012), (Huang and Hsu, 2008), (Lin et al., 2015), (Bruellmann et al.,
2016), (Amer and Aqel, 2015), (Tikhe et al., 2016).
Thresholding simply based on the histogram of the image usually fails when
the image exhibits considerable variation in contrast and illumination, resulting
in many pixels that can not be easily classified as first or second plane. One
solution to this problem is to try to estimate a ”shading function”, and then use
it to compensate for the pattern of non-uniform intensities. The commonly used
approach to compensate for irregularities, or when there is a lot of variation of
the intensity of the pixels related to the dominant object (in which case the
histogram-based thresholding has difficulties) is the use of variable threshold
based on local statistics of the pixels of the image. The studies in (Said et al.,
2006), (Geraets et al., 2007), (Lin et al., 2010), (Wang et al., 2016), (Nomir and
Abdel-Mottaleb, 2008b), (Kaur and Kaur, 2016), (Nomir and Abdel-Mottaleb,
2008a), (Keshtkar and Gueaieb, 2007), (Lin et al., 2013), (Indraswari et al.,
2015), (Mohamed Razali et al., 2014) applied local variable thresholding as
7

the main step for segmentation of the dental X-ray images.
Cluster-based. Clustering is a method used to make automatic grouping of
data according to a certain degree of similarity between the data. The criterion
of similarity depends on the problem to be solved. In general, the number
of groups to be detected must be informed as the initial parameter for the
algorithm to perform data clustering. Among the relevant papers, Alsmadi
(2015) used clustering to perform the segmentation of panoramic X-ray images,
while Son and Tuan (2016) proposed a clustering-based method to segment
X-rays of bitewing and periapical types.
Boundary-based. Boundary-based methods are used to search for discontinuities (point and edge detection) in the gray levels of the image. Thirty-four
percent (34%) of the relevant papers used boundary-based segmentation methods.
The classical boundary-based approach performs the search for points and
edges in images by detecting discontinuity in color or pixel intensities in images.
Among the works that used boundary-based methods, (Senthilkumaran, 2012b),
(Lin et al., 2012), (Razali et al., 2015), (Senthilkumaran, 2012a), (Gráfová et al.,
2013), (Trivedi et al., 2015) and (Economopoulos et al., 2008) used the classical
approach for point and edge detection to segment the images. A more recent
approach on boundary-based segmentation is known as active contour model
(Ali et al., 2015), (Niroshika et al., 2013), (Hasan et al., 2016), also called snakes,
which performs segmentation by delineating an object outline from an image.
The goal is to minimize the initialization of energy functions, and the stop
criterion is when the minimum energy is detected. The region that represents
the minimum energy value corresponds to the contour that best approaches
the perimeter of an object. Another recent boundary-based detection approach
is a variation of the active contour model known as level set method (LSM).
The LSM performs segmentation by means of geometric operations to detect
contours with topology changes. The studies found in (Ehsani Rad et al., 2013),
(Li et al., 2006), (Li et al., 2007), (An et al., 2012) used LSM to segment the
X-ray images.
Watershed-based. Watershed is a transformation defined in a grayscale image. The watershed transformation uses mathematical morphology to segment
an image in adjacent regions. Among the relevant articles selected, only (Li
et al., 2012) used the watershed-based segmentation to segment bitewing X-ray
images.
3.2. Type of the X-ray images
Approximately eighty percent (80%) of the reviewed papers used intra-oral
X-ray images. Only three of the reviewed papers used extra-oral panoramic
X-ray images. The studies addressed in (Geraets et al., 2007), (Son and Tuan,
2016) and (Trivedi et al., 2015) perform experiments with intra-oral and extraoral images. Table 4 summarizes the relevant papers grouped by the type of
X-ray image.
8

Table 4: Works grouped by X-ray images.
X-ray

Related works

Bitewing

(Jain and Chen, 2004), (Ehsani Rad et al.,
2013), (Senthilkumaran, 2012b), (Nomir and AbdelMottaleb, 2008a), (Lin et al., 2010), (Lin et al.,
2012), (Wang et al., 2016), (Keshtkar and Gueaieb,
2007), (Nomir and Abdel-Mottaleb, 2008b), (Modi
and Desai, 2011), (Ali et al., 2015), (Li et al., 2012),
(Kaur and Kaur, 2016)

Periapical

Cameriere et al. (2015), (Lin et al., 2014), (Li et al.,
2006), (Dighe and Revati, 2012), (Li et al., 2007),
(Huang and Hsu, 2008), (Lin et al., 2015), (Bruellmann et al., 2016), (Lin et al., 2013), (Niroshika
et al., 2013), (Tikhe et al., 2016), (Senthilkumaran,
2012a), (An et al., 2012), (Economopoulos et al.,
2008)

Panoramic

(Alsmadi, 2015), (Amer and Aqel, 2015), (Lurie
et al., 2012), (Ajaz and Kathirvelu, 2013), (Indraswari et al., 2015), (Mohamed Razali et al.,
2014), (Razali et al., 2015), (Hasan et al., 2016),
(Gráfová et al., 2013)

Bitewing / Periapical

(Said et al., 2006), (Abaza et al., 2009)

Bitewing / Panoramic

(Son and Tuan, 2016)

Periapical / Panoramic

(Geraets et al., 2007)

Bitewing / Periapical / Panoramic

(Trivedi et al., 2015)

3.3. Characteristics of the data sets used in the reviewed works
Sixty-one percent (61%) of the relevant papers used data sets containing
between 1 and 100 X-ray images ((Lin et al., 2012), (Li et al., 2007), (Son and
Tuan, 2016), (Alsmadi, 2015), (Lin et al., 2015), (Lin et al., 2010), (Cameriere
et al., 2015), (Jain and Chen, 2004), (Dighe and Revati, 2012), (Lin et al., 2014),
(Bruellmann et al., 2016), (Economopoulos et al., 2008), (Gráfová et al., 2013),
(Kaur and Kaur, 2016), (An et al., 2012), (Ehsani Rad et al., 2013), (Tikhe
et al., 2016), (Lin et al., 2013), (Ajaz and Kathirvelu, 2013), (Mohamed Razali
et al., 2014), (Indraswari et al., 2015), (Modi and Desai, 2011), (Amer and
Aqel, 2015), (Li et al., 2012), (Senthilkumaran, 2012a)). Eight of the reviewed
articles did not present information about the data set used ((Li et al., 2006),
(Senthilkumaran, 2012b), (Trivedi et al., 2015), (Niroshika et al., 2013), (Ali
et al., 2015), (Razali et al., 2015), (Keshtkar and Gueaieb, 2007), (Geraets et al.,
2007)). Four of the papers reviewed used between 101 and 200 images ((Wang
et al., 2016), (Nomir and Abdel-Mottaleb, 2008b), (Lurie et al., 2012), (Nomir
and Abdel-Mottaleb, 2008a)). Three used between 201 and 500 images ((Huang
and Hsu, 2008), (Abaza et al., 2009), (Hasan et al., 2016)). Only one among the
papers reviewed used more than 500 images in their experiments (Said et al.,
9

Figure 1: Number of works by number of images

2006). In general, the reviewed articles exploited data sets containing X-ray
images with small variations (i.e., varying only with respect to the number of
teeth). In addition, as shown in the previous section, there is a predominance of
intra-oral radiographs (which show only a part of the teeth) rather than extraoral radiographs (which present the entire dental structure in a single image).
Figure 1 depicts the number of works versus number of images used in each
group of work.
4. Evaluation of the segmentation methods
In our work, to evaluate the segmentation methods studied, we created a
methodology that consists of six stages. In the first stage, we started with
the acquisition of images through the orthopantomograph (device used for the
generation of orthopantomography images), and the collected images were classified into 10 categories according to the variety of structural characteristics of
the teeth. The second stage consists of annotating the images (obtaining the
binary images), which correspond to the demarcations of the objects of interest
in each analyzed image. After finishing the tooth annotation process, in the
third stage, the buccal region is annotated, as the region of interest (ROI) to
determine the actual image of the teeth. In the fourth stage, the statistics of
the gathered data set are calculated. The fifth and sixth consist in analyzing
the performance of the segmentation algorithms, using the metrics summarized
in Table 7, and in evaluating the results achieved by each segmentation method
studied.
4.1. Construction of the data set
The images used in our data set were acquired from the X-ray camera model:
ORTHOPHOS XG 5 / XG 5 DS / Ceph, manufactured by Sirona Dental Sys10

tems GmbH. X-rays were acquired at the Diagnostic Imaging Center of the
Southwest State University of Bahia (UESB). The radiographic images used for
this research were coded in order to identify the patient in the study5 .
The gathered data set consists of 1,500 annotated panoramic X-ray images.
The images have significant structural variations in relation to: the teeth, the
number of teeth, existence of restorations, existence of implants, existence of
appliances, existence of supernumerary teeth (referring to patients with more
than 32 teeth), and the size of the mouth and jaws. All images originally
obtained by the ORTHOPHOS XG 5 / XG 5 DS / Ceph ortopantomograph had
dimensions 2440 × 1292 pixels. The images were captured in gray level. The
work with panoramic X-ray images is more challenging, due to heterogeneity
reasons, among which the following stands out: 1) Different levels of noise
generated by the ortopantomograph; 2) Image of the vertebral column, which
covers the front teeth in some cases; 3) Low contrast, making morphological
properties complex.
To thoroughly benchmark the methods studied here, the 1,500 images were
distributed among 10 categories. The images were named, using whole numbers,
in sequential order by category, aiming at not identifying the patients in the
study. The process of categorizing the images was performed manually, selecting
images individually, counting tooth by tooth, as well as verifying structural
characteristics of the teeth. The images were classified according to the variety
of structural characteristics of the teeth (see Table 5). Finally, the images were
cut out to disregard non-relevant information (white border around the images
and part of the spine) generated by the orthopantomograph device. After the
clipping process, there was a change in the size of the images to 1991 × 1127
pixels, but without affecting the objects of interest (teeth), as shown in Figure
2. The cropped images were saved on the new dimension to be used in the
following stages, which will be presented in the next sections. Figure 3 shows
an X-ray image corresponding to each of the categories of our data set.
Image annotation. The process of annotating the images of our proposed
data set occurred in two parts. First, it was initiated by the upper jaw through
the annotation of the third right upper molar and making the annotation of
all the teeth of the upper arch to the third left upper molar. Then, the same
process was performed on the lower jaw with all the teeth, and in the same
direction as the upper jaw, from left to right, starting with the annotation of
the third right lower molar, and annotating all teeth from the lower arch to the
lower third molar. Figure 4 illustrates the tooth annotation process through a
panoramic X-ray image of the data set.
Determining ROI. For each image, after the annotation of the teeth, the
buccal region was also annotated, covering the whole region delineated by the
5 The use of the radiographs in the research was authorized by the National Commission for
Research Ethics (CONEP) and by the Research Ethics Committee (CEP), under the report
number 646,050, approved on 05/13/2014.

11

Figure 2: Example of the clipping and resizing of the data set images of the present work.

contour of the jaws. This process was carried out in view of preserving the
area containing all the teeth (objects of interest). Finally, the region of interest
(ROI) was determined by multiplying the values of the pixel array elements,
representing the original panoramic X-ray image, by its corresponding binary
matrix, resulting from the process of oral annotation. Figure 5 illustrates the
whole process to determine the ROI of the images.
Data set statistics. Table 5 presents the statistics of our data set: The categorization of the images, the total number of images used, the total of images
by category and the average of teeth of the images by category.
Statistics of the image ROIs. For all statistics, only the pixels in the image
ROIs were considered. The results of the statistical operations were used as
a parameter to perform the segmentation algorithms studied. The statistics
raised over the image ROIs were as follows:
The image statistics per category were organized in a single table to better
analyze the results found among the categories, as shown in Table 6. From
the analysis of that table, it was possible to compare the characteristics of
each data set category. For instance, category 5 is formed by images with
dental implants, which correspond to regions of high luminosity in the images,
resulting in pixels with greater intensity than those found in the images of the
other categories.
4.2. Performance analysis of the segmentation methods
The following metrics were used to evaluate the segmentation methods studied: Accuracy, specificity, precision, recall (sensitivity) and F-score, which are
12

Figure 3: Examples of images from the data set categories of present work: (a) Category 1;
(b) Category 2; (c) Category 3; (d) Category 4; (e) Category 5; (f ) Category 6;
(g) Category 7; (h) Category 8; (i) Category 9; (j) Category 10.

commonly used in the field of computer vision for performance analysis of segmentation. Table 7 presents a summary of these metrics.
4.2.1. Methodology of the performance analysis
Only the image ROIs were considered to calculate the metrics for the evaluation of the segmentation methods. The process presented in Figure 6 was carried
13

Figure 4: Annotation of the teeth.

Figure 5: Determining the ROI of the images.

14

Table 5: Categorization of data set images and average number of teeth per category.
Number
1
2
3
4
5
6
7
8
9
10

Category
Images with all the teeth, containing teeth with
restoration and with dental appliance
Images with all the teeth, containing teeth with
restoration and without dental appliance
Images with all the teeth, containing teeth without
restoration and with dental appliance
Images with all the teeth, containing teeth without
restoration and without dental appliance
Images containing dental implant
Images containing more than 32 teeth
Images missing teeth, containing teeth with
restoration and dental appliance
Images missing teeth, containing teeth with
restoration and without dental appliance
Images missing teeth, containing teeth without
restoration and with dental appliance
Images missing teeth, containing teeth without
restoration and without dental appliance

Images

Average
teeth

73

32

220

32

45

32

140

32

120
170
115

18
37
27

457

29

45

28

115

28

Table 6: Image statistics by category.
Category

Highest value

Lowest value

Mean

Entropy

Category 1
Category 2
Category 3
Category 4
Category 5
Category 6
Category 7
Category 8
Category 9
Category 10

253
250
248
215
254
230
255
253
251
214

10
16
13
20
5
18
7
11
9
20

108.30
108.29
107.25
107.31
109.36
100.43
108.50
106.72
107.33
105.94

6.93
6.83
6.88
6.82
6.94
6.86
6.88
6.89
6.89
6.70

out on all the segmented images obtained by each one of the 10 segmentation
methods analyzed. Figure 7 illustrates the steps of the performance evaluation over 10 segmentation methods (see also Table 9 for a list of the evaluated
methods).
4.2.2. Computing the metrics over the data set
Table 8 summarizes the process to calculate the accuracy for all images in
each category, using only one segmentation method. Parameters of each method
were optimized for best performance. On each category, the average accuracy
was computed. After that, to find the accuracy for all images in the data set,
the average accuracy was multiplied by the number of images in each category,
obtaining a weighted sum for all images in the data set (1, 500 images). By
dividing by the number of images in the whole data set, we were able to find the

15

Table 7: Metrics used to evaluate the segmentation methods studied.
Initial measures
Positive (P)
Negative (N)
True Positive (TP)
True Negative (TN)
False Positive (FP)
False Negative (FN)

Pixel is in a class of interest
Pixel is not in a class of interest
The pixel in the ground truth is positive, while method ranks the
pixel as positive
The pixel in; 3) the ground truth is negative, while method ranks
the pixel as negative
The pixel in the ground truth is negative, while method ranks
the pixel as positive
The pixel in the ground truth is positive, while method ranks the
pixel as negative
Metrics used for performance evaluation

Accuracy
Specificity
Precision

Sensitivity/Recall
F-score

Relation between total of hits on the total set of errors and hits.
This value is calculated by: (TP + TN)/(TP + FN + FP + TN)
Percentage of negative samples correctly identified on total negative samples. This value is calculated by: TN/(FP + TN)
Percentage of positive samples correctly classified on the total
of samples classified as positive. This value is calculated by:
TP/(TP + FP)
Percentage of positive samples correctly classified on the total of
positive samples. This value is calculated by: TP/(TP + FN)
Represents the harmonic average between precision and sensitivity. It is calculated by: 2*Recall*Precision/(Recall + Precision)

Table 8: Average accuracy per category.
Category (Images)
Category
Category
Category
Category
Category
Category
Category
Category
Category
Category

Accuracy

1 (73)
2 (220)
3 (45)
4 (140)
5 (120)
6 (170)
7 (115)
8 (457)
9 (45)
10 (115)

0.7585 x 73 = 55.3709
0.7471 x 220 = 164.3572
0.7682 x 45 = 34.5702
0.7909 x 140 = 110.7238
0.7506 x 120 = 90.0745
0.8295 x 170 = 141.0067
0.7401 x 115 = 85.1139
0.8351 x 457 = 381.6530
0.7646 x 45 = 34.4083
0.8005 x 115 = 92.0603

SUM

1189.34

Overall accuracy

0.7929

Overall accuracy = SUM/1,500

average accuracy of the data set for one segmentation method. The same process
was performed to calculate all other metrics (specificity, recall, precision and fscore), over each segmentation method. The segmentation methods evaluated
in this work are summarized in Table 9. Next, each one of the methods are
discussed and evaluated.

16

Figure 6: Obtaining the values of the metrics used over each image of the data set.

Table 9: Segmentation methods evaluated in the present study.
Category

Segmentation methods

Region-based
Thresholding-based
Clustering-based
Boundary-based
Watershed

1) Region growing; 2) Region splitting and merging
3) Basic global thresholding; 4) Niblack
5) Fuzzy C-means clustering
6) Sobel; 7) Canny; 8)Active contour without edges; 9) Level-set
10) Marker-controlled watershed

1) Region growing. Region growing is a method that groups pixels based
on a predefined criterion to create larger regions. A standard approach for
the region growing method is to perform calculations that generate sets of pixel
values, whose properties group them close to the center (centroids) of the values
we are looking for, so these values are used as seeds. Region growing needs two
parameters to perform the segmentation operations, as follows:
• Seeds - Initial points to start the growth of regions. In the present work,
the cells containing the X and Y coordinates of the centroids (center of
the objects of interest) of the tooth regions (objects of interest in the
images) were selected manually to serve as seeds. The seed values were
grouped into vectors for each corresponding image, which served as the
initial points for the execution of the region growing method;
• Dist - Threshold that will work as a basis to indicate the similarity between the pixels that will be part of the region or not. The parameter dist
17

Figure 7: Steps of the performance analysis of segmentation algorithms.

also corresponds to the conditional stop value of the algorithm. Thus, it is
used to verify when the difference of the mean intensity between the pixels
of a region and a new pixel becomes larger than the informed parameter,
and therefore there are no more pixels to be inserted in the regions. The
parameter dist also corresponds to the conditional stop value of the algorithm. The best value found for the dist parameter was 0.1.
2) Region splitting and merging. Segmentation based on division and union
of regions is generally performed in four basic steps: 1) The image as a whole is
considered as the area of initial interest; 2) The area of interest is examined to
decide which of the pixels satisfy some criteria of similarity; 3) if true, the area
of interest becomes part of a region in the image, receiving a label; 4) otherwise,
the area of interest is divided and each one is successively considered as area
of interest. After each division, a joining process is used to compare adjacent
regions, putting them together if necessary. This process continues until no
further division or no further union of regions are possible. The most granular
level of division that can occur is when there are areas that contain only one
pixel. Using this approach, all the regions that satisfy the similarity criterion are
filled with 1’s. Likewise, the regions that do not satisfy the similarity criterion
are filled with 0’s, thus creating a segmented image. The method needs two
18

parameters:
• qtdecomp - Minimum block size for decomposition (this parameter must
be a positive integer), and set to 1 in our evaluation;
• splitmerge - Similarity criterion used to indicate whether the region
(block) should be divided or not. In the present work, we compared the
standard deviation of the intensity of the pixels in the analyzed region.
If the standard deviation is greater than the lowest intensity value of the
pixels, then the region is divided.
3) Basic global thresholding. This method performs segmentation based on
the histogram of the image. Assuming that f (x, y) corresponds to the histogram
of an image, then to separate objects of interest from the background, an initial
threshold (T ) is chosen. Then any pixel of the image, represented by (x, y),
that is greater than T is marked as an object of interest, otherwise the pixel is
marked as the background. In our work, we used the following steps:
1. Estimate an initial value for the global limit, T (we used the average pixel
intensity of the ROI of each image analyzed);
2. Segment the image through the threshold (T ). Then, two groups of pixels
appear: G1 , referring to pixels with values greater than T and G2 , referring
to pixels with values less than or equal to T ;
3. Calculate the mean intensity values, m1 and m2 , of the pixels in G1 and
G2 , respectively;
4. Calculate (m1 + m2 )/2 to obtain a new threshold (T ) value;
5. Repeat steps 2 to 4 until the value of T , is less than a positive value
predefined by a parameter ∆T. The larger the ∆T, the less interactions
the method will perform. For the experiments, 0.5 was the best value
found for ∆T, in our work;
6. Finally, converting the grayscale image into a binary image using the
T / den threshold, where T is the threshold obtained in the previous
steps and den denote an integer value, which scales the maximum value of
the ratio of T / den to 1. The output image is binarized by replacing all
pixels of the input image that are of intensity greater than the threshold
T / den by the value 1 and all other pixels by the value 0.
4) Niblack method. Based on two measures of local statistics: mean and standard deviation within a neighborhood block of size n×n, a threshold T (x, y) for
each pixel is calculated. Then, as the neighborhood block moves, it involves
different neighborhoods, obtaining new thresholds, T , at each location (x, y).
Local standard deviation and mean are useful to determine local thresholds,
because they are descriptors of contrast and luminosity. When the contrast
or luminosity are found with great intensity in the images, they hinder the
segmentation process in methods that use a single global threshold, such as
the basic global thresholding. Adaptations of segmentation using local variable

19

thresholding have been proposed in the literature. However, the method originally proposed in (Niblack, 1985) was evaluated here. The local threshold is
calculated with a block of size n×n, according to
T (x, y) = m(x, y) + k ∗ σ(x, y) ,

(1)

where m(x,y) and σ(x,y) represent the mean and local standard deviation of the
local block of size n×n, respectively. k is an imbalance constant (also called bias)
that modifies the local value obtained by the local standard deviation. k equal
to 1 was adopted to avoid modifying the standard deviation locally calculated.
For each pixel, the following process for all the images was performed:
1. Calculate mean and standard deviation within the local block (x, y);
2. Calculate the threshold T (x, y);
3. If the value of the pixel is greater than T (x, y), one is assigned.
5) Fuzzy C-means clustering. Fuzzy C-Means starts from a set Xk = {X1 , X2 ,
X3 , ..., Xn } ∈ Rp , where Xk is a characteristic vector for all k ∈ {1, 2, ..., n},
and Rp is the p-dimensional space (Bezdek, 1981). A fuzzy pseudopartition,
denoted by P = {U1 , U2 , ..., Uc }, must satisfy the following
c
X

U1 (Xk ) = 1 ,

(2)

i=1

for every k ∈ {1, 2, ..., n}, and n denotes the number of elements of the set X.
The sum of the membership degrees of an element in all families must be equal
to one. And,
0<

n
X

Ui (Xk ) < n ,

(3)

k=1

for every i ∈ {1, 2, ..., c} and c represents the number of classes. Therefore, the
sum of the membership degrees of all the elements of a family must be less than
the number of elements in the set X. Ui is the degree of relevance for Xk in
a cluster. The way to determine if the algorithm based on the Fuzzy C-means
method finds an optimal fuzzy partition is defined by the objective function:
Jm =

n X
k
X

2

m
Uci
kXmn − Vmc k ,

(4)

t=1 c=1

where Vmc = (V11 , ..., Vmc ) is the matrix containing the centers of the clusters, M
is the fuzzy coefficient responsible for the degree of fuzing of the elements Xmn
and Vmc . The objective function is used to obtain the clusters by calculating
the Euclidean distance between the image data and the cluster centers. The
center Vmc of each cluster c(c = 1, ..., k) for an interaction t is given by
(t)
Vmc

(t)

Pn
=

M
i=1 (Uci ) Xmn
.
Pn
(t) M
i=1 (Uci )

20

(5)

To perform the segmentation using the Fuzzy C-means method, the following
parameters were used:
• Fuzzy coefficient M equal to 2, responsible for the degree of fuzzification;
• Stop criterion using a number of iterations (100), or if the method reaches
the minimum error rate (0.00001).
The formation of clusters continues until the maximum number of iterations
is completed, or when the minimum error rate is reached.
6) Sobel. Sobel works as an operator that calculates finite differences to identify the edges of the image. To perform the segmentation using the Sobel edge
detector, we used an automatically threshold T, calculated from the image pixels.
7) Canny. In Canny (Canny, 1986) detector, the edges are identified by observing the maximum location of the gradient of f (x, y). Canny performs the
following steps:
1. Smooth the Gaussian filter image first;
2. The local gradient, [gx2 + gy2 ]1/2 , and the edge direction, tan−1 (gx /gy ), are
computed at each point;
3. The edges are identified in step (2). Known the directions of the edge, a
non-maximum suppression is performed; this is done by tracing the border
and suppressing pixel values (setting them to zero) that are not considered
edge pixels. Two thresholds, T1 and T2, with T1 ¡ T2 (automatically
calculated based on each image);
4. Finally, the edges detection of the image is performed considering the
pixels that have values greater than T2.
8) Active contour without edges. This method is a variation of the model
originally proposed by Chan and Vese (Chan and Vese, 2001), and works differently from the classical edge detection methods. This is able to detect objects
whose boundaries are not defined by the gradient.
To perform the segmentation using the active contour without edges method,
the following parameters were used:
• Initial mask (a matrix of 0’s and 1’s), where the mask corresponds to 75%
of the image to be segmented);
• Total number of iterations (500, in our case);
• A stopping term equal to 0.1, in our work.

21

9) Level set method. This method is a variation of the active contour method.
Level set method (LSM) was originally proposed by Osher (1988). LSM can
perform numeric calculations of curves and surfaces without requiring predefined
criteria. The objective of LSM is to represent the limit of an object using a level
adjustment function, usually represented mathematically by the variable α. The
curves of the objects are obtained by calculating the value of γ through the set
of levels of α, given by:
γ = {(x, y)|α(x, y) = 0} ,

(6)

and the α function has positive values within the region delimited by the γ curve
and negative values outside the curve.
10) Marker-controlled watershed. The goal of the watershed segmentation
method is to separate two adjacent regions, which present abrupt changing in the
gradient values. Assuming that gradient values form a topographic surface with
valleys and mountains, brighter pixels (e.g. teeth in X-ray images) correspond to
those with the highest gradient, while the darker ones (e.g. valleys between teeth
in X-ray images) would correspond to those with the lowest gradient. A variation
of the watershed method is the marker-controlled watershed that prevents the
occurrence of the phenomenon known as super-segmentation (excess of pixels
that can not be attached to any other part of the image) using morphological
operations of opening and closing to make adjustments to the gray levels of the
image and avoid over-segmentation.
To segment the images in our data set by using the marker-controlled watershed method, the following steps are performed:
1.
2.
3.
4.

Compute a segmentation function, which tries to identify dark regions;
Compute markers of the target object;
Compute markers of the segments out of the target objects;
Calculate the transformation of the segmentation function to obtain the
position of the target objects and the positions of the background markers.

4.3. Result analysis
Tables 10 and 11 present samples of the results obtained by each of the segmentation methods evaluated, in each category. Table 12 summarizes the overall
averages obtained by calculating the metrics, which were applied to evaluate the
performance of each segmentation method. From Table 12, we can observe that
the splitting and merging-based and the sobel methods achieved almost perfect
results in the specificity (which corresponds to the true negatives). This could
be explained due to the characteristics of the two methods to focus on the edges.
Similarly, the fuzzy C-means and canny methods also had over 90% with respect
to specificity. In contrast, these four segmentation methods obtained poor results in relation to the recall metric (which privileges the true positives in the
evaluation). Thus, the images, segmented by algorithms based on splitting and

22

Table 10: Samples of the results of the segmentation methods evaluated (PART 1).
Method

Category 1

Category 2

Region
growing

Splitting
and merging

Basic global
threshold

Niblack
method

Fuzzy
c-means

Canny

Sobel

Active
contour
without
edges

Level set
method

Watershed

23

Category 3

Category 4

Category 5

Table 11: Samples of the results of the segmentation methods evaluated (PART 2).
Method

Category 6

Category 7

Region
growing

Splitting
and merging

Basic global
threshold

Niblack
method

Fuzzy
c-means

Canny

Sobel

Active
contour
without
edges

Level set
method

Watershed

24

Category 8

Category 9

Category 10

merging, fuzzy C-means, canny and sobel, showed a predominance of true negatives in their results. However, when segmentation results in a predominance of
elements of a class (for example, true negatives), it indicates that for a binary
classification problem, the result of the segmentation algorithm is equivalent to
the random suggestion of a class. Therefore, algorithms based on splitting and
merging, fuzzy C-means, canny and sobel presented poor results when applied
to the data set images used in the present work.
Table 12: Overall average results.
Method

Accuracy

Specificity

Precision

Recall

F-score

Region growing
Splitting/merging
Global thresholding
Niblack method
Fuzzy C-means
Canny
Sobel
Without edges
Level set method
Watershed

0.6810
0.8107
0.7929
0.8182
0.8234
0.7927
0.8025
0.8020
0.7637
0.7658

0.6948
0.9958
0.8191
0.8174
0.9159
0.9637
0.9954
0.8576
0.7842
0.7531

0.3553
0.8156
0.5202
0.5129
0.6186
0.4502
0.6663
0.5111
0.4776
0.4782

0.6341
0.0807
0.6931
0.8257
0.4525
0.1122
0.0360
0.5750
0.6808
0.8157

0.4419
0.1429
0.5621
0.6138
0.4939
0.1751
0.0677
0.5209
0.5224
0.5782

The results presented in Table 12 also show that the niblack reached the
highest value of the recall metric (approximately 83%), indicating that the images segmented by niblack presented the highest number of true positives (pixels
corresponding to the objects of interest in the analyzed images) and, therefore,
few false negatives with respect to the other methods of segmentation evaluated.
Niblack also obtained approximately 82% in relation to the specificity metric,
which corresponds to the true negatives that were correctly identified. Besides
niblack, only the marker-controlled watershed method reached above 80% of
the recall metric. The marker-controlled watershed obtained lower results than
the Nibliack method in all other analyzed metrics. The active contour without
edges and the level set segmentation methods obtained less than 70% of the
recall metric; these methods also achieved poorer results when compared to the
Niblack and marker-controlled watershed methods.
Considering the results, one can state that the segmentation process of
panoramic X-ray images of the teeth based on thresholding, achieved a significant performance improvement when a local threshold (niblack method)
was used, instead of using a single global threshold (basic global thresholding). Niblack was the one which presented superior performance to segment
teeth.
5. Discussion and conclusions
From the images obtained with the X-ray, the dentist can analyze the entire dental structure and construct (if necessary) the patient’s treatment plan.
However, due to the lack of adequate automated resources to aid the analysis

25

of dental X-ray images, the evaluation of these images occurs empirically, that
is to say, only using the experience of the dentist. The difficulty of analyzing
the dental X-ray images is still great when dealing with extra-oral radiographs,
because these images are not restricted to only an isolated part of the teeth,
as happens in the intra-oral images. In addition to the teeth, extra-oral X-rays
also show the temporomandibular regions (jaw joints with the skull) and details originated by the bones of the nasal and facial areas. Other information
on dental radiographs that make it difficult to analyze these images are: variations of patient-to-patient teeth, artifacts used for restorations and prostheses,
poor image qualities caused by some conditions, such as noise, low contrast,
homogeneity in regions that represent teeth and not teeth, space existing for a
missing tooth, and limitation of acquisition methods, which sometimes result in
unsuccessful segmentation. Therefore, the literature review carried out in the
present study has revealed that there is still room to find an adequate method
for segmentation of dental X-ray images that can be used as a basis for the
construction of specialized systems to aid in dental diagnosis.
It is noteworthy that the revised literature has been increasingly focused on
research to segment dental X-ray images based on thresholding (see Section 3.1
and Table 3). Eighty percent (80%) of the analyzed articles used intra-oral X-ray
images in their experiments. Public data sets used in the works analyzed in our
study indicate a preference in the literature for segmenting images that present
only an isolated part of the teeth, rather than using extra-oral X-rays that show
the entire dental structure of the mouth and face bones in a single image. The
results of the present study also show that the majority of the papers reviewed
(61%) worked with data sets containing between 1 and 100 dental x-ray images
(with only one work exploiting a data set with more than 500 images, containing
500 bitewing and 130 periapical dental radiographic films). This finding seems
to indicate that most of the methods are not thoroughly evaluated.
One contribution of the present study was the construction of a data set
with 1,500 panoramic X-ray images, characterizing a significant differential when
compared to other works. Our data set has a diversity of images, with different
characteristics and distributed in 10 categories that were defined. With our
diverse and great-in-size data set, we performed an in-depth evaluation of the
segmentation methods applied to the panoramic X-ray images of our dataset
teeth using the following metrics: Accuracy, specificity, precision, recall, and
F-score for performance measurement of the segmentation algorithms studied.
After the performance evaluation of 10 segmentation methods over our proposed
data set, we could conclude that none of the algorithms studied was able to
completely isolate the objects of interest (teeth) in the data set images used in
the present work, failing mainly because of the bone parts.
5.1. Future perspectives on exploiting learning-based segmentation methods
We can state that panoramic X-ray images of the teeth present characteristics that make the segmentation process difficult. It is realized that a possible
way for segmentation of images in this branch can be the use of methods based

26

on machine learning. Recently research has revealed interest in object recognition in images using segments (Keypoints and Lowe, 2004), (Arbel et al., 2012),
using techniques that combine segmentation and object recognition in images
(also known as semantic segmentation). The work carried out in (Carreira
et al., 2012), for example, presents how to explore grouping methods that calculate second order statistics in the form of symmetric matrices to combine image
recognition and segmentation. The method proposes to efficiently perform second order statistical operations over a large number of regions of the image,
finding clusters in shared areas of several overlapping regions. The role of image aggregation is to produce an overall description of a region of the image.
Thus, a single descriptor can summarize local characteristics within a region,
and can be used as input for a standard classifier. Most current clustering techniques calculate first-order statistics, for example, by performing the calculation
of the maximum or average value of the characteristics extracted from a cluster (Boureau et al., 2010). The work proposed by (Carreira et al., 2012) uses
different types of second-order statistics, highlighting the local descriptors scale
invariant feature transform (SIFT) (Keypoints and Lowe, 2004) and the Fisher
coding (Perronnin et al., 2010), which also uses second order statistics to recognize objects in the images to perform the semantic segmentation. Second-order
statistics could be exploited to segment dental X-ray images by learning the
shape of the teeth while performing the segmentation.
With no a priori information, finding segments of image objects is a remarkable skill of human vision. In the human visual system, when we look at an
image, not all hypotheses are equally perceived by different people. For example,
some people may recognize objects that are usually compact in their projection
in the image and others may not be able to perceive these objects. In a computational view, the work performed in Carreira and Cristian (2012) presents
a new proposal to generate and classify hypotheses of objects in an image using processes from bottom-up and mid-level bands. The objects are segmented
without prior knowledge about the their locations. The authors present how to
classify the hypotheses of the objects through a model to predict the image segments, considering the properties of the region that composes the objects in the
images studied. According to Carreira and Cristian (2012), real-world object
statistics are not easy to identify in segmentation algorithms, sometimes leading
to unsuccessful segmentation. One possibility to solve this problem would be to
obtain the parameters of the segmentation algorithm, forming a model of machine learning using large amounts of annotated data. However, the local scope
and inherently combinatory nature of the annotations of the images decrease the
effectiveness of segmentation. While energy-minimization image segmentation
problems generate multiple regions among objects, the work done in (Carreira
and Cristian, 2012) separated regions into individually connected components.
Some characteristics harm the segmentation process: For example, in an image
containing people hugged, segmenting people and their arms may require prior
knowledge of the number of arms displayed in the image and the locations of
the people may be in. Then, it is necessary to deal with such scenarios in an
ascending way, that is to say, based on strong clues as continuity of the pixels
27

that compose the objects of interest, and can be explored in the image analyzed.
Still in the field of energy minimization, an observation that must be taken into
account is the problem that leads to energy minimization that can be used to
identify regions of image objects (Tu and Zhu, 2006). In this sense, a promising
direction that can be followed towards to improve the segmentation in dental Xray images is the development of operations that can minimize energy functions
to highlight the regions that represent objects of interest in the orthopantomography images. A big concern here is the overlap of image characteristics of the
teeth with jaw and skull, in extra-oral X-rays images.
5.2. Deep learning-based segmentation
The work found in (Garcia-Garcia et al., 2017) provides a review of deep
learning methods on semantic segmentation in various application areas. First,
the authors describe the terminology of this field, as well as the mandatory
background concepts. Then the key data sets and challenges are set out to help
researchers on how to decide which ones best fit their needs and goals. The
existing methods are reviewed, highlighting their contributions and their significance in the field. The authors conclude that semantic segmentation has been
addressed with many success stories, although still remains an open problem.
Given the recent success of deep learning methods in several areas of image
pattern recognition, the use of these methods on a huge data set as ours could
be a breakthrough in the field. As presented in (Garcia-Garcia et al., 2017),
deep learning has proven to be extremely powerful to address the most diverse
segmentation problems, so we can expect a flurry of innovation and lines of research over the next few years, exploiting studies that apply deep learning and
semantic segmentation to dental X-ray images.
References
Abaza, A., Ross, A., and Ammar, H. (2009). Retrieving dental radiographs
for post-mortem identification. In Intl. Conf. on Image Processing, pages
2537–2540.
Ajaz, A. and Kathirvelu, D. (2013). Dental biometrics: Computer aided human
identification system using the dental panoramic radiographs. In Intl. Conf.
on Communication and Signal Processing, pages 717–721.
Ali, R. B., Ejbali, R., and Zaied, M. (2015). GPU-based Segmentation of Dental
X-ray Images using Active Contours Without. In Intl. Conf. on Intelligent
Systems Design and Applications, volume 1, pages 505–510.
Alsmadi, M. K. (2015). A hybrid fuzzy c-means and neutrosophic for jaw lesions
segmentation. Ain Shams Engineering Journal.
Amer, Y. Y. and Aqel, M. J. (2015). An efficient segmentation algorithm for
panoramic dental images. Procedia Computer Science, 65:718–725.

28

An, P.-l., Huang, P., Whe, P., Ang, H. U., Science, C., Engineering, I., and
Science, C. (2012). An automatic lesion detection method for dental xray images by segmentation using variational level set. In Proceedings of
the 2012 International Conference on Machine Learning and Cybernetics,
Xian, volume 1, pages 1821–1825.
Arbel, P., Berkeley, B., Rd, W., and Park, M. (2012). Semantic Segmentation
using Regions and Parts. pages 3378–3385.
Association, A. D. (1987). Dental radiographic examinations - recommendations for patient selection and limiting radiation exposure. Journal of the
American Medical Association, 257(14):1929–1936.
Bezdek, J. C. (1981). Pattern recognition with fuzzy objective function algorithms. SIAM Review, 25(3):442–442.
Boureau, Y.-L., Ponce, J., and LeCun, Y. (2010). A Theoretical Analysis of
Feature Pooling in Visual Recognition. Icml, pages 111–118.
Bruellmann, D., Sander, S., and Schmidtmann, I. (2016). The design of an fast
fourier filter for enhancing diagnostically relevant structures – endodontic
files. Computers in Biology and Medicine, 72:212–217.
Cameriere, R., De Luca, S., Egidi, N., Bacaloni, M., Maponi, P., Ferrante, L.,
and Cingolani, M. (2015). Automatic age estimation in adults by analysis of
canine pulp/tooth ratio: Preliminary results. Journal of Forensic Radiology
and Imaging, 3(1):61–66.
Canny, J. (1986). A computational approach to edge detection. IEEE Trans.
on Patt. Analysis and Machine Intellig., PAMI-8(6):679–698.
Carreira, J., Caseiro, R., Batista, J., and Sminchisescu, C. (2012). Semantic segmentation with second-order pooling. Lecture Notes in Computer Science,
7578:430–443.
Carreira, J. and Cristian, S. (2012). Cpmc: Automatic object segmentation using. IEEE Trans. on Pattern Analysis and Machine Intelligence,
34(7):1312–1328.
Chan, T. F. and Vese, L. A. (2001). Active contours without edges. IEEE
Trans. on Image Processing, 10(2):266–277.
Chen, S. and Leung, H. (2004). Survey over image thresholding techniques
and quantitative performance evaluation. Journal of Electronic Imaging,
13(1):220.
Dighe, S. and Revati, S. (2012). Preprocessing, Segmentation and Matching of
Dental Radiographs used in Dental Biometrics. 1(2278):52–56.

29

Economopoulos, T., Matsopoulos, G. K., Asvestas, P. A., Gröndahl, K., and
Gröndahl, H. G. (2008). Automatic correspondence using the enhanced
hexagonal centre-based inner search algorithm for point-based dental image
registration. Dentomaxillofacial Radiology, 37(4):185–204.
Ehsani Rad, A., Shafry, M., Rahim, M., and Norouzi, A. (2013). Digital dental
x-ray image segmentation and feature extraction. 11:3109–3114.
Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., and
Garcia-Rodriguez, J. (2017). A Review on Deep Learning Techniques Applied to Semantic Segmentation. pages 1–23.
Geraets, W. G. M., Verheij, J. G. C., van der Stelt, P. F., Horner, K., Lindh,
C., Nicopoulou-Karayianni, K., Jacobs, R., Harrison, E. J., Adams, J. E.,
and Devlin, H. (2007). Prediction of bone mineral density with dental
radiographs. Bone, 40(5):1217–1221.
Gráfová, L., Kašparová, M., Kakawand, S., Procházka, A., and Dostálová, T.
(2013). Study of edge detection task in dental panoramic radiographs.
Dentomaxillofacial Radiology, 42(7).
Hasan, M. M., Hassan, R., and Ismail, W. (2016). Automatic segmentation
of jaw from panoramic dental x-ray images using gvf snakes. In WAC,
volume 1, pages 1–6.
Huang, C. H. and Hsu, C. Y. (2008). Computer-assisted orientation of dental
periapical radiographs to the occlusal plane. Oral Surgery, Oral Medicine,
Oral Pathology, Oral Radiology and Endodontology, 105(5):649–653.
Indraswari, R., Arifin, A. Z., Navastara, D. A., and Jawas, N. (2015). Teeth
segmentation on dental panoramic radiographs using decimation-free directional filter banck thresholding and multistage adaptive thresholding. In
Intl. Conf. on Information, Communication Technology and System, volume 1, pages 49–54.
Jain, A. K. and Chen, H. (2004). Matching of dental X-ray images for human
identification. Pattern Recognition, 37(7):1519–1532.
Kaur, J. and Kaur, J. (2016). Dental image disease analysis using pso and backpropagation neural network classifier. Intl. Journal of Advanced Research
in Computer Science and Software Engineering, 6(4):158–160.
Keshtkar, F. and Gueaieb, W. (2007). Segmentation of dental radiographs using
a swarm intelligence approach. In Canadian Conference on Electrical and
Computer Engineering, pages 328–331.
Keypoints, S.-i. and Lowe, D. G. (2004). Distinctive image features from. Intl.
Journal of Computer Vision, 60(2):91–110.

30

Li, H., Sun, G., Sun, H., and Liu, W. (2012). Watershed algorithm based on
morphology for dental x-ray images segmentation. In Intl. Conference on
Signal Processing Proceedings, volume 2, pages 877–880.
Li, S., Fevens, T., Krzyzak, A., Jin, C., and Li, S. (2007). Semi-automatic
computer aided lesion detection in dental x-rays using variational level set.
Pattern Recognition, 40(10):2861–2873.
Li, S., Fevens, T., Krzyzak, A., and Li, S. (2006). An automatic variational level
set segmentation framework for computer aided dental x-rays analysis in
clinical environments. Computerized Medical Imag. and Graph., 30(2):65–
74.
Lin, P. L., Huang, P. W., Huang, P. Y., and Hsu, H. C. (2015). Alveolar
bone-loss area localization in periodontitis radiographs based on threshold
segmentation with a hybrid feature fused of intensity and the h-value of
fractional brownian motion model. Computer Methods and Programs in
Biomedicine, 121(3):117–126.
Lin, P. L., Huang, P. Y., and Huang, P. W. (2013). An effective teeth segmentation method for dental periapical radiographs based on local singularity. In
Intl. Conf. on System Science and Engineering, volume 1, pages 407–411.
Lin, P. L., Huang, P. Y., Huang, P. W., Hsu, H. C., and Chen, C. C. (2014).
Teeth segmentation of dental periapical radiographs based on local singularity analysis. Computer Methods and Programs in Biomedicine, 113(2):433–
445.
Lin, P. L., Lai, Y. H., and Huang, P. W. (2010). An effective classification and
numbering system for dental bitewing radiographs using teeth region and
contour information. Pattern Recognition, 43(4):1380–1392.
Lin, P. L., Lai, Y. H., and Huang, P. W. (2012). Dental biometrics: Human
identification based on teeth and dental works in bitewing radiographs.
Patt. Recog., 45(3):934–946.
Lurie, A., Tosoni, G. M., Tsimikas, J., and Fitz, W. (2012). Recursive hierarchic
segmentation analysis of bone mineral density changes on digital panoramic
images. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology,
113(4):549–558.
Modi, C. K. and Desai, N. P. (2011). A simple and novel algorithm for automatic selection of ROI for dental radiograph segmentation. In Canadian
Conference on Electrical and Computer Engineering, pages 000504–000507.
Mohamed Razali, M. R., Ahmad, N. S., Mohd Zaki, Z., and Ismail, W. (2014).
Region of adaptive threshold segmentation between mean, median and otsu
threshold for dental age assessment. In Intl. Conf. on Computer, Communications, and Control Technology, Proceedings, pages 353–356.

31

Niblack, W. (1985). An introduction to digital image processing. 1 edition.
Niroshika, U. A. A., Meegama, R. G. N., and Fernando, T. G. I. (2013). Active
contour model to extract boundaries of teeth in dental X-ray images. In
Proceedings of the 8th International Conference on Computer Science and
Education, ICCSE 2013, pages 396–401.
Nomir, O. and Abdel-Mottaleb, M. (2008a). Fusion of matching algorithms for
human identification using dental x-ray radiographs. IEEE Transactions
on Inform. Forensics and Security, 3(2):223–233.
Nomir, O. and Abdel-Mottaleb, M. (2008b). Hierarchical contour matching for
dental x-ray radiographs. Pattern Recognition, 41(1):130–138.
Oliveira, J. and Proença, H. (2011). Caries Detection in Panoramic Dental
X-ray Images, pages 175–190. Springer Netherlands, Dordrecht.
Osher, S. J. (1988). Fronts propagating with curvature dependent speed. Computational Physics, 79(1):1–5.
Paewinsky, E., Pfeiffer, H., and Brinkmann, B. (2005). Quantification of secondary dentine formation from orthopantomograms – a contribution to
forensic age estimation methods in adults. Intl. Journal of Legal Medicine,
119(1):27–30.
Perronnin, F., Jorge, S., Mensink, T., Perronnin, F., Jorge, S., Mensink, T.,
Kernel, F., Perronnin, F., and Jorge, S. (2010). Improving the fisher kernel for large-scale image classification to cite this version : Improving the
fisher kernel for large-scale image classification. In Springer-Verlag Berlin
Heidelberg, pages 143–156.
Quinn, R. A. and Sigl, C. C. (1980). Radiography in Modern Industry.
Razali, M. R. M., Ahmad, N. S., Hassan, R., Zaki, Z. M., and Ismail, W. (2015).
Sobel and canny edges segmentations for the dental age assessment. In Intl.
Conference on Computer Assisted System in Health, pages 62–66.
Said, E., Nassar, D., Fahmy, G., and Ammar, H. (2006). Teeth segmentation in
digitized dental x-ray films using mathematical morphology. IEEE Transactions on Inform. Forensics and Security, 1(2):178–189.
Senthilkumaran, N. (2012a). Fuzzy logic approach to edge detection for dental
x-ray image segmentation. 3(5):5236–5238.
Senthilkumaran, N. (2012b). Genetic algorithm approach to edge detection for
dental x-ray image segmentation. Intl. Journal of Advanced Research in
Computer Science and Electronics Engineering, 1(7):5236–5238.
Son, L. H. and Tuan, T. M. (2016). A cooperative semi-supervised fuzzy clustering framework for dental x-ray image segmentation. Expert Systems with
Applications, 46:380–393.
32

Tikhe, S., Naik, A., Bhide, S., Saravanan, T., and Kaliyamurthie, K. (2016).
Algorithm to Identify Enamel Caries and Interproximal Caries Using Dental
Digital Radiographs. In Intl. Advanced Computing Conference, IACC 2016,
pages 225–228.
Trivedi, D. N., Kothari, A. M., Shah, S., and Nikunj, S. (2015). Dental Image
Matching By Canny Algorithm for Human Identification. International
Journal of Advanced Computer Research, 4(17):985–990.
Tu, Z. and Zhu, S. C. (2006). Parsing images into regions, curves, and curve
groups. Intl. Journal of Computer Vision, 69(2):223–249.
Wang, C. W., Huang, C. T., Lee, J. H., Li, C. H., Chang, S. W., Siao, M. J., Lai,
T. M., Ibragimov, B., Vrtovec, T., Ronneberger, O., Fischer, P., Cootes,
T. F., and Lindner, C. (2016). A benchmark for comparison of dental
radiography analysis algorithms. Medical Image Analysis, 31:63–76.
Willems, G., Moulin-Romsee, C., and Solheim, T. (2002). Non-destructive
dental-age calculation methods in adults: intra- and inter-observer effects.
Forensic Science Intl., 126(3):221–226.

33

