food recognition
140856752
Image Augmentation Based Food Recognition with Convolutional Neural Networks
Image retrieval for food ingredients is important work, tremendously tiring, uninteresting, and expensive. Computer vision systems have extraordinary advancements in image retrieval with CNNs skills. But it is not feasible for small size food datasets using convolutional neural networks directly. In this study, a novel image retrieval approach is presented for small and medium scale food datasets, which both augments images utilizing image transformation techniques to enlarge the size of datasets, and promotes the average accuracy of food recognition with state of the art deep learning technologies. First, typical image transformation techniques are used to augment food images. Then transfer learning technology based on deep learning is applied to extract image features. Finally, a food recognition algorithm is leveraged on extracted deepfeature vectors. The presented image retrieval architecture is analyzed based on a smallscale food dataset which is composed of forty one categories of food ingredients and one hundred pictures for each category. Extensive experimental results demonstrate the advantages of image augmentation architecture for small and medium datasets using deep learning. The novel approach combines image augmentation, ResNet feature vectors, and SMO classification, and shows its superiority for food detection of small/medium scale datasets with comprehensive experiments.
Lili Pan Jiaohua Qin Hao Chen Xuyu Xiang Cong Li Ran Chen
204837612
Ingredient Guided Cascaded Multi Attention Network for Food Recognition
Recently, food recognition is gaining more attention in the multimedia community due to its various applications, e.g. multimodal foodlog and personalized healthcare. Most of existing methods directly extract visual features of the whole image using popular deep networks for food recognition without considering its own characteristics. Compared with other types of object images, food images generally do not exhibit distinctive spatial arrangement and common semantic patterns, and thus are very hard to capture discriminative information. In this work, we achieve food recognition by developing an Ingredient Guided Cascaded Multi Attention Network (IG CMAN) which is capable of sequentially localizing multiple informative image regions with multi scale from category level to ingredient level guidance in a coarse to fine manner. At the first level, IG CMAN generates the initial attentional region from the category supervised network with Spatial Transformer (ST) Taking this localized attentional region as the reference, IG CMAN combined ST with LSTM to sequentially discover diverse attentional regions with fine grained scales from ingredient guided sub network in the following levels. Furthermore, we introduce a new dataset ISIA Food 200 with 200 food categories from the list in the Wikipedia, about 200,000 food images and 319 ingredients. We conducted extensive experiment on two popular food datasets and newly proposed ISIA Food 200, and verified the effectiveness of our method. Qualitative results along with visualization further show that IG CMAN can introduce the explainability for localized regions, and is able to learn relevant regions for ingredients.
ACM Multimedia
Weiqing Min Linhu Liu Zhengdong Luo Shuqiang Jiang
17006311
Wide Slice Residual Networks for Food Recognition
Image based food recognition pose new challenges for mainstream computer vision algorithms. Recent works in the field focused either on hand crafted representations or on learning these by exploiting deep neural networks (DNN) Despite the success of DNN based works, these exploit off the shelf deep architectures which are not cast to the specific food classification problem. We believe that better results can be obtained if the architecture is defined with respect to an analysis of the food composition. Following such an intuition, this work introduces a new deep scheme that is designed to handle the food structure. In particular, we focus on the vertical food traits that are common to a large number of categories (i.e. 15% of the whole data in current datasets) Towards the final objective, we first introduce a slice convolution block to capture such specific information. Then, we leverage on the recent success of deep residual blocks and combine those with the sliced convolution to produce the classification score. Extensive evaluations on three benchmark datasets demonstrated that our solution has better performance than existing approaches (e.g. a top 1 accuracy of 90.27% on the Food 101 dataset)
2018 IEEE Winter Conference on Applications of Computer Vision (WACV)
Niki Martinel Gian Luca Foresti Christian Micheloni
4712140
A New Deep Learning Based Food Recognition System for Dietary Assessment on An Edge Computing Service Infrastructure
Literature has indicated that accurate dietary assessment is very important for assessing the effectiveness of weight loss interventions. However, most of the existing dietary assessment methods rely on memory. With the help of pervasive mobile devices and rich cloud services, it is now possible to develop new computer aided food recognition system for accurate dietary assessment. However, enabling this future Internet of Things based dietary assessment imposes several fundamental challenges on algorithm development and system design. In this paper, we set to address these issues from the following two aspects: (1) to develop novel deep learning based visual food recognition algorithms to achieve the best in class recognition accuracy; (2) to design a food recognition system employing edge computing based service computing paradigm to overcome some inherent problems of traditional mobile cloud computing paradigm, such as unacceptable system latency and low battery life of mobile devices. We have conducted extensive experiments with real world data. Our results have shown that the proposed system achieved three objectives: (1) outperforming existing work in terms of food recognition accuracy; (2) reducing response time that is equivalent to the minimum of the existing approaches; and (3) lowering energy consumption which is close to the minimum of the state of the art.
IEEE Transactions on Services Computing
Chang Liu Yu Cao Yan Luo Guanling Chen Vinod Vokkarane Ma Yunsheng Songqing Chen Peng Hou
56895278
Deep Convolutional Generative Adversarial Network Based Food Recognition Using Partially Labeled Data
Traditional machine learning algorithms using hand crafted feature extraction techniques (such as local binary pattern) have limited accuracy because of high variation in images of the same class (or intraclass variation) for food recognition tasks. In recent works, convolutional neural networks (CNNs) have been applied to this task with better results than all previously reported methods. However, they perform best when trained with large amount of annotated (labeled) food images. This is problematic when obtained in large volume, because they are expensive, laborious, and impractical. This article aims at developing an efficient deep CNN learning based method for food recognition alleviating these limitations by using partially labeled training data on generative adversarial networks (GANs) We make new enhancements to the unsupervised training architecture introduced by Goodfellow et al. which was originally aimed at generating new data by sampling a dataset. In this article, we make modifications to deep convolutional GANs to make them robust and efficient for classifying food images. Experimental results on benchmarking datasets show the superiority of our proposed method, as compared to the current state of the art methodologies, even when trained with partially labeled training data.
IEEE Sensors Letters
Bappaditya Mandal Niladri B Puhan Avijit Verma
211117398
Mining Discriminative Food Regions for Accurate Food Recognition
Automatic food recognition is the very first step towards passive dietary monitoring. In this paper, we address the problem of food recognition by mining discriminative food regions. Taking inspiration from Adversarial Erasing, a strategy that progressively discovers discriminative object regions for weakly supervised semantic segmentation, we propose a novel network architecture in which a primary network maintains the base accuracy of classifying an input image, an auxiliary network adversarially mines discriminative food regions, and a region network classifies the resulting mined regions. The global (the original input image) and the local (the mined regions) representations are then integrated for the final prediction. The proposed architecture denoted as PAR Net is end to end trainable, and highlights discriminative regions in an online fashion. In addition, we introduce a new fine grained food dataset named as Sushi 50, which consists of 50 different sushi categories. Extensive experiments have been conducted to evaluate the proposed approach. On three food datasets chosen (Food 101, Vireo 172, and Sushi 50) our approach performs consistently and achieves state of the art results (top1 testing accuracy of 90.4% 90.2% 92.0% respectively) compared with other existing approaches.
BMVC
Jianing Qiu Frank P -W Lo Yingnan Sun Siyao Wang Benny P L Lo
69842164
Survey of Different Approaches Used for Food Recognition
Food recognition is an ever growing field gaining rapid momentum in the past couple of years. Various approaches have been implemented to get accurate results by correctly identifying the food item. Traditional methods like the implementation of neural networks, SVMs, HMMs utilizing hand crafted features of the large data sets of food images are one way of developing food recognition systems. To improve the accuracy, modern methods using newer concepts of convolutional neural networks and deep learning which avoid the use of hand crafted features are being implemented to build even better food recognition systems. These newer methods require huge data sets of images of food items to work with to obtain good results. Besides approaches based on image recognition, other innovative images are also being explored for recognizing food images. Food items are being recognized using the cutting sounds, acoustic sensors, electronic tongues and so on.
Sandhya Arora Gauri Chaware Devangi Chinchankar Eesha Dixit Shevi Jain
188707792
Food recognition in hematophagous insects.
Hematophagous insects use heat, odors, visual cues and humidity emitted by vertebrate hosts to find them in space and time. Once they reach a host, they integrate multimodal information from its skin, and decide whether to bite or not. If skin conditions fulfil the insect's expectations, it bites and pumps a small quantity of blood. Again, only if the sampled blood fulfils the insect's feeding requirements, it continues with a full ingestion. Taste is involved in both timely linked evaluation processes via contact chemoreceptors located in different parts of their bodies, driving jointly food acceptance or rejection. However, the whole picture of how blood sucking insects evaluate the quality of a potential host is poorly understood. Here, I summarize the actual knowledge about the feeding decision making in blood sucking insects. Being typically involved in the transmission of diseases to humans or livestock, a deeper understanding about factors affecting an essential process as feeding in these insects could help us to find new strategies to reduce interactions.
Current opinion in insect science
Romina Barrozo
4772337
A Deep Learning based Food Recognition System for Lifelog Images
In this paper, we propose a deep learning based system for food recognition from personal life archive im ages. The system first identifies the eating moments based on multi modal information, then tries to focus and enhance the food images available in these moments, and finally, exploits GoogleNet as the core of the learning process to recognise the food category of the images. Preliminary results, experimenting on the food recognition module of the proposed system, show that the proposed system achieves 95.97% classification accuracy on the food images taken from the personal life archive from several lifeloggers, which potentially can be extended and applied in broader scenarios and for different types of food categories.
ICPRAM
Binh T Nguyen Duc-Tien Dang-Nguyen Dang Xuan Tien Thai Van Phat Cathal Gurrin
2040113
Food Recognition: A New Dataset, Experiments, and Results
We propose a new dataset for the evaluation of food recognition algorithms that can be used in dietary monitoring applications. Each image depicts a real canteen tray with dishes and foods arranged in different ways. Each tray contains multiple instances of food classes. The dataset contains 1027 canteen trays for a total of 3616 food instances belonging to 73 food classes. The food on the tray images has been manually segmented using carefully drawn polygonal boundaries. We have benchmarked the dataset by designing an automatic tray analysis pipeline that takes a tray image as input, finds the regions of interest, and predicts for each region the corresponding food class. We have experimented with three different classification strategies using also several visual descriptors. We achieve about 79% of food and tray recognition accuracy using convolutional neural networks based features. The dataset, as well as the benchmark framework, are available to the research community.
IEEE Journal of Biomedical and Health Informatics
Gianluigi Ciocca Paolo Napoletano Raimondo Schettini