ingredient recognition
210705951
Zero Shot Ingredient Recognition by Multi Relational Graph Convolutional Network
Recognizing ingredients for a given dish image is at the core of automatic dietary assessment, attracting increasing attention from both industry and academia. Nevertheless, the task is challenging due to the difficulty of collecting and labeling sufficient training data. On one hand, there are hundred thousands of food ingredients in the world, ranging from the common to rare. Collecting training samples for all of the ingredient categories is difficult. On the other hand, as the ingredient appearances exhibit huge visual variance during the food preparation, it requires to collect the training samples under different cooking and cutting methods for robust recognition. Since obtaining sufficient fully annotated training data is not easy, a more practical way of scaling up the recognition is to develop models that are capable of recognizing unseen ingredients. Therefore, in this paper, we target the problem of ingredient recognition with zero training samples. More specifically, we introduce multi relational GCN (graph convolutional network) that integrates ingredient hierarchy, attribute as well as co occurrence for zero shot ingredient recognition. Extensive experiments on both Chinese and Japanese food datasets are performed to demonstrate the superior performance of multi relational GCN and shed light on zero shot ingredients recognition.
AAAI
Jing-jing Chen Liangming Pan Zhipeng Wei Xiang Wang Chong-Wah Ngo Tat-Seng Chua
229689255
A Study of Multi Task and Region Wise Deep Learning for Food Ingredient Recognition
Food recognition has captured numerous research attention for its importance for health related applications. The existing approaches mostly focus on the categorization of food according to dish names, while ignoring the underlying ingredient composition. In reality, two dishes with the same name do not necessarily share the exact list of ingredients. Therefore, the dishes under the same food category are not mandatorily equal in nutrition content. Nevertheless, due to limited datasets available with ingredient labels, the problem of ingredient recognition is often overlooked. Furthermore, as the number of ingredients is expected to be much less than the number of food categories, ingredient recognition is more tractable in the real world scenario. This paper provides an insightful analysis of three compelling issues in ingredient recognition. These issues involve recognition in either image level or region level, pooling in either single or multiple image scales, learning in either single or multi task manner. The analysis is conducted on a large food dataset, Vireo Food 251, contributed by this paper. The dataset is composed of 169,673 images with 251 popular Chinese food and 406 ingredients. The dataset includes adequate challenges in scale and complexity to reveal the limit of the current approaches in ingredient recognition.
IEEE Transactions on Image Processing
Jingjing Chen Bin Zhu Chong-Wah Ngo Tat-Seng Chua Yugang Jiang
207240186
Deep based Ingredient Recognition for Cooking Recipe Retrieval
Retrieving recipes corresponding to given dish pictures facilitates the estimation of nutrition facts, which is crucial to various health relevant applications. The current approaches mostly focus on recognition of food category based on global dish appearance without explicit analysis of ingredient composition. Such approaches are incapable for retrieval of recipes with unknown food categories, a problem referred to as zero shot retrieval. On the other hand, content based retrieval without knowledge of food categories is also difficult to attain satisfactory performance due to large visual variations in food appearance and ingredient composition. As the number of ingredients is far less than food categories, understanding ingredients underlying dishes in principle is more scalable than recognizing every food category and thus is suitable for zero shot retrieval. Nevertheless, ingredient recognition is a task far harder than food categorization, and this seriously challenges the feasibility of relying on them for retrieval. This paper proposes deep architectures for simultaneous learning of ingredient recognition and food categorization, by exploiting the mutual but also fuzzy relationship between them. The learnt deep features and semantic labels of ingredients are then innovatively applied for zero shot retrieval of recipes. By experimenting on a large Chinese food dataset with images of highly complex dish appearance, this paper demonstrates the feasibility of ingredient recognition and sheds light on this zero shot problem peculiar to cooking recipe retrieval.
ACM Multimedia
Jingjing Chen Chong-Wah Ngo
209336679
AIFood: A Large Scale Food Images Dataset for Ingredient Recognition
In this paper, we introduce a large scale food images dataset namely AIFood, which is constructed to aim ingredient recognition in food image research. AIFood dataset includes 24 categories and totally 372,095 food images around the world. We collect food images from eight existing food image datasets and a food website. The food images are relabeled using 24 categories. We preliminarily label each image using existing food information, e.g. dish name or ingredient information. Next, we manually check food images to find out undiscovered ingredients and relabel them. Every image can be labeled more than one category. In addition, food images may have color cast or uneven contrast problems, which may disturb performance of image recognition system. So, we applied preprocessing method which contains automatic white balancing and contrast limited adaptive histogram equalization methods to improve visual quality of food images. We set constraints which are defined by luminance and chrominance of image to determine if the image is to be preprocessed.
TENCON 2019 2019 IEEE Region 10 Conference (TENCON)
Gwo-Giun Lee Chin-Wei Huang Jia-Hong Chen Shih-Yu Chen Hsiu-Ling Chen
86768737
RECIPE RETRIEVAL USING INGREDIENT RECOGNITION
Based on the generic object recognition, we propose a cooking recipe recommendation system on an android smart phone which includes food (vegetable and meats) detection involving object recognition. The system which we have proposed gives the real time recognition of the food stuff and using with a recommendation system, helps retrieve recipes including those food ingredients. When the user points a smart phone camera towards the food items, the recipes are retrieved instantly based on certain ranking.
Vaibhav Rajendra Suryawanshi
168798481
Recipe Recommendation Using Ingredient Recognition
Utilizing interactive media methods in frameworks that give help to clients has turned into a typical enthusiasm for late research. To accomplish an incomparable level of intelligence in these frameworks, considering clients' states and also clients' inclinations is critical. Be that as it may, most genuine frameworks don't consider clients' inclinations while giving help to clients. We plan a framework that chooses an attractive formula for a client to cook in view of inclination, at that point direct him through the cooking. The formula choice depends on individuals' normal translation of cooking formulas. We store clients' criticism (understandings) as shared separating, and this input is utilized to recognize properties of different formulas. Moreover, clients can take in more adaptably and viably by controlling their procedure utilizing a parallel progress display for cooking. Utilizing a parallel change show, clients will have the capacity to perform parallel activities in any request they need and the framework will encourage them as indicated by the move made.
Ajay Mozar K Vikram
207199278
Real time mobile recipe recommendation system using food ingredient recognition
In this paper, we propose a mobile cooking recipe recom mendation system employing object recognition for food ingredients such as vegetables and meats. The proposed system carries out object recognition on food ingredients in a real time way on an Android based smartphone, and recommends cooking recipes related to the recognized food ingredients. By only pointing a built in camera on a mobile device to food ingredients, the user can obtain a recipe list instantly. As an object recognition method, we adopt bag of features with SURF and color histogram extracted from multiple images as image features and linear SVM with the one vs rest strategy as a classifier. We built 30 kinds of food ingredient short video database for experiments. With this database, we achieved the 83.93% recognition rate within the top six candidates. In the experiment, we made user study by comparing mobile recipe recommendation systems with/without ingredient recognition.
IMMPD '12
Takuma Maruyama Yoshiyuki Kawano Keiji Yanai
204837612
Ingredient Guided Cascaded Multi Attention Network for Food Recognition
Recently, food recognition is gaining more attention in the multimedia community due to its various applications, e.g. multimodal foodlog and personalized healthcare. Most of existing methods directly extract visual features of the whole image using popular deep networks for food recognition without considering its own characteristics. Compared with other types of object images, food images generally do not exhibit distinctive spatial arrangement and common semantic patterns, and thus are very hard to capture discriminative information. In this work, we achieve food recognition by developing an Ingredient Guided Cascaded Multi Attention Network (IG CMAN) which is capable of sequentially localizing multiple informative image regions with multi scale from category level to ingredient level guidance in a coarse to fine manner. At the first level, IG CMAN generates the initial attentional region from the category supervised network with Spatial Transformer (ST) Taking this localized attentional region as the reference, IG CMAN combined ST with LSTM to sequentially discover diverse attentional regions with fine grained scales from ingredient guided sub network in the following levels. Furthermore, we introduce a new dataset ISIA Food 200 with 200 food categories from the list in the Wikipedia, about 200,000 food images and 319 ingredients. We conducted extensive experiment on two popular food datasets and newly proposed ISIA Food 200, and verified the effectiveness of our method. Qualitative results along with visualization further show that IG CMAN can introduce the explainability for localized regions, and is able to learn relevant regions for ingredients.
ACM Multimedia
Weiqing Min Linhu Liu Zhengdong Luo Shuqiang Jiang
199488797
From Market to Dish: Multi ingredient Image Recognition for Personalized Recipe Recommendation
Recognition of food ingredients enables applications on recipe recommendation for developing a healthier eating habit. Existing ingredients recognition methods largely rely on ideal images captured in a controlled environment, while ingredients are usually displayed unorderly in a complex environment in the market. We propose the multi ingredient recognition problem in the market and develop a Spatial Regularization Network (SRN) based method to solve it by using a newly collected multiple vegetable image dataset captured in the market. We further use the recognition result to develop a recipe recommendation system to satisfy the daily nutrition requirements and individual preference of each user. Experiments show that our multi ingredient recognition outperforms previous methods over 14% in mAP and recommendation model shows an improvement of over 23% in HR@10.
2019 IEEE International Conference on Multimedia and Expo (ICME)
Lin Zhang Jianbo Zhao Si Li Boxin Shi Ling-yu Duan
11911627
[Demo paper] mirurecipe: A mobile cooking recipe recommendation system with food ingredient recognition
In this demo, we demonstrate a cooking recipe recommendation system which runs on a consumer smartphone. The proposed system carries out object recognition on food ingredients in a real time way, and recommends cooking recipes related to the recognized food ingredients. By only pointing a built in camera on a mobile device to food ingredients, the user can obtain a recipe list instantly. The objective of the proposed system is to assist people who cook to decide a cooking recipe at grocery stores or at a kitchen. In the current implementation, the system can recognize 30 kinds of food ingredient in 0.15 seconds, and it achieved the 83.93% recognition rate within the top six candidates.
2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW)
Yoshiyuki Kawano Takanori Sato Takuma Maruyama Keiji Yanai