Shape recognition
129468
Shape matching and object recognition using shape contexts
This paper presents my work on computing shape models that are computationally fast and invariant basic transformations like translation, scaling and rotation. In this paper, I propose shape detection using a feature called shape context. Shape context describes all boundary points of a shape with respect to any single boundary point. Thus it is descriptive of the shape of the object. Object recognition can be achieved by matching this feature with a priori knowledge of the shape context of the boundary points of the object. Experimental results are promising on handwritten digits, trademark images.
2010 3rd International Conference on Computer Science and Information Technology
Serge J Belongie Jitendra Malik Jan Puzicha
69602756
Learning Multi View Representation With LSTM for 3 D Shape Recognition and Retrieval
Shape representation for 3 D models is an important topic in computer vision, multimedia analysis, and computer graphics. Recent multiview based methods demonstrate promising performance for 3 D shape recognition and retrieval. However, most multiview based methods ignore the correlations of multiple views or suffer from high computional cost. In this paper, we propose a novel multiview based network architecture for 3 D shape recognition and retrieval. Our network combines convolutional neural networks (CNNs) with long short term memory (LSTM) to exploit the correlative information from multiple views. Well pretrained CNNs with residual connections are first used to extract a low level feature of each view image rendered from a 3 D shape. Then, a LSTM and a sequence voting layer are employed to aggregate these features into a shape descriptor. The highway network and a three step training strategy are also adopted to boost the optimization of the deep network. Experimental results on two public datasets demonstrate that the proposed method achieves promising performance for 3 D shape recognition and the state of the art performance for the 3 D shape retrieval.
IEEE Transactions on Multimedia
Chao Ma Yulan Guo Jungang Yang Wei An
2407217
Multi view Convolutional Neural Networks for 3D Shape Recognition
A longstanding question in computer vision concerns the representation of 3D shapes for recognition: should 3D shapes be represented with descriptors operating on their native 3D formats, such as voxel grid or polygon mesh, or can they be effectively represented with view based descriptors? We address this question in the context of learning to recognize 3D shapes from a collection of their rendered views on 2D images. We first present a standard CNN architecture trained to recognize the shapes' rendered views independently of each other, and show that a 3D shape can be recognized even from a single view at an accuracy far higher than using state of the art 3D shape descriptors. Recognition rates further increase when multiple views of the shapes are provided. In addition, we present a novel CNN architecture that combines information from multiple views of a 3D shape into a single and compact shape descriptor offering even better recognition performance. The same architecture can be applied to accurately recognize human hand drawn sketches of shapes. We conclude that a collection of 2D views can be highly informative for 3D shape recognition and is amenable to emerging CNN architectures and their derivatives.
2015 IEEE International Conference on Computer Vision (ICCV)
Hang Su Subhransu Maji Evangelos Kalogerakis Erik G Learned-Miller
52044787
GVCNN: Group View Convolutional Neural Networks for 3D Shape Recognition
3D shape recognition has attracted much attention recently. Its recent advances advocate the usage of deep features and achieve the state of the art performance. However, existing deep features for 3D shape recognition are restricted to a view to shape setting, which learns the shape descriptor from the view level feature directly. Despite the exciting progress on view based 3D shape description, the intrinsic hierarchical correlation and discriminability among views have not been well exploited, which is important for 3D shape representation. To tackle this issue, in this paper, we propose a group view convolutional neural network (GVCNN) framework for hierarchical correlation modeling towards discriminative 3D shape description. The proposed GVCNN framework is composed of a hierarchical view group shape architecture, i.e. from the view level, the group level and the shape level, which are organized using a grouping strategy. Concretely, we first use an expanded CNN to extract a view level descriptor. Then, a grouping module is introduced to estimate the content discrimination of each view, based on which all views can be splitted into different groups according to their discriminative level. A group level description can be further generated by pooling from view descriptors. Finally, all group level descriptors are combined into the shape level descriptor according to their discriminative weights. Experimental results and comparison with state of the art methods show that our proposed GVCNN method can achieve a significant performance gain on both the 3D shape classification and retrieval tasks.
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Yifan Feng Zizhao Zhang Xibin Zhao R Ji Yue Gao
52077468
PVNet: A Joint Convolutional Network of Point Cloud and Multi View for 3D Shape Recognition
3D object recognition has attracted wide research attention in the field of multimedia and computer vision. With the recent proliferation of deep learning, various deep models with different representations have achieved the state of the art performance. Among them, point cloud and multi view based 3D shape representations are promising recently, and their corresponding deep models have shown significant performance on 3D shape recognition. However, there is little effort concentrating point cloud data and multi view data for 3D shape representation, which is, in our consideration, beneficial and compensated to each other. In this paper, we propose the Point View Network (PVNet) the first framework integrating both the point cloud and the multi view data towards joint 3D shape recognition. More specifically, an embedding attention fusion scheme is proposed that could employ high level features from the multi view data to model the intrinsic correlation and discriminability of different structure features from the point cloud data. In particular, the discriminative descriptions are quantified and leveraged as the soft attention mask to further refine the structure feature of the 3D shape. We have evaluated the proposed method on the ModelNet40 dataset for 3D shape classification and retrieval tasks. Experimental results and comparisons with state of the art methods demonstrate that our framework can achieve superior performance.
ACM Multimedia
Haoxuan You Yifan Feng R Ji Yue Gao
206441208
DeepPano: Deep Panoramic Representation for 3 D Shape Recognition
This letter introduces a robust representation of 3 D shapes, named DeepPano, learned with deep convolutional neural networks (CNN) Firstly, each 3 D shape is converted into a panoramic view, namely a cylinder projection around its principle axis. Then, a variant of CNN is specifically designed for learning the deep representations directly from such views. Different from typical CNN, a row wise max pooling layer is inserted between the convolution and fully connected layers, making the learned representations invariant to the rotation around a principle axis. Our approach achieves state of the art retrieval/classification results on two large scale 3 D model datasets (ModelNet 10 and ModelNet 40) outperforming typical methods by a large margin.
IEEE Signal Processing Letters
Baoguang Shi Song Bai Zhichao Zhou Xiang Bai
26491290
Improving 3D Shape Recognition withElectrostatic Friction Display
Electrovibration technology has the potential for seamless integration into ordinary smartphones and tablets to provide programmable haptic feedback. The aim of this work is to seek effective ways to improve 3D perception of visual objects rendered on an electrovibration display. Utilizing a gradient based algorithm, we first investigated whether rendering only lateral frictional force on an electrovibration display improves 3D shape perception compared to doing the same using a force feedback interface. We observed that although users do not naturally associate electrovibration patterns to geometrical shapes, they can map patterns to shapes with moderate accuracy if guidance or context is given. Motivated by this finding, we generalized the gradient based rendering algorithm to estimate the surface gradient for any 3D mesh and added an edge detection algorithm to render sharp edges. Then, we evaluated the advantages of our algorithm in a user study and found that our algorithm can notably improve the performance of 3D shape recognition when visual information is limited.
IEEE Transactions on Haptics
Reza Haghighi Osgouei Jin Ryong Kim Seungmoon Choi
20432407
Deformation driven shape correspondence via shape recognition
Many approaches to shape comparison and recognition start by establishing a shape correspondence. We "turn the table" and show that quality shape correspondences can be obtained by performing many shape recognition tasks. What is more, the method we develop computes a fine grained, topology varying part correspondence between two 3D shapes where the core evaluation mechanism only recognizes shapes globally. This is made possible by casting the part correspondence problem in a deformation driven framework and relying on a data driven "deformation energy" which rates visual similarity between deformed shapes and models from a shape repository. Our basic premise is that if a correspondence between two chairs (or airplanes, bicycles, etc. is correct, then a reasonable deformation between the two chairs anchored on the correspondence ought to produce plausible, "chair like" in between shapes. Given two 3D shapes belonging to the same category, we perform a top down, hierarchical search for part correspondences. For a candidate correspondence at each level of the search hierarchy, we deform one input shape into the other, while respecting the correspondence, and rate the correspondence based on how well the resulting deformed shapes resemble other shapes from ShapeNet belonging to the same category as the inputs. The resemblance, i.e. plausibility, is measured by comparing multi view depth images over category specific features learned for the various shape categories. We demonstrate clear improvements over state of the art approaches through tests covering extensive sets of man made models with rich geometric and topological variations.
ACM Trans. Graph.
Chenyang Zhu Renjiao Yi Wallace P Lira Ibraheem Alhashim Kai Xu Hao Zhang
27111283
Shape recognition by bag of skeleton associated contour parts
We associate a contour point with an object thickness value.We propose a new descriptor to combine both contour and skeleton information.Our method achieves the state of the arts results on several shape dataset. Contour and skeleton are two complementary representations for shape recognition. However combining them in a principal way is nontrivial, as they are generally abstracted by different structures (closed string vs graph) respectively. This paper aims at addressing the shape recognition problem by combining contour and skeleton according to the correspondence between them. The correspondence provides a straightforward way to associate skeletal information with a shape contour. More specifically, we propose a new shape descriptor, named Skeleton associated Shape Context (SSC) which captures the features of a contour fragment associated with skeletal information. Benefited from the association, the proposed shape descriptor provides the complementary geometric information from both contour and skeleton parts, including the spatial distribution and the thickness change along the shape part. To form a meaningful shape feature vector for an overall shape, the Bag of Features framework is applied to the SSC descriptors extracted from it. Finally, the shape feature vector is fed into a linear SVM classifier to recognize the shape. The encouraging experimental results demonstrate that the proposed way to combine contour and skeleton is effective for shape recognition, which achieves the state of the art performances on several standard shape benchmarks.
Pattern Recognit. Lett.
Wei Shen Yuan Jiang Wenjing Gao Dan Zeng Xinggang Wang
125715680
Leaf Shape Recognition using Centroid Contour Distance
This research recognizes the leaf shape using Centroid Contour Distance (CCD) as shape descriptor. CCD is an algorithm of shape representation contour based approach which only exploits boundary information. CCD calculates the distance between the midpoint and the points on the edge corresponding to interval angle. Leaf shapes that included in this study are ellips, cordate, ovate, and lanceolate. We analyzed 200 leaf images of tropical plant. Each class consists of 50 images. The best accuracy is obtained by 96.67% We used Probabilistic Neural Network to classify the leaf shape. Experimental results demonstrated the effectiveness of the proposed approach for shape recognition with high accuracy.
Abdurrasyid Hasim Yeni Herdiyeni Stephane Douady