9803776
Semantic propagation from relevance feedbacks
Relevance feedback has been a very useful tool to enhance the performance of content-based information retrieval (CBIR) systems. To fully make use of the precious user feedback provided to a system, we propose an approach named semantic propagation, which reveals the deep semantic relationships among objects in the database, given a set of relevance feedbacks between object pairs. In particular, we present two semantic propagation algorithms that are applicable to CIBR systems with a feature vector space model and a general metric space model, respectively. Experiments on a 3D model retrieval system and a logo image retrieval system are performed to show the effectiveness of the proposed methods.
15648497
Interactive browsing of 3D environment over the Internet
In this paper, we describe a system for wandering in a realistic environment over the Internet. The environment is captured by the concentric mosaic, compressed via the reference block coder (RBC), and accessed and delivered over the Internet through the virtual media (Vmedia) access protocol. Capturing the environment through the concentric mosaic is easy. We mount a camera at the end of a level beam, and shoot images as the beam rotates. The huge dataset of the concentric mosaic is then compressed through the RBC, which is specifically designed for both high compression efficiency and just-in-time (JIT) rendering. Through the JIT rendering function, only a portion of the RBC bitstream is accessed, decoded and rendered for each virtual view. A multimedia communication protocol -- the Vmedia protocol, is then proposed to deliver the compressed concentric mosaic data over the Internet. Only the bitstream segments corresponding to the current view are streamed over the Internet. Moreover, the delivered bitstream segments are managed by a local Vmedia cache so that frequently used bitstream segments need not be streamed over the Internet repeatedly, and the Vmedia is able to handle a RBC bitstream larger than its memory capacity. A Vmedia concentric mosaic interactive browser is developed where the user can freely wander in a realistic environment, e.g., rotate around, walk forward/backward and sidestep, even under a tight bandwidth of 33.6 kbps.
2141991
Spectral analysis for sampling image-based rendering data
Image-based rendering (IBR) has become a very active research area in recent years. The spectral analysis problem for IBR has not been completely solved. In this paper, we present a new method to parameterize the problem, which is applicable for general-purpose IBR spectral analysis. We notice that any plenoptic function is generated by light ray emitted/reflected/refracted from the object surface. We introduce the surface plenoptic function (SPF), which represents the light rays starting from the object surface. Given that radiance along a light ray does not change unless the light ray is blocked, SPF reduces the dimension of the original plenoptic function to 6D. We are then able to map or transform the SPF to IBR representations captured along any camera trajectory. Assuming some properties on the SPF, we can analyze the properties of IBR for generic scenes such as scenes with Lambertian or non-Lambertian surfaces and scenes with or without occlusions, and for different sampling strategies such as lightfield/concentric mosaic. We find that in most cases, even though the SPF may be band-limited, the frequency spectrum of IBR is not band-limited. We show that non-Lambertian reflections, depth variations and occlusions can all broaden the spectrum, with the latter two being more significant. SPF is defined for scenes with known geometry. When the geometry is unknown, spectral analysis is still possible. We show that with the "truncating windows" analysis and some conclusions obtained with SPF, the spectrum expansion caused by non-Lambertian reflections and occlusions can be quantatively estimated, even when the scene geometry is not explicitly known. Given the spectrum of IBR, we also study how to sample IBR data more efficiently. Our analysis is based on the generalized periodic sampling theory with arbitrary geometry. We show that the sampling efficiency can be up to twice of that when we use rectangular sampling. The advantages and disadvantages of generalized periodic sampling for IBR are also discussed.
6161007
Light field capturing with lensless cameras
We introduce a novel approach to capturing light field with lensless cameras. By moving the cameras back and forth, we capture a set of images. We show that it is possible to reconstruct the light field from these blurry images. The problem is formulized in a way similar to computer tomography, so that the light field can be reconstructed using existing algorithms. The light field can then be used to render 3D scenes. Synthetic examples are presented to show the effectiveness of the proposed method.
5718237
Compression and rendering of concentric mosaics with reference block codec (RBC)
Concentric mosaics have the ability to quickly capture a complete 3D view of a realistic environment and to enable a user to wander freely in the environment. However, the data amount of the concentric mosaics is huge. In this paper, we propose an algorithm to compress the concentric mosaic image array through motion compensation and residue coding, which we called reference block codec. A two-level index table is embedded in the compressed bitstream for random access. During the rendering, the entire compressed concentric mosaic scene is not fully expanded at any time. Instead, only the contents necessary to render the current view are decoded in real time. We denote such rendering scheme as just-in-time rendering. Four decoder caches are implemented to speed up the rendering.
12091703
Indexing and retrieval of 3D models aided by active learning
We demonstrate a system for indexing and retrieval of 3D models aided by active learning. We propose a new set of region-based features for 3D models. Each model is treated as a solid volume with a uniform density. Features such as the volume-surface ratio, the moment invariants and the Fourier transform coefficients are efficiently calculated from the mesh model directly. Comparable retrieval performance is achieved with other features such as the cord histogram, the 3D shape spectrum, etc. To further improve the performance, we incorporate hidden annotation into our system. We propose to use active learning to improve the annotation efficiency. We show that with active learning, the system can perform better than random annotation, and the retrieval result improves rapidly with the number of annotated samples. Moreover, relevance feedback is included in the system and combined with active learning, which provides better user-adoptive retrieval results.
45564986
Mining High-Level User Concepts with Multiple Instance Learning and Relevance Feedback for Content-Based Image Retrieval
Understanding and learning the subjective aspect of humans in Content-Based Image Retrieval has been an active research field during the past few years. However, how to effectively discover users' concept patterns when there are multiple visual features existing in the retrieval system still remains a big issue. In this book chapter, we propose a multimedia data mining framework that incorporates Multiple Instance Learning into the user relevance feedback in a seamless way to discover the concept patterns of users, especially where the user's most interested region and how to map the local feature vector of that region to the high-level concept pattern of users. This underlying mapping can be progressively discovered through the feedback and learning procedure. The role the user plays in the retrieval system is to guide the system mining process to his/her own focus of attention. The retrieval performance is tested to show the feasibility and effectiveness of the proposed multimedia data mining framework.
40273501
Pattern Discovery in Probabilistic Databases
Modeling probabilistic data is one of important issues in databases due to the fact that data is often uncertainty in real-world applications. So, it is necessary to identify potentially useful patterns in probabilistic databases. Because probabilistic data in 1NF relations is redundant, previous mining techniques don't work well on probabilistic databases. For this reason, this paper proposes a new model for mining probabilistic databases. A partition is thus developed for preprocessing probabilistic data in a probabilistic databases. We evaluated the proposed technique, and the experimental results demonstrate that our approach is effective and efficient.
18082444
Circuit Decompositions of Eulerian Graphs
Let G be an eulerian graph. For each vertex v?V(G), let P(v) be a partition of the edges incident with v and set P=?v?V(G)P(v), called a forbidden system of G. We say that P is admissible if |P?T|?12|T| for every P?P and every edge cut T of G. H. Fleischner and A. Frank (1990, J. Combin. Theory Ser. B50, 245?253) proved that if G is planar and P is any admissible forbidden system of G, then G has a circuit decomposition F such that |C?P|?1 for every C?F and every P?P. We generalize this result to all eulerian graphs that do not contain K5 as a minor. As a consequence, a conjecture of Sabidussi is settled for graphs that do not contain K5 as a minor. Also, as a byproduct, our proof provides a different approach to the circuit cover theorem of B. Alspach, L. A. Goddyn, and C.-Q. Zhang (1994, Trans. Amer. Math. Soc.344, No. 1, 131?154).
9917703
Multiple object retrieval for image databases using multiple instance learning and relevance feedback
The paper proposes a method to discover effectively users' concept patterns when multiple objects of interest (e.g., foreground and background objects) are involved in content-based image retrieval. The proposed method incorporates multiple instance learning into the user relevance feedback in a seamless way to discover where the user's objects/regions of most interest are and how to map the local features of that(those) region(s) to the user's high-level concepts. A three-layer neural network is used to model the underlying mapping progressively through the feedback and learning procedure
15778796
MIRAI: Multi-hierarchical, FS-Tree Based Music Information Retrieval System
With the fast booming of online music repositories, there is a need for content-based automatic indexing which will help users to find their favorite music objects in real time. Recently, numerous successful approaches on musical data feature extraction and selection have been proposed for instrument recognition in monophonic sounds. Unfortunately, none of these methods can be successfully applied to polyphonic sounds. Identification of music instruments in polyphonic sounds is still difficult and challenging, especially when harmonic partials are overlapping with each other. This has stimulated the research on music sound separation and new features development for content-based automatic music information retrieval. Our goal is to build a cooperative query answering system (QAS), for a musical database, retrieving from it all objects satisfying queries like "find all musical pieces in pentatonic scale with a viola and piano where viola is playing for minimum 20 seconds and piano for minimum 10 seconds". We use the database of musical sounds, containing almost 4000 sounds taken from the MUMs (McGill University Master Samples), as a vehicle to construct several classifiers for automatic instrument recognition. Classifiers showing the best performance are adopted for automatic indexing of musical pieces by instruments. Our musical database has an FS-tree (Frame Segment Tree) structure representation. The cooperativeness of QAS is driven by several hierarchical structures used for classifying musical instruments.