6161007
Light field capturing with lensless cameras
We introduce a novel approach to capturing light field with lensless cameras. By moving the cameras back and forth, we capture a set of images. We show that it is possible to reconstruct the light field from these blurry images. The problem is formulized in a way similar to computer tomography, so that the light field can be reconstructed using existing algorithms. The light field can then be used to render 3D scenes. Synthetic examples are presented to show the effectiveness of the proposed method.
15648497
Interactive browsing of 3D environment over the Internet
In this paper, we describe a system for wandering in a realistic environment over the Internet. The environment is captured by the concentric mosaic, compressed via the reference block coder (RBC), and accessed and delivered over the Internet through the virtual media (Vmedia) access protocol. Capturing the environment through the concentric mosaic is easy. We mount a camera at the end of a level beam, and shoot images as the beam rotates. The huge dataset of the concentric mosaic is then compressed through the RBC, which is specifically designed for both high compression efficiency and just-in-time (JIT) rendering. Through the JIT rendering function, only a portion of the RBC bitstream is accessed, decoded and rendered for each virtual view. A multimedia communication protocol -- the Vmedia protocol, is then proposed to deliver the compressed concentric mosaic data over the Internet. Only the bitstream segments corresponding to the current view are streamed over the Internet. Moreover, the delivered bitstream segments are managed by a local Vmedia cache so that frequently used bitstream segments need not be streamed over the Internet repeatedly, and the Vmedia is able to handle a RBC bitstream larger than its memory capacity. A Vmedia concentric mosaic interactive browser is developed where the user can freely wander in a realistic environment, e.g., rotate around, walk forward/backward and sidestep, even under a tight bandwidth of 33.6 kbps.
12091703
Indexing and retrieval of 3D models aided by active learning
We demonstrate a system for indexing and retrieval of 3D models aided by active learning. We propose a new set of region-based features for 3D models. Each model is treated as a solid volume with a uniform density. Features such as the volume-surface ratio, the moment invariants and the Fourier transform coefficients are efficiently calculated from the mesh model directly. Comparable retrieval performance is achieved with other features such as the cord histogram, the 3D shape spectrum, etc. To further improve the performance, we incorporate hidden annotation into our system. We propose to use active learning to improve the annotation efficiency. We show that with active learning, the system can perform better than random annotation, and the retrieval result improves rapidly with the number of annotated samples. Moreover, relevance feedback is included in the system and combined with active learning, which provides better user-adoptive retrieval results.
11474786
A Self-Reconfigurable Camera Array
This paper presents a self-reconfigurable camera array system that captures video sequences from an array of mobile cameras, renders novel views on the fly and reconfigures the camera positions to achieve better rendering quality. The system is composed of 48 cameras mounted on mobile platforms. The contribution of this paper is twofold. First, we propose an efficient algorithm that is capable of rendering high-quality novel views from the captured images. The algorithm reconstructs a view-dependent multi-resolution 2D mesh model of the scene geometry on the fly and uses it for rendering. The algorithm combines region of interest (ROI) identification, JPEG image decompression, lens distortion correction, scene geometry reconstruction and novel view synthesis seamlessly on a single Intel Xeon 2.4 GHz processor, which is capable of generating novel views at 4-10 frames per second (fps). Second, we present a view-dependent adaptive capturing scheme that moves the cameras in order to show even better rendering results. Such camera reconfiguration naturally leads to a nonuniform arrangement of the cameras on the camera plane, which is both view-dependent and scene-dependent.
15778343
Smart rebinning for compression of concentric mosaics
Concentric mosaics offer a quick solution to construct a virtual copy of a real environment, and navigate in the virtual environment. However, the huge amount of data associated with concentric mosaics is a heavy burden for its application. A 3D wavelet transform-based compressor has been proposed in previous work to compress the concentric mosaics. In this paper, we greatly improve the performance of the 3D wavelet coder with a data rearrangement mechanism called “smart rebinning”. The proposed scheme first aligns the concentric mosaic image shots along the horizontal direction and then rebins the shots into multi-perspective panoramas. Smart rebinning greatly improves the cross shot correlation and enables the coder to better explore the redundancy among shots. Experimental results show that the performance of the 3D wavelet coder improves an average of 4.3dB with the use of smart rebinning. The proposed coder outperforms MPEG-2 coding of concentric mosaics by an average of 3.7dB.
10867502
Robust Visual Tracking via Pixel Classification and Integration
We propose a novel framework for tracking non-rigid objects via pixel classification and integration (PCI). Given a new input frame, the tracker first performs object classification on each pixel and then finds the region that has the highest integral of scores. There are several key advantages of the proposed approach: it is computationally very efficient; it finds a global, instead of local (e.g., mean-shift), optimal solution within a search range; and it is inherently robust to different object scales with minimum extra computation. Within this framework, a mixture of long-term and short-term appearance model is further introduced to perform PCI. As a result, the tracker is able to adapt to both slow and rapid appearance changes without drifting. Challenging video sequences are presented to illustrate how the proposed tracker handles large motion, dramatic shape changes, scale variations, illumination variations and partial occlusions
9240156
TeraScope: distributed visual data mining of terascale data sets over photonic networks
TeraScope is a framework and a suite of tools for interactively browsing and visualizing large terascale data sets. Unique to TeraScope is its utilization of the Optiputer paradigm to treat distributed computer clusters as a single giant computer, where the dedicated optical networks that connect the clusters serve as the computer's system bus. TeraScope explores one aspect of the Optiputer architecture by employing a distributed pool of memory, called LambdaRAM, that serves as a massive data cache for supporting parallel data mining and visualization algorithms.
5113927
An unsupervised segmentation framework for texture image queries
In this paper a novel unsupervised segmentation framework for texture image queries is presented. The proposed framework consists of an unsupervised segmentation method for texture images, and a multi-filter query strategy. By applying the unsupervised segmentation method on each texture image, a set of texture feature parameters for that texture image can be extracted automatically. Based upon these parameters, an effective multi-filter query strategy which allows the users to issue texture-based image queries is developed The test results of the proposed framework on 318 texture images obtained from the MIT VisTex and Brodatz database are presented to show its effectiveness.
207326756
Estimating the amount of defocus through a wavelet transform approach
This paper presents a new technique for defocus estimation of a captured image. In our method, a ratio of the wavelet coefficients of high frequency correspond to a same image point at two different levels is used. For an edge point, it is shown that the ratio is related to the amount of defocus. Let α be the ratio of wavelet coefficients at the first level to that at the second level. The value of α decreases as the amount of defocus increase. In our experiments of iris image analysis, when α is larger than 0.5 the number of feature points in an image almost remain constant. It means that the image is little defocused and available for image recognition. Compared with Fourier methods, this technique is more robust. In addition, this method is fast enough to be used in auto-focus system for tracking moving objects.
9156106
Blind Signal Separation of Similar Pitches and Instruments in a Noisy Polyphonic Domain
In our continuing work on ”Blind Signal Separation” this paper focuses on extending our previous work [1] by creating a data set that can successfully perform blind separation of polyphonic signals containing similar instruments playing similar notes in a noisy environment. Upon isolating and subtracting the dominant signal from a base signal containing varying types and amounts of noise, even though we purposefully excluded any identical matches in the dataset, the signal separation system successfully built a resulting foreign set of synthesized sounds that the classifier correctly recognized. Herein, this paper presents a system that classifies and separates two harmonic signals with added noise. This novel methodology incorporates Knowledge Discovery, MPEG7-based segmentation and Inverse Fourier Transforms.
5200887
Damage pattern mining in hurricane image databases
We present a damage pattern mining framework for hurricane data on residential houses using aerial photographs with 1:3000 based scale. The vertical photographs are normally collected for an overview of a disaster area or for more detailed assessment. Damage on roof, especially for shingles torn off, is expected to be discovered in a more efficient and automatic way instead of going through the high-resolution aerial photograph for numerous details. The discovered damage patterns can then be used for hurricane damage assessment and impacts on different geographical areas. Our methodology is: (1) applying a novel and effective segmentation method on each residential house on the aerial photograph of one community, (2) using the segmentation results to obtain a set of indexing parameters for each house representing the damage level of roof cladding as well as the patterns of damage, (3) using these parameters to select several templates representing the damage patterns so that users can issue query-by-example (QBE) queries. The proposed segmentation method is an unsupervised simultaneous partition and class parameter estimation algorithm that considers the problem of segmentation as a joint estimation of the partition and class parameter variables. By utilizing this segmentation method, the indexing parameters can be obtained automatically. The QBE capability can assist in finding similar damage patterns on the roof of the residential houses in different locations in the image databases. Experiments based on the aerial photographs of Hurricane Andrew in 1992 are conducted and analyzed to show the effectiveness of the proposed hurricane damage pattern mining framework.