5996048
Scaling up all pairs similarity search
Given a large collection of sparse vector data in a high dimensional space, we investigate the problem of finding all pairs of vectors whose similarity score (as determined by a function such as cosine distance) is above a given threshold. We propose a simple algorithm based on novel indexing and optimization strategies that solves this problem without relying on approximation methods or extensive parameter tuning. We show the approach efficiently handles a variety of datasets across a wide setting of similarity thresholds, with large speedups over previous state-of-the-art approaches.
1599574
Interactive Refinement of Filtering Queries on Streaming Intelligence Data
Intelligence analysis involves routinely monitoring and correlating large amount of data streaming from multiple sources. In order to detect important patterns, the analyst normally needs to look at data gathered over a certain time window. Given the size of data and rate at which it arrives, it is usually impossible to manually process every record or case. Instead, automated filtering (classification) mechanisms are employed to identify information relevant to the analyst’s task. In this paper, we present a novel system framework called FREESIA (Filter REfinement Engine for Streaming InformAtion) to effectively generate, utilize and update filtering queries on streaming data.
9332901
On-Demand Information Portals for Disaster Situations
This paper describes our work on developing technology for rapidly assembling information portals that provide integrated access to and analysis of information from multiple sources in the case of any disaster. Many recent disasters (the S.E. Asian Tsunamis, the London subway bombings, the Katrina hurricane, to name a few) have demonstrated that a lot of valuable information becomes available in the hours and days immediately following the disaster, and such information is indeed valuable to disaster managers or even citizens in their response. In this paper we describe our work on developing information portals for disasters in general; we describe many key information processing capabilities and challenges that we consider important in such portals and also describe our approach to developing such capabilities.
2798845
Modeling and querying uncertain spatial information for situational awareness applications
Situational awareness (SA) applications monitor the real world and the entities therein to support tasks such as rapid decision-making, reasoning, and analysis. Raw input about unfolding events may arrive from variety of sources in the form of sensor data, video streams, human observations, and so on, from which events of interest are extracted. Location is one of the most important attributes of events, useful for a variety of SA tasks. In this paper, we propose an approach to model and represent (potentially uncertain) event locations described by human reporters in the form of free text. We analyze several types of spatial queries of interest in SA applications. Our experimental evaluation demonstrates the effectiveness of our approach.
14652056
A framework for refining similarity queries using learning techniques
In numerous applications that deal with similarity search, a user may not have an exact idea of his information need and/or may not be able to construct a query that exactly captures his notion of similarity. A promising approach to mitigate this problem is to enable the user to submit a rough approximation of the desired query and use the feedback on the relevance of the retrieved objects to refine the query. In this paper, we explore such a refinement strategy for a general class of SQL similarity queries. Our approach casts the refinement problem as that of learning concepts using examples. This is achieved by viewing the tuples on which a user provides feedback as a labeled training set for a learner. Under this setup, SQL query refinement consists of two learning tasks, namely learning the structure of the SQL query and learning the relative importance of the query components. The paper develops appropriate machine learning approaches suitable for these two learning tasks. The primary contribution of the paper is a general refinement framework that decides when each learner is invoked in order to quickly learn the user query. Experimental analyses over many real life datasets and queries show that our strategy outperforms the existing approaches significantly in terms of retrieval accuracy and query simplicity.
9927329
CAMAS: a citizen awareness system for crisis mitigation
1 ITR-Rescue, 5251 California Avenue, Suite 210, Irvine CA 92612-2815, Phone: 949-824-1147 http://www.itr-rescue.org 2 Union Bank Building, 400 Oceangate, Suite 1050, Long Beach CA 90802 Phone: 562-628-1675, http://imagecatinc.com ABSTRACT This demo paper provides a brief description of the intuition and design philosophy of CAMAS, one of the main testbeds being developed in the context of the RESCUE (Responding to the Unexpected) project [9]. The goal of our work is to enhance the mitigation capabilities of first responders in the event of a crisis by dramatically transforming their ability to collect, store, analyze, interpret, share and disseminate data. CAMAS specifically is a system designed to allow a variety of users, including the average citizen to report incidents and potentially hazardous situations, analyze and evaluate these reports and notify appropriate personnel for further action. The multidisciplinary approach incorporates a variety of information technologies: networks; distributed systems; databases; image and video processing; GIS; and machine learning, together with subjective information obtained through social science. Besides providing an overview of the CAMAS architecture, we describe the demonstration platform and specific experiments that will illustrate issues of event extraction, ranking, access control and visualization.
696914
Generalized principal component analysis (GPCA)
This paper presents an algebro-geometric solution to the problem of segmenting an unknown number of subspaces of unknown and varying dimensions from sample data points. We represent the subspaces with a set of homogeneous polynomials whose degree is the number of subspaces and whose derivatives at a data point give normal vectors to the subspace passing through the point. When the number of subspaces is known, we show that these polynomials can be estimated linearly from data; hence, subspace segmentation is reduced to classifying one point per subspace. We select these points optimally from the data set by minimizing certain distance function, thus dealing automatically with moderate noise in the data. A basis for the complement of each subspace is then recovered by applying standard PCA to the collection of derivatives (normal vectors). Extensions of GPCA that deal with data in a high-dimensional space and with an unknown number of subspaces are also presented. Our experiments on low-dimensional data show that GPCA outperforms existing algebraic algorithms based on polynomial factorization and provides a good initialization to iterative techniques such as k-subspaces and expectation maximization. We also present applications of GPCA to computer vision problems such as face clustering, temporal video segmentation, and 3D motion segmentation from point correspondences in multiple affine views.
47404061
Diversity and Multiplexing Tradeoff in General Fading Channels
The optimal tradeoff between diversity gain and multiplexing gain for multiple-inputmultiple-output (MIMO) channels has been studied recently under the independent and identically distributed (i.i.d.) Rayleigh-fading assumption. In this correspondence, this result is extended and the optimal tradeoff performance is derived for generalized fading channel conditions, including different fading types, nonidentical fading distributions, spatial correlation, and nonzero channel means. Our results include many known models as special cases and shed light on the effects of different channel parameters on the optimal tradeoff performance
8131766
Segmentation of a piece-wise planar scene from perspective images
We study and compare two novel embedding methods for segmenting feature points of piece-wise planar structures from two (uncalibrated) perspective images. We show that a set of different homographies can be embedded in different ways to a higher-dimensional real or complex space, so that each homography corresponds to either a complex bilinear form or a real quadratic form. Each embedding reveals different algebraic properties and relations of homographies. We give a closed-form segmentation solution for each case by utilizing these properties based on subspace-segmentation methods. These theoretical results show that one can intrinsically segment a piece-wise planar scene from 2-D images without explicitly performing any 3-D reconstruction. The resulting segmentation may make subsequent 3-D reconstruction much better-conditioned. We demonstrate the proposed methods with some convincing experimental results.
6626830
Optimization Criteria and Geometric Algorithms for Motion and Structure Estimation
Prevailing efforts to study the standard formulation of motion and structure recovery have recently been focused on issues of sensitivity and robustness of existing techniques. While many cogent observations have been made and verified experimentally, many statements do not hold in general settings and make a comparison of existing techniques difficult. With an ultimate goal of clarifying these issues, we study the main aspects of motion and structure recovery: the choice of objective function, optimization techniques and sensitivity and robustness issues in the presence of noise.We clearly reveal the relationship among different objective functions, such as “(normalized) epipolar constraints,” “reprojection error” or “triangulation,” all of which can be unified in a new “optimal triangulation” procedure. Regardless of various choices of the objective function, the optimization problems all inherit the same unknown parameter space, the so-called “essential manifold.” Based on recent developments of optimization techniques on Riemannian manifolds, in particular on Stiefel or Grassmann manifolds, we propose a Riemannian Newton algorithm to solve the motion and structure recovery problem, making use of the natural differential geometric structure of the essential manifold.We provide a clear account of sensitivity and robustness of the proposed linear and nonlinear optimization techniques and study the analytical and practical equivalence of different objective functions. The geometric characterization of critical points and the simulation results clarify the difference between the effect of bas-relief ambiguity, rotation and translation confounding and other types of local minima. This leads to consistent interpretations of simulation results over a large range of signal-to-noise ratio and variety of configurations.
2268413
Web object retrieval
The primary function of current Web search engines is essentially relevance ranking at the document level. However, myriad structured information about real-world objects is embedded in static Web pages and online Web databases. Document-level information retrieval can unfortunately lead to highly inaccurate relevance ranking in answering object-oriented queries. In this paper, we propose a paradigm shift to enable searching at the object level. In traditional information retrieval models, documents are taken as the retrieval units and the content of a document is considered reliable. However, this reliability assumption is no longer valid in the object retrieval context when multiple copies of information about the same object typically exist. These copies may be inconsistent because of diversity of Web site qualities and the limited performance of current information extraction techniques. If we simply combine the noisy and inaccurate attribute information extracted from different sources, we may not be able to achieve satisfactory retrieval performance. In this paper, we propose several language models for Web object retrieval, namely an unstructured object retrieval model, a structured object retrieval model, and a hybrid model with both structured and unstructured retrieval features. We test these models on a paper search engine and compare their performances. We conclude that the hybrid model is the superior by taking into account the extraction errors at varying levels.