2957894
InteMon: intelligent system monitoring on large clusters
InteMon is a prototype monitoring and mining system for large clusters. Currently, it monitors over 100 hosts of a prototype data center at CMU. It uses the SNMP protocol and it stores the monitoring data in an mySQL database. Then, it allows for visualization of the time-series data using a JSP web-based frontend interface for users.What sets it apart from other cluster monitoring systems is its ability to automatically analyze the monitoring data in real time and alert the users for potential anomalies. It uses state of the art stream mining methods, it has a sophisticated definition of anomalies (broken correlations among input streams), and it can also pinpoint the reason of the anomaly. InteMon has a user-friendly GUI, it allows the users to perform interactive mining tasks, and it is fully operational.
16117277
Hiding in the Crowd: Privacy Preservation on Evolving Streams through Correlation Tracking
We address the problem of preserving privacy in streams, which has received surprisingly limited attention. For static data, a well-studied and widely used approach is based on random perturbation of the data values. However, streams pose additional challenges. First, analysis of the data has to be performed incrementally, using limited processing time and buffer space, making batch approaches unsuitable. Second, the characteristics of streams evolve over time. Consequently, approaches based on global analysis of the data are not adequate. We show that it is possible to efficiently and effectively track the correlation and autocorrelation structure of multivariate streams and leverage it to add noise which maximally preserves privacy, in the sense that it is very hard to remove. Our techniques achieve much better results than previous static, global approaches, while requiring limited processing time and memory. We provide both a mathematical analysis and experimental evaluation on real data to validate the correctness, efficiency, and effectiveness of our algorithms.
8149166
Analysis of predictive spatio-temporal queries
Given a set of objects <i>S</i>, a spatio-temporal <i>window query q</i> retrieves the objects of <i>S</i> that will intersect the window during the (future) interval <i>q</i><sub><i>T</i></sub>. A <i>nearest neighbor query q</i> retrieves the objects of <i>S</i> closest to <i>q</i> during <i>q</i><sub><i>T</i></sub>. Given a threshold <i>d</i>, a spatio-temporal <i>join</i> retrieves the pairs of objects from two datasets that will come within distance <i>d</i> from each other during <i>q</i><sub><i>T</i></sub>. In this article, we present probabilistic cost models that estimate the <i>selectivity</i> of spatio-temporal window queries and joins, and the <i>expected distance</i> between a query and its nearest neighbor(s). Our models capture any query/object mobility combination (moving queries, moving objects or both) and any data type (points and rectangles) in arbitrary dimensionality. In addition, we develop specialized spatio-temporal histograms, which take into account both location and velocity information, and can be incrementally maintained. Extensive performance evaluation verifies that the proposed techniques produce highly accurate estimation on both uniform and non-uniform data.
803846
Selectivity estimation for predictive spatio-temporal queries
We propose a cost model for selectivity estimation of predictive spatio-temporal window queries. Initially, we focus on uniform data proposing formulae that capture both points and rectangles, and any type of object/query mobility combination (i.e., dynamic objects, dynamic queries or both). Then, we apply the model to nonuniform datasets by introducing spatio-temporal histograms, which in addition to the spatial, also consider the velocity distributions during partitioning. The advantages of our techniques are (i) high accuracy (1-2 orders of magnitude lower error than previous techniques), (ii) ability to handle all query types, and (iii) efficient handling of updates.
6738109
Neighborhood formation and anomaly detection in bipartite graphs
Many real applications can be modeled using bipartite graphs, such as users vs. files in a P2P system, traders vs. stocks in a financial trading system, conferences vs. authors in a scientific publication network, and so on. We introduce two operations on bipartite graphs: 1) identifying similar nodes (Neighborhood formation), and 2) finding abnormal nodes (Anomaly detection). And we propose algorithms to compute the neighborhood for each node using random walk with restarts and graph partitioning; we also propose algorithms to identify abnormal nodes, using neighborhood information. We evaluate the quality of neighborhoods based on semantics of the datasets, and we also measure the performance of the anomaly detection algorithm with manually injected anomalies. Both effectiveness and efficiency of the methods are confirmed by experiments on several real datasets.
60837177
Indexing and Retrieval of Historical Aggregate Information about Moving Objects
Spatio-temporal databases store information about the positions of individual objects over time. In many applications however, such as traffic supervision or mobile communication systems, only summarized data, like the average number of cars in an area for a specific period, or phones serviced by a cell each day, is required. Although this information can be obtained from operational databases, its computation is expensive, rendering online processing inapplicable. A vital solution is the construction of a spatiotemporal data warehouse. In this paper, we describe a framework for supporting OLAP operations over spatiotemporal data. We argue that the spatial and temporal dimensions should be modeled as a combined dimension on the data cube and present data structures, which integrate spatiotemporal indexing with pre-aggregation. While the well-known materialization techniques require a-priori knowledge of the grouping hierarchy, we develop methods that utilize the proposed structures for efficient execution of ad-hoc group-bys. Our techniques can be used for both static and dynamic dimensions.
11369262
GE-CKO: A Method to Optimize Composite Kernels for Web Page Classification
Most of current researches on Web page classification focus on leveraging heterogeneous features such as plain text, hyperlinks and anchor texts in an effective and efficient way. Composite kernel method is one topic of interest among them. It first selects a bunch of initial kernels, each of which is determined separately by a certain type of features. Then a classifier is trained based on a linear combination of these kernels. In this paper, we propose an effective way to optimize the linear combination of kernels. We proved that this problem is equivalent to solving a generalized eigenvalue problem. And the weight vector of the kernels is the eigenvector associated with the largest eigen-value. A support vector machine (SVM) classifier is then trained based on this optimized combination of kernels. Our experiment on the WebKB dataset has shown the effectiveness of our proposed method.
46084829
User Adaptation for Online Sketchy Shape Recognition
This paper presents a method of online sketchy shape recognition that can adapt to different user sketching styles. The adaptation principle is based on incremental active learning and dynamic user modeling. Incremental active learning is used for sketchy stroke classification such that important data can actively be selected to train the classifiers. Dynamic user modeling is used to model the user's sketching style in an incremental decision tree, which is then used to recognize the composite shapes dynamically by means of fuzzy matching. Experiments prove the proposed method both effective and efficient for user adaptation in online sketchy shape recognition.
10578152
CubeSVD: a novel approach to personalized Web search
As the competition of Web search market increases, there is a high demand for personalized Web search to conduct retrieval incorporating Web users' information needs. This paper focuses on utilizing clickthrough data to improve Web search. Since millions of searches are conducted everyday, a search engine accumulates a large volume of clickthrough data, which records who submits queries and which pages he/she clicks on. The clickthrough data is highly sparse and contains different types of objects (user, query and Web page), and the relationships among these objects are also very complicated. By performing analysis on these data, we attempt to discover Web users' interests and the patterns that users locate information. In this paper, a novel approach CubeSVD is proposed to improve Web search. The clickthrough data is represented by a 3-order tensor, on which we perform 3-mode analysis using the higher-order singular value decomposition technique to automatically capture the latent factors that govern the relations among these multi-type objects: users, queries and Web pages. A tensor reconstructed based on the CubeSVD analysis reflects both the observed interactions among these objects and the implicit associations among them. Therefore, Web search activities can be carried out based on CubeSVD analysis. Experimental evaluations using a real-world data set collected from an MSN search engine show that CubeSVD achieves encouraging search results in comparison with some standard methods.
62600747
Automatic cascade training with perturbation bias
Face detection methods based on cascade architecture have demonstrated fast and robust performance. Cascade learning is aided by the modularity of the architecture in which nodes are chained together to form a cascade. In this paper we present two new cascade learning results which address the decoupled nature of the cascade learning task. First, we introduce a cascade indifference curve framework, which connects the learning objectives for a node to the overall cascade performance. We derive a new cost function for node learning, which yields fully-automatic stopping conditions and improved detection performance. Second, we introduce the concept of perturbation bias, which leverages the statistical differences between target and non-target classes in a detection problem to obtain improved performance and robustness. We derive necessary and sufficient conditions for the success of the method and present experimental results.
14749669
Predicting Large Population Data Cumulative Match Characteristic Performance from Small Population Data
Given a biometric feature-space, in this paper we present a method to predict cumulative match characteristic (CMC) curve performance for a large population of individuals using a significantly smaller population to make the prediction. This is achieved by mathematically modelling the CMC curve. For a given biometric technique that extracts measurements of individuals to be used for identification, the CMC curve shows the probability of recognizing that individual within a database of measurements that are extracted from multiple individuals. As the number of individuals in the database increase, the probabilities displayed on the CMC curve decrease, which indicate the decreasing ability of the biometric technique to recognize individuals. Our mathematical model replicates this effect, and allows us to predict the identification performance of a technique as more individuals are added without physically needing to extract measurements from more individuals.