14876394
User Defined Aggregates for Logical Data Languages
A new wave of data-intensive and knowledge-based applications|such as data mining and decision support|require the introduction of complex application-speciic aggregate functions. In this paper, we propose extensions for deductive database systems to support these new applications. We develop constructs, formal semantics, and implementation techniques for user-deened aggregates, and describe their realization in an extended LDL++ system recently built at UCLA. With these extensions, the system can support on-line aggregation, roll-ups for data cubing, temporal aggregates for time-series, iceberg queries, and other recently proposed operators used in decision support and data mining procedures. We then discuss the application of this technology to other DBMSs, and in particular to the SQL3 speciications that support the notion of user-deened aggregates. We show that SQL3 suuers from limitations that severely restrict its use in new applications; thus we propose simple extensions similar to those used for LDL++ to overcome such limitations.
5103963
Adaptive Load Diffusion for Multiway Windowed Stream Joins
In this paper, we present an adaptive load diffusion operator to enable scalable processing of multiway windowed stream joins (MWSJs) using a cluster system. The load diffusion is achieved by a set of novel semantics-pre serving tuple routing algorithms. Different from previous work, the load diffusion operator can (1) preserve the MWSJ semantics while spreading tuples to different hosts for parallel join processing; (2) achieve fine-grained load balancing among distributed hosts; and (3) perform semantics-preserving online adaptations to maintain optimal performance in dynamic stream environments. We have implemented a prototype of the distributed MWSJ framework on top of the System S distributed stream processing system. Our experiment results based on both real data streams and synthetic workloads show that the load diffusion algorithms can efficiently scale-up the performance of MWSJ processing with low overhead.
2129781
ATLaS: A Native Extension of SQL for Data Mining
A lack of power and extensibility in their query languages has seriously limited the generality of DBMSs and hampered their ability to support new application domains. Considerable efforts by database researchers and commercial DBMS vendors have led to major extensions; yet there remain important applications—particularly data mining—that are not supported well in SQL-3. Thus, there is a pressing need for more general mechanisms for extending SQL and dealing with new application areas, particularly database-centric data mining. To satisfy this need, we allow database users to add new table functions and stream-oriented aggregate functions by defining them in SQL—rather than in external procedural languages as O-R DBMSs currently do. This simple extension turns SQL into a powerful database language, which can express a wide range of applications, including recursive queries, ROLAP aggregates, time-series queries, stream-oriented processing, and data mining functions. In addition to adding great power and flexibility to SQL, these extensions are conducive to performance and data independence. The paper also discusses the system we have developed to support these SQL extensions, and the architecture and techniques used in its realization.
5368193
Progressive modeling
Presently, inductive learning is still performed in a frustrating batch process. The user has little interaction with the system and no control over the final accuracy and training time. If the accuracy of the produced model is too low, all the computing resources are misspent. In this paper we propose a progressive modeling framework. In progressive modeling, the learning algorithm estimates online both the accuracy of the final model and remaining training time. If the estimated accuracy is far below expectation, the user can terminate training prior to completion without wasting further resources. If the user chooses to complete the learning process, progressive modeling will compute a model with expected accuracy in expected time. We describe one implementation of progressive modeling using ensemble of classifiers.
1498767
Clustering by pattern similarity in large data sets
Clustering is the process of grouping a set of objects into classes of similar objects. Although definitions of similarity vary from one clustering model to another, in most of these models the concept of similarity is based on distances, e.g., Euclidean distance or cosine distance. In other words, similar objects are required to have close values on at least a set of dimensions. In this paper, we explore a more general type of similarity. Under the pCluster model we proposed, two objects are similar if they exhibit a coherent pattern on a subset of dimensions. For instance, in DNA microarray analysis, the expression levels of two genes may rise and fall synchronously in response to a set of environmental stimuli. Although the magnitude of their expression levels may not be close, the patterns they exhibit can be very much alike. Discovery of such clusters of genes is essential in revealing significant connections in gene regulatory networks. E-commerce applications, such as collaborative filtering, can also benefit from the new model, which captures not only the closeness of values of certain leading indicators but also the closeness of (purchasing, browsing, etc.) patterns exhibited by the customers. Our paper introduces an effective algorithm to detect such clusters, and we perform tests on several real and synthetic data sets to show its effectiveness.
18937658
Mining associations by pattern structure in large relational tables
Association rule mining aims at discovering patterns whose support is beyond a given threshold. Mining patterns composed of items described by an arbitrary subset of attributes in a large relational table represents a new challenge and has various practical applications, including the event management systems that motivated this work. The attribute combinations that define the items in a pattern provide the structural information of the pattern. Current association algorithms do not make full use of the structural information of the patterns: the information is either lost after it is encoded with attribute values, or is constrained by a given hierarchy or taxonomy. Pattern structures convey important knowledge about the patterns. We present an architecture that organizes the mining space based on pattern structures. By exploiting the interrelationships among pattern structures, execution times for mining can be reduced significantly. This advantage is demonstrated by our experiments using both synthetic and real-life datasets.
31489027
A Graph Spectral Approach to Consistent Labelling
In this paper a new formulation of probabilistic relaxation labeling is developed using the theory of diffusion processes on graphs. Our idea is to formulate relaxation labelling as a diffusion process on the vector of object-label probabilities. According to this picture, the label probabilities are given by the state-vector of a continuous time random walk on a support graph. The state-vector is the solution of the heat equation on the support-graph. The nodes of the support graph are the Cartesian product of the object-set and label-set of the relaxation process. The compatibility functions are combined in the weight matrix of the support graph. The solution of the heat-equation is found by exponentiating the eigensystem of the Laplacian matrix for the weighted support graph with time. We demonstrate the new relaxation process on a toy labeling example which has been studied extensively in the early literature, and a feature correspondence matching problem abstracted in terms of relational graphs. The experiments show encouraging labeling and matching results.
19562173
Web-service-agents-based family wealth management system
The roiling financial markets, constantly changing tax law and increasing complexity of planning transaction increase the demand of aggregated family wealth management (FWM) services. However, current trend of developing such advisory systems is mainly focusing on financial or investment side. In addition, these existing systems lack of flexibility and are hard to be integrated with other organizational information systems, such as CRM systems. In this paper, a novel architecture of Web-service-agents-based FWM systems has been proposed. Multiple intelligent agents are wrapped as Web services and can communicate with each other via Web service protocols. On the one hand, these agents can collaborate with each other and provide comprehensive FWM advices. On the other hand, each service can work independently to achieve its own tasks. A prototype system for supporting financial advice is also presented to demonstrate the advances of the proposed Webservice- agents-based FWM system architecture.
7336686
Online aggregation
Aggregation in traditional database systems is performed in batch mode: a query is submitted, the system processes a large volume of data over a long period of time, and, eventually, the final answer is returned. This archaic approach is frustrating to users and has been abandoned in most other areas of computing. In this paper we propose a new online aggregation interface that permits users to both observe the progress of their aggregation queries and control execution on the fly. After outlining usability and performance requirements for a system supporting online aggregation, we present a suite of techniques that extend a database system to meet these requirements. These include methods for returning the output in random order, for providing control over the relative rate at which different aggregates are computed, and for computing running confidence intervals. Finally, we report on an initial implementation of online aggregation in POSTGRES.
16808221
A Collaborative Support Tool for Creativity Learning: Idea Storming Cube
Creativity plays an important role in many learning activities but how to enhance creativity with the support of modern computer tools deserves further studies. In this paper, we propose a game-based collaborative creativity support system called Idea storming cube to support creative thinking and help a user form a perspective-shift thinking habit. The system analyzes the knowledge acquired from the history of user inputs and compares it with the ideas possessed by the domain expert and other users in the current brainstorming group. The system is designed to provide user-, goal- and context-sensitive supports with this mechanism that stimulates more divergent thinking. We have implemented the proposed game with a magic-cube-like HCI for collaborative idea generation. A case study for preliminary evaluation of the system is also reported in this paper.
1367527
Space-Time Light Field Rendering
In this paper, we propose a novel framework called space-time light field rendering, which allows continuous exploration of a dynamic scene in both space and time. Compared to existing light field capture/rendering systems, it offers the capability of using unsynchronized video inputs and the added freedom of controlling the visualization in the temporal domain, such as smooth slow motion and temporal integration. In order to synthesize novel views from any viewpoint at any time instant, we develop a two-stage rendering algorithm. We first interpolate in the temporal domain to generate globally synchronized images using a robust spatial-temporal image registration algorithm followed by edge-preserving image morphing. We then interpolate these software-synchronized images in the spatial domain to synthesize the final view. In addition, we introduce a very accurate and robust algorithm to estimate subframe temporal offsets among input video sequences. Experimental results from unsynchronized videos with or without time stamps show that our approach is capable of maintaining photorealistic quality from a variety of real scenes.