611504
Utilizing histogram information
Many commercial database management systems (e.g., DB2, Oracle, etc.) make use of histograms of the value distribution of individual attributes of relations in order to make good selections of query execution plans. These histograms contain partial information about the actual distribution, such as which attribute values occur most frequently, and how often each one occurs, and what value occurs at the kth quantile when the values are sorted.In this paper, we quantitatively assess the information gain (or uncertainty reduction) due to each of these types of histogram information both individually and in combination. Correspondingly, we observe how the accuracy of estimating frequencies of individual values improves with the availability of each of these types of histogram information. We suggest guidelines for constructing histograms tailored to each individual attribute depending on the characteristics of its attribute value distribution.
38802324
Experiments with improved approximate mean value analysis algorithms
Abstract Approximate mean value analysis (MVA) is a popular technique for analyzing queueing networks because of the efficiency and accuracy that it affords. In this paper, we present a new software package, called the improved approximate mean value analysis library (IAMVAL) , which can be easily integrated into existing commercial and research queueing network analysis packages. The IAMVAL packages include two new approximate MVA algorithms, the queue line (QL) algorithm and the fraction line (FL) algorithm, for analyzing multiple class separable queueing networks. The QL algorithm is always more accurate than, and yet has approximately the same computational efficiency as, the Bard–Schweitzer proportional estimation (PE) algorithm, which is currently the most widely used approximate MVA algorithm. The FL algorithm has the same computational cost and, in noncongested separable queueing networks where queue lengths are quite small, yields more accurate solutions than both the QL and PE algorithms.
9669585
Conceptual construction on incomplete survey data
The raw survey data for data mining are often incomplete. The issues of missing data in knowledge discovery are often ignored in data mining. This article presents the conceptual foundations of data mining with incomplete survey data, and proposes query processing for knowledge discovery and a set of query functions for the conceptual construction in survey data mining. Through a case, this paper demonstrates that conceptual construction on incomplete data can be accomplished by using artificial intelligence tools such as self-organizing maps.
35913610
Knowledge Discovery Through Self-Organizing Maps: Data Visualization and Query Processing
Abstract. In data mining, the usefulness of a data pattern depends on the user of the database and does not solely depend on the statistical strength of the pattern. Based on the premise that heuristic search in combinatorial spaces built on computer and human cognitive theories is useful for effective knowledge discovery, this study investigates how the use of self-organizing maps as a tool of data visualization in data mining plays a significant role in human–computer interactive knowledge discovery. This article presents the conceptual foundations of the integration of data visualization and query processing for knowledge discovery, and proposes a set of query functions for the validation of self-organizing maps in data mining.
27642253
A multi-dimensional histogram for selectivity estimation and fast approximate query answering
Histograms have been widely used for selectivity estimation in query optimization, as well as for fast approximate query answering in many data mining, OLAP, and data visualization applications. This paper presents a new type of multi-dimensional histogram, the multi-dimensional VI histogram. Unlike other types of multi-dimensional histograms, which are seldom used in practice due to their high construction costs, the multi-dimensional VI histogram can be constructed in just one scan through the data. Through a set of experiments, we show that the multi-dimensional VI histogram is capable of providing more accurate estimations than the techniques currently used in major commercial database management systems, including IBM DB2, Oracle Database, and Microsoft SQL Server.
18741188
Experiments with Improved Approximate Mean Value Analysis Algorithms
Approximate Mean Value Analysis (MVA) is a popular technique for analyzing queueing networks because of the efficiency and accuracy that it affords. In this paper, we present a new software package, called the Improved Approximate Mean Value Analysis Library (IAMVAL), which can be easily integrated into existing commercial and research queueing network analysis packages. The IAMVAL packages includes two new approximate MVA algorithms, the Queue Line (QL) algorithm and the Fraction Line (FL) algorithm, for analyzing multiple class separable queueing networks. The QL algorithm is always more accurate than, and yet has approximately the same computational efficiency as, the Bard-Schweitzer Proportional Estimation (PE) algorithm, which is currently the most widely used approximate MVA algorithm. The FL algorithm has the same computational cost and, in noncongested separable queueing networks where queue lengths are quite small, yields more accurate solutions than both the QL and PE algorithms.
19006969
Water drops on surfaces
We present a physically-based method to enforce contact angles at the intersection of fluid free surfaces and solid objects, allowing us to simulate a variety of small-scale fluid phenomena including water drops on surfaces. The heart of this technique is a virtual surface method, which modifies the level set distance field representing the fluid surface in order to maintain an appropriate contact angle. The surface tension that is calculated on the contact line between the solid surface and liquid surface can then capture all interfacial tensions, including liquid-solid, liquid-air and solid-air tensions. We use a simple dynamic contact angle model to select contact angles according to the solid material property, water history, and the fluid front's motion. Our algorithm robustly and accurately treats various drop shape deformations, and handles both flat and curved solid surfaces. Our results show that our algorithm is capable of realistically simulating several small-scale liquid phenomena such as beading and flattened drops, stretched and separating drops, suspended drops on curved surfaces, and capillary action.
18937658
Mining associations by pattern structure in large relational tables
Association rule mining aims at discovering patterns whose support is beyond a given threshold. Mining patterns composed of items described by an arbitrary subset of attributes in a large relational table represents a new challenge and has various practical applications, including the event management systems that motivated this work. The attribute combinations that define the items in a pattern provide the structural information of the pattern. Current association algorithms do not make full use of the structural information of the patterns: the information is either lost after it is encoded with attribute values, or is constrained by a given hierarchy or taxonomy. Pattern structures convey important knowledge about the patterns. We present an architecture that organizes the mining space based on pattern structures. By exploiting the interrelationships among pattern structures, execution times for mining can be reduced significantly. This advantage is demonstrated by our experiments using both synthetic and real-life datasets.
4407504
Optimal ISP subscription for Internet multihoming: algorithm design and implication analysis
Multihoming is a popular method used by large enterprises and stub ISPs to connect to the Internet to reduce cost and improve performance. Recently researchers have studied the potential benefits of multihoming and proposed protocols and algorithms to realize these benefits. They focus on how to dynamically select which ISPs to use for forwarding and receiving packets, and assume that the set of subscribed ISPs is given a priori. In practice, a user often has the freedom to choose which subset of ISPs among all available ISPs to subscribe to. We call the problem of how to choose the optimal set of ISPs the ISP subscription problem. In this paper, We design a dynamic programming algorithm to solve the ISP subscription problem optimally. We also design a more efficient algorithm for a large class of common pricing functions. Using real traffic traces and realistic pricing data, we show that our algorithm reduces users' cost. Next we study how ISPs respond to users' optimal ISP subscription by adjusting their pricing strategies. We call this problem the ISP pricing problem. Using a realistic charging model, we formulate the problem as a non-cooperative game. We first prove that if cost is the only criterion used by a user to determine which subset of ISPs to subscribe to, at any equilibrium all ISPs receive zero revenue. We then study a more practical formulation in which different ISPs provide different levels of reliability and users choose ISPs to both improve reliability and reduce cost. We analyze this problem and show that at any equilibrium an ISP's revenue is positive and determined by its reliability.
13221842
Rank-R approximation of tensors using image-as-matrix representation
We present a novel multilinear algebra based approach for reduced dimensionality representation of image ensembles. We treat an image as a matrix, instead of a vector as in traditional dimensionality reduction techniques like PCA, and higher-dimensional data as a tensor. This helps exploit spatio-temporal redundancies with less information loss than image-as-vector methods. The challenges lie in the computational and memory requirements for large ensembles. Currently, there exists a rank-R approximation algorithm which, although applicable to any number of dimensions, is efficient for only low-rank approximations. For larger dimensionality reductions, the memory and time costs of this algorithm become prohibitive. We propose a novel algorithm, for rank-R approximations of third-order tensors, which is efficient for arbitrary R but for the important special case of 2D image ensembles, e.g. video. Both of these algorithms reduce redundancies present in all dimensions. Rank-R tensor approximation yields the most compact data representation among all known image-as-matrix methods. We evaluated the performance of our algorithm vs. other approaches on a number of datasets with the following two main results. First, for a fixed compression ratio, the proposed algorithm yields the best representation of image ensembles visually as well as in the least squares sense. Second, proposed representation gives the best performance for object classification.
14953442
Estimating the Selectivity of XML Path Expression with Predicates by Histograms
Selectivity estimation of path expressions in querying XML data plays an important role in query optimization. A path expression may contain multiple branches with predicates, each of which having its impact on the selectivity of the entire query. In this paper, we propose a novel method based on 2-dimensional value histograms to estimate the selectivity of path expressions embedded with predicates. The value histograms capture the correlation between the structures and the values in the XML data. We define a set of operations on the value histograms as well as on the traditional histograms that capture nodes positional distribution. We then construct a cost tree based on such operations. The selectivity of any node (or branch) in a path expression can be estimated by executing the cost tree. Compared with previous methods (which ignore value distribution) our method offers much better estimation accuracy.