3382815
PSQL: A Query Language for Probabilistic Relational Data
Abstract Database systems based on the relational model have become very popular for the storage of large volumes of business-related data. Although the relational model provides a great range of advantages over other data models, it was designed to support deterministic data and, consequently, it lacks a comprehensive way for handling uncertain data. However, in all real-world environments, uncertainty in data values is a common occurrence. Therefore, there is a need to extend the relational model so that data uncertainty can be captured explicitly. In an earlier work, we have proposed a probabilistic relational algebra that is a consistent extension of the conventional relational algebra and is reducible to the latter. The contribution of this work is the development of a non-procedural probabilistic query language—structured around the popular SQL—that can be built on top of that algebra. We formalize the extended syntax, provide examples to illustrate the syntax, and examine query formulation from a decision-theoretic perspective.
33218100
A Distance-Based Approach to Entity Reconciliation in Heterogeneous Databases
In modern organizations, decision makers must often be able to quickly access information from diverse sources in order to make timely decisions. A critical problem facing many such organizations is the inability to easily reconcile the information contained in heterogeneous data sources. To overcome this limitation, an organization must resolve several types of heterogeneity problems that may exist across different sources. We examine one such problem called the entity heterogeneity problem, which arises when the same real-world entity type is represented using different identifiers in different applications. A decision-theoretic model to resolve the problem is proposed. Our model uses a distance measure to express the similarity between two entity instances. We have implemented the model and tested it on real-world data. The results indicate that the model performs quite well in terms of its ability to predict whether two entity instances should be matched or not. The model is shown to be computationally efficient. It also scales well to large relations from the perspective of the accuracy of prediction. Overall, the test results imply that this is certainly a viable approach in practical situations.
14335537
A Data Perturbation Approach to Privacy Protection in Data Mining
Advances in data mining techniques have raised growing concerns about privacy of personal information. Organizations that use their customers’ records in data mining activities are forced to take actions to protect the privacy of the individuals involved. A common practice for many organizations today is to remove the identity-reated attributes from customer records before releasing them to data miners or analysts. In this study, we investigate the effect of this practice and demonstrate that a majority of the records in a dataset can be uniquely identified even after identity related attributes are removed. We propose a data perturbation method that can be used by organizations to prevent such unique identification of individual records, while providing the data to analysts for data mining. The proposed method attempts to preserve the statistical properties of the data based on privacy protection parameters specified by the organization. We show that the problem can be solved in two phases, with a linear programming formulation in phase one (to preserve the marginal distribution), followed by a simple Bayes-based swapping procedure in phase two (to preserve the joint distribution). The proposed method is compared with a random perturbation method in classification performance on two real-world datasets. The results of the experiments indicate that it significantly outperforms the random method.
29720896
Assessing data quality for information products
Data quality is a critical issue in the design and implementation of databases. Often, the information provided to decision-makers is derived from heterogeneous databases. To determine the quality characteristics of information provided to decision-makers (derived data), it is necessary to determine how quality characteristics associated with diverse data sources affect the quality of the derived data. We discuss a methodology to determine two data quality characteristics, accuracy and incompleteness, that are of critical importance to decision-makers. We examine how the quality metrics of source data affect metrics for data derived using relational algebra operations.
476243
Reconciling Attribute Values from Multiple Data Sources
Because of the heterogeneous nature of multiple data sources, data integration is often one of the most challenging tasks of today’s information systems. While the existing literature has focused on problems such as schema integration and entity identification, our current study attempts to answer a basic question: When an attribute value for a real-world entity is recorded differently in two databases, how should the “best” value be chosen from the set of possible values? We first show how probabilities for attribute values can be derived, and then propose a framework for deciding the cost-minimizing value based on the total cost of type I, type II, and misrepresentation errors.
500117
A probabilistic relational model and algebra
Although the relational model for databases provides a great range of advantages over other data models, it lacks a comprehensive way to handle incomplete and uncertain data. Uncertainty in data values, however, is pervasive in all real-world environments and has received much attention in the literature. Several methods have been proposed for incorporating uncertain data into relational databases. However, the current approaches have many shortcomings and have not established an acceptable extension of the relational model. In this paper, we propose a consistent extension of the relational model. We present a revised relational structure and extend the relational algebra. The extended algebra is shown to be closed, a consistent extension of the conventional relational algebra, and reducible to the latter.
27571789
Computation of intraprocedural dynamic program slices
Abstract Dynamic slicing algorithms are used in interactive applications such as program debugging and testing. Therefore, these algorithms need to be very efficient. In this context, we propose three intraprocedural dynamic slicing algorithms which are more space and time efficient than the existing algorithms. Two of the proposed algorithms compute precise dynamic slices of structured programs using Program Dependence Graph as an intermediate representation. To compute precise dynamic slices of unstructured programs, we introduce the concepts of jump dependence and Unstructured Program Dependence Graph. The third algorithm uses Unstructured Program Dependence Graph as the intermediate program representation, and computes precise dynamic slices of unstructured programs. We show that each of our proposed algorithms is more space and time efficient than the existing algorithms.
2016087
Enumeration of 9-Variable Rotation Symmetric Boolean Functions Having Nonlinearity > 240
The existence of 9-variable Boolean functions having nonlinearity strictly greater than 240 has been shown very recently (May 2006) by Kavut, Maitra and Yucel; a few functions with nonlinearity 241 have been identified by a heuristic search in the class of Rotation Symmetric Boolean Functions (RSBFs). In this paper, using combinatorial results related to the Walsh spectra of RSBFs, we efficiently perform the exhaustive search to enumerate the 9-variable RSBFs having nonlinearity > 240 and found that there are 8 ×189 many functions with nonlinearity 241 and there is no RSBF having nonlinearity > 241. We further prove that among these functions, there are only two which are different up to the affine equivalence. This is found by utilizing the binary nonsingular circulant matrices and their variants. Finally we explain the coding theoretic significance of these functions. This is the first time orphan cosets of R(1, n) having minimum weight 241 are demonstrated for n = 9. Further they provide odd weight orphans for n = 9; earlier these were known for certain n ≥11.
16820690
Introducing Local Optimization for Effective Initialization and Crossover of Genetic Decision Trees
We introduce a new genetic operator, Reduction, that recties decision trees not correct syntactically and at the same time removes the redundant sections within, while preserving its accuracy during operation. A novel approach to crossover is presented that uses the reduction operator to systematically extract building blocks spread out over the entire second parent to create a subtree that is valid and particularly useful in the context it replaces the subtree in the rst parent. The crossover introduced also removes unexplored code from the ospring and hence prevents redundancy and bloating. Overall, reduction can be viewed as a local optimization step that directs the population, generated initially or over generations through crossovers, to potentially good regions in the search space so that reproduction is performed in a highly correlated landscape with a global structure. Lexical convergence is also ensured implying identical individuals always produce the same ospring.
4723885
Learning while solving problems in best first search
We investigate the role of learning in search-based systems for solving optimization problems. We use a learning model, where the values of a set of features can be used to induce a clustering of the problem state space. The feasible set of h* values corresponding to each cluster is called h*set. If we relax the optimality guarantee, and tolerate a risk factor, the distribution of h*set can be used to expedite search and produce results within a given risk of suboptimality. The off-line learning method consists of solving a batch of problems by using A* to learn the distribution of the h*set in the learning phase. This distribution can be used to solve the rest of the problems effectively. We show how the knowledge acquisition phase can be integrated with the problem solving phase. We present a continuous online learning scheme that uses an "anytime" algorithm to learn continuously while solving problems.
9695284
A Hybrid Model for Part-of-Speech Tagging and its Application to Bengali
This paper describes our work on Bengali Part of Speech (POS) tagging using a corpus-based approach. There are several approaches for part of speech tagging. This paper deals with a model that uses a combination of supervised and unsupervised learning using a Hidden Markov Model (HMM). We make use of small tagged corpus and a large untagged corpus. We also make use of Morphological Analyzer. Bengali is a highly ambiguous and relatively free word order language. We have obtained an overall accuracy of 95%.