9695284
A Hybrid Model for Part-of-Speech Tagging and its Application to Bengali
This paper describes our work on Bengali Part of Speech (POS) tagging using a corpus-based approach. There are several approaches for part of speech tagging. This paper deals with a model that uses a combination of supervised and unsupervised learning using a Hidden Markov Model (HMM). We make use of small tagged corpus and a large untagged corpus. We also make use of Morphological Analyzer. Bengali is a highly ambiguous and relatively free word order language. We have obtained an overall accuracy of 95%.
16820690
Introducing Local Optimization for Effective Initialization and Crossover of Genetic Decision Trees
We introduce a new genetic operator, Reduction, that recties decision trees not correct syntactically and at the same time removes the redundant sections within, while preserving its accuracy during operation. A novel approach to crossover is presented that uses the reduction operator to systematically extract building blocks spread out over the entire second parent to create a subtree that is valid and particularly useful in the context it replaces the subtree in the rst parent. The crossover introduced also removes unexplored code from the ospring and hence prevents redundancy and bloating. Overall, reduction can be viewed as a local optimization step that directs the population, generated initially or over generations through crossovers, to potentially good regions in the search space so that reproduction is performed in a highly correlated landscape with a global structure. Lexical convergence is also ensured implying identical individuals always produce the same ospring.
27571789
Computation of intraprocedural dynamic program slices
Abstract Dynamic slicing algorithms are used in interactive applications such as program debugging and testing. Therefore, these algorithms need to be very efficient. In this context, we propose three intraprocedural dynamic slicing algorithms which are more space and time efficient than the existing algorithms. Two of the proposed algorithms compute precise dynamic slices of structured programs using Program Dependence Graph as an intermediate representation. To compute precise dynamic slices of unstructured programs, we introduce the concepts of jump dependence and Unstructured Program Dependence Graph. The third algorithm uses Unstructured Program Dependence Graph as the intermediate program representation, and computes precise dynamic slices of unstructured programs. We show that each of our proposed algorithms is more space and time efficient than the existing algorithms.
14335537
A Data Perturbation Approach to Privacy Protection in Data Mining
Advances in data mining techniques have raised growing concerns about privacy of personal information. Organizations that use their customers’ records in data mining activities are forced to take actions to protect the privacy of the individuals involved. A common practice for many organizations today is to remove the identity-reated attributes from customer records before releasing them to data miners or analysts. In this study, we investigate the effect of this practice and demonstrate that a majority of the records in a dataset can be uniquely identified even after identity related attributes are removed. We propose a data perturbation method that can be used by organizations to prevent such unique identification of individual records, while providing the data to analysts for data mining. The proposed method attempts to preserve the statistical properties of the data based on privacy protection parameters specified by the organization. We show that the problem can be solved in two phases, with a linear programming formulation in phase one (to preserve the marginal distribution), followed by a simple Bayes-based swapping procedure in phase two (to preserve the joint distribution). The proposed method is compared with a random perturbation method in classification performance on two real-world datasets. The results of the experiments indicate that it significantly outperforms the random method.
33218100
A Distance-Based Approach to Entity Reconciliation in Heterogeneous Databases
In modern organizations, decision makers must often be able to quickly access information from diverse sources in order to make timely decisions. A critical problem facing many such organizations is the inability to easily reconcile the information contained in heterogeneous data sources. To overcome this limitation, an organization must resolve several types of heterogeneity problems that may exist across different sources. We examine one such problem called the entity heterogeneity problem, which arises when the same real-world entity type is represented using different identifiers in different applications. A decision-theoretic model to resolve the problem is proposed. Our model uses a distance measure to express the similarity between two entity instances. We have implemented the model and tested it on real-world data. The results indicate that the model performs quite well in terms of its ability to predict whether two entity instances should be matched or not. The model is shown to be computationally efficient. It also scales well to large relations from the perspective of the accuracy of prediction. Overall, the test results imply that this is certainly a viable approach in practical situations.
15753347
Basic Theory in Construction of Boolean Functions with Maximum Possible Annihilator Immunity
So far there is no systematic attempt to construct Boolean functions with maximum annihilator immunity. In this paper we present a construction keeping in mind the basic theory of annihilator immunity. This construction provides functions with the maximum possible annihilator immunity and the weight, nonlinearity and algebraic degree of the functions can be properly calculated under certain cases. The basic construction is that of symmetric Boolean functions and applying linear transformation on the input variables of these functions, one can get a large class of non-symmetric functions too. Moreover, we also study several other modifications on the basic symmetric functions to identify interesting non-symmetric functions with maximum annihilator immunity. In the process we also present an algorithm to compute the Walsh spectra of a symmetric Boolean function with O(n2) time and O(n) space complexity.
40802326
Constructing Efficient Belief Network Structures With Expert Provided Information
Presents a technique to construct efficient belief network structures for application areas where large amounts of data are available and information on the ordering of the variables can be obtained from domain experts. We identify classes of networks that are efficient for propagating beliefs. We formulate the problem as one of determining the belief network representation from a given class that best represents the data. We use the I-Divergence measure which is known to have certain desirable properties for evaluating different approximations. We present some theoretical findings that characterize the nature of solutions that are obtained. These theoretical results lead to an efficient solution procedure for finding the best network representation. We also discuss other information that may be reasonably obtained from experts, and show how such information leads to improving the efficiency of the technique to find the best network structure.
29720896
Assessing data quality for information products
Data quality is a critical issue in the design and implementation of databases. Often, the information provided to decision-makers is derived from heterogeneous databases. To determine the quality characteristics of information provided to decision-makers (derived data), it is necessary to determine how quality characteristics associated with diverse data sources affect the quality of the derived data. We discuss a methodology to determine two data quality characteristics, accuracy and incompleteness, that are of critical importance to decision-makers. We examine how the quality metrics of source data affect metrics for data derived using relational algebra operations.