5083193
A Comparative Study on Feature Selection in Text Categorization
This paper is a comparative study of feature selection methods in statistical learning of text categorization The focus is on aggres sive dimensionality reduction Five meth ods were evaluated including term selection based on document frequency DF informa tion gain IG mutual information MI a test CHI and term strength TS We found IG and CHI most e ective in our ex periments Using IG thresholding with a k nearest neighbor classi er on the Reuters cor pus removal of up to removal of unique terms actually yielded an improved classi cation accuracy measured by average preci sion DF thresholding performed similarly Indeed we found strong correlations between the DF IG and CHI values of a term This suggests that DF thresholding the simplest method with the lowest cost in computation can be reliably used instead of IG or CHI when the computation of these measures are too expensive TS compares favorably with the other methods with up to vocabulary reduction but is not competitive at higher vo cabulary reduction levels In contrast MI had relatively poor performance due to its bias towards favoring rare terms and its sen sitivity to probability estimation errors
8555900
Unsupervised Learning of Arabic Stemming Using a Parallel Corpus
This paper presents an unsupervised learning approach to building a non-English (Arabic) stemmer. The stemming model is based on statistical machine translation and it uses an English stemmer and a small (10 K sentences) parallel corpus as its sole training resources. No parallel text is needed after the training phase. Monolingual, unannotated text can be used to further improve the stemmer by allowing it to adapt to a desired domain or genre. Examples and results will be given for Arabic, but the approach is applicable to any language that needs affix removal. Our resource-frugal approach results in 87.5% agreement with a state of the art, proprietary Arabic stemmer built using rules, affix lists, and human annotated text, in addition to an unsupervised component. Task-based evaluation using Arabic information retrieval indicates an improvement of 22-38% in average precision over unstemmed text, and 96% of the performance of the proprietary stemmer above.
1743726
Hypertext Categorization using Hyperlink Patterns and Meta Data
Hypertext poses new text classi cation research challenges as hyperlinks, content of linked documents, and meta data about related web sites all provide richer sources of information for hypertext classi cation that are not available in traditional text classi cation. We investigate the use of such information for representing web sites, and the e ectiveness of di erent classi ers (Naive Bayes, Nearest Neighbor, and Foil) in exploiting those representations. We nd that using words in web pages alone often yields suboptimal performance of classi ers, compared to exploiting additional sources of information beyond document content. On the other hand, we also observe that linked pages can be more harmful than helpful when the linked neighborhoods are highly \noisy" and that links have to be used in a careful manner. More importantly, our investigation suggests that meta data which is often available, or can be acquired using Information Extraction techniques, can be extremely useful for improving classi cation accuracy. Finally, the relative performance of the di erent classi ers being tested gives us insights into the strengths and limitations of our algorithms for hypertext classi cation.
7098682
High-performing feature selection for text classification
This paper reports a controlled study on a large number of filter feature selection methods for text classification. Over 100 variants of five major feature selection criteria were examined using four well-known classification algorithms: a Naive Bayesian (NB) approach, a Rocchio-style classifier, a k-nearest neighbor (kNN) method and a Support Vector Machine (SVM) system. Two benchmark collections were chosen as the testbeds: Reuters-21578 and small portion of Reuters Corpus Version 1 (RCV1), making the new results comparable to published results. We found that feature selection methods based on chi2 statistics consistently outperformed those based on other criteria (including information gain) for all four classifiers and both data collections, and that a further increase in performance was obtained by combining uncorrelated and high-performing feature selection methods.The results we obtained using only 3% of the available features are among the best reported, including results obtained with the full feature set.
10555290
From Lasso regression to Feature vector machine
Lasso regression tends to assign zero weights to most irrelevant or redundant features, and hence is a promising technique for feature selection. Its limitation, however, is that it only offers solutions to linear models. Kernel machines with feature scaling techniques have been studied for feature selection with non-linear models. However, such approaches require to solve hard non-convex optimization problems. This paper proposes a new approach named the Feature Vector Machine (FVM). It reformulates the standard Lasso regression into a form isomorphic to SVM, and this form can be easily extended for feature selection with non-linear models by introducing kernels defined on feature vectors. FVM generates sparse solutions in the nonlinear feature space and it is much more tractable compared to feature scaling kernel machines. Our experiments with FVM on simulated data show encouraging results in identifying the small number of dominating features that are non-linearly correlated to the response, a task the standard Lasso fails to complete.
14181364
Probabilistic score estimation with piecewise logistic regression
Well-calibrated probabilities are necessary in many applications like probabilistic frameworks or cost-sensitive tasks. Based on previous success of asymmetric Laplace method in calibrating text classifiers' scores, we propose to use piecewise logistic regression, which is a simple extension of standard logistic regression, as an alternative method in the discriminative family. We show that both methods have the flexibility to be piecewise linear functions in log-odds, but they are based on quite different assumptions. We evaluated asymmetric Laplace method, piecewise logistic regression and standard logistic regression over standard text categorization collections (Reuters-21578 and TRECAP) with three classifiers (SVM, Naive Bayes and Logistic Regression Classifier), and observed that piecewise logistic regression performs significantly better than the other two methods in the log-loss metric.
14477293
To Select or To Weigh: A Comparative Study of Model Selection and Model Weighing for SPODE Ensembles
An ensemble of Super-Parent-One-Dependence Estimators (SPODEs) offers a powerful yet simple alternative to naive Bayes classifiers, achieving significantly higher classification accuracy at a moderate cost in classification efficiency. Currently there exist two families of methodologies that ensemble candidate SPODEs for classification. One is to select only helpful SPODEs and uniformly average their probability estimates, a type of model selection. Another is to assign a weight to each SPODE and linearly combine their probability estimates, a methodology named model weighing. This paper presents a theoretical and empirical study comparing model selection and model weighing for ensembling SPODEs. The focus is on maximizing the ensemble's classification accuracy while minimizing its computational time. A number of representative selection and weighing schemes are studied, providing a comprehensive research on this topic and identifying effective schemes that provide alternative trades-off between speed and expected error.
4914546
Multiple temporal consistency states for dynamical verification of upper bound constraints in grid workflow systems
Conventional upper bound constraint verification in grid workflow systems is based on the key assumption that an upper bound constraint only has two states: consistency and inconsistency. However, due to complexity of grid workflows and dynamic availability of participating grid services, this assumption is too restrictive as there may be some intermediate states. Therefore, in this paper, we introduce four states for an upper bound constraint. Namely, we treat conventional consistency as strong consistency and divide conventional inconsistency into weak consistency, weak inconsistency and strong inconsistency. Correspondingly, we develop their verification methods. For weak consistency, we present some algorithms on how to adjust it to strong consistency without triggering exception handling as in conventional work. For weak inconsistency, we analyse why it can rely on less costly exception handling than conventional work. The final evaluation demonstrates that our four-state approach can achieve better cost-effectiveness than the conventional two-state approach
17411004
Parameter tuning for induction-algorithm-oriented feature elimination
Feature selection has long been an active research topic in machine learning. Beginning with an empty set of features, it selects features most necessary for learning a target concept. Feature elimination, a newer technique, starts out with a full set of features and eliminates those most unnecessary for learning the target concept. Feature elimination tends to be more effective, can capture interacting features more easily, and suffers less from feature interaction than feature selection. Because the most unnecessary features are eliminated from the beginning, they will not mislead the induction process in terms of efficiency or accuracy. Induction-algorithm-oriented feature elimination, with particular parameter configurations, can achieve higher predictive accuracy than existing popular feature selection approaches. We propose two sets of well-tuned parameters based on empirical analysis. To understand how to achieve the best performance possible from IAOFE, we conducted a comprehensive analysis of IAOFE parameter tuning.
61052099
Memory-Model-Sensitive Data Race Analysis
We present a “memory-model-sensitive” approach to validating correctness properties for multithreaded programs. Our key insight is that by specifying both the inter-thread memory consistency model and the intra-thread program semantics as constraints, a program verification task can be reduced to an equivalent constraint solving problem, thus allowing an exhaustive examination of all thread interleavings precisely allowed by a given memory model. To demonstrate, this paper formalizes race conditions according to the new Java memory model, for a simplified but non-trivial source language. We then describe the implementation of a memory-model-sensitive race detector using constraint logic programming (CLP). In comparison with conventional program analysis, our approach does not offer the same kind of performance and scalability due to the complexity involved in exact formal reasoning. However, we show that a formal semantics can serve more than documentation purposes — it can be applied as a sound basis for rigorous property checking, upon which more scalable methods can be derived.
2702750
A new look at multimedia synchronization in distributed environments
A multimedia system requires to handle automatically and uniformly the generation, manipulation, presentation, storage, and communication of independent discrete and continuous media data. The synchronization between various kinds of media data is a key issue in an integrated multimedia system, and has drawn a considerable attention. In this paper we investigate the synchronization problem in distributed multimedia environments and propose a new approach. Based on a globally synchronized clock, we present an isochronous clock-driven protocol for multimedia synchronization in a distributed environment. We show that the isochronous approach is simple, easy to understand, and more importantly, can guarantee the temporal ordering (including causal order and /spl Delta/-causal order). We believe that with the new developments in the improved NTP over Internet and very high accuracy of synchronized clocks over ATM and LANs, the clock-based distributed synchronization would provide a promising, yet simple and intuitive, alternative to the existing approaches.