6922975
Similarity-Based Estimation of Word Cooccurrence Probabilities
In many applications of natural language processing it is necessary to determine the likelihood of a given word combination. For example, a speech recognizer may need to determine which of the two word combinations "eat a peach" and "eat a beach" is more likely. Statistical NLP methods determine the likelihood of a word combination according to its frequency in a training corpus. However, the nature of language is such that many word combinations are infrequent and do not occur in a given corpus. In this work we propose a method for estimating the probability of such previously unseen word combinations using available information on "most similar" words.We describe a probabilistic word association model based on distributional word similarity, and apply it to improving probability estimates for unseen word bigrams in a variant of Katz's back-off model. The similarity-based method yields a 20% perplexity improvement in the prediction of unseen bigrams and statistically significant reductions in speech-recognition error.
2480472
Similarity-Based Methods for Word Sense Disambiguation
We compare four similarity-based estimation methods against back-off and maximum-likelihood estimation methods on a pseudo-word sense disambiguation task in which we controlled for both unigram and bigram frequency. The similarity-based methods perform up to 40% better on this particular task. We also conclude that events that occur only once in the training set have major impact on similarity-based estimates.
7521453
Bootstrapping Lexical Choice via Multiple-Sequence Alignment
An important component of any generation system is the mapping dictionary, a lexicon of elementary semantic expressions and corresponding natural language realizations. Typically, labor-intensive knowledge-based methods are used to construct the dictionary. We instead propose to acquire it automatically via a novel multiple-pass algorithm employing multiple-sequence alignment, a technique commonly used in bioinformatics. Crucially, our method lever-ages latent information contained in multi-parallel corpora --- datasets that supply several verbalizations of the corresponding semantics rather than just one.We used our techniques to generate natural language versions of computer-generated mathematical proofs, with good results on both a per-component and overall-output basis. For example, in evaluations involving a dozen human judges, our system produced output whose readability and faithfulness to the semantic input rivaled that of a traditional generation system.
6305097
Measures of Distributional Similarity
We study distributional similarity measures for the purpose of improving probability estimation for unseen cooccurrences. Our contributions are three-fold: an empirical comparison of a broad range of measures; a classification of similarity functions based on the information that they incorporate; and the introduction of a novel function that is superior at evaluating potential proxy distributions.
6713452
Distributional Clustering of English Words
We describe and evaluate experimentally a method for clustering words according to their distribution in particular syntactic contexts. Words are represented by the relative frequency distributions of contexts in which they appear, and relative entropy between those distributions is used as the similarity measure for clustering. Clusters are represented by average context distributions derived from the given words according to their probabilities of cluster membership. In many cases, the clusters can be thought of as encoding coarse sense distinctions. Deterministic annealing is used to find lowest distortion sets of clusters: as the annealing parameter increases, existing clusters become unstable and subdivide, yielding a hierarchical "soft" clustering of the data. Clusters are used as the basis for class models of word coocurrence, and the models evaluated with respect to held-out test data.
2032603
Similarity-Based Approaches to Natural Language Processing
Statistical methods for automatically extracting information about associations between words or documents from large collections of text have the potential to have considerable impact in a number of areas, such as information retrieval and natural-language-based user interfaces. However, even huge bodies of text yield highly unreliable estimates of the probability of relatively common events, and, in fact, perfectly reasonable events may not occur in the training data at all. This is known as the sparse data problem. Traditional approaches to the sparse data problem use crude approximations. We propose a different solution: if we are able to organize the data into classes of similar events, then, if information about an event is lacking, we can estimate its behavior from information about similar events. This thesis presents two such similarity-based approaches, where, in general, we measure similarity by the Kullback-Leibler divergence, an information-theoretic quantity. 
Our first approach is to build soft, hierarchical clusters: soft, because each event belongs to each cluster with some probability; hierarchical, because cluster centroids are iteratively split to model finer distinctions. Our clustering method, which uses the technique of deterministic annealing, represents (to our knowledge) the first application of soft clustering to problems in natural language processing. We use this method to cluster words drawn from 44 million words of Associated Press Newswire and 10 million words from Grolier's encyclopedia, and find that language models built from the clusters have substantial predictive power. Our algorithm also extends with no modification to other domains, such as document clustering. 
Our second approach is a nearest-neighbor approach: instead of calculating a centroid for each class, we in essence build a cluster around each word. We compare several such nearest-neighbor approaches on a word sense disambiguation task and find that as a whole, their performance is far superior to that of standard methods. In another set of experiments, we show that using estimation techniques based on the nearest-neighbor model enables us to achieve perplexity reductions of more than 20 percent over standard techniques in the prediction of low-frequency events, and statistically significant speech recognition error-rate reduction.
10735662
MAS platforms as an enabler of enterprise mobilisation: the state of the art
One of the main application areas for multi-agent systems technology is enterprise mobilization, wherein the main business process actors are nomadic workers. An agent's autonomy, sociality and intelligence are highly prized features when it comes to supporting those mobile workers who are geographically isolated from the main knowledge source (i.e. the corporate Intranet) and are frequently moving from one location to another. Based on experience gained from two field trials of applications (built using for multi-agent systems technology and running on lightweight handheld devices) that support mobile business processes for telecommunications service provisioning and maintenance, this paper proposes desirable metrics for any multi-agent systems platform intended for enterprise mobilisation use. These metrics are then used to compare a number of existing multi-agent systems platforms, and based on the results, this paper identifies some areas for improvement.
6016075
Concept-based component libraries and optimizing compilers
This paper discusses concept lattices and some of their applications in component library development and compiler optimizations. Ongoing work on concept-based user-extensible simplification (type-based optimizer generators) and library transformations is first reviewed, then a new application--categorization of algorithms via precise descriptions of their performance characteristics--is outlined, with Standard Template Library and Boost Graph Library algorithm concept lattices as example.
1094026
Generic Graph Algorithms for Sparse Matrix Ordering
Fill-reducing sparse matrix orderings have been a topic of active research for many years. Although most such algorithms are developed and analyzed within a graph-theoretical framework, for reasons of performance the corresponding implementations are typically realized with programming languages devoid of language features necessary to explicitly represent graph abstractions. Recently, generic programming has emerged as a programming paradigm capable of providing high levels of performance in the presence of programming abstractions. In this paper we present an implementation of the Minimum Degree ordering algorithm using the newly-developed Generic Graph Component Library. Experimental comparisons show that, despite our heavy use of abstractions, our implementation has performance indistinguishable from that of a widely used Fortran implementation.
40898248
Using logic models to predict the detection behavior of statistical timing defects
In this paper, we study the possibility of using logic defect-level prediction models to predict the detection behavior of statistical timing defects. We compare two known logic models: the Williams-Brown (WB) model and the Mercer-Park-GrimailaDworak (MPGD) model. In the WB-model, the defect coverage is replaced by the n-detection transition fault coverage. We first demonstrate that both logic models may fail to predict the detection of statistical timing defects. Then, we propose an improved WB model based upon selection of the hard-to-detect transition faults. We show that, by selecting a proper subset of the hard-to-detect transition faults, the detection behavior of these faults can correlate well to the detection behavior of statistical timing defects. We explain our findings through statistical delay defect injection and simulation, and report results based upon various benchmark circuits.
29502567
On silicon-based speed path identification
Speed path identification is an indispensable step for pushing the design timing wall and for developing the final speed binning strategy in production test. For complex high-performance designs, pre-silicon timing tools have so far not been able to deliver satisfactory results in predicting the actual speed limiting paths on the silicon. The actual speed paths are mostly uncovered through test and silicon debug, where tremendous manual effort is involved. This paper presents a novel approach as the first step for automating the speed path identification process. Our approach is silicon-based, meaning that timing information is extracted through testing of silicon sample chips. We call this step silicon learning. Based on silicon learning, we present an iterative flow for speed path identification. Experimental results are presented to explain the new methodologies and to demonstrate the effectiveness of our techniques.