14001621
Discriminative Language Modeling with Conditional Random Fields and the Perceptron Algorithm
This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the perceptron algorithm, and a method based on conditional random fields (CRFs). The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The perceptron algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. However, using the feature set output from the perceptron algorithm (initialized with their weights), CRF training provides an additional 0.5% reduction in word error rate, for a total 1.8% absolute reduction from the baseline of 39.2%.
2965282
Exploiting auxiliary distributions in stochastic unification-based grammars
This paper describes a method for estimating conditional probability distributions over the parses of "unification-based" grammars which can utilize auxiliary distributions that are estimated by other means. We show how this can be used to incorporate information about lexical selectional preferences gathered from other sources into Stochastic "Unification-based" Grammars (SUBGs). While we apply this estimator to a Stochastic Lexical-Functional Grammar, the method is general, and should be applicable to stochastic versions of HPSGs, categorial grammars and transformational grammars.
5454540
Edit Detection and Parsing for Transcribed Speech
We present a simple architecture for parsing transcribed speech in which an edited-word detector first removes such words from the sentence string, and then a standard statistical parser trained on transcribed speech parses the remaining words. The edit detector achieves a misclassification rate on edited words of 2.2%. (The NULL-model, which marks everything as not edited, has an error rate of 5.9%.) To evaluate our parsing results we introduce a new evaluation metric, the purpose of which is to make evaluation of a parse tree relatively indifferent to the exact tree position of EDITED nodes. By this metric the parser achieves 85.3% precision and 86.5% recall.
739426
Joint and Conditional Estimation of Tagging and Parsing Models
This paper compares two different ways of estimating statistical language models. Many statistical NLP tagging and parsing models are estimated by maximizing the (joint) likelihood of the fully-observed training data. However, since these applications only require the conditional probability distributions, these distributions can in principle be learnt by maximizing the conditional likelihood of the training data. Perhaps somewhat surprisingly, models estimated by maximizing the joint were superior to models estimated by maximizing the conditional, even though some of the latter models intuitively had access to "more information".
5925726
A Resource Sensitive Interpretation of Lexical Functional Grammar
This paper investigates whether the fundamental linguistic insights and intuitions of Lexical Functional Grammar (LFG), which is usually presented as a “constraint-based” linguistic theory, can be reformulated in a “resource sensitive” framework using a substructural modal logic. In the approach investigated here, LFG's f-descriptions are replaced with expressions from a multi-modal propositional logic (with permutation and possibly limited contraction). In effect, the feature structure “unification” basis of LFG's f-structures is replaced with a very different resource based mechanism. It turns out that some linguistic analyses that required non-monotonic devices in LFG (such as the “constraint equations” in the Andrews (1982) analysis of Icelandic) can be straightforwardly expressed in the framework presented here. Moreover, a Curry–Howard correspondence between proofs in this logic and λ-terms provides a semantic interpretation as a by-product of the process of showing syntactic well-formedness.
9027893
A Look at Parsing and Its Applications
This paper provides a brief introduction to recent work in statistical parsing and its applications. We highlight successes to date, remaining challenges, and promising future work.
17383450
Effectiveness of a machine-level, global optimizer
We present an overview of the design of a machine-code-level, global (intraprocedural) optimizer that supports several front-ends producing code for the Hewlett-Packard Precision Architecture family of machines. The basic optimization strategy is described, including information about the division of responsibilities between various components of the compiler. Optimization algorithms are described, including a discussion of the dataflow information they require. Measurements showing the collective and individual effects of various optimizer components are presented.
The performance data presented here was collected using a preliminary version of the optimizer. Development is continuing and further improvements are expected.
930298
On Compressing Encrypted Data without the Encryption Key
When it is desired to transmit redundant data over an insecure and bandwidth-constrained channel, it is customary to first compress the redundant data and then encrypt it for security reasons. In this paper, we investigate the novelty of reversing the order of these steps, i.e. first encrypting and then compressing. Although counter-intuitive, we show surprisingly that through the use of coding with side information principles, this reversal in order is indeed possible. In fact, for lossless compression, we show that the theoretical compression gain is unchanged by performing encryption before compression. We show that the cryptographic security of the reversed system is directly related to the strength of the key generator.
10445058
Time series classification using Gaussian mixture models of reconstructed phase spaces
A new signal classification approach is presented that is based upon modeling the dynamics of a system as they are captured in a reconstructed phase space. The modeling is done using full covariance Gaussian mixture models of time domain signatures, in contrast with current and previous work in signal classification that is typically focused on either linear systems analysis using frequency content or simple nonlinear machine learning models such as artificial neural networks. The proposed approach has strong theoretical foundations based on dynamical systems and topological theorems, resulting in a signal reconstruction, which is asymptotically guaranteed to be a complete representation of the underlying system, given properly chosen parameters. The algorithm automatically calculates these parameters to form appropriate reconstructed phase spaces, requiring only the number of mixtures, the signals, and their class labels as input. Three separate data sets are used for validation, including motor current simulations, electrocardiogram recordings, and speech waveforms. The results show that the proposed method is robust across these diverse domains, significantly outperforming the time delay neural network used as a baseline.
15400971
When Are Two Protocols the Same?
A number of protocols based on the formal dialogue games of philosophy have recently been proposed for interactions between autonomous agents. Several of these proposals purport to assist agents engaged in the same types of interactions, such as persuasions and negotiations, and are superficially different. How are we to determine whether or not these proposals are substantially different? This chapter considers this question and explores several alternative definitions of equivalence of protocols.
8126535
An adaptive approach for texture modelling
Markov random fields (MRFs) continue to be an important and useful representation for modelling textured images. Standard methods for MRF image modeling make use of the equivalent Gibbs distribution (GD) to express the joint probabilities of groups of neighboring pixels. The authors investigate a new approach to the use of the GD in image modeling. Specifically, they develop an adaptive approach to the formation of clique potential functions for the distribution. Traditional tools, such as the multi-level logistic (MLL) model, have been based on the use of a predetermined and identical set of potential functions. In the present paper it is shown that by incorporating additional parameters into the model in order to control the shape of these functions, it is possible to arrive at a more complete parametrization of the image. A simple model based on this concept is described and implemented, and image simulations using the well-known Gibbs sampler algorithm are constructed to demonstrate the usefulness of an adaptive set of potential functions.<<ETX>>