257038
Apportioning Development Effort in a Probabilistic LR Parsing System Through Evaluation
We describe an implemented system for robust domain-independent syntactic parsing of English, using a unification-based grammar of part-ofspeech and punctuation labels coupled with a probabilistic LR parser. We present evaluations of the system’s performance along several different dimensions; these enable us to assess the contribution that each individual part is making to the success of the system as a whole, and thus prioritise the effort to be devoted to its further enhancement. Currently, the system is able to parse around 80% of sentences in a substantial corpus of general text containing a number of distinct genres. On a random sample of 250 such sentences the system has a mean crossing bracket rate of 0.71 and recall and precision of 83% and 84% respectively when evaluated against manually-disambiguated analyses.
1954
Corpus Annotation for Parser Evaluation
We describe a recently developed corpus annotation scheme for evaluating parsers that avoids shortcomings of current methods. The scheme encodes grammatical relations between heads and dependents, and has been used to mark up a new publicdomain corpus of naturally occurring English text. We show how the corpus can be used to evaluate the accuracy of a robust parser, and relate the corpus to extant resources.
9629533
The Automatic Acquisition of Verb Subcategorisations and Their Impact on the Performance of an HPSG Parser
We describe the automatic acquisition of a lexicon of verb subcategorisations from a domain-specific corpus, and an evaluation of the impact this lexicon has on the performance of a “deep”, HPSG parser of English. We conducted two experiments to determine whether the empirically extracted verb stems would enhance the lexical coverage of the grammar and to see whether the automatically extracted verb subcategorisations would result in enhanced parser coverage. In our experiments, the empirically extracted verbs enhance lexical coverage by 8.5%. The automatically extracted verb subcategorisations enhance the parse success rate by 15% in theoretical terms and by 4.5% in practice. This is a promising approach for improving the robustness of deep parsing.
12738746
Software support for practical grammar development
Even though progress in theoretical linguistics does not necessarily rely on the construction of working programs, a large proportion of current research in syntactic theory is facilitated by suitable computational tools. However, when natural language processing applications seek to draw on the results from new developments in theories of grammar, not only the nature of the tools has to change, but they face the challenge of reconciling the seemingly contradictory requirements of notational perspicuity and efficiency of performance. In this paper, we present a comparison and an evaluation of a number of software systems for grammar development, and argue that they are inadequate as practical tools for building wide-coverage grammars. We discuss a number of factors characteristic of this task, demonstrate how they influence the design of a suitable software environment, and describe the implementation of a system which has supported efficient development of a large computational grammar of English.
1627677
High Precision Extraction of Grammatical Relations
A parsing system returning analyses in the form of sets of grammatical relations can obtain high precision if it hypothesises a particular grammatical relation only when it is certain that the relation is correct. We operationalise this technique -in a statistical parser using a manually-developed wide-coverage grammar of English - by only returning relations that form part of all analyses licensed by the grammar. We observe an increase in precision from 75% to over 90% (at the cost of a reduction in recall) on a test corpus of naturally-occurring text.
45352
A Practical Approach to Multiple Default Inheritance for Unification-Based Lexicons
This paper describes a unification-based lexicon system for NLP applications that incorporates mechanisms for multiple default inheritance. Such systems are intractable in the general case---the approach adopted here places a number of restrictions on the inheritance hierarchy in order to remove some of the sources of complexity while retaining more desirable properties. Implications of the design choices are discussed, comparisons are drawn with related work in computational linguistics and AI, and illustrative examples from the lexicons of German and English are given.
39105600
Estimating the Reliability of Regeneration-Based Replica Control Protocols
The accessibility of vital information can be enhanced by replicating the data on several sites and employing a consistency control protocol to manage the replicas. The reliability of a replicated data object depends on maintaining a viable set of current replicas. When storage is limited, it may not be feasible to simply replicate a data object at enough sites to achieve the desired level of reliability. Regeneration approximates the reliability provided by additional replicas for a modest increase in storage costs, and is applicable whenever a new replica of a data object can be created faster than a system failure can be repaired. Regeneration enhances reliability by creating new replicas on other sites in response to site failures. Several strategies for replica maintenance are considered, and the benefits of each are analyzed using simulation and both algebraic and numeric solutions to systems of differential equations. >
2198481
Participatory analysis: shared development of requirements from scenarios
Participatory design typically focuses on envisionment and evaluation activities. We explored a method for pushing the participatory activities further “upstream” in the design process, to the initial analysis of requirements. We used a variant of the task-artifact fi-arnework, carrying out a participatory claims analysis during a design workshop fm a project addressing collaborative science education. The analysis used videotaped classroom sessions as source material. The participant-teachers were highly engaged by the analysis process and contributed significantly to the analysis results. We conclude that the method has promise as a technique for evoking self-reflection and analysis in a participatory design setting.
2269827
Training wheels in a user interface
New users of high-function application systems can become frustrated and confused by the errors they make in the early stages of learning. A training interface for a commercial word processor was designed to make typical and troublesome error states “unreachable,” thus eliminating the sources of some new-user learning problems. Creating a training environment from the basic function of the system itself afforded substantially faster learning coupled with better learning achievement and better performance on a comprehension post-test. A control group spent almost a quarter of their time recovering from the error states that the training interface blocked off. We speculate on how this training strategy might be refined, and more generally, on how function should be organized in a user interface.
14571844
The process of ISD methodology selection and use: a case study
This paper provides empirically-based explanations for the observed low, and partial, use of information systems development (ISD) methodologies. The research reported in this paper examines the process of methodology selection and use in an intensive field study of the early stages of systems development and notes ongoing adaptation of methodologies in response to changing contingencies. The paper concludes that, in many circumstances, methodology adaptation is a necessary part of successful systems development. It also suggests that deeper understanding of the attitudes of different stakeholders to selecting and using ISD methodologies may help to explain this adaptation.
15057511
Completing design in use: closing the appropriation cycle
Users appropriate a technology innovation as they adapt and adapt to its capabilities. This paper argues that the appropriation of technology innovations – information and communication technologies such as devices and systems – is actually part of the design process. The design of a technology innovation is completed by users as they appropriate it. The contributions of the paper are to draw attention to the crucial role played by users’ actions in completing the design process and to examine the implications for the design and implementation of technology innovations. The challenge for designers is to design malleable technologies that can be adapted to users’ organisational, social and personal practices, and then to harvest users’ needs from the appropriated innovation in order to improve its design. For managers, trainers and IS staff involved in the implementation of innovations, the challenge is to encourage and support users’ appropriation activities.