4484500
Perception of Blended Emotions: From Video Corpus to Expressive Agent
Real life emotions are often blended and involve several simultaneous superposed or masked emotions. This paper reports on a study on the perception of multimodal emotional behaviors in Embodied Conversational Agents. This experimental study aims at evaluating if people detect properly the signs of emotions in different modalities (speech, facial expressions, gestures) when they appear to be superposed or masked. We compared the perception of emotional behaviors annotated in a corpus of TV interviews and replayed by an expressive agent at different levels of abstraction. The results provide insights on the use of such protocols for studying the effect of various models and modalities on the perception of complex emotions.
15015024
Annotating Multimodal Behaviors Occurring During Non Basic Emotions
The design of affective interfaces such as credible expressive characters in story-telling applications requires the understanding and the modeling of relations between realistic emotions and behaviors in different modalities such as facial expressions, speech, hand gestures and body movements. Yet, research on emotional multimodal behaviors has focused on individual modalities during acted basic emotions. In this paper we describe the coding scheme that we have designed for annotating multimodal behaviors observed during mixed and non acted emotions. We explain how we used it for the annotation of videos from a corpus of emotionally rich TV interviews. We illustrate how the annotations can be used to compute expressive profiles of videos and relations between non basic emotions and multimodal behaviors.
14471368
A Unified Framework for Constructing Multimodal Experiments and Applications
Inspired by a Wizard of Oz (WOZ) simulation experiment, we developed a working prototype of a system that enables users to interact with a map display through synergistic combinations of pen and voice. To address many of the issues raised by multimodal fusion, our implementation employed a distributed multi-agent framework to coordinate parallel competition and cooperation among processing components. Since then, the agent-based infrastructure has been enhanced with a collaboration technology, creating a framework in which multiple humans and automated agents can naturally interact within the same graphical workspace.Our current endeavour is the leveraging of this architecture to create a unified implementation framework for simultaneously developing both WOZ simulated systems and their fully-automated counterparts. Bootstrapping effects made possible by such an approach are illustrated by an experiment currently under way in our laboratory: as a naive subject draws, writes, and speaks requests to a (simulated) interactive map, a hidden Wizard responds as efficiently as possible using our best fullyautomated system, through either standard graphical interface devices or multimodal combinations of pen and voice. The input choices made by both subject and Wizard are invaluable, and the data collected from each can be applied directly to evaluating and improving the automated part of the system.
12923227
Levels of Representation in the Annotation of Emotion for the Specification of Expressivity in ECAs
In this paper we present a two-steps approach towards the creation of affective Embodied Conversational Agents (ECAs): annotation of a real-life nonacted emotional corpus and animation by copy-synthesis. The basis of our approach is to study how coders perceive and annotate at several levels the emotions observed in a corpus of emotionally rich TV video interviews. We use their annotations to specify the expressive behavior of an agent at several levels. We explain how such an approach can be useful for providing knowledge as input for the specification of non-basic patterns of emotional behaviors to be displayed by the ECA (e.g. which perceptual cues and levels of annotation are required for enabling the proper recognition of the emotions).
28154888
Architecture of a Framework for Generic Assisting Conversational Agents
In this paper, we focus on the notion of Assisting Conversational Agents (ACA) that are embodied agents dedicated to the function of assistance for novice users of software components and/or web services. We discuss the main requirements of such agents and we emphasize the genericity issue arising in the dialogical part of such architectures. This prompts us to propose a mediator-based framework, using a dynamic symbolic representation of the runtime of the assisted components. Then we define three strategies for the development of the mediators that are validated by the implementation of experiences taken in various situations.
27507077
Temporal codes within a typology of cooperation between modalities
The Human-Machine Communication Department of LIMSI (LIMSI Report 1994), a laboratory of the French National Scientific Research Agency (CNRS), conducts research in the field of Pattern Recognition, Artificial Intelligence and Pattern Generation. Several approaches are currently investigated there for studying interactions between modalities: studies about the cognitive processes involved in human intermodal tasks (Denis and Cocude 1992; Gryl 1994; Daniel et al. 1994), image and language interactions for learning (Bordeaux 1993), human factors (Castaing and True 1993), symbolic approaches for designing multimodal human-computer interfaces (Bellik and Teil 1993; Bares et al. 1992), spatial reasoning (Ligozat 1992; Briffault 1992).
26598527
Large Tandem Queueing Networks with Blocking
Systems consisting of many queues in series have been considered by Glynn and Whitt (1991) and Baccelli, Borovkov and Mairesse (2000). We extend their results to apply to situations where the queues have finite capacity and so various types of “blocking” can occur. The models correspond to max-plus type recursions, of simple form but in infinitely many dimensions; they are related to “percolation” problems of finding paths of maximum weight through a 2-dimensional lattice with random weights at the vertices. Topics treated include: laws of large numbers for the speed of customers progressing through the system; stationary behaviour for systems with external arrival processes; a functional central limit theorem describing the behaviour of the “front of the wave” progressing through a system which starts empty; stochastic orderings for waiting times of customers at successive queues. Several open problems are noted.
5918485
A framework for dynamic Byzantine storage
We present a framework for transforming several quorum-based protocols so that they can dynamically adapt their failure threshold and server count, allowing them to be reconfigured in anticipation of possible failures or to replace servers as desired. We demonstrate this transformation on the dissemination quorum protocol. The resulting system provides confirmable wait-free atomic semantics while tolerating Byzantine failures from the clients or servers. The system can grow without bound to tolerate as many failures as desired. Finally, the protocol is optimal and fast: only the minimal number of servers - 3f + 1 - is needed to tolerate any f failures and, in the common case, reads require only one message round-trip.
44345524
Extracting Keyphrases from Spoken Audio Documents
Spoken audio documents are becoming more and more common on the World Wide Web, and this is likely to be accelerated by the widespread deployment of broadband technologies. Unfortunately, speech documents are inherently hard to browse because of their transient nature. One approach to this problem is to label segments of a spoken document with keyphrases that summarise them. In this paper, we investigate an approach for automatically extracting keyphrases from spoken audio documents. We use a keyphrase extraction system (Extractor) originally developed for text, and apply it to errorful Speech Recognition transcripts, which may contain multiple hypotheses for each of the utterances. We show that keyphrase extraction is an "easier" task than full text transcription and that keyphrases can be extracted with reasonable precision from transcripts with Word Error Rates (WER) as high as 62%. This robustness to noise can be attributed to the fact that keyphrase words have a lower WER than non-keyphrase words and that they tend to have more redundancy in the audio. From this we conclude that keyphrase extraction is feasible for a wide range of spoken documents, including less-than-broadcast casual speech. We also show that including multiple utterance hypotheses does not improve the precision of the extracted keyphrases.
8592573
Contextual Spelling Correction Using Latent Semantic Analysis
Contextual spelling errors are defined as the use of an incorrect, though valid, word in a particular sentence or context. Traditional spelling checkers flag misspelled words, but they do not typically attempt to identify words that are used incorrectly in a sentence. We explore the use of Latent Semantic Analysis for correcting these incorrectly used words and the results are compared to earlier work based on a Bayesian classifier.
2946678
Conventional Metaphor and the Lexicon
Metaphor and other forms of non-literal language are essential parts of language which have direct bearing on theories of lexical semantics. Neither narrow theories of lexical semantics, nor theories relying solely on world knowledge are sufficient to account for our ability to generate and interpret non-literal language. This paper presents an emerging approach that may provide such an account. This approach is based on systematic representations that capture non-literal language conventions, and mechanisms that can dynamically understand and learn new uses as they are encountered.