Semantic recognition of throat vibration signal
71149920
Bottleneck feature mediated DNN based feature mapping for throat microphone speech recognition
Throat microphones are more robust to environmental noises than usual acoustic microphones such as close talk microphones because they detect speech signals through skin vibrations rather than by air transmission. Throat microphones, however, cannot be used in conventional speech recognition systems because their acoustic characteristics are much different from those of the acoustic microphones. In this study, we propose a deep neural network (DNN) based feature mapping method for throat microphone speech recognition. To utilize a large amount of training data recorded by acoustic microphones and effectively reduce the acoustic mismatch between the throat and acoustic microphones, we tried to use the bottleneck features to mediate between them. Evaluation results for a large vocabulary speech recognition task of Japanese free conversation revealed that the proposed system had a 45.8% lower character error rate (75.5% 40.9% than the typical MFCC system trained from the acoustic microphone data.
2018 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)
Takahito Suzuki Jun Ogata Takashi Tsunakawa Masafumi Nishida Masafumi Nishimura
44548625
Throat microphone speech recognition using mfcc
The Throat Microphone (TM) is a non acoustic device, relying on the vibrations of vocal folds rather than the audible sound produced. Correctly capturing vocal fold vibrations is difficult due to poor signal representation capabilities. The system recognizes the TM vibrations and produces the corresponding speech sound. This is done by extracting features from the spectrum of the TM vibrations and comparing the obtained features with the values stored in a database. The extracted features include characteristic features of the speech waveform called Mel Frequency Cepstral Coefficients (MFCC) The selection of the closest speech signal is chosen by the minimum mean square error estimation method, where the signal in the database whose corresponding MFCC values show the least difference from the input speech MFCCs is selected. This system has the potential of having applications for giving voice to those with defective speech and in military communications.
2017 International Conference on Networks Advances in Computational Technologies (NetACT)
Amritha Vijayan Bipil Mary Mathai Karthik Valsalan Riyanka Raji Johnson Lani Rachel Mathew K Gopakumar
226245811
Ontology and signal analysis based fault identification for ultra precision flycutting machines
The ultra precision single point diamond flycutting method has many positive characteristics such as high production efficiency, good repeatability, environmente friendly and easy to maintain, it can produce fine optical surfaces directly. But the motion errors which would influence the quality of the machined surfaces are very weak and always coupled, it is hard to achieve the error detection and error source identification for ultra precision flycutting machines. To solve this problem, different measuring data such as spindle error, feed error, vibrations and temperatures of different components, and compressed air pressure were collected synchronously in real time, and then we established the fault diagnosis method based on ontology and semantic technology to analysis the acquired measuring signals. The method includes signal analysis and feature extraction of raw data, fault recognition based on continuous Gaussian mixture density hidden Markov model (CGHMM) and semantic mapping technique which relating signal analysis to ontology elements. It combines the advantages of signal analysis and ontology, to identify fault information timely and accurately. Taking the spindle vibration signal of the machine tool as an example. Under different working conditions, feature extraction of the spindle vibration signals were acquired. The fault identification is carried out by CGHMM. Finally, semantic mappings and knowledge reasoning for the identification result can validate the performance of this method, several error souces which can introduce mid spacial frequency patterns on the machined surfaces were located and solved separately.
Chenhui An Jinlei Ma Wanlian Yang Qiao Xu Jianfeng Zhang Xiangyang Lei Junqing Wang
62410255
Pattern Recognition and Image Analysis 7th Iberian Conference, IbPRIA 2015, Santiago de Compostela, Spain, June 17 19, 2015, Proceedings
Pattern Recognition and Machine Learning. Spatiotemporal Stacked Sequential Learning for Pedestrian Detection. Social Signaling Descriptor for Group Behaviour Analysis. A New Smoothing Method for Lexicon Based Handwritten Text Keyword Spotting. Empirical Evaluation of Different Feature Representations for Social Circles Detection. Source Target Source Classification Using Stacked Denoising Autoencoders. On the Impact of Distance Metrics in Instance Based Learning Algorithms. Multi class Boosting for Imbalanced Data. Computer Vision. Object Discovery Using CNN Features in Egocentric Videos. Prototype Generation on Structural Data Using Dissimilarity Space Representation: A Case of Study. Estimation and Tracking of Partial Planar Templates to Improve VSLAM. Human Centered Scene Understanding Based on 3D Long Term Tracking Data. Fast Head Pose Estimation for Human Computer Interaction. Structured Light System Calibration for Perception in Underwater Tanks. Dense 3D SLAM in Dynamic Scenes Using Kinect. Scene Recognition Invariant to Symmetrical Reflections and Illumination Conditions in Robotics. System for Medical Mask Detection in the Operating Room Through Facial Attributes. Image and Signal Processing. New Method for Obtaining Optimal Polygonal Approximations. Noise Decomposition Using Polynomial Approximation. Color Correction for Image Stitching by Monotone Cubic Spline Interpolation. Structured Output Prediction with Hierarchical Loss Functions for Seafloor Imagery Taxonomic Categorization. Escaping Path Approach with Extended Neighborhood for Speckle Noise Reduction. Adaptive Line Matching for Low Textured Images. Unsupervised Approximation of Digital Planar Curves. On the Modification of Binarization Algorithms to Retain Grayscale Information for Handwritten Text Recognition. Applications. Improving the Minimum Description Length Inference of Phrase Based Translation Models. A Kinect Based System to Assess Lymphedema Impairments in Breast Cancer Patients. Arabic Writer Identification Using Local Binary Patterns (LBP) of Handwritten Fragments. A Fuzzy C Means Algorithm for Fingerprint Segmentation. Word Graph Based Applications for Handwriting Documents: Impact of Word Graph Size on Their Performances. Temporal Segmentation of Digital Colposcopies. Goal Driven Phenotyping Through Spectral Imaging for Grape Aromatic Ripeness Assessment. Medical Image. Robust 3D Active Shape Model for the Segmentation of the Left Ventricle in MRI. Kernel Based Feature Relevance Analysis for ECG Beat Classification. Spatial Dependent Similarity Metric Supporting Multi atlas MRI Segmentation. Color Detection in Dermoscopy Images Based on Scarce Annotations. Pattern Recognition and Machine Learning. Spectral Clustering Using Friendship Path Similarity. R Clustering for Egocentric Video Segmentation. Applying Basic Features from Sentiment Analysis for Automatic Irony Detection. Exploiting the Bin Class Histograms for Feature Selection on Discrete Data. Time Series Prediction Based on Kernel Adaptive Filtering with Cyclostationary Codebooks. Latent Topic Encoding for Content Based Retrieval. Dissimilarity Based Learning from Imbalanced Data with Small Disjuncts and Noise. Binary and Multi class Parkinsonian Disorders Classification Using Support Vector Machines. Peripheral Nerve Segmentation Using Speckle Removal and Bayesian Shape Models. Measuring Scene Detection Performance. Threshold of Graph Based Volumetric Segmentation. Brain Neural Data Analysis with Feature Space Defined by Descriptive Statistics. Extremely Overlapping Vehicle Counting. Sentence Clustering Using Continuous Vector Space Representation. Online Learning of Stochastic Bi automaton to Model Dialogues. Single Channel Separation Between Stationary and Non stationary Signals Using Relevant Information. Computer Vision. A New Trajectory Based Motion Segmentation Benchmark Dataset (UdG MS15) Escritoire: A Multi touch Desk with e Pen Input for Capture, Management and Multimodal Interactive Transcription of Handwritten Documents. Person Enrollment by Face Gait Fusion. Homographic Class Template for Logo Localization and Recognition. Multimodal Object Recognition Using Random Clustering Trees. Videogrammetry System for Wind Turbine Vibration Monitoring. Canonical Views for Scene Recognition in Mobile Robotics. Crater Detection in Multi ring Basins of Mercury. Iterative Versus Voting Method to Reach Consensus Given Multiple Correspondences of Two Sets. Extracting Categories by Hierarchical Clustering Using Global Relational Features. A Calibration Algorithm for Multi camera Visual Surveillance Systems Based on Single View Metrology. 3D Guided Multiscale Sliding Window for Pedestrian Detection. A Feature Based Gaze Estimation Algorithm for Natural Light Scenarios. Image and Signal Processing. Fast Simple Linear Iterative Clustering by Early Candidate Cluster Elimination. Pectoral Muscle Segmentation in Mammograms Based on Cartoon Texture Decomposition. PPG Beat Reconstruction Based on Shape Models and Probabilistic Templates for Signals Acquired with Conventional Smartphones. Peripheral Nerves Segmentation in Ultrasound Images Using Non linear Wavelets and Gaussian Processes. Improving Diffusion Tensor Estimation Using Adaptive and Optimized Filtering Based on Local Similarity. Dimension Reduction of Hyperspectral Image with Rare Event Preserving. Applications. Clustering of Strokes from Pen Based Music Notation: An Experimental Study. Image Analysis Based Automatic Detection of Transmission Towers Using Aerial Imagery. A Sliding Window Framework for Word Spotting Based on Word Attributes. Combining Statistical and Semantic Knowledge for Sarcasm Detection in Online Dialogues. Estimating Fuel Consumption from GPS Data. A Bag of phonemes Model for Homeplace Classification of Mandarin Speakers. A Gaussian Process Emulator for Estimating the Volume of Tissue Activated During Deep Brain Stimulation. Genetic Seam Carving: A Genetic Algorithm Approach for Content Aware Image Retargeting. Analysis of Expressiveness of Portuguese Sign Language Speakers. Automatic Eye Localization Multi block LBP vs. Pyramidal LBP Three Levels Image Decomposition for Eye Visual Appearance Description. Clinical Evaluation of an Automatic Method for Segmentation and Characterization of the Thoracic Aorta with and Without Aneurysm Patients. Combined MPEG7 Color Descriptors for Image Classification: Bypassing the Training Phase. A Multi platform Graphical Software for Determining Reproductive Parameters in Fishes Using Histological Image Analysis.
IbPRIA
Roberto Paredes Jaime S Cardoso Xose Manuel Pardo
136234895
Flexible and wearable electronic silk fabrics for human physiological monitoring
The development of textile based devices for human physiological monitoring has attracted tremendous interest in recent years. However, flexible physiological sensing elements based on silk fabrics have not been realized. In this paper, ZnO nanorod arrays are grown in situ on reduced graphene oxide coated silk fabrics via a facile electro deposition method for the fabrication of silk fabric based mechanical sensing devices. The data show that well aligned ZnO nanorods with hexagonal wurtzite crystalline structures are synthesized on the conductive silk fabric surface. After magnetron sputtering of gold electrodes, silk fabric based devices are produced and applied to detect periodic bending and twisting. Based on the electric signals, the deformation and release processes can be easily differentiated. Human arterial pulse and respiration can also be real time monitored to calculate the pulse rate and respiration frequency, respectively. Throat vibrations during coughing and singing are detected to demonstrate the voice recognition capability. This work may not only help develop silk fabric based mechanical sensing elements for potential applications in clinical diagnosis, daily healthcare monitoring and voice recognition, but also provide a versatile method for fabricating textile based flexible electronic devices.
Cuiping Mao Huihui Zhang Zhisong Lu
204958582
Speech Emotion Recognition Measuring Driver States from Voice Characteristics
Detecting emotions and driver states in a broad sense (e.g. annoyance, anger, and fatigue) can help to enhance acceptance and comfort of these systems themselves as well as accident related safety. Adapting the system output, e.g. in speech interfaces of Driver Assistance Systems to the actual emotional and energetic driver states might enhance the acceptance of this empathic, emotional intelligent communication due to its improved naturalism. In addition, comfort and comprehensiveness might be improved if the system's output is adapted to the driver's actual e.g. stress impaired attentional and cognitive resources. Furthermore, the detection of distracted driver states (e.g. pain, anger, emotional intensive conversation) could be beneficial from the viewpoint of safety concerns. Using voice communication as an indicator of emotion would have the following advantages: obtaining speech data is non obtrusive, free from sensor application, calibration efforts, and robust against climatic environmental conditions. Several validation studies analysing e.g. basic emotions, fatigue, stress are presented in this paper reaching recognition rates for 2 class problems of about 80 90 Finally, current limitations and future demands (as e.g. freely accessible speech databases) for an efficient progress in speech emotions recognition are discussed. Measuring Driver States Many efforts have been reported in the literature for automatic measurement of emotional states. These systems mainly focus on (a) electropysiological data (e.g. EEG: Golz, Sommer, Holzbrecher Schnupp, 2007; Honig, Batliner Noth, 2007) and (b) behavioural expression data (gross body movement, head movement, mannerism, and facial expression; Horlings, Datcu Rothkrantz, 2008) in order to characterize the user state. But these electrode(EOG/EEG reaching 15% error rate for fatigue detection; Sommer Golz, 2005) or videobased instruments still do not fulfil the demands of an everyday life measurement system. The major drawbacks are (a) a lack of robustness against environmental and individual specific variations (e.g. bright light, wearing correction glasses, angle of face or being of Asian race) and (b) a lack of comfort and longevity due to electrode sensor application. In contrast to these electrodeor video based instruments, the utilization of voice communication as an indicator for emotional states could match the demands of everyday life measurement. Contact free measurements as voice analysis are non obtrusive (not interfering with the primary driving task) and favourable for emotion detection, since an application of sensors would cause annoyance, additional stress and often impairs working capabilities and mobility demands. In addition, speech is easy to record even under extreme environmental conditions (bright light, high humidity and temperature) requires merely cheap, durable, and maintenance free sensors and most importantly, it utilizes already existing communication system hardware. Furthermore, speech data is omnipresent in many professional driver settings. Given these obvious advantages, the renewed interest in computational demanding analyses of vocal expressions has been enabled just recently by the advances in computer processing speed (Batliner, Steidl, Schuller et al, 2006; Juslin Scherer, 2005; Owren Bachorowski, 2007; Scherer, Johnstone Klasmeyer, 2003; Schuller, Batliner, Seppi et al. 2007) Speech Emotion Recognition Measuring Driver States from Voice Characteristics 99 Acoustic Features The acoustic speech emotion recognition research is mainly based on phonetics, general signal processing and computational intelligence research (e.g. Batliner, Steidl, Schuller et al. 2006) How emotions are expressed in the voice can be analyzed acoustically by measuring the characteristics of the speech wave form radiating from nostrils and mouth. Accordingly, acoustic features can be divided referring to auditive perceptual concepts into prosody (pitch, intensity, rhythm, pause pattern, and speech rate) articulation (slurred speech, reduction and elision phenomena) and speech quality (timbre: breathy, whispery, tense, sharp, hoarse, or modal voice) A popular approach prefers the fusion of purely signal processing based features without any known auditive perceptual correlates and perceptual acoustic features as e.g. the speech intensity (loudness) fundamental frequency (pitch) voiced/unvoiced duration (rhythm) and several voice quality measures. Suitable measures for the voice quality category whisperiness are level of the first harmonic, bandwidth of the first formant, large amount of interharmonic noise, and steep spectral tilt. The spectrum of tense voices is flatter than that of lax voices. Creaky voice is characterized by a relatively flat spectrum, i.e. relatively strong upper harmonics. Finally, harsh and rough voices are characterized by a large amount of jitter, shimmer, and interharmonic noise. Typical acoustic features which implicitly cover the categories mentioned above and are used frequently in Speech Emotion Recognition (SER) (see Table 1) are (a) fundamental frequency, (b) intensity, (c) duration of voiced/unvoiced speech segments, (d) harmonics to noise ratio (HNR) (e) formant positions (F1 F7) (f) formant bandwidths (Fbw1 Fbw7) (g) mel frequency cepstrum coefficients (MFCCs) (h) linear frequency cepstrum coefficients (LFCCs) (i) linear predictive coding coefficients, and (j) spectral features derived from the long term average spectrum (LTAS) Table 1. Basic acoustic feature contours (frame level descriptors) Frame level based feature Description Fundamental frequency (F0) acoustic equivalent to pitch; rate of vocal fold vibration; maximum of the autocorrelation function; models prosodic structure; speech melody indicator Energy models intensity, based on the amplitude in different intervals; average squared amplitude within a predefined time segment; stressing structure Harmonics to noise Ratio (HNR) spectral energy in voiced vs. unvoiced segments; ratio between harmonic and aperiodic signal energy; breathiness indicator Formant position (F1 F7) resonance frequencies of the vocal tract (VT) depending strongly on its actual shape; represent spectral maxima, and are known to model spoken content and speaker characteristics; influenced by lower jaw angle, tongue body angle, tongue body horizontal location, tongue tip angle, tongue tip horizontal location, relative lip height, lip protrusion, velum height Formants bandwidth (Fbw1 Fbw7) model VT shape and energy loss of speech signal due to VT elasticity (yielding wall effect) viscoeleasticity of VT tissue or heat conduction induced changes of air flow (jet streams, turbulences) width of the spectral band containing significant formant energy (3 dB threshold) Duration of voiced unvoiced segments models temporal speech rhythm aspects as speech rate and pause structure Mel frequency cepstrum coefficients (MFCCs) "spectrum of the spectrum" have been proven beneficial in speech emotion recognition, and speech recognition tasks; homomorphic transform with equidistant band pass filters on the Mel scale; Holistic and decorrelated representation of spectrum Linear frequency cepstrum coefficients (LFCCs) similar to MFCC but without the perceptually oriented transformation into the mel frequency scale; emphasize changes or periodicity in the spectrum, while being relatively robust against noise Jarek Krajewski, Elmar Noth und Anton Batliner 100 Linear Predictive Coding (LPC) Provides an accurate and economical representation of the envelope of the short time power spectrum. One of the most powerful speech coding analysis techniques providing very accurate estimates of speech parameters and is known as being relatively efficient for computation at the same time Long Term Average Spectrum (LTAS) averages out formant information; giving general spectral trends; relative amount of energy within predefined frequency bands; speech quality The speaker's emotional state expresses itself through prosodic phonetic information as pitch, intensity, speaking rate, duration, articulation and voice quality (see Figure 2) Furthermore, this prosody related information conveyed by speech can be enriched by paralinguistic information, which contains short, emotional non speech expressions, comprising both clear non speech sounds and interjections with a phonemic structure (cf. affective bursts, Schroeder, 2001) Laughing, sniffs, sobs, or morphing speech into a cry "noooo " swearing, yawning, interjections (e.g. "wow!" deep breathing as sound of relief are typical examples of these paralinguistic events, which might be used for speech emotion recognition (Truong van Leeuwen, 2007) Furthermore, emotional information is carried in the human speech over the information channel of syntax and semantics (lexical content) Syntactic channels include information as part of speech categories (POS; e.g. number of nouns, non inflected adjectives) other syntax related information (n grams, bag of words, elaborated vs. simple construction, many vs. few word per sentence; many vs. few negations; 1 person singular use, tentative words, present vs. past tense) Linguistic Inquiry and Word Count (LIWC; e.g. categories as anger words, inclusive words, social processes) and MRC psycholinguistic features (concreteness of words, imageability; cf. Mairesse, Walker, Mehl Moore, 2007) Moreover, the analysis of semantic channels might include: topic selection (e.g. self focused, pleasure vs. problem talk, single vs. multiple topics, strict selection vs. think out loud bag of words features) and conversational behaviour characteristics (listen vs. initiate conversation, back channel behaviour, formal style) The major drawbacks of SER using syntactic or semantic information are the difficulties related to an automatic identification of these categories. In addition, a word tagger
Jarek Krajewski Elmar Noth Anton Batliner
55057021
The Influence of Phonetic Symbolism on Memory for Advertisements
This experiment investigated the influence of phonetic symbolism on memory in the context of a marketing study. Participants first judged the appeal and probable success of advertisements for large and small products that were paired either congruently or incongruently with nonword brand names according to size, and then completed surprise recall and recognition tasks about the brand names, products, and pairings. It was predicted that participants would have better memory for brand names, products, and pairings that displayed a congruent vs. incongruent relationship between the associated size of the brand name vowel and the size of the product. The results showed some support for these hypotheses, but primarily for back vowellarge product pairings. These findings suggest that using phonetic symbolism in brand names for large products will result in consumers having better memory for those brand names and products, which has implications for how companies name and advertise future products. PHONETIC SYMBOLISM AND ADVERTISEMENT MEMORY Allen 4 The Influence of Phonetic Symbolism on Memory for Advertisements Clorox. Kraft. Kleenex. Chapstick. These words are easily recognizable as brand names, and these brand names quickly bring to mind the products with which they are associated. All advertisers strive to create memorable brand names because, as seen when wandering through aisles of the grocery store, a quick glance at a brand name may be the only interaction potential buyers have with a product before deciding whether to purchase it (Gontijo Zhang, 2007) At the same time, advertisers want potential buyers to know what the positive attributes of their products are, such as fast acting for medicine or long lasting for light bulbs and air fresheners (Keller et al. 1998) The competing interests of memorability and informativeness thus make developing brand names a challenging task. While research in the realm of language suggests that brand names can communicate important product attributes solely through the sounds of their brand names, little research has examined whether the phonetic properties of brand names influence memorability. This study aims to examine this link by investigating whether participants have a better memory for brand names, products, and their pairings when there is a congruent vs. incongruent relationship between the size conveyed by the vowel sound of the brand name and the actual size of the product. Phonetic Symbolism in Language What is language? To understand how brand names' vowels can convey characteristics of referent products, it is first necessary to explore the basics of human language. Given that so many animals have communication systems but humans are the only creatures that use spoken language, Charles Hockett published The Origin of Speech in 1960 which detailed 13 qualities that distinguish human language from other animal communication systems: "vocal auditory channel," PHONETIC SYMBOLISM AND ADVERTISEMENT MEMORY Allen 5 "broadcast transmission and directional reception," "transitoriness," "interchangeability," "total feedback," "specialization," "semanticity," "arbitrariness," "discreteness," "displacement," "productivity," "traditional transmission," and "duality of patterning." Definitions of all of these criteria are provided in Figure 1 below. Figure 1: Hockett's Criteria of Language From: Hauser, M. D. (1996) The Evolution of Communication. Cambridge, Massachusetts: MIT Press. Three of these criteria warrant further discussion due to their relevance in brand name creation. "Discreteness" signifies that a sound produced at any given frequency is only heard as PHONETIC SYMBOLISM AND ADVERTISEMENT MEMORY Allen 6 one phoneme due to the distinct ranges of each sound within a language (Hockett, 1960) For example, in English, all bilabial stops with a voice onset time (time between release of sound and vibration of vocal cords) between zero and forty milliseconds are strictly perceived as the phoneme b, whereas those with a voice onset time of 40 milliseconds or more are strictly perceived as p (Carroll, 2004) "Duality of patterning" explains that humans can take the finite number of phonemes within their language and create an enormous lexicon because the same phoneme groups can have entirely different meanings based on their arrangement, as seen with "late" and "tale" (Hockett, 1960) Finally, "arbitrariness" signifies that there is no link in meaning between the words with which we label objects, and the properties of those objects; for example, there is no property of the object table or the word table that necessitates that the two are linked (Hockett, 1960) Is language truly arbitrary? Challenges from onomatopoeia and phonetic symbolism Arbitrariness, or arbitrary symbolic reference, is one of the most defining characteristics of human language because it is used in very few other animal communication systems (Hockett, 1960) Since there is no connection between the sounds of words and the properties of their referent objects, arbitrary symbolic reference thus allows humans to create an extensive number of words to communicate about the various things they encounter in the world (Hockett, 1960) While Hockett believed that language is entirely arbitrary, two phenomena in the English language, onomatopoeia and phonetic symbolism, challenge this assumption. Onomatopoeia and phonetic symbolism are both types of sound symbolism, which is defined as "the direct linkage between sound and meaning" (Hinton et al. 1994, p. 1) According to Hinton, Nichols, Ohala (1994) there are four types of sound symbolism: corporeal, imitative, synesthetic, and conventional. Corporeal sound symbolism encompasses sounds that PHONETIC SYMBOLISM AND ADVERTISEMENT MEMORY Allen 7 reveal a speaker's inner physical or emotional state, such as with "Aaugh!" demonstrating frustration or "Achoo!" demonstrating illness (Hinton et al. 1994) Imitative sound symbolism includes the phenomenon of onomatopoeia, in which words convey sounds in the environment, such as "bzz" and "knock" (Hinton et al. 1994) Synesthetic sound symbolism, or phonetic symbolism, refers to the idea that the same vowels and consonants are repeatedly used to convey specific sensory properties of objects, such as size or shape; one example is the use of high vowels to convey smallness, such as with "tiny" and "little" (Hinton et al. 1994) Finally, conventional sound symbolism reflects that certain phonemes are associated with specific meanings in a given language, such as the "gl" sound in English being associated with light, as seen in "glitter" and "gleam" (Hinton et al. 1994) In all types of sound symbolism, the phonetic properties of certain words convey some level of meaning about their referent objects, thus challenging language's supposed arbitrariness (Hinton et al. 1994) The main issue of interest in the present research, however, is to specifically investigate the ways in which phonetic symbolism challenges arbitrary symbolic reference. To understand how words can convey sensory properties, it is first necessary to discuss how consonants and vowels are produced in the vocal tract. Vowels and consonants differ because vowels are produced without blocking the airflow from the lungs, while consonants briefly block airflow at some point in their production (Carroll, 2004) Consonants differ from one another in place of articulation, manner of articulation, and voicing, while vowels only differ from each other in place of articulation and positioning of the tongue (Carroll, 2004) Places of articulation for consonants include bilabial, labiodental, dental, alveolar, palatal, velar, and glottal; these sounds are ordered from those that are produced most forward in the mouth (bilabial) to those that are produced in the back of the throat (velar) (Carroll, 2004) PHONETIC SYMBOLISM AND ADVERTISEMENT MEMORY Allen 8 The manners of articulation for consonants are stops, fricatives, affricates, nasals, liquids, and glides (Carroll, 2004) Stops completely obstruct airflow for a brief period and then release the sound, while fricatives, liquids, and glides involve a partial obstruction of airflow that still allows sounds to be produced (Carroll, 2004) Affricates are combinations of stops and fricatives, while nasals are consonants whose airflow travels through the nasal cavity (Carroll, 2004) Finally, consonants are either voiced or voiceless; voiced consonants involve immediate vibration of the vocal cords as the sound is released, while voiceless consonants involve a brief delay between sound release and vibration of the vocal cords (Carroll, 2004) Vowels are articulated in the front, middle, or back of the mouth while the tongue is in a high, middle, or low position (Carroll, 2004) Examples of the different types of consonants and vowels in the English language are shown in Figures 2 and 3 respectively. Figure 2: Consonants with the English Language as a Function of Voicing and Manner and Place of Articulation Place of Articulation Manner of Articulation Voicing Bilabial Labiodental Dental Alveolar Palatal Velar Glottal Stops Voiced Voiceless p (pat) b (bat) t (tack) d (dig) k (cat) g (get) Fricatives Voiced Voiceless f (fat) v (vat) Th (thin) d (then) s (sat) z (zap) s (fish) z (azure) h (hat) Affricates Voiced Voiceless c (church) j (judge) Nasals m (mat) n (nat) NG (sing) Liquids l (late) r (rate) Glides w (win) y (yet) Source: (Gluckberg Danks, 1975) PHONETIC SYMBOLISM AND ADVERTISEMENT MEMORY Allen 9 Figure 3: Vowels within the English Language as a Function of Tongue Position and Place of Articulation Position of Tongue Place of Articulation Front Center Back i (beet) u (boot) High U (book) I (bit) o (bode) e (baby) (sofa) Middle e (bet) o (bought) ae (bat) (but) Low a (palm) Source: (Gluckberg Danks, 1975) Studies have found that vowels can convey different attributes depending on their place
Molly Allen
37668176
Mobile Games for Training Tactile Perception
Tactile interactive multimedia propose a wide spectrum of developmental games both for visually impaired children and adults. While some simulators can produce strong vibro tactile sensations, the discrimination of several tactile patterns remains quite poor. Skin sensitivity is not enough for remembering and recognizing vibration patterns (tactons) and their combinations. Short term tactile memory is the crucial factor in educational and vocational environments for deaf and blind people. We designed a vibro tactile pen and software to create tactons and semantic sequences of vibro tactile patterns on mobile devices (iPAQ pocket PC) We propose special games to facilitate learning and manipulation by tactons. The techniques are based on gesture recognition and spatial temporal mapping for imaging vibro tactile signals. The proposed approach and the tools implemented allow creating a new kind of mobile communication environment for deaf and blind people.
ICEC
Grigori E Evreinov Tatiana Evreinova Roope Raisamo
16230352
A testbed for voice based robot control
A hardware and software configuration is being developed to support research and development in speech recognition (SR) as applied to the interactive control of devices. A research goal is to investigate new signal processing techniques and parametric representations such as those used in speech coding to improve communication effectiveness for person to person communications, and to apply similar types of processing to improve the performance of machine recognition. SR systems are often developed and trained using prerecorded databases, which often do not capture the range of variation that can be expected in reality. For example, words spoken while reading or in casual conversation sound substantially different in live environments including noise, vibration, distractions, or emotional stress. A live system is desired to set up games or other situations which introduce excitement and competition, with the resulting changes in vocal characteristics. Therefore, the testbed includes high quality recognition software and allows a high degree of interactivity. It is intended to support testing and improvement of speech recognition algorithms and software in a realistic environment. It should be suitable for research (improved audio feature extraction algorithms, model adaptation, etc) engineering (reliable data acquisition front ends) and human factors evaluation. The paper discusses the motivation, chosen software and hardware approach, and some results. I plan to demonstrate the robot responding to various navigation commands using either a conventional microphone or a modified throat microphone.
2008 IEEE International Conference on Technologies for Practical Robot Applications
Joseph Rothweiler
111915430
Plenary lecture III: Enhancement and restoration
Persons that suffer from diseases such as throat cancer require that their larynx and vocal cords be extracted by a surgical operation, and then require rehabilitation in order to be able to reintegrate to their individual, social, familiar and work activities. To accomplish this, different methods have been used, such as: The esophageal speech, the use of tracheoesophagical prosthetics and the Artificial Larynx Transducer (ALT) also known as "electronic larynx" The ALT, which has the form of a handheld device, introduces an excitation in the vocal track by applying a vibration against the external walls of the neck. This excitation is then modulated by the movement of the oral cavity to produce the speech sound. This transducer is attached to the speaker's neck, and in some cases in the speaker's cheeks. The ALT is very easy using even for new patients, although the voice produced by these transducers is unnatural and with low quality, besides that it is distorted by the ALT produced background noise. The Esophageal speech is produced through the compression of the contained air in the vocal tract with the tongue. This air is swallowed and as passing through the esophageal pharynx segment produces a vibration of the esophageal upper muscle, bringing about the speech. The generated sound is similar to a burp, the tone is commonly very low and the timbre generally harsh. In ALT as well as in esophageal speech, the voiced segments are the most affected part of speech. Several approaches have been proposed to improve the quality and intelligibility of ALT produced, as well as esophageal speech signals. Some of them reduce the ALT produced background noise by using cepstral root subtraction or adaptive filtering. However the speech quality produced by these approaches is still poor. Another approach intended to improve the speech quality estimating the frequency band from 4 KHz to 8 KHz using the frequency band from 300Hz to 4 KHz. Although this approach may be an attractive alternative, it must be still improved. A promising approach is based on speech conversion techniques which carry out a spectral conversion using vector quantization methods. A similar approach based on a pattern recognition approach, has also been proposed, in which, firstly the voiced segments are detected and identified. Then the voiced segments are replaced by their equivalent voiced segments of normal speech while the unvoiced segments are kept without change. Finally the voiced, unvoiced and silence segments are concatenated together to produce the restored speech. These approaches perform fairly well although still present some problems because the spectral conversion reduce a continuous spectral space into a discrete code book, which may produce a distortion that still must be reduced. This speech presents a review of alaryngeal speech enhancement systems, providing also evaluation results to show the improvement in the quality and intelligibility of produced speech.
Hector Perez-Meana