- Alexander I. Rudnicky, M. Sakamoto. 1989. Transcription conventions and evaluation techniques for spoken language system research. Abstract: We describe the transcription conventions currently in use for spontaneous speech at Carnegie Mellon University. Two sets of conventions are described, a detail-rich system for wizard experi ments, and a more rigid evaluation system designed for purposes of SLS evaluation. The latter is suitable for automatic scoring using the existing NBS (now NIST) scoring software. A sample wizard transcription is included as well as a sample of live-system transcription together with system output Transcripts can be used to generate a number of diagnostic metrics useful for system evaluation. The research described in this paper was sponsored by the Defense Advanced Research Projects Agency (DOD), ARPA Order No. 5167, monitored by SPAWAR under contract N00039-85-C-0163. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Defense Advanced Research Projects Agency or of the US Government. Abstract This note describes the transcription conventions currently in use for spontaneous speech at Carnegie Mellon University. Two sets of conventions arc described, a detail-rich system for wizard experiments, and a more rigid system designed for purposes of SLS evaluation. The latter is suitable for automatic scoring using the existing NBS (now NIST) scoring software. A sample wizard transcription is included as well as a sample of live-system transcription together with system output. Transcripts can be used to generate a number of diagnostic metrics useful for system evaluation.
