[speaker001:] And we're going. So the only status ite well, first of all we h haven't decided whether we're [vocalsound] Meeting Recorder data issues, or recognition this week. I think we were recognition.
[speaker003:] Wha what was on the list? Th the [disfmarker] I mean, I sent you a couple things, although I don't remember them.
[speaker001:] You only sent me one thing, which was demo status. And asking which one we were on this week.
[speaker003:] Right. Ah. That was the second thing. Right. [vocalsound] Right.
[speaker001:] So should we simply assert that this week we are recognition, and next week data issues,
[speaker006:] Hmm.
[speaker002:] I think that's correct.
[speaker001:] and [disfmarker]?
[speaker006:] Yeah. I think so, too.
[speaker002:] Yeah.
[speaker001:] And, uh [disfmarker]
[speaker006:] Yeah.
[speaker001:] So, I think what we should probably do is any quick, small stuff we can do every week. So [disfmarker] like Morgan asked about the demo status. We can go ahead and talk about that a little bit. And then do [disfmarker] then alternate in more depth.
[speaker003:] By the way, I'll [disfmarker] I [disfmarker] I won't be here [pause] next Thursday. I'll be out of town.
[speaker001:] OK.
[speaker003:] But [disfmarker]
[speaker001:] Actually, I [pause] may not be here either. So I gotta double check the dates. But, [vocalsound] anyway. So, uh, demo status. First of all, I did a little thing for Liz with the Transcriber tool, that, um [disfmarker] first of all, it uses the forced alignments, so that the words appear, uh, in their own segments, rather than in long [disfmarker] in long chunks.
[speaker008:] Oh.
[speaker001:] She said that that [disfmarker] she thought that was a much better idea for the other stuff she's working on.
[speaker008:] Yeah. That's great.
[speaker001:] Um [disfmarker] And that works fine except it's even slower to load. It's already pretty slow to load.
[speaker004:] Yeah. It's more segments?
[speaker005:] Yeah. [vocalsound] It's slow.
[speaker004:] Or [disfmarker]? Yeah.
[speaker008:] Is that because the transcripts get longer?
[speaker004:] Yeah.
[speaker008:] The f transcript file gets longer?
[speaker001:] Yeah. Yep.
[speaker004:] Yep.
[speaker001:] And the Transcriber tool is just not very good n at [disfmarker] [vocalsound] at that.
[speaker008:] But th but that's [disfmarker] that's n You didn't have to change the software for that yet. Right?
[speaker001:] Correct.
[speaker008:] It's just formatting the right kind of, uh, XML [disfmarker]?
[speaker001:] Yeah, it's just writing conversion tools from the format that the aligner [disfmarker] Actually th he did a SRT file for it.
[speaker008:] Mm hmm. OK.
[speaker001:] And then just back into Transcriber [disfmarker] Transcriber format.
[speaker008:] Oh, good. That's very good. OK.
[speaker001:] Yeah. So my [disfmarker] my decision was, for the first pass for this demo that Liz was talking about [pause] I decided that I would do, um, only enough to get it working, as opposed to any coding.
[speaker008:] Mm hmm. Right.
[speaker001:] And so the other thing I sh she wanted to display the stylized F zeroes, I think they're called? Is that right?
[speaker005:] Yeah. The [pause] linear fit.
[speaker008:] Mm hmm.
[speaker001:] And, uh [disfmarker] So what I did is I just took the file with those in it, converted it so that it looks like an audio file.
[speaker008:] Right.
[speaker001:] And, so you s it shows that instead of the wavefile. And so that [disfmarker] that's working,
[speaker008:] Cool.
[speaker001:] and I think it actually looks pretty good.
[speaker008:] Mm hmm.
[speaker001:] Um, I'd like someone who's more familiar with it to look at it because when I was looking at it, [vocalsound] we seemed to have lots of stuff going on when no one's saying anything.
[speaker005:] That's just background speech.
[speaker001:] Ah.
[speaker005:] Yeah. So.
[speaker002:] So do you have to pad that out? Uh, so that it looks like it's an eight kilohertz sampled thing?
[speaker001:] No. I [comment] [disfmarker] the audio file you can specify any sampling rate.
[speaker002:] Or [disfmarker]?
[speaker001:] And so I s I specified [disfmarker] instead of, you know, sixteen thousand or eight thousand, I specified a hundred.
[speaker002:] Mmm.
[speaker001:] Um, and, the only problem with that is that there's a bug in Transcriber, that if the sample rate is too low, when it tries to compute the shape file, it fails.
[speaker008:] Hmm.
[speaker001:] Um, and crashes. Um [disfmarker] But the solution to that is just, set the option so it doesn't compute the shape file, [vocalsound] and it will work and the only problem with that is you can't, uh, zoom out on it. You can zoom in, but not out.
[speaker003:] What's a shape file?
[speaker001:] The shape file is [disfmarker] If you think about a wavefile, sixteen thousand samples per second is way too many to display on the screen.
[speaker003:] Mm hmm.
[speaker001:] So what Transcriber does, is it computes a [disfmarker] another thing to display based on the waveform.
[speaker005:] We're talking about [disfmarker] We're talking about that demo.
[speaker007:] Yeah. [comment] [nonvocalsound] Tried that, and it died.
[speaker001:] And it displays it at [disfmarker]
[speaker005:] Yeah. Did it?
[speaker001:] And it allows you to show m many different resolutions. So there's a little user interface component that lets you sh select the resolution.
[speaker007:] Great!
[speaker003:] I see.
[speaker001:] And if you don't compute the wavefile, you can't zoom out. You can't get a larger view of it. But you can zoom in.
[speaker008:] Hmm.
[speaker001:] Um [disfmarker] And that's alright, because at [disfmarker] at a hundred samples that's already pretty far [pause] out. And, uh [disfmarker] so I think it looks pretty good, but I'll let Liz look at it and see what she thinks.
[speaker007:] I [disfmarker] I got the wavefile but [disfmarker] Sorry. I got the wavefile, but I couldn't get the words yet. But the [disfmarker] the wavefile part looks [disfmarker] looks good.
[speaker001:] OK. We should [disfmarker] If you were having problems with the words, we should figure out why.
[speaker007:] I [disfmarker] I'll have done [disfmarker] I'm probably doing something wrong.
[speaker001:] OK.
[speaker007:] Sorry this microphone's moving around. I can't put this over my ear.
[speaker006:] It could [disfmarker] Do you have to put it in your ear?
[speaker001:] You c you clip that part over your ear.
[speaker007:] I can [disfmarker] I can do that but there's no orientation where the [disfmarker] Darn!
[speaker001:] Anyway. [comment] We'll all watch Liz play with the mike.
[speaker006:] I Does it really need to go in her ear?
[speaker001:] Um [disfmarker]
[speaker006:] That [disfmarker] that bud? It doesn't have to go in her ear. Right?
[speaker001:] Uh, no. It doesn't have to, but that [disfmarker] that's [disfmarker] I find that's the only way to wear it.
[speaker006:] Oh, wow.
[speaker001:] Is that the bud's in the ear and that the link is over it.
[speaker003:] This bud's for [disfmarker]
[speaker001:] But, so, anyway, I think that looks pretty good.
[speaker003:] No.
[speaker001:] The only [disfmarker] the only other thing we might wanna do with that is be able to display more than one waveform.
[speaker007:] Hmm.
[speaker001:] And that actually shouldn't be too slow, uh, because it's much lower resolution than a full waveform.
[speaker004:] Yeah.
[speaker001:] The problem with it is just it does require coding.
[speaker005:] Yeah. It migh
[speaker001:] And so it would be much better to get, uh, Dave Gelbart to do that than me, because he's familiar with the code, and is more likely to be able to get it to work quickly.
[speaker005:] It'd be nice if we can do, like, a quick hack, just so we can play [pause] the audio file, too.
[speaker007:] Right.
[speaker005:] Um, with th with the display.
[speaker001:] Oh, OK.
[speaker005:] Like, even if we [disfmarker] I think that even if we didn't display the waveform, [comment] it might be better to, rather, play the waveform than display it. I mean, like, if we were to choose [disfmarker] I don't know, if I were to choose between one or the other, I'd rather have it played.
[speaker001:] Mm hmm. I understand what you mean.
[speaker007:] Yeah.
[speaker005:] And then displayed.
[speaker001:] Right. Ps But for the demo maybe it doesn't matter. I'm not sure whether you wanna do the demo live anyway, or just screen shots of what we have. The problem with doing it live is it takes so long to load, that, um [disfmarker]
[speaker003:] I don't know.
[speaker008:] So this [disfmarker] the [disfmarker] this [disfmarker] uh, the sluggishness of the loading is all due to the parsing of the XML format.
[speaker001:] Um, w I was talking to Dave Gelbart about that
[speaker008:] Right?
[speaker001:] and apparently it's not actually the parsing of the XML raw [disfmarker] that going from the XML to an internal [pause] t tree structure is pretty fast.
[speaker008:] Hmm.
[speaker001:] But then it walks the tree to assemble its dat internal data structures and that's slow.
[speaker008:] Mmm.
[speaker002:] Seems like you should be able to spawn that off into a background process, because [vocalsound] not everything is displayed in that tree at once. Right?
[speaker001:] Uh, no. But what it does is it actually assembles all the user interface components then.
[speaker008:] But [disfmarker] d y
[speaker002:] Right.
[speaker001:] And then displays all the user interface components.
[speaker003:] U I'm [disfmarker] I'm confused.
[speaker002:] Seems like you wanna ass
[speaker003:] Uh, is [disfmarker] is this downloading something that happens once?
[speaker001:] Yes.
[speaker003:] And [disfmarker] and then when you d display different things, it's fine? So, in that case [disfmarker]
[speaker008:] No. Whenever you load a new [pause] meeting or a new [pause] transcript.
[speaker001:] A new transcript.
[speaker004:] Yep.
[speaker003:] Well, a new meeting, a transcript. Right.
[speaker008:] Right.
[speaker003:] But [disfmarker] but [disfmarker] i i for [disfmarker] but for [disfmarker]
[speaker001:] Or audio file. Well, actually the audio files are pretty fast, too.
[speaker003:] for presentation in, uh [disfmarker] Uh [disfmarker] I wouldn't be [disfmarker]
[speaker004:] Yeah.
[speaker005:] Yeah.
[speaker008:] You just have to have the thing running before you open your laptop.
[speaker005:] Yeah.
[speaker007:] Yeah.
[speaker001:] Right. The only problem with that is if anything goes wrong [pause] or if you wanna switch from one thing to another.
[speaker007:] Right.
[speaker005:] Yeah.
[speaker003:] Go wrong?
[speaker004:] Yeah.
[speaker003:] I see. Yeah.
[speaker007:] Right. I guess for the demo you can always play [disfmarker] just store the pieces that you're gonna display and play those as separate files, if we can't, you know, actually do it.
[speaker008:] Just make shorter files.
[speaker007:] But it's [disfmarker] you know, it [disfmarker] it's [disfmarker]
[speaker001:] That's true. We could just subset it.
[speaker007:] Yeah.
[speaker001:] That's a good idea.
[speaker008:] Yeah.
[speaker001:] That's actually probably the right thing to do.
[speaker007:] And just make it l
[speaker004:] Yep.
[speaker001:] You know, just take f ten minutes instead of an hour and a half.
[speaker004:] Yeah. Th that's what I did for [disfmarker] for my talk.
[speaker003:] Oh! Oh, you're downloading a whole meeting.
[speaker001:] Yeah.
[speaker003:] Oh, yeah that [@ @].
[speaker004:] Yeah.
[speaker001:] Yeah. So that [disfmarker] that's actually [pause] the [disfmarker] definitely the way to do it.
[speaker004:] Yeah.
[speaker001:] That's a good idea.
[speaker003:] Yeah. And then still do it ahead of time, but then at least you're covered if [disfmarker] if, uh [disfmarker] [vocalsound] if there's a problem.
[speaker001:] Yeah, if there are any problems.
[speaker007:] Right.
[speaker001:] Yeah, I mean, even five minutes is probably enough.
[speaker004:] Huh.
[speaker007:] Right. So, what happened to [disfmarker] Is it st possible at all to display the words in their aligned locations?
[speaker001:] That's what I did.
[speaker007:] OK. So. Sorry,
[speaker001:] Yeah.
[speaker008:] You missed that part.
[speaker007:] I missed [disfmarker] OK. Great! But it [disfmarker] OK. I couldn't get the words and the waveform at the same time for some reason, and there must be some [disfmarker] I'll [disfmarker] I'll work on it with Don and see what I'm doing wrong.
[speaker001:] Yeah. I mean, just ask [disfmarker] Just come by my office. I can show you as well.
[speaker007:] OK. Great! Well, thanks a lot. That's really great.
[speaker001:] Right. And for the information retrieval, uh, Don has been working on that. So.
[speaker005:] Yeah. So, it's coming along.
[speaker001:] It looks like it
[speaker005:] Um Just hacking [pause] Dan's code and m stepping through it. But, I think [pause] it's close.
[speaker001:] Great.
[speaker005:] And we should be there pretty soon, with at least, like [disfmarker] at least with, you know, being able to search over c certain amount of meetings, just, like, really basic stuff. Just asking fo looking for a word, and looking through a bunch of different meetings. And if we have time, I'll also add, you know, like, choosing which speakers you wanna include, and stuff. But [disfmarker]
[speaker003:] OK. Well, I'm gonna start working on this the week after next, so that's the point when I'll need to look more carefully at what y what [disfmarker] what you guys have.
[speaker005:] So, is the end of the month still [pause] the [disfmarker] the d?
[speaker003:] Right. The week after [disfmarker] th th The Monday the week after next is July second, which is the first day I get back.
[speaker005:] OK.
[speaker003:] So
[speaker005:] OK.
[speaker001:] Yeah. So I think for the L stuff Liz was talking about, we have something that'll work now. And Liz can look at it and see if she wants anything else. Maybe we can work on doing [disfmarker] displaying multiple [disfmarker] or displaying one and playing back the other.
[speaker007:] So do you think it's [pause] reasonable to display more than one [pause] before the demo? Cuz [disfmarker]
[speaker001:] Um, I think I'd h I'd have to ask Dave. I did it once before and it was just so slow to scroll, that I gave up.
[speaker007:] Huh.
[speaker001:] But, the advantage is that these things are much lower sampling rate.
[speaker007:] Right.
[speaker001:] And so then it might be alright.
[speaker007:] OK. Let me know.
[speaker005:] Morgan, when's the demo?
[speaker003:] Well, uh, I'm giving a talk on [vocalsound] July sixteenth. It's a Mon Monday in four weeks?
[speaker008:] So [disfmarker] If [disfmarker] if raw speed is the problem [pause] This thing is written in Tcl.
[speaker003:] Three weeks?
[speaker001:] Tcl.
[speaker008:] Right? Y I mean, John Osterhaut, uh [disfmarker] Y you know, he started his own company based on Tcl stuff, and maybe they have the native code compiler or something.
[speaker001:] I mean, we could check. I don't think they do. Um, there was actually a [pause] Java back end that apparently is actually a little faster. It generates byte code.
[speaker008:] Hmm.
[speaker001:] But, uh [disfmarker]
[speaker003:] It's always exciting to hear that Java's faster than something.
[speaker001:] Yeah. Well, e everything is faster than Tcl TK.
[speaker003:] Yeah.
[speaker001:] It's a string substitution language, basically. I should probably beep that out in case John Osterhaut ever listens.
[speaker006:] But Tcl is wonderful.
[speaker001:] But [disfmarker] Well, it is wonderful. It is, for prototyping and user interface. It's just really [disfmarker] the language is awful.
[speaker006:] There you go. Oh.
[speaker003:] Beep.
[speaker001:] Beep, y right.
[speaker006:] Yeah. [comment] [vocalsound] Yeah. I like it.
[speaker001:] But let me tell you how I really feel.
[speaker003:] We're all entitled to our opinions here.
[speaker001:] Yep.
[speaker003:] Um. Yeah. So it's [disfmarker] Yeah. I think it must be three and a half weeks.
[speaker006:] It's great.
[speaker003:] Uh, cuz July [disfmarker] [vocalsound] The [disfmarker] the meeting is July sixteenth through eighteenth. And, uh, my talk's the first day. So.
[speaker005:] OK.
[speaker003:] I'm flying out there the Sunday before. So. Um [disfmarker]
[speaker008:] Hmm.
[speaker003:] I guess, you know, it'd be desirable [pause] if, a week ahead of that, we basically had [disfmarker] thought we had it, which would allow a week for
[speaker001:] Oh, that's right.
[speaker005:] For realizing we don't?
[speaker003:] re iterating. Yeah. Pretty much. Yeah.
[speaker001:] Then the other issue related to that is data release. If we wanna show this in public, it should be releas So, I, uh, haven't gotten any other replies from the original email asking for approval. So I sent out another set this morning.
[speaker003:] I th I saw that.
[speaker001:] And, uh, we'll see if we get any responses.
[speaker007:] I just did it.
[speaker001:] Very good.
[speaker007:] But it is [disfmarker] I did wanna say that, um [disfmarker]
[speaker001:] Did you notice I put in the filter?
[speaker007:] No! No! No!
[speaker001:] Go ahead.
[speaker007:] I just figured you p
[speaker001:] There's a link there that now says if you want to search by [disfmarker] filter by a regular expression, you can.
[speaker007:] Oh, my gosh. OK.
[speaker001:] I put that in just for you.
[speaker007:] Terrific. Well, since you didn't answer the emai So there was a q question I had asked Adam whether it's possible to search only for your own name [disfmarker] your own utterances, so that you know you don't have to go through the whole meeting, and [disfmarker] [vocalsound] Um, and I didn't hear back. So I thought "OK. It's probably too hard. He's overloaded. I won't say anything. I'll just do it". Great. OK. So, anyway I looked at everybody else's stu
[speaker001:] It's actually an arbitrary [disfmarker] arbitrary regular expression.
[speaker007:] Good.
[speaker001:] But if you search your name, you'll get all of the things you said and any time anyone said your name.
[speaker007:] So th That's great.
[speaker001:] So.
[speaker003:] And it's [disfmarker] it's case insensitive?
[speaker001:] Correct.
[speaker003:] Yeah.
[speaker007:] That's great.
[speaker002:] Did you actually look through your [pause] transcripts? Or [disfmarker] you just approved them all.
[speaker007:] Uh [pause] Well
[speaker002:] I just approved all mine. I didn't look at them.
[speaker007:] I sort of spot checked. I was trying to remember [disfmarker]
[speaker008:] Oh, darn. I haven't done that yet. Er [disfmarker] OK.
[speaker007:] I couldn't find the [disfmarker] the keywords for things [vocalsound] that I thought I had said wrong. So.
[speaker001:] It's hard to find.
[speaker007:] That makes it g
[speaker002:] That's a compliment to you. He said it's hard to find things you say wrong.
[speaker001:] Hmm.
[speaker007:] It's hard to find anything that you say in these.
[speaker001:] Yep.
[speaker007:] Great. Well, thanks for the filter.
[speaker001:] You really do have to sort of r
[speaker007:] Uh, it's really useful, tha Cuz if you're only at part of a meeting, or something
[speaker003:] So, we have our first information retrieval example. It's a regular expression
[speaker001:] Yeah. That's right.
[speaker007:] Yeah!
[speaker003:] searcher.
[speaker007:] That's actually [disfmarker] Well, it's useful.
[speaker001:] And it demonstrates why it doesn't work, because you really wanna go acro more than one meeting.
[speaker003:] Yeah. Yeah.
[speaker001:] And you need a better user interface for displaying the results.
[speaker007:] But this helps a lot.
[speaker001:] So. Yeah.
[speaker003:] You wanna say, "Where are al where [disfmarker] Find all the contentious things I said."
[speaker007:] Great. Thanks.
[speaker001:] Yeah, really.
[speaker002:] Find everything that should be bleeped.
[speaker001:] That's right. Th we do have that bi nice marker [disfmarker] is that, n n because we all know we're being recorded, whenever anyone says anything like that, we then have a conversation about bleeping it out.
[speaker007:] Right. We Yeah. You can search for "beep" or "bleep".
[speaker001:] So. Yep.
[speaker003:] Yeah.
[speaker007:] In somebody else's turn.
[speaker001:] Um. Oh. And also we actually have a few people who have still not filled out speaker forms. Specifically in the NSA ones, and I noticed that when I tried to, uh [disfmarker] uh, generate the transcripts for NSA. That there are a few with no speaker forms. And so, uh, I have a [disfmarker] I sent out yet another this morning, which I think makes six total emails that I've sent to these people, and so I think we need to escalate to some other method of trying to contact them.
[speaker003:] Um [pause] [vocalsound] Right.
[speaker008:] Stalk them [@ @] at their [disfmarker]
[speaker003:] Has [disfmarker] has [disfmarker] has Joachim Sokol replied?
[speaker008:] Like in the morning, when I leave for work.
[speaker001:] Nope.
[speaker003:] Or [disfmarker]?
[speaker006:] I think, maybe talk to him first in person? That's what I would [pause] think.
[speaker003:] He's [pause] not around, is the only problem.
[speaker006:] Mm hmm. Oh, is that right? Oh. I saw him [disfmarker] I saw him on Tuesday.
[speaker003:] Yeah. Otherwise, it'd be a good idea. Yeah. He popped in. But, I mean, he's basically gone.
[speaker006:] Oh, OK.
[speaker008:] Cold calling at lunch time? Uh, dinner time, I mean.
[speaker001:] Well, if I could find phone numbers, that would certainly work.
[speaker006:] Well, did you ask Lila?
[speaker001:] But.
[speaker006:] Cuz I bet she has this information.
[speaker001:] Yeah, that's a good idea. I'll ask her if she can con track some of them down.
[speaker003:] Yeah.
[speaker006:] Yeah.
[speaker003:] Yeah. And [disfmarker] and tell her [disfmarker] You know. Tell her your specific problem. She'll fix [disfmarker]
[speaker006:] And it [disfmarker] M And, uh, then there's still m um [disfmarker] Miguel is still an active member of the group and he's [disfmarker] What I mean is, he's an active member and he's still here.
[speaker003:] He's right there. Yeah.
[speaker001:] Mm hmm.
[speaker006:] Very helpful.
[speaker001:] Yeah, I didn't actually see who they all were. Um, a couple of them were, like, people at IBM who were here for one of the IBM meetings
[speaker003:] Yeah.
[speaker001:] and one [disfmarker] a guy from SRI who was at one of the SRI meetings. And so, uh, those might be harder to track down.
[speaker006:] Most of them, though, really were visitors here and Lila should have contacts with them.
[speaker001:] Yep.
[speaker006:] Nice meetings, by the way. Yeah.
[speaker001:] I mean, they [disfmarker] they were people who didn't have accounts at ICSI, so they're [disfmarker] they're harder to find.
[speaker006:] Well, not the ones that [disfmarker] Are you su? Are you sure?
[speaker001:] Am I sure about what?
[speaker006:] Yeah. NSA one and NSA three?
[speaker001:] There were other people also.
[speaker006:] We're talking about those?
[speaker001:] There were other c other people also who didn't ha fill out the speaker forms, in addition to the N S
[speaker003:] I i in [disfmarker] in other meetings.
[speaker001:] Yeah.
[speaker003:] Yeah.
[speaker006:] Oh, oh. I see. OK. Fine.
[speaker003:] Well, SRI people, easy to f find.
[speaker007:] Yeah.
[speaker001:] Yeah.
[speaker003:] And, uh, IBM people, also. Just let us know. I mean [disfmarker] Just [disfmarker] we certainly have their email.
[speaker006:] But I knew everybody in the NSA meetings.
[speaker001:] Mm hmm.
[speaker006:] So th I'm sure that we have, uh, fresh, you know, information on them.
[speaker001:] Right. Yeah. None of the e emails bounced, so I know they're going somewhere.
[speaker006:] Good. OK.
[speaker003:] Right.
[speaker001:] That's all I have. You wanna talk about recognition?
[speaker008:] Chuck, you wanna talk about recognition?
[speaker002:] I haven't done anything.
[speaker001:] J T Liz, you wanna talk about recognition?
[speaker002:] I was [disfmarker]
[speaker001:] Thilo, you wanna talk about recognition?
[speaker002:] I was away for a couple of days. So. I haven't done a [disfmarker]
[speaker007:] We're sort of in a stage where we're [disfmarker] uh, Don's going through getting some of the next meetings that Jane s has. And, uh, you know, creating a second database. So we haven't actually run anything yet. We need to get a critical mass for that.
[speaker008:] However, I just got an email from Thilo saying that we are ready to run [disfmarker] I mean, we have segmentations for the old meetings that [pause] are from his segmenter,
[speaker007:] And check the segmentations.
[speaker005:] Mm hmm.
[speaker007:] Yeah.
[speaker008:] and so [disfmarker] You [disfmarker] you had three different versions, with different, like, pause thresholds [pause] between the segments?
[speaker007:] Great.
[speaker004:] Yep. Yep. Yeah. Just [disfmarker] Yeah. Just smooth the [disfmarker] the output of the [disfmarker] of the detector.
[speaker008:] Right. And you recommended using the one with two s maximum of two seconds?
[speaker004:] Yeah.
[speaker007:] A [disfmarker] What do you mean, a different pause threshold?
[speaker004:] Yep.
[speaker008:] But two s
[speaker004:] Or you can [disfmarker] Yeah. You can use the one with one second or whatever.
[speaker007:] Do you mean a [disfmarker]?
[speaker004:] I [disfmarker] I [disfmarker] I [disfmarker] There's no [disfmarker] not much difference between the [disfmarker] the one second and the two second one.
[speaker008:] Mm hmm. I mean, the only advantage to using the longer threshold would be that you run less risk of missing some [disfmarker] some speech.
[speaker001:] Backchannel.
[speaker008:] Right?
[speaker004:] And I think [disfmarker] wouldn't it be better to [disfmarker] to have a little longer sequences for the recognizer? B eh because of the language model? As sometimes it happens that [disfmarker] that it cuts off within a s
[speaker007:] Yeah.
[speaker008:] But two seconds is pretty long. So [disfmarker]
[speaker004:] Yeah. But, we can be sure that as [disfmarker] Or we can be [disfmarker] er not [disfmarker] not totally sure,
[speaker007:] No, it's not bad. That's good.
[speaker004:] but we can be somehow sure that there is nothing [disfmarker] not [disfmarker] no speech between those.
[speaker008:] Hmm. Hmm.
[speaker007:] i ye Yeah. th u
[speaker004:] So, it [disfmarker] it [disfmarker]
[speaker002:] What does the two second threshold mean?
[speaker004:] I think i it doesn't [disfmarker]
[speaker007:] I th I think that's g that's good.
[speaker004:] It's the same as in the [disfmarker] the smoother for the [disfmarker] [vocalsound] IBM thing.
[speaker001:] It combines them if it's [disfmarker] if the pause is longer than [disfmarker]
[speaker004:] Yeah.
[speaker007:] It's [disfmarker] it's no more than six words. So [disfmarker] Roughly. On average.
[speaker002:] Ah.
[speaker004:] So. Yeah.
[speaker008:] Hmm.
[speaker004:] So.
[speaker007:] That's pretty good, I think.
[speaker001:] Right. So the [disfmarker] the trade off is you get longer utterances, but you miss fewer utterances.
[speaker007:] It's better than a [disfmarker]
[speaker004:] But [disfmarker] Yeah. But [disfmarker] but the [disfmarker] the chunks are already sh i in general, are short. So [pause] I [disfmarker] I t I think it would be better to have [disfmarker] to have more of them concatenated together, in order to have better language model
[speaker008:] Mmm.
[speaker004:] or language modeling.
[speaker008:] I think two seconds [disfmarker] mmm
[speaker004:] I don't know.
[speaker008:] I would maybe go with one second. I don't know, it's a [disfmarker]
[speaker001:] Well, take a look [disfmarker]
[speaker004:] Yeah. Do that. Yeah.
[speaker008:] See what the length distribution is.
[speaker004:] Yeah. But [disfmarker] but y there's [disfmarker] there's really not much difference between the one second and the two second.
[speaker008:] Yeah. Really?
[speaker002:] I wouldn't think that the language model would continue across two seconds.
[speaker004:] So just [pause] take the one [disfmarker] the one second one.
[speaker008:] Uh [disfmarker] bu I'm [disfmarker] I'm [disfmarker] I'm just scared that with two seconds you get [disfmarker] you get, um [disfmarker] [vocalsound] you [disfmarker] you get false recognitions.
[speaker007:] Yeah. Well, yeah. Y you do, becau
[speaker008:] You're gonna [disfmarker] Yeah, you're gonna hurt yourself occasionally by having [disfmarker] missing the language model context. But you might hurt yourself more by having misrecognitions due to [pause] background speech, or, uh, y noise, or whatever.
[speaker004:] I [disfmarker] I'm not too afraid ab about that as w when there [disfmarker] when there would be something [disfmarker] some background speech or something [disfmarker] there wo there would be a [disfmarker] a chunk in another [vocalsound] channel,
[speaker008:] Oh, I see. Then [disfmarker] Oh, I see. OK.
[speaker007:] Yeah. I think it's better.
[speaker004:] and when there is s something in between, I con I [disfmarker] I do not concatenate them.
[speaker008:] Oh, right. Oh, that's [disfmarker]
[speaker007:] The longer is better.
[speaker004:] It's just when there is [disfmarker] when they are sequentially and [disfmarker]
[speaker007:] There's [disfmarker] there's [disfmarker] th
[speaker008:] Mm hmm. OK. Sure.
[speaker004:] So. I wou I would use th
[speaker008:] We can try them all and see which works better.
[speaker007:] There's a l there's a lot of these cases just like now, where I c people say "uh [disfmarker] uh" when they're trying to ta
[speaker008:] Mm hmm.
[speaker007:] and there's about a half second pause to a second in between and then another word,
[speaker004:] Yep.
[speaker008:] OK.
[speaker007:] and it's much better if we can keep those together, I think.
[speaker001:] Yeah. It's, uh, funny looking at some of the transcripts. I was filtering by [pause] person,
[speaker008:] Mm hmm.
[speaker001:] and in one of the [disfmarker] one of the early meetings, one pers particular person, almost the only thing they said the entire meeting was "yeah", "uh huh". It was just a whole list of them. It was very funny.
[speaker006:] I bet I know who that was.
[speaker008:] OK. So [disfmarker] so we need to split the waveforms, then.
[speaker006:] Yeah.
[speaker008:] Or do you already have them split up? No, you don't. Right?
[speaker004:] No.
[speaker008:] So [disfmarker] so, I guess Don [pause] would need your help to [disfmarker] to create a new set of split, uh, meetings.
[speaker005:] Sure.
[speaker007:] You know, you just fake the format that you take as input with the synch times to a new set of synch times,
[speaker008:] If [disfmarker]
[speaker004:] We could [disfmarker]
[speaker008:] Right.
[speaker004:] Mmm.
[speaker001:] Uh, do we know about disk?
[speaker007:] and [disfmarker]
[speaker008:] But th Oh, yeah. There's that pressure.
[speaker001:] Uh, Abbott disk?
[speaker003:] Uh. I know they're in.
[speaker001:] OK.
[speaker003:] And, uh [disfmarker] But I don't know. I think he was [disfmarker] Wasn't he asking about [disfmarker]?
[speaker002:] Wel He had a problem. Right?
[speaker001:] Yep.
[speaker004:] Yep.
[speaker007:] Oh.
[speaker003:] Well, there was an issue. He wanted to take it down. And then he tried [disfmarker]
[speaker002:] H he did, and then it didn't work,
[speaker001:] Couldn't format them.
[speaker002:] and
[speaker001:] No.
[speaker002:] I didn't hear anything after that.
[speaker001:] The only reason I'm asking is, you're gonna need space to split them up. And so I wanted to make sure we had some available for you.
[speaker005:] I still have, like, [vocalsound] probably [pause] six, seven, eight gig on my disk.
[speaker001:] OK. So we're OK [disfmarker] we're still OK for another couple days, then?
[speaker007:] And I have [disfmarker] I have like another un backed [disfmarker] I ha have another six gig, which Jeremy, if you're not using, can [disfmarker] on XA.
[speaker005:] Yeah.
[speaker003:] Wel
[speaker001:] So we're OK for [disfmarker] for a couple weeks, then.
[speaker003:] Markham probably needs u H he probably needs us to approve another time to take things down.
[speaker007:] To you.
[speaker003:] Right? In order to do that?
[speaker002:] Yeah. He [disfmarker] he didn't say [disfmarker] he hasn't said anything to me about it.
[speaker003:] I thought he [disfmarker] I thought he said in that mail that he would need to take it down another time. So I think he [disfmarker]
[speaker002:] Yeah. I think [disfmarker]
[speaker001:] Y yeah. He just didn't say when. So [disfmarker]
[speaker002:] Yeah. He [disfmarker]
[speaker003:] Well, no. I think he wanted u us to tell him when.
[speaker007:] How about during the picnic?
[speaker001:] Ah. Yeah. I'm sure he'd appreciate that.
[speaker003:] Yeah. I'm sure he'd love to [disfmarker]
[speaker001:] Well, m my feeling about that
[speaker003:] Fortunately, Markham's not a transcriber. But, um [disfmarker]
[speaker001:] is p is p
[speaker007:] Sorry. [comment] [vocalsound] Beep.
[speaker001:] Well, OK. That's the point. So it's Jane [pause] that we have to coordinate that through. Uh, what I was gonna say is, as soon as possible, and I'm willing to not work for an hour to get it done. Uh, but [disfmarker]
[speaker003:] "I might not work."
[speaker008:] Oops. [comment] I'm sorry.
[speaker004:] Oops!
[speaker005:] Whoa!
[speaker003:] "I'm [disfmarker] I'm willing to not work for an hour."
[speaker001:] Because when Abbott's [disfmarker]
[speaker003:] I know you're willing to not work for an hour.
[speaker001:] Yeah, right.
[speaker003:] But, I th
[speaker001:] Because when Abbott is down you can't work.
[speaker007:] You're really dedicated, if you're [disfmarker] No matter how you parse that one.
[speaker005:] Yea
[speaker008:] OK.
[speaker001:] But, uh, I think the per the people it disrupts the most are the transcribers.
[speaker006:] Well, you know, uh, I [disfmarker] All I need to do is mail, um [disfmarker] send them m a mail like two days in advance so they can schedule their time. I did that with the last outage. I j I wrote to them letting them know that this w
[speaker001:] OK.
[speaker008:] Hmm.
[speaker003:] I So, OK, it sounds like Markham should almost decide when he wants to do it, and tell us, as long as [disfmarker]
[speaker006:] was not, um
[speaker001:] So early next week. And just as long as we have a little warning.
[speaker003:] Yeah.
[speaker007:] So that means we can't, um, save meeting data either.
[speaker006:] Yeah.
[speaker007:] Right?
[speaker001:] Uh, just not during that time when it's down. But that [disfmarker] it should be down for an hour.
[speaker007:] So it just can't [disfmarker] so we can't have two meetings in a row where the first meeting's during that hour. That's all I meant.
[speaker001:] Right. Well, no, we can store them here.
[speaker007:] That [disfmarker] that's happening, like, today.
[speaker001:] We can store them here. You [disfmarker] we just run the risk [pause] that if you have a crash we lose the data.
[speaker007:] Temporarily.
[speaker001:] So.
[speaker002:] They're stored on Popcorn.
[speaker006:] This is on Popcorn? Or [disfmarker]?
[speaker007:] Well, no. I mea I mean, the fir Oh, jus OK.
[speaker001:] Yep.
[speaker006:] This is on Popcorn or something?
[speaker001:] Yep.
[speaker006:] Yeah? OK.
[speaker003:] We store [disfmarker] we store our data on Popcorn.
[speaker001:] Anything else?
[speaker003:] That's really great.
[speaker001:] Excuse me?
[speaker003:] I was just thinking we store our data on Popcorn. How many [disfmarker] how many institutes can you [vocalsound] say do that?
[speaker001:] Yep.
[speaker008:] Pop goes the data.
[speaker003:] OK. Uh, g megabytes and mega many megabytes, too. Um, what [disfmarker] what, uh [disfmarker]?
[speaker001:] We have a kernel on Popcorn, too.
[speaker008:] Right.
[speaker006:] That's very good.
[speaker003:] So uh, what [disfmarker] what's on your queue for [disfmarker] for recognition experiments?
[speaker008:] u un Can I have butter on my meeting?
[speaker003:] Let's talk about that for a second, maybe.
[speaker006:] What was the question?
[speaker002:] Uh [disfmarker]
[speaker003:] What [disfmarker] what was on his queue for recognition experiments?
[speaker006:] OK.
[speaker002:] Um, I'm rebuilding the net that we're gonna use for the tandem stuff. And so what I'm doing is, [vocalsound] um, putting in the stream reader into the Quicknet libraries for the SRI feature files.
[speaker001:] Mmm.
[speaker002:] An Which is the right way to do it. I mean, when we did our first experiments and I was, [vocalsound] uh, creating SRI feature files from the ICSI front end, I just had perl scripts, you know, and [vocalsound] hacked a bunch of stuff together just to get it going. But the r the right way to do it is to integrate it in with the ICSI tools. And so, that's what I'm doing now. And so once I get that done, then I'll generate the P files I need. Cuz we already have the feature files in the SRI format.
[speaker003:] Mm hmm.
[speaker002:] So what I need to do is, make it so that the [disfmarker] the Quicknet stuff can read those. And, uh [disfmarker]
[speaker008:] I is that independent or related to [pause] also being able to write out the, uh, feature file [pause] i in the SRI format f r
[speaker002:] It's both. There's a [disfmarker] there's an input stream and an output stream.
[speaker001:] Input reader and an output stream.
[speaker008:] Oh, OK. So then you could use, um [disfmarker] You could use, um [disfmarker] [vocalsound] [comment] uh, like Feacalc and s just specify as an output format
[speaker001:] Yeah.
[speaker002:] Yeah.
[speaker001:] Yeah. That's the point.
[speaker008:] the [disfmarker] the [disfmarker] Oh, OK. Th t I'm just ignorant about the [vocalsound] sof software architecture of this thing.
[speaker002:] Yeah.
[speaker001:] Yeah. So, if [disfmarker] if you [disfmarker] Right. Quicknet is a very nice stream based library,
[speaker008:] Uh huh.
[speaker001:] so without too much effort, once he has the classes written we can incorporate it into all the standard tools.
[speaker008:] Oh, cool.
[speaker001:] So.
[speaker008:] Great.
[speaker002:] So, then it's t uh, tandem experiments after that?
[speaker003:] Uh huh.
[speaker001:] And at some point, I'd like to get back to, uh, porting Quicknet to the multiprocessor Linux box.
[speaker003:] Yeah.
[speaker001:] You know, I [disfmarker] I have forward passes working, but I haven't done training yet.
[speaker008:] So, b speaking of Linux. So [disfmarker] [vocalsound] Th there's some i impetus at, um, [vocalsound] SRI to actually [vocalsound] u u p th [comment] uh, build [disfmarker] support Linux as a platform. So.
[speaker001:] Mm hmm.
[speaker008:] What that means is, [vocalsound] once we have, uh, everything running on Linux we can also use a Li [vocalsound] eh
[speaker007:] n Run all our jobs on your machines.
[speaker008:] Yeah. Exac
[speaker002:] We don't have too many. We just have that [disfmarker] just have a few Linux machin
[speaker008:] I mean, if you can't use all the processors on whatever machine, we'll help you with that.
[speaker001:] Yeah. That's right. Well, that's the nice thing about it, is that [disfmarker] i since it's coarse parallelism you don't have to do anything special.
[speaker008:] Right. Exactly.
[speaker001:] So. I mean that would be a fine use for b for that machine.
[speaker008:] Yeah. So it's just, uh [disfmarker]
[speaker001:] Five more processors.
[speaker003:] Oh, I know what it was [disfmarker]
[speaker008:] Uh. Yeah.
[speaker003:] Um. Yeah. U um [disfmarker] Some [disfmarker]
[speaker008:] Or if [disfmarker] uh, you know, in the future, if Linux machines become like way cheaper, than, [vocalsound] uh, you know, Solaris machines, then [pause] you know, that wouldn't be a reason not to [pause] use Linux anymore.
[speaker001:] Yep.
[speaker008:] So.
[speaker003:] Yeah. Um, I think it would be ne neat at some point [pause] in this [pause] to do, um, a recognition, uh, pass on one of the PZM mikes for these s same meetings that you've been gi n gi bi
[speaker008:] For the meeting?
[speaker007:] Yeah.
[speaker003:] I mean, it's gonna be terrible,
[speaker008:] Mm hmm.
[speaker003:] but, you know, we [disfmarker] we just don't know how terrible.
[speaker007:] Ye It's also an interesting problem to come up with the reference.
[speaker008:] Mm hmm.
[speaker003:] And [disfmarker]
[speaker007:] So, the reference file for the relative times at which [disfmarker]
[speaker001:] Oh, yeah. That's really hard.
[speaker007:] So [disfmarker] Well, i it's an interesting question, because I was thinking well you can force align the transcriber transcripts, and then of course, you try to merge them in time.
[speaker001:] It's not determined.
[speaker003:] Mm hmm.
[speaker007:] But how do you score? "
[speaker001:] I think the first pass is [pause] throw out words which are overlapped.
[speaker007:] I mean it's just an interesting problem.
[speaker001:] That would be a good first pass. Just ignore everything that has any overlap.
[speaker008:] Mmm.
[speaker007:] Right. I mean, there's a whole sort of dis
[speaker003:] Yeah. Cuz you have a set of scores about that,
[speaker007:] Right. We [disfmarker] Right.
[speaker003:] so maybe then, that wouldn't be so bad.
[speaker007:] But there's a whole interesting discussion, cuz of course the alignments are not perfect either,
[speaker003:] Oh.
[speaker007:] and so [disfmarker] um. In fact, we actually don't have a [disfmarker] a p
[speaker001:] Right.
[speaker003:] Yeah.
[speaker007:] I mean, it it's worth trying.
[speaker003:] But [disfmarker] that'd be a hell of a lot better than what we do with just these.
[speaker007:] I [disfmarker]
[speaker003:] And [disfmarker]
[speaker007:] Right. Right.
[speaker003:] and [disfmarker] and, again, if you rule out the overlap, you have some numbers for that, cuz that's yet another [disfmarker] But, I [disfmarker] I'm just concerned, of course, about that [disfmarker]
[speaker007:] Oh, I see. You mean when only one person is [pause] talki
[speaker001:] Yep.
[speaker007:] Yes. We could [disfmarker] we could try tha
[speaker003:] Cuz you have scores for that for the other case.
[speaker007:] Right. Right. Exactly.
[speaker008:] Mm hmm.
[speaker003:] And [disfmarker]
[speaker007:] Yeah. We [disfmarker] we should try [disfmarker]
[speaker003:] We just don't know how bad it will be. I mean, one of the things that Dave was noticing [disfmarker] we were talking this morning [disfmarker] is that i it seems like [disfmarker] and we do don't know this in detail, but it seems like you're getting a lot from the channel adaptation, the speaker adaptation, and so forth.
[speaker005:] Hmm.
[speaker008:] Mmm. Mm hmm.
[speaker003:] Um. So you are, already, in that recognizer, doing something that is likely to affect, uh, the [disfmarker] the far field microphone, uh, formant.
[speaker008:] Mmm. Right.
[speaker003:] So, it may not [disfmarker] I mean, it's gonna be bad,
[speaker008:] Right.
[speaker003:] but, uh, it may not be, like, "won't decode" kind of bad.
[speaker008:] Right.
[speaker003:] It might [disfmarker] might only be that it goes from forty percent to eighty, or something like that.
[speaker001:] Yeah, eighty percent. Yeah.
[speaker008:] Right.
[speaker007:] Do you assume you know the speaker when you do this?
[speaker003:] I want us to assume the exac whatever it was you assumed when you did the other [disfmarker] the [disfmarker] the close mike.
[speaker008:] You just di you just [disfmarker]
[speaker007:] Well, th r Well, there, there's only one person who it can be, because they own that microphone.
[speaker008:] Right.
[speaker007:] I'm just wondering [disfmarker] There's [disfmarker]
[speaker008:] Yeah. That [disfmarker] that becomes another problem, actually.
[speaker005:] Well, and then you have the gender detection
[speaker007:] No It's not a pro
[speaker004:] Yeah.
[speaker001:] Well, but for a [disfmarker] But, for s for scoring, you can do it or not do it as you choose.
[speaker007:] Right. And [disfmarker]
[speaker003:] Oh, I'm [disfmarker]
[speaker007:] R right. But in terms of [disfmarker] for norm
[speaker008:] n s
[speaker001:] So.
[speaker003:] But you're saying for this [disfmarker] For the adaptation, you mean.
[speaker008:] So, d s so [disfmarker] fo not [disfmarker] well, for everything.
[speaker007:] for adaptation. Right.
[speaker008:] For [disfmarker] s for [disfmarker] f even feature normalization, for, uh, vocal tract length estimation,
[speaker007:] You know, do you do a supervised adaptation [disfmarker]?
[speaker003:] Right.
[speaker007:] Ye All of these adaptations ha
[speaker008:] all of [disfmarker] all of these assume you know who's speaking.
[speaker007:] assume that the same person [disfmarker]
[speaker008:] So,
[speaker004:] Mmm.
[speaker008:] you would have to do a speaker segmentation first on the far field micropho signal.
[speaker007:] Yeah.
[speaker001:] N well, but you can use the [disfmarker] when you're doing the scoring [disfmarker] Since you're [disfmarker] you're gonna be scoring against transcript, you can use [disfmarker]
[speaker008:] You mean you wanna cheat.
[speaker001:] Well, you're doing that anyway.
[speaker007:] N Well. I was just aski
[speaker008:] No. If [disfmarker] i i
[speaker006:] Ooo. I don't like that term.
[speaker001:] So ch chea try to cheat in the same way that you're doing with the close talking.
[speaker006:] I don't like that term.
[speaker003:] I [disfmarker]
[speaker007:] Actually, the s Those are two [disfmarker]
[speaker006:] I don't like that term.
[speaker003:] I have [disfmarker]
[speaker008:] OK. We g we're gonna bleep that out.
[speaker003:] I [disfmarker] I have a suggestion. Do the simplest thing first.
[speaker001:] Yeah, right.
[speaker003:] Because we're gonna want to know that anyway.
[speaker007:] So, wait, the simplest thing is you cheat, saying [disfmarker]
[speaker003:] In other words, if you d No. It's [disfmarker] No. It's even simpler thing than that, is just that you don't know.
[speaker008:] You mean you do you don't do all those normalizations.
[speaker001:] Yeah.
[speaker003:] Yeah.
[speaker007:] Oh, you m y totally unadapted.
[speaker001:] Just do [disfmarker] or a free ri Yeah.
[speaker003:] Yeah. Because you can get a number [disfmarker] uh, for that with the other as well.
[speaker007:] Uh. Right.
[speaker003:] Right? You can turn those things off. Right?
[speaker008:] Um, actually [disfmarker] [vocalsound] We don't have any models.
[speaker007:] Yeah. i i
[speaker008:] Um, you can [disfmarker]
[speaker007:] No. You can [disfmarker]
[speaker008:] Um [disfmarker]
[speaker007:] You can use a speaker [disfmarker] eh [disfmarker] What about gender detection?
[speaker008:] A actually, it's that [disfmarker] it's [disfmarker] it's [disfmarker] We would have to retrain models that are not [disfmarker] that have none of that stuff, uh, in it. But actually we could [disfmarker] We can just run it, assuming that it's all one speaker, basically. And see what happens.
[speaker003:] Yeah. Yeah. Yeah. And then put it in correctly and see how much that helps.
[speaker007:] Yeah.
[speaker008:] Yeah.
[speaker003:] I mean, I was just thinking, do the one that's easiest first, because you wanna know [pause] how much that's helping you in these cases anyhow.
[speaker008:] Actually [disfmarker]
[speaker004:] But d do you have gender dependent models?
[speaker008:] Actually, no. Th th Sorry.
[speaker004:] A Are the models gender dependent?
[speaker005:] Mmm.
[speaker008:] Yeah.
[speaker001:] Yeah. They're all gender dependent.
[speaker007:] Yeah.
[speaker004:] So
[speaker007:] Yeah. So. You can r
[speaker001:] So we would have to at least do that.
[speaker008:] No, actually [disfmarker]
[speaker007:] No. You can run both.
[speaker004:] Yeah.
[speaker007:] And you can s
[speaker004:] Yeah.
[speaker008:] No, actually, what [disfmarker] Here's [disfmarker] here's what we would usually do u under these circumstances.
[speaker001:] And pick whichever's better.
[speaker008:] We would actually [disfmarker] we would run some sort of segmentation. Thilo's is as [pause] good as any, probably. Um, and then we would do an unsupervised clustering of [disfmarker] of the segments, to [disfmarker] and [disfmarker] and put the similar ones into bins that would be sort of pseudo speakers.
[speaker003:] Mm hmm.
[speaker008:] And then we would do our standard processing on these pseudo speakers. And that turns out to work very well on Broadcast News, SPINE [disfmarker] those types of tasks, where you don't have the speaker segmentation given to you.
[speaker001:] S does the clustering [disfmarker]? Do you give it sort of a target number of clusters? Or is it [disfmarker]? [comment] adapted in some way?
[speaker008:] Um, you can either do it by target number or by some measure of dissimilarity that you use as a threshold.
[speaker001:] Mm hmm. That's what I'm just thinking one of the big differences with Broadcast News and these meetings is we have m many fewer participants.
[speaker007:] The other thing is that you actually have direction here.
[speaker008:] Right.
[speaker007:] So, unlike these corpora that are recorded with other microphones, like [disfmarker] The right way to do this I guess, w you know, in the future would be, well, in general, Thilo's sitting there, and this PZM is gonna [disfmarker] You know, he's roughly [pause] in that location.
[speaker001:] Speaker ID. Yeah. Well, there're different ways of thinking about it. I mean that [disfmarker] that would be true if w you had a meeting situation with multiple mikes. But if you only had your PDA sitting in front of you [disfmarker]
[speaker007:] Well, any case where the people are not all sitting at the same place and they're not moving around too much.
[speaker001:] And you have more than one mike.
[speaker003:] Yeah. If you don't have one [disfmarker] more than one mike, you don't have a very good handle on location.
[speaker007:] W well, you have distance, and you have [disfmarker]
[speaker003:] That's [disfmarker] it's not enough.
[speaker007:] I mean, that, um, Jane's [disfmarker] i n The pickup of Adam on this mike is gonna be different than me, in terms of energy and so forth over the whole meeting.
[speaker001:] Oh, so just from clustering. You might be able to cluster it better because of that.
[speaker007:] You might get some clustering from the speaker and some of it from the characteristics of the distance,
[speaker003:] Mmm.
[speaker001:] From mike. Yeah.
[speaker003:] Yeah.
[speaker007:] and [disfmarker]
[speaker003:] But, say, if [disfmarker] if you had a [disfmarker] a cardioid mike or something sitting someplace, then [disfmarker] [vocalsound] sitting there, then it would [disfmarker] Its [disfmarker] its response to him would be about the same as the response to him, and so on.
[speaker001:] And transfer functions.
[speaker007:] Right. Exactly.
[speaker008:] Well, you can do [disfmarker] [vocalsound] You can [disfmarker] you can [disfmarker] you can do certain normalizations like, you know, gain control,
[speaker001:] Well, I think there're lots of [disfmarker] lots of ways of doing it.
[speaker007:] They're both picked up in the clustering, I guess.
[speaker003:] Mm hmm.
[speaker008:] uh, before you do the clustering to rule out those [disfmarker] those types of things.
[speaker007:] Or to just do the clustering, knowing that you're capturing both. It's just that the kind of clustering we've done before hasn't had that, uh, distance factor, or location factor in it in the same way.
[speaker003:] I see. Yeah.
[speaker007:] And so we're not really modeling it directly. If somebody does, we sh maybe we add that
[speaker008:] Mmm.
[speaker001:] Yeah. That's an interesting [disfmarker]
[speaker007:] because I think it would be a pretty big difference. When you listen, you can sort of tell where people are,
[speaker001:] It's a big difference. Yeah.
[speaker008:] OK. That would be fun [disfmarker] fun to try.
[speaker007:] not which s you know, side, but [disfmarker]
[speaker001:] Well, humans are really good at that [disfmarker] transfer function through the head, and things like that.
[speaker007:] Right. Even with one microphone.
[speaker001:] So you know, even if you only have one ear, you can still get [disfmarker] get good transfers.
[speaker003:] Yeah. Yeah.
[speaker007:] Right. So our [disfmarker] our clustering is not gonna be intelligent that way.
[speaker001:] So
[speaker007:] It's just gonna pick up whatever energy difference, or whatever, is [disfmarker]
[speaker001:] Yeah.
[speaker003:] But [disfmarker] Anyway, I [disfmarker] I'd [disfmarker] it'd be neat to have that, because you know we've been at this for a little while and we don't have [disfmarker] have [disfmarker] any results yet with [disfmarker] with conversational speech at a distance.
[speaker008:] Mm hmm. OK.
[speaker003:] So, um [disfmarker]
[speaker008:] Hmm.
[speaker001:] Yeah.
[speaker003:] We should at least get [pause] a first one.
[speaker001:] Something. Yeah.
[speaker007:] OK.
[speaker006:] Mm hmm.
[speaker003:] Um, and the other thing [disfmarker] This would kind of be a Hail Mary, but [disfmarker] but, uh [disfmarker] [vocalsound] Uh, Dave does have this stuff that is helping on digits, and, you know, and so with [disfmarker] then it'd be, you know, just throw that in and see.
[speaker008:] Oh, yeah. Then we should [disfmarker]
[speaker001:] Yeah. So it'd be cool to see if it helped.
[speaker008:] Well, first you have to filter the whole training set and retrain.
[speaker007:] Yeah.
[speaker005:] Mm hmm.
[speaker003:] Yeah.
[speaker001:] That would be [disfmarker] ft quick,
[speaker007:] Mm hmm.
[speaker001:] since I think he did it in Matlab.
[speaker003:] Uh, well, he can do it in something else.
[speaker006:] Hmm. Interesting.
[speaker001:] Yep.
[speaker003:] But, I mean [disfmarker] [vocalsound] you know, it's [disfmarker]
[speaker001:] Can't you export C from Matlab,
[speaker003:] Um [disfmarker] Actually, we're experimenting with phase stuff now,
[speaker001:] or is that Mathematica?
[speaker003:] and [disfmarker] and, uh, thi this, uh [disfmarker] first result he got, uh, was really great. It actually, uh [disfmarker] uh [disfmarker] [vocalsound] didn't exactly eliminate the reverberation, but it completely got rid of the speech.
[speaker008:] Oh. That would solve all of our problems.
[speaker006:] Wow.
[speaker003:] Yeah. Well, I was thinking that.
[speaker008:] Wouldn't it?
[speaker001:] Well, so just take the inverse and you're fine.
[speaker003:] So, it's [disfmarker] That's [disfmarker] You think I didn't s [vocalsound] tell him that? No. I got pretty excited, because it completely got rid of the speech. So, I was thinking well, so,
[speaker001:] So that's a speech detector. That's great.
[speaker006:] That's interesting.
[speaker003:] you know, it could be useful for lots of things ". So, we have to [disfmarker]
[speaker006:] Did it get rid of other stuff, too, though?
[speaker008:] Hmm.
[speaker003:] What?
[speaker006:] Did it get rid of other stuff besides the speech?
[speaker001:] Yeah. Just subtract that [disfmarker]
[speaker003:] Well, we have to sort of check that out.
[speaker001:] subtract that from the original signal and you're set.
[speaker003:] That's [disfmarker]
[speaker006:] OK. Wow. Interesting.
[speaker003:] Yeah. Yeah.
[speaker008:] Right. Then you can do like an [disfmarker] you can estimate the [disfmarker] the noise estimates.
[speaker001:] Noise estimate. Signal to noise.
[speaker008:] Right? Yeah.
[speaker001:] That's great.
[speaker007:] Mm hmm.
[speaker003:] Reminds me of when th when Herve and I were first playing with c uh, context dependent things for nets, and [disfmarker] and at one point we took out the speech input, so we [vocalsound] only had priors, and our performance went up.
[speaker008:] Mmm
[speaker006:] Wow.
[speaker001:] I guess that's why Herve always talks about using the priors as one of the mixtures in [disfmarker] in his all ways combos.
[speaker008:] Hmm.
[speaker003:] S Yeah. Well, of course it was a bug. But, I mean, it's [disfmarker] it's a [disfmarker] but it was pretty f f [vocalsound] It was pretty funny anyway.
[speaker001:] But still [disfmarker]
[speaker006:] Wow. Interesting.
[speaker008:] Yeah.
[speaker006:] Hmm.
[speaker001:] So if you run [disfmarker] er, your recognizer with all probabilities equal, what do you get out? Probably garbage.
[speaker002:] Whatever the language model says.
[speaker001:] I bet the pruning [disfmarker]
[speaker004:] Yep.
[speaker001:] The pruning probably prunes everything out.
[speaker008:] You get out Switchboard. That's just the lang the language model.
[speaker001:] Yeah. That's right.
[speaker003:] So that's how it was generated.
[speaker008:] Ha Yeah. So we have this new speaker adaptation. Um A [disfmarker] b Oh, it's a s sort of feature normalization t uh, like f speaker adaptation, which, uh, which I wr which I [pause] wrote about in the last status report, which seems to be helping about a percent and a half on Hub five. So, um. We haven't tried that yet on the meetings, uh, but hopefully it'll help there, too.
[speaker006:] I wanna ask, um [disfmarker] So, you know that the data [disfmarker] I' I have upgraded it considerably. So I've probably made [disfmarker] I probably corrected something like, well, s It's really a substantial amount of things that I've caught, changed [disfmarker] um, added to it. Including a lot of, uh, backchannels. So, when you're d running things, if you run it on the old [disfmarker] So. If you [disfmarker] if you [disfmarker] run it on the new version, then the numbers will be [disfmarker] um, and you compare it to the [disfmarker] to runs on the old version, then you're gonna end up with more of an improvement than would actually be the case.
[speaker008:] Hmm.
[speaker001:] Different.
[speaker008:] Mm hmm. Well, we [disfmarker] we have a frozen [disfmarker] we do all our experiments with a frozen version of the transcripts as of, I don't know
[speaker006:] As of February?
[speaker008:] A a as of [disfmarker] no, a little [disfmarker] I don't know. When [disfmarker] when did we grab the transcripts?
[speaker006:] So a l Like the HLT paper?
[speaker005:] The b for the f? It was m m March, probably.
[speaker007:] F Sorry.
[speaker008:] The [disfmarker]
[speaker007:] For these meetings?
[speaker008:] We're talking about which [disfmarker] which version we're using for evaluating the recognition. Which version of the transcripts.
[speaker007:] Right. They're somewhere in between January and late March, or something like that?
[speaker005:] Yeah. Something around there.
[speaker006:] OK. Well, so long as you have the same baseline, then [pause] you'll be able to tell.
[speaker001:] But they are channelized ones, though?
[speaker008:] No, no. Yeah. O obviously. Yeah.
[speaker007:] N N They're meetings that are now channelized, but they were not from [disfmarker]
[speaker006:] So long as i so long as it's the same baseline you'll be able to tell.
[speaker008:] The [disfmarker] uh, the other thing is [disfmarker] Th the [disfmarker] the [disfmarker]
[speaker006:] But [disfmarker] but I'm just gonn I'm just saying that if you were to compare that with running that on the new data, that it would be an [disfmarker] a more optimistic outcome.
[speaker008:] Hmm. And [disfmarker] Well, the [disfmarker] and the other thing is, it takes only a [disfmarker] a minute to rescore all the old outputs with [disfmarker] If you had new transcripts, then we j we just re rescore the old [disfmarker]
[speaker001:] Right. Cuz you haven't done any training.
[speaker008:] Sorry?
[speaker001:] Right. Cuz we're not doing it for training.
[speaker007:] Yeah. We haven't a modified the recognizer at all.
[speaker001:] Right. So it's really [disfmarker] it would be really easy to re do it.
[speaker007:] So, actually, um, at some point we should update and rescore everything with, you know, the corrected transcripts,
[speaker008:] W we just [disfmarker] We save [disfmarker]
[speaker001:] Yeah. It'd be interesting just to see i how much it changes.
[speaker008:] Well, the [disfmarker] the [disfmarker]
[speaker007:] just to h
[speaker001:] I bet it wouldn't change a lot.
[speaker008:] Right.
[speaker005:] What are the nature of most of the changes?
[speaker006:] Yeah.
[speaker001:] We can take [disfmarker] we can have a pool.
[speaker007:] Sometimes the changes are, um, cases where the recognizer would get it wrong anyway, cuz it was some word that we didn't have in the vocabulary,
[speaker001:] That's just what I was thinking.
[speaker008:] So, I [disfmarker] I [disfmarker] The thing is, when [disfmarker]
[speaker007:] or [disfmarker] But it does help to get the backchannels back in, and things like that.
[speaker008:] Right. So, whenever the [disfmarker] Right now, the s the scoring is based on segments. Um, which is not great because, for instance [disfmarker] So [disfmarker] so, the [disfmarker] the [nonvocalsound] other way to do the scoring is using a [disfmarker] a NIST format called STM. S uh, Segment Time Marked,
[speaker001:] Yep. I [disfmarker] we know about it.
[speaker008:] where [disfmarker] So, um, I have to convert the, uh, transcripts into this format, and then the scoring program actually looks at the times. And, uh, you know, it [disfmarker] You can have a different segmentation in your recognizer output than in your references, and it will still do the right thing. So, that's what we need to basically, uh, to [disfmarker]
[speaker001:] Tran
[speaker008:] Hmm?
[speaker001:] Transcriber will export STM.
[speaker008:] Oh.
[speaker001:] In case you care.
[speaker008:] Well. But then there's other changes. So. I mean, there's other [disfmarker]
[speaker001:] OK.
[speaker008:] We [disfmarker] we strip away a lot of the mark up, uh, in the transcripts, which, you know, isn't relevant to the scoring of the speech recognition output.
[speaker001:] Right.
[speaker008:] So
[speaker007:] Would that [disfmarker]? Doesn't that only change the scoring if a word has moved into a different segment? I mean, I don't think that's [disfmarker] har I hardly ever see that. I think most of them are pretty good.
[speaker008:] No. I mean if [disfmarker] suppo I [disfmarker] I assume you also changed some boundaries.
[speaker006:] Mmm. I did.
[speaker008:] Right? So, if we want to use new transcripts with a different segmentation, then we can't use them in the current way we do scoring. We have to [disfmarker] m m switch to this o
[speaker007:] Oh, you mean you would need to re run the recognition.
[speaker008:] No. We have to r There's a difference [disfmarker]
[speaker007:] I mean, if you re ran the recognition, then you just run it [disfmarker]
[speaker008:] Right. Right.
[speaker007:] Oh, I see. If you wanna use the old [disfmarker] You can actually never, though, really infer what you would get with a different [disfmarker]
[speaker008:] That's true.
[speaker007:] You know? It's [disfmarker] it's probably more fair to re run the whole th re run that. Uh, in other words, it's not really a scoring script problem. It's [disfmarker]
[speaker008:] No. But if you w just want to see what [disfmarker] Like, suppose you fixed some typ No, Jane fixed some typos and you wanna see what effect does that have on the word error, then we can f we c
[speaker001:] Right. But [disfmarker] but [disfmarker] If the segments change, that won't work.
[speaker007:] Y Yeah. Yeah. You [disfmarker] It'll sort of work, but it's not exactly what you would maybe get from recognition.
[speaker001:] Well, I mean, what happens if you break one segment into two?
[speaker007:] You never know.
[speaker001:] Suddenly the don they don't match at all and you can't line them up anymore.
[speaker008:] No, but that's what the pr Well, that's what I'm saying.
[speaker007:] Well, you c You [disfmarker] you s
[speaker008:] You can line them up based on the times.
[speaker007:] You can [disfmarker] Yeah.
[speaker008:] So the scoring program with [disfmarker] if you give it an STM reference file, it will actually compare the words based on their time marks.
[speaker001:] Right.
[speaker007:] Yeah.
[speaker008:] So therefore, you can, um [disfmarker]
[speaker001:] Does STM do it per word, or por per utterance?
[speaker008:] It's per utterance, but it [disfmarker] it allows [disfmarker] As long as you hypothesize the word in the right segment in the reference, it gives you credit for that.
[speaker001:] Th what I thought. Alright.
[speaker008:] So it does a [disfmarker] [vocalsound] it does a word alignment, like you have to do for scoring,
[speaker007:] Mm hmm.
[speaker001:] And then it does also some [disfmarker]
[speaker008:] but it constrains the words to lie within the t the time bins of the reference.
[speaker001:] Within the segment. Yep. I see.
[speaker008:] And [disfmarker] fo for you to get a credit for it. So.
[speaker006:] That sounds great.
[speaker008:] So [disfmarker] so, it's [disfmarker] it should be just a straightforward re formatting issue of the references. So.
[speaker006:] Yeah. I mean, t y I was thinking, th the other day, that this [vocalsound] is not just conversational speech. The fact that is has so much technological jargon in it. I it makes it [disfmarker] u considerably harder to, um [disfmarker] me e e e to transcribe and to [disfmarker] to double check and all those things. So, [vocalsound] I think you're gonna find a substantial gain in terms of the word ac accuracy. So long as those words are in your vocabulary.
[speaker007:] Well, it definitely helps with, um, forced alignment, too,
[speaker001:] One percent.
[speaker007:] because, you know, whe when we know the [disfmarker] the [disfmarker] the true words and we're adding them to the vocabulary and training a language model and so forth for future meetings, especially the [disfmarker] like, the front end meetings or meetings with a lot of [pause] jargon in them, that is not represented in Switchboard or CallHome, then it [disfmarker] then it really helps.
[speaker006:] Well, all of them are. All of our [disfmarker] all of our meetings have a lot of jargon in them. But we know what those words are.
[speaker007:] So Yeah.
[speaker006:] I mean, I don't know how you get them into the vocabulary, but it would seem [@ @] [disfmarker] Now, I have to [disfmarker] I have to [vocalsound] [disfmarker] I really need to, uh, raise a question about the term "cheating". OK? And the reason is, um, if I understand that, "cheating" is a term which is used to apply for basically what all of linguistics [disfmarker] corpus linguistics does, and what [disfmarker] what my transcribers are doing and what I do, [vocalsound] which is a methodology whereby you actually physically mark things in the data.
[speaker001:] All [disfmarker] al
[speaker006:] Like the transcription, like the words that are actually there.
[speaker001:] All we mean by that is that we're giving the recognizer more information than it would have if you were running it raw, over a meeting that no person has ever listened to or transcribed.
[speaker006:] OK. I mean, that [disfmarker] that part is OK. But I [disfmarker] but I do wonder sometimes if it might be possible to use [vocalsound] a term that's a little bit less evaluative, like, um, "hand marked".
[speaker003:] Uh, but cheating is [disfmarker] is [disfmarker] p Sure. But chea but cheating is [disfmarker] is [disfmarker] is [disfmarker] pretty commonly used to mean this.
[speaker008:] It's a [disfmarker] it's a technical term.
[speaker007:] It [disfmarker] I guess it [disfmarker] It's been around for a long time,
[speaker003:] It really is [disfmarker]
[speaker001:] It is. Yep.
[speaker007:] and it [disfmarker] it's sort of, um [disfmarker] Because it's so strong a word, people don't take it that seriously,
[speaker003:] Yeah. Yeah. I i i i
[speaker007:] in [disfmarker] It's not negatively viewed. It just really means [disfmarker]
[speaker003:] I it's [disfmarker] i It's even more than that. I think it [disfmarker] it really gives a very strong perspective, that you know that what you were doing is not an un biased experiment. If you don't say that, then people think "oh, they did that and they threw that, but that doesn't represent what would happen in the real world". But if you say "we did a n cheating experiment", which is really the standard way you'd say it, it says you deliberately put in a piece of me information that you would not have in the real world so that you can learn something. Just as part [disfmarker] as your process. So it's [disfmarker] I d I don't know, I [disfmarker]
[speaker006:] OK. I guess what I'm [disfmarker] I'm thinking, is just in [disfmarker] if [disfmarker] uh, when these are presented in an inte interdisciplinary context, it might be nice to add that explanation?
[speaker003:] i
[speaker006:] Cuz otherwise it sounds like a pejorative statement on an alternative methodology? Cuz it sounds like [disfmarker] it sort of devalues the [disfmarker] the, um, all [disfmarker] other approach, which is to [disfmarker] to put those distinctions in.
[speaker003:] Maybe. I [disfmarker] I've heard this at ICSLP for years and years and years, though. I mean, it's [disfmarker] it's pretty [disfmarker] pretty common. People say this.
[speaker008:] If [disfmarker] i Well, you i i in a [disfmarker] To a broader audience you could call it a diagnostic experiment, rather than a cheating experiment.
[speaker007:] Actually, Jane is right. Like, in conversation analysis, I've never heard people use this, because they're not using an automatic system. So, it really [disfmarker]
[speaker008:] But they don't run experiments.
[speaker003:] Yeah.
[speaker007:] Right. I'm j Well,
[speaker006:] They can do experiments.
[speaker007:] you can do experiments, but the [disfmarker] it's cheating relative to what we call a system where we can completely control this black box. And it's not a very smart system. It [disfmarker] it only knows what we give it. And if w If it knows more than we would really give it when it runs, we call it cheating. But it's [disfmarker] Yeah, it's f only used in a community that does some type of computational modeling,
[speaker003:] Yeah. In pattern recognition, or in [disfmarker]
[speaker007:] I think. It's really not used in any kind of community doing experiments on human perception,
[speaker003:] Yeah.
[speaker007:] or [disfmarker]
[speaker003:] Right.
[speaker007:] You know. So [pause] she certainly can define it.
[speaker003:] Yeah. It's machine [disfmarker] machine experiments. Um, [vocalsound] I mean, w when [disfmarker] when the, uh, the neural net w uh [disfmarker] wave hit in the mid eighties, and by the late eighties there were [disfmarker] uh, we were reviewing thousands of papers that were [vocalsound] coming out in neural nets. It was really hot. Everybody thought it would do everything. Um, and a really common error that people were making was, they were just reporting their, uh [disfmarker] uh, classification results on the data that they were training on. And so I think it was [disfmarker] it was [disfmarker] it was very important for people then who were doing something diagnostic to say "hey, I know I'm doing something that isn't kosher".
[speaker008:] Hmm.
[speaker003:] And to make it really clear that they knew it. So that was, I think, why [disfmarker] why it became a popular term.
[speaker006:] Yeah, it just seems like, you know, i if it were "bootstrapping", or if it were [disfmarker] I mean, there are other ways to maybe get the point across.
[speaker003:] But it isn't bootstrapping. Uh, it's [disfmarker] it's really cheating.
[speaker001:] It's us it's using information you wouldn't normally have.
[speaker003:] It's just ng Yeah. Yeah.
[speaker006:] Hu Well. OK.
[speaker003:] But it's cheating in a way that's pril It's [disfmarker] it's announcing to everybody "hey, I'm cheating, by doing this."
[speaker001:] So.
[speaker003:] It's saying [disfmarker] so it's all above the table. I'm actually using this other thing.
[speaker006:] OK.
[speaker003:] That's [disfmarker] that's [disfmarker] that's the reason.
[speaker006:] OK. Alright.
[speaker003:] Bootstrapping would imply it was actually legitimate in some kind of way.
[speaker001:] Hmm.
[speaker006:] Well, I [disfmarker] I think [disfmarker]
[speaker003:] But [disfmarker]
[speaker001:] We're not de legitimizing the data. We're de legitimizing the experiment. We're not saying that the data is cheating data. We're saying th we are cheating by using this data.
[speaker003:] Yeah.
[speaker006:] OK.
[speaker003:] Yeah.
[speaker001:] Because normally you wouldn't have that data available.
[speaker006:] OK. OK.
[speaker003:] Yeah. Anyway.
[speaker006:] Does seem to me that it [disfmarker] that it carries over some baggage with it. That, uh, [vocalsound] i that [disfmarker] I can understand it in context as you described it. But, it [disfmarker] it seems to me that [disfmarker] u um, it does import some negative evaluation, that, um, would maybe not be good if [disfmarker]
[speaker003:] It has some shock value.
[speaker006:] It has shock value, and
[speaker003:] Which is part of why it's used.
[speaker006:] to the wr to the wrong audience, I think that that might be t [vocalsound] a negative shock.
[speaker007:] Yeah. I thi I think to the wrong audience, I agree that, um, p
[speaker003:] Uh huh.
[speaker008:] It's like a disclaimer.
[speaker007:] to the wrong audie we should just explain what it [disfmarker] what it means here, and that it's a common term,
[speaker006:] Yeah. We started with hand marked data, or with, uh, t you know, hand transcribed data,
[speaker007:] and we're [disfmarker]
[speaker006:] or [disfmarker]
[speaker001:] Shoot.
[speaker006:] Yeah. Did you break the microphone?
[speaker007:] Yeah.
[speaker001:] Just the clip.
[speaker006:] Nnn. OK.
[speaker008:] Now we have to l delete that expletive.
[speaker006:] OK. Well, I feel better now. Thank you very much.
[speaker007:] Hmm. The first time I heard that, I [disfmarker] I thought, you know, the same thing
[speaker005:] Thank you, Andreas.
[speaker007:] and I guess you ju after a while it becomes almost a [disfmarker] It's a bu bit of a [disfmarker] a humbling thing when somebody says that.
[speaker001:] It's p it [disfmarker] it really is part of the jargon.
[speaker003:] But it's also is [disfmarker]
[speaker007:] If they get good results but we were cheating on this feature because we took it for granted even though we [pause] can't really assume that, then it's actually the opposite.
[speaker003:] Right.
[speaker007:] It's [disfmarker]
[speaker006:] The trouble is, that [disfmarker] you know, I understand it in that context, but it [disfmarker] but it is almost resentful. It's almost like, you know, resentful of the data, resentful of the, um, the hard work that's going into preparing the data
[speaker001:] Well [disfmarker]
[speaker006:] is [disfmarker]
[speaker003:] No, no. It has [disfmarker] I don't think you're saying the data is cheating.
[speaker007:] Well, it's [disfmarker] it's [disfmarker]
[speaker003:] I think you're saying "in my experiment, I cheated in this way".
[speaker001:] That's what I was saying.
[speaker004:] Yeah.
[speaker003:] I mean, another is what Liz was talking about, how in the Switchboard tests [disfmarker] in all the Switchboard tests we've been g doing, we've been making the same er uh, same [disfmarker] uh, uh, standard [disfmarker] using the same standard way of [disfmarker] of getting the data to test on. Which means that we weren't actually running it on [disfmarker] on, uh, on data that had [disfmarker] had no speech. And, in a sense, that was cheating.
[speaker006:] OK.
[speaker003:] So i so it [disfmarker] it's [disfmarker] it's a good wake up call if people "well, we have this performance but you have to keep in mind, we're doing [disfmarker] we're [disfmarker] we're not doing the whole real task in this way and this way and this way. But, I'll convince you that [disfmarker] that it's still important for you to listen to what I have to say next because of this and this."
[speaker006:] OK. OK.
[speaker003:] And so it's just a way of putting it all out on the table, uh, as opposed to [disfmarker]
[speaker001:] Uh, and it's used for a lot of different types of data. So, segment whether you have segmentation or not, is it male or female or not [disfmarker] [vocalsound] Um, do you know the signal to noise?
[speaker003:] Yeah.
[speaker006:] Mmm.
[speaker001:] Like, that's another one I see all the time, where [vocalsound] you assume it's known.
[speaker003:] Yeah. Sure.
[speaker006:] Oh, interesting.
[speaker003:] Yeah.
[speaker001:] And you say it's cheating because [pause] you don't actually compute it.
[speaker008:] Hmm.
[speaker006:] I didn't know that. Huh!
[speaker003:] In [disfmarker] In the multi band experiments, in the first one, we really wanted to find out, "what if you knew which band was really noisy?"
[speaker006:] Uh huh!
[speaker001:] N Right.
[speaker003:] I mean, suppose you just know that.
[speaker001:] Same cheating.
[speaker003:] And then [disfmarker] then, even if you know that, can that help you? I mean, what can [disfmarker] what strategy can you do, to do well without that particular band in the spectrum?
[speaker008:] Hmm.
[speaker003:] And so that was important to know as a baseline. And [disfmarker] and then, once you knew that, then s you go "well, now how do I know that that's noisy?"
[speaker008:] Hmm.
[speaker006:] OK.
[speaker003:] And [disfmarker] and [disfmarker] and, uh [disfmarker] But, y you know, I [disfmarker] I should also say that there's a lot of [disfmarker] It's not just the word "cheating". But there's lots of other things that we talk about, [vocalsound] which, as soon as you go outside of your narrow little group, it gets very, very confusing to people.
[speaker001:] Mm hmm.
[speaker006:] Oh, sure. Terminology is always context dependent.
[speaker003:] And [disfmarker] I i i i
[speaker006:] No question about it. It's just [pause] that it seems like [pause] the [disfmarker] the point without the negative evaluation would be [disfmarker] However, you know, if i I understand your point. That it [disfmarker] th it has a long tradition in this field [disfmarker]
[speaker001:] It's not even really negative.
[speaker006:] I didn't realize. And is used [disfmarker] This is interesting what [disfmarker] what Adam said about it being used also for a bunch of other, uh, dimensions.
[speaker003:] Well, the example I was thinking of, also, was this w thing that [disfmarker] that Herve and [disfmarker] and Hynek and I made about, uh, increasing the error rate. And, so we [disfmarker] we did a number of papers, and talks, and so forth about, uh [disfmarker] w uh, i the [disfmarker] the virtues of increasing the error rate.
[speaker006:] Oh.
[speaker003:] OK. And [disfmarker] I mean, and the whole point of it was, not that it was good to increase your error rate, but it was good to be willing to risk increasing your error rate by trying risky things, and trying [disfmarker] Because there's this notion of a local minimum, that if you just have [vocalsound] some system that's very complex, and you t you [disfmarker] [vocalsound] you turn some knobs to try to make it better and better, you'll never get out of this local minimum.
[speaker001:] Fine tune little bits.
[speaker006:] Mm hmm.
[speaker003:] You have to be willing [vocalsound] to jump to something that's quite different.
[speaker008:] Mmm.
[speaker006:] Interesting.
[speaker003:] And the first time you jump to something quite different, or maybe the first ten times or a hundred times you do, [vocalsound] it's gonna be much much worse, because you've optimized the other system. So the effect [disfmarker] the immediate effect is gonna be to increase your error rate.
[speaker006:] Interesting.
[speaker003:] And so it was [disfmarker] we had a couple papers like "Towards increasing the error rate in speech" and [disfmarker] and so on.
[speaker001:] Hmm.
[speaker006:] Huh!
[speaker003:] And we really did get feedback from a few people, some of whom were, you know, fairly senior, that [disfmarker] that, um, "Well, you know, you sh you [disfmarker] you c We're c really concerned about you misleading people into thinking they should be increasing the error rate."
[speaker008:] Hmm.
[speaker006:] Yeah.
[speaker003:] And, [vocalsound] uh, and we go [disfmarker] we thought "Oh, come on." But.
[speaker001:] Did you read the paper?
[speaker008:] But weren't you cheating in those experiments?
[speaker006:] Yeah.
[speaker003:] That's [disfmarker] we weren't. That's why we increased the error rate.
[speaker006:] Interesting.
[speaker007:] Actually, I think of cheating as [pause] a way to do some work, where you can't address all of the computational tasks. Like, if we wanna study, um, speaker habits, but we can't do speaker detection. But we wanna assume [disfmarker] Let's say, we know this is Jane and we know this is Chuck, even though automatically to look at the habits, we would need to also first figure that out.
[speaker003:] Mm hmm.
[speaker007:] But we can sort of cheat on that factor because it's somebody else's research. And then we just [disfmarker] So it's [disfmarker] Then we wanna make a proof of concept for [disfmarker]
[speaker001:] What [disfmarker] what this [disfmarker] w Right. What would this be like if [disfmarker] w it were perfect?
[speaker007:] If someone else [disfmarker]
[speaker001:] If this component were perfect?
[speaker007:] Right. Eh [disfmarker] or [disfmarker] w But we really can't work on that problem.
[speaker004:] Yep.
[speaker007:] We don't have time, we're not interested, or w whatever [disfmarker]
[speaker001:] Hmm.
[speaker007:] And so you [disfmarker] you assume it's given
[speaker001:] It's too hard, usually.
[speaker007:] because it's, um [disfmarker] you wanna go forward with your research, and assume that you have that information.
[speaker006:] Mm hmm.
[speaker007:] So, I guess if [disfmarker] I don't ever think of it as negative. More like, it's something we're not building.
[speaker001:] It's not pejorative at all.
[speaker006:] Well. [vocalsound] The term itself is. You know?
[speaker001:] But, it's not pejorative towards the data.
[speaker006:] But [disfmarker] but it [disfmarker] And it's pejorative with respect to a certain purpose.
[speaker001:] It's pejorative towards [disfmarker]
[speaker006:] I understand.
[speaker001:] It's pejorative to o ourselves.
[speaker006:] I [disfmarker] I wanted to raise the issue.
[speaker008:] S self [disfmarker]
[speaker001:] Right? To say "I am cheating in this experiment" is not saying that the data is bad.
[speaker003:] Yeah.
[speaker006:] Yeah.
[speaker003:] But [disfmarker]
[speaker001:] It's saying that my experiment is bad.
[speaker003:] Yeah. Anyway, it's colloquial,
[speaker006:] Anyway.
[speaker003:] an and [disfmarker] and, uh, it's [disfmarker] it's interesting to hear that [disfmarker] that someone coming from a different direction [disfmarker] it s it sounds the way it sounds to you. But [disfmarker] but, uh, I [disfmarker] I'd never heard that before.
[speaker001:] Yep.
[speaker006:] OK. That's interesting. OK.
[speaker003:] So, so i so, that's interesting.
[speaker006:] Well, I wanted to raise the issue, and um,
[speaker003:] Mm hmm.
[speaker006:] I [disfmarker] I, uh [comment] [disfmarker] I appreciate the discussion. I wanted to ask one other question which is a different [pause] matter, which is with respect to, um, this thing that you've been r working on for the recording monitoring script? So. The idea [disfmarker] You had this script that you're working on, to be sure that the microphone values are in [disfmarker] are kept in [disfmarker]?
[speaker001:] Oh, right. I haven't gotten back to that recently.
[speaker006:] OK. [vocalsound] OK.
[speaker001:] But
[speaker008:] Hmm.
[speaker001:] I assume you're saying you want me to [pause] get back to it.
[speaker006:] Cuz, y you know [disfmarker] Well, I was just wondering if [disfmarker] i i Because I [disfmarker] I am finding that, um, in double checking, I've run across, u um, one data set where the microphone was off early on, and then two other [pause] speakers [disfmarker] their microphones went off later.
[speaker001:] Hmm.
[speaker006:] And this is out of, like, seven speakers. So out of seven speakers, four microphones [disfmarker] Y you know, basically, three [disfmarker] three microphones bak basically flaked, which makes it hard for the data to be [pause] used for all possible purposes.
[speaker002:] D Do you think the battery ran out?
[speaker006:] That's what I think.
[speaker002:] Or [disfmarker]?
[speaker006:] I think the two that flaked late, I think it was the battery. But, you know, if the script could, um, you know, alert the recording person to that [disfmarker] I mean, I don't know if there's a way to replace a battery, if it happens in the middle of a meeting.
[speaker001:] Well [disfmarker]
[speaker006:] Maybe that's hopeless anyway.
[speaker001:] Well, you can, but you'll lose a lot of data.
[speaker006:] But [disfmarker]
[speaker004:] Yeah.
[speaker001:] But, I mean, that doesn't really help because often the recording person isn't in the room.
[speaker006:] Oh, I see.
[speaker001:] So, what are you gonna do? I mean it will [disfmarker] w the person [disfmarker] if you're looking up at the board and I disable the screensaver, you will see that the mike is off, but that doesn't necessarily help.
[speaker006:] OK. Then I guess that raises the question of, [vocalsound] uh, whether we should screen the data before they get transcribed.
[speaker001:] So
[speaker006:] Because, although I think that the data are still useful in terms of providing content, and [disfmarker] and that, I know that having three out of seven microphones out of commission during some part of the meeting restricts the, uh, usefulness of the data [pause] for other purposes.
[speaker007:] Yeah.
[speaker001:] Mm hmm.
[speaker007:] I [disfmarker] I think that's a really good idea.
[speaker006:] And maybe you don't wanna have it [disfmarker]
[speaker007:] Because for all kinds of studies, we don't really enjoy meetings where some of the [disfmarker] where the signal's going off at times. It just makes it hard to study any kind of parameters. So, if there's a way to check the signal quality before transcribing it and you find any problem at all, it'd be much better to go to another meeting, I think.
[speaker006:] I actually think that Thilo's [disfmarker] n in a [disfmarker] in a [disfmarker] k a [disfmarker] You know, when you do the pre segmenter and you [disfmarker] and you run across troubles [disfmarker] He [disfmarker] he runs across s some of these that are [disfmarker]
[speaker004:] Yeah. Yeah. Some of these [pause] things are captured by [disfmarker] Yeah. Um, by looking at [disfmarker] at the minimum and maximum and whatever
[speaker006:] So, we have some part of that.
[speaker004:] you find in the channels, and [disfmarker]
[speaker001:] It's just hard to tell between that and just someone not talking.
[speaker006:] I c
[speaker004:] But [disfmarker] Yep. Yep. Yeah. Yeah.
[speaker001:] So.
[speaker006:] Well, there the [disfmarker] And the other aspect of it is, um, that, when the microphone is not well adjusted, then y i i even if it's not a lapel mike, you can get lapel mike type behavior.
[speaker001:] Right.
[speaker006:] Cuz I'm [disfmarker] I'm expecting, for example, with this, that you're gonna end up, [vocalsound] you know, picking up other peoples' signals.
[speaker007:] I've already written some notes here, so I [disfmarker]
[speaker001:] Mm hmm.
[speaker007:] Um, right. Uh [disfmarker]
[speaker006:] Yeah. I mean, it's just [disfmarker] you know, the microphone is intended for certain adjustments.
[speaker001:] Well, we should be getting new equipment in, so we don't have to use the earplug any more.
[speaker007:] OK. But it's my ear. I'm sure it's something wrong with my head,
[speaker001:] It's your hair.
[speaker004:] Hmm.
[speaker007:] but [disfmarker] Um, actually, is there a way, though, to use whatever you're using for background noise to check post hoc that a microphone was constantly on? I mean [disfmarker]
[speaker001:] I mean, y d you [disfmarker] you can do sort of a check,
[speaker007:] Uh, like this o
[speaker001:] but it will be very hard to tell the difference between that and, um, someone not talking.
[speaker003:] Just so it doesn't [disfmarker] when the microphone's dead, it doesn't put out zeros?
[speaker001:] No.
[speaker003:] Or [disfmarker]? Mm hmm.
[speaker008:] What does it put out?
[speaker005:] Really?
[speaker003:] Johnson noise?
[speaker007:] But then how are y how are you detecting during a meeting?
[speaker001:] Little bit of noise.
[speaker003:] Yeah.
[speaker001:] I use a threshold. If it's below a particular value, it [disfmarker] it flashes yellow. So as I [disfmarker] eh [disfmarker] but it's not perfect.
[speaker007:] But is it better than nothing?
[speaker001:] Yeah, probably.
[speaker007:] Cuz it would really be [disfmarker]
[speaker001:] I mean, this is the reason why I haven't gotten back to it, is cuz my first pass at it didn't really work because all the mikes have different noise levels.
[speaker007:] Right.
[speaker008:] Hmm.
[speaker001:] And so I have to do something a little more clever.
[speaker005:] It's just the noise from the connections and the [disfmarker] everything.
[speaker007:] But you could [disfmarker]
[speaker001:] Yep. Yep.
[speaker007:] So, you could run that post hoc on a already recorded meeting, in the sense that, you know, not everyone [disfmarker] as you just said, you won't be there. And then if we find any problems, have the transcribers listen and [disfmarker]
[speaker001:] Yep.
[speaker007:] I really think it's better not to transcribe a meeting that's gonna have problems once you've spent all this effort.
[speaker001:] Yep.
[speaker006:] The only argument for doing so would be with reference to the content. Content maps.
[speaker007:] Yeah. Yeah.
[speaker006:] Uh [disfmarker] but mayb maybe then it should be done in the old original way, w instead of channelizing and having the, uh, uh, you know, elaborate coding of the backchannels.
[speaker008:] Mmm. Yeah. You could still [disfmarker] eh
[speaker001:] Yeah. There's [disfmarker] eh [disfmarker] the s the standard deviation of the signal gives you a good clue. I mean, if that is too low, then you can be pretty sure that it's, uh, empty.
[speaker007:] Mm hmm.
[speaker005:] Hmm.
[speaker004:] Yeah. Yeah. Sometimes you [disfmarker] you capture that when you mix it together, and [disfmarker]
[speaker001:] Right. Exactly.
[speaker004:] Yeah. Yeah.
[speaker006:] And actually, an [disfmarker] an alternative to even doing that level of transcription would be to have a transcriber listen and [disfmarker] and m you know, maybe just [disfmarker] Oh, I don't know if a transcr uh [disfmarker] Well, someone who's associated with the meeting could have, like, a summary of points handled in the meeting. You know, maybe if we could pay one person who knows that subject matter to do an outline of the meeting's content.
[speaker001:] Uh huh.
[speaker006:] Instead of losing the content altogether.
[speaker008:] Well, it's even [disfmarker] If y if you could still transcribe the words based on the [pause] far field microphone or something, [nonvocalsound] you could, uh, still use it for, say, language modeling,
[speaker001:] Is there a [disfmarker]? Far field. Yeah.
[speaker008:] you know.
[speaker006:] I guess the troub trouble in my mind is that it [disfmarker] that it's not a very neat corpus if you say these data are available,
[speaker007:] Yeah.
[speaker008:] Right.
[speaker006:] but these are imperfect,
[speaker007:] Yeah.
[speaker006:] because of the f you know, the batteries flaked. "
[speaker001:] Well, we'll just [disfmarker] We'll [disfmarker] we'll just have to note those.
[speaker007:] I'd mu We're [disfmarker] I'd much rather have, you know, meetings that have all the channels, even if we had to skip a meeting or something,
[speaker008:] Well, if [disfmarker] Right.
[speaker007:] just for [pause] all these other purposes. So.
[speaker006:] And we have so many.
[speaker008:] I guess there is no [disfmarker]
[speaker005:] You could put them at the back of the queue or something.
[speaker001:] Well, I [disfmarker] I [disfmarker] I think we can't throw away that data,
[speaker007:] Yeah.
[speaker001:] cuz otherwise we'll end up with very few meetings.
[speaker007:] Right. But may just [disfmarker]
[speaker008:] there's [disfmarker] there's no shortage of meetings,
[speaker005:] Yeah.
[speaker001:] But
[speaker004:] Yep.
[speaker008:] so
[speaker007:] Right. There [disfmarker] but there's a shortage of trans transcription power.
[speaker008:] we can afford [disfmarker] we can a
[speaker006:] We have such a backlog. I mean, to be able to get through the backlog of the [disfmarker] of the good meetings, it would be nice to not have energies diverted.
[speaker008:] Hmm.
[speaker007:] Yeah.
[speaker001:] Do we have a [disfmarker]? I'm sorry for interrupting. Do we have an EDU meeting at four?
[speaker006:] That's fine.
[speaker008:] Hmm.
[speaker006:] Oh, good question.
[speaker007:] You know what? They can their four o'lock cancelled,
[speaker004:] I don't think so.
[speaker007:] and they're starting at four twenty, and I'll set up at four fifteen.
[speaker001:] OK. Great. Good.
[speaker008:] Hmm. [vocalsound] Hey, I bet there's tea.
[speaker003:] Yeah, but there's [disfmarker]
[speaker001:] Cuz otherwise I was gonna say we have to cancel.
[speaker007:] Yeah.
[speaker003:] There's tea, also, though.
[speaker008:] Yeah.
[speaker006:] OK.
[speaker007:] Yeah. So you'll have si
[speaker003:] So, maybe we should [disfmarker] maybe we should do a, uh, [vocalsound] simultaneous digits [disfmarker] digits read. In the interest of [disfmarker] of getting the snacks.
[speaker001:] A simultaneous digit. OK.
[speaker006:] Mm hmm.
[speaker008:] Hey, I've never done one of those.
[speaker003:] Oh, you never have? Oh, it's a treat.
[speaker007:] You have to plug your ears or just prepare not to laugh.
[speaker008:] Right.
[speaker003:] OK.
[speaker001:] Everyone ready? S reading simultaneous digits? Three! Two! One!
[speaker006:] Yeah. Yep. Yep.
[speaker008:] Sev
[speaker003:] OK, another successful babble.
[speaker001:] And [disfmarker]