[speaker001:] OK.
[speaker003:] OK. So the one [disfmarker] [vocalsound] [vocalsound] one thing I knew I wanted to talk about was about, uh, sort of last minute stuff to to, uh, try to get some recognition results.
[speaker005:] Recognition results
[speaker003:] Yeah.
[speaker005:] for [disfmarker]
[speaker003:] So, uh, on [disfmarker] on [disfmarker] on meeting data. And so, I'm [disfmarker] I'm not sure exactly what you're doing already, and [disfmarker] and there's some stuff I've talked to Dave [disfmarker]
[speaker005:] Well, we I just started recognition on the on Thilo's [vocalsound] segments, which was [disfmarker] but using the far far the close talking microphone.
[speaker003:] OK. So [disfmarker] Um [disfmarker]
[speaker005:] And you wanted [disfmarker] I know you wanted the far field [pause] data.
[speaker003:] Right. So [disfmarker] so, we have some stuff with no overlap, uh, for which there would be near [disfmarker] near field results.
[speaker005:] Uh huh.
[speaker003:] We wanted to get the far field results for that. And then this [disfmarker] this real, uh, long shot thing would be, that we'd apply Dave's processing to, uh, potentially training and test data
[speaker005:] Mm hmm.
[speaker003:] and do the [disfmarker] look at the same thing. And in talking this morning with, uh, Chuck and with Dave [vocalsound] one thought was to use [disfmarker] We couldn't remember how different the numbers were, but if you just [pause] worked with males only and used the short training there're [disfmarker] there're [disfmarker] uh, I think Chuck's recollection was that when he was doing the feature stuff, it took maybe a day and a half to do the training.
[speaker005:] To retrain?
[speaker003:] Yeah.
[speaker005:] Uh Yeah. Um. That's about right. Actually, it should probably be [disfmarker] It depends on who else is using machines, but we have more machines now. So.
[speaker001:] That's true.
[speaker005:] It's more like a day, probably.
[speaker003:] Um, how much worse is the short training set than the large one, in terms of the ultimate performance?
[speaker005:] Mmm, it's like [disfmarker] Something like three [disfmarker] three percent [disfmarker] three or four percent absolute.
[speaker003:] Yep. So, it's [disfmarker] that should be fine for this, I would think. So we [disfmarker] an and you have the short [disfmarker] you have short training results for the close [pause] case?
[speaker005:] Um, not for meetings. Because we didn't train [disfmarker] we didn't re ever recognize with the [disfmarker] with the small models [pause] on meeting data. But I [disfmarker] I have the models, so I could run reco
[speaker003:] So, how do you know it's [disfmarker]? Oh, it's three percent on [disfmarker] on, uh, on Hub five.
[speaker005:] On [disfmarker]
[speaker001:] Hub five.
[speaker003:] I see. Yeah. But we have the models so we could get that number, and [disfmarker] So the question is, what [disfmarker]? w
[speaker005:] I mean, the [disfmarker] the [disfmarker] mmm The recognition also takes [pause] non negligible amount of time. So [disfmarker] we might wanna restrict it to, maybe, a few meetings, if you want to do a full comparison.
[speaker003:] It has to be enough so that [disfmarker]
[speaker005:] Uh.
[speaker003:] I mean, it's the non overlap only. Um [disfmarker] [comment] And [disfmarker] it has to be enough to be sort of comparable to what you folks were seeing and what you reported already.
[speaker005:] Hmm.
[speaker003:] I mean [disfmarker]
[speaker005:] Well, do we have the [disfmarker] do we have the processed data? That [disfmarker] that's also [disfmarker]
[speaker003:] No. He has to create that. But [disfmarker] but [disfmarker] s but [disfmarker] so [disfmarker]
[speaker005:] Right.
[speaker003:] Um [disfmarker] We have [pause] a whole parallel set of things over here which are all with digits.
[speaker005:] Mmm. I see.
[speaker003:] And [disfmarker] and [disfmarker] and Dave has been working with that and there's all of those issues. But [disfmarker] [vocalsound] I know that if I [pause] go in with something that's not just digits it would be [pause] good. And, um, so [disfmarker] We already have these results that you [disfmarker] I mean, on [disfmarker] uh, a l a lot tha tha a particular test set, that you [disfmarker] that we reported at HLT.
[speaker005:] Right.
[speaker003:] Um, it'd be nice to have something more than that. And w we had talked about was having distant [disfmarker] Um, and then, uh, if we could on top of that [disfmarker] I mean, so this is gonna be a lot worse. Right? Whatever comparison we [disfmarker] w w one would presume.
[speaker005:] Mm hmm.
[speaker003:] But we don't know how much worse,
[speaker005:] Right.
[speaker003:] w w uh, which is certainly one interesting thing. And then, um, Dave [disfmarker] I think we figured that it'd probably take a day or two to compute the [disfmarker] the, uh [disfmarker] uh [disfmarker] Well [disfmarker] How many hours of training [disfmarker]?
[speaker005:] Actually, I did retrain [disfmarker] I recently retrained, um, [vocalsound] for another reason, on the full training set. And that took only [disfmarker] I think it took only two days.
[speaker003:] Yeah.
[speaker005:] So, it's actually conceivable to do [disfmarker] use the full training set.
[speaker003:] Yeah, but we also have to do this other processing, so having a smaller training set, if it's only a few percent difference, it might be [disfmarker] be worth doing it.
[speaker005:] Oh, I see.
[speaker003:] But m How big is the small [pause] training [pause] set? Do you remember?
[speaker005:] Hmm. [comment] [vocalsound] [comment] Uh. I don't know, something [disfmarker] something like [disfmarker] something between [comment] thirty and fifty hours, maybe. I f I forget the exac
[speaker003:] It's around there. And ha and [disfmarker] and male is roughly half of that,
[speaker005:] Right.
[speaker003:] or [disfmarker]?
[speaker001:] Well, that's only male.
[speaker003:] Or [disfmarker] or [disfmarker] or was that only male?
[speaker005:] Uh. Actually, I don't know. I'd [disfmarker] [vocalsound] I [disfmarker] I can look it up. It's [disfmarker] it's [disfmarker] it's just, uh, I don't know the [disfmarker] remember the [disfmarker] the number.
[speaker001:] We could only jus just do the male only
[speaker003:] OK.
[speaker001:] right? Or will we run into trouble when we g?
[speaker005:] Well, the males [pause] account for most of this meeting data anyhow.
[speaker001:] Yeah.
[speaker005:] So [disfmarker] Yeah. I would say we [disfmarker] you do only males.
[speaker003:] Yeah. So [disfmarker] Yeah. So that's certainly part of the issue, is that right now he's [disfmarker] he hasn't written his stuff for efficiency. Yeah. It's [disfmarker] it's in Matlab and so on, and [disfmarker] [vocalsound] And, uh, it's not an impossible amount of time.
[speaker005:] Mm hmm.
[speaker003:] We [disfmarker] we were guestimating it was like one and a half times faster than real time, or something? So, if there's thirty hours of data, you can calculate that he can do, uh, the enhancement in a day
[speaker005:] u
[speaker003:] and something. So [disfmarker] But [vocalsound] if we were dealing with two hundred hours or something, I think it'd be [pause] prohibitive.
[speaker005:] No. It's definitely [disfmarker] It's less than a hundred hours, for sure. It's [disfmarker]
[speaker003:] Yeah.
[speaker005:] It's probably actually, uh [disfmarker] It's [disfmarker] uh, I think it's around thirty hours just for [disfmarker] for one gender.
[speaker001:] That's what I was thinking.
[speaker003:] Yeah.
[speaker005:] Yeah.
[speaker001:] Yeah.
[speaker003:] Yeah. So, I mean, it's a bit of a push, but it seems like, OK, we've got some models, we've got some training data, we have software that works, he's got a method that helps with, you know, other ta another task. Um [disfmarker] It, you know, appears to be, you know, debugged.
[speaker001:] S
[speaker005:] Mm hmm.
[speaker003:] Um [disfmarker]
[speaker001:] So [disfmarker] We Yo So, one thing I was wondering is, did you already do that middle one or should we re do that one, too?
[speaker005:] No. I didn't do that, because we haven't even cut the waveforms for that.
[speaker001:] You didn't do that. OK. Yeah. That's what I was gonna say next.
[speaker005:] So [disfmarker]
[speaker001:] We hafta [pause] cut the, uh [disfmarker]
[speaker005:] Right. So.
[speaker001:] W s so is [disfmarker] Morgan, is the plan to just pick one of the far field mikes?
[speaker004:] Yeah. [vocalsound] [vocalsound] [vocalsound] [vocalsound] [vocalsound] Yeah. We did.
[speaker001:] Uh [disfmarker]
[speaker005:] And there's a bit of a question whether you want to use [disfmarker] um, what segmentations you want to use.
[speaker003:] Yes.
[speaker005:] Uh [disfmarker] Uh, David just [disfmarker] Um, I'm sorry. Don just, uh, created a new version of the first meetings that we had previously recognized, but with different segmentation. And so [disfmarker] [vocalsound] [vocalsound] Um [disfmarker] It would be nice [disfmarker] I mean, if the results are comparable to what we had before [disfmarker] to use those segmentations, because th then we could claim that everything's automatic.
[speaker001:] Do you know [pause] when he'll have the [disfmarker] the comparison?
[speaker005:] Right. Well, I'm [disfmarker] as I said, I just started [pause] the recognizer, um [disfmarker] It will [disfmarker] [vocalsound] uh, [vocalsound] it will probably be [vocalsound] a couple hours before [disfmarker] before I have some results.
[speaker003:] Oh. Well, that's not bad.
[speaker001:] Oh, OK.
[speaker005:] So [disfmarker]
[speaker001:] So [disfmarker] cuz I don't think the new data will be ready, uh, for a couple days may probably.
[speaker003:] You mean the training.
[speaker001:] So, the training.
[speaker005:] But the segmentations matter for the filtering.
[speaker003:] Yeah.
[speaker005:] Right? Because [disfmarker]
[speaker003:] For the test.
[speaker005:] For the test set. Yeah. So,
[speaker003:] Yeah, fo
[speaker001:] Yeah.
[speaker005:] need to be [disfmarker] But, first [disfmarker] first, of course, you would wanna process the [pause] training data, because we wanna get that started.
[speaker003:] Right.
[speaker005:] Yeah.
[speaker003:] Yeah. I mean, it [disfmarker] it'd be really great if it was all automatic, but I think that, you know, given the pressure of time, if [disfmarker] i i I mean, since you're gonna find out in a short amount of time, that's great.
[speaker005:] Mm hmm.
[speaker003:] But i if [disfmarker] if it doesn't work out, I think we would rather charge ahead with the older segmentations
[speaker005:] Mm hmm. Right.
[speaker003:] and [disfmarker] Um, and we were gonna use one of the P Z I don I don't know what [disfmarker] Probably whatever one you've been using for [disfmarker] for [disfmarker] for the digits.
[speaker001:] I think it's that one. Right? F.
[speaker003:] Is it this one?
[speaker002:] It's e F.
[speaker001:] F. Yeah, that's it.
[speaker003:] I [disfmarker] I'd [disfmarker] which [disfmarker]? That's F? How do you know?
[speaker001:] That's F. It's the second nearest the machine room.
[speaker003:] Oh. Bien sur OK. [vocalsound] Alright, so [disfmarker] Bet they'll have fun with that one. OK, so. Uh [disfmarker]
[speaker001:] OK. So, uh, I just want to make sure I understand what we need to run. Um [disfmarker] Let's see. So, it's [disfmarker] OK, so if we're talking about [disfmarker] Let's [disfmarker] let's assume that we're gonna use the new segmentations. We need to, um, [vocalsound] run recognition o just looking at the no overlap column. Basically, we have to d r do recognitions for all three of those cases. Right?
[speaker003:] Mm hmm. Right.
[speaker001:] Um, because [pause] we're gonna be using the [disfmarker] just the male, uh, model [disfmarker] short training set for the male.
[speaker005:] Mm hmm.
[speaker001:] So we need to have results for all three of those, even though we have [disfmarker]
[speaker005:] Ma maybe you should limit ourselves to the Meeting Recorder meetings?
[speaker001:] OK.
[speaker005:] Um, if you were gonna cut down on the test set, I would suggest that.
[speaker003:] Well [disfmarker] Maybe. But, I [disfmarker] I mean, how long does it take for the test?
[speaker005:] Actually, the longer [disfmarker] the Robustness meetings take longer, because there's this one speaker who talks a lot. And so the [disfmarker] [vocalsound] Um. No. It's because for all the [disfmarker] for the adaptation and normalization steps, you cannot [disfmarker] you have to d you have to, uh, um, you cannot chop it up into small pieces. So, you're sort of limited by how long the longest speaker, uh, is s speaking. So how much data there is from the [disfmarker] the speaker who talks the most. So, [vocalsound] um, you parallelize across different speakers,
[speaker001:] Mm hmm.
[speaker005:] but [vocalsound] you know, if you have a bunch of speakers who speak very little and then one wh who [disfmarker] who speaks a lot, then [vocalsound] effectively, everybody waits for the longest one to process.
[speaker003:] Right.
[speaker001:] Right. Yeah.
[speaker003:] Bu But what [disfmarker] what was your result for [disfmarker] uh, that we had at the HLT?
[speaker005:] So.
[speaker003:] Was that a combination of me?
[speaker005:] That was both types of meetings, but most [disfmarker] but there were only two Robustness meetings, and four or five, uh, Meeting Recorders.
[speaker001:] And if we're re doing the baseline anyways, it [disfmarker] it [disfmarker] it would be OK.
[speaker005:] Right.
[speaker001:] Right? I mean [disfmarker] To [disfmarker] to just limit ourselves to a smaller [disfmarker] How long would it take to run recognition, if we did that?
[speaker005:] Oh. Uh, I [disfmarker] I [disfmarker] I don't have [disfmarker] I don't have a good [pause] gue
[speaker001:] I mean, is [disfmarker] is it like a day or is it [pause] a few hours or [disfmarker] roughly?
[speaker005:] For everything? For all the meetings?
[speaker001:] For [disfmarker] yeah, let's say we just did the Meeting Recorder meetings for our test set.
[speaker005:] Uh. Um. It's probably more than a day, but probably less than two.
[speaker001:] Oh, really. I didn't realize each test took that long.
[speaker005:] Well, i No. I mean for all the meetings. Because it's [disfmarker] [vocalsound] Again, it's, um [disfmarker]
[speaker003:] So you were doing like [disfmarker]
[speaker005:] So each meeting [disfmarker] each meetings takes, uh, something like [disfmarker] Again, we [disfmarker] we [disfmarker] I ran [disfmarker] when we ran these, we were sort of short on machines, and, um, I don't know, I [disfmarker] I would estimate maybe four hours per meeting. Something like that.
[speaker001:] Four hours per meeting.
[speaker005:] Right.
[speaker001:] Wow.
[speaker003:] Yeah. But if you [disfmarker] So, if you do half a dozen meetings, that's [disfmarker] that's about a day. We also have more machines now.
[speaker005:] Right. Right. So that's why I'm saying I'm not sure how they would scale with more machines.
[speaker003:] well Yeah. Yeah. I mean, if we had about six [disfmarker] A six hour test set's not bad. Right?
[speaker005:] Right.
[speaker001:] S six hour test set. Six meetings. OK.
[speaker003:] You know? Right? I mean, a lot of the evaluations have been [disfmarker]
[speaker005:] We have MR [disfmarker] two, th We have two, three, four, fi I think there are four Meeting Recorder meetings that we worked with.
[speaker003:] Four that you worked with?
[speaker004:] I think it's [disfmarker] Is it the same set as [pause] the alignments?
[speaker005:] Yeah.
[speaker004:] I think it's five Meeting Recorder meetings.
[speaker005:] Five? OK.
[speaker003:] That would be OK, too. I mean, I'm [disfmarker] i So, if they have a set that they worked with, and you [disfmarker] you got [disfmarker] Did you do similarly in performance between them and the other meetings, or was it [disfmarker]?
[speaker005:] Uh, with the Robust compared to Robustness? Yeah. The big variation is by [disfmarker] whether it's a native speaker or not.
[speaker003:] Yeah.
[speaker005:] And whether [vocalsound] it's, um [disfmarker] Uh, I think that's the o actually [disfmarker] and [disfmarker] and of course what, um, you know, whether it's lapel or, uh, headset microphone.
[speaker003:] And overlap or not.
[speaker005:] Yeah.
[speaker003:] Yeah. So maybe just with the [disfmarker] the [disfmarker] the Meeting Recorder set of the [disfmarker] De th that you did before.
[speaker005:] And we can exclude [disfmarker] we don't need to recognize the [pause] non natives, because we know that [disfmarker] I mean, in fact, we excluded them previously from [disfmarker]
[speaker003:] Yeah. So we want to do the same [disfmarker] same thing.
[speaker005:] Right.
[speaker001:] OK. So [disfmarker] Alright. So if we got a list of the, uh, segmentations for these five Meeting Recorder meetings, we could start, uh, the first two experiments going right away, using the short male models.
[speaker005:] Mm hmm.
[speaker001:] So, you could get those going while Dave is, um, creating waveforms for the r retraining the short male models.
[speaker003:] Yeah. [vocalsound] Once we know which segmentations we're using. Yeah.
[speaker001:] Right. OK. OK. And then [disfmarker] OK. So, do we w also want to run that bottom experiment without retraining the short male models on his thing? Did you want that? Or [disfmarker]?
[speaker003:] Um, I [disfmarker] I agree that that would be an interesting thing to do, but I sort of regard it as secondary.
[speaker001:] OK. So, we'll save that.
[speaker003:] So if there's sort of machines sitting around and people sitting around and they're waiting for [vocalsound] other things to finish, then sure. But [disfmarker] Uh, Chuck had been asking about that earlier as kind of a control to know, um [disfmarker] Cuz, I mean, you could imagine a fantasy in which you said that Dave's processing [vocalsound] made the, uh, far microphone like the near microphone. In which case you shouldn't have to actually retrain.
[speaker005:] Mm hmm.
[speaker003:] But it's [disfmarker] it's not really true. It's [disfmarker] it's sort of fantasy. It does [disfmarker] it does muck up the data in [disfmarker] in some funny ways.
[speaker005:] Mm hmm.
[speaker003:] And so, I'm [disfmarker] I'm kind of questioning that.
[speaker001:] Well, i it [disfmarker]
[speaker003:] But [disfmarker] But [disfmarker]
[speaker001:] Well, on a more basic level, also, it means that that third experiment, there are actually two differences between the other experiments, not one.
[speaker003:] Right.
[speaker001:] So, it's hard to know [disfmarker]
[speaker003:] It involves retraining and it involves a [disfmarker]
[speaker001:] Right.
[speaker003:] Uh, that's right. I mean the other thing which [disfmarker] which it might come in [disfmarker] to is if there was some problem in the retraining. I mean, maybe you'd just have some mechanical thing we do wrong.
[speaker001:] Mm hmm.
[speaker003:] Uh, that, uh [disfmarker] since Dave's experience was that it didn't help as much if you didn't retrain, but it does help some, that we would hopefully see that.
[speaker001:] Right. Mm hmm.
[speaker003:] So, that [disfmarker] that's [disfmarker] that's true.
[speaker005:] Wait, did [disfmarker]?
[speaker003:] I i
[speaker005:] So when you used original [disfmarker] the original models, and you just process the test set in this way, d do you get any [disfmarker] do you get decent performance or not?
[speaker002:] u [vocalsound] I [disfmarker] I [disfmarker] I think, um, for the far mike HTK system I was using, it did help somewhat.
[speaker005:] Mm hmm.
[speaker002:] I could re check that. But it was such a bad baseline that I don't know what that means.
[speaker005:] Mmm. Right. Right. OK.
[speaker002:] Cuz the baseline word error rate was around forty percent on digits.
[speaker005:] Mm hmm.
[speaker001:] On the far field?
[speaker002:] Right.
[speaker003:] Right. [vocalsound] So [disfmarker]
[speaker005:] OK. Well, I'll [disfmarker] I can get started on the [disfmarker] well, the first [disfmarker] the one that already has a cross there. We need to re do that with small models. Right.
[speaker003:] Yeah.
[speaker005:] And then, have to ask, um, I guess, Don to, uh, cut the, um, cut the segments for the sh for the tis distant mike. Uh, uh [disfmarker] that's [disfmarker] [vocalsound] So we would be using the same channel for each [disfmarker] fo for everything?
[speaker003:] Yeah.
[speaker001:] Mm hmm.
[speaker005:] OK. So.
[speaker003:] I mean, do you ha do you have to rely on his segmentations at all to do the top one?
[speaker005:] No, no. We would use the same segmentations, but he needs to extract [disfmarker] extract the wavef form segments from a different channel.
[speaker003:] Oh. OK. Got it.
[speaker005:] Right.
[speaker001:] So when you said you were gonna start that top one, were you gonna use the new segmentations?
[speaker005:] Mm hmm. Um [disfmarker] Yeah.
[speaker001:] OK.
[speaker005:] If [disfmarker] assuming that the performance turns out to be comparable with [disfmarker] with the old experiments and the old segmentations.
[speaker001:] Right. OK.
[speaker005:] Now there's the issue of [disfmarker] Oh, OK. So there's the issue of speaker normalization. So, with the distant microphone you wouldn't know which speaker is talking. Right?
[speaker001:] Can we [disfmarker]?
[speaker003:] We [disfmarker] we talked about this before. I think what we were saying was that, um, the very fact that in both cases we're ignoring the overlap section means that, um, uh, we're to some extent finessing that. So, um, I think, for the purposes of just determining whether a far field microphone [disfmarker] uh, what the effect of the far field microphone is, we should do the same to both.
[speaker005:] Mm hmm. I see. So you want to cheat?
[speaker003:] I mean [disfmarker] We want to i incorporate [comment] [vocalsound] certain data that would not be available during final tests, [comment] uh, under a [disfmarker] a full fair test of it, much as we are in the [disfmarker] all the numbers that we have so far.
[speaker005:] OK. So we assume [disfmarker] we assume knowledge of the speakers as [disfmarker] as, um [disfmarker] [vocalsound] in a way that's compatible with the close talking test set.
[speaker004:] Oops.
[speaker003:] Yeah. We [disfmarker] we simply wanna determine what's the difference in performance due to being distant versus close.
[speaker005:] OK. Oh, OK.
[speaker001:] So, does that mean you turn off speaker normalization when you run it? Or you just let it do what it would do, an?
[speaker005:] No. It means [disfmarker] No. It just means you [disfmarker] you group together the segments [pause] that by magic you know belong to one speaker, and [disfmarker] and treat [disfmarker]
[speaker003:] I mean to a lesser extent you had that same magic the other way, too, because you have leakage into other microphones. Right? But, it's just you're using the fact that [vocalsound] this is where this person is.
[speaker005:] Right. Right.
[speaker003:] Right? So.
[speaker005:] But, um [disfmarker]
[speaker003:] It's just easier to do.
[speaker005:] Well, in the new test, actually, that's not true. So [disfmarker] [vocalsound] Again, if this [disfmarker] if these new segmentations work OK, then we [disfmarker] then it's a fair [disfmarker] it's a completely fair [pause] test.
[speaker003:] Yeah. So, how do you determine what you use to group together to be a [disfmarker] a [disfmarker]?
[speaker005:] You group together all the data coming in through one channel and where [pause] Thilo's speech detector has [disfmarker] has determined that there is speech. And that speech is [disfmarker] is deemed to come from that speaker, whether that's true or not. So if you get some cross talk from another microphone, then you just process this [disfmarker] it as if it were from that speaker.
[speaker003:] The only other alternative would be to turn off speaker adaptation in both.
[speaker005:] Well, that's more of a problem. I mean, because it's [disfmarker] You can just pretend it's some kind of gene I mean you can pretend it's all from one speaker and do all this processing the same, but then you're gonna get results that are worse on account of not doing proper speaker normalization
[speaker003:] Mm hmm.
[speaker005:] and you're gonna have [disfmarker] So, you could certainly do better than that by doing, for instance [disfmarker] uh, cluster the segments, which is what we do, say, in a Broadcast News system, where you don't have speaker labels. But that would be another processing step that I'm [disfmarker] I would have to [pause] debug first, and so forth, and so we wanna avoid that. So I agree with you. We should [disfmarker] [vocalsound] we should, uh, do the [disfmarker] you know, this sort of cheating experiment.
[speaker003:] OK. Yeah. An and e so that will tell us [pause] what the difference it between the mikes, and then, uh, in order to [disfmarker] The [disfmarker] the other difference that we'd have to take care of is that, uh [disfmarker] yeah, we [disfmarker] we don't have a mike that, uh, is particular to a person. And so we'll have to do some clustering, and that'll be another [disfmarker] [vocalsound] another, uh, issue, too.
[speaker005:] Mm hmm.
[speaker003:] But, it [disfmarker] it [disfmarker] I could be wrong, but it seems to me that [disfmarker] that the speaker [disfmarker] the [disfmarker] the level of degradation that you get from having the distant mike in a normal acoustic is much greater than what you get from, say, not applying speaker adaptation or applying speaker adaptation. I think that the [disfmarker] I mean, we'll see. But [disfmarker] but, I think that the kind of gains that we've seen from speaker adaptation [vocalsound] on Hub five sort of things are like a few percent. Right? And [disfmarker]
[speaker005:] It's not just speaker adaptation. It's the whole norm feature normalization process. I it's spea uh, all that is speaker based. You know, so we [disfmarker] So, in that I'm, um [disfmarker] Y you know, d d b the most important, of course, is the [pause] cepstral mean subtraction.
[speaker003:] Yeah.
[speaker005:] And that [disfmarker] I don't know if we [disfmarker] we never really [disfmarker] I don't remember, because it's so far [disfmarker] [vocalsound] s so long ago that we didn't do that on a per speaker basis,
[speaker003:] It doesn't make that much difference, I think.
[speaker005:] but [disfmarker]
[speaker003:] I would doubt that it would be a huge amount of difference for that. So, I mean, I [disfmarker] I think that that difference would definitely be marginal. I think the main thing is to do something [disfmarker] to do some cepstral mean subtraction on some level. And, uh, so what's different about this processing is just that we're doing it at a much longer time [pause] scale. Right? But [disfmarker] Um [disfmarker]
[speaker005:] A and by the way,
[speaker002:] What wo
[speaker005:] it's [disfmarker] actually we're [disfmarker] we're already [disfmarker] If we use the same segmentations that we use for the close talking microphone, then [vocalsound] the segmentations assume that we have access to all channels and cross correlate them,
[speaker003:] That's right.
[speaker005:] so [vocalsound] there's no point in not using that knowledge for speaker identification.
[speaker003:] Yeah.
[speaker001:] Mm hmm.
[speaker002:] Uh, I I think also for the log spectral mean subtraction, uh, we wanna know which speaker's talking when,
[speaker003:] Yeah.
[speaker002:] cuz we wanna chain together the audio from one particular speaker to calculate the mean
[speaker005:] Yeah.
[speaker002:] and subtract it, and we don't [disfmarker]
[speaker005:] Right.
[speaker003:] Right.
[speaker005:] OK.
[speaker003:] Um. Yeah, I guess. But, um, I also think that a again once we got into it, that, um, using some kind of clustering would probably work reasonably well there, too. Certainly for the [disfmarker] the two microphone case, which we're not gonna mess with because it's another whole deal with the low quality microphones, um, we ought to be able to at least tell that it appears that things are coming from a particular direction. So we ought to be able to use that information, um, as well. But, I think we might be able to do not too bad a job of separating out sp uh, segments that appear to come from a single speaker both in terms of s acoustic similarity and in terms of direction. So, I mean [disfmarker] But that's another research thing to do and [vocalsound] probably won't get done the next week.
[speaker005:] Right. So what [disfmarker] what is the schedule here?
[speaker003:] Well, I mean, I'm [disfmarker] I'm leaving for [vocalsound] for the, uh, the New Orleans meeting, uh, next Saturday, and [disfmarker] and, um,
[speaker005:] Mm hmm.
[speaker003:] it'd be kinda nice to have some results at least a day or two before that, so that I could figure out what I wanted to say about it.
[speaker005:] Mm hmm. Oh, we'll call you when you get there.
[speaker001:] You'll have email. Right?
[speaker003:] Yeah. Uh, not to mention that [disfmarker] that, uh, Mari's putting together this report next week, too, you know. So [disfmarker] Uh, what we were hoping was that over the weekend we could do, uh, the, um, calculation on the training set and, uh [disfmarker] uh, maybe, you know, we could [disfmarker] by the end of the weekend, we could have the top one, and [disfmarker] and then [vocalsound] early next week, do these. If we had enough machines, maybe do them in parallel.
[speaker005:] Mm hmm.
[speaker003:] So that by the middle of the week we had s had some kind of result. I mean, it's [disfmarker] it's one of these [pause] Hail Mary kinds of things. I mean, it [disfmarker] it, uh [disfmarker] might [disfmarker] might not [pause] work out. But, uh, f figured I may as well ask for it.
[speaker005:] OK. I'll ask [disfmarker]
[speaker001:] So [disfmarker]?
[speaker005:] The other thing is, um [disfmarker] and I'll ask Don which is easier to process in terms of creating these [disfmarker] the [disfmarker] the test data for the far [disfmarker] far microphone. If [disfmarker] if it turns out that for some reason it's easier for him to [vocalsound] use the old [disfmarker] um, [vocalsound] the [disfmarker] the [disfmarker] the old, uh,
[speaker001:] Segmentation?
[speaker005:] segmentations, then we'll just use that, I figure.
[speaker003:] Right.
[speaker005:] Um [disfmarker]
[speaker001:] S So, um. I don't want you to have to be burdened with doing a lot of stuff. What [disfmarker] what can I do to [disfmarker]? Y y you said it would be easy for you to do that [pause] top one there, and [pause] I guess Don can do the segmentations of the, uh, channel F?
[speaker005:] Mm hmm. Um [disfmarker]
[speaker001:] I mean, I can certainly help with, uh, retraining [pause] the short male models,
[speaker005:] Right.
[speaker001:] uh, once we have the new data.
[speaker003:] You have models of short males?
[speaker005:] Yeah. Um [disfmarker] Right. Uh, let's see. The [disfmarker] the You [disfmarker] you can [disfmarker] I mean [vocalsound] you could [disfmarker] you could run the, um, you c Basically, once the, um, whe the top [disfmarker] the top one's done, you could easily re run the whole set of experiments.
[speaker001:] The top one is done. I c I can re do the [pause] next one. Yeah. Mm hmm.
[speaker005:] Uh, I mean, manage the jobs and so forth,
[speaker001:] Yeah. Sure.
[speaker005:] uh [disfmarker] [vocalsound] Um,
[speaker001:] Yeah, the [disfmarker] the bottom one would b just be a matter of pointing it at a new set of files and kicking it off,
[speaker005:] that's all [disfmarker]
[speaker001:] so that would be re I mean, th not the bottom one, but the middle one
[speaker005:] Mm hmm.
[speaker001:] would be really easy once you've got the top one going.
[speaker005:] Yeah.
[speaker001:] I could do that.
[speaker005:] Right.
[speaker001:] OK. So I guess I just need to get Don to, uh [disfmarker]
[speaker005:] Right. So, somehow the [disfmarker] Assuming he uses the new naming scheme, then he should call [pause] the waveforms the [disfmarker] so, the waveform names have the, you know, meeting [disfmarker] meeting ID, and the [pause] microphone, and the, [vocalsound] um [disfmarker] I guess, the channel and the microphone and the speaker, um, [vocalsound] speaker [disfmarker] some something that identifies the speaker.
[speaker001:] Mm hmm.
[speaker005:] So [disfmarker]
[speaker001:] To keep it the same but j just change them all to channel F?
[speaker005:] Exactly. So, uh [disfmarker] well, you still need to be able to distinguish the different speakers. That's the key point.
[speaker001:] Right. Right.
[speaker005:] Because, if we wanna do what we just discussed [disfmarker] So [disfmarker] [vocalsound] Uh, uh, the [disfmarker] the best [disfmarker] the easiest way to do that would be to just take [disfmarker] You know, you make the channel be channel F,
[speaker001:] Yeah.
[speaker005:] but then keep the speaker names the same as they would be in the old [disfmarker] in the close talking, uh, version.
[speaker001:] Yeah. OK. OK. OK. And so that's something that Don would do [disfmarker] right? [disfmarker] when he creates these.
[speaker005:] Right. Exactly.
[speaker001:] OK. OK. So will you talk to him about that, or do you want me to talk to him?
[speaker005:] I, uh, I [disfmarker] I can talk to him.
[speaker001:] OK. And then the bottom [pause] one in terms of the test will be [disfmarker]? Uh [disfmarker] That will just be a copy of the one above it, except for [pause] different models from training.
[speaker002:] Well, we also have to mean subtract the test data.
[speaker001:] Ah. OK. So, we need to run [disfmarker]? OK. Well, once we have the new, uh [disfmarker] Well, once I do that, uh, second experiment, we'll have the, um, files, and I can give you those to [disfmarker] to process.
[speaker002:] OK. And there's, um, S so the way this means subtraction expects to work, is it expects to have, um, this continuous stream of audio data from a particular speaker to operate on. And it goes along it with this sliding window, calculating the mean using the data in the window, and then subtracting that.
[speaker001:] So, do you create this continuous stream from the individual utterance files?
[speaker002:] I mean, uh [disfmarker] That's [disfmarker] that's how I've been doing it, just by concatenating files together. Um, and if these files [disfmarker] and the since they're individual utterance files, um, s long silence periods are removed, which is a good thing. Because this method might estimate the mean badly, if it had to face long silence periods. But that does mean that I need as much [disfmarker] I need twice as much disk space as the original set cuz I need [disfmarker] while I'm running it [disfmarker] cuz I need to create this intermediate set, um, of these big files,
[speaker001:] Yeah.
[speaker002:] and then [pause] create the [disfmarker] finally, the mean subtracted, um, little files. And then I can get rid of the big files. But st while I'm doing the processing, I'll nee I need twice as much disk space.
[speaker001:] OK. I'm gonna [disfmarker] I'll check with, um, Markham, and see what happened with the [disfmarker] the disks. He went to put those on a couple of weeks ago and something [disfmarker]
[speaker003:] I think [pause] Markham's out on [pause] vacation. I think, che check with Dave.
[speaker001:] Oh, OK. I'll ask David, then.
[speaker004:] Yeah.
[speaker003:] Yeah.
[speaker001:] You haven't seen new disks pop up, have you?
[speaker004:] Nope. I was wondering [pause] if they were [pause] in.
[speaker001:] OK. Yeah.
[speaker005:] Well, they grow them on trees now.
[speaker003:] I thought they were like mushrooms.
[speaker001:] Well, he went to put them on [disfmarker]
[speaker005:] Just, you shake them and they fall down.
[speaker003:] They're popping up.
[speaker001:] Yeah. He went to put them on and then something happened. And he sent around a note saying "Oh, uh, it didn't work, and we'll have to schedule another time."
[speaker004:] Yeah. Yeah. Something w went wrong.
[speaker001:] And then, didn't see [disfmarker]
[speaker004:] Nothing happened.
[speaker001:] nothing happened.
[speaker004:] No? Yeah.
[speaker001:] So [disfmarker] I'll check with David about that. OK.
[speaker005:] OK.
[speaker003:] Yeah. Cuz we still all have tha the [disfmarker] that other one going, which is the, uh [disfmarker] the Macrophone training.
[speaker002:] Right, an and [disfmarker] So [disfmarker] so, Andreas, um,
[speaker005:] Mm hmm?
[speaker002:] in U doctor speech data SRI Hub five, there's this, uh, Hub five training set.
[speaker005:] Mmm. Mm hmm.
[speaker002:] Now, is that the long training set there?
[speaker005:] Mm hmm. That's everything. Yeah,
[speaker002:] OK.
[speaker005:] so [disfmarker] So, I can give you a list of the short [pause] version.
[speaker002:] OK. I th I think you already did, actually.
[speaker005:] So you can [disfmarker] Oh, OK.
[speaker002:] OK. And so, say the Macrophone files that are included in this short training, are just a subset of the Macrophone files. Right?
[speaker005:] That's right.
[speaker002:] OK. So [disfmarker] so, um [disfmarker] when you [disfmarker] You did some TI digits t t experiments training on Macrophone.
[speaker005:] Yeah.
[speaker002:] Um. But that's not necessarily any less data [pause] than the SRI Hub five set. It's not a [disfmarker] it's not a subset of the short SRI Hu Hub five set. Right?
[speaker005:] Um [disfmarker] Wel No, it is. Th the [disfmarker] Sorry. Um. Can you repeat the question?
[speaker002:] Uh, whe when you trained on Macrophone, um, to do those digits experiments, did you use the entire Macrophone corpus?
[speaker005:] There wa it is a subse Yeah. No. Only the portion that was in the Hub five training set.
[speaker002:] Oh. OK.
[speaker003:] That was in the Hub five small training set?
[speaker005:] Well, the Hub five small training set contains as much Macrophone as the large training set, for historical reasons.
[speaker003:] Yes.
[speaker005:] Yeah.
[speaker002:] OK OK. So [disfmarker] Um [disfmarker]
[speaker005:] So, do you have that processed there, then [disfmarker] right? Because you already did [disfmarker] did y didn't you already do that experiment?
[speaker002:] I [comment] I [disfmarker] I got confused, cuz I thought [disfmarker] I thought you were using [vocalsound] the whole Macrophone set.
[speaker005:] Mm hmm. No.
[speaker002:] Um. OK. Well, if [disfmarker] if [disfmarker] if I just need to use that subset, I [disfmarker] I can get it processed. I actually [pause] got [disfmarker] I think I got f into it before, and then I thought I was doing the wrong thing and I stopped. And it shouldn it shouldn't take that long to do.
[speaker005:] Mmm. OK. Right.
[speaker003:] OK.
[speaker005:] And you need only the males. So.
[speaker002:] Oh, OK.
[speaker001:] So, basically, Dave, so y for you to get your processing going, you need the list of the, uh, wave Well, I guess it'll dep you'll [disfmarker] you'll [disfmarker] we'll need to get the segmentations.
[speaker002:] Yeah.
[speaker001:] Figure out whether we're using the new or the old [pause] from Don. And then, uh, from that you need the [disfmarker] from the segmentations you'll have the list of wavefiles that the short, uh, set is trained on. And then [vocalsound] you'll need disk space. And once you've got [pause] those things, then you can start your processing. Right?
[speaker002:] Yeah.
[speaker001:] OK.
[speaker005:] Does this [disfmarker] th this [disfmarker]? It it's sort of [disfmarker] f f not very nice to use the small training set for another reason, which is that the [vocalsound] you also are losing on [disfmarker] Again, because you don't use [pause] all the data you have for one speaker. So, the normalizations you compute for your training speakers will be, uh, crummier [pause] than they would in the large training set. So, [vocalsound] um, I have to [disfmarker] So, to make it really a matching experiment, I have to find [disfmarker] uh, I have to use short models that were trained on normalizations that were also only estimated on the [pause] short set. Which is, uh [disfmarker]
[speaker001:] Do you have this, uh [disfmarker]?
[speaker005:] I think so. I've [disfmarker] I [disfmarker] I have to check. In any case, I could retrain short models within [vocalsound] a few hours actually at [disfmarker] if I use machines at SRI.
[speaker003:] I wonder about that, though. I mean, because all we're doing [disfmarker] The only reason we're using the short training set is [disfmarker] is for speed. And there [disfmarker] we're not really making any claims about using a smaller training set.
[speaker005:] Yeah.
[speaker003:] So as long as we're not using any testing data from [disfmarker]
[speaker005:] No. But the thing is, if [disfmarker] if we used [disfmarker] if we used the whole training set for normalizations, then [pause] David would have to process much more data,
[speaker003:] Yeah.
[speaker005:] which [disfmarker] That's a [disfmarker] that's one bottleneck, for us right, in terms of get
[speaker003:] Oh, you mean for [disfmarker] for his normalizations.
[speaker005:] Yeah.
[speaker003:] Oh, oh, oh. I'm sorry.
[speaker005:] Right. So, you wanna do the exact same thing,
[speaker003:] Right.
[speaker005:] or else [pause] you'll have apples and oranges.
[speaker003:] Yeah.
[speaker005:] So. It doesn't make [disfmarker] I don't think it makes that much of a difference. It's just this little detail that if you can take care of that, then you should.
[speaker003:] Mm hmm.
[speaker005:] I [disfmarker] I think I have [disfmarker] I have the models, I have [disfmarker] [comment] I have, um [disfmarker] let's see, um [disfmarker] Yeah. And if not I can retrain [pause] those models very quickly.
[speaker001:] Oh, there's [disfmarker] there's one other issue, uh, and that is that Dave throws out speakers that have less than twelve seconds of training data. And he said there were a few in the Macrophone set like that. So, do we need to wait to find out who he's gonna throw out, so that we create a new set of short models that don't include those speakers?
[speaker005:] Uh [disfmarker]
[speaker003:] Say this again? Sorry I missed it.
[speaker001:] So i in [disfmarker] [vocalsound] Th the problem is that if we proceed like we just described, um, [vocalsound] when he goes to m um, [vocalsound] create the new s training [pause] data with his processing, he throws out some speakers. So the t two training sets won't be identical.
[speaker005:] Yeah.
[speaker003:] He throws out some speakers that are [disfmarker] that are very small.
[speaker001:] Yeah. Have just a little bit.
[speaker005:] Yeah. I don't think it'll make a [disfmarker] matter.
[speaker003:] Yeah. I think there was only a few
[speaker001:] OK.
[speaker003:] we thought to be the case.
[speaker005:] In fact, I thought about throwing those out too,
[speaker003:] Righ -?
[speaker005:] because when I heard how little speech there was for some of them, I thought they can only hurt your models, because they're [disfmarker] again their normalizations will be all [disfmarker] all [disfmarker] all over the map,
[speaker001:] Yeah.
[speaker003:] Right.
[speaker005:] and you won't get very [disfmarker] very clean models from them, anyhow. So.
[speaker001:] So you think it's OK, then?
[speaker005:] Yeah. In fact, if [disfmarker] if you wanna do this, uh, to speed things up, um, you [disfmarker] we can leave out the Macrophone data altogether. That hurt [disfmarker] Actually. Oh, no. Sorry. Not in the short. Then you have too little data. OK. Sorry. Forget that. Um, When you use [disfmarker] when you go to the large training set, then [pause] leaving out Macrophone actually sometimes helps you, because it's [disfmarker] it it's just not relevant to the [disfmarker] to the meeting and [disfmarker] or to conversational speech anyway.
[speaker001:] It's read.
[speaker005:] OK. Yeah. Leave it out, and [disfmarker] [vocalsound] Um, in the event that I retrain the short models, um, why don't you give me a list of the files that you throw out, and I [disfmarker] I'll throw them out, too.
[speaker002:] Right.
[speaker005:] And then we have complete [disfmarker] [vocalsound] completely [pause] identical training conditions.
[speaker001:] We sh M Right. A actually you should be able to figure out, Dave, right, once you know the segmentations, uh, who you're going to [disfmarker] which speakers will get left out, even before you run your processing.
[speaker002:] Mm hmm.
[speaker001:] Right?
[speaker002:] The segmentations?
[speaker001:] Yeah. The segmentations from Don?
[speaker005:] But th the segmentations are only [disfmarker] they only affect the test set.
[speaker001:] Once we [disfmarker]?
[speaker002:] Oh.
[speaker005:] We're talking about the training speakers.
[speaker001:] Ah. Right, right, right, right, right.
[speaker003:] No. The training set he could go through right now, and see how [disfmarker] how long the [disfmarker]
[speaker005:] Right.
[speaker001:] Right. But I'm just wondering how long it will take to get that information.
[speaker002:] Um [disfmarker]
[speaker005:] He already has the in you already have the information.
[speaker002:] I [disfmarker] I have i I have it for [disfmarker] for Macrophone, um, already, I think,
[speaker005:] Right?
[speaker002:] and, um, I think by tomorrow I'll have it for th for the rest.
[speaker005:] Mm hmm. OK. Alright.
[speaker001:] OK.
[speaker005:] OK.
[speaker003:] At that one? [vocalsound] Maybe. Uh [disfmarker] Been looking at synthesizers?
[speaker004:] Synthesizers?
[speaker003:] You were looking at Festival.
[speaker004:] Yeah, yeah. I was doing something for [pause] the SmartKom data collectioners.
[speaker003:] Yeah.
[speaker004:] Robert has taken his laptop t back to Germany so we needed a new synthesis machine. And we have now a SUN workstation in the library, which does the synthesis and the Festival speech s s system is running on
[speaker001:] Sorry, Robert did what? Robert took what?
[speaker004:] His laptop where, uh [disfmarker] which we used for the SmartKom data collection, for the synthesis.
[speaker001:] Uh huh.
[speaker004:] And so he took it to Germany and [pause] so we couldn't do any data collection.
[speaker001:] Oh. Is he gone now?
[speaker004:] Uh [disfmarker] No. He's just gone to a SmartKom workshop.
[speaker001:] Oh. Oh, oh. Oh, OK.
[speaker004:] And so, we have now the SUN in the library which can do that. And I looked into the ts F zero thing and talked to [disfmarker] to Liz, and it seems that it's quite s what she wants, but we'll [vocalsound] have to think about the [disfmarker] the energy thing [disfmarker] uh, what we wanna do.
[speaker003:] Right. This was a business about, uh, um, coming up with something that [disfmarker] that was purely prosodic.
[speaker005:] Mm hmm.
[speaker003:] And so, uh, we're just gonna use a pitch detector, drive a synthesizer, and since it doesn't have a hook in it for, uh, modifying energy, we'll have a little box at the output that'll modify the energy. So [disfmarker]
[speaker005:] OK.
[speaker003:] So. Rrrrr rrrrm Something like that.
[speaker001:] Are you [disfmarker] are you interfacing to that thing with the C plus plus Or are you [disfmarker] is there another interface that you use?
[speaker004:] For w for Festival?
[speaker001:] Yeah.
[speaker004:] Uh, you can just [pause] use it from the [disfmarker] Yeah. Basically from the command line, and the defining the phones, whatever you want to have synthesized,
[speaker001:] Oh.
[speaker004:] and [vocalsound] give the F zero targets,
[speaker001:] And it saves it in a [disfmarker]
[speaker004:] and then ts get [disfmarker] gives out a [disfmarker] a waveform, and I [disfmarker] I want to manipulate the [disfmarker] the waveform, then.
[speaker001:] I see. Oh, that's neat.
[speaker003:] OK? Digits?
[speaker001:] Ah. OK.
[speaker003:] That's all folks.