PhD A: OK, we're going.PhD C: Eight, eightPhD D: This is three.PhD C: Three.PhD D: Yep. Yep.Professor B: Test. Hmm. Let's see. Move it bit. Test Test OK, I guess it's alright. So, let's see. Yeah, Barry's not here and Dave's not here. Um, I can say about {disfmarker} just q just quickly to get through it, that Dave and I submitted this ASRU.PhD A: This is for ASRU.Professor B: Yeah. So. Um. Yeah, it's {disfmarker} it's interesting. I mean, basically we're dealing with rever reverberation, and, um, when we deal with pure reverberation, the technique he's using works really, really well. Uh, and when they had the reverberation here, uh, we'll measure the signal - to - noise ratio and it's, uh, about nine DB. So,PhD D: Hmm.Professor B: um,PhD A: You mean, from the actual, uh, recordingsProfessor B: a fair amount of {disfmarker}PhD D: kPhD A: It's nine DBProfessor B: Yeah. Yeah. Um {disfmarker} And actually it brought up a question which may be relevant to the Aurora stuff too. Um, I know that when you figured out the filters that we're using for the Mel scale, there was some experimentation that went on at {disfmarker} at, uh {disfmarker} at OGI. Um, but one of the differences that we found between the two systems that we were using, {comment} the {disfmarker} the Aurora HTK system baseline system {comment} and the system that we were {disfmarker} the {disfmarker} the uh, other system we were using, the uh, the SRI system, was that the SRI system had maybe a, um, hundred hertz high - pass. And the, uh, Aurora HTK, it was like twenty.PhD D: Yep. S sixty - four.Professor B: Uh.PhD D: S sixty - four.Professor B: Sixty - four Uh.PhD D: Yeah, if you're using the baseline.Professor B: Is that the ba band centerPhD D: No, the edge.Professor B: The edge is really, uh, sixty - fourPhD D: Yeah.Professor B: For some reason, uh, Dave thought it was twenty,PhD D: So the, uh, center would be somewhere around like hundredProfessor B: but.PhD D: and {disfmarker} hundred and {disfmarker} hundred {disfmarker} hundred and {disfmarker} maybe {disfmarker} it's like {disfmarker} fi hundred hertz.Professor B: But do you know, for instance, h how far down it would be at twenty hertz What the {disfmarker} how much rejection would there be at twenty hertz, let's sayPhD D: At twenty hertz.Professor B: Yeah, any idea what the curve looks likePhD D: Twenty hertz frequency {disfmarker} Oh, it's {disfmarker} it's zero at twenty hertz, right The filterPhD C: Yea - actually, the left edge of the first filter is at sixty - four.PhD D: Sixt - s sixty - four.PhD C: So {disfmarker}PhD D: So anything less than sixty - four is zero.PhD C: Mmm.Professor B: It's actually set to zero What kind of filter is thatPhD C: Yeah.PhD D: Yeah.Professor B: Is this {disfmarker} oh, from the {disfmarker} from {disfmarker}PhD C: It {disfmarker} This is the filter bank in the frequency domain that starts at sixty - four.Professor B: Oh, so you, uh {disfmarker} so you really set it to zero, the FFTPhD D: Yeah,PhD C: Yeah.PhD D: yeah. So it's {disfmarker} it's a weight on the ball spectrum. Triangular weighting.Professor B: Right. OK. Um {disfmarker} OK. So that's {disfmarker} that's a little different than Dave thought, I think. But {disfmarker} but, um, still, it's possible that we're getting in some more noise. So I wonder, is it {disfmarker} @ @ Was there {disfmarker} their experimentation with, uh, say, throwing away that filter or something And, uh {disfmarker}PhD D: Uh, throwing away the firstProfessor B: Yeah.PhD D: Um, yeah, we {disfmarker} we've tried including the full {disfmarker} full bank. Right From zero to four K.PhD C: Mm - hmm.PhD D: And that's always worse than using sixty - four hertz.Professor B: Right, but the question is, whether sixty - four hertz is {disfmarker} is, uh, too, uh, low.PhD D: Yeah, I mean, make it a hundred or soProfessor B: Yeah.PhD D: I t I think I've tried a hundred and it was more or less the same, or slightly worse.Professor B: On what test setPhD D: On the same, uh, SpeechDat - Car, Aurora.Professor B: Um, it was on the SpeechDat - Car.PhD D: Yeah. So I tried a hundred to four K. Yeah.Professor B: Um,PhD D: So it was {disfmarker}Professor B: and on {disfmarker} and on the, um, um, {vocalsound} TI - digits alsoPhD D: No, no, no. I think I just tried it on SpeechDat - Car.Professor B: Mmm. That'd be something to look at sometime because what, um, eh, he was looking at was performance in this room.PhD D: Mm - hmm.Professor B: Would that be more like {disfmarker} Well, you'd think that'd be more like SpeechDat - Car, I guess, in terms of the noise. The SpeechDat - Car is more, uh, sort of roughly stationary, a lot of it. And {disfmarker} and TI - digits maybe is not so much as {disfmarker}PhD D: Yeah.PhD C: Mm - hmm.Professor B: Yeah.PhD D: Yeah.Professor B: Mm - hmm. OK. Well, maybe it's not a big deal. But, um {disfmarker} Anyway, that was just something we wondered about. But, um, uh, certainly a lot of the noise, uh, is, uh, below a hundred hertz. Uh, the signal - to - noise ratio, you know, looks a fair amount better if you {disfmarker} if you high - pass filter it from this room.PhD D: Yeah.Professor B: But, um {disfmarker} but it's still pretty noisy. Even {disfmarker} even for a hundred hertz up, it's {disfmarker} it's still fairly noisy. The signal - to - noise ratio is {disfmarker} is {disfmarker} is actually still pretty bad.PhD C: Mm - hmm.PhD A: Hmm.Professor B: So, um, I mean, the main {disfmarker} the {disfmarker} the {disfmarker}PhD A: So that's on th that's on the f the far field ones though, right Yeah.Professor B: Yeah, that's on the far field. Yeah, the near field's pretty good.PhD A: So wha what is, uh {disfmarker} what's causing thatProfessor B: Well, we got a {disfmarker} a video projector in here, uh, and, uh {disfmarker} which we keep on during every {disfmarker} every session we record,PhD A: Yeah.Professor B: which, you know, I {disfmarker} I {disfmarker} w we were aware ofPhD A: Uh - huh.Professor B: but {disfmarker} but we thought it wasn't a bad thing.PhD A: Yeah.Professor B: I mean, that's a nice noise source. Uh, and there's also the, uh {disfmarker} uh, air conditioning.PhD A: Hmm.Professor B: Which, uh, you know, is a pretty low frequency kind of thing.PhD A: Mm - hmm.Professor B: But {disfmarker} but, uh {disfmarker} So, those are {disfmarker} those are major components, I think,PhD A: I see.Professor B: uh, for the stationary kind of stuff.PhD A: Mmm.Professor B: Um, but, um, it, uh {disfmarker} I guess, I {disfmarker} maybe I said this last week too but it {disfmarker} it {disfmarker} it really became apparent to us that we need to {disfmarker} to take account of noise. And, uh, so I think when {disfmarker} when he gets done with his prelim study I think {vocalsound} one of the next things we'd want to do is to take this, uh {disfmarker} uh, noise, uh, processing stuff and {disfmarker} and, uh {disfmarker} uh, synthesize some speech from it.PhD A: When are his prelimsProfessor B: And then {disfmarker} Um, I think in about, um, a little less than two weeks.PhD A: Oh. Wow.Professor B: Yeah. Yeah. So. Uh, it might even be sooner. Uh, let's see, this is the sixteenth, seventeenth Yeah, I don't know if he's before {disfmarker} It might even be in a week.PhD A: So, IProfessor B: A week,PhD A: Huh. I {disfmarker} I guessed that they were gonna do it some time during the semesterProfessor B: week and a half.PhD A: but they'll do it any time, huhProfessor B: They seem to be {disfmarker} Well, the semester actually is starting up.PhD A: Is it alreadyProfessor B: Yeah, the semester's late {disfmarker} late August they start here.PhD A: Yikes.Professor B: So they do it right at the beginning of the semester.PhD A: Yeah.Professor B: Yeah. So, uh {disfmarker} Yep. I mean, that {disfmarker} that was sort of one {disfmarker} I mean, the overall results seemed to be first place in {disfmarker} in {disfmarker} in the case of either, um, artificial reverberation or a modest sized training set. Uh, either way, uh, i uh, it helped a lot. And {disfmarker} But if you had a {disfmarker} a really big training set, a recognizer, uh, system that was capable of taking advantage of a really large training set {disfmarker} I thought that {disfmarker} One thing with the HTK is that is has the {disfmarker} as we're using {disfmarker} the configuration we're using is w s is {disfmarker} being bound by the terms of Aurora, we have all those parameters just set as they are. So even if we had a hundred times as much data, we wouldn't go out to, you know, ten or t or a hundred times as many Gaussians or anything. So, um, it's kind of hard to take advantage of {disfmarker} of {disfmarker} of big chunks of data. Uh, whereas the other one does sort of expand as you have more training data.PhD C: Mm - hmm.PhD D: Mmm, yeah.Professor B: It does it automatically, actually. And so, um, uh, that one really benefited from the larger set. And it was also a diverse set with different noises and so forth. Uh, so, um, that, uh {disfmarker} that seemed to be {disfmarker} So, if you have that {disfmarker} that better recognizer that can {disfmarker} that can build up more parameters, and if you, um, have the natural room, which in this case has a p a pretty bad signal - to - noise ratio, then in that case, um, the right thing to do is just do {disfmarker} u use speaker adaptation. And {disfmarker} and not bother with {disfmarker} with this acoustic, uh, processing. But I think that that would not be true if we did some explicit noise - processing as well as, uh, the convolutional kind of things we were doing.PhD C: Mm - hmm.Professor B: So. That's sort of what we found.PhD D: Hmm.PhD A: I, um {disfmarker} {vocalsound} uh, started working on the uh {disfmarker} Mississippi State recognizer. So, I got in touch with Joe and {disfmarker} and, uh, from your email and things like that.PhD D: Oh, OK.PhD A: And, uh, they added me to the list {disfmarker} uh, the mailing list.PhD D: OK, great.PhD A: And he gave me all of the pointers and everything that I needed. And so I downloaded the, um {disfmarker} There were two things, uh, that they had to download. One was the, uh, I guess the software. And another wad {disfmarker} was a, um, sort of like a sample {disfmarker} a sample run. So I downloaded the software and compiled all of that. And it compiled fine.PhD D: Eight.PhD A: No problems.PhD D: Oh, eh, great.PhD A: And, um, I grabbed the sample stuff but I haven't, uh, compiled it.PhD D: That sample was released only yesterday or the day before, rightPhD A: No {disfmarker} Well, I haven't grabbed that one yet. So there's two.PhD D: Oh, there is another short sample set {disfmarker}PhD A: There was another short one, yeah.PhD D: o o sample.PhD A: And so I haven't grabbed the latest one that he just, uh, put out yet.PhD D: OK. Oh, OK. F Yeah, OK.PhD A: So. Um, but, the software seemed to compile fine and everything, so. And, um, So.Professor B: Is there any word yet about the issues about, um, adjustments for different feature sets or anythingPhD A: No, I {disfmarker} I d You asked me to write to him and I think I forgot to ask him about that. Or if I did ask him, he didn't reply.Professor B: Yeah.PhD A: I {disfmarker} I don't remember yet. Uh, I'll {disfmarker} I'll d I'll double check that and ask him again.Professor B: Yeah. Yeah, it's like that {disfmarker} that could r turn out to be an important issue for us.PhD D: Hmm. Mmm.PhD A: Yeah. Yeah.Professor B: Yeah.PhD D: Cuz they have it {disfmarker}PhD A: Maybe I'll send it to the list. Yeah.PhD D: Cuz they have, uh, already frozen those in i insertion penalties and all those stuff is what {disfmarker} I feel. Because they have this document explaining the recognizer.PhD A: Uh - huh.PhD D: And they have these tables with, uh, various language model weights, insertion penalties.PhD A: OK, I haven't seen that one yet.PhD D: uPhD A: So.PhD D: Uh, it's th it's there on that web.PhD A: OK.PhD D: And, uh, on that, I mean, they have run some experiments using various insertion penalties and all those {disfmarker}PhD A: And so they've picked {disfmarker} the values.PhD D: Yeah, I think they pi pPhD A: Oh, OK.PhD D: yeah, they picked the values from {disfmarker}PhD A: OK.Professor B: For r w what test setPhD D: Uh, p the one that they have reported is a NIST evaluation, Wall Street Journal.Professor B: But that has nothing to do with what we're testing on, rightPhD C: Mm - hmm.PhD D: You know. No. So they're, like {disfmarker} um {disfmarker} So they are actually trying to, uh, fix that {disfmarker} those values using the clean, uh, training part of the Wall Street Journal. Which is {disfmarker} I mean, the Aurora. Aurora has a clean subset.Professor B: Right.PhD D: I mean, they want to train it and then this {disfmarker} they're going to run some evaluations.Professor B: So they're set they're setting it based on thatPhD D: Yeah.Professor B: OK. So now, we may come back to the situation where we may be looking for a modification of the features to account for the fact that we can't modify these parameters.PhD A: Yeah.Professor B: But, um,PhD D: Yeah.Professor B: uh {disfmarker} but it's still worth, I think, just {disfmarker} since {disfmarker} you know, just chatting with Joe about the issue.PhD A: Yeah, OK. Do you think that's something I should just send to himProfessor B: Um {disfmarker}PhD A: or do you think I should send it to this {disfmarker} there's an {disfmarker} a m a mailing list.Professor B: Well, it's not a secret. I mean, we're, you know, certainly willing to talk about it with everybody, but I think {disfmarker} I think that, um {disfmarker} um, it's probably best to start talking with him just to {disfmarker}PhD A: OK.Professor B: Uh @ @ {comment} you know, it's a dialogue between two of you about what {disfmarker} you know, what does he think about this and what {disfmarker} what {disfmarker} you know {disfmarker} what could be done about it.PhD A: Yeah. OK.Professor B: Um, if you get ten people in {disfmarker} involved in it there'll be a lot of perspectives based on, you know, how {disfmarker}PhD A: Yeah.Professor B: you know.PhD A: Right.Professor B: Uh {disfmarker} But, I mean, I think it all should come up eventually,PhD A: OK.Professor B: but if {disfmarker} if {disfmarker} if there is any, uh, uh, way to move in {disfmarker} a way that would {disfmarker} that would, you know, be more open to different kinds of features. But if {disfmarker} if, uh {disfmarker} if there isn't, and it's just kind of shut down and {disfmarker} and then also there's probably not worthwhile bringing it into a larger forum where {disfmarker} where political issues will come in.PhD A: Yeah. OK.PhD D: Oh. So this is now {disfmarker} it's {disfmarker} it's compiled under SolarisPhD A: Yeah.PhD D: Yeah, OK.PhD A: Yep.PhD D: Because he {disfmarker} there was some mail r saying that it's {disfmarker} may not be stable for Linux and all those.PhD A: Yeah. Yeah, i that was a particular version.PhD D: SUSIPhD A: Yeah, SUSI or whatever it wasPhD D: yeah. Yeah, yeah.PhD A: but we don't have that.PhD D: Yeah, OK.PhD A: So. Should be OK.PhD D: OK, that's fine.PhD A: Yeah, it compiled fine actually.PhD D: Yeah.PhD A: No {disfmarker} no errors. Nothing. So.Professor B: Uh, this is slightly off topicPhD D: That's good.Professor B: but, uh, I noticed, just glancing at the, uh, Hopkins workshop, uh, web site that, uh, um {disfmarker} one of the thing I don't know {disfmarker} Well, we'll see how much they accomplish, but one of the things that they were trying to do in the graphical models thing was to put together a {disfmarker} a, uh, tool kit for doing, uh r um, arbitrary graphical models for, uh, speech recognition.PhD A: Hmm.Professor B: So {disfmarker} And Jeff, uh {disfmarker} the two Jeffs werePhD A: Who's the second JeffProfessor B: Uh {disfmarker} Oh, uh, do you know Geoff ZweigPhD A: No.Professor B: Oh. Uh, he {disfmarker} he, uh {disfmarker} he was here for a couple yearsPhD A: Oh, OK.Professor B: and he, uh {disfmarker} got his PHD. He {disfmarker} And he's, uh, been at IBM for the last couple years.PhD A: Oh, OK.Professor B: So.PhD A: Wow. That would be neat.Professor B: Uh, so he did {disfmarker} he did his PHD on dynamic Bayes - nets, uh, for {disfmarker} for speech recognition. He had some continuity built into the model, presumably to handle some, um, inertia in the {disfmarker} in the production system, and, um {disfmarker}PhD A: Hmm.Professor B: So.PhD D: Hmm.PhD C: Um, I've been playing with, first, the, um, VAD. Um, {vocalsound} so it's exactly the same approach, but the features that the VAD neural network use are, uh, MFCC after noise compensation. Oh, I think I have the results.Professor B: What was it using beforePhD C: Before it was just P LPhD D:PhD C: So.PhD D: Yeah, it was actually {disfmarker} No. Not {disfmarker} I mean, it was just the noisy features I guess.PhD C: Yeah,PhD D: Yeah, yeah, yeah,PhD C: noisy {disfmarker} noisy features.PhD D: not compensated.PhD C: Um {disfmarker} This is what we get after {disfmarker} This {disfmarker} So, actually, we, yeah, here the features are noise compensated and there is also the LDA filter. Um, and then it's a pretty small neural network which use, um, {vocalsound} nine frames of {disfmarker} of six features from C - zero to C - fives, plus the first derivatives. And it has one hundred hidden units.PhD A: Is that nine frames u s uh, centered around the current frame Or {disfmarker}PhD C: Yeah. Mm - hmm.Professor B: S so, I'm {disfmarker} I'm sorry, there's {disfmarker} there's {disfmarker} there's how many {disfmarker} how many inputsPhD C: So it's twelve times nine.Professor B: Twelve times nine inputs, and a hundred, uh, hidden.PhD C: Hidden andPhD D: Two outputs.PhD C: two outputs.Professor B: Two outputs. OK. So I guess about eleven thousand parameters, which {disfmarker} actually shouldn't be a problem, even in {disfmarker} in small phones. Yeah.PhD C: Mm - hmm.PhD A: So, I'm {disfmarker} I'm {disfmarker} s so what is different between this and {disfmarker} and what you {disfmarker}PhD C: It should be OK. So the previous syst It's based on the system that has a fifty - three point sixty - six percent improvement. It's the same system. The only thing that changed is the n a p eh {disfmarker} a es the estimation of the silence probabilities.PhD A: Ah. OK.PhD C: Which now is based on, uh, cleaned features.Professor B: And, it's a l it's a lot better.PhD A: Wow.PhD C: Yeah.Professor B: That's great.PhD C: Um {disfmarker} So it's {disfmarker} it's not bad, but the problem is still that the latency is too large.Professor B: What's the latencyPhD C: Because {disfmarker} um {disfmarker} the {disfmarker} the latency of the VAD is two hundred and twenty milliseconds. And, uh, the VAD is used uh, i for on - line normalization, and it's used before the delta computation. So if you add these components it goes t to a hundred and seventy, rightProfessor B: I {disfmarker} I'm confused. You started off with two - twenty and you ended up with one - seventyPhD C: With two an two hundred and seventy.Professor B: Two - seventy.PhD C: If {disfmarker} Yeah, if you add the c delta comp delta computationProfessor B: Oh.PhD C: which is done afterwards. Um {disfmarker}Professor B: So it's two - twenty. I the is this {disfmarker} are these twenty - millisecond frames Is that why Is it after downsampling or {disfmarker}PhD C: The two - twenty is one hundred milliseconds for the um {disfmarker} No, it's forty milliseconds for t for the, uh, uh, cleaning of the speech. Um {disfmarker} then there is, um, the neural network which use nine frames. So it adds forty milliseconds.Professor B: a OK.PhD C: Um, after that, um, you have the um, filtering of the silence probabilities. Which is a million filter it, and it creates a one hundred milliseconds delay. So, um {disfmarker}Professor B:PhD D: Plus there is a delta at the input.PhD C: Yeah, and there is the delta at the input which is,Professor B: One hundred milliseconds for smoothing.PhD C: um {disfmarker} So it's {disfmarker} @ @ {disfmarker}Professor B: Uh, median.PhD C:PhD D: It's like forty plus {disfmarker} forty {disfmarker} plus {disfmarker}Professor B: And then forty {disfmarker}PhD C: Mmm. Forty {disfmarker} This forty plus twenty, plus one hundred.Professor B: forty pPhD C: Uh {disfmarker}PhD D: So it's two hundred actually.PhD C: Yeah, there are twenty that comes from {disfmarker} There is ten that comes from the LDA filters also. RightPhD D: Oh, OK.PhD C: Uh, so it's two hundred and ten, yeah.PhD D: If you are using {disfmarker}Professor B: Uh {disfmarker}PhD C: Plus the frame,PhD D: t If you are using three frames {disfmarker}PhD C: so it's two - twenty.PhD D: If you are phrasing f {comment} using three frames, it is thirty here for delta.PhD C: Yeah, I think it's {disfmarker} it's five frames, but.PhD D: So five frames, that's twenty. OK, so it's who un {comment} two hundred and ten.Professor B: Uh, p Wait a minute. It's forty {disfmarker} {vocalsound} forty for the {disfmarker} for the cleaning of the speech,PhD C: So. Forty cleaning.Professor B: forty for the I N {disfmarker} ANN, a hundred for the smoothing.PhD C: Yeah.Professor B: Well, but at ten {disfmarker},PhD C: Twenty for the delta.Professor B: Twenty for delta.PhD D: At th {nonvocalsound} At the input. I mean, that's at the input to the net.PhD C: Yeah.Professor B: Delta at input to netPhD D: And there iPhD C: Yeah.PhD D: Yeah. So it's like s five, six cepstrum plus delta at nine {disfmarker} nine frames of {disfmarker}Professor B: And then ten milliseconds for {disfmarker}PhD D: Fi - There's an LDA filter.Professor B: ten milliseconds for LDA filter, and t and ten {disfmarker} another ten milliseconds you said for the framePhD C: For the frame I guess. I computed two - twenty {disfmarker} Yeah, well, it's {disfmarker} I guess it's for the fr {disfmarker} the {disfmarker}Professor B: OK. And then there's delta besides thatPhD C: So this is the features that are used by our network and then afterwards, you have to compute the delta on the, uh, main feature stream,Professor B: OK.PhD C: which is um, delta and double - deltas, which is fifty milliseconds.Professor B: Yeah. No, I mean, the {disfmarker} after the noise part, the forty {disfmarker} the {disfmarker} the other hundred and eighty {disfmarker} Well, I mean, Wait a minute. Some of this is, uh {disfmarker} is, uh {disfmarker} is in parallel, isn't it I mean, the LDA {disfmarker} Oh, you have the LDA as part of the V D - uh, VAD Or {disfmarker}PhD C: The VAD use, uh, LDA filtered features also.Professor B: Oh, it doesPhD C: Mm - hmm.Professor B: Ah. So in that case there isn't too much in parallel. Uh {disfmarker}PhD C: No. There is, um, just downsampling, upsampling, and the LDA.Professor B: Um, so the delta at the end is how muchPhD C: It's fifty.PhD D: It's {disfmarker}Professor B: Fifty. Alright. So {disfmarker}PhD C: But well, we could probably put the delta, um, {vocalsound} before on - line normalization. It should not that make a big difference,PhD A: What if you used a smaller window for the deltaPhD C: because {disfmarker}PhD A: Could that help a little bit I mean, I guess there's a lot of things you could do to {disfmarker}PhD C: Yeah.Professor B: Yeah.PhD C: Yeah,Professor B: SoPhD C: but, nnn {disfmarker}Professor B: Yeah. So if you {disfmarker} if you put the delta before the, uh, ana on - line {disfmarker} If {disfmarker} Yeah {disfmarker}PhD C: Mm - hmm.Professor B: uh {disfmarker} then {disfmarker} then it could go in parallel.PhD C: Cuz iProfessor B: And then y then you don't have that additive {disfmarker}PhD C: Yeah,PhD D: Yep.PhD C: cuz the time constant of the on - line normalization is pretty long compared to the delta window,Professor B: OK.PhD C: so. It should not make {disfmarker}Professor B: OK. And you ought to be able to shove tw, uh {disfmarker} sh uh {disfmarker} pull off twenty milliseconds from somewhere else to get it under two hundred, right I mean {disfmarker}PhD A: Is two hundred the dProfessor B: The hundred millaPhD C: Mm - hmm.Professor B: mill a hundred milliseconds for smoothing is sort of an arbitrary amount. It could be eighty and {disfmarker} and probably do @ @ {disfmarker}PhD C: Yeah,PhD A: i a hunPhD C: yeah.PhD A: uh {disfmarker} Wh - what's the baseline you need to be under Two hundredProfessor B: Well, we don't know. They're still arguing about it.PhD C:PhD A: Oh.Professor B: I mean, if it's two {disfmarker} if {disfmarker} if it's, uh {disfmarker} if it's two - fifty, then we could keep the delta where it is if we shaved off twenty. If it's two hundred, if we shaved off twenty, we could {disfmarker} we could, uh, meet it by moving the delta back.PhD A: So, how do you know that what you have is too much if they're still decidingProfessor B: Uh, we don't, but it's just {disfmarker} I mean, the main thing is that since that we got burned last time, and {disfmarker} you know, by not worrying about it very much, we're just staying conscious of it.PhD A: Uh - huh. Oh, OK, I see.Professor B: And so, th I mean, if {disfmarker} if {disfmarker} if a week before we have to be done someone says," Well, you have to have fifty milliseconds less than you have now" , it would be pretty frantic around here. So {disfmarker}PhD A: Ah, OK.Professor B: Uh {disfmarker}PhD A: But still, that's {disfmarker} that's a pretty big, uh, win. And it doesn't seem like you're {disfmarker} in terms of your delay, you're, uh, that {disfmarker}Professor B: He added a bit on, I guess, because before we were {disfmarker} we were {disfmarker} had {disfmarker} were able to have the noise, uh, stuff, uh, and the LVA be in parallel.PhD C: Hmm.Professor B: And now he's {disfmarker} he's requiring it to be done first.PhD C: Well, but I think the main thing, maybe, is the cleaning of the speech, which takes forty milliseconds or so.Professor B: Right. Well, so you say {disfmarker} let's say ten milliseconds {disfmarker} seconds for the LDA.PhD C: And {disfmarker} and {disfmarker} but {disfmarker} the LDA is, well, pretty short right now.Professor B: Well, ten. And then forty for the other.PhD C: Yeah.PhD D: Yeah, the LDA {disfmarker} LDA {disfmarker} we don't know, is, like {disfmarker} is it very crucial for the features, rightPhD C: No. I just {disfmarker} This is the first try.PhD D: Yeah.Professor B: Right,PhD C: I mean, I {disfmarker} maybe the LDA's not very useful then.Professor B: so you could start pulling back,PhD D: S s hProfessor B: but {disfmarker}PhD D: Yeah,Professor B: But I think you have {disfmarker}PhD D: lProfessor B: I mean, you have twenty for delta computation which y now you're sort of doing twice, right But yo w were you doing that beforePhD C: Mmm. Well, in the proposal, um, the input of the VAD network were just three frames, I think.PhD D: On the {disfmarker} in the {disfmarker} Mm - hmm. Just {disfmarker} Yeah, just the static, no delta.Professor B: Right.PhD C: Uh, static features.Professor B: So, what you have now is fort uh, forty for the {disfmarker} the noise, twenty for the delta, and ten for the LDA. That's seventy milliseconds of stuff which was formerly in parallel,PhD C:Professor B: right So I think,PhD C: Mm - hmm.Professor B: you know, that's {disfmarker} that's the difference as far as the timing, rightPhD C: Yeah.Professor B: Um, and you could experiment with cutting various pieces of these back a bit, but {disfmarker} I mean, we're s we're not {disfmarker} we're not in terrible shape.PhD A: Yeah, that's what it seems like to me. It's pretty good.Professor B: Yeah.PhD C: Mm - hmm.Professor B: It's {disfmarker} it's not like it's adding up to four hundred milliseconds or something.PhD A: Where {disfmarker} where is this {disfmarker} where is this fifty - seven point O two in {disfmarker} in comparison to the last evaluationProfessor B: Well, it's {disfmarker} I think it's better than anything, uh, anybody got.PhD A: Oh, is that rightPhD C: Yeah. The best was fifty - four point five.Professor B: Yeah.PhD D: Point sPhD A: Oh.Professor B: Yeah. UhPhD C: And our system was forty - nine, but with the neural network.PhD A: Wow. So this is almost ten percent.Professor B: With the f with the neural net. Yeah, and r and {disfmarker}PhD C: It wouldPhD D: Yeah, so this is {disfmarker} this is like the first proposal. The proposal - one. It was forty - four, actually.Professor B: Yeah. Yeah. And we still don't have the neural net in. So {disfmarker} so it's {disfmarker}PhD A: Wow.Professor B: You know. So it's {disfmarker} We're {disfmarker} we're doing better.PhD A: This is {disfmarker} this is really good.Professor B: I mean, we're getting better recognition. I mean, I'm sure other people working on this are not sitting still either, but {disfmarker}PhD A: Yeah.Professor B: but {disfmarker} but, uh {disfmarker} Uh, I mean, the important thing is that we learn how to do this better, and, you know. So. Um, Yeah. So, our, um {disfmarker} Yeah, you can see the kind of {disfmarker} kind of numbers that we're having, say, on SpeechDat - Car which is a hard task, cuz it's really, um {disfmarker} I think it's just sort of {disfmarker} sort of reasonable numbers, starting to be. I mean, it's still terriPhD C: Mm - hmm. Yeah, even for a well - matched case it's sixty percent error rate reduction,Professor B: Yeah.PhD C: which is {disfmarker}Professor B: Yeah. Probably half. Good!PhD C: Um, Yeah. So actually, this is in between {vocalsound} what we had with the previous VAD and what Sunil did with an IDL VAD. Which gave sixty - two percent improvement, rightPhD D: Yeah, it's almost that.PhD C: So {disfmarker}PhD D: It's almost an average somewhere around {disfmarker}PhD C: Yeah.PhD D: Yeah.PhD A: What was that Say that last part againPhD C: So, if you use, like, an IDL VAD, uh, for dropping the frames,PhD D: o o Or the best we can get.PhD C: the best that we can get {disfmarker} i That means that we estimate the silence probability on the clean version of the utterances. Then you can go up to sixty - two percent error rate reduction, globally.PhD A: Mmm.PhD C: Mmm {disfmarker} Yeah.PhD A: So that would be even {disfmarker} That wouldn't change this number down here to sixty - twoPhD C: Yeah.Professor B: Yeah. So you {disfmarker} you were getPhD C: If you add a g good v very good VAD, that works as well as a VAD working on clean speech,PhD A: Yeah. Yeah.PhD C: then you wou you would go {disfmarker}PhD A: So that's sort of the best you could hope for.PhD C: Mm - hmm.PhD A: I see.Professor B: Probably. Yeah. So fi si fifty - three is what you were getting with the old VAD.PhD C: Yeah.Professor B: And, uh {disfmarker} and sixty - two with the {disfmarker} the, you know, quote, unquote, cheating VAD. And fifty - seven is what you got with the real VAD.PhD C: Mm - hmm.Professor B: That's great.PhD C: Uh, yeah, the next thing is, I started to play {disfmarker} Well, I don't want to worry too much about the delay, no. Maybe it's better to waitProfessor B: OK.PhD C: for the decisionProfessor B: Yeah.PhD C: from the committee. Uh, but I started to play with the, um, {vocalsound} {vocalsound} uh, tandem neural network. Mmm I just did the configuration that's very similar to what we did for the February proposal. And {disfmarker} Um. So. There is a f a first feature stream that use uh straight MFCC features.Professor B: Mm - hmm.PhD C: Well, these features actually. And the other stream is the output of a neural network, using as input, also, these, um, cleaned MFCC. Um {disfmarker}PhD A: Those are th those are th what is going into the tandem netPhD C: I don't have the comp MmmPhD A: Those twoPhD C: So there is just this feature stream, {comment} the fifteen MFCC plus delta and double - delta.Professor B: No.PhD A: YeahPhD C: Um, so it's {disfmarker} makes forty - five features {comment} that are used as input to the HTK. And then, there is {disfmarker} there are more inputs that comes from the tandem MLP.PhD A: Oh, oh. OK. I see.Professor B: Yeah, h he likes to use them both,PhD A: Uh - huh.Professor B: cuz then it has one part that's discriminative,PhD C: Yeah. Um {disfmarker}Professor B: one part that's not.PhD A: Right. OK.PhD C: So, um, uh, yeah. Right now it seems that {disfmarker} i I just tested on SpeechDat - Car while the experiment are running on your {disfmarker} on TI - digits. Well, it improves on the well - matched and the mismatched conditions, but it get worse on the highly mismatched. Um,PhD A: Compared to these numbersPhD C: Compared to these numbers, yeah. Um,Professor B: yPhD C: like, on the well - match and medium mismatch, the gain is around five percent relative, but it goes down a lot more, like fifteen percent on the HM case.Professor B: You're just using the full ninety featuresPhD C: The {disfmarker}Professor B: Y you have ninety featuresPhD C: i I have, um {disfmarker} From the networks, it's twenty - eight. So {disfmarker}Professor B: And from the other side it's forty - five.PhD C: So, d i It's forty - five.Professor B: So it's {disfmarker} you have seventy - three features,PhD C: Yeah.Professor B: and you're just feeding them like that.PhD C: Yeah.Professor B: There isn't any KLT or anythingPhD C: Mm - hmm. There's a KLT after the neural network, as {disfmarker} as before.PhD A: That's how you get down to twenty - eightPhD C: Yeah.PhD A: Why twenty - eightPhD C: I don't know.PhD A: Oh.PhD C: Uh. It's {disfmarker} i i i It's because it's what we did for the first proposal. We tested, uh, trying to go downPhD A: Ah.Professor B: It's a multiple of seven.PhD C: and Yeah.PhD D: Yeah.PhD C: So {disfmarker} Um.PhD D: Yeah.PhD C: I wanted to do something very similar to the proposal as a first {disfmarker} first try.PhD D: Yeah.PhD A: I see.Professor B: Yeah.PhD A: Yeah. That makes sense.PhD C: But we have to {disfmarker} for sure, we have to go down, because the limit is now sixty features.Professor B: Yeah.PhD C: So, uh, we have to find a way to decrease the number of features. Um {disfmarker}PhD A: So, it seems funny that {disfmarker} I don't know, maybe I don't u quite understand everything, {comment} but that adding features {disfmarker} I guess {disfmarker} I guess if you're keeping the back - end fixed. Maybe that's it. Because it seems like just adding information shouldn't give worse results. But I guess if you're keeping the number of Gaussians fixed in the recognizer, then {disfmarker}Professor B: Well, yeah.PhD C: Mmm.Professor B: But, I mean, just in general, adding information {disfmarker} Suppose the information you added, well, was a really terrible feature and all it brought in was noise.PhD A: Yeah.Professor B: Right So {disfmarker} so, um {disfmarker} Or {disfmarker} or suppose it wasn't completely terrible, but it was completely equivalent to another one feature that you had, except it was noisier.PhD A: Uh - huh.Professor B: Right In that case you wouldn't necessarily expect it to be better at all.PhD A: Oh, yeah, I wasn't necessarily saying it should be better. I'm just surprised that you're getting fifteen percent relative worse on the welProfessor B: Uh - huh.PhD C: But it's worse.Professor B: On the highly mismatched condition.PhD A: On the highly mismatch.PhD C: Yeah, I {disfmarker}PhD A: Yeah.Professor B: So," highly mismatched condition" means that in fact your training is a bad estimate of your test.PhD C: Uh - huh.Professor B: So having {disfmarker} having, uh, a g a l a greater number of features, if they aren't maybe the right features that you use, certainly can e can easily, uh, make things worse. I mean, you're right. If you have {disfmarker} if you have, uh, lots and lots of data, and you have {disfmarker} and your {disfmarker} your {disfmarker} your training is representative of your test, then getting more sources of information should just help. But {disfmarker} but it's {disfmarker} It doesn't necessarily work that way.PhD A: Huh.PhD C: Mm - hmm.Professor B: So I wonder, um, Well, what's your {disfmarker} what's your thought about what to do next with itPhD C: Um, I don't know. I'm surprised, because I expected the neural net to help more when there is more mismatch, as it was the case for the {disfmarker}Professor B: Mm - hmm.PhD D: So, was the training set same as the p the February proposal OK.PhD C: Yeah, it's the same training set, so it's TIMIT with the TI - digits', uh, noises, uh, added.PhD D:Professor B: Mm - hmm.PhD C: Um {disfmarker}Professor B: Well, we might {disfmarker} uh, we might have to experiment with, uh better training sets. Again. But,PhD C: Mm - hmm.Professor B: I {disfmarker} The other thing is, I mean, before you found that was the best configuration, but you might have to retest those things now that we have different {disfmarker} The rest of it is different, right So, um, uh, For instance, what's the effect of just putting the neural net on without the o other {disfmarker} other pathPhD C: Mm - hmm.Professor B: I mean, you know what the straight features do.PhD C: Yeah.Professor B: That gives you this. You know what it does in combination.PhD C: Mm - hmm.Professor B: You don't necessarily know what {disfmarker}PhD A: What if you did the {disfmarker} Would it make sense to do the KLT on the full set of combined features Instead of just on the {disfmarker}PhD C: Yeah. I g I guess. Um. The reason I did it this ways is that in February, it {disfmarker} we {disfmarker} we tested different things like that, so, having two KLT, having just a KLT for a network, or having a global KLT.PhD A: Oh, I see.PhD C: And {disfmarker}PhD A: So you tried the global KLT beforePhD C: Well {disfmarker}PhD A: and it didn't really {disfmarker}PhD C: Yeah. And, uh, th Yeah.PhD A: I see.PhD C: The differences between these configurations were not huge, but {disfmarker} it was marginally better with this configuration.PhD A: Uh - huh. Uh - huh.Professor B: But, yeah, that's obviously another thing to try,PhD C: Um.Professor B: since things are {disfmarker} things are different.PhD C: Mm - hmm. Mm - hmm.Professor B: And I guess if the {disfmarker} These are all {disfmarker} so all of these seventy - three features are going into, um, the, uh {disfmarker} the HMM.PhD C: Yeah.Professor B: And is {disfmarker} are {disfmarker} i i are {disfmarker} are any deltas being computed of tha of themPhD C: Of the straight features, yeah.Professor B: n Not of the {disfmarker}PhD C: So. But n th the, um, tandem features are u used as they are.Professor B: Are not.PhD C: So, yeah, maybe we can add some context from these features also as {disfmarker} Dan did in {disfmarker} in his last work.Professor B: Could. i Yeah, but the other thing I was thinking was, um {disfmarker} Uh, now I lost track of what I was thinking. But.PhD A: What is the {disfmarker} You said there was a limit of sixty features or somethingPhD C: Mm - hmm.PhD A: What's the relation between that limit and the, um, forty - eight {disfmarker} uh, forty eight hundred bits per secondProfessor B: Oh, I know what I was gonna say.PhD C: Um, not {disfmarker} no relation.Professor B: No relation.PhD A: So I {disfmarker} I {disfmarker} I don't understand,PhD C: The f the forty - eight hundred bits is for transmission of some features.PhD A: because i I mean, if you're only using hPhD C: And generally, i it {disfmarker} s allows you to transmit like, fifteen, uh, cepstrum.Professor B: The issue was that, um, this is supposed to be a standard that's then gonna be fed to somebody's recognizer somewhere which might be, you know, it {disfmarker} it might be a concern how many parameters are use {disfmarker} u used and so forth. And so, uh, they felt they wanted to set a limit. So they chose sixty. Some people wanted to use hundreds of parameters and {disfmarker} and that bothered some other people.PhD A: Uh - huh.Professor B: u And so they just chose that. I {disfmarker} I {disfmarker} I think it's kind of r arbitrary too. But {disfmarker} but that's {disfmarker} that's kind of what was chosen. I {disfmarker} I remembered what I was going to say. What I was going to say is that, um, maybe {disfmarker} {vocalsound} maybe with the noise removal, uh, these things are now more correlated. So you have two sets of things that are kind of uncorrelated, uh, within themselves, but they're pretty correlated with one another.PhD C: Mm - hmm.Professor B: And, um, they're being fed into these, uh, variants, only Gaussians and so forth, and {disfmarker} and, uh,PhD C: Mm - hmm.Professor B: so maybe it would be a better idea now than it was before to, uh, have, uh, one KLT over everything, to de - correlate it.PhD C: Mm - hmm. Yeah, I see.Professor B: Maybe. You know.PhD D: What are the S N Rs in the training set, TIMITPhD C: It's, uh, ranging from zero to clean Yeah. From zero to clean.PhD D: Mm - hmm.Professor B: Yeah. So we found this {disfmarker} this, uh {disfmarker} this Macrophone data, and so forth, that we were using for these other experiments, to be pretty good.PhD C: Mm - hmm.Professor B: So that's {disfmarker} i after you explore these other alternatives, that might be another way to start looking, is {disfmarker} is just improving the training set.PhD C: Mm - hmm.Professor B: I mean, we were getting, uh, lots better recognition using that, than {disfmarker} Of course, you do have the problem that, um, u i {comment} we are not able to increase the number of Gaussians, uh, or anything to, uh, uh, to match anything. So we're only improving the training of our feature set, but that's still probably something.PhD A: So you're saying, add the Macrophone data to the training of the neural net The tandem netProfessor B: Yeah, that's the only place that we can train.PhD A: Yeah.Professor B: We can't train the other stuff with anything other than the standard amount,PhD A: Right.Professor B: so. Um, um {disfmarker}PhD A: What {disfmarker} what was it trained on again The one that you usedPhD C: It's TIMIT with noise.PhD A: Uh - huh.Professor B: Yeah.PhD C: So, yeah, it's rather a small {disfmarker}Professor B: How big is the net, by the wayPhD C: Um, Uh, it's, uh, five hundred hidden units. And {disfmarker}Professor B: And again, you did experiments back then where you made it bigger and it {disfmarker} and that was {disfmarker} that was sort of the threshold point. Much less than that, it was worse,PhD C: Yeah.Professor B: andPhD C: Yeah.Professor B: much more than that, it wasn't much better. Hmm.PhD C: Yeah. @ @PhD D: So is it {disfmarker} is it though the performance, big relation in the high ma high mismatch has something to do with the, uh, cleaning up that you {disfmarker} that is done on the TIMIT after adding noisePhD C:PhD D: So {disfmarker} it's {disfmarker} i All the noises are from the TI - digits,PhD C: Yeah.PhD D: right So you {disfmarker} iPhD C: Um {disfmarker} They {disfmarker} k uh {disfmarker}PhD D: Well, it it's like the high mismatch of the SpeechDat - Car after cleaning up, maybe having more noise than the {disfmarker} the training set of TIMIT after clean {disfmarker} s after you do the noise clean - up.PhD C: Mmm.PhD D: I mean, earlier you never had any compensation, you just trained it straight away.PhD C: Mm - hmm.PhD D: So it had like all these different conditions of S N Rs, actually in their training set of neural net.PhD C: Mm - hmm. Mm - hmm.PhD D: But after cleaning up you have now a different set of S N Rs, rightPhD C: Yeah.PhD D: For the training of the neural net.PhD C: Mm - hmm.PhD D: And {disfmarker} is it something to do with the mismatch that {disfmarker} that's created after the cleaning up, like the high mismatch {disfmarker}PhD C: You mean the {disfmarker} the most noisy occurrences on SpeechDat - Car might be a lot more noisy than {disfmarker}PhD D: Mm - hmm. Of {disfmarker} that {disfmarker} I mean, the SNR after the noise compensation of the SpeechDat - Car.Professor B: Oh, so {disfmarker} Right. So the training {disfmarker} the {disfmarker} the neural net is being trained with noise compensated stuff.PhD C: Maybe.PhD D: Yeah.PhD C: Yeah, yeah.Professor B: Which makes sense,PhD D: Yeah.Professor B: but, uh, you're saying {disfmarker} Yeah, the noisier ones are still going to be, even after our noise compensation, are still gonna be pretty noisy.PhD D: Yeah.PhD C: Mm - hmm.PhD D: Yeah, so now the after - noise compensation the neural net is seeing a different set of S N Rs than that was originally there in the training set. Of TIMIT. Because in the TIMIT it was zero to some clean.Professor B: Right. Yes.PhD D: So the net saw all the SNR @ @ conditions.Professor B: Right.PhD D: Now after cleaning up it's a different set of SNR.Professor B: Right.PhD D: And that SNR may not be, like, com covering the whole set of S N Rs that you're getting in the SpeechDat - Car.Professor B: Right, but the SpeechDat - Car data that you're seeing is also reduced in noise by the noise compensation.PhD C: Yeah.PhD D: Yeah, yeah, yeah, yeah, it is. But, I'm saying, there could be some {disfmarker} some issues of {disfmarker}Professor B: So.PhD C: Mm - hmm.Professor B: Yeah.PhD C: Well, if the initial range of SNR is different, we {disfmarker} the problem was already there before. And {disfmarker}Professor B: Yeah.PhD C: Because {disfmarker} Mmm {disfmarker}Professor B: Yeah, I mean, it depends on whether you believe that the noise compensation is equally reducing the noise on the test set and the training set.PhD C: Hmm.Professor B: Uh {disfmarker}PhD D: On the test set, yeah.Professor B: Right I mean, you're saying there's a mismatch in noise that wasn't there before,PhD D: Hmm. Mm - hmm.Professor B: but if they were both the same before, then if they were both reduic reduced equally, then, there would not be a mismatch.PhD D: Mm - hmm.Professor B: So, I mean, this may be {disfmarker} Heaven forbid, this noise compensation process may be imperfect, but. Uh, so maybe it's treating some things differently.PhD C: Yeah, uh {disfmarker}PhD D: Well, I {disfmarker} I don't know. I {disfmarker} I just {disfmarker} that could be seen from the TI - digits, uh, testing condition because, um, the noises are from the TI - digits, right Noise {disfmarker}PhD C: Yeah. So {disfmarker}PhD D: So cleaning up the TI - digits and if the performance goes down in the TI - digits mismatch {disfmarker} high mismatch like this {disfmarker}PhD C: Clean training, yeah.PhD D: on a clean training, or zero DB testing.PhD C: Yeah, we'll {disfmarker} so we'll see. Uh.PhD D: Yeah.PhD C: Maybe.PhD D: Then it's something to do.PhD C: Mm - hmm.Professor B: I mean, one of the things about {disfmarker}PhD C: Yeah.Professor B: I mean, the Macrophone data, um, I think, you know, it was recorded over many different telephones.PhD C: Mm - hmm.Professor B: And, um, so, there's lots of different kinds of acoustic conditions. I mean, it's not artificially added noise or anything. So it's not the same. I don't think there's anybody recording over a car from a car, but {disfmarker} I think it's {disfmarker} it's varied enough that if {disfmarker} if doing this adjustments, uh, and playing around with it doesn't, uh, make it better, the most {disfmarker} uh, it seems like the most obvious thing to do is to improve the training set. Um {disfmarker} I mean, what we were {disfmarker} uh {disfmarker} the condition {disfmarker} It {disfmarker} it gave us an enormous amount of improvement in what we were doing with Meeting Recorder digits, even though there, again, these m Macrophone digits were very, very different from, uh, what we were going on here. I mean, we weren't talking over a telephone here. But it was just {disfmarker} I think just having a {disfmarker} a nice variation in acoustic conditions was just a good thing.PhD C: Mm - hmm. Yep.PhD D: Mmm.PhD C: Yeah, actually {vocalsound} to s eh, what I observed in the HM case is that the number of deletion dramatically increases. It {disfmarker} it doubles.Professor B: Number of deletions.PhD C: When I added the num the neural network it doubles the number of deletions. Yeah, so I don't you know {vocalsound} how to interpret that, but, mmm {disfmarker}Professor B: Yeah. Me either.PhD C: tPhD A: And {disfmarker} and did {disfmarker} an other numbers stay the same Insertion substitutions stay the samePhD C: They p stayed the same,PhD A: RoughlyPhD C: they {disfmarker} maybe they are a little bit uh, lower.PhD A: Uh - huh.PhD C: They are a little bit better. Yeah. But {disfmarker}Professor B: Did they increase the number of deletions even for the cases that got betterPhD C: Mm - hmm.Professor B: Say, for the {disfmarker} I mean, it {disfmarker}PhD C: No, it doesn't.Professor B: So it's only the highly mismatchedPhD C: No.Professor B: And it {disfmarker} Remind me again, the" highly mismatched" means that the {disfmarker}PhD C: Clean training and {disfmarker}Professor B: Uh, sorryPhD C: It's clean training {disfmarker} Well, close microphone training and distant microphone, um, high speed, I think.Professor B: Close mike training {disfmarker}PhD C: Well {disfmarker} The most noisy cases are the distant microphone for testing.Professor B: Right. So {disfmarker} Well, maybe the noise subtraction is subtracting off speech.PhD C: Separating. Yeah.Professor B: WhPhD C: But {disfmarker} Yeah. I mean, but without the neural network it's {disfmarker} well, it's better. It's just when we add the neural networks.Professor B: Yeah, right.PhD C: The feature are the same except that {disfmarker}Professor B: Uh, that's right, that's right. Um {disfmarker}PhD A: Well that {disfmarker} that says that, you know, the, um {disfmarker} the models in {disfmarker} in, uh, the recognizer are really paying attention to the neural net features.PhD C: Yeah.PhD A: Uh.PhD C: Mm - hmm.Professor B: But, yeah, actually {disfmarker} {nonvocalsound} the TIMIT noises {pause} are sort of a range of noises and they're not so much the stationary driving kind of noises, right It's {disfmarker} it's pretty different. Isn't itPhD C: Uh, there is a car noise. So there are f just four noises. Um, uh," Car" , I think," Babble" ,PhD D:" Babble." PhD C:" Subway" , right and {disfmarker}PhD D:" Street" or" Airport" or something.PhD C: and {disfmarker}" Street" isn't {disfmarker}PhD D: Or" Train station" .PhD C:" Train station" , yeah.PhD D: Yeah.PhD C: So {disfmarker} it's mostly {disfmarker} Well," Car" is stationary,Professor B: Mm - hmm.PhD C:" Babble" , it's a stationary background plus some voices,Professor B: Mm - hmm.PhD C: some speech over it. And the other two are rather stationary also.Professor B: Well, I {disfmarker} I think that if you run it {disfmarker} Actually, you {disfmarker} maybe you remember this. When you {disfmarker} in {disfmarker} in the old experiments when you ran with the neural net only, and didn't have this side path, um, uh, with the {disfmarker} the pure features as well, did it make things better to have the neural netPhD C: Mm - hmm.Professor B: Was it about the same Uh, w iPhD C: It was {disfmarker} b a little bit worse.Professor B: Than {disfmarker}PhD C: Than just the features, yeah.Professor B: So, until you put the second path in with the pure features, the neural net wasn't helping at all.PhD C: Mm - hmm.Professor B: Well, that's interesting.PhD C: It was helping, uh, if the features are b were bad,Professor B: Yeah.PhD C: I mean. Just plain P L Ps or M FProfessor B: Yeah.PhD C: C Cs. as soon as we added LDA on - line normalization, and {vocalsound} all these things, then {disfmarker}Professor B: They were doing similar enough things. Well, I still think it would be k sort of interesting to see what would happen if you just had the neural net without the side thing.PhD C: Yeah,Professor B: And {disfmarker} and the thing I {disfmarker} I have in mind is, uh, maybe you'll see that the results are not just a little bit worse.PhD C: mm - hmm.Professor B: Maybe that they're a lot worse. You know And, um {disfmarker} But if on the ha other hand, uh, it's, say, somewhere in between what you're seeing now and {disfmarker} and {disfmarker} and, uh, what you'd have with just the pure features, then maybe there is some problem of a {disfmarker} of a, uh, combination of these things, or correlation between them somehow.PhD C: Mm - hmm.Professor B: If it really is that the net is hurting you at the moment, then I think the issue is to focus on {disfmarker} on, uh, improving the {disfmarker} the net.PhD C: Yeah,Professor B: Um.PhD C: mm - hmm.Professor B: So what's the overall effe I mean, you haven't done all the experiments but you said it was i somewhat better, say, five percent better, for the first two conditions, and fifteen percent worse for the other one But it's {disfmarker} but of course that one's weighted lower,PhD C: Y yeah, oh. Yeah.Professor B: so I wonder what the net effect is.PhD C: I d I {disfmarker} I think it's {disfmarker} it was one or two percent. That's not that bad, but it was l like two percent relative worse on SpeechDat - Car. I have to {disfmarker} to check that. Well, I have {disfmarker} I will.PhD D: Well, it will {disfmarker} overall it will be still better even if it is fifteen percent worse, because the fifteen percent worse is given like f w twenty - five {disfmarker} point two five eight.Professor B: Right.PhD C: Mm - hmm. Hmm.Professor B: Right. So the {disfmarker} so the worst it could be, if the others were exactly the same, is four,PhD D: Is it like {disfmarker}Professor B: and {disfmarker} and, uh, in fact since the others are somewhat better {disfmarker}PhD D: Yeah, so it's four. Is i So either it'll get cancelled out, or you'll get, like, almost the same.Professor B: Uh.PhD C: Yeah, it was {disfmarker} it was slightly worse.PhD D: Slightly bad. Yeah.PhD C: Um,Professor B: Yeah, it should be pretty close to cancelled out.PhD D: Yeah.PhD A: You know, I've been wondering about something.PhD C: Mm - hmm.PhD A: In the, um {disfmarker} a lot of the, um {disfmarker} the Hub - five systems, um, recently have been using LDA. and {disfmarker} and they, um {disfmarker} They run LDA on the features right before they train the models. So there's the {disfmarker} the LDA is {disfmarker} is right there before the H MPhD D: Yeah.PhD A: So, you guys are using LDA but it seems like it's pretty far back in the process.PhD D: Uh, this LDA is different from the LDA that you are talking about. The LDA that you {disfmarker} saying is, like, you take a block of features, like nine frames or something, {comment} and then do an LDA on it,PhD A: Yeah. Uh - huh.PhD D: and then reduce the dimensionality to something like twenty - four or something like that.PhD A: Yeah, you c you c you can.PhD D: And then feed it to HMM.PhD A: I mean, it's {disfmarker} you know, you're just basically iPhD D: Yeah, so this is like a two d two dimensional tile.PhD A: You're shifting the feature space. Yeah.PhD D: So this is a two dimensional tile. And the LDA that we are f applying is only in time, not in frequency {disfmarker} high cost frequency. So it's like {disfmarker} more like a filtering in time, rather than doing a rPhD A: Ah. OK. So what i what about, um {disfmarker} i u what i w I mean, I don't know if this is a good idea or not, but what if you put {disfmarker} ran the other kind of LDA, uh, on your features right before they go into the HMMPhD D: Uh, it {disfmarker}PhD C: Mm - hmm. No, actually, I think {disfmarker} iPhD D: mPhD C: Well. What do we do with the ANN is {disfmarker} is something like that except that it's not linear. But it's {disfmarker} it's like a nonlinear discriminant analysis.PhD A: Yeah. Right, it's the {disfmarker} It's {disfmarker} Right. The {disfmarker} So {disfmarker} Yeah, so it's sort of like {disfmarker}PhD C: But.PhD A: The tandem stuff is kind of like i nonlinear LDA.PhD C: Yeah. It's {disfmarker}PhD A: I gPhD C: Yeah.PhD A: Yeah.Professor B: Yeah.PhD A: But I mean, w but the other features that you have, um, th the non - tandem ones,PhD C: Uh. Mm - hmm. Yeah, I know. That {disfmarker} that {disfmarker} Yeah. Well, in the proposal, they were transformed u using PCA, but {disfmarker}PhD A: Uh - huh.PhD C: Yeah, it might be that LDA could be better.Professor B: The a the argument i is kind of i in {disfmarker} and it's not like we really know, but the argument anyway is that, um, uh, we always have the prob I mean, discriminative things are good. LDA, neural nets, they're good.PhD A: Yeah.Professor B: Uh, they're good because you {disfmarker} you {disfmarker} you learn to distinguish between these categories that you want to be good at distinguishing between. And PCA doesn't do that. It {disfmarker} PAC - PCA {disfmarker} low - order PCA throws away pieces that are uh, maybe not {disfmarker} not gonna be helpful just because they're small, basically.PhD A: Right.Professor B: But, uh, the problem is, training sets aren't perfect and testing sets are different. So you f you {disfmarker} you face the potential problem with discriminative stuff, be it LDA or neural nets, that you are training to discriminate between categories in one space but what you're really gonna be g getting is {disfmarker} is something else.PhD A: Uh - huh.Professor B: And so, uh, Stephane's idea was, uh, let's feed, uh, both this discriminatively trained thing and something that's not. So you have a good set of features that everybody's worked really hard to make,PhD A: Yeah.Professor B: and then, uh, you {disfmarker} you discriminately train it, but you also take the path that {disfmarker} that doesn't have that,PhD A: Uh - huh.Professor B: and putting those in together. And that {disfmarker} that seem So it's kind of like a combination of the {disfmarker} uh, what, uh, Dan has been calling, you know, a feature {disfmarker} uh, you know, a feature combination versus posterior combination or something. It's {disfmarker} it's, you know, you have the posterior combination but then you get the features from that and use them as a feature combination with these {disfmarker} these other things. And that seemed, at least in the last one, as he was just saying, he {disfmarker} he {disfmarker} when he only did discriminative stuff, i it actually was {disfmarker} was {disfmarker} it didn't help at all in this particular case.PhD A: Yeah.Professor B: There was enough of a difference, I guess, between the testing and training. But by having them both there {disfmarker} The fact is some of the time, the discriminative stuff is gonna help you.PhD A: Mm - hmm.Professor B: And some of the time it's going to hurt you,PhD A: Right.Professor B: and by combining two information sources if, you know {disfmarker} if {disfmarker} if {disfmarker}PhD A: So you wouldn't necessarily then want to do LDA on the non - tandem features because now you're doing something to them that {disfmarker}Professor B: That i i I think that's counter to that idea.PhD A: Yeah, right.Professor B: Now, again, it's {disfmarker} we're just trying these different things. We don't really know what's gonna work best. But if that's the hypothesis, at least it would be counter to that hypothesis to do that.PhD A: Right.Professor B: Um, and in principle you would think that the neural net would do better at the discriminant part than LDA.PhD A: Right. Yeah. Well {disfmarker} yProfessor B: Though, maybe not.PhD A: Yeah. Exactly. I mean, we, uh {disfmarker} we were getting ready to do the tandem, uh, stuff for the Hub - five system, and, um, Andreas and I talked about it, and the idea w the thought was," Well, uh, yeah, that i you know {disfmarker} th the neural net should be better, but we should at least have uh, a number, you know, to show that we did try the LDA in place of the neural net, so that we can you know, show a clear path.Professor B: Right.PhD A: You know, that you have it without it, then you have the LDA, then you have the neural net, and you can see, theoretically. So. I was just wondering {disfmarker} I {disfmarker} I {disfmarker}Professor B: Well, I think that's a good idea.PhD A: Yeah.Professor B: Did {disfmarker} did you do thatPhD A: Um. No.Professor B: or {disfmarker} tha that's a {disfmarker}PhD A: That's what {disfmarker} that's what we're gonna do next as soon as I finish this other thing. So.Professor B: Yeah. Yeah. No, well, that's a good idea. I {disfmarker} I {disfmarker}PhD A: We just want to show.Professor B: i Yeah.PhD A: I mean, it {disfmarker} everybody believes it,Professor B: Oh, no it's a gPhD A: but you know, we just {disfmarker}Professor B: No, no, but it might not {disfmarker} not even be true.PhD A: Yeah.Professor B: I mean, it's {disfmarker} it's {disfmarker} it's {disfmarker} it's {disfmarker} it's a great idea. I mean, one of the things that always disturbed me, uh, in the {disfmarker} the resurgence of neural nets that happened in the eighties was that, um, a lot of people {disfmarker} Because neural nets were pretty easy to {disfmarker} to use {disfmarker} a lot of people were just using them for all sorts of things without, uh, looking at all into the linear, uh {disfmarker} uh, versions of them.PhD A: Yeah. Mm - hmm. Yeah.Professor B: And, uh, people were doing recurrent nets but not looking at IIR filters, and {disfmarker} You know, I mean, uh, so I think, yeah, it's definitely a good idea to try it.PhD A: Yeah, and everybody's putting that on their {vocalsound} systems now, and so, I that's what made me wonder about this,Professor B: Well, they've been putting them in their systems off and on for ten years,PhD A: but.Professor B: but {disfmarker} but {disfmarker} but, uh,PhD A: Yeah, what I mean is it's {disfmarker} it's like in the Hub - five evaluations, you know, and you read the system descriptions and everybody's got, {vocalsound} you know, LDA on their features.Professor B: And now they all have that. I see.PhD A: And so.Professor B: Yeah.PhD A: Uh.PhD C: It's the transformation they're estimating on {disfmarker} Well, they are trained on the same data as the final HMM are.PhD A: Yeah, so it's different. Yeah, exactly. Cuz they don't have these, you know, mismatches that {disfmarker} that you guys have.PhD C: Mm - hmm.PhD A: So that's why I was wondering if maybe it's not even a good idea.PhD C: Mm - hmm.PhD A: I don't know. I {disfmarker} I don't know enough about it,PhD C: Mm - hmm.PhD A: but {disfmarker} Um.Professor B: I mean, part of why {disfmarker} I {disfmarker} I think part of why you were getting into the KLT {disfmarker} Y you were describing to me at one point that you wanted to see if, uh, you know, getting good orthogonal features was {disfmarker} and combining the {disfmarker} the different temporal ranges {disfmarker} was the key thing that was happening or whether it was this discriminant thing, right So you were just trying {disfmarker} I think you r I mean, this is {disfmarker} it doesn't have the LDA aspect but th as far as the orthogonalizing transformation, you were trying that at one point, rightPhD C: Mm - hmm.Professor B: I think you were.PhD C: Mm - hmm. Yeah.Professor B: Does something. It doesn't work as well. Yeah. Yeah.PhD D: So, yeah, I've been exploring a parallel VAD without neural network with, like, less latency using SNR and energy, um, after the cleaning up. So what I'd been trying was, um, uh {disfmarker} After the b after the noise compensation, n I was trying t to f find a f feature based on the ratio of the energies, that is, cl after clean and before clean. So that if {disfmarker} if they are, like, pretty c close to one, which means it's speech. And if it is n if it is close to zero, which is {disfmarker} So it's like a scale @ @ probability value. So I was trying, uh, with full band and multiple bands, m ps uh {disfmarker} separating them to different frequency bands and deriving separate decisions on each bands, and trying to combine them. Uh, the advantage being like it doesn't have the latency of the neural net if it {disfmarker} if it canProfessor B: Mm - hmm.PhD D: g And {pause} it gave me like, uh, one point {disfmarker} One {disfmarker} more than one percent relative improvement. So, from fifty - three point six it went to fifty f four point eight. So it's, like, only slightly more than a percent improvement,Professor B: Mm - hmm.PhD D: just like {disfmarker} Which means that it's {disfmarker} it's doing a slightly better job than the previous VAD,Professor B: Mm - hmm.PhD D: uh, at a l lower delay.Professor B: Mm - hmm.PhD D: Um, so, um {disfmarker}Professor B: But {disfmarker} i d I'm sorry,PhD D: so {disfmarker} uProfessor B: does it still have the median {pause} filter stuffPhD D: It still has the median filter.Professor B: So it still has most of the delay,PhD D: So {disfmarker}Professor B: it just doesn't {disfmarker}PhD D: Yeah, so d with the delay, that's gone is the input, which is the sixty millisecond. The forty plus {pause} twenty.Professor B: Well, w iPhD D: At the input of the neural net you have this, uh, f nine frames of context plus the delta.Professor B: Oh, plus the delta,PhD C: Mm - hmm.Professor B: right. OK.PhD D: Yeah. So that delay, plus the LDA.Professor B: Mm - hmm.PhD D: Uh, so the delay is only the forty millisecond of the noise cleaning, plus the hundred millisecond smoothing at the output.Professor B: Mm - hmm. Mm - hmm.PhD D: Um. So. Yeah. So the {disfmarker} the {disfmarker} di the biggest {disfmarker} The problem f for me was to find a consistent threshold that works {pause} well across the different databases, because I t I try to make it work on tr SpeechDat - CarProfessor B: Mm - hmm.PhD D: and it fails on TI - digits, or if I try to make it work on that it's just the Italian or something, it doesn't work on the Finnish.Professor B: Mm - hmm.PhD D: So, um. So there are {disfmarker} there was, like, some problem in balancing the deletions and insertions when I try different thresholds.Professor B: Mm - hmm.PhD D: So {disfmarker} The {disfmarker} I'm still trying to make it better by using some other features from the {disfmarker} after the p clean up {disfmarker} maybe, some, uh, correlation {disfmarker} auto - correlation or some s additional features of {disfmarker} to mainly the improvement of the VAD. I've been trying.Professor B: Now this {disfmarker} this {disfmarker} this, uh," before and after clean" , it sounds like you think that's a good feature. That {disfmarker} that, it {disfmarker} you th think that the, uh {disfmarker} the {disfmarker} i it appears to be a good feature, rightPhD D: Mm - hmm.Professor B: What about using it in the neural netPhD D: Yeah.PhD C: Yeah, eventually we could {disfmarker} could justPhD D: Yeah, so {disfmarker} Yeah, so that's the {disfmarker} Yeah. So we've been thinking about putting it into the neural net also.Professor B: Yeah.PhD D: Because they did {disfmarker} that itself {disfmarker}PhD C: Then you don't have to worry about the thresholds and {disfmarker}PhD D: There's a threshold and {disfmarker} Yeah.Professor B: Yeah.PhD C: but just {disfmarker}PhD D: Yeah. So that {disfmarker} that's, uh {disfmarker}Professor B: Yeah. So if we {disfmarker} if we can live with the latency or cut the latencies elsewhere, then {disfmarker} then that would be a, uh, good thing.PhD D: Yeah. Yeah.Professor B: Um, anybody {disfmarker} has anybody {disfmarker} you guys or {disfmarker} or Naren, uh, somebody, tried the, uh, um, second th second stream thing Uh.PhD D: Oh, I just {disfmarker} I just h put the second stream in place and, uh ran one experiment, but just like {disfmarker} just to know that everything is fine.Professor B: Uh - huh.PhD D: So it was like, uh, forty - five cepstrum plus twenty - three mel {disfmarker} log mel.Professor B: Yeah.PhD D: And {disfmarker} and, just, like, it gave me the baseline performance of the Aurora, which is like zero improvement.Professor B: Yeah. Yeah.PhD D: So I just tried it on Italian just to know that everything is {disfmarker} But I {disfmarker} I didn't export anything out of it because it was, like, a weird feature set.Professor B: Yeah.PhD D: So.Professor B: Yeah. Well, what I think, you know, would be more what you'd want to do is {disfmarker} is {disfmarker} is, uh, put it into another neural net. RightPhD C: Mm - hmm.PhD D: Yeah, yeah, yeah, yeah.Professor B: And then {disfmarker} But, yeah, we're {disfmarker} we're not quite there yet. So we have to {vocalsound} figure out the neural nets, I guess.PhD C: Yeah.PhD D: The uh, other thing I was wondering was, um, if the neural net, um, has any {disfmarker} because of the different noise con unseen noise conditions for the neural net, where, like, you train it on those four noise conditions, while you are feeding it with, like, a additional {disfmarker} some four plus some {disfmarker} f few more conditions which it hasn't seen, actually,PhD C: Mm - hmm.PhD D: from the {disfmarker} f f while testing.PhD C: Yeah, yeah. Right.PhD D: Um {disfmarker} instead of just h having c uh, those cleaned up t cepstrum, sh should we feed some additional information, like {disfmarker} The {disfmarker} the {disfmarker} We have the VAD flag. I mean, should we f feed the VAD flag, also, at the input so that it {disfmarker} it has some additional discriminating information at the inputPhD C: Hmm - hmm! Um {disfmarker}Professor B: Wh - uh, the {disfmarker} the VAD whatPhD D: We have the VAD information also available at the back - end.Professor B: Uh - huh.PhD D: So if it is something the neural net is not able to discriminate the classes {disfmarker}Professor B: Yeah.PhD D: I mean {disfmarker} Because most of it is sil I mean, we have dropped some silence f We have dropped so silence framesProfessor B: Mm - hmm.PhD D: No, we haven't dropped silence frames still.PhD C: Uh, still not. Yeah.PhD D: Yeah. So {disfmarker}PhD C: ThPhD D: the b b biggest classification would be the speech and silence. So, by having an additional, uh, feature which says" this is speech and this is nonspeech" , I mean, it certainly helps in some unseen noise conditions for the neural net.PhD A: What {disfmarker} Do y do you have that feature available for the test dataPhD D: Well, I mean, we have {disfmarker} we are transferring the VAD to the back - end {disfmarker} feature to the back - end. Because we are dropping it at the back - end after everything {disfmarker} all the features are computed.PhD A: Oh, oh, I see.PhD D: So {disfmarker}PhD A: I see.PhD D: so the neural {disfmarker} so that is coming from a separate neural net or some VAD.PhD A: OK. OK.PhD D: Which is {disfmarker} which is certainly giving aPhD A: So you're saying, feed that, also, into {pause} the neural net.PhD D: to {disfmarker} Yeah. So it it's an {disfmarker} additional discriminating information.PhD A: Yeah. Yeah. Right.PhD D: So that {disfmarker}Professor B: You could feed it into the neural net. The other thing {comment} you could do is just, um, p modify the, uh, output probabilities of the {disfmarker} of the, uh, uh, um, neural net, tandem neural net, {comment} based on the fact that you have a silence probability.PhD D: Mm - hmm.Professor B: RightPhD C: Mm - hmm.Professor B: So you have an independent estimator of what the silence probability is, and you could multiply the two things, and renormalize.PhD C: Yeah.Professor B: Uh, I mean, you'd have to do the nonlinearity part and deal with that. Uh, I mean, go backwards from what the nonlinearity would, you know {disfmarker} would be.PhD D: Through {disfmarker} t to the soft max.Professor B: But {disfmarker} but, uh {disfmarker}PhD C: Yeah, so {disfmarker} maybe, yeah, when {disfmarker}PhD A: But in principle wouldn't it be better to feed it in And let the net do thatProfessor B: Well, u Not sure.PhD A: Hmm.Professor B: I mean, let's put it this way. I mean, y you {disfmarker} you have this complicated system with thousands and thousand parametersPhD A: Yeah.Professor B: and you can tell it, uh," Learn this thing." Or you can say," It's silence! Go away!" I mean, I mean, i Doesn't {disfmarker} I think {disfmarker} I think the second one sounds a lot more direct.PhD A: What {disfmarker} what if you {disfmarker}Professor B: Uh.PhD A: Right. So, what if you then, uh {disfmarker} since you know this, what if you only use the neural net on the speech portionsProfessor B: Well, uh,PhD C: That's what {disfmarker}PhD A: Well, I guess that's the same. Uh, that's similar.Professor B: Yeah, I mean, y you'd have to actually run it continuously,PhD A: But I mean {disfmarker} I mean, train the net only on {disfmarker}Professor B: but it's {disfmarker} @ @ {disfmarker} Well, no, you want to train on {disfmarker} on the nonspeech also, because that's part of what you're learning in it, to {disfmarker} to {disfmarker} to generate, that it's {disfmarker} it has to distinguish between.PhD D: Speech.PhD A: But I mean, if you're gonna {disfmarker} if you're going to multiply the output of the net by this other decision, uh, would {disfmarker} then you don't care about whether the net makes that distinction, rightProfessor B: Well, yeah. But this other thing isn't perfect.PhD A: Ah.Professor B: So that you bring in some information from the net itself.PhD A: Right, OK. That's a good point.Professor B: Yeah. Now the only thing that {disfmarker} that bothers me about all this is that I {disfmarker} I {disfmarker} I {disfmarker} The {disfmarker} the fact {disfmarker} i i It's sort of bothersome that you're getting more deletions.PhD C: Yeah. But {disfmarker} So I might maybe look at, is it due to the fact that um, the probability of the silence at the output of the network, is, uh,Professor B: Is too high.PhD C: too {disfmarker} too high or {disfmarker}Professor B: Yeah. So maybe {disfmarker} So {disfmarker}PhD C: If it's the case, then multiplying it again by {disfmarker} i by somethingPhD D: It may not be {disfmarker} it {disfmarker}Professor B: Yeah.PhD C: Mm - hmm.PhD D: Yeah, it {disfmarker} it may be too {disfmarker} it's too high in a sense, like, everything is more like a, um, flat probability.Professor B: Yeah.PhD C: Oh - eee - hhh.PhD D: So, like, it's not really doing any distinction between speech and nonspeech {disfmarker}PhD C: Uh, yeah.PhD D: or, I mean, different {disfmarker} among classes.Professor B: Yeah.PhD C: Mm - hmm.PhD A: Be interesting to look at the {disfmarker} Yeah, for the {disfmarker} I wonder if you could do this. But if you look at the, um, highly mism high mismat the output of the net on the high mismatch case and just look at, you know, the distribution versus the {disfmarker} the other ones, do you {disfmarker} do you see more peaks or somethingPhD C: Yeah. Yeah, like the entropy of the {disfmarker} the output,PhD A: Yeah.Professor B: Yeah, for instance.PhD C: or {disfmarker}Professor B: But I {disfmarker} buPhD C: It {disfmarker} it seems that the VAD network doesn't {disfmarker} Well, it doesn't drop, uh, too many frames because the dele the number of deletion is reasonable. But it's just when we add the tandem, the final MLP, and then {disfmarker}Professor B: Yeah. Now the only problem is you don't want to ta I guess wait for the output of the VAD before you can put something into the other system,PhD C: uProfessor B: cuz that'll shoot up the latency a lot, right Am I missing something herePhD C: But {disfmarker}PhD D: Mm - hmm.PhD C: Yeah. Right.Professor B: Yeah. So that's maybe a problem with what I was just saying. But {disfmarker} but {disfmarker} I I guess {disfmarker}PhD A: But if you were gonna put it in as a feature it means you already have it by the time you get to the tandem net, rightPhD D: Um, well. We {disfmarker} w we don't have it, actually,Professor B: No.PhD D: because it's {disfmarker} it has a high rate energy {disfmarker}PhD A: Ah.PhD D: the VAD has a {disfmarker}Professor B: Yeah.PhD A: OK.Professor B: It's kind of done in {disfmarker} I mean, some of the things are, not in parallel, but certainly, it would be in parallel with the {disfmarker} with a tandem net.PhD A: Right.Professor B: In time. So maybe, if that doesn't work, um {disfmarker} But it would be interesting to see if that was the problem, anyway. And {disfmarker} and {disfmarker} and then I guess another alternative would be to take the feature that you're feeding into the VAD, and feeding it into the other one as well.PhD C: Mm - hmm.Professor B: And then maybe it would just learn {disfmarker} learn it better.PhD C: Mm - hmm.Professor B: Um {disfmarker} But that's {disfmarker} Yeah, that's an interesting thing to try to see, if what's going on is that in the highly mismatched condition, it's, um, causing deletions by having this silence probability up {disfmarker} up too high,PhD C: Mm - hmm.Professor B: at some point where the VAD is saying it's actually speech.PhD C: Yeah.Professor B: Which is probably true.PhD C: So, mProfessor B: Cuz {disfmarker} Well, the V A if the VAD said {disfmarker} since the VAD is {disfmarker} is {disfmarker} is right a lot, uh {disfmarker}PhD C: Yeah.Professor B: Hmm. Anyway. Might be.PhD C: Mm - hmm.Professor B: Yeah. Well, we just started working with it. But these are {disfmarker} these are some good ideas I think.PhD C: Mm - hmm. Yeah, and the other thing {disfmarker} Well, there are other issues maybe for the tandem, like, uh, well, do we want to, w uh n Do we want to work on the targets Or, like, instead of using phonemes, using more context dependent unitsPhD A: For the tandem net you meanPhD C: Well, I'm {disfmarker} Yeah.PhD A: Hmm.PhD C: I'm thinking, also, a w about Dan's work where he {disfmarker} he trained {vocalsound} a network, not on phoneme targets but on the HMM state targets. And {disfmarker} it was giving s slightly better results.Professor B: Problem is, if you are going to run this on different m test sets, including large vocabulary,PhD C: Yeah. Yeah.Professor B: um,PhD C: Uh {disfmarker}Professor B: I think {disfmarker}PhD C: Mmm. I was just thinking maybe about, like, generalized diphones, and {disfmarker} come up with a {disfmarker} a reasonable, not too large, set of context dependent units, and {disfmarker} and {disfmarker} Yeah. And then anyway we would have to reduce this with the KLT.Professor B: Yeah.PhD C: So. But {disfmarker} I don't know.Professor B: Yeah. Well, maybe. But I d I d it {disfmarker} it {disfmarker} i it's all worth looking at,PhD C: Mm - hmm.Professor B: but it sounds to me like, uh, looking at the relationship between this and the {disfmarker} speech noise stuff is {disfmarker} is {disfmarker} is probably a key thing.PhD C: Mm - hmm.Professor B: That and the correlation between stuff.PhD A: So if, uh {disfmarker} if the, uh, high mismatch case had been more like the, uh, the other two cases {comment} in terms of giving you just a better performance, {comment} how would this number have changedPhD C: Mm - hmm. Oh, it would be {disfmarker} Yeah. Around five percent better, I guess. If {disfmarker} if {disfmarker} iPhD A: y Like sixtyProfessor B: Well, we don't know what's it's gonna be the TI - digits yet. He hasn't got the results back yet.PhD C: Yeah. If you extrapolate the SpeechDat - Car well - matched and medium - mismatch, it's around, yeah, maybe five.PhD A: Uh - huh. Yeah. So this would be sixty - twoProfessor B: Sixty - two.PhD A: Which is {disfmarker}Professor B: Yeah.PhD C: Sixty - two, yeah.PhD D: Somewhere around sixty, must be. Right Yeah.PhD C: Well, it's around five percent, because it's {disfmarker} s Right If everything is five percent.PhD D: Yeah. Yeah.PhD A: All the other ones were five percent,PhD C: Mm - hmm.PhD A: the {disfmarker}Professor B: Yeah.PhD C: I d I d I just have the SpeechDat - Car right now, so {disfmarker}PhD A: Yeah.PhD C: It's running {disfmarker} it shou we should have the results today during the afternoon,PhD A: Hmm.PhD C: but {disfmarker} Well.Professor B: Hmm. Well {disfmarker} Um {disfmarker} So I won't be here for {disfmarker}PhD A: When {disfmarker} When do you leaveProfessor B: Uh, I'm leaving next Wednesday. May or may not be in in the morning. I leave in the afternoon. Um,PhD A: But you're {disfmarker}Professor B: so I {disfmarker}PhD A: are you {disfmarker} you're not gonna be around this afternoonProfessor B: Yeah.PhD A: Oh.Professor B: Oh, well. I'm talking about next week. I'm leaving {disfmarker} leaving next Wednesday.PhD A: Uh - huh.Professor B: This afternoon {disfmarker} uh {disfmarker} Oh, right, for the Meeting meeting Yeah, that's just cuz of something on campus.PhD A: Ah, OK, OK.Professor B: Yeah. But, um, yeah, so next week I won't, and the week after I won't, cuz I'll be in Finland. And the week after that I won't. By that time you'll be {disfmarker} {comment} Uh, you'll both be gone {pause} from here. So there'll be no {disfmarker} definitely no meeting on {disfmarker} on September sixth. Uh,PhD A: What's September sixthProfessor B: and {disfmarker} Uh, that's during Eurospeech.PhD A: Oh, oh, right. OK.Professor B: So, uh, Sunil will be in Oregon. Uh, Stephane and I will be in Denmark. Uh {disfmarker} Right So it'll be a few weeks, really, before we have a meeting of the same cast of characters. Um, but, uh {disfmarker} I guess, just {disfmarker} I mean, you guys should probably meet. And maybe Barry {disfmarker} Barry will be around. And {disfmarker} and then uh, uh, we'll start up again with Dave and {disfmarker} Dave and Barry and Stephane and us on the, uh, twentieth. No. Thirteenth About a monthPhD A: So, uh, you're gonna be gone for the next three weeks or somethingProfessor B: I'm gone for two and a half weeks starting {disfmarker} starting next Wed - late next Wednesday.PhD A: So that's {disfmarker} you won't be at the next three of these meetings. Is that rightProfessor B: Uh, I won't {disfmarker} it's probably four because of {disfmarker} is it three Let's see, twenty - third, thirtieth, sixth. That's right, next three. And the {disfmarker} the third one won't {disfmarker} probably won't be a meeting, cuz {disfmarker} cuz, uh, Su - Sunil, Stephane, and I will all not be here.PhD A: Oh, right. Right.Professor B: Um {disfmarker} Mmm. {comment} So it's just, uh, the next two where there will be {disfmarker} there, you know, may as well be meetings,PhD A: OK.Professor B: but I just won't be at them. And then starting up on the thirteenth, {nonvocalsound} uh, we'll have meetings again but we'll have to do without Sunil here somehow.PhD A: When do you go backProfessor B: So.PhD D: Thirty - first, August.Professor B: Yeah. Yeah. So. Cool.PhD A: When is the evaluation November, or somethingProfessor B: Yeah, it was supposed to be November fifteenth. Has anybody heard anything differentPhD C: I don't know. The meeting in {disfmarker} is the five and six of December. So {disfmarker}PhD D: p s It's like {disfmarker} Yeah, it's tentatively all full. Yeah.PhD C: Mm - hmm.PhD D: Uh, that's a proposed date, I guess.PhD C: Yeah, um {disfmarker} so the evaluation should be on a week before or {disfmarker}PhD A: Yeah.Professor B: Yep. But, no, this is good progress. So. Uh {disfmarker} OK.PhD A: Should we do digitsProfessor B: Guess we're done. Digits Yep.PhD A: OK.Professor B: It's a wrap.
The VAD neural net had twelve times nine inputs and two outputs. The system had improved significantly. It added some latency but the team could potentially run it in parallel.
Summarize the discussion on VAD results