[speaker001:] Alright. We're on.
[speaker002:] Test, um. Test, test, test. Guess that's me. Yeah. OK.
[speaker004:] Ooh, Thursday.
[speaker002:] So. There's two sheets of paper in front of us.
[speaker005:] Yeah. So.
[speaker001:] What are these?
[speaker002:] This is the arm wrestling?
[speaker003:] Uh. Yeah, we formed a coalition actually.
[speaker005:] Yeah. Almost.
[speaker003:] We already made it into one.
[speaker002:] Oh, good.
[speaker003:] Yeah.
[speaker005:] Yeah.
[speaker002:] Excellent. That's the best thing.
[speaker005:] Mm hmm.
[speaker002:] So, tell me about it.
[speaker005:] So it's [disfmarker] well, it's [pause] spectral subtraction or Wiener filtering, um, depending on if we put [disfmarker] if we square the transfer function or not.
[speaker002:] Right.
[speaker005:] And then with over estimation of the noise, depending on the, uh [disfmarker] the SNR, with smoothing along time, um, smoothing along frequency.
[speaker002:] Mm hmm.
[speaker005:] It's very simple, smoothing things.
[speaker002:] Mm hmm.
[speaker005:] And, um, [vocalsound] the best result is [vocalsound] when we apply this procedure on FFT bins, uh, with a Wiener filter.
[speaker002:] Mm hmm.
[speaker005:] And there is no noise addition after [disfmarker] after that.
[speaker002:] OK.
[speaker005:] So it's good because [vocalsound] [vocalsound] it's difficult when we have to add noise to [disfmarker] to [disfmarker] to find the right level.
[speaker002:] OK.
[speaker001:] Are you looking at one in [disfmarker] in particular of these two?
[speaker005:] Yeah. So the sh it's the sheet that gives fifty f three point sixty six.
[speaker002:] Mm hmm.
[speaker005:] Um, [vocalsound] the second sheet is abo uh, about the same. It's the same, um, idea but it's working on mel bands, [vocalsound] and it's a spectral subtraction instead of Wiener filter, and there is also a noise addition after, uh, cleaning up the mel bins. Mmm. Well, the results are similar.
[speaker002:] Yeah. I mean, [vocalsound] it's [disfmarker] [comment] it's actually, uh, very similar.
[speaker005:] Mm hmm.
[speaker002:] I mean, [vocalsound] if you look at databases, uh, the, uh, one that has the smallest [disfmarker] smaller overall number is actually better on the Finnish and Spanish, uh, but it is, uh, worse on the, uh, Aurora [disfmarker]
[speaker005:] It's worse on [disfmarker]
[speaker002:] I mean on the, uh, TI TI digits,
[speaker005:] on the multi condition in TI digits. Yeah.
[speaker002:] uh, uh. Um.
[speaker005:] Mmm.
[speaker002:] So, it probably doesn't matter that much either way. But, um, when you say u uh, unified do you mean, uh, it's one piece of software now, or [disfmarker]?
[speaker005:] So now we are, yeah, setting up the software.
[speaker002:] Mm hmm.
[speaker005:] Um, it should be ready, uh, very soon. Um, and we
[speaker001:] So what's [disfmarker] what's happened? I think I've missed something.
[speaker002:] OK. So a week ago [disfmarker] maybe you weren't around when [disfmarker] when [disfmarker] when Hynek and Guenther and I [disfmarker]?
[speaker003:] Hynek was here.
[speaker001:] Yeah. I didn't.
[speaker002:] Oh, OK. So [disfmarker] Yeah, let's summarize. Um [disfmarker] And then if I summarize somebody can tell me if I'm wrong, which will also be possibly helpful. What did I just press here? I hope this is still working.
[speaker005:] p p p
[speaker002:] We, uh [disfmarker] we looked at, [nonvocalsound] uh [disfmarker] anyway we [disfmarker] [vocalsound] after coming back from QualComm we had, you know, very strong feedback and, uh, I think it was [vocalsound] Hynek and Guenter's and my opinion also that, um, you know, we sort of spread out to look at a number of different ways of doing noise suppression. But given the limited time, uh, it was sort of time to [pause] choose one.
[speaker001:] Mm hmm. Mmm.
[speaker002:] Uh, and so, uh, th the vector Taylor series hadn't really worked out that much. Uh, the subspace stuff, uh, had not been worked with so much. Um, so it sort of came down to spectral subtraction versus Wiener filtering.
[speaker001:] Hmm.
[speaker002:] Uh, we had a long discussion about how they were the same and how they were d uh, completely different.
[speaker001:] Mm hmm.
[speaker002:] And, uh, I mean, fundamentally they're the same sort of thing but the math is a little different so that there's a [disfmarker] a [disfmarker] [vocalsound] there's an exponent difference in the index [disfmarker] you know, what's the ideal filtering, and depending on how you construct the problem.
[speaker001:] Uh huh.
[speaker002:] And, uh, I guess it's sort [disfmarker] you know, after [disfmarker] after that meeting it sort of made more sense to me because [vocalsound] um, if you're dealing with power spectra then how are you gonna choose your error? And typically you'll do [disfmarker] choose something like a variance. And so that means it'll be something like the square of the power spectra.
[speaker003:] Mm hmm.
[speaker002:] Whereas when you're [disfmarker] when you're doing the [disfmarker] the, uh, um, [vocalsound] looking at it the other way, you're gonna be dealing with signals and you're gonna end up looking at power [disfmarker] uh, noise power that you're trying to reduce. And so, eh [disfmarker] so there should be a difference [vocalsound] of [disfmarker] you know, conceptually of [disfmarker] of, uh, a factor of two in the exponent.
[speaker001:] Mm hmm.
[speaker002:] But there're so many different little factors that you adjust in terms of [disfmarker] of, uh, [vocalsound] uh, over subtraction and [disfmarker] and [disfmarker] and [disfmarker] and [disfmarker] and so forth, um, that [vocalsound] arguably, you're c and [disfmarker] and [disfmarker] and the choice of do you [disfmarker] do you operate on the mel bands or do you operate on the FFT beforehand. There're so many other choices to make that are [disfmarker] are almost [disfmarker] well, if not independent, certainly in addition to [pause] the choice of whether you, uh, do spectral subtraction or Wiener filtering, that, um, [vocalsound] [@ @] again we sort of felt the gang should just sort of figure out which it is they wanna do and then let's pick it, go forward with it. So that's [disfmarker] that was [disfmarker] that was last week. And [disfmarker] [vocalsound] and, uh, we said, uh, take a week, go arm wrestle, you know,
[speaker004:] Oh.
[speaker002:] figure it out. I mean, and th the joke there was that each of them had specialized in one of them. And [disfmarker] and so they [disfmarker]
[speaker001:] Oh, OK.
[speaker002:] so instead they went to Yosemite and bonded, and [disfmarker] and they came out with a single [disfmarker] single piece of software. So it's [vocalsound] another [disfmarker] another victory for international collaboration. So. Uh.
[speaker001:] So [disfmarker] so you guys have combined [disfmarker] or you're going to be combining the software?
[speaker005:] Oh boy.
[speaker003:] Well, the piece of software has, like, plenty of options, like you can parse command line arguments. So depending on that, it [disfmarker] it becomes either spectral subtraction or Wiener filtering.
[speaker001:] Oh, OK.
[speaker003:] So, ye
[speaker002:] Well, that's fine,
[speaker001:] They're close enough.
[speaker002:] but the thing is [disfmarker] the important thing is that there is a piece of software that you [disfmarker] that we all will be using now.
[speaker003:] Yeah. Yeah.
[speaker002:] Yes.
[speaker005:] Yeah.
[speaker003:] There's just one piece of software.
[speaker002:] Yeah.
[speaker005:] I need to allow it to do everything and even more [disfmarker] more than this. Well, if we want to, like, optimize different parameters of [disfmarker]
[speaker003:] Right. Parameters. Yeah.
[speaker002:] Sure.
[speaker005:] Yeah, we can do it later. But, still [disfmarker] so, there will be a piece of software with, [vocalsound] [vocalsound] uh, will give this system, the fifty three point sixty six, by default and [disfmarker]
[speaker002:] Mm hmm.
[speaker001:] How [disfmarker] how is [disfmarker] how good is that?
[speaker005:] Mm hmm.
[speaker001:] I [disfmarker] I [disfmarker] I don't have a sense of [disfmarker]
[speaker005:] It's just one percent off of the [pause] best proposal.
[speaker003:] Best system.
[speaker005:] It's between [disfmarker] i we are second actually if we take this system. Right?
[speaker003:] Yeah.
[speaker002:] Yeah.
[speaker001:] OK. Compared to the last evaluation numbers? Yeah.
[speaker002:] But, uh [disfmarker]
[speaker005:] Mm hmm. Yeah.
[speaker003:] Yeah.
[speaker002:] w which we sort of were before but we were considerably far behind. And the thing is, this doesn't have neural net in yet for instance.
[speaker005:] Mm hmm.
[speaker002:] You know?
[speaker001:] Hmm.
[speaker002:] So it [disfmarker] so, um, it's [disfmarker] it it's not using our full bal bag of tricks, if you will.
[speaker001:] Mm hmm.
[speaker002:] And, uh, and it [disfmarker] it is, uh, very close in performance to the best thing that was there before. Uh, but, you know, looking at it another way, maybe more importantly, uh, [vocalsound] we didn't have any explicit noise, uh, handling [disfmarker] stationary [disfmarker] dealing with [disfmarker] e e we didn't explicitly have anything to deal with stationary noise.
[speaker001:] Mm hmm.
[speaker002:] And now we do.
[speaker001:] So will the [pause] neural net operate on the output from either the Wiener filtering or the spectral subtraction?
[speaker002:] Well, so [disfmarker] so [disfmarker] so argu arguably, I mean, what we should do [disfmarker]
[speaker001:] Or will it operate on the original?
[speaker002:] I mean, I gather you have [disfmarker] it sounds like you have a few more days of [disfmarker] of nailing things down with the software and so on. But [disfmarker] and then [disfmarker] but, um, [vocalsound] arguably what we should do is, even though the software can do many things, we should for now pick a set of things, th these things I would guess,
[speaker005:] Mm hmm.
[speaker002:] and not change that. And then focus on [pause] everything that's left. And I think, you know, that our goal should be by next week, when Hynek comes back, [vocalsound] uh, to [disfmarker] uh, really just to have a firm path, uh, for the [disfmarker] you know, for the time he's gone, of [disfmarker] of, uh, what things will be attacked. But I would [disfmarker] I would [disfmarker] I would thought think that what we would wanna do is not futz with this stuff for a while because what'll happen is we'll change many other things in the system,
[speaker001:] Mm hmm.
[speaker002:] and then we'll probably wanna come back to this and possibly make some other choices. But, um.
[speaker001:] But just conceptually, where does the neural net go? Do [disfmarker] do you wanna h run it on the output of the spectrally subtracted [disfmarker]?
[speaker005:] Mmm.
[speaker002:] Well, depending on its size [disfmarker] Well, one question is, is it on the, um, server side or is it on the terminal side? Uh, if it's on the server side, it [disfmarker] you probably don't have to worry too much about size.
[speaker001:] Mm hmm.
[speaker002:] So that's kind of an argument for that. We do still, however, have to consider its latency. So the issue is [disfmarker] is, um, [vocalsound] for instance, could we have a neural net that only looked at the past?
[speaker001:] Right.
[speaker002:] Um, what we've done in uh [disfmarker] in the past is to use the neural net, uh, to transform, [vocalsound] um, all of the features that we use. So this is done early on. This is essentially, [vocalsound] um, um [disfmarker] I guess it's [disfmarker] it's more or less like a spee a speech enhancement technique here [disfmarker] right? [disfmarker]
[speaker001:] Mm hmm.
[speaker002:] where we're just kind of creating [vocalsound] new [disfmarker] if not new speech at least new [disfmarker] new FFT's that [disfmarker] that have [disfmarker] you know, which could be turned into speech [disfmarker]
[speaker005:] Mm hmm.
[speaker002:] uh, that [disfmarker] that have some of the noise removed.
[speaker001:] Mm hmm.
[speaker002:] Um, after that we still do a mess of other things to [disfmarker] to produce a bunch of features.
[speaker001:] Right.
[speaker002:] And then those features are not now currently transformed [vocalsound] by the neural net. And then the [disfmarker] the way that we had it in our proposal two before, we had the neural net transformed features and we had [vocalsound] the untransformed features, which I guess you [disfmarker] you actually did linearly transform with the KLT,
[speaker005:] Yeah. Yeah. Right.
[speaker002:] but [disfmarker] but [disfmarker] but [disfmarker] uh, to orthogonalize them [disfmarker] but [disfmarker] [vocalsound] but they were not, uh, processed through a neural net. And Stephane's idea with that, as I recall, was that [vocalsound] you'd have one part of the feature vector that was very discriminant and another part that wasn't,
[speaker001:] Mm hmm.
[speaker002:] uh, which would smooth things a bit for those occasions when, uh, the testing set was quite different than what you'd trained your discriminant features for. So, um, all of that is [disfmarker] is, uh [disfmarker] still seems like a good idea. The thing is now we know some other constraints. We can't have unlimited amounts of latency. Uh, y you know, that's still being debated by the [disfmarker] by people in Europe but, [vocalsound] uh, no matter how they end up there, it's not going to be unlimited amounts, so we have to be a little conscious of that.
[speaker001:] Yeah.
[speaker002:] Um. So there's the neural net issue. There's the VAD issue. And, uh, there's the second stream [pause] thing. And I think those that we [disfmarker] last time we agreed that those are the three things that have to get, uh, focused on.
[speaker001:] What was the issue with the VAD?
[speaker002:] Well, better [comment] ones are good.
[speaker001:] And so the w the default, uh, boundaries that they provide are [disfmarker] they're OK, but they're not all that great?
[speaker002:] I guess they still allow two hundred milliseconds on either side or some?
[speaker005:] Mm hmm.
[speaker002:] Is that what the deal is?
[speaker005:] Uh, so th um, they keep two hundred milliseconds at the beginning and end of speech. And they keep all the [disfmarker]
[speaker001:] Outside the beginnings and end.
[speaker005:] Yeah. And all the speech pauses,
[speaker001:] Uh huh.
[speaker005:] which is [disfmarker] Sometimes on the SpeechDat Car you have pauses that are more than one or two seconds.
[speaker001:] Wow.
[speaker005:] More than one second for sure. Um.
[speaker001:] Hmm.
[speaker005:] Yeah. And, yeah, it seems to us that this way of just dropping the beginning and end is not [disfmarker] We cou we can do better, I think,
[speaker001:] Mm hmm.
[speaker005:] because, um, [vocalsound] with this way of dropping the frames they improve [pause] over the baseline by fourteen percent and [vocalsound] Sunil already showed that with our current VAD we can improve by more than twenty percent.
[speaker001:] On top of the VAD that they provide?
[speaker003:] No.
[speaker005:] Just using either their VAD or our current VAD.
[speaker003:] Our way.
[speaker001:] Oh, OK.
[speaker005:] So, our current VAD is [disfmarker] is more than twenty percent, while their is fourteen.
[speaker001:] Theirs is fourteen?
[speaker005:] Yeah.
[speaker001:] I see. Huh.
[speaker005:] So. Yeah. And [pause] another thing that we did also is that we have all this training data for [disfmarker] let's say, for SpeechDat Car. We have channel zero which is clean, channel one which is far field microphone. And if we just take only the, um, VAD probabilities computed on the clean signal and apply them on the far field, uh, test utterances, [vocalsound] then results are much better.
[speaker001:] Mm hmm.
[speaker005:] In some cases it divides the error rate by two.
[speaker001:] Wow.
[speaker005:] So it means that there are stim [comment] still [disfmarker] If [disfmarker] if we can have a good VAD, well, it would be great.
[speaker001:] How [disfmarker] how much latency does the, uh [disfmarker] does our VAD add? Is it significant,
[speaker005:] Uh, right now it's, um, a neural net with nine frames.
[speaker001:] or [disfmarker]?
[speaker005:] So it's forty milliseconds plus, um, the rank ordering, which, uh, should be
[speaker003:] Like another ten frames.
[speaker005:] ten [disfmarker] Yeah. So, right now it's one hundred and forty [pause] milliseconds.
[speaker004:] Rank. Oh.
[speaker002:] With the rank ordering [disfmarker]? I'm sorry.
[speaker003:] The [disfmarker] the [disfmarker] the smoothing [disfmarker] the m the [disfmarker] the filtering of the probabilities.
[speaker005:] The [disfmarker] The, um [disfmarker]
[speaker003:] on the R.
[speaker005:] Yeah. It's not a median filtering. It's just [disfmarker] We don't take the median value. We take something [disfmarker] Um, so we have eleven, um, frames.
[speaker002:] Oh, this is for the VAD.
[speaker005:] And [disfmarker]
[speaker003:] Yeah.
[speaker005:] for the VAD, yeah [disfmarker] and we take th the third.
[speaker003:] Yeah.
[speaker002:] Oh, OK.
[speaker003:] Yeah.
[speaker004:] Dar
[speaker005:] Um.
[speaker002:] Yeah. Um.
[speaker005:] Mmm.
[speaker002:] So [disfmarker] [comment] Yeah, I was just noticing on this that it makes reference to delay. So what's the [disfmarker]? If you ignore [disfmarker] Um, the VAD is sort of in [disfmarker] in parallel, isn't i isn't it, with [disfmarker] with the [disfmarker]? I mean, it isn't additive with the [disfmarker] the, uh, LDA and the Wiener filtering, and so forth.
[speaker003:] The LDA? Yeah. So [disfmarker] so what happened right now, we removed the delay of the LDA.
[speaker002:] Right?
[speaker005:] Mm hmm.
[speaker002:] Yeah.
[speaker003:] So we [disfmarker] I mean, if [disfmarker] so if we [disfmarker] if [disfmarker] so which is like if we reduce the delay of VA So, the f the final delay's now ba is f determined by the delay of the VAD, because the LDA doesn't have any delay. So if we re if we reduce the delay of the VAD, I mean, it's like effectively reducing the delay.
[speaker001:] How [disfmarker] how much, uh, delay was there on the LDA?
[speaker003:] So the LDA and the VAD both had a hundred millisecond delay. So and they were in parallel, so which means you pick either one of them [disfmarker]
[speaker001:] Mmm.
[speaker003:] the [disfmarker] the biggest, whatever.
[speaker002:] Mm hmm.
[speaker001:] I see.
[speaker003:] So, right now the LDA delays are more.
[speaker002:] And there [disfmarker]
[speaker001:] Oh, OK.
[speaker002:] And there didn't seem to be any, uh, penalty for that?
[speaker003:] Pardon?
[speaker002:] There didn't seem to be any penalty for making it causal?
[speaker003:] Oh, no. It actually made it, like, point one percent better or something, actually.
[speaker002:] OK.
[speaker003:] Or something like that
[speaker002:] Well, may as well, then.
[speaker003:] and [disfmarker]
[speaker002:] And he says Wiener filter is [disfmarker] is forty milliseconds delay.
[speaker003:] Yeah. So that's the one which Stephane was discussing, like [disfmarker]
[speaker005:] Mmm.
[speaker002:] So is it [disfmarker]? The smoothing?
[speaker003:] Yeah. The [disfmarker] you smooth it and then delay the decision by [disfmarker] So.
[speaker002:] Right. OK. So that's [disfmarker] that's really not [disfmarker] not bad. So we may in fact [disfmarker] we'll see what they decide. We may in fact have, [vocalsound] um, the [disfmarker] the, uh, latency time available for [disfmarker] to have a neural net. I mean, sounds like we probably will.
[speaker003:] Mm hmm.
[speaker002:] So. That'd be good. Cuz I [disfmarker] cuz it certainly always helped us before. So. Uh.
[speaker001:] What amount of latency are you thinking about when you say that?
[speaker002:] Well, they're [disfmarker] you know, they're disputing it. You know, they're saying, uh [disfmarker] one group is saying a hundred and thirty milliseconds and another group is saying two hundred and fifty milliseconds.
[speaker001:] Mmm.
[speaker002:] Two hundred and fifty is what it was before actually. So,
[speaker001:] Oh.
[speaker002:] uh, some people are lobbying [disfmarker] lobbying [comment] to make it shorter.
[speaker001:] Hmm.
[speaker002:] Um. And, um.
[speaker001:] Were you thinking of the two fifty or the one thirty when you said we should [pause] have enough for the neural net?
[speaker002:] Well, it just [disfmarker] it [disfmarker] when we find that out it might change exactly how we do it, is all. I mean, how much effort do we put into making it causal?
[speaker001:] Oh, OK.
[speaker002:] I mean, [vocalsound] I think the neural net will probably do better if it looks at a little bit of the future.
[speaker001:] Mm hmm.
[speaker002:] But, um, it will probably work to some extent to look only at the past. And we ha you know, limited machine and human time, and [vocalsound] effort. And, you know, how [disfmarker] how much time should we put into [disfmarker] into that? So it'd be helpful if we find out from the [disfmarker] the standards folks whether, you know, they're gonna restrict that or not.
[speaker001:] Mm hmm.
[speaker002:] Um. But I think, you know, at this point our major concern is making the performance better and [disfmarker] and, um, [vocalsound] if, uh, something has to take a little longer in latency in order to do it that's [pause] you know, a secondary issue.
[speaker001:] Mm hmm.
[speaker002:] But if we get told otherwise then, you know, we may have to c clamp down a bit more.
[speaker004:] Mmm.
[speaker003:] So, the one [disfmarker] one [disfmarker] one difference is that [disfmarker] was there is like we tried computing the delta and then doing the frame dropping.
[speaker004:] S
[speaker005:] Mm hmm.
[speaker003:] The earlier system was do the frame dropping and then compute the delta on the [disfmarker]
[speaker002:] Uh huh.
[speaker003:] So this [disfmarker]
[speaker001:] Which could be a kind of a funny delta. Right?
[speaker003:] Yeah.
[speaker002:] Oh, oh. So that's fixed in this. Yeah, we talked about that.
[speaker003:] Yeah.
[speaker005:] Yeah. Uh huh.
[speaker003:] So we have no delta. And then [disfmarker]
[speaker002:] Good.
[speaker003:] So the frame dropping is the last thing that we do. So, yeah, what we do is we compute the silence probability, convert it to that binary flag,
[speaker002:] Uh huh.
[speaker003:] and then in the end you c up upsample it to [vocalsound] match the final features number of [disfmarker]
[speaker005:] Mm hmm.
[speaker001:] Did that help then?
[speaker003:] It seems to be helping on the well matched condition. So that's why this improvement I got from the last result. So. And it actually r reduced a little bit on the high mismatch, so in the final weightage it's b b better because the well matched is still weighted more than [disfmarker]
[speaker002:] So, [@ @] I mean, you were doing a lot of changes. Did you happen to notice how much, [vocalsound] uh, the change was due to just this frame dropping problem? What about this?
[speaker003:] Uh, y you had something on it. Right?
[speaker005:] Just the frame dropping problem. Yeah. But it's [disfmarker] it's difficult. Sometime we [disfmarker] we change two [disfmarker] two things together and [disfmarker] But it's around [pause] maybe [disfmarker] it's less than one percent.
[speaker002:] Uh huh.
[speaker003:] Yeah.
[speaker005:] It [disfmarker]
[speaker002:] Well. [vocalsound] But like we're saying, if there's four or five things like that then [vocalsound] pretty sho soon you're talking real improvement.
[speaker005:] Yeah. Yeah. And it [disfmarker] Yeah. And then we have to be careful with that also [disfmarker] with the neural net
[speaker002:] Yeah.
[speaker005:] because in [comment] the proposal the neural net was also, uh, working on [disfmarker] after frame dropping.
[speaker002:] Mm hmm.
[speaker005:] Um.
[speaker002:] Oh, that's a real good point.
[speaker005:] So. Well, we'll have to be [disfmarker] to do the same kind of correction.
[speaker002:] It might be hard if it's at the server side. Right?
[speaker005:] Mmm. Well, we can do the frame dropping on the server side or we can just be careful at the terminal side to send a couple of more frames before and after, and [disfmarker] So. I think it's OK.
[speaker002:] OK.
[speaker001:] You have, um [disfmarker] So when you [disfmarker] Uh, maybe I don't quite understand how this works, but, um, couldn't you just send all of the frames, but mark the ones that are supposed to be dropped? Cuz you have a bunch more bandwidth. Right?
[speaker002:] Well, you could. Yeah. I mean, it [disfmarker] it always seemed to us that it would be kind of nice to [disfmarker] in addition to, uh, reducing insertions, actually use up less bandwidth.
[speaker001:] Yeah. Yeah.
[speaker002:] But nobody seems to have [vocalsound] cared about that in this [pause] evaluation.
[speaker001:] And that way the net could use [disfmarker]
[speaker002:] So.
[speaker001:] If the net's on the server side then it could use all of the [pause] frames.
[speaker003:] Yes, it could be. It's, like, you mean you just transferred everything and then finally drop the frames after the neural net. Right?
[speaker001:] Mm hmm.
[speaker005:] Mm hmm.
[speaker003:] Yeah. That's [disfmarker] that's one thing which [disfmarker]
[speaker001:] But you could even mark them, before they get to the server.
[speaker003:] Yeah. Right now we are [disfmarker] Uh, ri Right now what [disfmarker] wha what we did is, like, we just mark [disfmarker] we just have this additional bit which goes around the features, [vocalsound] saying it's currently a [disfmarker] it's a speech or a nonspeech.
[speaker001:] Oh, OK.
[speaker003:] So there is no frame dropping till the final features, like, including the deltas are computed.
[speaker001:] I see.
[speaker003:] And after the deltas are computed, you just pick up the ones that are marked silence and then drop them.
[speaker001:] Mm hmm. I see.
[speaker002:] So it would be more or less the same thing with the neural net, I guess, actually.
[speaker005:] Mm hmm.
[speaker001:] I see.
[speaker003:] So. Yeah, that's what [disfmarker] that's what [disfmarker] that's what, uh, this is doing right now.
[speaker001:] I see. OK.
[speaker002:] Yeah.
[speaker005:] Mm hmm.
[speaker002:] Um. OK. So, uh, what's, uh [disfmarker]? That's [disfmarker] that's a good set of work that [disfmarker] that, uh [disfmarker]
[speaker003:] Just one more thing. Like, should we do something f more for the noise estimation, because we still [disfmarker]?
[speaker002:] Yeah. I was wondering about that.
[speaker003:] Yeah.
[speaker005:] Mm hmm.
[speaker002:] That was [disfmarker] I [disfmarker] I had written that down there. Um [disfmarker]
[speaker005:] So, we, uh [disfmarker] actually I did the first experiment. This is [pause] with just fifteen frames. Um. We take the first fifteen frame of each utterance to it,
[speaker002:] Yeah.
[speaker005:] and average their power spectra. Um. I tried just plugging the, um, [vocalsound] uh, Guenter noise estimation on this system, and it [disfmarker] uh, it got worse. Um, but of course I didn't play [pause] with it.
[speaker002:] Uh huh.
[speaker005:] But [disfmarker] Mm hmm. Uh, I didn't [pause] do much more [pause] for noise estimation. I just tried this, and [disfmarker]
[speaker002:] Hmm. Yeah. Well, it's not surprising it'd be worse the first time.
[speaker005:] Mm hmm.
[speaker002:] But, um, it does seem like, you know, i i i i some compromise between always depending on the first fifteen frames and a a always depending on a [disfmarker] a pause is [disfmarker] is [disfmarker] is a good idea. Uh, maybe you have to weight the estimate from the first teen [disfmarker] fifteen frames more heavily than [disfmarker] than was done in your first attempt. But [disfmarker]
[speaker005:] Mm hmm.
[speaker002:] but [disfmarker]
[speaker005:] Yeah, I guess.
[speaker002:] Yeah. Um. No, I mean [disfmarker] Um, do you have any way of assessing how well or how poorly the noise estimation is currently doing?
[speaker005:] Mmm. No, we don't.
[speaker002:] Yeah.
[speaker005:] We don't have nothing [pause] that [disfmarker]
[speaker003:] Is there [disfmarker] was there any experiment with [disfmarker]? Well, I [disfmarker] I did [disfmarker] The only experiment where I tried was I used the channel zero VAD for the noise estimation and frame dropping. So I don't have a [disfmarker] [vocalsound] I don't have a split, like which one helped more.
[speaker005:] Yeah.
[speaker003:] So. It [disfmarker] it was the best result I could get.
[speaker005:] Mm hmm.
[speaker003:] So, that's the [disfmarker]
[speaker002:] So that's something you could do with, um, this final system. Right? Just do this [disfmarker] everything that is in this final system except, [vocalsound] uh, use the channel zero.
[speaker003:] Mm hmm. For the noise estimation.
[speaker002:] Yeah.
[speaker003:] Yeah. We can try something.
[speaker002:] And then see how much better it gets.
[speaker003:] Mm hmm. Sure.
[speaker002:] If it's, you know, essentially not better, then [pause] it's probably not worth
[speaker005:] Yeah.
[speaker002:] any more.
[speaker003:] Yeah. But the Guenter's argument is slightly different. It's, like, ev even [disfmarker] even if I use a channel zero VAD, I'm just averaging the [disfmarker] [vocalsound] the s power spectrum. But the Guenter's argument is, like, if it is a non stationary [pause] segment, then he doesn't update the noise spectrum. So he's, like [disfmarker] he tries to capture only the stationary part in it. So the averaging is, like, [vocalsound] different from [pause] updating the noise spectrum only during stationary segments. So, th the Guenter was arguing that, I mean, even if you have a very good VAD, averaging it, like, over the whole thing is not a good idea. Because you're averaging the stationary and the non stationary, and finally you end up getting something
[speaker002:] I see.
[speaker003:] which is not really the s because, you [disfmarker] anyway, you can't remove the stationary part fr I mean, non stationary part from [vocalsound] the signal. So [disfmarker]
[speaker002:] Not using these methods anyway. Yeah.
[speaker003:] Yeah. So you just [pause] update only doing [disfmarker] or update only the stationary components. Yeah. So, that's [disfmarker] so that's still a slight difference from what Guenter is trying
[speaker002:] Well, yeah. And [disfmarker] and also there's just the fact that, um, eh, uh, although we're trying to do very well on this evaluation, um, we actually would like to have something that worked well in general.
[speaker003:] Yeah, yeah.
[speaker002:] And, um, relying on having fifteen frames at the front or something is [disfmarker] is pretty [disfmarker] I mean, you might, you might not.
[speaker003:] Mmm.
[speaker005:] Mm hmm.
[speaker002:] So, um. Um, it'd certainly be more robust to different kinds of input if you had at least some updates. Um.
[speaker005:] Mm hmm.
[speaker002:] But, um. Well, I don't know. What [disfmarker] what do you, uh [disfmarker] what do you guys see as [disfmarker] as being what you would be doing in the next week, given wha what's [pause] happened?
[speaker003:] Cure the VAD?
[speaker005:] Yeah.
[speaker001:] What was that?
[speaker003:] VAD.
[speaker001:] Oh.
[speaker003:] And [disfmarker]
[speaker002:] OK.
[speaker005:] So, should we keep the same [disfmarker]? I think we might try to keep the same idea of having a neural network, but [vocalsound] training it on more data and adding better features, I think, but [disfmarker] because the current network is just PLP features. Well, it's trained on noisy [pause] PLP [disfmarker]
[speaker003:] Just the cepstra. Yeah.
[speaker005:] PLP features computed on noisy speech. But [vocalsound] [vocalsound] there is no nothing particularly robust in these features.
[speaker001:] So, I I uh [disfmarker]
[speaker003:] No.
[speaker005:] There's no RASTA, no [disfmarker]
[speaker001:] So, uh, I [disfmarker] I don't remember what you said [vocalsound] the answer to my, uh, question earlier. Will you [disfmarker] will you train the net on [disfmarker] after you've done the spectral subtraction or the Wiener filtering?
[speaker002:] This is a different net.
[speaker001:] Oh.
[speaker003:] So we have a VAD which is like neur that's a neural net.
[speaker005:] Oh, yeah. Hmm.
[speaker001:] Oh, you're talking about the VAD net.
[speaker003:] Yeah.
[speaker001:] OK.
[speaker005:] Mm hmm.
[speaker001:] I see.
[speaker003:] So that [disfmarker] that VAD was trained on the noisy features.
[speaker001:] Mm hmm.
[speaker003:] So, right now we have, like, uh [disfmarker] we have the cleaned up features, so we can have a better VAD by training the net on [pause] the cleaned up speech.
[speaker001:] Mm hmm. I see. I see.
[speaker003:] Yeah, but we need a VAD for uh noise estimation also. So it's, like, where do we want to put the VAD? Uh, it's like [disfmarker]
[speaker001:] Can you use the same net to do both, or [disfmarker]?
[speaker003:] For [disfmarker]
[speaker001:] Can you use the same net that you [disfmarker] that I was talking about to do the VAD?
[speaker003:] Mm hmm. Uh, it actually comes at v at the very end. So the net [disfmarker] the final net [disfmarker] I mean, which is the feature net [disfmarker]
[speaker001:] Mm hmm.
[speaker003:] so that actually comes after a chain of, like, LDA plus everything. So it's, like, it takes a long time to get a decision out of it. And [disfmarker] [vocalsound] and you can actually do it for final frame dropping, but not for the VA f noise estimation.
[speaker001:] Mm hmm.
[speaker002:] You see, the idea is that the, um, initial decision to [disfmarker] that [disfmarker] that you're in silence or speech happens pretty quickly.
[speaker001:] Oh, OK.
[speaker003:] Hmm.
[speaker002:] And that [disfmarker]
[speaker001:] Cuz that's used by some of these other [disfmarker]?
[speaker002:] Yeah. And that's sort of fed forward, and [disfmarker] and you say "well, flush everything, it's not speech anymore".
[speaker001:] Oh, OK. I see.
[speaker003:] Yeah.
[speaker001:] I thought that was only used for doing frame dropping later on.
[speaker002:] Um, it is used, uh [disfmarker] Yeah, it's only used f Well, it's used for frame dropping. Um, it's used for end of utterance
[speaker005:] Mmm.
[speaker002:] because, you know, there's [disfmarker] [vocalsound] if you have [pause] more than five hundred milliseconds of [disfmarker] of [disfmarker] of nonspeech then you figure it's end of utterance or something like that.
[speaker001:] Mm hmm.
[speaker002:] So, um.
[speaker005:] And it seems important for, like, the on line normalization. Um. We don't want to update the mean and variance during silen long silence portions. Um. So it [disfmarker] it has to be done before
[speaker001:] Oh. I see.
[speaker005:] this mean and variance normalization. Um.
[speaker002:] Um. Yeah. So probably the VAD and [disfmarker] and maybe testing out the noise [pause] estimation a little bit. I mean, keeping the same method but [disfmarker] but, uh, [vocalsound] seeing if you cou but, um noise estimation could be improved.
[speaker005:] Mm hmm.
[speaker002:] Those are sort of related issues. It probably makes sense to move from there. And then, uh, [vocalsound] later on in the month I think we wanna start including the [pause] neural net at the end. Um. OK. Anything else?
[speaker005:] The Half Dome was great.
[speaker002:] Good. Yeah. You didn't [disfmarker] didn't fall.
[speaker003:] Well, yeah.
[speaker002:] That's good. Our e our effort would have been devastated if you guys had [comment] [vocalsound] run into problems.
[speaker001:] So, Hynek is coming back next week, you said?
[speaker002:] Yeah, that's the plan.
[speaker001:] Hmm.
[speaker002:] I guess the week after he'll be, uh, going back to Europe, and so we wanna [disfmarker]
[speaker001:] Is he in Europe right now or is he up at [disfmarker]?
[speaker002:] No, no. He's [disfmarker] he's [disfmarker] he's dropped into the US. Yeah. Yeah.
[speaker001:] Oh. Hmm.
[speaker002:] So. Uh. [vocalsound] So, uh. Uh, the idea was that, uh, we'd [disfmarker] we'd sort out where we were going next with this [disfmarker] with this work before he, uh, left on this next trip. Good. [vocalsound] [vocalsound] Uh, Barry, you just got through your [vocalsound] quals, so I don't know if you [vocalsound] have much to say. But, uh.
[speaker004:] Mmm. No, just, uh, looking into some [disfmarker] some of the things that, um, [vocalsound] uh, John Ohala and Hynek, um, gave as feedback, um, as [disfmarker] as a starting point for the project. Um. In [disfmarker] in my proposal, I [disfmarker] I was thinking about starting from a set of, uh, phonological features, [vocalsound] or a subset of them. Um, but that might not be necessarily a good idea according to, um, John.
[speaker001:] Mm hmm.
[speaker004:] He said, uh, um, these [disfmarker] these phonological features are [disfmarker] are sort of figments of imagination also.
[speaker001:] Mm hmm.
[speaker004:] Um. S
[speaker002:] In conversational speech in particular.
[speaker004:] Ye
[speaker002:] I think you can [disfmarker] you can put them in pretty reliably in synthetic speech. But [vocalsound] we don't have too much trouble recognizing synthetic speech since we create it in the first place. So, it's [disfmarker]
[speaker004:] Right. Yeah. So, um, a better way would be something more [disfmarker] more data driven,
[speaker001:] Mm hmm.
[speaker004:] just looking at the data and seeing what's similar and what's not similar.
[speaker001:] Mm hmm.
[speaker004:] So, I'm [disfmarker] I'm, um, taking a look at some of, um, [vocalsound] Sangita's work on [disfmarker] on TRAPS. She did something where, um [disfmarker] [vocalsound] w where the TRAPS learn She clustered the [disfmarker] the temporal patterns of, um, certain [disfmarker] certain phonemes in [disfmarker] in m averaged over many, many contexts. And, uh, some things tended to cluster.
[speaker001:] Mm hmm.
[speaker004:] Right? You know, like stop [disfmarker] stop consonants clustered really well.
[speaker001:] Hmm.
[speaker004:] Um, silence was by its own self.
[speaker001:] Mm hmm.
[speaker004:] And, uh, um, [vocalsound] v vocalic was clustered.
[speaker001:] Mm hmm.
[speaker004:] And, [vocalsound] um, so, [vocalsound] those are [pause] interesting things to [disfmarker]
[speaker001:] So you're [disfmarker] now you're sort of looking to try to gather a set of these types of features?
[speaker004:] Right.
[speaker001:] Mm hmm.
[speaker004:] Yeah. Just to see where [disfmarker] where I could start off from,
[speaker001:] Mm hmm.
[speaker004:] uh, you know? A [disfmarker] a [disfmarker] a set of small features and continue to iterate and find, uh, a better set.
[speaker001:] Mm hmm.
[speaker004:] Yeah.
[speaker002:] OK. Well, short meeting. That's OK.
[speaker001:] Yeah.
[speaker002:] OK. So next week hopefully we'll [disfmarker] can get Hynek here to [disfmarker] to join us and, uh, uh.
[speaker001:] Should we do digits?
[speaker002:] Digits, digits. OK, now.
[speaker001:] Go ahead, Morgan.
[speaker002:] Alright. Let me get my glasses on so I can [pause] see them.
[speaker001:] You can start.
[speaker002:] OK.
[speaker001:] OK. And we're off.
[speaker002:] Mm