Professor B: OK.PhD C: Oh, I don't {disfmarker}PhD A: I think I'm zero.Professor B: Wow! Unprecedented.PhD C: Hello, hello, hello, hello.PhD E: AhGrad F: Wh - what causes the crashPhD A: Did you fix somethingPhD C: Hello.PhD E: Five, five.PhD C: Hello, hello.Grad F: Oh, maybe it's the turning {disfmarker} turning off and turning on of the mike, rightProfessor B: Uh, you think that's you Oh.PhD C: Aaa - aaa - aaa.Grad F: Yeah, OK, mine's working.PhD C: OK. That's me.Professor B: OK. OK. So, um I guess we are {pause} um {pause} gonna do the digits at the end. UhPhD D: Channel {disfmarker} channel three, yeah.PhD C: Channel two.PhD D: OK.PhD E: Mmm, channel five Doesn't workProfessor B: Yeah, that's the mike number there, uh {pause} Uh, mike number five, and {pause} channel {disfmarker} channel four.PhD C: Two.PhD A: Is it written on her sheet, I believe.PhD E: No Ah,PhD D: Mike four.Grad F: Watch this.PhD E: era el cuatro.Grad F: Yep, that's me.PhD E: Yeah.PhD A: But, channelPhD E: Yeah yeah yeah.Professor B: This is you.PhD E: OK. I saw that. Ah {disfmarker} yeah, it's OK.Professor B: Yeah. And I'm channel uh two I think,PhD C: Ooo.Professor B: or channel {disfmarker}PhD C: I think I'm channel two.Professor B: Oh, I'm channel {disfmarker} must be channel one. Channel onePhD E: Channel {disfmarker} {vocalsound} I decided to talk about that.Professor B: Yes, OK. OK. So uh {pause} I also copied uh the results that we all got in the mail I think from uh {disfmarker} {pause} from OGI and we'll go {disfmarker} go through them also. So where are we on {disfmarker} {pause} on uh {vocalsound} {pause} our runsPhD D: Uh so. {pause} uh {disfmarker} We {disfmarker} So {pause} As I was already said, we {disfmarker} we mainly focused on uh four kind of features.Professor B: Excuse me.PhD D: The PLP, the PLP with JRASTA, the MSG, and the MFCC from the baseline Aurora.Professor B: Mm - hmm.PhD D: Uh, and we focused for the {disfmarker} the test part on the English and the Italian. Um. We've trained uh several neural networks on {disfmarker} so {disfmarker} on the TI - digits English {pause} and on the Italian data and also on the broad uh {pause} English uh French and uh Spanish databases. Mmm, so there's our result tables here, for the tandem approach, and um, actually what we {disfmarker} we @ @ observed is that if the network is trained on the task data it works pretty well.Professor B: OK. Our {disfmarker} our uh {disfmarker} {pause} There's a {disfmarker} {pause} We're pausing for a photo {disfmarker}PhD C: Chicken on the grill. Try that corner.PhD A: How about over th from the front of the roomPhD C: Yeah, it's longer.Professor B: We're pausing for a photo opportunity here. Uh. {vocalsound} Uh. So.Grad F: Oh wait wait wait wait wait. Wait.PhD C: Get out of the {disfmarker} Yeah.Grad F: Hold on. Hold on.Professor B: OK.Grad F: Let me give you a black screen.Professor B: He's facing this way. What OK, this {disfmarker} this would be a {pause} good section for our silence detection.Grad F: OK.PhD C: Mm - hmm.Professor B: Um Oh.Grad F: Musical chairs everybody!Professor B: OK. So um, {pause} you were saying {pause} about the training data {disfmarker} Yeah.PhD D: Yeah, so if the network is trained on the task data um {pause} tandem works pretty well. And uh actually we have uh, results are similar Only on,PhD A: Do you mean if it's trained only on {disfmarker} On data from just that task,PhD D: yeah.PhD A: that languagePhD D: Just that task. But actually we didn't train network on {pause} uh both types of data I mean {pause} uh {pause} phonetically ba phonetically balanced uh data and task data.PhD A: Mmm.PhD D: We only did either task {disfmarker} task data or {pause} uh broad {pause} data.PhD A: Mm - hmm.PhD D: Um {pause} Yeah. So,Professor B: So how {disfmarker} I mean {disfmarker} clearly it's gonna be good thenPhD A: So what's thProfessor B: but the question is how much {pause} worse is it {pause} if you have broad data I mean, {pause} my assump From what I saw from the earlier results, uh I guess last week, {pause} was that um, {pause} if you {pause} trained on one language and tested on another, say, that {pause} the results were {disfmarker} were relatively poor.PhD D: Mmm. Yeah.Professor B: But {disfmarker} but the question is if you train on one language {pause} but you have a broad coverage {pause} and then test in another, {pause} does that {disfmarker} {pause} is that improve things {pause} i c in comparisonPhD D: If we use the same languageProfessor B: No, no, no. Different lang So {pause} um {pause} If you train on TI - digits {pause} and test on Italian digits, {pause} you do poorly, {pause} let's say.PhD D: Mm - hmm.Professor B: I don't have the numbers in front of me,PhD D: But {disfmarker} Yeah but I did not uh do that.Professor B: so I'm just imagining. E So, you didn't train on {pause} TIMIT and test on {disfmarker} {pause} on Italian digits, sayPhD D: We {disfmarker} No, we did four {disfmarker} four kind of {disfmarker} of testing, actually. The first testing is {pause} with task data {disfmarker} So, with nets trained on task data. So for Italian on the Italian speech @ @. The second test is trained on a single language um with broad database, but the same language as the t task data.Professor B: OK.PhD D: But for Italian we choose Spanish which {pause} we assume is close to Italian. The third test is by using, um the three language databaseProfessor B: W which in {disfmarker}PhD D: and the fourth isProfessor B: It has three languages. That's including the w the {disfmarker} {pause} the {disfmarker}PhD D: This includes {disfmarker}Professor B: the one that it's {disfmarker}PhD D: Yeah.PhD A: InPhD D: But {pause} not digits. I mean it's {disfmarker}PhD A: The three languages {pause} is not digits,Professor B: Right.PhD A: it's the broad {pause} data. OK.PhD D: Yeah And the fourth test is uh {pause} excluding from these three languages the language {pause} that is {pause} the task language.Professor B: Oh, OK, yeah, so, that is what I wanted to know.PhD D: Yeah.Professor B: I just wasn't saying it very well, I guess.PhD D: Uh, yeah. So um {pause} for uh TI - digits for ins example {pause} uh when we go from TI - digits training to {pause} TIMIT training {pause} uh we lose {pause} uh around ten percent, uh. The error rate increase u of {disfmarker} of {disfmarker} of ten percent, relative.Professor B: Relative. Right.PhD D: So this is not so bad. And then when we jump to the multilingual data it's uh it become worse and, well Around uh, let's say, {pause} twenty perc twenty percent further.Professor B: Ab - about how muchPhD D: So. Yeah.Professor B: Twenty percent furtherPhD D: Twenty to {disfmarker} to thirty percent further. Yeah.PhD A: And so, remind me, the multilingual stuff is just the broad data. Right It's not the digits.PhD D: Yeah.PhD A: So it's the combination of {pause} two things there. It's {pause} removing the {pause} task specific {pause} training and {pause} it's adding other languages.PhD D: Yeah. Yeah.PhD A: OK.PhD D: But the first step is al already removing the task s specific from {disfmarker} from {disfmarker}PhD A: Already, right right right.PhD D: So.PhD A: So they were sort of building {pause} herePhD D: And we lose {disfmarker}PhD A: OKPhD D: Yeah. Uh {pause} So, basically when it's trained on the {disfmarker} the multilingual broad data {pause} um or number {disfmarker} so, the {disfmarker} the {pause} ratio of our error rates uh with the {pause} baseline error rate is around {pause} uh one point one.Professor B: Yes. {vocalsound} And it's something like one point three of {disfmarker} of the {pause} uh {disfmarker}PhD D: So.Professor B: I i if you compare everything to the first case at the baseline, you get something like one point one for the {disfmarker} for the using the same language but a different task, and something like one point three {pause} for three {disfmarker} three languages {pause} broad stuff.PhD D: No no no. Uh same language we are at uh {disfmarker} for at English at O point eight. So it improves, {pause} compared to the baseline. But {disfmarker} So. Le - let me.Professor B: I {disfmarker} I {disfmarker} I'm sorry.PhD D: Tas - task dataProfessor B: I {disfmarker} I {disfmarker} I meant something different by baselinePhD D: we are u Yeah.Professor B: So let me {disfmarker} let me {disfmarker} Um, {pause} so, {pause} um {disfmarker}PhD D: Mmm.Professor B: OK, fine. Let's {disfmarker} let's use the conventional meaning of baseline.PhD D: Hmm.Professor B: I {disfmarker} I {disfmarker} By baseline here I meant {pause} uh using the task specific data.PhD D: Oh yeah, the f Yeah, OK.Professor B: But uh {disfmarker} {pause} uh, because that's what you were just doing with this ten percent.PhD D: Yeah.Professor B: So I was just {disfmarker} I just trying to understand that.PhD D: Yeah. Sure.Professor B: So if we call {pause} a factor of w just one, just normalized to one, the word error rate {pause} that you have {pause} for using TI - digits as {disfmarker} as {pause} training and TI - digits as test,PhD D: Mmm.Professor B: uh different words, I'm sure,PhD D: Mm - hmm.Professor B: but {disfmarker} {pause} but uh, uh the same {pause} task and so on.PhD D: Mm - hmm.Professor B: If we call that" one" , {pause} then what you're saying is {pause} that the word error rate {pause} for the same language but using {pause} uh different training data than you're testing on, say TIMIT and so forth, {pause} it's one point one.PhD D: Mm - hmm. Yeah, it's around one point one.Professor B: Right. And if it's {disfmarker}PhD D: Yeah.Professor B: you {pause} do {pause} go to {pause} three languages including the English, {pause} it's something like one point three. That's what you were just saying, I think.PhD D: Ye Uh, more actually.PhD A: One point fourPhD D: If I {disfmarker} Yeah.PhD A: So, it's an additional thirty percent.PhD D: What would you say Around one point fourProfessor B: OK.PhD D: yeah.Professor B: And if you exclude {pause} English, {pause} from this combination, what's thatPhD D: If we exclude English, {pause} um {pause} there is {pause} not much difference with the {pause} data with English.Professor B: Aha!PhD D: So. Yeah.Professor B: That's interesting. {pause} That's interesting. Do you see Because {disfmarker} Uh,PhD D: Uh.Professor B: so {disfmarker} No, that {disfmarker} that's important. So what {disfmarker} what it's saying here is just that" yes, there is a reduction {pause} in performance, {pause} when you don't {pause} um {pause} have the s {pause} when you don't have {pause} umPhD A: Task data.Professor B: Wait a minute, th th the {disfmarker}PhD D: Hmm.Professor B: No, actually {pause} it's interesting. So it's {disfmarker} So when you go to a different task, there's actually not so {pause} different. It's when you went to these {disfmarker} So what's the difference between two and three Between the one point one case and the one point four case I'm confused.PhD A: It's multilingual.PhD D: Yeah. The only difference it's {disfmarker} is that it's multilingual {disfmarker} UmProfessor B: Cuz in both {disfmarker} in both {disfmarker} both of those cases, you don't have the same task.PhD D: Yeah. Yeah sure.Professor B: So is {disfmarker} is the training data for the {disfmarker} for this one point four case {disfmarker} does it include the training data for the one point one casePhD D: Uh yeah.Grad F: Yeah, a fraction of it.PhD D: A part of it, yeah.Professor B: How m how much bigger is itPhD D: Um {pause} It's two times,Grad F: Yeah, um.PhD D: actually Yeah. Um. The English data {disfmarker} {pause} No, the multilingual databases are two times the {pause} broad English {pause} data. We just wanted to keep this, w well, not too huge. So.Professor B: So it's two times, but it includes the {disfmarker} but it includes the broad English data.PhD D: I think so. Do you {disfmarker} Uh, Yeah.Professor B: And the broad English data is what you got this one point one {pause} with. So that's TIMIT basically rightPhD D: Yeah.Grad F: Mm - hmm.Professor B: So it's band - limited TIMIT. This is all eight kilohertz sampling.PhD D: Mm - hmm.Grad F: Mm - hmm.PhD D: Yeah.Grad F: Downs Right.Professor B: So you have band - limited TIMIT, {pause} gave you uh almost as good as a result as using TI - digits {pause} on a TI - digits test. OKPhD D: HmmProfessor B: Um {pause} and {pause} um But, {pause} when you add in more training data but keep the neural net the same size, {pause} it {pause} um performs worse on the TI - digits. OK, now all of this is {disfmarker} {pause} This is noisy {pause} TI - digits, I assume Both training and testPhD D:Professor B: Yeah. OK. Um OK. Well. {pause} We {disfmarker} we {disfmarker} we may just need to uh {disfmarker} So I mean it's interesting that h going to a different {disfmarker} different task didn't seem to hurt us that much, and going to a different language um It doesn't seem to matter {disfmarker} The difference between three and four is not particularly great, so that means that {pause} whether you have the language in or not is not such a big deal.PhD D: Mmm.Professor B: It sounds like um {pause} uh {pause} we may need to have more {pause} of uh things that are similar to a target language or {disfmarker} I mean. {pause} You have the same number of parameters in the neural net, you haven't increased the size of the neural net, and maybe there's just {disfmarker} {pause} just not enough {pause} complexity to it to represent {pause} the variab increased variability in the {disfmarker} in the training set. That {disfmarker} that could be. Um {pause} So, what about {disfmarker} So these are results with {pause} uh th {pause} that you're describing now, that {pause} they are pretty similar for the different features or {disfmarker} {pause} or uh {disfmarker}PhD D: Uh, let me check. Uh.Professor B: Yeah.PhD D: So. This was for the PLP,Professor B: Yeah.PhD D: Um. The {disfmarker} Yeah. For the PLP with JRASTA the {disfmarker} {pause} the {disfmarker} we {disfmarker} This is quite the same {pause} tendency, {pause} with a slight increase of the error rate, {pause} uh if we go to {disfmarker} to TIMIT. And then it's {disfmarker} it gets worse with the multilingual. Um. Yeah. There {disfmarker} there is a difference actually with {disfmarker} b between PLP and JRASTA is that {pause} JRASTA {pause} seems to {pause} perform better with the highly mismatched {pause} condition {pause} but slightly {disfmarker} slightly worse {pause} for the well matched condition. Mmm.Professor B: I have a suggestion, actually, even though it'll delay us slightly, would {disfmarker} would you mind {pause} running into the other room and making {pause} copies of this Cuz we're all sort of {disfmarker} If we c if we could look at it, while we're talking, I think it'd bePhD D: Yeah, yeah. OK.Professor B: uh {disfmarker} {pause} Uh, I'll {disfmarker} I'll sing a song or dance or something while you {vocalsound} do it, too.PhD A: So um {disfmarker}Grad F: Alright.PhD A: Go ahead. Ah, while you're gone I'll ask s some of my questions.Professor B: Yeah.PhD A: Um.Professor B: Yeah. Uh, this way and just slightly to the left, yeah.PhD A: The um {disfmarker} What was {disfmarker} Was this number {pause} forty or {disfmarker} It was roughly the same as this one, {pause} he said When you had the two language versus the three languageProfessor B: Um. That's what he was saying.PhD A: That's where he removed English,Grad F: Yeah.PhD A: rightProfessor B: Right.Grad F: It sometimes, actually, depends on what features you're using.Professor B: Yeah. But {disfmarker} but i it sounds like {disfmarker}Grad F: Um, but {disfmarker} {vocalsound} {vocalsound} He {disfmarker} Mm - hmm.Professor B: I mean. That's interesting because {pause} it {disfmarker} it seems like what it's saying is not so much that you got hurt {pause} uh because {pause} you {pause} uh didn't have so much representation of English, because in the other case you don't get hurt any more, at least when {pause} it seemed like uh it {disfmarker} it might simply be a case that you have something that is just much more diverse,PhD A: Mm - hmm.Professor B: but you have the same number of parameters representing it.PhD A: Mm - hmm. I wonder {disfmarker} were um all three of these nets {pause} using the same output This multi - language {pause} uh labellingGrad F: He was using uh sixty - four phonemes from {pause} SAMPA.PhD A: OK, OK.Grad F: Yeah.PhD A: So this would {disfmarker} {pause} From this you would say," well, it doesn't really matter if we put Finnish {pause} into {pause} the training of the neural net, {pause} if there's {pause} gonna be, {pause} you know, Finnish in the test data." RightProfessor B: Well, it's {disfmarker} it sounds {disfmarker} {pause} I mean, we have to be careful, cuz we haven't gotten a good result yet.PhD A: Yeah.Professor B: And comparing different bad results can be {pause} tricky.PhD A: Hmm.Professor B: But I {disfmarker} I {disfmarker} I {disfmarker} {pause} I think it does suggest that it's not so much uh {pause} uh cross {pause} language as cross type of speech.PhD A: Mm - hmm.Professor B: It's {disfmarker} it's um {disfmarker} {vocalsound} But we did {disfmarker} Oh yeah, the other thing I was asking him, though, is that I think that in the case {disfmarker} Yeah, you {disfmarker} you do have to be careful because of com compounded results. I think we got some earlier results {pause} in which you trained on one language and tested on another and you didn't have {pause} three, but you just had one {pause} language. So you trained on {pause} one type of digits and tested on another. Didn - Wasn't there something of that Where you, {pause} say, trained on Spanish and tested on {disfmarker} on TI - digits, or the other way around Something like thatPhD E: No.Professor B: I thought there was something like that, {pause} that he showed me {pause} last week. We'll have to wait till we get {disfmarker}PhD A: Yeah, that would be interesting.Professor B: Um, This may have been what I was asking before, Stephane, but {disfmarker} {pause} but, um, wasn't there something that you did, {pause} where you trained {pause} on one language and tested on another I mean no {disfmarker} no mixture but just {disfmarker}Grad F: I'll get it for you.PhD D: Uh, no, no.Professor B: We've never just trained on one langPhD D: Training on a single language, you mean, and testing on the other oneProfessor B: Yeah.PhD D: Uh, no.PhD E: Not yet.PhD D: So the only {pause} task that's similar to this is the training on two languages, and {comment} that {disfmarker}Professor B: But we've done a bunch of things where we just trained on one language. Right I mean, you haven't {disfmarker} you haven't done all your tests on multiple languages.PhD D: Uh, No. Either thi this is test with {pause} uh the same language {pause} but from the broad data, or it's test with {pause} uh different languages also from the broad data, excluding the {disfmarker} So, it's {disfmarker} it's three or {disfmarker} three and four.PhD E: The early experiment that {disfmarker}PhD A: Did you do different languages from digitsPhD D: Uh. No. You mean {pause} training digits {pause} on one language and using the net {pause} to recognize on the otherPhD A: Digits on another languagePhD D: No.Professor B: See, I thought you showed me something like that last week. You had a {disfmarker} you had a little {disfmarker}PhD D: Uh, {pause} No, I don't think so.Professor B: Um What {disfmarker}PhD C: These numbers are uh {pause} ratio to baselineProfessor B: So, I mean wha what's the {disfmarker}PhD D: So.Professor B: This {disfmarker} this chart {disfmarker} this table that we're looking at {pause} is um, show is all testing for TI - digits, or {disfmarker}Grad F: Bigger is worse.PhD D: So you have uh basically two {pause} uh parts.Grad F: This is error rate, I think.PhD C: Ratio.Grad F: No. {pause} No.PhD D: The upper part is for TI - digitsGrad F: Yeah, yeah, yeah.PhD D: and it's divided in three {pause} rows {pause} of four {disfmarker} four rows each.Grad F: Mm - hmm.Professor B: Yeah.PhD D: And the first four rows is well - matched, then the s the second group of four rows is mismatched, and {pause} finally highly mismatched. And then the lower part is for Italian and it's the same {disfmarker} {pause} the same thing.PhD A: So, so the upper part is training {pause} TI - digitsPhD D: So. It's {disfmarker} it's the HTK results, I mean. So it's {pause} HTK training testings {pause} with different kind of featuresPhD A: Ah.PhD D: and what appears in the {pause} uh left column is {pause} the networks that are used for doing this.Professor B: Hmm.PhD D: So. Uh Yeah.Professor B: Well, What was is that i What was it that you had {pause} done {pause} last week when you showed {disfmarker} Do you remember Wh - when you showed me {pause} the {disfmarker} your table last weekPhD D: It - It was part of these results. Mmm. Mmm.PhD A: So where is the baseline {pause} for the TI - digits {pause} located in herePhD D: You mean the HTK Aurora baselinePhD A: Yeah.PhD D: It's uh the one hundred number. It's, well, all these numbers are the ratio {pause} with respect to the baseline.PhD A: Ah! Ah, OK, OK.Professor B: So this is word {disfmarker} word error rate, so a high number is bad.PhD D: Yeah, this is {pause} a word error rate ratio.PhD E: Yeah.PhD A: OK, I see.PhD D: Yeah. So, seventy point two means that {pause} we reduced the error rate uh by thirty {disfmarker} thirty percent.PhD A: OK, OK, gotcha.PhD D: So.Professor B: OK, {vocalsound} so if we takePhD D: Hmm.Professor B: uh um let's see PLP {pause} uh with on - line {pause} normalization and {pause} delta - del so that's this thing you have circled here {pause} in the second column,PhD D: Yeah.Professor B: um {pause} and" multi - English" refers to whatPhD D: To TIMIT. Mmm. Then you have {pause} uh MF, {pause} MS and ME which are for French, Spanish and English. And, yeah. Actually I {disfmarker} {pause} I uh forgot to say that {pause} the multilingual net are trained {pause} on {pause} uh {pause} features without the s derivatives uh but with {pause} increased frame numbers. Mmm. And we can {disfmarker} we can see on the first line of the table that it {disfmarker} it {disfmarker} {pause} it's slightly {disfmarker} slightly worse when we don't use delta but it's not {disfmarker} {pause} not that much.Professor B: Right. So w w So, I'm sorry. I missed that. What's MF, MS and MEPhD A: Multi - French, Multi - SpanishPhD D: So. Multi - French, Multi - Spanish, and Multi - English.Professor B: Uh OK. So, it's {pause} uh {pause} broader vocabulary. Then {disfmarker} And {disfmarker}PhD D: Yeah.Professor B: OK so I think what I'm {disfmarker} what I saw in your smaller chart that I was thinking of was {disfmarker} was {pause} there were some numbers I saw, I think, that included these multiple languages and it {disfmarker} and I was seeing {pause} that it got worse. I {disfmarker} I think that was all it was. You had some very limited results that {disfmarker} at that pointPhD D: Yeah.Professor B: which showed {pause} having in these {disfmarker} these other languages. In fact it might have been just this last category, {pause} having two languages broad that were {disfmarker} where {disfmarker} where English was removed. So that was cross language and the {disfmarker} and the result was quite poor. What I {disfmarker} {pause} we hadn't seen yet was that if you added in the English, it's still poor.PhD D: Yeah.Professor B: Uh {vocalsound} {vocalsound} Um now, what's the noise condition {pause} um {pause} of the training data {disfmarker}PhD D: Still poor.Professor B: Well, I think this is what you were explaining. The noise condition is the same {disfmarker} It's the same uh Aurora noises uh, in all these cases {pause} for the training.PhD D: Yeah. Yeah.Professor B: So there's not a {pause} statistical {disfmarker} sta a strong st {pause} statistically different {pause} noise characteristic between {pause} uh the training and testPhD D: No these are the s s s same noises,Professor B: and yet we're seeing some kind of effect {disfmarker}PhD D: yeah. At least {disfmarker} at least for the first {disfmarker} {pause} for the well - matched,Grad F: Well matched condition.Professor B: Right.PhD D: yeah.Professor B: So there's some kind of a {disfmarker} a {disfmarker} an effect from having these {disfmarker} uh this broader coverage um Now I guess what we should try doing with this is try {pause} testing these on u this same sort of thing on {disfmarker} you probably must have this {pause} lined up to do. To try the same t {pause} with the exact same training, do testing on {pause} the other languages.PhD D: Mmm.Professor B: On {disfmarker} on um {disfmarker} So. Um, oh I well, wait a minute. You have this here, for the Italian. That's right. OK, so, {pause} So.PhD D: Yeah. Yeah, so for the Italian the results are {vocalsound} uh {pause} stranger um {pause} Mmm. So what appears is that perhaps Spanish is {pause} not very close to Italian because uh, well, {pause} when using the {disfmarker} the network trained only on Spanish it's {disfmarker} {pause} the error rate is {pause} almost uh twice {pause} the baseline error rate.Professor B: Mm - hmm.PhD D: Mmm. {vocalsound} Uh.Professor B: Well, I mean, let's see. Is there any difference in {disfmarker} So it's in {pause} the uh {disfmarker} So you're saying that {pause} when you train on English {pause} and {pause} uh {pause} and {disfmarker} and test on {disfmarker}PhD D: Yeah.Professor B: No, you don't have training on English testing {disfmarker}PhD D: There {disfmarker} there is {disfmarker} another difference, is that the noise {disfmarker} the noises are different.Professor B: In {disfmarker} in whatPhD D: Well, For {disfmarker} for the Italian part I mean the {pause} uh {pause} the um {pause} networks are trained with noise from {pause} Aurora {disfmarker} TI - digits,PhD E: Aurora - two.PhD D: mmm.Professor B: And the noise is different in thPhD D: Yeah. And perhaps the noise are {pause} quite different from the noises {pause} in the speech that Italian.Professor B: Do we have any um {pause} test sets {pause} uh in {pause} any other language that um have the same noise as in {pause} the AuroraPhD D: And {disfmarker}PhD E: Mmm, no.PhD D: No.PhD A: Can I ask something real quick In {disfmarker} in the upper part {disfmarker} {pause} in the English {pause} stuff, {pause} it looks like the very best number is sixty point nine and that's in the uh {disfmarker} {pause} the third {pause} section in the upper part under PLP JRASTA, sort of the middle columnPhD D: Yeah.PhD A: I is that {pause} a noisy conditionPhD D: Yeah.PhD A: So that's matched training Is that what that isPhD D: It's {disfmarker} no, the third part, so it's uh {pause} highly mismatched. So. Training and {pause} test noise are different.PhD A: So {disfmarker} why do you get your best number in {disfmarker} Wouldn't you get your best number in the clean casePhD C: Well, it's relative to the um {pause} baseline mismatchingPhD D: Yeah.PhD A: Ah,PhD D: Yeah. Yeah.PhD A: OK so these are not {disfmarker} OK, alright, I see.PhD C: Yeah.PhD A: OK. And then {disfmarker} so, in the {disfmarker} in the um {disfmarker} {pause} in the {pause} non - mismatched clean case, {pause} your best one was under MFCC That sixty - one point fourPhD D: Yeah. {pause} But it's not a clean case. It's {pause} a noisy case but {pause} uh training and test noises are the same.PhD A: Oh! So this upper thirdPhD D: So {disfmarker} Yeah.PhD A: Uh that's still noisyPhD D: Yeah.PhD A: Ah, OK.PhD D: So it's always noisy basically,PhD A: Mm - hmm.PhD D: and, {pause} well, the {disfmarker}PhD A: I see.PhD D: Mmm.Professor B: OK Um {pause} So uh, I think this will take some {pause} looking at, thinking about. But, {pause} what is uh {disfmarker} what is currently running, that's {disfmarker} uh, i that {disfmarker} just filling in the holes here or {disfmarker} or {disfmarker} {comment} {pause} pretty muchPhD D: Uh, no we don't plan to fill the holesProfessor B: OK.PhD D: but {pause} actually there is something important, is that {pause} um we made a lot of assumption concerning the on - line normalization and we just noticed {pause} uh recently that {pause} uh the {pause} approach that we were using {pause} was not {pause} uh {pause} leading to very good results {pause} when we {pause} used the straight features to HTK. Um {pause} {pause} Mmm. So basically d {pause} if you look at the {disfmarker} at the left of the table, {pause} the first uh row, {pause} with eighty - six, one hundred, and forty - three and seventy - five, these are the results we obtained for Italian {pause} uh with {pause} straight {pause} mmm, PLP features {pause} using on - line normalization.Professor B: Mm - hmm.PhD D: Mmm. And the, mmm {disfmarker} what's {pause} in the table, just {pause} at the left of the PLP twelve {pause} on - line normalization column, so, the numbers seventy - nine, fifty - four and {pause} uh forty - two {pause} are the results obtained by uh Pratibha with {pause} uh his on - line normalization {disfmarker} uh her on - line normalization approach.PhD A: Where is that seventy - nine, fiftyProfessor B: Uh, it's just sort of sitting right on the uh {disfmarker} the column line.PhD D: So.PhD E: Fifty - one This {disfmarker}PhD A: Oh I see, OK.Professor B: Uh. {pause} Yeah.PhD D: Just {disfmarker} uh Yeah. So these are the results of {pause} OGI with {pause} on - line normalization and straight features to HTK. And the previous result, eighty - six and so on, {pause} are with our {pause} features straight to HTK.Professor B: Yes. Yes.PhD D: So {pause} what we see that {disfmarker} is {disfmarker} there is that um {pause} uh the way we were doing this was not correct, but {pause} still {pause} the networks {pause} are very good. When we use the networks {pause} our number are better that {pause} uh Pratibha results.PhD E: We improve.Professor B: So, do you know what was wrong with the on - line normalization, or {disfmarker}PhD D: Yeah. There were diff there were different things and {pause} basically, {pause} the first thing is the mmm, {pause} alpha uh {pause} value. So, the recursion {pause} uh {pause} part. um, {pause} I used point five percent, {pause} which was the default value in the {disfmarker} {pause} in the programs here. And Pratibha used five percent.Professor B: UhPhD D: So it adapts more {pause} quicklyProfessor B: Yes. Yeah.PhD D: Um, but, yeah. I assume that this was not important because {pause} uh previous results from {disfmarker} from Dan and {disfmarker} show that basically {pause} the {pause} both {disfmarker} both values g give the same {disfmarker} same {pause} uh results. It was true on uh {pause} TI - digits but it's not true on Italian.Professor B: Mm - hmm.PhD D: Uh, second thing is the initialization of the {pause} stuff. Actually, {pause} uh what we were doing is to start the recursion from the beginning of the {pause} utterance. And using initial values that are the global mean and variances {pause} measured across the whole database.Professor B: Right. Right.PhD D: And Pratibha did something different is that he {disfmarker} uh she initialed the um values of the mean and variance {pause} by computing {pause} this on the {pause} twenty - five first frames of each utterance. Mmm. There were other minor differences, the fact that {pause} she used fifteen dissities instead s instead of thirteen, and that she used C - zero instead of log energy. Uh, but the main differences concerns the recursion. So. {pause} Uh, I changed the code uh and now we have a baseline that's similar to the OGI baseline.Professor B: OK.PhD D: We {disfmarker} It {disfmarker} it's slightly {pause} uh different because {pause} I don't exactly initialize the same way she does. Actually I start, {pause} mmm, I don't wait to a fifteen {disfmarker} twenty - five {disfmarker} twenty - five frames {pause} before computing a mean and the variance {pause} to e to {disfmarker} to start the recursion.PhD C: Mm - hmm.Professor B: Yeah.PhD D: I {disfmarker} I use the on - line scheme and only start the re recursion after the twenty - five {disfmarker} {pause} twenty - fifth frame. But, well it's similar. So {pause} uh I retrained {pause} the networks with {pause} these {disfmarker} well, the {disfmarker} the {disfmarker} the networks are retaining with these new {pause} features.Professor B: Mm - hmm.PhD D: And, yeah.Professor B: OK.PhD D: So basically what I expect is that {pause} these numbers will a little bit go down but {pause} perhaps not {disfmarker} not so muchProfessor B: Right.PhD D: because {pause} I think the neural networks learn perhaps {pause} to {disfmarker}Professor B: Right.PhD D: even if the features are not {pause} normalized. It {disfmarker} it will learn how to normalize and {disfmarker}Professor B: OK, but I think that {pause} given the pressure of time we probably want to draw {disfmarker} because of that {pause} especially, we wanna draw some conclusions from this, do some reductions {pause} in what we're looking at,PhD D: Yeah.Professor B: and make some strong decisions for what we're gonna do testing on before next week. So do you {disfmarker} are you {disfmarker} w did you have something going on, on the side, with uh multi - band {pause} or {disfmarker} on {disfmarker} on this,PhD D: Yeah {vocalsound} IProfessor B: or {disfmarker}PhD D: No, I {disfmarker} we plan to start this uh so, act actually we have discussed uh {pause} @ @ um, these {disfmarker} what we could do {pause} more as a {disfmarker} as a research and {disfmarker} {pause} and {pause} we were thinking perhaps that {pause} uh {pause} the way we use the tandem is not {disfmarker} Uh, well, there is basically perhaps a flaw in the {disfmarker} in the {disfmarker} the stuff because {pause} we {pause} trained the networks {disfmarker} If we trained the networks on the {disfmarker} on {pause} a language and a t or a specific {pause} task,Professor B: Mm - hmm.PhD D: um, what we ask is {disfmarker} to the network {disfmarker} is to put the bound the decision boundaries somewhere in the space.Professor B: Mmm.PhD D: And uh {pause} mmm and ask the network to put one, {pause} at one side of the {disfmarker} for {disfmarker} for a particular phoneme at one side of the boundary {disfmarker} decision boundary and one for another phoneme at the other side. And {pause} so there is kind of reduction of the information there that's not correct because if we change task {pause} and if the phonemes are not in the same context in the new task, {pause} obviously the {pause} decision boundaries are not {disfmarker} {pause} should not be at the same {pause} place.Professor B: I diPhD D: But the way the feature gives {disfmarker} The {disfmarker} the way the network gives the features is that it reduce completely the {disfmarker} {pause} it removes completely the information {disfmarker} {pause} a lot of information from the {disfmarker} the features {pause} by uh {pause} uh {pause} placing the decision boundaries at {pause} optimal places for {pause} one kind of {pause} data but {pause} this is not the case for another kind of data.Professor B: It's a trade - off,PhD D: So {disfmarker}Professor B: right Any - anyway go ahead.PhD D: Yeah. So uh what we were thinking about is perhaps {pause} um one way {pause} to solve this problem is increase the number of {pause} outputs of the neural networks. Doing something like, um {pause} um phonemes within context and, well, basically context dependent phonemes.Professor B: Maybe. I mean, I {disfmarker} I think {pause} you could make {pause} the same argument, it'd be just as legitimate, {pause} for hybrid systems {pause} as well. Right.PhD D: Yeah but, we know that {disfmarker}Professor B: And in fact, {pause} th things get better with context dependent {pause} versions. RightPhD D: Ye - yeah but here it's something different. We want to have featuresProfessor B: Yeah.PhD D: uh well, {pause} um.Professor B: Yeah, but it's still true {pause} that what you're doing {pause} is you're ignoring {disfmarker} you're {disfmarker} you're coming up with something to represent, {pause} whether it's a distribution, {pause} probability distribution or features, you're coming up with a set of variables {pause} that are representing {pause} uh, {pause} things that vary w over context.PhD D: Mm - hmm.Professor B: Uh, and you're {pause} putting it all together, ignoring the differences in context. That {disfmarker} that's true {pause} for the hybrid system, it's true for a tandem system. So, for that reason, when you {disfmarker} in {disfmarker} in {disfmarker} in a hybrid system, {pause} when you incorporate context one way or another, {pause} you do get better scores.PhD D: Yeah.Professor B: OK But I {disfmarker} it's {disfmarker} it's a big deal {pause} to get that. I {disfmarker} I'm {disfmarker} I'm sort of {disfmarker} And once you {disfmarker} the other thing is that once you represent {disfmarker} start representing more and more context {pause} it is {pause} uh {pause} much more {pause} um specific {pause} to a particular task in language. So um Uh, the {disfmarker} {pause} the acoustics associated with {pause} uh a particular context, for instance you may have some kinds of contexts that will never occur {pause} in one language and will occur frequently in the other, so the qu the issue of getting enough training {pause} for a particular kind of context becomes harder. We already actually don't have a huge amount of training data umPhD D: Yeah, but {disfmarker} mmm, I mean, {pause} the {disfmarker} the way we {disfmarker} we do it now is that we have a neural network and {pause} basically {pause} the net network is trained almost to give binary decisions.Professor B: Right.PhD D: And {pause} uh {disfmarker} binary decisions about phonemes. Nnn {disfmarker} Uh It's {disfmarker}Professor B: Almost. But I mean it {disfmarker} it {disfmarker} it does give a distribution.PhD D: Yeah.Professor B: It's {disfmarker} and {disfmarker} and {pause} it is true that if there's two phones that are very similar, {pause} that {pause} uh {pause} the {disfmarker} {pause} i it may prefer one but it will {pause} give a reasonably high value to the other, too.PhD D: Yeah. Yeah, sure but uh {pause} So basically it's almost binary decisions and {pause} um the idea of using more {pause} classes is {pause} to {pause} get something that's {pause} less binary decisions.Professor B: Oh no, but it would still be even more of a binary decision. It {disfmarker} it'd be even more of one. Because then you would say {pause} that in {disfmarker} that this phone in this context is a one, {pause} but the same phone in a slightly different context is a zero.PhD D: But {disfmarker} yeah, but {disfmarker}Professor B: That would be even {disfmarker} even more distinct of a binary decision. I actually would have thought you'd wanna go the other way and have fewer classes.PhD D: Yeah, but if {disfmarker}Professor B: Uh, I mean for instance, the {disfmarker} the thing I was arguing for before, but again which I don't think we have time to try, {pause} is something in which you would modify the code so you could train to have several outputs on and use articulatory featuresPhD D: Mmm. Mm - hmm.Professor B: cuz then that would {disfmarker} that would go {disfmarker} {pause} that would be much broader and cover many different situations. But if you go to very very fine categories, it's very {pause} binary.PhD D: Mmm. Yeah, but I think {disfmarker} Yeah, perhaps you're right, but you have more classes so {pause} you {disfmarker} you have more information in your features. So, {vocalsound} Um {pause} You have more information in the {pause} uhProfessor B: Mm - hmm. True.PhD D: posteriors vector um which means that {disfmarker} But still the information is relevantProfessor B: Mm - hmm.PhD D: because it's {disfmarker} it's information that helps to discriminate,Professor B: Mm - hmm.PhD D: if it's possible to be able to discriminate {pause} among the phonemes in context.Professor B: Well it's {disfmarker} it's {disfmarker} {pause} it's an interesting thought.PhD D: But the {disfmarker}Professor B: I mean we {disfmarker} we could disagree about it at lengthPhD D: Mmm.Professor B: but the {disfmarker} the real thing is if you're interested in it you'll probably try itPhD D: Mmm.Professor B: and {disfmarker} {pause} and {pause} we'll see. But {disfmarker} but what I'm more concerned with now, as an operational level, is {pause} uh, you know,PhD D: Mmm.Professor B: what do we do in four or five days Uh, and {disfmarker} {pause} so we have {pause} to be concerned {pause} with Are we gonna look at any combinations of things, you know once the nets get retrained so you have this problem out of it.PhD D: Mmm.Professor B: Um, are we going to look at {pause} multi - band Are we gonna look at combinations of things Uh, what questions are we gonna ask, uh now that, I mean, {pause} we should probably turn shortly to this O G I note. Um, how are we going to {pause} combine {pause} with what they've been focusing on Uh, {pause} Uh we haven't been doing any of the L D A RASTA sort of thing.PhD D: Mm - hmm.Professor B: And they, although they don't talk about it in this note, um, {pause} there's um, {pause} the issue of the {pause} um Mu law {pause} business {pause} uh {pause} versus the logarithm, um, {pause} so.PhD D: Mm - hmm.Professor B: So what i what is going on right now What's right {disfmarker} you've got {pause} nets retraining, Are there {disfmarker} is there {disfmarker} are there any H T K {pause} trainings {disfmarker} testings going onPhD D: NPhD E: I {disfmarker} I {disfmarker} I'm trying the HTK with eh, {pause} PLP twelve on - line delta - delta and MSG filter {pause} together.Professor B: The combination, I see.PhD E: The combination, yeah. But I haven't result {vocalsound} at this moment.Professor B: MSG and {disfmarker} and PLP.PhD E: Yeah.Professor B: And is this with the revised {pause} on - line normalizationPhD E: Ye - Uh, with the old {pause} older,PhD D: Yeah.Professor B: Old one. So it's using all the nets for thatPhD E: yeah.Professor B: but again we have the hope that it {disfmarker} {pause} We have the hope that it {disfmarker} {pause} maybe it's not making too much difference,PhD E: Yeah. But {pause} We can know soon.Professor B: but {disfmarker} butPhD E: Maybe.Professor B: yeah.PhD E: I don't know.PhD D: Yeah.Professor B: Uh, OK.PhD D: Uh so there is this combination, yeah. Working on combination obviously.PhD E: Mm - hmm.PhD D: Um, I will start work on multi - band. And {pause} we {pause} plan to work also on the idea of using both {pause} features {pause} and net outputs.PhD E:PhD D: Um. And {pause} we think that {pause} with this approach perhaps {pause} we could reduce the number of outputs of the neural network. Um, So, get simpler networks, because we still have the features. So we have um {pause} come up with um {pause} different kind of {pause} broad phonetic categories. And we have {disfmarker} Basically we have three {pause} types of broad phonetic classes. Well, something using place of articulation which {disfmarker} which leads to {pause} nine, I think, {pause} broad classes. Uh, another which is based on manner, which is {disfmarker} is also something like nine classes. And then, {pause} something that combine both, and we have {pause} twenty f {pause} twenty - fiveGrad F: Twenty - seven.PhD D: Twenty - seven broad classes. So like, uh, oh, I don't know, like back vowels, front vowels.Professor B: So what you do {disfmarker} um I just wanna understandPhD D: Um For the moments we do not {disfmarker} don't have nets,Professor B: so {pause} You have two net or three nets Was this How many {disfmarker} how many nets do you have No nets.PhD D: I mean, {pause} It's just {disfmarker} Were we just changing {pause} the labels to retrain nets {pause} with fewer out outputs.PhD E: Begin to work in this. We are @ @.Professor B: Right. But {disfmarker} but I didn't understand {disfmarker}PhD D: And then {disfmarker} Mm - hmm.Professor B: Uh. {pause} the software currently just has {disfmarker} uh a {disfmarker} allows for I think, the one {disfmarker} one hot output. So you're having multiple nets and combining them, or {disfmarker} Uh, how are you {disfmarker} how are you coming up with {disfmarker} If you say {pause} uh {pause} If you have a place {pause} characteristic and a manner characteristic, how do you {disfmarker}PhD D: It - It's the single net,PhD A: I think they have one output.PhD D: yeah.Professor B: Oh, it's just one net.PhD D: It's one net with {pause} um {pause} twenty - seven outputsPhD E: Yeah.Grad F: mm - hmmPhD D: if we have twenty - seven classes,Professor B: I see. I see, OK.PhD D: yeah. So it's {disfmarker} Well, it's basically a standard net with fewer {pause} classes.Professor B: So you're sort of going the other way of what you were saying a bit ago instead of {disfmarker} yeah.PhD D: Yeah, but I think {disfmarker} Yeah. B b including the features, yeah.Grad F: But including the features.PhD E: Yeah.PhD D: I don't think this {pause} will work {pause} alone. I think it will get worse because Well, I believe the effect that {disfmarker} of {disfmarker} of too reducing too much the information is {pause} basically {disfmarker} basically what happensProfessor B: Uh - huh.PhD D: and {disfmarker}Professor B: But you think if you include that {pause} plus the other features,PhD D: but {disfmarker} Yeah, because {pause} there is perhaps one important thing that the net {pause} brings, and OGI show showed that, is {pause} the distinction between {pause} sp speech and silence Because these nets are trained on well - controlled condition. I mean the labels are obtained on clean speech, and we add noise after. So this is one thing And But perhaps, something intermediary using also {pause} some broad classes could {disfmarker} could bring so much more information. Uh.Professor B: So {disfmarker} so again then we have these broad classes and {disfmarker} well, somewhat broad. I mean, it's twenty - seven instead of sixty - four, {pause} basically. And you have the original features.PhD D: Yeah.Professor B: Which are PLP, or something.PhD D: Yeah.Professor B: And then uh, just to remind me, all of that goes {pause} into {disfmarker} uh, that all of that is transformed by uh, uh, K - KL or something, or {disfmarker}PhD D: Mm - hmm. There will probably be,PhD E: Mu.PhD D: yeah, one single KL to transform everythingProfessor B: Right.PhD D: or {vocalsound} {pause} uh,PhD E: No transform the PLPPhD D: perPhD E: and only transform the other I'm not sure.Professor B: Well no,PhD D: This is {pause} still something {pause} thatProfessor B: I think {disfmarker} I see.PhD D: yeah, we {pause} don't know {disfmarker}Professor B: So there's a question of whether you would {disfmarker}PhD E: Two e @ @ it's one.PhD D: Yeah.Professor B: Right. Whether you would transform together or just one. Yeah. Might wanna try it both ways. But that's interesting. So that's something that you're {disfmarker} you haven't trained yet but are preparing to train, and {disfmarker}PhD D: Yeah.Professor B: Yeah. Um {pause} {pause} Yeah, so I think Hynek will be here Monday.PhD D: Mmm.Professor B: Monday or Tuesday. SoPhD D: Uh, yeah.Professor B: So I think, you know, we need to {pause} choose the {disfmarker} choose the experiments carefully, so we can get uh key {disfmarker} {pause} key questions answered {pause} uh before thenPhD D: Mm - hmm.Professor B: and {pause} leave other ones aside even if it {pause} leaves incomplete {pause} tables {vocalsound} {pause} someplace, uh {pause} uh, it's {disfmarker} it's really time to {disfmarker} {pause} time to choose.PhD D: Mm - hmm.Professor B: Um, let me pass this out, {pause} by the way. Um These are {disfmarker} Did {disfmarker} did {disfmarker} {pause} did I interrupt youPhD E: Yeah, I have one.Professor B: Were there other things that you wanted to {disfmarker}PhD D: Uh, no. I don't think so.PhD E:PhD D: Yeah, I have one.Grad G: Oh, thanks.Professor B: Ah! {pause} OK. {pause} OK, we have {pause} lots of them.PhD E: We have one.Professor B: OK, so {vocalsound} um, Something I asked {disfmarker} So they're {disfmarker} they're doing {pause} the {disfmarker} the VAD I guess they mean voice activity detection So again, it's the silence {disfmarker} So they've just trained up a net {pause} which has two outputs, I believe. Um {vocalsound} I asked uh {pause} Hynek whether {disfmarker} I haven't talked to Sunil {disfmarker} I asked Hynek whether {pause} they compared that to {pause} just taking the nets we already had {pause} and summing up the probabilities.PhD D: Mm - hmm.Professor B: Uh. {pause} To get the speech {disfmarker} voice activity detection, or else just using the silence, {pause} if there's only one {pause} silence output. Um {pause} And, he didn't think they had, um. But on the other hand, maybe they can get by with a smaller net and {pause} maybe {pause} sometimes you don't run the other, maybe there's a computational advantage to having a separate net, anyway.PhD D: Mm - hmm.Professor B: So um Their uh {disfmarker} {pause} the results look pretty good. Um, {pause} I mean, not uniformly.PhD D: Yeah.Professor B: I mean, there's a {disfmarker} an example or two {pause} that you can find, where it made it slightly worse, but {pause} uh in {disfmarker} in all but a couple {pause} examples.PhD D: Mmm.Professor B: Uh.PhD E: But they have a question of the result. Um how are trained the {disfmarker} the LDA filter How obtained the LDA filterPhD D: Mmm.Professor B: I I'm sorry. I don't understand your question.PhD E: Yes, um the LDA filter {pause} needs some {pause} training set {pause} to obtain the filter. Maybe I don't know exactly how {pause} they are obtained.Professor B: It's on {pause} training.PhD E: Training, with the training test of each {disfmarker} You understand meProfessor B: No.PhD E: Yeah, uh for example, {pause} LDA filter {pause} need a set of {disfmarker} {pause} a set of training {pause} to obtain the filter.Professor B: Yes.PhD E: And maybe {pause} for the Italian, for the TD {pause} TE on for Finnish, these filter are {disfmarker} are obtained with their own training set.Professor B: Yes, I don't know. That's {disfmarker} that's {disfmarker} so that's a {disfmarker} that's a very good question, then {disfmarker} now that it {disfmarker} {pause} I understand it. It's" yeah, where does the LDA come from" In the {disfmarker} In {pause} earlier experiments, they had taken LDA {pause} from a completely different database, rightPhD E: Yeah. Yeah, because maybe it the same situation that the neural network training with their ownPhD D: Mmm.PhD E: set.Professor B: So that's a good question. Where does it come from Yeah, I don't know. Um, {pause} but uh to tell you the {pause} truth, I wasn't actually looking at the LDA so much when I {disfmarker} I was looking at it I was {pause} mostly thinking about the {disfmarker} {pause} the VAD. And um, it ap {pause} it ap Oh what does {disfmarker} what does ASP Oh that's {disfmarker}PhD D: The features, yeah. Yeah.PhD E: I don't understand alsoProfessor B: It says" baseline ASP" .PhD E: what is {disfmarker} {pause} what is the difference between ASP and uh baseline overPhD C: ASP.PhD D: Yeah, I don't know.PhD E: This is {disfmarker}Professor B: Anybody know {pause} any {disfmarker}PhD C: Oh. There it is.Professor B: Um Cuz there's" baseline Aurora" {pause} above it.PhD C: Mm - hmm.Professor B: And it's {disfmarker} This is mostly better than baseline, although in some cases it's a little worse, in a couple cases.PhD C: Well, it says baseline ASP is twenty - three mill {pause} minus thirteen.PhD E: Yeah.Professor B: Yeah, it says what it is. But I don't how that's different {pause} from {disfmarker}PhD C: From the baseline. {comment} OK.Professor B: I think this was {disfmarker} {pause} I think this is the same point we were at when {disfmarker} when we were up in Oregon.PhD E: Yeah.PhD D: I think {disfmarker} {pause} I think it's the C - zero {disfmarker} using C - zero instead of log energy.PhD E: Ah, OK, mm - hmm.PhD D: Yeah, it's this.Professor B: Oh. OK.PhD E: yeah.PhD D: It should be that, yeah.PhD A: They s they say in here that the VAD is not used as an additional feature.Professor B: Shouldn't it be {disfmarker}PhD D: Because {disfmarker}PhD A: Does {disfmarker} does anybody know how they're using itProfessor B: Yeah. So {disfmarker} so what they're doing here is, {pause} iPhD D: Yeah.Professor B: if you look down at the block diagram, {pause} um, {pause} they estimate {disfmarker} they get a {disfmarker} {pause} they get an estimate {pause} of whether it's speech or silence,PhD A: But that {disfmarker}Professor B: and then they have a median filter of it.PhD A: Mm - hmm.Professor B: And so um, {pause} basically they're trying to find stretches. The median filter is enforcing a {disfmarker} i it having some continuity.PhD A: Mm - hmm.Professor B: You find stretches where the {pause} combination of the {pause} frame wise VAD and the {disfmarker} {pause} the median filter say that there's a stretch of silence. And then it's going through and just throwing the data away.PhD C: Hmm.Professor B: Right So um {disfmarker}PhD A: So it's {disfmarker} it's {disfmarker} I don't understand. You mean it's throwing out frames Before {disfmarker}Professor B: It's throwing out chunks of frames, yeah. There's {disfmarker} the {disfmarker} the median filter is enforcing that it's not gonna be single cases of frames, or isolated frames.PhD A: Yeah.Professor B: So it's throwing out frames and the thing is {pause} um, {pause} what I don't understand is how they're doing this with H TPhD A: Yeah, that's what I was just gonna ask.Professor B: This is {disfmarker}PhD A: How can you just throw out framesProfessor B: Yeah. Well, you {disfmarker} you can,PhD D: iProfessor B: right I mean y you {disfmarker} you {disfmarker}PhD D: Yeah.Professor B: it stretches again. For single frames I think it would be pretty hard.PhD A: Yeah.Professor B: But if you say speech starts here, speech ends there.PhD A: Mm - hmm.Professor B: RightPhD C: Huh.PhD D: Yeah. Yeah, you can basically remove the {disfmarker} the frames from the feature {disfmarker} feature files.Professor B: Yeah. Yeah, so I mean in the {disfmarker} i i in the {disfmarker} in the decoding, you're saying that we're gonna decode from here to here.PhD D: I tPhD A: Mm - hmm.Professor B: I think they're {disfmarker} they're {disfmarker} they're treating it, {pause} you know, like uh {disfmarker} well, it's not isolated word, but {disfmarker} but connected, you know, the {disfmarker} the {disfmarker}PhD A: In the text they say that this {disfmarker} this is a tentative block diagram of a possible configuration we could think of. So that sort of sounds like they're not doing that yet.Professor B: Well. {pause} No they {disfmarker} they have numbers though, right So I think they're {disfmarker} they're doing something like that. I think that they're {disfmarker} they're {disfmarker} I think what I mean by tha that is they're trying to come up with a block diagram that's plausible for the standard. In other words, it's {disfmarker} uh {disfmarker} I mean from the point of view of {disfmarker} of uh reducing the number of bits you have to transmit it's not a bad idea to detect silence anyway.PhD A: Yeah. Yeah. I'm just wondering what exactly did they do up in this table if it wasn't this.Professor B: Um. But it's {disfmarker} the thing is it's that {disfmarker} that {disfmarker} that's {disfmarker} that's I {disfmarker} I {disfmarker} Certainly it would be tricky about it intrans in transmitting voice, {pause} uh uh for listening to, is that these kinds of things {pause} uh cut {pause} speech off a lot.PhD A: Mm - hmm.Professor B: Right And so {pause} umPhD A: Plus it's gonna introduce delays.Professor B: It does introduce delays but they're claiming that it's {disfmarker} it's within the {disfmarker} {pause} the boundaries of it.PhD A: Mmm.Professor B: And the LDA introduces delays, and b {pause} what he's suggesting this here is a parallel path so that it doesn't introduce {pause} uh, any more delay. I it introduces two hundred milliseconds of delay but at the same {pause} time the LDA {pause} down here {disfmarker} I don't know {disfmarker} Wh what's the difference between TLDA and SLDAPhD C: Temporal and spectral.Professor B: Ah, thank you.PhD E: Temporal LDA.Professor B: Yeah, you would know that.PhD C: YeahProfessor B: So um. The temporal LDA does in fact include the same {disfmarker} so that {disfmarker} I think he {disfmarker} well, by {disfmarker} by saying this is a b a tentative block di diagram I think means {pause} if you construct it this way, this {disfmarker} this delay would work in that wayPhD A: Ah.Professor B: and then it'd be OK. They {disfmarker} they clearly did actually remove {pause} silent sections in order {disfmarker} because they {pause} got these {pause} word error rate {pause} results. So um I think that it's {disfmarker} it's nice to do that in this because in fact, it's gonna give a better word error result and therefore will help within an evaluation. Whereas to whether this would actually be in a final standard, I don't know. Um. Uh, as you know, part of the problem with evaluation right now is that the {pause} word models are pretty bad and nobody wants {disfmarker} {pause} has {disfmarker} has approached improving them. So {pause} it's possible that a lot of the problems {pause} with so many insertions and so forth would go away if they were better word models {pause} to begin with. So {pause} this might just be a temporary thing. But {disfmarker} But, on the other hand, and maybe {disfmarker} maybe it's a decent idea. So um The question we're gonna wanna go {pause} through next week when Hynek shows up I guess is given that we've been {disfmarker} if you look at what we've been trying, we're uh looking at {pause} uh, by then I guess, combinations of features and multi - band Uh, and we've been looking at {pause} cross - language, cross {pause} task {pause} issues. And they've been not so much looking at {pause} the cross task uh multiple language issues. But they've been looking at uh {disfmarker} {pause} at these issues. At the on - line normalization and the uh {pause} voice activity detection. And I guess when he comes here we're gonna have to start deciding about {pause} um what do we choose {pause} from what we've looked at {pause} to um blend with {pause} some group of things in what they've looked at And once we choose that, {pause} how do we split up the {pause} effort Uh, because we still have {disfmarker} even once we choose, {pause} we've still got {pause} uh another {pause} month or so, I mean there's holidays in the way, but {disfmarker} but uh {pause} I think the evaluation data comes January thirty - first so there's still a fair amount of time {pause} to do things together it's just that they probably should be somewhat more coherent between the two sites {pause} in that {disfmarker} that amount of time.PhD A: When they removed the silence frames, did they insert some kind of a marker so that the recognizer knows it's {disfmarker} {pause} knows when it's time to back trace or somethingProfessor B: Well, see they, I {disfmarker} I think they're Um. I don't know the {disfmarker} {pause} the specifics of how they're doing it. They're {disfmarker} {pause} they're getting around the way the recognizer works because they're not allowed to {pause} um, change the scripts {pause} for the recognizer, {pause} I believe.PhD A: Oh, right. Maybe they're just inserting some nummy frames or somethingProfessor B: So. Uh. Uh, you know that's what I had thought. But I don't {disfmarker} I don't think they are.PhD A: Hmm.Professor B: I mean that's {disfmarker} sort of what {disfmarker} the way I had imagined would happen is that on the other side, yeah you p put some low level noise or something. Probably don't want all zeros.PhD A: Hmm.Professor B: Most recognizers don't like zeros but {vocalsound} but {pause} you know, {pause} put some epsilon in or some randPhD A: Yeah.Professor B: sorry epsilon random variable {pause} in or something.PhD A: Some constant vector. I mean i w Or something {disfmarker}Professor B: Maybe not a constant but it doesn't, uh {disfmarker} don't like to divide by the variance of that, but I mean it'sPhD A: That's right. But something that {disfmarker} what I mean is something that is {pause} very distinguishable from {pause} speech.Professor B: Mm - hmm.PhD A: So that the {disfmarker} the silence model in HTK will always pick it up.Professor B: Yeah. So I {disfmarker} I {disfmarker} that's what I thought they would do. or else, uh {pause} uh maybe there is some indicator to tell it to start and stop, I don't know.PhD A: Hmm.Professor B: But whatever they did, I mean they have to play within the rules of this specific evaluation.PhD A: Yeah.Professor B: We c we can find out.PhD A: Cuz you gotta do something. Otherwise, if it's just a bunch of speech, stuck together {disfmarker}Professor B: No they're {disfmarker}PhD A: Yeah.Professor B: It would do badlyPhD A: Yeah, right.Professor B: and it didn't so badly, right So they did something.PhD A: Yeah, yeah.Professor B: Yeah. Uh. So, OK, So I think {pause} this brings me up to date a bit. It hopefully brings other {pause} people up to date a bit. And um Um {pause} I think {disfmarker} Uh, I wanna look at these numbers off - line a little bit and think about it and {disfmarker} {pause} and talk with everybody uh, {pause} outside of this meeting. Um, but uh No I mean it sounds like {disfmarker} I mean {pause} there {disfmarker} there {disfmarker} there are the usual number of {disfmarker} of {pause} little {disfmarker} little problems and bugs and so forth but it sounds like they're getting ironed out. And now we're {pause} seem to be kind of in a position to actually {pause} uh, {pause} look at stuff and {disfmarker} and {disfmarker} and compare things. So I think that's {disfmarker} that's pretty good. Um {pause} I don't know what the {disfmarker} One of the things I wonder about, {pause} coming back to the first results you talked about, is {disfmarker} is {pause} how much, {pause} uh {pause} things could be helped {pause} by more parameters. And uh {disfmarker} {pause} And uh how many more parameters we can afford to have, {vocalsound} {pause} in terms of the uh computational limits. Because anyway when we go to {pause} twice as much data {pause} and have the same number of parameters, particularly when it's twice as much data and it's quite diverse, um, I wonder if having twice as many parameters would help.PhD D: Mm - hmm.Professor B: Uh, just have a bigger hidden layer. Uh But {disfmarker} I doubt it would {pause} help by forty per cent. But {vocalsound} {pause} but uhPhD D: Yeah.Professor B: Just curious. How are we doing on the {pause} resources Disk, and {disfmarker}PhD D: I think we're alright,Professor B: OK.PhD D: um, {pause} not much problems with that.Professor B: ComputationPhD D: It's OK.Professor B: We {disfmarker}PhD D: Well this table took uh {pause} more than five days to get back.Professor B: Yeah. Yeah, well.PhD D: But {disfmarker} Yeah.Professor B: Are {disfmarker} were you folks using Gin That's a {disfmarker} that just died, you knowPhD D: Mmm, no. You were using Gin {comment} perhaps, yeah No.PhD E: No.Professor B: No Oh, that's good.Grad F: It just died.Professor B: OK. Yeah, {pause} we're gonna get a replacement {pause} server that'll be a faster server, {pause} actually.PhD E: Yes.Professor B: That'll be {disfmarker} It's a {pause} seven hundred fifty megahertz uh SUNPhD D: Hmm. {comment} Mm - hmm.Professor B: uh {pause} But it won't be installed for {pause} a little while.PhD C: Tonic.Professor B: U Go ahead.Grad G: Do we {disfmarker} Do we have that big new IBM machine the, I think in thProfessor B: We have the {pause} little tiny IBM machine {vocalsound} {pause} that might someday grow up to be a big {pause} IBM machine. It's got s slots for eight, uh IBM was donating five, I think we only got two so far, processors. We had originally hoped we were getting eight hundred megahertz processors. They ended up being five fifty. So instead of having eight processors that were eight hundred megahertz, we ended up with two {pause} that are five hundred and fifty megahertz. And more are supposed to come soon and there's only a moderate amount of dat of memory. So I don't think {pause} anybody has been sufficiently excited by it to {pause} spend much time {pause} uh {pause} with it, but uh {vocalsound} Hopefully, {pause} they'll get us some more {pause} parts, soon and {disfmarker} Uh, yeah, I think that'll be {disfmarker} once we get it populated, {pause} that'll be a nice machine. I mean we will ultimately get eight processors in there. And uh {disfmarker} and uh a nice amount of memory. Uh so it'll be a pr pretty fast Linux machine.Grad G: And if we can do things on Linux, {pause} some of the machines we have going already, like SwedeProfessor B: Mm - hmm.Grad G: Um It seems pretty fast.Professor B: Mm - hmm.Grad G: But {disfmarker} I think Fudge is pretty fast too.Professor B: Yeah, I mean you can check with uh {pause} Dave Johnson. I mean, it {disfmarker} it's {disfmarker} {pause} I think the machine is just sitting there. And it does have two processors, you know and {disfmarker} {pause} Somebody could do {disfmarker} {pause} you know, uh, check out {pause} uh the multi - threading {pause} libraries. And {pause} I mean i it's possible that the {disfmarker} I mean, I guess the prudent thing to do would be for somebody to do the work on {disfmarker} {pause} on getting our code running {pause} on that machine with two processors {pause} even though there aren't five or eight. There's {disfmarker} there's {disfmarker} there's gonna be debugging hassles and then we'd be set for when we did have five or eight, to have it really be useful. But. {pause} Notice how I said somebody and {vocalsound} turned my head your direction. That's one thing you don't get in these recordings. You don't get the {disfmarker} {pause} don't get the visuals but {disfmarker}Grad G: I is it um {pause} mostly um the neural network trainings that are {pause} um slowing us down or the HTK runs that are slowing us downProfessor B: Uh, I think yes. Uh, {vocalsound} Isn't that right I mean I think you're {disfmarker} you're sort of held up by both, right If the {disfmarker} if the neural net trainings were a hundred times faster {pause} you still wouldn't {pause} be anything {disfmarker} running through these a hundred times faster because you'd {pause} be stuck by the HTK trainings,PhD D: Mmm.Professor B: rightPhD D: Yeah.Professor B: But if the HTK {disfmarker} I mean I think they're both {disfmarker} It sounded like they were roughly equal Is that about rightPhD D: Yeah.Professor B: Yeah.Grad G: Because, um {pause} I think that'll be running Linux, and Sw - Swede and Fudge are already running Linux so, {pause} um I could try to get {pause} um the train the neural network trainings or the HTK stuff running under Linux, and to start with I'm {pause} wondering which one I should pick first.Professor B: Uh, probably the neural net cuz it's probably {disfmarker} it {disfmarker} it's {disfmarker} {pause} it's um {disfmarker} Well, I {disfmarker} I don't know. They both {disfmarker} HTK we use for {pause} um {pause} this Aurora stuff Um {pause} Um, I think {pause} It's not clear yet what we're gonna use {pause} for trainings uh {disfmarker} Well, {pause} there's the trainings uh {disfmarker} is it the training that takes the time, or the decoding Uh, is it about equal {pause} between the two For {disfmarker} for AuroraPhD D: For HTKProfessor B: For {disfmarker} Yeah. For the AuroraPhD D: Uh Training is longer.Professor B: OK.PhD D: Yeah.Professor B: OK. Well, I don't know how we can {disfmarker} I don't know how to {disfmarker} Do we have HTK source Is that {disfmarker} Yeah.PhD D: Mmm.Professor B: You would think that would fairly trivially {disfmarker} the training would, anyway, th the testing {pause} uh I don't {disfmarker} I don't {pause} think would {pause} parallelize all that well. But I think {pause} that {pause} you could {pause} certainly do d um, {pause} distributed, sort of {disfmarker} {pause} Ah, no, it's the {disfmarker} {pause} each individual {pause} sentence is pretty tricky to parallelize. But you could split up the sentences in a test set.PhD A: They have a {disfmarker} they have a thing for doing that and th they have for awhile, in H T And you can parallelize the training.Professor B: YeahPhD A: And run it on several machinesProfessor B: Aha!PhD A: and it just basically keeps counts. And there's something {disfmarker} {pause} a final {pause} thing that you run and it accumulates all the counts together.Professor B: I see.PhD D: Mmm.PhD A: I don't what their scripts are {pause} set up to do for the Aurora stuff, but {disfmarker}PhD D: Yeah.Professor B: Something that we haven't really settled on yet is other than {pause} this Aurora stuff, {pause} uh what do we do, large vocabulary {pause} training slash testing {pause} for uh tandem systems. Cuz we hadn't really done much with tandem systems for larger stuff. Cuz we had this one collaboration with CMU and we used SPHINX. Uh, we're also gonna be collaborating with SRI and we have their {disfmarker} have theirs. Um {pause} So {pause} I don't know Um. So I {disfmarker} I think the {disfmarker} the advantage of going with the neural net thing is that we're gonna use the neural net trainings, no matter what, for a lot of the things we're doing,Grad G: OK.Professor B: whereas, w exactly which HMM {disfmarker} Gaussian - mixture - based HMM thing we use is gonna depend uh So with that, maybe we should uh {vocalsound} go to our {nonvocalsound} digit recitation task. And, it's about eleven fifty. Canned. Uh, I can {disfmarker} I can start over here. Great, uh, could you give Adam a call. Tell him to He's at two nine seven seven.Grad F: Oh.Professor B: OK. I think we can {vocalsound} @ @ You know Herve's coming tomorrow, right Herve will be giving a talk, yeah, talk at eleven. Did uh, did everybody sign these consent Er everybody Has everyone signed a consent form before, on previous meetings You don't have to do it again each time Yes. microphones off
The professor thought that the model did not get hurt that much because of the removal of English. He also thought it would be worthwhile to train on one language and test on another.
What did the Professor think about the use of English?