Stochastic RNNs without Teacher-Forcing
NeurIPS2018 NIPS2018 NLP deep learning RNN
We present a stochastic non-autoregressive RNN that does not require teacher-forcing for training. The content is based on our 2018 NeurIPS paper:

Deep State Space Models for Unconditional Word Generation
https://arxiv.org/abs/1806.04550
 Hi everybody, my name is Florian and Janik was nice enough to host me here as a guest to talk about Stochastic RNNs without teacher forcing. This is based on recent work, deep state space models for unconditional word generation, which we presented at this year's New RIPs. And if you feel like any more details, please check out the paper. We focus on a de facto standard training hack for any RNNs that generate text. It's called teacher forcing and it's used in any model, whether unconditional or conditional, such as in a sentence autoencoder or in a translation model. To understand where teacher forcing comes from, we first need to understand where text generation comes from. For the good or the bad, and here we will focus on the bad, text generation has its roots in language modeling. So language modeling is the problem of predicting the next word, given all the previous words. People used to use ANGRA models for this, but today people use recurrent neural networks to do that. Such recurrent neural networks or RNNs factorize the joint observation probability of a sequence that I here depict as W into independent softmax distributions over individual tokens. So for every time step, there's a softmax function. And the softmax is conditioned on a hidden state. And all the magic of the RNN goes into the function that gives you the new state, given the old hidden state. Usually this is called a transition function, F, and as an input it gets the last state and the last word. So F could be a GUO function or an LSTM function. Just like any other language model, you can turn this into a generative model of text. Let's look at the dependencies that you would have at test time. There's initial hidden state H1. We sample a new word. We use our transition function F and it gives us the new state H2. Then we can sample a new word W2, feed it back, get a new state, sample a new word, feed it back. It's important to note that all the stochasticity in the output is solely due to the stochasticity in the sampling process, because the transition function is deterministic. So far there's nothing to complain about. But so far I've only talked about test time. At training time there is a catch. This is where teacher forcing kicks in. It turns out that you can't learn this model by basing the evolution of the hidden states on your own predictions. You have to use teacher forcing and that means you substitute your own prediction by the ground truth. So at training time there's no sampling loop. You just take the ground truth token and feed it into your state transition function. So that feels unintuitive because at test time we do something else than we do at training time. And it's also known in the literature for a few years to cause biases. So why is that problematic? Remember we come from language modeling. In language modeling we could argue that if our only goal is to predict one word given the previous words, then of course we can use the ground truth context to ground truth previous words. But if we're interested in generating like longer sequences, then we need to learn what to memorize. And in particular we need to become robust against our own predictions because we might make mistakes at test time and there's no ground truth at test time. Just to get this confirmed by somebody who has worked in the field for years, at the NeurIPS representation learning workshop Alex Grave mentioned teacher forcing as one of the big three problems for autoregressive models. And in his own words, teacher forcing might lead to predict one step ahead, not many and potentially brittle generation and myopic representations. How have people addressed teacher forcing so far? There are approaches to try to mitigate the problem. For example, by blending together these two views, training time and test time, so that sometimes you use your own prediction during training, but sometimes you use the ground truth. We believe for a rigorous model of text generation, we need a rigorous model of uncertainty. This should be an integral part of any generative model and therefore it should be the same model both at training time and test time without any hacks. We propose a fundamentally different approach by proposing a new transition function. The new transition function is non autoregressive. That means it depends on the last stage, ht-1, but it doesn't depend on the last word. That means teacher forcing is not an option anymore, but it also means teacher forcing is not a problem anymore. Instead, the transition function accepts a white noise vector as the second input. Now you might wonder why do we need noise at all as an input to the transition function? Well, for a given prefix, there might be different continuations. So we need some source of entropy to model the entropy in different continuations. The rest of the paper pretty much focuses on the following two questions. A. Which function f is powerful enough to turn the most simple noise source, just the standard Gaussian vector, into something that is powerful enough to replace the autoregressive feedback mechanism of a standard RNN? And the second question is, of course, how do we train this? What framework do we train this in? And it will turn out that variational flows are suitable functions f and variational inference is the right framework to train them. So here's the roadmap to complete the model. First, we need to cast the generative model as a probabilistic method because so far I've only sketched a procedure that involves sampling some noise and then applying some function and then predicting observations. Then we need to propose a variational inference model so that we can do maximum likelihood training. We will derive an elbow, which is our objective. Then in the paper, we also describe how the tightness of the elbow can be improved. And here I will finish by talking a bit about the evaluation and what we do to inspect the model. Since this work is based a lot on variational flows, let me give you a quick summary of variational flows. A variational flow is a diffeomorphism f, which maps from what I will call a simple noise space, Xi, to a complex noise space, H. And here I'm already using the notation for our sequence model. Simply by the change of variable formula, we know that the probability of an event H in the complex space is simply the probability of the event in the simplest space Xi as given by the inverse of f times a Jacobian term with respect to f evaluated at Xi. How can we use this in our sequential setting? First, let me fix some notation because sequential models are pretty prone to overloaded notation. I'll write time as t running from 1 to capital T. And whenever I talk about a sequence of variables like w, I don't index them. I just write w without an index. And only when I need a specific element, I'll write it as wt. Let's formalize the generative model. We start out with the probability of observing a sequence w. And since we use the latent variable model, we marginalize out the latent variables H. And then we will assume that the overall dependencies between hidden states H and observations w follow like an HMM type of dependency. That means the new state only depends on the last state and the current observation only depends on the current state. And now the question is how do we model these transitions? I've so far pitched the ideas of sampling noise and then using some transition function f. And we have seen flows already. Now we are ready to combine the two. We propose a transition function fg, which has the signature as I mentioned before. It gets a hidden state and noise vector as an input. And it gives you a new state as an output. This can be seen as a conditional flow because any ht minus 1, any last state, inserted as the first argument into fg, induces a flow which maps from the simple noise distribution to the space of new hidden states. And as I've said before, for the prior distribution in the simple noise space, we simply assume it's a standard Gaussian. Let's look at this graphically, because in the end this is a graphical model. I copied over the formulas from the last slide. And at the bottom you see the graphical model. First we have a sequence of stochastic variables Xi. Those deterministically induce via the transition function f, via the flow, a sequence of hidden states. And those independently predict the observations. All the magic is in the transition. So let me sketch this process here in the big circle. How do we get from the last state h2 to the new state h3? Let's say h2 encodes a prefix and there are two possible continuations. They're equally likely in the corpus, so there are two potential new states. The blue state h3 and the yellow state h3. I've sketched the standard Gaussian noise distribution at the top. There are yellow samples and there are blue samples. The flow realizes a mapping that takes any yellow sample and maps it to the yellow hidden state. And it maps any blue sample to the blue hidden state. So with probability one half in this situation, we either get a blue or a yellow sample from the simple noise distribution. And it will induce new states, blue h3 or the yellow h3. So far we have proposed the generative model. Now the question is how do we train it if we don't know the hidden states? The answer is variational inference and in particular, amortized variational inference. The key idea of variational inference is to introduce a parameterized approximate inference model. How do we propose such a model? Well, a good recipe is to first look at a true posterior. The probability of a state sequence given an observation sequence. The true posterior turns out to factorize into individual components, which give us the probability of a state given the last state and the future observations. It turns out that we can formulate this inference model using two ingredients that should be familiar. First, we use a transition function Fq, which induces a flow. It has the same signature as Fg for the generative model. And we use a noise source q. But now the noise source isn't uninformative anymore. In variational inference, the inference network is informed about the data. So there's a base distribution q of Xi t, which is allowed to look at the data Wt. Now compare this to teacher forcing. In teacher forcing, we substitute our own predictions by inserting ground truth information into the generative model. In variational inference, it's very clear how to use the data. The data enters through the inference model and it enters in the form of future observation because the past observation we want to store in the hidden state. It remains to derive an elbow, which is the usual evidence lower bound objective used for variational inference. Any elbow, whether it's in a sequential setting or not, factorizes into two parts, a reconstruction loss and a model mismatch term. Here, reconstruction loss means probability of observation given a state. And model mismatch is between the generative model P and the inference model q. This is what is usually written as a KL divergence. To derive our elbow, we follow the literature on flows. In the first step, we introduced the flow on the inference model Fq. We turn the expectation with respect to the complex state space H into an expectation with respect to the simple noise distribution. And then, of course, at the same time, the flow appears inside the expectation. And we get the log-determinant terms that I've mentioned before. In the second step, we introduced the generative flow Fg using the same change of variable technique. It's possible to write out the elbow in a way so that there's only one Jacobian term for both flows and so that the generative model always appears as the inverse concatenated with the inference flow. In a second, I'll show you what the interpretation of that is. Let's quickly recap what we've seen so far. There's a generative model. It consists of a generative flow Fg and an uninformed noise source. There's an inference model, which contains an inference flow Fq and a simple base distribution across the noise variables q of xi. In the elbow, the two flows appear concatenated, and we can interpret this in the following way. The inference model q proposes a noise vector, xi t, that is informed about the future. The inference flow maps this to a hidden state. At the hidden state, the reconstruction loss lives. This is where we pay a price for making a bad prediction. However, the inference model cannot encode all the possible information about the future into the hidden state, ht, because the mapping continues to the simple noise space of the generative model. And the inference model must make sure that the proposal also covers significant probability mass under the uninformed prior. This trade-off between reconstruction and model mismatch is common to all elbows. But here we highlight the special situation where we have two flows, one for the inference model and one for the generative model. In our paper, we also show how we can use the recently proposed important weighted autoencoder to improve the tightness of our bound, but I'll skip those steps here. Instead, let's quickly talk about evaluation. We apply our model to unconditional generation. So why in hell would somebody look into unconditional generation? Well, actually, it turns out it's harder than conditional generation. If you know what the French sentence looks like, it's much easier to continue a partial English translation. But it's not only harder, it's also more interesting to inspect which information does a sequence model need to store and which information can it forget. We use two metrics to evaluate our model. First, we look at sequence cross entropy. So we compare the model's sequence distribution to the data sequence distribution. Usually estimating the data distribution is impossible. You don't want to say that the probability of a sentence is how many times the sentence has appeared in the training data. However, for words, we can use unigram frequencies of words in a corpus as a pretty reliable estimate. Also, we can get an estimate of our model's probability assigned to a sequence by using MC sampling. We take the marginal likelihood, sample k trajectories, and assess the probability that the trajectories assigned to the given sequence. Since our model is not autoregressive, the sequence isn't tied to an observation. So we can actually use the same sequences of hidden states to evaluate probabilities for all the words in the vocabulary. Since we've pitched our noise model as the key to contribution to our generative model, we want to empirically verify that the model is being used. Working with a clean probabilistic model allows us to use tools from probability theory to assess that. We use the mutual information between a noise vector at time t and the observation of time t. So this measures how much information in the output is actually due to the noise model. Before showing you the numbers, let's quickly go across the parameterization of our model. For the flows, we look at shift scaling transformations. And if the scaling g is lower triangular, we can compute efficiently the Jacobian determinant. We also look at real NVP and we compose flows by concatenation. The base distribution of our inference model depends on the future observations, which we summarize using a GRU RNN. The base distribution itself is a diagonal Gaussian. We use a state size of 8 and also run some experiments for 16 and 32. All the numbers are in the paper, so here are just the take-home messages. We are on par or better than a domestic RNN with teacher forcing trained at the same state size. Also, we observed that a powerful generative flow is essential to achieve good performance. Furthermore, we can confirm that important weightless elbow improved the results. This is the first model applying generative flows to sequence modeling. So naturally, we are interested in comparing the expressiveness of fg and fq. Our paper has a table that compares four choices for both flows. Our findings are that the generative flow should be powerful and the inference flow should be slightly less powerful. To understand our noise model, we look at the mutual information at every time step and show a box spot for all of them. Initially, the mutual information is highest, which means the initial character is most important to remember. The noise model is never being ignored and we see increased variance in the remaining time steps because we are averaging here across different sequences. A non-autoregressive model needs to have lower entropy in the observation model because any underentropy under the observation model is being forgotten because there is no feedback. The purple line shows you the observation model entropy during training. The dashed red line shows you the entropy on the observation model of a baseline. So indeed, we have lower entropy in the observation model and at the same time in green, you see the mutual information increasing. Let's summarize our findings. Using variational flows, non-autoregressive modeling of sequences is possible and teacher forcing is not necessary. At the same time, we get a noise model that is the driving factor of the sequence model and is easy to interpret. For any details, please check out the paper and for any questions, shoot me an email.
 Hi everybody, my name is Florian and Janik was nice enough to host me here as a guest to talk about
 Stochastic RNNs without teacher forcing. This is based on recent work, deep state space models for
 unconditional word generation, which we presented at this year's New RIPs. And if you feel like any more details,
 please check out the paper. We focus on a de facto standard training hack for any RNNs that generate
 text. It's called teacher forcing and it's used in any model, whether unconditional or conditional,
 such as in a sentence autoencoder or in a translation model. To understand where teacher forcing comes from,
 we first need to understand where text generation comes from. For the good or the bad, and here we will focus on the bad,
 text generation has its roots in language modeling. So language modeling is the problem of predicting the next word,
 given all the previous words. People used to use ANGRA models for this, but today people use recurrent neural networks to do that.
 Such recurrent neural networks or RNNs factorize the joint observation probability of a sequence that I here depict as W
 into independent softmax distributions over individual tokens. So for every time step, there's a softmax function.
 And the softmax is conditioned on a hidden state. And all the magic of the RNN goes into the function that gives you the new state,
 given the old hidden state. Usually this is called a transition function, F, and as an input it gets the last state and the last word.
 So F could be a GUO function or an LSTM function. Just like any other language model, you can turn this into a generative model of text.
 Let's look at the dependencies that you would have at test time. There's initial hidden state H1. We sample a new word.
 We use our transition function F and it gives us the new state H2. Then we can sample a new word W2, feed it back,
 get a new state, sample a new word, feed it back. It's important to note that all the stochasticity in the output is solely due to the stochasticity
 in the sampling process, because the transition function is deterministic. So far there's nothing to complain about.
 But so far I've only talked about test time. At training time there is a catch. This is where teacher forcing kicks in.
 It turns out that you can't learn this model by basing the evolution of the hidden states on your own predictions.
 You have to use teacher forcing and that means you substitute your own prediction by the ground truth.
 So at training time there's no sampling loop. You just take the ground truth token and feed it into your state transition function.
 So that feels unintuitive because at test time we do something else than we do at training time.
 And it's also known in the literature for a few years to cause biases. So why is that problematic?
 Remember we come from language modeling. In language modeling we could argue that if our only goal is to predict one word given the previous words,
 then of course we can use the ground truth context to ground truth previous words.
 But if we're interested in generating like longer sequences, then we need to learn what to memorize.
 And in particular we need to become robust against our own predictions because we might make mistakes at test time and there's no ground truth at test time.
 Just to get this confirmed by somebody who has worked in the field for years, at the NeurIPS representation learning workshop Alex Grave mentioned
 teacher forcing as one of the big three problems for autoregressive models.
 And in his own words, teacher forcing might lead to predict one step ahead, not many and potentially brittle generation and myopic representations.
 How have people addressed teacher forcing so far? There are approaches to try to mitigate the problem.
 For example, by blending together these two views, training time and test time, so that sometimes you use your own prediction during training,
 but sometimes you use the ground truth. We believe for a rigorous model of text generation, we need a rigorous model of uncertainty.
 This should be an integral part of any generative model and therefore it should be the same model both at training time and test time without any hacks.
 We propose a fundamentally different approach by proposing a new transition function.
 The new transition function is non autoregressive. That means it depends on the last stage, ht-1, but it doesn't depend on the last word.
 That means teacher forcing is not an option anymore, but it also means teacher forcing is not a problem anymore.
 Instead, the transition function accepts a white noise vector as the second input.
 Now you might wonder why do we need noise at all as an input to the transition function?
 Well, for a given prefix, there might be different continuations.
 So we need some source of entropy to model the entropy in different continuations.
 The rest of the paper pretty much focuses on the following two questions.
 A. Which function f is powerful enough to turn the most simple noise source, just the standard Gaussian vector,
 into something that is powerful enough to replace the autoregressive feedback mechanism of a standard RNN?
 And the second question is, of course, how do we train this? What framework do we train this in?
 And it will turn out that variational flows are suitable functions f and variational inference is the right framework to train them.
 So here's the roadmap to complete the model.
 First, we need to cast the generative model as a probabilistic method because so far I've only sketched a procedure
 that involves sampling some noise and then applying some function and then predicting observations.
 Then we need to propose a variational inference model so that we can do maximum likelihood training.
 We will derive an elbow, which is our objective.
 Then in the paper, we also describe how the tightness of the elbow can be improved.
 And here I will finish by talking a bit about the evaluation and what we do to inspect the model.
 Since this work is based a lot on variational flows, let me give you a quick summary of variational flows.
 A variational flow is a diffeomorphism f, which maps from what I will call a simple noise space, Xi, to a complex noise space, H.
 And here I'm already using the notation for our sequence model.
 Simply by the change of variable formula, we know that the probability of an event H in the complex space is simply the probability of the event
 in the simplest space Xi as given by the inverse of f times a Jacobian term with respect to f evaluated at Xi.
 How can we use this in our sequential setting?
 First, let me fix some notation because sequential models are pretty prone to overloaded notation.
 I'll write time as t running from 1 to capital T.
 And whenever I talk about a sequence of variables like w, I don't index them. I just write w without an index.
 And only when I need a specific element, I'll write it as wt.
 Let's formalize the generative model.
 We start out with the probability of observing a sequence w.
 And since we use the latent variable model, we marginalize out the latent variables H.
 And then we will assume that the overall dependencies between hidden states H and observations w follow like an HMM type of dependency.
 That means the new state only depends on the last state and the current observation only depends on the current state.
 And now the question is how do we model these transitions?
 I've so far pitched the ideas of sampling noise and then using some transition function f.
 And we have seen flows already. Now we are ready to combine the two.
 We propose a transition function fg, which has the signature as I mentioned before.
 It gets a hidden state and noise vector as an input.
 And it gives you a new state as an output.
 This can be seen as a conditional flow because any ht minus 1, any last state, inserted as the first argument into fg,
 induces a flow which maps from the simple noise distribution to the space of new hidden states.
 And as I've said before, for the prior distribution in the simple noise space, we simply assume it's a standard Gaussian.
 Let's look at this graphically, because in the end this is a graphical model.
 I copied over the formulas from the last slide.
 And at the bottom you see the graphical model.
 First we have a sequence of stochastic variables Xi.
 Those deterministically induce via the transition function f, via the flow, a sequence of hidden states.
 And those independently predict the observations.
 All the magic is in the transition.
 So let me sketch this process here in the big circle.
 How do we get from the last state h2 to the new state h3?
 Let's say h2 encodes a prefix and there are two possible continuations.
 They're equally likely in the corpus, so there are two potential new states.
 The blue state h3 and the yellow state h3.
 I've sketched the standard Gaussian noise distribution at the top.
 There are yellow samples and there are blue samples.
 The flow realizes a mapping that takes any yellow sample and maps it to the yellow hidden state.
 And it maps any blue sample to the blue hidden state.
 So with probability one half in this situation, we either get a blue or a yellow sample from the simple noise distribution.
 And it will induce new states, blue h3 or the yellow h3.
 So far we have proposed the generative model.
 Now the question is how do we train it if we don't know the hidden states?
 The answer is variational inference and in particular, amortized variational inference.
 The key idea of variational inference is to introduce a parameterized approximate inference model.
 How do we propose such a model?
 Well, a good recipe is to first look at a true posterior.
 The probability of a state sequence given an observation sequence.
 The true posterior turns out to factorize into individual components,
 which give us the probability of a state given the last state and the future observations.
 It turns out that we can formulate this inference model using two ingredients that should be familiar.
 First, we use a transition function Fq, which induces a flow.
 It has the same signature as Fg for the generative model.
 And we use a noise source q.
 But now the noise source isn't uninformative anymore.
 In variational inference, the inference network is informed about the data.
 So there's a base distribution q of Xi t, which is allowed to look at the data Wt.
 Now compare this to teacher forcing.
 In teacher forcing, we substitute our own predictions by inserting ground truth information into the generative model.
 In variational inference, it's very clear how to use the data.
 The data enters through the inference model and it enters in the form of future observation
 because the past observation we want to store in the hidden state.
 It remains to derive an elbow, which is the usual evidence lower bound objective used for variational inference.
 Any elbow, whether it's in a sequential setting or not, factorizes into two parts,
 a reconstruction loss and a model mismatch term.
 Here, reconstruction loss means probability of observation given a state.
 And model mismatch is between the generative model P and the inference model q.
 This is what is usually written as a KL divergence.
 To derive our elbow, we follow the literature on flows.
 In the first step, we introduced the flow on the inference model Fq.
 We turn the expectation with respect to the complex state space H into an expectation with respect to the simple noise distribution.
 And then, of course, at the same time, the flow appears inside the expectation.
 And we get the log-determinant terms that I've mentioned before.
 In the second step, we introduced the generative flow Fg using the same change of variable technique.
 It's possible to write out the elbow in a way so that there's only one Jacobian term for both flows
 and so that the generative model always appears as the inverse concatenated with the inference flow.
 In a second, I'll show you what the interpretation of that is.
 Let's quickly recap what we've seen so far.
 There's a generative model.
 It consists of a generative flow Fg and an uninformed noise source.
 There's an inference model, which contains an inference flow Fq
 and a simple base distribution across the noise variables q of xi.
 In the elbow, the two flows appear concatenated, and we can interpret this in the following way.
 The inference model q proposes a noise vector, xi t, that is informed about the future.
 The inference flow maps this to a hidden state.
 At the hidden state, the reconstruction loss lives.
 This is where we pay a price for making a bad prediction.
 However, the inference model cannot encode all the possible information about the future into the hidden state, ht,
 because the mapping continues to the simple noise space of the generative model.
 And the inference model must make sure that the proposal also covers significant probability mass under the uninformed prior.
 This trade-off between reconstruction and model mismatch is common to all elbows.
 But here we highlight the special situation where we have two flows, one for the inference model and one for the generative model.
 In our paper, we also show how we can use the recently proposed important weighted autoencoder to improve the tightness of our bound, but I'll skip those steps here.
 Instead, let's quickly talk about evaluation.
 We apply our model to unconditional generation.
 So why in hell would somebody look into unconditional generation?
 Well, actually, it turns out it's harder than conditional generation.
 If you know what the French sentence looks like, it's much easier to continue a partial English translation.
 But it's not only harder, it's also more interesting to inspect which information does a sequence model need to store and which information can it forget.
 We use two metrics to evaluate our model.
 First, we look at sequence cross entropy.
 So we compare the model's sequence distribution to the data sequence distribution.
 Usually estimating the data distribution is impossible.
 You don't want to say that the probability of a sentence is how many times the sentence has appeared in the training data.
 However, for words, we can use unigram frequencies of words in a corpus as a pretty reliable estimate.
 Also, we can get an estimate of our model's probability assigned to a sequence by using MC sampling.
 We take the marginal likelihood, sample k trajectories, and assess the probability that the trajectories assigned to the given sequence.
 Since our model is not autoregressive, the sequence isn't tied to an observation.
 So we can actually use the same sequences of hidden states to evaluate probabilities for all the words in the vocabulary.
 Since we've pitched our noise model as the key to contribution to our generative model, we want to empirically verify that the model is being used.
 Working with a clean probabilistic model allows us to use tools from probability theory to assess that.
 We use the mutual information between a noise vector at time t and the observation of time t.
 So this measures how much information in the output is actually due to the noise model.
 Before showing you the numbers, let's quickly go across the parameterization of our model.
 For the flows, we look at shift scaling transformations.
 And if the scaling g is lower triangular, we can compute efficiently the Jacobian determinant.
 We also look at real NVP and we compose flows by concatenation.
 The base distribution of our inference model depends on the future observations, which we summarize using a GRU RNN.
 The base distribution itself is a diagonal Gaussian.
 We use a state size of 8 and also run some experiments for 16 and 32.
 All the numbers are in the paper, so here are just the take-home messages.
 We are on par or better than a domestic RNN with teacher forcing trained at the same state size.
 Also, we observed that a powerful generative flow is essential to achieve good performance.
 Furthermore, we can confirm that important weightless elbow improved the results.
 This is the first model applying generative flows to sequence modeling.
 So naturally, we are interested in comparing the expressiveness of fg and fq.
 Our paper has a table that compares four choices for both flows.
 Our findings are that the generative flow should be powerful and the inference flow should be slightly less powerful.
 To understand our noise model, we look at the mutual information at every time step and show a box spot for all of them.
 Initially, the mutual information is highest, which means the initial character is most important to remember.
 The noise model is never being ignored and we see increased variance in the remaining time steps because we are averaging here across different sequences.
 A non-autoregressive model needs to have lower entropy in the observation model because any underentropy under the observation model is being forgotten because there is no feedback.
 The purple line shows you the observation model entropy during training.
 The dashed red line shows you the entropy on the observation model of a baseline.
 So indeed, we have lower entropy in the observation model and at the same time in green, you see the mutual information increasing.
 Let's summarize our findings.
 Using variational flows, non-autoregressive modeling of sequences is possible and teacher forcing is not necessary.
 At the same time, we get a noise model that is the driving factor of the sequence model and is easy to interpret.
 For any details, please check out the paper and for any questions, shoot me an email.