- E. Xing, F. Xu, Abhishek Chugh. 2014. 29 : Posterior Regularization. Abstract: This is the last lecture which tends to tie together everything we learn so far. What we learned this semester doesn’t have to be practiced and applied in an isolated fashion. There is a possibility of grand integration with a method that benefits from many aspects. The main title here, posterior regularization, is actually smaller than what is going to be covered in this lecture. This lecture is not just about regularization, but about the integrative paradigm for learning graphical models. The leaning of graphical models is primarily built on the maximum likelihood principle because that is the most common loss function we define on the graphs. However, as we have seen from previous lectures, there are multiple ways to define alternative loss functions. For example, we can put prior distribution to Bayesian estimation over the model and parameters, so that in the end we can choose to optimize the posterior probability of the model given the data. Very recently, we also have learned some kernel methods, which are new ways of designing loss function structures over the graphical models. In the last lecture, we also learned to bring the max-margin principle as an alternative to drive the derivation of an optimal graphical model in terms of coefficients on certain features or potential functions. All these methods have advantages as well as disadvantages. The greedy question to ask is, can we land on the middle part, which is the integration of everything, with the hope to harness the advantages of all of these methods as shown in Figure 1. We hope the disadvantages are mutually exclusive while the advantages can coexist. One of the recent attempt to do so is the regularized Bayesian inference.
