We propose a general matrix-valued multiple kernel learning framework for high-dimensional nonlinear multivariate regression problems. This framework allows a broad class of mixed norm regularizers, including those that induce sparsity, to be imposed on a dictionary of vector-valued Reproducing Kernel Hilbert Spaces (Michelli and Pontil, 2005). We develop a highly scalable and eigendecomposition-free Block coordinate descent procedure that orchestrates two inexact solvers: a Conjugate Gradient (CG) based Sylvester equation solver for solving vector-valued Regularized Least Squares (RLS) problems, and a specialized Sparse approximate SDP solver (Hazan, 2008) for learning output kernels. As an application of our framework, we show how high-dimensional causal inference tasks can be naturally cast as sparse function estimation problems within our framework, leading to novel nonlinear extensions of Grouped Graphical Granger Causality techniques. The algorithmic developments and extensive empirical studies are complemented by theoretical analyses in terms of Rademacher generalization bounds.
['Vikas Sindhwani', 'Aurelie C. Lozano', 'Ha Quang Minh']
Scalable Matrix-valued Kernel Learning and High-dimensional Nonlinear Causal Inference