input gradient alignment
220646948
Attributional Robustness Training Using Input Gradient Spatial Alignment
Interpretability is an emerging area of research in trustworthy machine learning. Safe deployment of machine learning system mandates that the prediction and its explanation be reliable and robust. Recently, it has been shown that the explanations could be manipulated easily by adding visually imperceptible perturbations to the input while keeping the model's prediction intact. In this work, we study the problem of attributional robustness (i.e. models having robust explanations) by showing an upper bound for attributional vulnerability in terms of spatial correlation between the input image and its explanation map. We propose a training methodology that learns robust features by minimizing this upper bound using soft margin triplet loss. Our methodology of robust attribution training \textit{ART} achieves the new state of the art attributional robustness measure by a margin of \approx$ 6 18 on several standard datasets, ie. SVHN, CIFAR 10 and GTSRB. We further show the utility of the proposed robust training technique \textit{ART} in the downstream task of weakly supervised object localization by achieving the new state of the art performance on CUB 200 dataset.
ECCV
Mayank Kumar Singh Nupur Kumari Puneet Mangla Abhishek Sinha Vineeth N Balasubramanian Balaji Krishnamurthy
219708484
Gradient Alignment in Deep Neural Networks
One cornerstone of interpretable deep learning is the high degree of visual alignment that input gradients, i.e.,the gradients of the output w.r.t. inputs, exhibit with the input data. This alignment is assumed to arise as a result of the model's generalization, justifying its use for interpretability. However, recent work has shown that it is possible to 'fool' models into having arbitrary gradients while achieving good generalization, thus falsifying the assumption above. This leaves an open question: if not generalization, what causes input gradients to align with input data? In this work, we first show that it is simple to 'fool' input gradients using the shift invariance property of softmax, and that gradient structure is unrelated to model generalization. Second, we re interpret the logits of standard classifiers as unnormalized log densities of the data distribution, and find that we can improve this gradient alignment via a generative modelling objective called this http URL show this, we derive a novel approximation to the score matching objective that eliminates the need for expensive Hessian computations, which may be of independent interest.Our experiments help us identify one factor that causes input gradient alignment in models, that being the approximate generative modelling behaviour of the normalized logit distributions.
ArXiv
Suraj Srinivas Francois Fleuret
59606132
Semi Supervised Learning by Label Gradient Alignment
We present label gradient alignment, a novel algorithm for semi supervised learning which imputes labels for the unlabeled data and trains on the imputed labels. We define a semantically meaningful distance metric on the input space by mapping a point (x, y) to the gradient of the model at (x, y) We then formulate an optimization problem whose objective is to minimize the distance between the labeled and the unlabeled data in this space, and we solve it by gradient descent on the imputed labels. We evaluate label gradient alignment using the standardized architecture introduced by Oliver et al. (2018) and demonstrate state of the art accuracy in semi supervised CIFAR 10 classification.
ArXiv
Jacob Jackson John Schulman
49556250
Glass Reflection Removal Using Co Saliency Based Image Alignment and Low Rank Matrix Completion in Gradient Domain
The images taken through glass often capture a target transmitted scene as well as undesired reflected scenes. In this paper, we propose a novel reflection removal algorithm using multiple glass images taken from slightly different camera positions. We first find co saliency maps for input multiple glass images based on the center prior assumption, and then align multiple images reliably with respect to the transmitted scene by selecting feature points with high co saliency values. The gradients of the transmission images are consistent while the gradients of the reflection images are varying across the aligned multiple glass images. Based on this observation, we compute gradient reliability such that the pixels belonging to consistent salient edges of the transmission image are assigned high reliability values. We restore the gradients of the transmission images and suppress the gradients of the reflection images by formulating a low rank matrix completion problem in gradient domain. Finally, we reconstruct desired transmission images from the restored transmission gradients. Experimental results show that the proposed algorithm removes the reflection artifacts from glass images faithfully and outperforms the existing methods on challenging glass images with diverse characteristics.
IEEE Transactions on Image Processing
Byeong-Ju Han Jae-Young Sim
226964434
Data efficient Alignment of Multimodal Sequences by Aligning Gradient Updates and Internal Feature Distributions
The task of video and text sequence alignment is a pre requisite step toward joint understanding of movie videos and screenplays. However, supervised methods face the obstacle of limited realistic training data. With this pa per, we attempt to enhance data efficiency of the end to end alignment network NeuMATCH [15] Recent research [56] suggests that network components dealing with different modalities may overfit and generalize at different speeds, creating difficulties for training. We propose to employ (1) layer wise adaptive rate scaling (LARS) to align the magnitudes of gradient updates in different layers and balance the pace of learning and (2) sequence wise batch normalization (SBN) to align the internal feature distributions from different modalities. Finally, we leverage random projection to reduce the dimensionality of input features. On the YouTube Movie Summary dataset, the combined use of these technique closes the performance gap when the pretraining on the LSMDC dataset is omitted and achieves the state of the art result. Extensive empirical comparisons and analysis reveal that these techniques improve optimization and regularize the network more effectively than two different setups of layer normalization.
2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
Jianan Wang Boyang Li Xiangyu Fan Jing-Jhih Lin Yanwei Fu
7423311
Alignment of Multimodal Sensory Input in the Superior Colliculus through a Gradient Matching Mechanism
The superior colliculus (SC) is a midbrain structure that integrates visual, somatosensory, and auditory inputs to direct head and eye movements. Each of these modalities is topographically mapped and aligned with the others to ensure precise behavioral responses to multimodal stimuli. While it is clear that neural activity is instructive for topographic alignment of inputs from the visual cortex (V1) and auditory system with retinal axons in the SC, there is also evidence that activity independent mechanisms are used to establish topographic alignment between modalities. Here, we show that the topography of the projection from primary somatosensory cortex (S1) to the SC is established during the first postnatal week. Unlike V1 SC projections, the S1 SC projection does not bifurcate when confronted with a duplicated retinocollicular map, showing that retinal input in the SC does not influence the topography of the S1 SC projection. However, S1 SC topography is disrupted in mice lacking ephrin As, which we find are expressed in graded patterns along with their binding partners, the EphA4 and EphA7, in both S1 and the somatosensory recipient layer of the SC. Together, these data support a model in which somatosensory inputs into the SC map topographically and establish alignment with visual inputs in the SC using a gradient matching mechanism.
The Journal of Neuroscience
Jason W Triplett Ann H Phan Jena Yamada David A Feldheim
235446621
Gradient trained Weights in Wide Neural Networks Align Layerwise to Error scaled Input Correlations
Recent works have examined how deep neural networks, which can solve a variety of difficult problems, incorporate the statistics of training data to achieve their success. However, existing results have been established only in limited settings. In this work, we derive the layerwise weight dynamics of infinite width neural networks with nonlinear activations trained by gradient descent. We show theoretically that weight updates are aligned with input correlations from intermediate layers weighted by error, and demonstrate empirically that the result also holds in finite width wide networks. The alignment result allows us to formulate backpropagation free learning rules, named Align zero and Align ada, that theoretically achieve the same alignment as backpropagation. Finally, we test these learning rules on benchmark problems in feedforward and recurrent neural networks and demonstrate, in wide networks, comparable performance to backpropagation.
ArXiv
Akhilan Boopathy Ila R Fiete
27198202
Image Alignment by Online Robust PCA via Stochastic Gradient Descent
Aligning a given set of images is usually conducted in batch mode manner, which not only requires large amount of memory but also adjusts all the previous transformations to register an input image. To address this issue, we propose a novel approach to image alignment by incorporating the geometric transformation into online robust principal component analysis (PCA) Instead of calculating the warp update using noisy input samples like the conventional methods, we suggest directly linearizing the object function by performing warp update on the recovered samples, which corresponds to an efficient inverse composition algorithm. Since the basis matrix is kept constant for a given sample, both the latent vector and warp update can be very efficiently computed. Moreover, we present two basis updating methods for robust PCA, including the closed form solution and stochastic gradient descent scheme. We have conducted the extensive experiments on the real world tasks of background subtraction with camera motion and visual tracking on the challenging video sequences, whose promising results demonstrate the efficacy of our presented approach.
IEEE Transactions on Circuits and Systems for Video Technology
Wenjie Song Jianke Zhu Yang Li Chun Chen
7637082
Beam Tracking for Interference Alignment in Time Varying MIMO Interference Channels: A Conjugate Gradient Based Approach
In this paper, an adaptive beam tracking algorithm for interference alignment (IA) in time varying multiple input multiple output (MIMO) interference channels is presented. It is shown that obtaining a set of interference aligning transmit beamforming matrices is equivalent to minimizing a certain Rayleigh quotient, and an approach based on the conjugate gradient (CG) method combined with metric projection is applied to this minimization problem to construct an adaptive algorithm for interference aligning beam design. The convergence of the proposed algorithm in static channels is established, and the steady state behavior of the proposed algorithm in time varying channels is investigated by numerical simulations. The performance of the proposed algorithm is evaluated numerically, and numerical results show that the proposed algorithm performs well with low computational complexity.
IEEE Transactions on Vehicular Technology
Junseo Lee Heejung Yu Youngchul Sung
218628827
Reconfigurable Intelligent Surface for Interference Alignment in MIMO Device to Device Networks
In multiple input multiple output (MIMO) device to device (D2D) networks, interference and rank deficient channels are the critical bottlenecks for achieving high degrees of freedom (DoFs) In this paper, we propose a reconfigurable intelligent surface (RIS) assisted interference alignment strategy to simultaneously mitigate the co channel interference and cope with rank deficient channels, thereby improving the feasibility of interference alignment conditions and in turn increasing the achievable DoFs. The key enabler is a general low rank optimization approach that maximizes the achievable DoFs by jointly designing the phase shift and transceiver coding matrices. To address the unique challenges of the rank objective function, the unit modulus constraints, and the bilinear constraint resulting from coupled optimization variables, we develop a block structured Riemannian pursuit method by solving fixed rank and unit modulus constrained least square subproblems along with rank increase. Finally, to achieve good DoF performance, we develop unified Riemannian conjugate gradient algorithms to alternately optimize the fixed rank transceiver matrix and the unit modulus constrained phase shifter by exploiting the non compact Stiefel manifold and the complex circle manifold, respectively. Numerical results demonstrate the effectiveness of deploying an RIS and the superiority of the proposed block structured Riemannian pursuit method in terms of the achievable DoFs compared to the state of the art methods.
2021 IEEE International Conference on Communications Workshops (ICC Workshops)
Min Fu Yong Zhou Yuanming Shi