Problem
=======


This is the first time I have posted a question on this site and it may not be suitable for this venue, which is primarily used for research questions in maths. If someone finds it unsuitable, it could be migrated to other sites like Maths StackExchange.


In a [paper](https://arxiv.org/pdf/1712.06541.pdf) I am reading now, the author claims



> 
> ... if some network computes a non-trivial function, and the product of its Schatten $p$-norm (for any $p<\infty$) is bounded, then there must be at least one parameter matrix, which is not far from being rank-1.
> 
> 
> 


Mathematically, this means that if the output of network $N\_{\mathbf{W}\_{1}^d}(\mathbf{x})$ is not trivial, and $\prod\_{i=1}^d\Vert \mathbf{W}\_i\Vert\_p\leq M$, then there exists $r\in \{1,2,\cdots,d\}$ such that $rank(\mathbf{W}\_r)=1$.


However, the authors do not provide *any* proof of this claim. Furthermore, all their following theorems are motivated by this claim.


I am not sure if this is obvious, but could someone help me understand why this claim holds? Thank you in advance.



Existence of rank-1 weight matrix in some type of deep neural network
I just skimmed the paper, but it seems that this is later formalized into Theorem 3, a proof of which is in the appendix.


