Let $A = [A\_1, \ldots, A\_m] \in \mathbb{R}^{n \times md}$, where for all $i=1,\ldots,m$, $A\_i \in \mathbb{R}^{n \times d}$, $d>1$. Let $x = [x\_1,\ldots,x\_m]^\top \in \mathbb{R}^m$ with $\|x\|\_2 \leq \varepsilon$, then what is a tight upper bound of $\big\| \sum\_{i=1}^m A\_i x\_i \big\|\_2$, i.e., the spectral norm of a weighted sum of blocks matrices $A\_i$ ($A\_i$ is not a vector here), in terms of the largest singular value $\sigma\_{\max}(A)$ of $A$ and $\varepsilon$?


A direct calculation gives
$\big\| \sum\_{i=1}^m A\_i x\_i \big\|\_2 \leq \sum\_{i=1}^m \|A\_i\|\_2 |x\_i| \leq \sigma\_{\max}(A) \|x\|\_1 \leq \sqrt{m} \sigma\_{\max}(A) \varepsilon$. But is it possible to get $$\big\| \sum\_{i=1}^m A\_i x\_i \big\|\_2 \leq \sigma\_{\max}(A) \varepsilon$$


why or why not?


When $d=1$, it simply holds from the Cauchy-Schwarz inequality. When $m=1$, this also holds trivially. But what if $m>1$ and $d>1$? 



spectral norm of block-wise sums of matrices
If I understand correctly, $\sum A\_i x\_i = A (x \otimes I\_d)$, so
$$
\|\sum A\_i x\_i\| \leq \|A\|\, \|x\otimes I\_d\| = \|A\|\, \|x\|,
$$
where the last inequality holds because $\|M\otimes N\|=\|M\|\,\|N\|$ for all matrices $M$, $N$ (which itself holds because the singular values of $M\otimes N$ are obtained by multiplying those of $M$ and $N$).


