Let $X\_1,\dots,X\_n$ be vectors in $\mathbb{R^d}$. Assume all of the vectors are inside the unite $\ell\_2$ ball, but outside the ball of radius $r$ for some $r \in (0,1)$, i.e. $r \leq \|X\_i\| \leq 1$ . Let $P$ be a vector in the probability simplex $\Delta\_n$ with $P\_i>0$ for all $i$. Consider the second moment matrix $\Sigma(P) = \sum\_{i=1}^n P\_i X\_i X\_i^\top$. Assume the $X\_i$s are such that $\Sigma(P)$ is full rank. Does the following bound always hold? If not, when does it hold?
$$\|\Sigma(P)^{-1}X\_j\| \leq \frac{1}{r P\_j} \quad \forall j\in \{1,\dots,n\}$$
For instance, if $n=d$ and $X\_i=e\_i$ are the canonical basis vectors of $\mathbb{R}^d$, then this bounds holds with equality.



Is there a bound on the norm of the product of second moment matrix with random vector?
$\newcommand\Si{\Sigma}$
$\newcommand\X{\mathbf X}$
The answer is no. Indeed, let $p\_i:=P\_i$, $p:=P$, $\X:=(X\_1,\dots,X\_n)$, and $\Si\_\X:=\Si(p)$. At least one of the vectors $\Si^{-1}X\_j$ is nonzero, for some $j$, because otherwise the matrix
$$I=\Si\_\X^{-1}\Si\_\X=\sum\_1^n p\_i\Si\_\X^{-1}X\_i X\_i^T$$
would be zero. So, there is some $j$ such that $c:=\|\Si\_\X^{-1}X\_j\|>0$. Replacing now $\X$ by $a\X$ for real $a>0$ and letting $a\to0$, we will have 
$$\|\Si\_{a\X}^{-1}(aX\_j)\|=\frac1a\,\|\Si\_\X^{-1} X\_j\|=\frac ca\to\infty,$$
so that the inequality 
$$\|\Si\_{a\X}^{-1}(aX\_j)\|\le\frac1{p\_j}$$
will fail to hold for small enough $a$. 




---


The OP has edited the question, thus invalidating this answer. However, even after the edit, the answer remains no. E.g., let $n=2$, $p\_1=p\_2=1/2$, $X\_1:=[1,1]^T/\sqrt2$, and $X\_2:=[1,1-h]^T/\sqrt2$ with $h\downarrow0$. Then $\|X\_1\|=1$, $1\ge\|X\_2\|\sim1$, but 
$$\|\Si\_{\X}^{-1}(X\_1)\|=\frac{2 \sqrt{2} \sqrt{h^2-2 h+2}}{h}
\sim\frac4h
\not\lesssim2=\frac1{p\_1}.$$


