I have the following question related to multivariable moment generating functions. Given two vectors $u,X\in\mathbb{R}^d$, I want to characterize the quantity $e^{u^T X}$ in terms of "powers" of the vector $X$. To see what I mean, note that by the power series expansion of the exponential:
$$e^{u^T X} = \sum\_{n=0}^\infty \frac{1}{n!}\left(u^T X\right)^n$$
which is "sort of" in terms of powers of $X$, but they're all mixed up with "powers of $u$". However, for $n\leq 2$ we can break up the dependence and make the dependence on $X$ very explicit. For $n=0$ there's nothing special (everything is constant), and for $n=1$ we are left with the inner product $u^T X = \langle u, X \rangle$, so $X$ itself characterizes the "first power of $X$". For $n=2$, we may write:
$$\left(u^T X\right)^2 = \left(u^T X\right)\left(u^T X\right) = \left(u^T X\right)\left(X^T u\right) = u^T \Sigma u$$
where $\Sigma = XX^T$ characterizes the "second power of $X$" in the matrix product $u^T \Sigma u$.


If we try to do the same for $n=3$ however, there is no way to characterize an "$n$th power of $X$" as a matrix - seemingly, we need an object of dimension $d\times d\times d$. I did some research into this in the setting of multidimensional Taylor series expansions, and it would seem that the appropriate object for the "$n$th moment of $X$" is a tensor of dimension $d\times d \times \overset{n}{...} \times d$. My question is, in this setting (the power series for $e^{u^T X}$), how can I write the product $(u^T X)^n$ as a general "tensor product", and how can I generally write the "power of $X$" that determines the product, given $n$ copies of the vector $u$?


My motivation for this question is in studying the moment generating function when $X$ is a random vector. In the one-dimensional case ($d=1$), assuming the moment generating function exists in a small neighbourhood around zero, the moments $(1, E[X], E[X^2], ...)$ uniquely determine the distribution. I'm interested in the multidimensional analogue, where given a positive radius of convergence around zero for the multidimensional moment generating function, the countable sequence $(1, E[X], E[XX^T], \overset{?}{...})$ uniquely determines the distribution of $X$.


Thank you!


Edit: I don't know much about the tensor product, but I found one resource that said the tensor product of two vectors $v, w$ is defined as $v\otimes w = vw^T$. I'd blindly guess then, since this is the form of the second-order term $E[XX^T]$, that the "$n$th moment of $X$" should be defined as
$$M\_n = E\left[\bigotimes\_{k=1}^{n} X\right]$$
where supposedly the empty product convention yields the scalar $1$. Is this the right idea? If so, does anyone have any resources that would be good for a beginner here? I'm not even sure how I would write the product of this object with $n$ copies of $u$ to get back to the scalar $\left(u^T X\right)^n$.



Multi-variable expansion of $e^{u^T X}$ as a power series in terms of tensors
Define the tensor $v^{\otimes k}$ with elements $[v^{\otimes k}]\_{i\_1,i\_2,\ldots i\_k}=v\_{i\_1}v\_{i\_2}\cdots v\_{i\_k}$ and denote the full contraction of two tensors by $A\odot B=\sum\_{i\_1,i\_2,\ldots i\_k}A\_{i\_1,i\_2,\ldots i\_k}B\_{i\_k,\ldots i\_2,i\_1}$, then you can write
$$e^{u^T X} = \sum\_{n=0}^\infty \frac{1}{n!}u^{\otimes n}\odot X^{\otimes n}.$$


