Feel free to restrict the function space to a Hilbert space or to a RKHS. Given a probability distribution on it when can we define a ``covariance operator" for it and when would it also have a well-defined notion of eigenfunctions for it? 



About covariance operators for probability distributions on a function space
Let $X$ be a random vector taking values in a
separable Hilbert space $H$ such that $E\|X\|^2<\infty$ and $E X=\mu$. Then the corresponding covariance operator $R\colon H\to H$ is defined by the formula 
\begin{equation\*}
 Rx:=E\langle x,X-\mu\rangle (X-\mu)=E\overline{\langle X-\mu,x\rangle}(X-\mu) 
\end{equation\*} 
for $x\in H$, so that for any vectors $x$ and $y$ in $H$, 
\begin{equation\*}
 \langle Rx,y\rangle=E\overline{\langle X-\mu,x\rangle}\langle X-\mu,y\rangle, 
\end{equation\*}
the covariance of the "$x$-coordinate" $\langle X,x\rangle$ and the "$y$-coordinate" $\langle X,y\rangle$ of the random vector $X$. 


A brief account on the compactness of the covariance operator and its spectral decomposition is given in Appendices E and F of the paper at \url{<http://projecteuclid.org/euclid.ejs/1460463653>}. 


If $X$ is a zero-mean square-integrable stochastic process on a closed bounded interval $I$, with a continuous covariance function, then, by the Karhunen--Loève theorem (\url{<https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem#The_Wiener_process>}), 
$X$ admits a representation as a series of the form $\sum\_k Z\_k e\_k$, where the $e\_k$'s form an orthonormal basis of $L^2(I)$ of eigenvectors of the covariance operator of $X$ and the $Z\_k$'s are zero-mean random variables such that $EZ\_jZ\_k=\lambda\_j\delta\_{jk}$, where $\lambda\_j$ is the eigenvalue corresponding to the eigenvector $e\_j$, and $\delta\_{jk}$ is the Kronecker symbol. 
The specific Karhunen--Loève decompositions for the Brownian motion (Wiener process) and the Brownian bridge can also be found at \url{<https://en.wikipedia.org/wiki/Karhunen%E2%80%93Lo%C3%A8ve_theorem#The_Wiener_process>}. 


