Let $c(i,i')$ be a kernel function on a reasonable index space $I$. Choose a dense sequence of points $\{i\_1, i\_2, \cdots \} \subseteq I$, and define the one-point kernel functions $k\_n := c(\cdot, i\_n)$. Let $U$ be the Hilbert span of these functions; by the reproducing kernel property, $\langle k\_n, k\_{n'} \rangle := c(i\_n, i\_{n'})$.


Using the [Gram-Schmidt procedure](https://en.wikipedia.org/wiki/Gram-Schmidt_procedure), we convert the sequence $\{k\_n\}$ into an orthogonal basis $\{\hat k\_n\}$, by subtracting away the projection onto the previous axes (without normalization). Define the "novel variance" $\hat \sigma\_n^2 := \langle \hat k\_n, \hat k\_n \rangle$, which represents the new contribution to variance from data located at $i\_n$.


Now, define $\Sigma^2 := \sum \hat \sigma\_n^2$, which represents all the possible novel variance to be found in $I$. It is not difficult to show that $\Sigma^2$ does not depend on the choice of basis.


**Question:** If $I$ is compact, then is $\Sigma^2 < \infty$?



Finiteness of "novel variance" from a kernel on a compact space
You haven't specified what role is played by the topology of $I$, so my default assumption will be that the kernel is continuous, in which case the answer to your question is negative.


For a counterexample, let $I$ be the one-point compactification of $\mathbb{N}$, that is, $\{1,2,\dots,\infty\}$, and take $k\_{nm} := \delta\_{nm} n^{-1}$. Then the basis is already orthogonal, so $\hat\sigma\_n^2 = k\_{nn}$, and $\sum\_n \hat\sigma\_n^2 = \infty$.


