Recall that a Hilbert space $\mathcal{H}$ is a reproducing kernel Hilbert space (RKHS) if the elements of $\mathcal{H}$ are functions on a certain set $X$ and for any $a\in X$, the linear functional $f\mapsto f(a)$ is bounded on $\mathcal{H}$. By Riesz Representation Theorem, there exists an element $K\_a\in\mathcal{H}$ such that 
$$f(a) = \langle f, K\_a\rangle\ \text{ for all } \ f\in\mathcal{H}.$$
The function $K(x,y) = K\_y(x) = \langle K\_y, K\_x\rangle$ defined on $X\times X$ is called the reproducing kernel function of $\mathcal{H}$.


It is well known and easy to show that for any orthonormal basis $\{e\_m\}\_{m=1}^{\infty}$ for $\mathcal{H}$, we have the formula
$$K(x,y) = \sum\_{m=1}^{\infty}e\_m(x)\overline{e\_m(y)},\tag{Eqn 1}$$
where the convergence is pointwise on $X\times X$.


My question concerns the converse of the above statement.


**Question:** if $\{g\_m\}\_{m=1}^{\infty}$ is a sequence of functions in $\mathcal{H}$ such that $$K(x,y) = \sum\_{m=1}^{\infty}g\_m(x)\overline{g\_m(y)}\tag{Eqn 2}$$ for all $x,y\in X$. Is the sequence $\{g\_m\}\_{m=1}^{\infty}$ an orthonormal basis for $\mathcal{H}$?


The answer to this question is clearly negative since equation (Eqn 1) can be re-written as
$$K(x,y) = \frac{e\_1(x)}{\sqrt{2}}\overline{\frac{e\_1(y)}{\sqrt{2}}}+\frac{e\_1(x)}{\sqrt{2}}\overline{\frac{e\_1(y)}{\sqrt{2}}}+\sum\_{m=2}^{\infty}e\_m(x)\overline{e\_m(y)}$$
and clearly $\{e\_1/\sqrt{2}, e\_1/\sqrt{2}, e\_2, \ldots\}$ is **not** an orthonormal basis for $\mathcal{H}$. So the following additional condition should be added: the sequence $\{g\_m\}\_{m=1}^{\infty}$ is **linearly independent**.


The following proof suggests that the answer is affirmative. (For those who are familiar with the proof of the Moore-Aronszajn's Theorem in the theory of RKHS, the proof here looks similar.)
Assume that we have (Eqn 2) and the sequence $\{g\_m\}\_{m=1}^{\infty}$ is linearly independent. Let $\mathcal M$ be the linear space spanned by the functions $\{g\_m\}\_{m=1}^{\infty}$. Define a sesquilinear form on $\mathcal M$ as
\begin{align\*}
\left\langle\sum\_{\text{finite sum}}a\_jg\_j, \sum\_{\text{finite sum}}b\_kg\_k\right\rangle\_{\mathcal M} = \sum\_{\text{finite sum}} a\_j\overline{b}\_j.
\end{align\*}
Since $\{g\_m\}\_{m=1}^{\infty}$ is a linearly independent set, the above definition is **well-defined**. Note that $\{g\_m\}\_{m=1}^{\infty}$ is an orthonormal set in $\langle,\rangle\_{\mathcal M}$. For any $f\in\mathcal M$ and $x\in X$, we have
\begin{align\*}
f(x) = \sum\_{\text{finite sum}}\langle f,g\_m\rangle\_{\mathcal M}\,g\_m(x).
\end{align\*}
Cauchy-Schwarz's inequality gives
\begin{align\*}
|f(x)| & \leq \Big(\sum\_{\text{finite sum}}|\langle f,g\_m\rangle\_{\mathcal M}|^2\Big)^{1/2}\Big(\sum\_{\text{finite sum}}|g\_m(x)|^2\Big)^{1/2} \leq \|f\|\_{\mathcal M}\sqrt{K(x,x)}.
\end{align\*}
Let $\widetilde{\mathcal M}$ be the Hilbert space completion of $\mathcal M$. The standard argument shows that $\widetilde{\mathcal M}$ is a RKHS of functions on $X$. What is the kernel of $\widetilde{\mathcal M}$? Since $\{g\_m\}\_{m=1}^{\infty}$ is an orthonormal set and its span is dense in $\widetilde{\mathcal M}$, it is an orthonormal basis for $\widetilde{\mathcal M}$. The kernel of $\widetilde{\mathcal M}$ then can be computed as $$\sum\_{m=1}^{\infty}g\_m(x)\bar{g}\_m(y),$$ which is the same as $K(x,y)$. Therefore, $\widetilde{\mathcal M}$ is the same as $\mathcal H$ (they consist of the same functions and the inner products on the two spaces are equal). Consequently, $\{g\_m\}\_{m=1}^{\infty}$ is an orthonormal basis for $\mathcal{H}$. This completes the proof.


**Counterexample:** On the other hand, there are counterexamples that provide a negative answer to the question in the infinite dimensional case. 


What part of the above proof is incorrect? I have checked but could not figure out what went wrong.


Thank you.



Orthonormal bases on Reproducing Kernel Hilbert Spaces
The error is in this line:



> 
> The standard argument shows that $\widetilde{\mathcal{M}}$ is an RKHS of functions on $X$.
> 
> 
> 


In fact, this is not generally true. The completion $\widetilde{\mathcal{M}}$ may not be naturally identified with a space of functions on $X$.


The "obvious" way that one would try to prove this is as follows. Consider an element $\phi \in \widetilde{\mathcal{M}}$. Since $\mathcal{M}$ is dense in its completion, there is a sequence $\{f\_n\} \subset \mathcal{M}$ such that $f\_n \to \phi$ in $\widetilde{\mathcal{M}}$-norm, which is an extension of the $\mathcal{M}$-norm. In particular, $\{f\_n\}$ is $\mathcal{M}$-norm Cauchy. Because of your inequality $|f(x)| \le \|f\|\_{\mathcal{M}} \sqrt{K(x,x)}$ (\*), we have that $\{f\_n(x)\}$ is Cauchy in $\mathbb{R}$ for each $x$. So $\{f\_n(x)\}$ converges to a number which we may call $f\_\phi(x)$; that is, $f\_n \to f\_\phi$ pointwise. It is also easy to show that $f\_{\phi}$ does not depend on the choice of sequence $f\_n \to \phi$, so the linear map $\phi \mapsto f\_\phi$ of $\widetilde{\mathcal{M}}$ into $\mathbb{R}^X$ is well defined. Let's call this map $T$.


So the "obvious" thing to do is to look at the image of $T$, which is of course a function space, and make it a Hilbert space by pushing forward the $\widetilde{\mathcal{M}}$-norm. If this works, then the same inequality (\*) will show that the evaluation map is continuous in this norm, and we have ourselves an RKHS.


The problem is that $T$ might fail to be injective. In other words, we could have a nonzero $\phi$ for which $f\_\phi = 0$. In that case, pushing forward the norm will not work; it will not be well defined on the image of $T$.


This is exactly what happens in the counterexample you discuss in the comments. As you suggest, consider $\ell^2$ as an RKHS on $X = \mathbb{N} = \{1,2,\dots\}$, with its usual orthonormal basis $\{e\_n\}$, and let $v = \sum\_n \frac{1}{n} e\_n$. Let $H = \{v\}^\perp$ with the same $\ell^2$ inner product; being a closed subspace of $\ell^2$, $H$ is an RKHS.


Let $P : \ell^2 \to H$ be the orthogonal projection and let $g\_n = P e\_n$. The $g\_n$ are linearly independent; if $0 = a\_1 g\_1 + \dots + a\_n g\_n = P(a\_1 e\_1 + \dots + a\_n e\_n)$ then $a\_1 e\_1 + \dots + a\_n e\_n$ is a scalar multiple of $v$, which is only possible if all $a\_i$ are 0.


Following your construction, let $\mathcal{M} \subset H$ be the linear span of $\{g\_n\}$ and let $\langle\cdot, \cdot\rangle\_{\mathcal{M}}$ be the inner product on $\mathcal{M}$ which makes the $\{g\_n\}$ orthonormal.


Set $h\_m = \sum\_{n=1}^m \frac{1}{n} g\_n$. Clearly the sequence $\{h\_m\}$ is Cauchy in $\mathcal{M}$, so it converges in the completion $\widetilde{\mathcal{M}}$ to some $\phi$. It is also clear that $\|\phi\|\_\widetilde{\mathcal{M}}^2 = \frac{\pi^2}{6}$ so in particular $\phi \ne 0$.


But on the other hand, we have $h\_m = \sum\_{n=1}^m \frac{1}{n} P e\_n = P\left(\sum\_{n=1}^m \frac{1}{n} e\_n\right)$. By continuity of $P$, we have $h\_m \to Pv = 0$ in $\ell^2$, and thus also pointwise. So $f\_\phi = 0$.




---


This is basically an example of a somewhat paradoxical fact that's bitten me before. Let $X,Y$ be Banach spaces, and suppose $E \subset X$ is dense. It's well known that every bounded operator $T : E \to Y$ has a unique bounded extension $\tilde{T} : X \to Y$. But it's possible that $T$ is injective while $\tilde{T}$ is not. Indeed, we've just constructed an example, by letting $E = \mathcal{M}$, $X = \widetilde{\mathcal{M}}$, $Y = \ell^2$, and $T : E \to Y$ the inclusion map.


