I'm a CS student and I'm trying to learn RKHS theory to understand the passages made in [this paper](http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.74.6694) .
Among the bibliography I'm using there are ["On the mathematical fundamentals of learning"](http://www.ams.org/journals/bull/2002-39-01/S0273-0979-01-00923-5/home.html) and ["Learning with Kernels"](http://mitpress.mit.edu/0262194759).
I think I've got a good grasp of the relevant theory, but there's still something that bugs me.


As far as I understood a RKHS is a Hilbert space $H\subseteq \mathbb{C}^X$, where $X$ is a generic set of objects, with inner product $\langle f,g\rangle=\int\_X\int\_X \alpha(x')\beta(x)k(x,x')dxdx'$ with $\alpha,\beta \in \mathbb{C}^X$, $f=\int\_X\alpha(x')k(x,x')dx'$ and $g=\int\_X\beta(x')k(x,x')dx'$ such that $H=\overline{\mathrm{span}\{k\_x|x\in X\}}$ and $\langle f,k\_x\rangle=f(x)$ where $k\_x=k(.,x)\;\forall x\in X$ and $k$ is the (Mercer) kernel for $H$. Let's call this "*continuos-whole*" definition.


However, given $D=\{x\_1,x\_2,...x\_n\}\subset X$ a subspace $H\_D\subset H$ could be defined by restricting the definition above to $H\_D=\overline{\mathrm{span}\{k\_{x\_i}|x\_i\in D\}}=\{f\in \mathbb{C}^X|f=\overset{i=s}{\underset{i=1}{\sum}} \alpha\_ik(x,x\_i)\quad \alpha\_i\in \mathbb{C} \;, r\in \mathbb{N}\}$ and thus define $\langle f,g\rangle\_{H\_D}=\overset{i=s}{\sum\_{i=1}}\overset{j=s}{\sum\_{j=1}}\alpha\_i\beta\_jk(x\_i,x\_j)$. Let this be the "*discrete-finite*" definition


Assuming this is correct, the references tend to define $H$ as both being spanned by $\{k\_x|x \in X\}$ and having inner product $\langle .,.\rangle\_{H\_D}$, and that would be possible if and only if $\overline{\mathrm{span}\{k\_x|x\in X\}}=\overline{\mathrm{span}\{k\_{x\_i}|x\_i\in D\}}$ which seems bogus to me.


Then, regarding the article, there's a passage of which I'm unsure. Let $H$ be a RKHS, i.e. a Hilbert space s.t. every point-evaluation functional is bounded, which would entail a "whole-continuous" definition. Then $H$ can be orthogonal decomposed in $H=H\_D\oplus H\_D^\bot$ where $H\_D^\bot \bot H\_D \rightarrow \langle f,g\rangle=0 \forall f\in H\_D \;\wedge\; g\in H\_D^\bot$. The paper says that $H\_D^\bot=\{g|g(x\_i)=0\forall x\_i\in D\}$. Starting from the definition of orthogonality, I carried out the following proof:
$\forall f \in H\_D \;\wedge\; g \in H\_D^\top$ $\langle f,g \rangle=\int\_X\int\_X \alpha(x')\beta(x)k(x,x')dxdx'=\int\_X \beta(x)f(x)dx$ $=\sum\_{i \in 1..s}\alpha\_i\int\_X \beta(x)k(x,x\_i)dx=\sum\_{i \in 1..s} \alpha\_i g(x\_i)=0 \forall \alpha\_i \in \mathbb{C}\rightarrow g(x\_i)=0 \forall i\in 1..s$, for which I applied the reproducing property and the definiton of $g$ in this order. is this correct? 


EDIT:I know the definition via the Riesz theorem, however I'm referring to the definiton I found in "Learning with kernels" (pg. 36):


"Definition 2.9 (Reproducing Kernel Hilbert Space) Let $X$ be a nonempty set and by $H$ a Hilbert space of functions $f :X\rightarrow\mathbb{R}$ . Then $H$ is called
a reproducing kernel Hilbert space endowed with the dot product $\langle .,. \rangle$ (and the norm $||f|| : \sqrt \langle f ,f\rangle $ ) if there exists afunction $k :X\times X \rightarrow \mathbb{R}$ with the following properties:


1. $k$ has the reproducing property
$\langle f, k(x,. )\rangle= f(x)$ for all $f\in H$ ;
2. $k$ spans $H$ , i.e. $H=\overline{\mathrm{span}\{k(x,.)| x \in X\}}$ where $\overline{X}$ denotes the completion of the set $X$"


, and, to my understanding, it is based on the theorems characterising RKHS on Mercer kernels (see pg. 35 of "On the mathematical foundations of Learning").
The approach they take, as far as I've understood, is inside-out: they start from a Lp space $H=\overline{\mathrm{span}\{k(x,.)| x \in X\}}$ with $k$ being a (Mercer) kernel, for which define a suitable inner product to get a Hilbert space that is also a RKHS. They defined a Mercer kernel to be a function $X^2\rightarrow \mathbb{R}$ that gives rise to a definite positive Gram matrix $K\_{ij}=k(x\_i,x\_j)$ for every $D=\{x\_1,x\_2..x\_n\}\subset X$.



Doubts on Reproducing Kernel Hilbert Spaces and orthogonal decomposition
To answer your second question:


$H\_D = \operatorname{span}(k\_x : x \in D) $ is a finite dimensional (and hence closed) subspace of $H$. A function $f \in H$ is orthogonal to each $k\_x, x \in D$ precisely when $\langle f , k\_x \rangle = f(x) = 0$. In other words, $H\_D^\perp = ( f \in H : f(x\_i) = 0, x\_i \in D)$ and of course $H = H\_D \oplus H\_D^\perp$.


