Let $\mathcal{X}$ be some nice space, let $\Phi$ be some ordered space, and let $K :\mathcal{X} \times \mathcal{X} \times \Phi \to \mathbf{R}$ be a positive-semidefinite kernel indexed by a hyperparameter $\phi$, i.e. for any $\phi \in \Phi$, the mapping $K(\cdot, \cdot, \phi) :\mathcal{X} \times \mathcal{X} \to \mathbf{R}$ is a positive-semidefinite kernel in the sense of Mercer.


Now, fix a finite subset $\mathcal{X}\_N = \{ x\_1 , \ldots, x\_N \} \subseteq \mathcal{X}$ of size $N$, and consider the mapping $\mathbf{K}$ which takes a hyperparameter $\phi$ and maps it to the corresponding $N \times N$ kernel matrix, i.e.


\begin{align}
\mathbf{K} ( \phi) \in \mathrm{PSD}\_{N \times N}, \quad \\
\mathbf{K} ( \phi)\_{i, j} = K (x\_i, x\_j, \phi).
\end{align}


I'm interested in examples of parametric kernels $K$ for which this mapping is order-preserving (for any set $\mathcal{X}\_N$), in the sense that when $\phi\_1 \prec \phi\_2$ with respect to the ordering on $\Phi$, it follows that $\mathbf{K} ( \phi\_1 ) \prec \mathbf{K} ( \phi\_2)$ in the semidefinite ordering on matrices.


Some simple examples of which I'm already aware:


* Let $K\_1$, $K\_2$ be two PSD kernels, let $\Phi = \mathbf{R}\_+$, and let $K(x, y, \phi) = K\_1 (x, y) + \phi \cdot K\_2 (x, y)$.
* Let $\{ K\_i : i \in \mathbf{N} \}$ be a countable collection of PSD kernels, let $\Phi = \mathbf{N}$, and let $K(x, y, \phi) = \sum\_{i \leqslant \phi} K\_i (x, y)$.


I'd be particularly interested in cases where such an ordering holds as e.g. the length scale of a kernel varies, or similar.



Monotonicity of kernel matrices with respect to hyperparameters
This example may be a little bit ridiculous, but suppose we take $\mathcal{X}=\mathbb{R}$ and let $\Phi$ be any parametric subset of the set of PSD kernels itself. We define
$$ \mathbf{K}(\phi)\_{i,j} = \sum\_{i,j}\phi(x\_i, x\_j) $$


We will say that $\phi\_1 \preceq \phi\_2$ iff for any $N$, $x\_1,...,x\_N$, and $c\_1,...,c\_N\geq 0$, we have
$$\sum\_{i,j} c\_i c\_j (\phi\_2(x\_i,x\_j)-\phi\_1(x\_i,x\_j)) \geq 0$$
This gives a partial order on $\Phi$ that (by construction) respects the OP's property of interest.


Possibly the OP is less interested in the preceding construction and more interested in concrete parametric examples. If so, geodesics of the cone of PSD matrices provide some interesting examples. For example, take $K\_1, K\_2$ as two PSD kernels, and let $\Phi=[0,1]$. For a fixed $N$, let $A$ and $B$ be the two corresponding PSD matrices. Then define
$$K(x,y,\phi)=A^{1/2}\exp(\phi \log(A^{-1/2}BA^{-1/2}))A^{1/2}$$
More inspiration along that line might be found at Bonnabel and Sepulchre's ["Riemannian Metric and Geometric Mean for
Positive Semidefinite Matrices of Fixed Rank"](https://arxiv.org/pdf/0807.4462.pdf)


