\section{Introduction}

We suppose that we have observed data $(Y_1,X_1),\ldots, (Y_n,X_n)$ from a strictly stationary process $(Y_k,X_k)_{k\in \mathbb{Z}}$  that are assumed to follow  a general  functional linear regression model of the form
\begin{equation}\label{mod}
Y_k=\varrho(X_k)+\varepsilon_k.
\end{equation}
Here $Y_k=(Y_k(t)\colon t\in [0,1])$ is a curve in a normed function space $H_2$, the covariates $X_k$ take values in a normed space $H_1$ and are distributed so that $X_k$ is independent of the model error $\varepsilon_k$, and $\varrho$ is a linear operator mapping $H_1$ to $H_2$. For example, $X_k$ might be a single curve living in the same space as the response, in which case \eqref{mod} describes simple linear function-on-function regression. This setting also includes functional autoregressive models \citep{bosq:2000} when $X_k = Y_{k-1}$. Generally though, $X_k$ might be comprised of several curves, a mixture of curves and scalar covariates, etc., and more detailed assumptions on the nature of the space $H_2$ will follow.

Suppose $(Y,X)$ is a generic pair following \eqref{mod}. The goal of this paper is to introduce and study methods to consistently estimate the conditional distribution of $Y$ given $X$, $P(Y\in A|X)$, for some specific sets of interest $A\subset H_2$. By choosing appropriate sets $A$, one may make inference on a wide range of interesting properties of $Y$:
\begin{enumerate}[topsep=5pt,itemsep=2pt]
\item  Often we are interested in some transformation $T$ of the response, and then might consider sets of the form
$A=\{y\colon T(y)\in B\}$. For instance, %
when $Ty=\lambda(\{t : y(t) \in B\})$, with $\lambda$ denoting standard Lebesgue measure on $[0,1]$, $A$ contains curves  that occupy a range of interest for a certain amount of time. More generally, when $T(y)$ is a scalar, then we are often interested in the conditional distribution function $$F(z|X)=P(Y\in T^{-1}(-\infty,z]|X).$$

\item Similarly, when $Z=T(Y)$ is again a scalar, for $p\in (0,1)$, we may wish to estimate the conditional quantile function $q_p(Z|X):=\inf\{z\in \mathbb{R}\colon F(z|X)\geq p\}$. In financial applications and when $p$ is close to zero or one, then estimating $q_p(Z|X)$ is related to  Value-at-Risk (VaR) estimation. See \cite{kato:2012} and \cite{sang:2020}.
\item We might wish to choose $A$ such that it yields a prediction set for $Y$, so that $P(Y\in A_p|X)=p$ for a given $p\in (0,1)$. Estimating $P(Y\in A_p|X)$ can be used to appropriately calibrate $A_p$. See \cite{goldsmith:2013}, \cite{choi:2016},  \cite{liebl:2019}, \cite{hyndman:shang:2009}, and \cite{paparoditis:2020} for a review of methods for constructing prediction sets for functional responses and parameters.
\end{enumerate}

At this point, when referring to examples (a) and (b), an important remark is necessary. In the case where $Z=T(Y)$ is scalar, it might appear more natural to directly employ some scalar-on-function regression with response variable $Z$. \emph{However, one of the main strengths of the approach we pursue and which is a clear distinction to competitive methods, is that we first model the entire response curve and then extract the feature of interest.
This has the advantage that we can harness the full information contained in the functional responses when estimating the conditional distribution of $Z$.}

Aside from interest in the general problem, this work was primarily motivated by the statistical challenge of forecasting aspects of response curves $Y_k$ describing daily electricity prices. The specific data that we consider consists of hourly electricity prices, demand, and wind energy production in Spain over the period from 2014 to 2019, which includes observations from 2191 days (the data are available at \texttt{www.esios.ree.es}). We project the hourly data onto a basis of 18 twice differentiable B-splines to construct daily  price, demand, and wind energy production curves, as illustrated in Figure~\ref{fig:spanishdata}. The price of electricity naturally fluctuates based on supply and demand, and exhibits daily, weekly, and yearly seasonality. The rather predictable variation in demand does not influence
the price as much as surges in wind energy production, especially if they occur on days with weak demand. Letting $Y_k$ denote the price curves and $X_k$ the vector of the demand and wind curves, both adjusted for yearly seasonality and trends, we then model $Y_k$ using an FAR(7) model with exogenous variables
\begin{equation}\label{e:spainfarx}
 Y_k = \sum_{i=1}^7 \Psi_i  Y_{k-i} + \varrho X_k + \varepsilon_k,
\end{equation}
where $\Psi_1,...,\Psi_7$ denote autoregressive operators; see  \cite{gonzalez:munoz:perez:2018}. The details of this are explained in Section~\ref{s:realdata:sim}, but for now it suffices to acknowledge that this is a regression model of the form \eqref{mod}. For such electricity price curves, their likelihood of falling within sets of the following type are of particular interest:

\begin{Example}[Level sets]\label{E:levelsets}
Let
\[
A_{\alpha,z} = \big\{ y\in H_2\colon \lambda(t\colon y(t)>\alpha)\leq z \big\}
\]
for some $z\in [0,1]$ and $\alpha\in \mathbb{R}$. $A_{\alpha,z}$ contains curves that stay a limited amount of time $z$ above a threshold $\alpha$.
\end{Example}

Forecasting whether price or demand curves will spend prolonged periods of time above certain levels is useful in anticipating volatility in continuous intraday electricity markets, and planning for peak loads \citep{vilar:2012}. This falls within the scope of the general problem we consider.

\begin{figure}[t]
\includegraphics[width=\textwidth, trim=0 18 0 2mm,clip]{Spain-price-2014-11-z-0point5.pdf}
\includegraphics[width=\textwidth, trim=0 29 0 1mm,clip]{Spain-demand-2014-11.pdf}
\includegraphics[width=\textwidth, trim=0 18 0 1mm,clip]{Spain-wind-2014-11.pdf}
\includegraphics[width=\textwidth, trim=3 5 0 0mm,clip]{Spain-price-prob-2014-11.pdf}
\caption{Spanish electricity data on price, demand, and wind energy production during two weeks in November 2014. Price curves are colored blue or red according to whether or not
they lie in the level set $\{y \in H_2: \lambda(t: y(t) > 50) \leq 0.5 \}$.
The bar plot on the bottom shows
the estimated conditional probability for $Y_k$ to lie in this set,
with the decision threshold $1/2$, indicated by a dotted line, whether the event occurred is indicated by black dots.}
\label{fig:spanishdata}
\end{figure}

The literature in functional data analysis on regression models of the form \eqref{mod} is vast, although the most frequent problems considered regarding consistent estimation in model \eqref{mod} are (i) how to find a consistent estimator $\hat\varrho_n$  of $\varrho$, and (ii) how to forecast consistently, i.e.\ to guarantee that $\hat\varrho_n(X)-\varrho(X)\to 0$ suitably in probability. Moreover, the majority of the literature on the topic of  function-on-function linear regression concentrates on the  setting when $H_1=H_2=L^2[0,1]$, the separable Hilbert space of square integrable functions on $[0,1]$, equipped with its standard inner product and norm.  \cite{ramsay:silverman:2006} for example proposes a double truncation scheme based on functional principal component analysis to estimate $\varrho$ in this setting, and \cite{mas:2007}, \cite{imaizumi:kato:2018} derive a convergence rate for
$\| \hat\varrho_n - \varrho \|_\mathcal{S}$ in a ``single-truncation'' estimation scheme based on an increasing (in the sample size) number of principal components, where here $\| \cdot \|_\mathcal{S}$ denotes the Hilbert--Schmidt norm. Similar consistency results for the resulting forecasts in functional linear regression can be found in \cite{crambes:mas:2013}, and under general stationarity conditions and in the FAR setting in \cite{hormann:kidzinski:2015} and \cite{aue:2015}. Estimating the operator $\varrho$ can be viewed as a special case of estimating the conditional mean $E(Y|X)$, and this general problem has also been extensively considered; see \cite{Chiou04functionalresponse}, \cite{ferraty:2012:fonfregressionbootstrap}, and \cite{wang:chiou:muller:review}.

The problem of estimating the conditional distribution of $Y$ given $X$ has been comparatively far less studied. Numerous methods have been proposed to estimate the conditional distribution of a scalar response $Y$ with a functional covariate $X$, including  \cite{chen:muller:2012:funquantile}, \cite{kato:2012}, \cite{yao:suechee:wang}, \cite{wang:chiou:muller:review}, and \cite{sang:2020}, who propose estimators based on quantile regression, and \cite{ferraty:vieu:2006}, who propose Nadaraya--Watson style kernel-smoothed estimators. Estimating the conditional distribution of $Y$ when $Y$ and $X$ take values in a general function space is largely unexplored to our knowledge, even in the context of model \eqref{mod}. \cite{fernandez:guillas:manteiga:2005:bootstrap} and \cite{paparoditis:2020} develop bootstrap procedures based on functional principal component analysis to produce prediction sets in the context of forecasting with Hilbertian FAR models, which can be viewed as a special case of this problem. For functional data taking values in $L^2[0,1]$, \cite{chen:muller:2016} and \cite{fan:muller:2021} develop methods for estimating the conditional distribution of $Y$ given $X$ assuming $X$ and $Y$ are jointly Gaussian, and that the conditional distribution of the response has sample paths satisfying natural differentiability conditions.

A technical problem that is encountered in consistently estimating the conditional probability $P(Y\in A | X)$ is that one could at most expect consistent estimation for continuity sets of the distribution of the response, i.e. sets $A$ for which $P(Y\in \partial A )=0$. This property evidently depends strongly on the choice of the space $H_2$, as well as the norm that it is equipped with. For many interesting examples, the metric on the space $L^2[0,1]$ is too weak to allow for meaningful continuity sets $A$. An illustrative example is simple prediction band sets of the form $A=\{y\colon \lambda( t : a(t)<y(t)<b(t))=1\}$, where $a$ and $b$ are continuous functions on $[0,1]$, in which case $\partial A=A$ when $A$ is viewed as a subset of $L^2[0,1]$. More appropriate spaces to handle many interesting examples in functional data analysis involving path properties of the response, such as level sets, are the spaces $C[0,1]$, the space of continuous function on $[0,1]$ equipped with the supremum norm, or the Sobolev spaces equipped with their canonical norms; see \cite{brezis:2010}. For these latter spaces, the problem of estimating, and consistently forecasting with, $\varrho$ has been only lightly studied to date, and in specialized settings; see \cite{pumo:1999}, \cite{RUIZMEDINA:2019:banachFAR}, and \cite{bosq:2000} in the context of FAR estimation. Further, the problem of consistently estimating the conditional distribution $P(Y\in A | X)$ in these settings has not been studied, to our knowledge. We also refer the reader to \cite{dette:kokot:aue:2020} for a review of functional data analysis methods in $C[0,1]$.

In this paper, we propose natural procedures to estimate $P(Y\in A |X)$, in which we first estimate $\varrho$ with a suitably consistent estimator $\hat{\varrho}_n$, and then either (i) resample the estimated residuals $\hat{\varepsilon}_{k,n} = Y_k  - \hat{\varrho}_n(X_k)$ to estimate $P(Y\in A |X)$ with the empirical distribution of $\hat{\varrho}_n(X) + \hat{\varepsilon}_{k,n}$, or (ii) assuming Gaussianity of the model errors $\varepsilon_k$, we estimate $P(Y\in A |X)$ using simulation by modelling $Y$ conditioned on $X$ as a Gaussian process with mean $\hat{\varrho}_n(X)$, and covariance estimated from the residual sequence $\hat{\varepsilon}_{k,n}$. We establish general conditions on the estimator $\hat{\varrho}_n$ in both the settings when $H_2$ is a separable Hilbert space, and when $H_2$ is $C[0,1]$, such that these procedures will lead to consistent estimation of $P(Y\in A |X)$.
Subsequent to this, we define an estimator $\hat{\varrho}_n$ that we show satisfies these conditions under regularity assumptions on the operator $\varrho$, and the process $(Y_k,X_k)_{k\in \mathbb{Z}}$, which allow for serial dependence of both the response and covariates. In the space $H_2=C[0,1]$, we introduce a number of examples of sets $A$ of potential interest, compute their boundaries, and establish under what conditions $P(Y\in \partial A )=0$, which can be non-trivial even when $Y$ is a Gaussian process. In several simulation studies and data analyses, the proposed methods generally outperformed competing methods in which $P(Y\in A |X)$ is estimated using functional logistic regression, functional Nadaraya--Watson estimation, or functional quantile regression. Another advantage, which derives from the simple form of our estimators for $P(Y\in A |X)$, is that they satisfy basic properties of a probability measure, e.g.\ they are monotone in $A$. While this may seem like an obvious requirement, it is not necessarily fulfilled by some competing approaches.

The rest of the paper is organized as follows. In Section~\ref{sec-1}, we formally introduce the methods described above to estimate $P(Y\in A |X)$, and present results on their consistency, including results on uniform consistency over monotone families of sets $A$ that are relevant in constructing prediction sets with a specified coverage and quantile function estimates. These results depend on the properties of the estimator $\hat{\varrho}_n$, and we define an estimator based on functional principal component analysis and single truncation scheme, and establish that it leads to consistent estimation of $P(Y\in A |X)$ when $H_2$ is a separable Hilbert space and when $H_2=C[0,1]$ in Section~\ref{s:theoretical}. Section~\ref{s:partialA} presents numerous examples of sets $A$ of interest, and a discussion of their boundaries in $C[0,1]$. A number of competing methods are introduced in Section~\ref{s:realdata:sim}, and these are compared and studied with the proposed methods in several simulations studies and real data illustrations. The proofs of all technical results follow these main sections.

\section{Estimation procedures and consistency results}\label{sec-1}

We let $\int = \int_{0}^{1}$, and for $f,g \in L^2[0,1]$, we use the notation $\langle f , g \rangle = \int f(t)g(t)dt$ to denote the standard inner product on $L^2[0,1]$, with induced norm $\|\cdot\|_{L^2}^2  = \langle \cdot , \cdot \rangle$.
In order to consider path properties of functions in $H_2$, we consider the space $C[0,1]$ equipped with the supremum norm $\|f\|_\infty = \sup_{t\in [0,1]} \big| f(t) \big|$. While $L^2[0,1]$ is a separable Hilbert space, $C[0,1]$ is a Banach space, with their norms satisfying $\|\cdot\|_{L^2} \leq \|\cdot\|_{\infty}$. The space $C[0,1]$ may hence be naturally embedded in $L^2[0,1]$.
We use the tensor product notation $\otimes$ to denote the operator $a\otimes b(\cdot) = a\langle \cdot,b\rangle$ if $b$ is viewed as an element of a Hilbert space, and the kernel integral operator with kernel $a\otimes b (t,s) = a(t)b(s)$ if $b$ is viewed as an element of $C[0,1]$. We assume that the covariate space $H_1$ is a Hilbert space with norm
$\|\cdot\|_{H_1}$. In order to lighten the notation, and when it is clear from the context, we write $\|\cdot\|$ in place of a specific norm on either the space $H_1$ or $H_2$. In the context of Hilbert spaces,
we use $\|\cdot\|_\mathcal{S}$ and $\|\cdot\|_1$ to denote the Hilbert--Schmidt norm and the trace norms of operators, respectively.

We assume throughout that the covariates $X_k$ and the model errors $\varepsilon_k$ satisfy the following independence condition, which we do not explicitly state in the below results, but take as granted.

\begin{Assumption}\label{a:errorind} In model \eqref{mod}, $E\varepsilon_k = 0$, and $\varepsilon_k$  is independent from $(X_j)_{j\leq k}$ for all $k \in \mathbb{Z}$.
\end{Assumption}

In order to formally describe the methods we use to estimate $P(Y\in A|X)$, we assume that we may consistently estimate $\varrho$ with an estimator $\hat{\varrho}_n$ based on the sample  $(Y_1,X_1),...,(Y_n,X_n)$. Specific conditions on this estimator, and examples satisfying these conditions, will follow. The first method we describe is based on applying an i.i.d.\ bootstrap to the estimated residuals.

\medskip

\noindent\underline{ Algorithm~1, Residual Bootstrap (abbreviated {\bf boot}):}
\begin{enumerate}
\item Estimate $\varrho$ in \eqref{mod} with $\hat\varrho_n$.
\item Calculate the model residuals $\hat\varepsilon_{k,n} = Y_k-\hat\varrho_n(X_k)$.
\item Define the estimator of $P(Y\in A|X)$ as
\[
\hat{P}_n^\text{B}(Y\in A|X)=\frac{1}{n}\sum_{k=1}^n \mathds{1}\{\hat\varrho_n(X)+\hat\varepsilon_{k,n}\in A\}.
\]
\end{enumerate}

We show below that this estimator is consistent under quite mild conditions. The algorithm {\bf boot} can be applied without specific distributional assumptions on the errors.
If though the model errors $\varepsilon_k$ are thought to be Gaussian processes, then we may take this into account in estimating $P(Y\in A |X)$. As the mean of the model errors is zero by assumption, in this case their distribution is determined by their covariance $\Gamma = \var(\varepsilon_k)$, which is defined by
\[
\var(\varepsilon_k) = E\big[ \varepsilon_k \otimes \varepsilon_k \big].
\]
The above algorithm may then be adapted as follows:

\medskip

\noindent\underline{ Algorithm~2, Gaussian process simulation (abbreviated {\bf Gauss}):}
\begin{enumerate}
\item Estimate $\varrho$ in \eqref{mod} with $\hat\varrho_n$.
\item Calculate the model residuals $\hat\varepsilon_{k,n} = Y_k-\hat\varrho_n(X_k)$.
\item Estimate the empirical covariance operator
\begin{align}
\widehat\Gamma_{\varepsilon,n} = \frac{1}{n}\sum_{k=1}^n (\hat\varepsilon_{k,n}-\bar{\varepsilon}_{\cdot,n})\otimes(\hat\varepsilon_{k,n}-\bar{\varepsilon}_{\cdot,n}),
\label{e:gammahat}
\end{align}
where $\bar{\varepsilon}_{\cdot,n}=\frac{1}{n}\sum_{k=1}^n \hat\varepsilon_{k,n}$. Let $\hat{\nu}_1 \ge \hat{\nu}_2 \ge \cdots $ denote the ordered eigenvalues of $\widehat\Gamma_{\varepsilon,n}$, with corresponding eigenfunctions $(\hat{\psi}_j)_{j\ge 1}$ satisfying $\widehat\Gamma_{\varepsilon,n}(\hat{\psi}_j) = \hat{\nu}_j \hat{\psi}_j$, $\langle \hat{\psi}_j, \hat{\psi}_\ell \rangle = \mathds{1}\{j = \ell\}$.

\item  Let $\{Z_i, \; i\ge 1\}$ denote a sequence of i.i.d.\ standard normal random variables, independent of the sample $(Y_1,X_1),...,(Y_n,X_n)$, and define
\[
\varepsilon^{(n)} = \sum_{j=1}^{\infty}\hat{\nu}_j^{1/2}  Z_j \hat{\psi}_j.
\]
Note that conditionally on the sample, in particular on $\widehat\Gamma_{\varepsilon,n}$, $\varepsilon^{(n)}$ is a Gaussian process with mean zero and covariance operator $\widehat\Gamma_{\varepsilon,n}$. Define the estimator of $P(Y\in A|X)$ as
\begin{align}\label{eq:gprob}
\hat{P}^\text{G}_{n}(Y\in A|X) =P(\hat\varrho_n(X)+\varepsilon^{(n)}\in A|X).
\end{align}

The right hand side above can be approximated by Monte-Carlo simulation. To do so, generate an i.i.d.\ sample  conditionally on $\widehat\Gamma_{\varepsilon,n}$,  $(\varepsilon_k^{(n)})_{k\geq 1}$, distributed as $\varepsilon^{(n)}$, by simulating independent standard Gaussian sequences $\{Z_{i,k}, \; i\ge 1\}_{k\ge1}$  and setting
\[
\varepsilon^{(n)}_k = \sum_{j=1}^{\infty}\hat{\nu}_j^{1/2}  Z_{j,k} \hat{\psi}_j.
\]
The right hand side of \eqref{eq:gprob} can be estimated, for a large $M$, by
\[
\frac{1}{M}\sum_{k=1}^M \mathds{1}\{\hat\varrho_n(X)+\varepsilon_k^{(n)}\in A\}.
\]

\end{enumerate}

\begin{Remark} {\rm  The scaling $1/n$ in the definition of $\widehat\Gamma_{\varepsilon,n}$
does not take into account the degrees of freedom $T_n$ lost in the estimation of the regression operator $\varrho$. It has thus been advocated, for example in \cite{crambes:2016},  to instead divide by $n - T_n$, where $T_n$ is related to the dimension of the dimensionality reduction technique used in estimating $\hat{\varrho}_n$. If $ET_n = o(n)$, as is the case for most estimation approaches,
the resulting scaling difference is asymptotically negligible.
Some authors also propose splitting the sample
and estimating the regression operator and the noise covariance operator
on separate parts of the sample in order to reduce the bias of the estimator $\widehat\Gamma_{\varepsilon,n}$; see \cite{crambes:mas:2013}. }
\end{Remark}

\begin{Remark} {\rm
Algorithm {\bf Gauss} can be extended to other parametric distributions of the noise.
A notable example for this are infinite dimensional elliptic distributions, where
$\varepsilon_k = \Xi_k   \varepsilon^\prime_k$
with two independent random variables
$\varepsilon^\prime_k \in H_2$, which is Gaussian, and $\Xi_k \geq 0$ from a known univariate parametric distribution. The following investigation can easily be adapted to this setting. For details on elliptical distributions of functional data, we refer to \cite{boente:2014}. }
\end{Remark}

We now aim to establish consistency results for these algorithms. In order to keep the results as general as possible and allow for different estimators of $\varrho$, these results are stated in terms of the following consistency properties of $\hat{\varrho}_n$.

\begin{Assumption}\label{a:consist}
The estimator $\hat\varrho_n$ is such that
\begin{enumerate}[(a),topsep=0pt,itemsep=0pt]
\item \label{a:consistOut}
its out-of-sample prediction is consistent,
i.e.\ if $X \stackrel{d}{=} X_1$, and is independent from the sample, then
\[
\|\hat\varrho_n(X)-\varrho(X)\| \stackrel{P}{\to} 0,
\quad  n \to \infty.
\]
\item \label{a:consistIn}
its in-sample prediction is consistent,
i.e.\ let $K_n$ be independent from the sample and
uniformly distributed on $\{1,\dots,n\}$, then
\[
\|\hat\varrho_n(X_{K_n})-\varrho(X_{K_n})\| \stackrel{P}{\to} 0,
\quad  n \to \infty.
\]
\end{enumerate}
\end{Assumption}

In order to establish the consistency of the algorithm {\bf Gauss}, we additionally need conditions on the estimator of the covariance
operator of the model errors defined by \eqref{e:gammahat}. We state two conditions depending on whether $H_2$ is a separable Hilbert space, or $H_2 = C[0,1]$.

\begin{Assumption}\label{a:consist2}
 $H_2$ is a separable Hilbert space.
The estimator $\hat\varrho_n$ is such that
\[
E \Bigg\| \frac 1n \sum_{i=1}^n (\hat\varrho_n - \varrho) X_k \otimes X_k (\hat\varrho_n - \varrho)^* \Bigg\|_1 \to 0, \mbox{ as }
\quad n\to\infty.
\]

\end{Assumption}

\begin{Assumption}\label{a:consist2C}
~
\begin{enumerate}[(a),topsep=0pt,itemsep=0pt]
\item $H_2 = C[0,1]$. The estimator $\hat\varrho_n$ is such that
\[
\sup_{t,s\in[0,1]} \Bigg| \frac 1n \sum_{i=1}^n
(\hat \varrho_n - \varrho)(X_k)\otimes (\hat \varrho_n - \varrho)(X_k)(t,s)
\Bigg| \stackrel{P}{\to} 0,
\quad n \to \infty.
\]
\item The estimated variance of the model errors,
\[
V_n^2(t,s) = \var\big( \varepsilon^{(n)}(t) - \varepsilon^{(n)}(s) \big| \widehat\Gamma_{\varepsilon,n} \big)
 = \widehat\Gamma_{\varepsilon,n}(t,t) - 2 \widehat\Gamma_{\varepsilon,n}(t,s) + \widehat\Gamma_{\varepsilon,n}(s,s)
\]
satisfies the H\"{o}lder condition
\[
V_n^2(t,s) < M_V^2 \, |t-s|^{2\alpha}, \quad    t,s\in[0,1],
\]
for some $0<\alpha\leq 1$,
where $M_V$ is a positive random variable with
$E M_V < \infty$.
\end{enumerate}
\end{Assumption}

Assumption~\ref{a:consist2C} is a $C[0,1]$ analog of Assumption~\ref{a:consist2}, but with the addition of Assumption~\ref{a:consist2C}~(b) that implicitly demands a degree of  continuity of $\varrho(X_k)$ and the model errors $\varepsilon_k$. Under the above assumptions, we can now formulate our main consistency results.

\begin{Theorem}\label{t:main1}
Suppose that Assumption~\ref{a:consist} holds
and that $P(Y\in \partial A)=0$.
Then
$\hat P^\text{B}_n(Y\in A|X)\stackrel{P}{\to} P(Y\in A|X)$ as $n\to\infty$.
\end{Theorem}

\begin{Theorem}\label{t:main2}
Suppose that Assumption~\ref{a:consist}~\ref{a:consistOut} holds and
either Assumption~\ref{a:consist2} or Assumption~\ref{a:consist2C} holds.
Assume that
$(\varepsilon_k)_{k\geq 1}$ are i.i.d.\ Gaussian random variables in $H_2$,
and that $P(Y\in \partial A)=0$.
Then
$\hat P^\text{G}_n(Y\in A|X)\stackrel{P}{\to} P(Y\in A|X)$ as $n\to\infty$.
\end{Theorem}

Theorems~\ref{t:main1} and \ref{t:main2} show that consistent estimation of $P(Y\in A |X)$ can be achieved by both {\bf boot} and {\bf Gauss} when $Y$ takes values in either a separable Hilbert space, or $C[0,1]$, under natural consistency conditions on $\hat{\varrho}_n$, and when $A$ is a continuity set of the response $Y$. We note that these results can be readily extended to sets $A$ that, rather than being fixed, are dependent on the predictor $X$, as well as using the estimator $\hat{\varrho}_n$, so long as there is a certain degree of continuity in relating $\{Y\in A\}$ to $\hat{\varrho}_n(X)$. This is often of interest when constructing prediction sets for the response $Y$, as in the following examples in which it is natural to consider $H_2 = C[0,1]$.

\begin{Example}[Pointwise and uniform prediction sets]\label{example:pred} Suppose $a$ and $b$ are positive functions in $C[0,1]$. Given a covariate $X$, let, for $s\in [0,1]$,
\[
\hat{A}_{a,b}^{(n)}(s)= \big\{y\in C[0,1] \colon \hat{\varrho}_n(X)(s)- a(s) \le y(s) \le \hat{\varrho}_n(X)(s)+ b(s)  \big\} \;\; \text{(Point prediction sets)},
\]
and
\[
\hat{U}_{a,b}^{(n)}= \big\{y\in C[0,1] \colon \lambda(t : \hat{\varrho}_n(X)(t)- a(t) \le y(t) \le \hat{\varrho}_n(X)(t)+ b(t) )=1 \big\} \;\; \text{(Uniform prediction sets)}.
\]
These approximate the sets
\[
 A_{a,b}(s)= \big\{y\in C[0,1] \colon  \varrho(X)(s)- a(s) \le y(s) \le  \varrho(X)(s)+ b(s)  \big\},
\]
and
\[
U_{a,b}= \big\{y\in C[0,1] \colon \lambda(t : \varrho(X)(t)- a(t) \le y(t) \le  \varrho(X)(t)+ b(t) )=1 \big\} .
\]
\end{Example}

\begin{Corollary}\label{cor-predsets}
For some $s\in [0,1]$, let $\hat{A}_{a,b}^{(n)}(s)$, $\hat{U}_{a,b}^{(n)}$, $A_{a,b}(s)$, and $U_{a,b}$ be defined in Example~\ref{example:pred}. Suppose that  $P(Y \in \partial A_{a,b}(s))=0$. %
If Assumption~\ref{a:consist} holds, then
\begin{equation}
\label{e:randomsets}
\hat P^\text{B}_n(Y\in \hat{A}_{a,b}^{(n)}(s)|X)\stackrel{P}{\to} P(Y\in A_{a,b}(s)|X),\quad\text{as $n\to\infty$.}
\end{equation}
If Assumptions~\ref{a:consist}~\ref{a:consistOut} and \ref{a:consist2C} hold, then \eqref{e:randomsets} holds with $\hat P^\text{G}_n$ instead of $P^\text{B}_n$. Under $P(Y \in \partial U_{a,b})=0$, the analogue results hold with the sets $\hat{U}_{a,b}^{(n)}$ and $U_{a,b}$.
\end{Corollary}

\subsection{Uniform consistency over monotone families of sets}\label{s:uniform:mono}

For a potentially unbounded  interval $[a,b] \subset \overline{\mathbb{R}}$, we call a family $\mathcal{A}=\{A_\xi\colon \xi\in [a,b]\}$ of measurable subsets of $H_2$ monotone if the sets $A_\xi$ are increasing or decreasing in $\xi$. Suppose that $A_\xi$ is increasing, the decreasing case can be handled similarly, and that we are interested in finding
\[
\xi_p(X)=\inf\{\xi \in [a,b]\colon P(Y \in A_\xi | X ) \geq p\},\quad p\in (0,1).
\]
As an example where this problem is relevant, consider a scalar transformation of the response $Z=T(Y)$, and suppose we wish to estimate the conditional quantile of $Z$ given the covariate $X$
\begin{align*}
q_p(Z|X)&=\inf\{\xi \in [a,b]\colon P(Z\leq \xi | X ) \geq p\}=\inf\{\xi \in [a,b]\colon P\big(Y \in T^{-1}\big([a,\xi]\big) | X \big) \geq p\}.
\end{align*}
The sets $A_\xi:= T^{-1}\big([a,\xi]\big)$ evidently define a monotone family. Consistent scalar-on-function quantile regression can hence be cast as the problem of consistently estimating $\xi_p(X)$ from the sample, which can be done using $\hat{P}_n^\text{B}$ or $\hat{P}_n^\text{G}$. To this end we consider the estimator
\begin{align}
\hat \xi_p^\text{B} (X):= \inf \big\{ \xi \in [a,b]\colon\, \hat P^\text{B}_n(Y \in A_\xi | X ) \geq p \big\}.
\label{e:predictionxi}
\end{align}
We note that based on the definition  of $\hat{P}^\text{B}_n$, $p\mapsto \hat\xi_p^\text{B}(X)$ is a non-decreasing function in $p$. The same holds for $\hat \xi_p^\text{G}(X)$, which is defined using $\hat{P}_n^\text{G}$. While this observation is rather trivial, in  other approaches to scalar-on-function quantile regression one often has to take special
care in order to guarantee monotonicity of estimators of $q_p(Z|X)$, see e.g.\ \cite{kato:2012}.

The goal is now to show that  $\hat\xi_p^\text{B}(X)\stackrel{P}{\to}\xi_p(X)$ and $\hat\xi_p^\text{G}(X)\stackrel{P}{\to}\xi_p(X)$. In order to do so, we need the following
uniform convergence result for the estimated conditional probabilities.
\begin{Proposition}\label{p:uniformconv}
Let $\{ A_\xi\colon \xi \in [a,b] \}$
be a monotone family of sets such that $P(Y \in A_\xi | X)$ is a.s. continuous in $\xi$.
Suppose the estimator $\hat P_n(Y\in A_\xi|X)$ is non-decreasing, right-continuous, and satisfies
$\hat P_n(Y\in A_\xi|X)\stackrel{P}{\to}  P(Y \in A_\xi | X)$ for all $\xi \in [a,b]$.
Then
\[
\sup_{\xi \in [a,b]} \Big| \hat P_n(Y \in A_\xi | X) - P(Y \in A_\xi | X) \Big| \stackrel{P}{\to} 0,
\quad n \to \infty.
\]
\end{Proposition}
We note that  both $\hat P_n^\text{B}(Y\in A_\xi|X)$ and
$\hat P_n^\text{G}(Y\in A_\xi|X)$ satisfy the conditions of Proposition~\ref{p:uniformconv} under the conditions of Theorems~\ref{t:main1} and \ref{t:main2}.

\begin{Corollary}\label{cor:unif}
Define $\hat\xi_p(X)$ as in  \eqref{e:predictionxi} for a some general estimator $\hat P_n(Y \in A_\xi | X)$.
Under the assumptions of Proposition~\ref{p:uniformconv} with increasing sets $A_\xi$, we
have $P(Y\in A_{\hat\xi_p(X)}|X)\stackrel{P}{\to} p$. If $P(Y \in A_\xi | X)$ is strictly increasing in $\xi$,
then $\hat \xi_p(X) \stackrel{P}{\to} \xi_p(X)$.
\end{Corollary}

\section{Estimation of the regression operator}\label{s:theoretical}

In this section we aim to define an estimator $\hat{\varrho}_n$ that satisfies the consistency conditions detailed in Assumptions~\ref{a:consist}, \ref{a:consist2}, and \ref{a:consist2C}. In order to do so, we make the following assumptions on model \eqref{mod}.

\begin{Assumption}\label{a:hilbertsetting}
\begin{enumerate}[(a),topsep=0pt,itemsep=0pt]
\item $H_1$ is a separable Hilbert space.
\item The process $(X_k)_{k \in \mathbb{Z}}$ has mean zero, and is $L^4$-$m$-approximable
in  $H_1$ (see \cite{hormann:kokoszka:2010}).
\item The operator $\varrho\colon H_1\to H_2$ is a bounded linear operator.
\item The sequence $(\varepsilon_k)_{k \in \mathbb{Z}} $ is a mean zero, i.i.d.\ sequence in $H_2$, and satisfies $E\|\varepsilon_k\|^4<\infty$.
\end{enumerate}
\end{Assumption}

Assumption~\ref{a:hilbertsetting}~(b) supposes that $X_k$ is a (strongly) stationary and ergodic sequence with $E\|X_k\|_{H_1}^4<\infty$, and allows the $X_k$ to be weakly serially dependent in a certain sense. \cite{hormann:kokoszka:2010} show that many commonly studied stationary time series in function space, like FAR processes or functional analogs of GARCH processes, are $L^4$-$m$-approximable under suitable moment conditions.

The estimator that we consider is a truncated (functional) principal components based estimator. Let the empirical covariance operator of $X_k$, and the empirical cross-covariance operator between $Y_k$ and $X_k$, be denoted as
\[
\widehat C_{XX} = \frac 1 n \sum_{k=1}^n X_k \otimes X_k, \quad\text{and}\quad  \widehat C_{YX} = \frac 1 n \sum_{k=1}^n Y_k \otimes X_k.
\]

Letting $\langle \cdot , \cdot \rangle_{H_1}$ denote the inner product on $H_1$, we note that $\widehat C_{XX}$ defines a non-negative sequence of eigenvalues $\hat \lambda_i$, and eigenfunctions $\hat v_i$, satisfying $\widehat C_{XX}(\hat{v}_i) = \hat{\lambda}_i \hat{v}_i $, $\langle \hat{v}_i , \hat{v}_j \rangle_{H_1} = \mathds{1}\{i=j\}$. We then define
\begin{equation}\label{e:rhohat}
\hat \varrho_n(x) := \sum_{i=1}^{T_n} \frac{1}{\hat \lambda_i} \, \widehat C_{YX} \, \hat v_i \otimes \hat v_i (x),
\end{equation}

The estimator \eqref{e:rhohat} only truncates the covariance operator of $X$ in order to obtain a feasible approximation to $\widehat C_{XX}^{-1}$, yielding a so-called ``single-truncated'' estimator. The asymptotic properties of these estimated operators have e.g.\ been studied in \cite{mas:2007} and \cite{hormann:kidzinski:2015}. In order to select the truncation parameter $T_n$ in such a way that leads to asymptotic consistency of $\hat{\varrho}_n$, we use the following criterion:
\begin{equation}\label{e:chooseTn}
T_n = \max\big\{j \geq 1 \colon \hat \lambda_j \geq m_n^{-1} \big\},
\quad \text{with }\; m_n \to \infty.
\end{equation}
Here $m_n$ is a tuning parameter, tending to infinity at a rate specified in the results below.

We note that another standard way to select $T_n$ is to use the percentage of variance explained (PVE) approach, which entails taking
\[
T_n = \min\left\{ d : \frac{\sum_{j=1}^{d} \hat{\lambda}_j}{\sum_{j=1}^{\infty} \hat{\lambda}_j } \ge v \right\},
\]
where $v$ is a user specified percentage treated as a tuning parameter.  While the criterion in \eqref{e:chooseTn} is more transparent in terms of describing the asymptotic consistency of $\hat{\varrho}_n$, since it gives a direct description of the decay rate of the sequence of eigenvalues $\hat{\lambda}_j$, in applications the PVE criterion is prevailing, due to its ease of interpretation. By choosing the associated tuning parameters appropriately, the two criteria may be made comparable.

Now we present results which imply Assumptions~\ref{a:consist}--\ref{a:consist2C}, and hence the consistency of the estimators in the Algorithms {\bf boot} and {\bf Gauss}.
\begin{Proposition}\label{p:main1}
Suppose that $H_2$ is a separable Hilbert space, Assumption~\ref{a:hilbertsetting} holds and
we define $\hat\varrho_n$ as in \eqref{e:rhohat} with $m_n = o\big(\sqrt n\big)$.
Then Assumption~\ref{a:consist} holds.
\end{Proposition}

\begin{Proposition}\label{p:main2}
Suppose that $H_2$ is a separable Hilbert space,
Assumption~\ref{a:hilbertsetting} holds
and that the true regression operator
$\varrho$ is Hilbert--Schmidt. If $\hat\varrho_n$ is defined as in \eqref{e:rhohat} with $m_n = o\big(\sqrt n\big)$,
then Assumption~\ref{a:consist2} holds.
\end{Proposition}

In the case when $H_2 = C[0,1]$, we add the following assumption in addition to Assumption~\ref{a:hilbertsetting}, supposing a degree of smoothness to $\varrho(X)$ and $\varepsilon_k$:
\begin{Assumption}\label{a:continuoussetting} If $H_2 = C[0,1]$, and for some $0 < \alpha \leq 1$,
\begin{enumerate}[(a),topsep=0pt,itemsep=0pt]
\item The model errors $\varepsilon_k$ a.s.\ satisfy the
Hölder condition
\begin{align}
\big| \varepsilon_k(t) - \varepsilon_k(s) \big| &< M_k \, |t-s|^\alpha
\label{e:holdercondition}
\end{align}
where $M_k$  is a positive random variable independent from $X_k$, with $E M_k^2 < \infty$.
\item
For all $x \in H_1$,
the regression operator $\varrho$ satisfies
\[
\big| \varrho x(t) - \varrho x(s) \big| \leq M_\varrho \, \|x\| \, |t-s|^\alpha,
\]
where $M_\varrho$ is a finite constant.
\end{enumerate}
\end{Assumption}

Assumption~\ref{a:continuoussetting}~(a) is fulfilled by a wide range of stochastic processes,
most notably the Brownian motion and the fractional Brownian motion.
Since $\varrho$ is linear, Assumption~\ref{a:continuoussetting}~(b) is a natural formulation of
the Hölder condition for the conditional mean of the response.
In particular, this implies that $\varrho$ is a bounded, compact operator.

\begin{Remark}{\rm
Suppose $H_1=L^2[0,1]$, so that model \eqref{mod} describes function-on-function regression. A frequently employed class of operators $\varrho$ in this setting are  kernel integral  operators, defined by a continuous kernel $\rho \in C[0,1]^2$  as
\[
\varrho x(t) = \int \rho(t,u) x(u) du.
\]
If there exists an $a \in H_1$ such that almost everywhere
\[
\big| \rho(t,u) - \rho(s,u) \big| < a(u) \, |t-s|^\alpha,
\]
then one can easily verify that
\begin{align*}
\big| \varrho x(t) - \varrho x(s) \big|
 &\leq \|a\| \, \|x\| \, |t-s|^\alpha,
\end{align*}
and thus
Assumption~\ref{a:continuoussetting}~(b) is fulfilled.}
\end{Remark}

\begin{Proposition}\label{p:mainc1}
Suppose that Assumption~\ref{a:hilbertsetting} and
Assumption~\ref{a:continuoussetting}~(a) hold, and
we define $\hat\varrho_n$ as in \eqref{e:rhohat}
with $m_n = o\big( n^{\alpha/2} \big)$.
Then Assumption~\ref{a:consist} holds.
\end{Proposition}

\begin{Proposition}\label{p:mainc2}
Suppose that
Assumption~\ref{a:hilbertsetting} and
Assumption~\ref{a:continuoussetting} hold, and
we define $\hat\varrho_n$ as in \eqref{e:rhohat}
with $m_n = o\big( n^{\alpha/2} \big)$.
Then Assumption~\ref{a:consist2C} holds.
\end{Proposition}

The proofs of Propositions~\ref{p:main1}--\ref{p:mainc1} are relegated to Section~\ref{s:proofs},
while the proof of Proposition~\ref{p:mainc2} is given in Appendix~\ref{a:additionalproofs}.

We conclude this section with some technical discussion.
We begin by noting that the sequence $m_n$, which controls
how many principal components of $X_k$ are used in forming $\hat{\varrho}_n$, can be of asymptotically higher order if $H_2$ is a Hilbert space compared to the setting when $H_2=C[0,1]$, and $\alpha < 1$.
In the case where the Hölder exponent in Assumption~\ref{a:continuoussetting} is $\alpha=1$,
the responses $Y_k$ are Lipschitz continuous, which implies they are weakly differentiable. As a result one may then take $H_2=W$, a separable Hilbert space, leading back to the rate condition $m_n = o(\sqrt{n})$. The order $n^{\alpha/2}$ is sufficient but not sharp. In fact, for $\alpha < 1/2$, a different proof yields that $m_n = o\big( n^{1/(2+\alpha^{-1})} \big)$ also leads to consistency. In the case of the Brownian motion, $\alpha=1/2$ demands that $m_n=o\big(n^{1/4}\big)$.
This is still of higher order than that suggested to be used by \cite{hormann:kidzinski:2015}
for consistent estimation of the regression operator in Hilbert spaces.

Our second technical remark concerns the choice of $H_1$. Assumption~\ref{a:hilbertsetting}~(a) requires $H_1$ to be a Hilbert space. While typically this is not a restriction, some care needs to be taken in the case of an FAR model. Here, when we study continuous functions, we choose $H_2=C[0,1]$. While it is natural to assume that the covariate and response space coincide for an FAR (i.e.\ requiring $H_1=C[0,1]$, too), this is not necessarily the case. For example when we consider a kernel integral operator  $\varrho$ with a continuous kernel, then we may still use $H_1=L^2[0,1]$ and $H_2=C[0,1]$ using the natural embedding of $C[0,1]$ in $L^2[0,1]$.
Alternatively we can resort to the Sobolev space $H_1=W^{1,2}[0,1]$ of once (weakly) differentiable functions in $L^2[0,1]$ equipped with the norm $\|f\|_W = \|f\|_{L^2}  + \|f'\|_{L^2} $; see Chapter 8 of \cite{brezis:2010}. The space $W^{1,2}[0,1]$ is a separable Hilbert space, and because $\|\cdot\|_{\infty} \leq \|\cdot\|_{W}$,
the space $W^{1,2}[0,1]$ can be embedded in $C[0,1]$. This will allow for more general class of continuous operators (for example, including pointwise evaluations).
When viewed with the moment conditions on $\|X_k\|_{H_1}$ implicit to Assumption~\ref{a:hilbertsetting}~(b),
this can be done so long as the covariates $X_k$ are sufficiently smooth.

\section{Some further examples of events $A$}\label{s:partialA}

In addition to level sets and pointwise or uniform prediction sets mentioned in Examples~\ref{E:levelsets} and \ref{example:pred} above, in this section we list some specific examples of sets $A$ that are of interest for the data that we discuss, and which might be useful in other applications.

\begin{Example}[Contrast sets] For some $\gamma\in H_2$ and $a\in\mathbb{R}$ let
\[
A = \left\{ y\in H_2\colon \int_0^1 \gamma(t)y(t)dt>a \right \}.
\]
For example, when $\gamma\equiv 1$, then $A$ is the set of curves which are in average above level $a$.
If $\gamma(t)=2\mathds{1}\{t\leq 1/2\}-1$, or $\gamma(t)=1/2-t$ and $a=E\int_0^1y(t)dt$,
then the set $A$ can be identified as functions with decreasing trend.
\end{Example}

\begin{Example}[Extremal sets.]\label{E:extremalsets}
Let $H_2=C[0,1]$ and let $d\in\mathbb{R}$ and
\[
A = \left\{ y \in H_2\colon \max_{u\in[0,1]} y(u) > d \right\}.
\]
Then $A$ contains functions which will exceed a certain threshold $d$. Note that this is the compliment of a boundary set with bounds $\alpha = -\infty$ and $\beta = d$.
\end{Example}

\begin{Example}[Excursion sets.] Let $H_2=C[0,1]$, $d\in\mathbb{R}$ and $c\in (0,1)$. Set
\[
A = \left\{ y \in H_2\colon \exists \; 0 \leq a < b \leq 1
\text{ with }
b-a \geq c \text{ s.t. } \min_{u\in[a,b]} y(u) > d \right\}.
\]
Then $A$ are the functions which uninterruptedly stay strictly above a certain threshold
for a certain amount of time.
\end{Example}

A crucial condition in  Theorems~\ref{t:main1} and \ref{t:main2}  is that $P(Y\in \partial A)=0$. Below we discuss some examples for which this requirement is fulfilled. For the purpose of illustration, we give details in the case of level sets (Example~\ref{E:levelsets}) and $H_2=C[0,1]$. The other examples can be explored similarly.

\begin{Proposition}\label{l:level1}
Let $\alpha \in \mathbb{R}$ and $z \in [0,1)$.
We define $A=\{y\in C[0,1]\colon \lambda(y>\alpha)\leq z\}$.
The following conditions imply $P(Y\in\partial A)=0$:
\begin{align}
& (i) \; P\big(\lambda(Y=\alpha)>0\big)=0 \quad\text{and}\quad   (ii)\; P\big(\lambda(Y>\alpha)=z\big)=0 && \text{for $z\in (0,1)$},\label{e:condlevel}\\
 &P\big(\sup_{t\in [0,1]} Y(t)=\alpha\big)=0 && \text{for $z=0$}.\label{e:condlevel2}
\end{align}
\end{Proposition}

 The conditions in \eqref{e:condlevel} and \eqref{e:condlevel2} are satisfied by many well known processes, including Brownian motion. They are also generally satisfied by continuously differentiable Gaussian processes under standard non-degeneracy conditions. Such processes might be used to model functional data generated by applying standard smoothing operations, for instance using cubic-splines or trigonometric polynomials, to raw discrete data. We note that comparable differentiability conditions are assumed in \cite{fan:muller:2021}. The following proposition, whose proof we defer to Section~\ref{s:proofs}, describes these conditions.

\begin{Proposition}\label{gauss-lemm}
Suppose that $Y$ is a continuously differentiable Gaussian process with covariance kernel $C_Y$. If $C_Y(t,t)  > 0$ for all $t\in [0,1]$, then \eqref{e:condlevel}(i) holds. For $\ell \in \mathbb{N}$, and $0\le t_1< \cdots  < t_\ell \le 1$, let
\[
r_Y(t,s) = \frac{C_Y(t,s)}{[C_Y(t,t) C_Y(s,s)]^{1/2}}, \mbox{ and } R_{t_1,...,t_\ell} = \{ r_Y(t_i,t_j)\}_{1\le i,j \le \ell} \in \mathbb{R}^{\ell\times \ell}.
\]
If in addition for all $\ell\in \mathbb{N}$ and $0\le t_1< \cdots  < t_\ell \le 1$, there exists constants $c_1, c_2 > 0 $ such that $det( R_{t_1,...,t_\ell}) \ge c_1 \min_{ 1 \le i\ne j \le \ell } | t_i - t_j |^{c_2}$, then  \eqref{e:condlevel}(ii) holds.
If $Y$ is twice continuously differentiable, and
$(Y(t_1),\dots,Y(t_\ell),Y'(t_1),\dots,Y'(t_\ell),Y''(t_1),\dots,Y''(t_\ell))$ has a non-degenerate distribution, then  \eqref{e:condlevel2} holds.
\end{Proposition}

If $A = \{y \in H_2: \langle y, \gamma \rangle > c \}$
is a contrast set with some $\gamma \in H_2$, $c \in \mathbb{R}$,
then from the continuity of the inner product it follows that
$\partial A = \{y \in H_2: \langle y, \gamma \rangle = c \}$, both for $H_2=L^2[0,1]$, and $H_2=C[0,1]$. As for the boundary set
$B_{\alpha,\beta} = \{y\in H_2\colon y([0,1]) \subseteq [\alpha, \beta] \}$, with $H_2=C[0,1]$, $\partial B_{\alpha,\beta} =
\{y\in H_2\colon \sup_{t\in [0,1]} y(t) = \beta \vee \inf_{t\in [0,1]} y(t) = \alpha \}.$

\section{Simulation experiments and data illustrations}\label{s:realdata:sim}

In this section we present the results of several simulation experiments and real data analyses that aimed to evaluate and compare the performance of the algorithms {\bf boot} and {\bf Gauss}, and illustrate their application. We begin by defining some alternate methods that may be used to estimate $P(Y\in A |X)$, and we describe two recent procedures proposed for functional quantile regression and construction of prediction sets in functional data prediction, respectively.

\subsection{Competing methods}

A simple method to estimate $P(Y\in A |X)$ is to employ functional binomial regression.
This entails positing the model
\[
P(Y \in A | X=x) = g\big( \beta_0 + \langle x, \beta \rangle \big)
\]
for some $\beta_0 \in \mathbb{R}$ and $\beta \in L^2[0,1]$, and a link function $g$
that can be chosen from a variety of possibilities, but is most often
the logistic link function, or the cumulative distribution function of a standard normal random variable (the ``probit link''). For more details of such models, we refer to \cite{muller:stadtmuller:2005} and \cite{mousavi:sorensen:2017}. One drawback of note in applying logistic regression in this setting is that changing the set $A$ necessitates refitting the model, which can be computationally cumbersome, and further, as a consequence, the resulting estimators of $P(Y \in A | X)$ need not be monotone with respect to increasing (or decreasing) sets $A$. An approach to adjust such estimators to restore monotonicity is to use rearrangement or isotonization, as discussed in e.g.  \cite{chernozhukov:2010}.

Since the exact relationship between the function $X$ and the event $\{Y \in A\}$
is unknown and difficult to describe in parametric terms, even under model \eqref{mod},
another promising approach is to use nonparametric techniques such a kernel estimators.
Generalizing the method found in Section~5.4 of \cite{ferraty:vieu:2006},
the conditional distribution $P(Y \in A | X = x) = E(\mathds{1}\{Y \in A\} | X = x)$
can be estimated by the functional extension of the
Nadaraya--Watson estimator
\begin{align}
\hat P^\text{NW}(Y\in A|X=x) = \frac{ \sum_{i=1}^n K\big( h^{-1} d(x,X_i) \big)  \; \mathds{1}\{Y_i \in A\} }
                   { \sum_{i=1}^n K\big( h^{-1} d(x,X_i) \big) },
\label{e:nadwat}
\end{align}
where $K$ is a kernel function on the nonnegative real numbers,
$d$ is a distance measure on $H_1$, and
$h > 0$ is a smoothing parameter corresponding to the bandwidth of the kernel.
Note that while the choice of $K$ is typically unproblematic,
the choice of $d$ is more intricate and is often taken to depend on the data.
The bandwidth $h$ represents the trade-off between bias (oversmoothing) and error (undersmoothing),
and is normally taken to decrease with the sample size $n$. \cite{ferraty:vieu:2006} establish consistency conditions for the estimator \eqref{e:nadwat}
in the case when the sequence $\{ (X_k,Y_k)\colon k\geq 1 \}$ is $\alpha$-mixing and $Y$ is scalar. When we apply this method below, we take $K$ to be the standard Gaussian kernel, $d$ to be the norm on $H_1$, and select $h$ using cross-validation.  We note that similarly to functional logistic regression based estimators, a draw back of these estimators is that if one changes the set $A$, then the bandwidth $h$ in general should be recalibrated, and the resulting estimators need not be monotone in $A$ if the bandwidth $h$ is not held fixed for all sets $A$.

Similar options may be derived from the local linear functional estimator,
which improves upon the Nadaraya--Watson estimator by including a linear terms of the form $\langle x - X_i, \beta \rangle$ into the computation of the weights; see \cite{berlinet:2011}. The $k$-nearest neighbors (kNN) functional estimator
is a variation on the Nadaraya--Watson estimator with adaptive bandwidth,
i.e. $h$ is the smallest number such that
$\big|\{ X_i\colon d(x,X_i) \leq h \}\big| = k$.
The kNN estimator has been shown to be consistent for non-parametric regression
by \cite{kudraszow:2013}.

In order to evaluate the proposed algorithms for the construction of prediction sets, we compared to the method of \citet{paparoditis:2020} in the setting of forecasting FAR(1) processes $Y_k-\mu = \varrho (Y_{k-1}-\mu) + \varepsilon_k$. Subsequent to forming the estimator $\hat{\varrho}_n$ using functional principal component analysis, their method entails performing a (sieve) bootstrap on the functional principal component scores of the residuals $\hat{\varepsilon}_{k,n}$ in order to estimate the distribution of the prediction error.
$Y_{n+1}$ is then forecast by $\widehat Y_{n+1} = \hat{\mu}+ \hat{\varrho}_n(Y_n-\hat\mu)$, and uniform prediction sets for $Y_{n+1}$ are constructed of the form
\[
\{ y \in C[0,1] :   \widehat Y_{n+1}(t) + L \, \sigma_{n+1}(t) \le y(t) \le  \widehat Y_{n+1}(t) + U \, \sigma_{n+1}(t), \mbox{ for all } t\in[0,1] \},
\]
where for a specified coverage level $1-\alpha$,
\[
\sigma_{n+1}^2(t) = \widehat\var( \varepsilon^{(n)}(t)), \mbox{ and with }
 M = \sup_{t\in[0,1]} \frac{ |\varepsilon^{(n)}(t) | }{\sigma_{n+1}(t)},
\quad
L = Q_{\alpha/2}(  M),
\mbox{ and }
U = Q_{1-\alpha/2}(M).
\]

In the setting of scalar-on-function quantile regression, we compare to the method of \cite{sang:2020}, which entails for a scalar response $T(Y)$ modelling
\[
T(Y) = g\big( \beta_0 + \langle x, \beta \rangle \big)+ \varepsilon,
\]
where $g$ is an assumed to be unknown link function. The link function $g$ as well as the parameter function $\beta$ are assumed to be linear combinations of splines, and estimated in order to estimate the level $p$ quantile of $T(Y)$ by  minimizing the check function loss
\[
\rho_p(y) = \big(p - \mathds{1}\{y \leq 0\}\big) \, y,
\]
subject also to a roughness penalty on the functions $g$ and $\beta$.

\subsection{Construction of prediction sets}\label{s:pred}
 Following the simulation experiment considered in \cite{paparoditis:2020}, we construct a time series of continuous functions as follows:
\begin{align}
Y_k(t) = \int_0^1 \rho(t,s) Y_{k-1}(s) ds + b\cdot Y_{k-2}(t) + B_k(t), \quad 1\leq k \leq n, \; t\in[0,1],
\label{e:paparoditis}
\end{align}
where $\rho(t,s) = 0.34\, e^{(t^2+s^2)/2}$, and $B_k$ is a standard Brownian motion.
We fit an FAR(1) model to each simulated sample where
we chose the truncation parameter $T_n$ using the PVE criterion with $v=0.85$. This is the same value as used in  \cite{paparoditis:2020}.
If $b$ is chosen as $0$, the FAR(1) model is correctly specified, whereas with $b=0.4$, there is a model misspecification that should be detrimental
to the quality of the model predictions. Following the method proposed in \cite{paparoditis:2020} and as described above, we constructed uniform prediction sets  to forecast each series 1-step ahead, with nominal coverage probabilities of 80\% and 95\%. This was repeated independently 1000 times, with sample sizes $n \in \{ 100,200,400,800\}$. While the model for the forecast is the same for both methods, the difference between our approach and \cite{paparoditis:2020} is in the methods used to estimate the noise distribution $\varepsilon^{(n)}$. These results are summarised in Table~\ref{tab:paparoditis} in terms of empirical coverage probabilities from the 1000 replications.

In the case $n=100$ and $b=0$, the method {\bf boot} yielded empirical coverage probabilities that were up to 4--6 percentage points below the method of \cite{paparoditis:2020}, which are both below the nominal level. Apart from this notable exception, the empirical coverage probabilities are comparable to those of \cite{paparoditis:2020}, and were closer to nominal coverage in 10 out of 16 cases considered. The results of {\bf Gauss} were generally better, which is to be expected since the model errors are Gaussian processes, especially for the nominal coverage probability of 95\%.

\begin{table}
\centering
{\small
\begin{tabular}{llcccccccc}
  \hline
\multicolumn{2}{l}{Nominal} & \multicolumn{2}{c}{$n=100$} & \multicolumn{2}{c}{$n=200$} & \multicolumn{2}{c}{$n=400$} & \multicolumn{2}{c}{$n=800$} \\
\multicolumn{2}{l}{coverage} & $b=0$ & $b=0.4$ & $b=0$ & $b=0.4$ & $b=0$ & $b=0.4$ & $b=0$ & $b=0.4$ \\
  \hline
  80\% & {\bf boot}       &         0.683  &         0.694  &         0.745  &         0.754  &         0.777  &         0.778  &         0.789  &         0.791  \\ %
       & {\bf Gauss}       &         0.703  & \textbf{0.716} &         0.756  & \textbf{0.763} &         0.781  & \textbf{0.783} &         0.791  & \textbf{0.793} \\ %
 & \textit{P., S. (2020)} & \textbf{0.740} &         0.689  & \textbf{0.766} &         0.740  & \textbf{0.791} &         0.768  & \textbf{0.803} &         0.786  \\ %
  \hline
  95\% & {\bf boot}       &         0.861  &         0.872  &         0.913  &         0.917  &         0.933  &         0.936  &         0.944  &         0.944  \\ %
       & {\bf Gauss}       &         0.898  & \textbf{0.904} & \textbf{0.927} & \textbf{0.931} & \textbf{0.940} & \textbf{0.943} & \textbf{0.946} & \textbf{0.946} \\ %
 & \textit{P., S. (2020)} & \textbf{0.902} &         0.856  &         0.918  &         0.899  &         0.927  &         0.913  &         0.936  &         0.924  \\ %
   \hline
\end{tabular}
}
\caption{\label{tab:paparoditis} Empirical coverage probabilities of uniform prediction intervals for the data generating process \eqref{e:paparoditis} calculated via {\bf boot}, as well as using the method of \cite{paparoditis:2020}, abbreviated  \textit{P., S. (2020)}. }
\end{table}

\subsection{Comparison to functional GLM and Nadaraya--Watson estimation}\label{ss:pm10data}

\begin{figure}[t]
\begin{minipage}[b]{0.60\textwidth}
\includegraphics[width=\textwidth]{PM10-realdata.pdf}
\includegraphics[width=\textwidth]{PM10-simdata.pdf}
\end{minipage}
\includegraphics[width=0.39\textwidth]{PM10-operator-contour.pdf}
\caption{Top left: the raw PM$_{10}$ measurements (blue) with the fitted curves (black).
Bottom left: simulated synthetic PM$_{10}$ data (black) with $\alpha = \sqrt{50}$ (red) that we considered in the level set case.
Right: the kernel operator $\varrho(t,s)$ used in the data generating process.}
\label{fig:PM10DGP}
\end{figure}

In this simulation experiment, we generated synthetic data
under model \eqref{mod} in such a way that it resembled a real functional time series derived from daily square-root transformed
PM$_{10}$ concentration curves constructed by smoothing half-hourly measurements of PM$_{10}$.
This is done using the function {\tt Data2fd} in the {\tt fda} package with default settings; see \cite{fdapackage}.
PM$_{10}$ concentration denotes the concentration in air of respirable coarse particles
having a diameter less than 10$\mu m$, and the data that we consider was collected in Graz, Austria over the period from  October 1st, 2010 to March 31st, 2011. An illustration of these data is given in Figure~\ref{fig:PM10DGP}, and they are available in the {\tt ftsa} package in {\tt R}; see \cite{hynd:shang:ftsa:2020}.

We use these data as a means to devise a realistic data generating process.
To this end, we first fit an FAR$(1)$ model to the square-root transformed PM$_{10}$ curves.
The estimator for the FAR operator $\varrho$ obtained in this way differs from operators typically used in simulation settings in that the estimator for the kernel is highly asymmetric, as illustrated in the right hand panel of Figure~\ref{fig:PM10DGP}.

With the estimated sample mean and the fitted FAR operator, we then generate synthetic FAR$(1)$ time series samples by drawing model errors $\varepsilon_k$ from a Gaussian distribution, with the covariance operator estimated from the residuals of the FAR(1) model fit to the original data. This can be done as in the algorithm {\bf Gauss}. The first 30 observations are dropped as part of the burn-in phase.  A snapshot of the raw data in comparison to the synthetic data can be seen in Figure~\ref{fig:PM10DGP}. In this manner we may generate time series of arbitrary sample sizes that are similar to the original PM$_{10}$ data. We generated 1000 independent samples for each sample size $n \in \{50, 100, 250, 1000\}$. Then, for 50 different values of predictors $Y_0^*$, simulated independently from the stationary distribution of the data generating process,  we estimated the conditional probability of $Y^*_{1}$ lying in the level set $P(\lambda(Y_{1}^* > \sqrt{50}) \leq 0.5| Y_0^*)$ for each such sample. For each of the 50 predictors, we also approximated the true probability using Monte-Carlo simulation ($n_{\text{MC}} = 10000$) from the data generating process.

We compared the estimators from algorithms {\bf boot} and {\bf Gauss}, as well as from a logistic functional GLM, and Nadaraya--Watson estimation.
The number $T_n$ of principal components used to estimate $\varrho$ was chosen using  criterion \eqref{e:chooseTn}, so that
\[
T_n = \max\big\{j \geq 1 \colon \hat \lambda_j \geq m_n^{-1} \hat \lambda_1 \big\},
\quad \text{with }\; m_n = 5  n^{0.45}.
\]
We introduce $\hat \lambda_1$ into  the definition of $T_n$ so that the criterion does not depend on the scale of the eigenvalues, yielding a more practicable way of choosing $T_n$.
For $n=1000$, this approximately covers 98\% of the variance of the simulated curves in the sense of the PVE criterion.
Naturally, less variance is covered in for smaller sample sizes.
For the logistic GLM, we used the approach suggested by \cite{muller:stadtmuller:2005}
and took the truncated Karhunen--Loève expansion as the predictor.
In order to keep the methods comparable,
we used the same number $T_n$ of principal components for our algorithms and for the functional GLM.
We calibrated the bandwidth $h$ for the Nadaraya--Watson estimator using leave-one-out cross-validation on each generated sample.
The results in terms of the root mean squared error (RMSE) over the 1000 simulations are displayed in Figure~\ref{fig:PM10RMSElevel}.
Because it is difficult to visualize this for the 50 different predictors, we present boxplots  summarizing the RMSE of each method over all predictors $Y_0^*$.  More details on the results for a variety of specific values of $Y_0^*$ can also be found in Table~\ref{tab:pm10rmselevel} in the Appendix.

\begin{figure}
\includegraphics[width=\textwidth]{PM10-RMSE-a-sqrt50-z05.pdf}
\caption{ RMSE of $\hat P$ for 50 random predictors $Y_0^*$ and 1000 independent simulations of samples of size $n\in \{ 50, 100, 250, 1000 \}$ based on the estimators {\bf boot}, {\bf Gauss}, functional logistic regression, and Nadaraya--Watson estimators of the probability $P(\lambda(Y_{1}^* > \sqrt{50}) \leq 0.5| Y_0^*)$.}
\label{fig:PM10RMSElevel}
\end{figure}

We observed that algorithms~{\bf boot}  and  {\bf Gauss} exhibited similar predictive performance in both examples and over all  sample sizes. These methods clearly outperformed functional logistic regression and Nadaraya--Watson estimation in estimating the conditional probability of level sets. The proposed methods achieved a similar mean squared error in this case to functional logistic regression with about a quarter of the sample size.
The performance of the Nadaraya--Watson estimator was poor compared to the other methods considered in both cases
and varied strongly depending on the predictor $Y_0^*$. In Appendix~\ref{s:additionalsim}, we also present results in which we considered contrast sets rather than level sets, in which case the same overall pattern was observed, although the results were more comparable across the four methods.

\begin{table}
\centering
\begin{tabular}{|r|rr|rr|rr|rr|}
  \hline
 & \multicolumn{2}{c|}{$n=50$} & \multicolumn{2}{c|}{$n=100$} & \multicolumn{2}{c|}{$n=250$} & \multicolumn{2}{c|}{$n=1000$} \\
 & \multicolumn{2}{c|}{$p=0.98$} & \multicolumn{2}{c|}{$p=0.99$} & \multicolumn{2}{c|}{$p=0.996$} & \multicolumn{2}{c|}{$p=0.999$} \\
$Y_0^*$ & \textbf{boot} & \textbf{Gauss} & \textbf{boot} & \textbf{Gauss} & \textbf{boot} & \textbf{Gauss} & \textbf{boot} & \textbf{Gauss} \\
  \hline
  1 & 0.770 & 0.704 & 0.559 & 0.491 & 0.457 & 0.356 & 0.487 & 0.434 \\
  2 & 0.530 & 0.437 & 0.396 & 0.292 & 0.333 & 0.170 & 0.341 & 0.189 \\
  3 & 0.517 & 0.419 & 0.432 & 0.287 & 0.348 & 0.199 & 0.308 & 0.137 \\
  4 & 0.595 & 0.501 & 0.464 & 0.371 & 0.404 & 0.262 & 0.347 & 0.187 \\
  5 & 0.589 & 0.480 & 0.470 & 0.355 & 0.378 & 0.247 & 0.336 & 0.144 \\
   \hline
\end{tabular}
\caption{\label{tab:pm10varlevel} RMSE for $\hat\alpha_p$, 5 different predictors and 1000 replications. We estimate $\hat\alpha_p$ such that $P(\lambda(Y_{1}^* > \alpha_p)) \leq 0.5 | Y_0^*) = p$, where $p = 1-n^{-1}$.}
\end{table}

Although the estimator {\bf Gauss} performs similarly to {\bf boot} in the above example, it can be expected that {\bf boot} runs into problems when $P(Y \in A|X)$ is close to $0$ or $1$, since {\bf boot} only uses the $n$ estimated model residuals  to estimate $P(Y \in A|X)$, whereas in producing the estimator \textbf{Gauss},
one can generate a Monte-Carlo sample of residuals as large as needed to give a non-degenerate estimate of these  probabilities, which can be expected to be accurate if the Gaussian assumption is plausible.  To highlight this, we present the results of a short simulation study in which for a probability $p_n=1-1/n$, we aimed to estimate $\alpha_p$  using {\bf boot} and {\bf Gauss}  such that $P(\lambda(Y_{1}^* > \alpha_{p_n}) \leq 0.5 | Y_0^*) = p_n$. This problem is hence related to the  Value-at-Risk estimation. We compared the RMSE of $\hat\alpha_{p_n}$ from the two algorithms for 50 different realizations of the predictor $Y_0^*$ that were simulated from the same data generating process. We note that the value of $\alpha_p$ varies between 7.26 and 11.77, depending on $Y_0^*$ and $p_n$. In Table~\ref{tab:pm10varlevel}, we present the results from a subset of five predictors $Y_0^*$ that were representative of the variability observed in the simulated series. It is apparent from these results that {\bf Gauss} outperforms {\bf boot} in all cases, and the relative advantage increases with sample size.
If we look at the results for all 50 predictors, RMSE of $\hat\alpha_p$ decreases by about 15\% for $n=50$, 22\% for $n=100$, 35\% for $n=250$ and 42\% for $n=1000$. This gives some indication of the difference in performance that can be expected between the two methods in forecasting extreme quantiles or events whenever the Gaussian assumption is plausible.

\subsection{Functional quantile regression}\label{ss:quantilereg}

In this application, we compare to the data analysis of \cite{sang:2020}.
As in our previous example, those authors consider the functional time series of daily square-root transformed PM$_{10}$ concentration curves constructed by smoothing half-hourly measurements of PM$_{10}$.

The goal of the analysis is to compare forecasts of the quantiles of the maximum values $M_t = \max_{u\in[0,1]} Y_{t}(u)$ (we remark the relation to Example~\ref{E:extremalsets}), where $Y_t(u)$ is the transformed PM$_{10}$ curve on day $t$ at intraday time $u$. As the covariate the curve $Y_{t-1}$ is used. Now we model the relationship between $(Y_t,Y_{t-1})$ by a FAR(1) process and apply the method {\bf boot} to estimate the conditional quantile of $M_t$.
We select the truncation parameter $T_n$ in order to explain 98\% of the variance in the variables $Y_t$
since for this fixed sample size, tuning $T_n$ by an asymptotic criterion is not meaningful.
At a quantile level $p$, we compared these methods by
5-fold cross-validation the mean
check-function loss $\rho_p\big( M_i - \hat q_p(M_i|Y_{i-1})$.
We did this for seven different quantile levels $p \in \{0.05, 0.15, 0.25, 0.50, 0.75, 0.85, 0.95\}$. The experiment was repeated on 50 random splits of the data set. The results are displayed in Figure~\ref{fig:sangcao}. In 87.4\% of the cases, {\bf boot} outperformed the
functional single-index quantile regression model of \cite{sang:2020} in terms of the loss considered.
This advantage was much smaller for more central quantiles and became more apparent for the more extreme quantiles.

\begin{figure}[t]
\centering
\includegraphics[width=0.90\textwidth]{SangCaoComparison.pdf}
\caption{Performance of {\bf boot} compared to the
functional single-index quantile regression model proposed by \cite{sang:2020}. The prediction error is compared using 5-fold cross-validation on 50 random splits of the PM$_{10}$ data set.}\label{fig:sangcao}
\end{figure}

\subsection{Spanish electricity price data}\label{ss:electricitydata}

\begin{table}
\centering
\begin{tabular}{lrrrrrrrrrr}
  \hline
      &     $\alpha$    & \hspace{5mm} 30 & 35 & 40 & 45 & 50 & 55 & 60 & 65 & 70 \\
  \hline
      &  \textbf{boot}  & \textbf{0.03} & \textbf{0.06} & \textbf{0.10} & \textbf{0.13} & \textbf{0.16} & \textbf{0.24} &         0.23  & \textbf{0.19} & \textbf{0.16} \\
$z=0$ &  GLM            &         0.11  &         0.17  &         0.23  &         0.36  &         0.30  &         0.24  & \textbf{0.22} &         0.21  &         0.24  \\
      &  N--W           &         0.05  &         0.10  &         0.20  &         0.21  &         0.22  &         0.31  &         0.29  &         0.27  &         0.26  \\
  \hline
      &  \textbf{boot}  & \textbf{0.05} & \textbf{0.07} & \textbf{0.10} & \textbf{0.15} & \textbf{0.15} & \textbf{0.19} & \textbf{0.17} & \textbf{0.16} & \textbf{0.12} \\
$z=\frac 16$ &  GLM     &         0.22  &         0.18  &         0.22  &         0.25  &         0.18  &         0.19  &         0.17  &         0.19  &         0.24  \\
      &  N--W           &         0.08  &         0.14  &         0.18  &         0.22  &         0.25  &         0.26  &         0.23  &         0.22  &         0.18  \\
  \hline
      &  \textbf{boot}  & \textbf{0.05} & \textbf{0.10} & \textbf{0.12} & \textbf{0.13} & \textbf{0.20} & \textbf{0.17} & \textbf{0.15} & \textbf{0.13} & \textbf{0.09} \\
$z= \frac 26$ &  GLM    &         0.25  &         0.15  &         0.26  &         0.23  &         0.23  &         0.17  &         0.23  &         0.20  &         0.39  \\
      &  N--W           &         0.10  &         0.16  &         0.23  &         0.23  &         0.26  &         0.22  &         0.24  &         0.22  &         0.14  \\
  \hline
      &  \textbf{boot}  & \textbf{0.08} & \textbf{0.10} & \textbf{0.11} & \textbf{0.17} & \textbf{0.20} & \textbf{0.17} & \textbf{0.15} & \textbf{0.11} &         0.07  \\
$z= \frac 36$ &  GLM    &         0.11  &         0.15  &         0.26  &         0.27  &         0.23  &         0.19  &         0.18  &         0.20  &         0.37  \\
      &  N--W           &         0.14  &         0.17  &         0.26  &         0.24  &         0.27  &         0.23  &         0.23  &         0.17  & \textbf{0.07} \\
  \hline
      &  \textbf{boot}  & \textbf{0.09} & \textbf{0.11} & \textbf{0.14} & \textbf{0.19} & \textbf{0.22} & \textbf{0.18} & \textbf{0.12} & \textbf{0.08} &         0.03  \\
$z= \frac 46$ &  GLM    &         0.12  &         0.18  &         0.27  &         0.26  &         0.25  &         0.24  &         0.20  &         0.40  &         0.12  \\
      &  N--W           &         0.16  &         0.20  &         0.23  &         0.26  &         0.28  &         0.25  &         0.19  &         0.13  & \textbf{0.02} \\
  \hline
      &  \textbf{boot}  & \textbf{0.14} & \textbf{0.17} & \textbf{0.20} & \textbf{0.24} & \textbf{0.23} & \textbf{0.14} & \textbf{0.08} & \textbf{0.02} & \textbf{0.00} \\
$z= \frac 56$ &  GLM    &         0.21  &         0.26  &         0.25  &         0.26  &         0.27  &         0.17  &         0.21  &         0.13  &         0.10  \\
      &  N--W           &         0.24  &         0.29  &         0.29  &         0.31  &         0.29  &         0.19  &         0.09  &         0.03  &         0.00  \\
   \hline
\end{tabular}
\caption{\label{tab:spaincrossentropy} The cross-entropy of the estimated conditional probability $P(\lambda(Y >\alpha) \leq z)$ for different values $\alpha$ and $z$, evaluated on the test set. The comparison value GLM is a logit regression model with the same predictors. N--W is the Nadaraya--Watson estimator. The smallest value in each cell is marked in bold font, and any apparent ties are merely a result of the rounding to two digits.}
\end{table}

We now return to the Spanish electricity price data that we gave as an introductory example. We take as the goal of this analysis to compare estimates for the conditional probability that the price curves will lie in specified level sets, given the covariates of demand, and wind energy production. In order to compare the various methods for doing this, we split the data into a training and testing set  by randomly taking four months from each year, and assigning them to the test set, which created a 2:1 split between the training and testing set.  Since we used 6 years of this data, the training set thus consists of 1453 days,
and the test set consists of 731 days.
Let $Z_t$ denote one of functional variables electricity price, demand or wind energy production. Then these variables were deseasonalized as follows:
\[
\widetilde Z_t = Z_t - Z_t^{(Y)} - Z_t^{(W)},
\]
where $Z_t^{(Y)}$ is the yearly seasonality obtained
by taking the mean for each day of the year and smoothing the result
using a rolling mean with a window size of 21 days.
$Z_t^{(W)}$ is the weekly seasonality that is estimated as the mean for each day of the week.
For the wind curves, no weekly seasonality was removed. In order to employ the methods {\bf boot} and {\bf Gauss}, we fit the FARX(7) model described in \eqref{e:spainfarx} using the estimator introduced in Section~\ref{s:theoretical} with the data in the training set. The truncation parameter $T_n$ was again chosen in order to explain 98\% of the variance of the covariates.

In order to compare the estimated conditional probabilities to the realized outcomes on the test set, we used the cross-entropy measure.
The cross-entropy of a distribution $P$ relative to a distribution $Q$
is defined as
$H(P,Q) = -\mathbb{E}_P\big[ \log(q(Y)) \big]$,
where $q$ is the probability mass function of $Q$; see Section 2.8 of \cite{murphy2012machine}. Given the realisations $y_i = \mathds{1}\{Y_i \in A\}$, $i \in \{1,...,N\}$, and corresponding estimated conditional probabilities $\hat{p}_i= \hat{P}(Y_i \in A | X_i)$ in the testing set of size $N$, the plug-in estimator of the cross-entropy estimated on the test set is
\[
\hat{H}(\hat{P}) = -\frac 1 N \sum_{k=1}^N [y_k \log\big( \hat{p}_k \big)+ (1-y_k) \log\big(1- \hat{p}_k \big)].
\]

We considered level sets of the form $A= \{ y \in C[0,1] : \lambda(Y > \alpha) \leq z)\}$ for various values of $\alpha$ and $z$, and calculated the cross-entropy on the test set of estimates of $P(Y \in A |X)$ using the method {\bf boot}, as well as for functional logistic regression, which was estimated using the same covariates (and PVE criterion) as those considered in generating the estimator in {\bf boot}, as well as functional Nadaraya--Watson estimation with a Gaussian kernel with the predictors
\emph{demand}, \emph{wind} and \emph{lagged price}, and the bandwidth parameters were selected using leave-one-out cross-validation on the training set. We do not present the results for the method {\bf Gauss}, as the results are again very similar to the method {\bf boot}. The estimated cross entropies on the test set for each set $A$ considered are presented in Table~\ref{tab:spaincrossentropy}. The smallest value in each cell is marked in bold font.

The method {\bf boot} achieved lower values of cross-entropy on the test set compared to the competing methods for most combinations of $\alpha$ and $z$. {\bf boot} had higher estimated cross-entropy in one case compared to functional logistic regression model,
and two cases compared to functional Nadaraya--Watson estimation. The values of $\alpha$ and $z$ considered were chosen such in a way
that most price curves belong to at least one set, and so we do not think this superior performance resulted from the sets $A$ focusing on outcomes that are well modelled using the FARX(7) model.

\section{Summary}

We considered two methods, based on either a residual bootstrap or Gaussian process simulation, to estimate the conditional distribution $P(Y\in A|X)$, where $Y$ and $X$ satisfy the functional linear regression model \eqref{mod}. We showed under mild consistency conditions on the estimated regression operator $\hat{\varrho}_n$ that these methods lead to consistent estimation, in particular in the setting where $Y$ is assumed to be an element of the Banach space $C[0,1]$, which allows for the consideration of sets $A$ that describe more detailed path properties of the response. We put forward one example of an operator estimator $\hat{\varrho}_n$ that has the specified consistency properties under natural regularity conditions on the covariates, which allow for weak serial dependence, and on the choice of tuning parameters. In several simulation experiments and data analyses we observed that these methods generally outperformed prominent competitors, which are often more complicated to implement.

\addcontentsline{toc}{section}{References}


\section{Introduction}

With the ever larger data output volume of modern astronomical observatories,
automated data processing pipelines are an indispensable part of any such project.
New observatory projects are now aware that data analysis (DA) software
needs a significant lead time and has to be budgeted for carefully.
But even with adequate financial means, it is nearly impossible for the DA software
to be ready {\it and verified} the moment the observatory records its first ``real'' data.
Any complex and novel observatory/detector system will have its unexpected features
and quirks which cannot be foreseen and simulated ahead of time.
As these unexpected features of the real data are gradually becoming apparent and understood, the
DA software needs to be adapted and equipped with corresponding heuristics. This is the usual commissioning
phase which is to be kept as short as possible such that scientific results can be produced soon.
Usually, a compromise is made and the new observatory/instrument is made available to
scientific users with a reduced set of features which is then gradually expanded
as commissioning of additional features is completed.
But even for relatively small sets of new capabilities, the commissioning to the point
of the release of a fully automated pipeline (PL), can be a long process of many months
if not years because the work requires know-how which is only available to a small expert team
that cannot arbitrarily be enlarged.
In this paper, I am presenting an approach to prototyping the automated PL
which has helped the ALMA observatory \citep{Cortes2020} since 2012 to shorten the time-to-release
of new capabilities by ca. one year. Its usefulness is not confined to the realm of radio astronomy.

\section{Three-stage development start-up}
At the beginning of the construction of a new observatory, the arrival of the first real data is years away.
At this point, the data model for the raw data needs to be defined and the creation of simulated raw data prepared.
Then the development of the DA software can proceed in the following three stages:

{\bf 1 - Scriptable toolkit:}
The first layer of functionality which needs to be provided for commissioning scientists and
later science users of the observatory, is a kit of DA tools from which an arbitrary
sequence of calibration and analysis steps can be constructed. These tools need to be bound
to a convenient scripting language, e.g. Python. It needs to include efficient methods to
import raw data into variables/arrays of the scripting language, perform common operations
on the imported data, and derive calibration information which can subsequently be stored
and applied to the raw data producing calibrated data in a convenient and efficient storage format.
Furthermore tools for data visualization need to be provided including interactive GUIs
for exploring the raw and calibrated data.
Finally, tools for transforming the calibrated data into astronomical images can already
be developed in this first phase based purely on simulated data.

{\bf 2 - Script generator:}
Once the beginning of instrument commissioning comes in sight and the scriptable toolkit
has reached some maturity, the development of the second layer of functionality can commence:
a tool to create standard data processing scripts for the given instrument - the {\it script generator} (SG).

This tool can be understood as a prototype PL: it examines a given set of raw data and then
outputs a (nearly) complete processing script tailored to the given dataset and
using the commands from the scriptable toolkit. The SG already contains many of the heuristics
which will later be at work in the automated PL but it (at least initially, in its first releases)
doesn't attempt to do the full job. Sections of the process which are not yet fully understood or
which are too complex to automate, may still be omitted in the output script draft such that
users can complete it manually.

\articlefigure[width=.5\textheight]{X9-005_f1.eps}{processgant}{Simplified Gantt chart of the data analysis software development for an
arbitrary observatory following the approach described in this paper. $\Delta T$ is the time period by which new capabilities
can be released earlier because an SG was employed as a PL prototype. The exact time scale will depend on the project complexity. Continuation of the development of data model and toolkit is not shown.}

The resulting SG tool is a help for commissioning scientists and data analysts.
It automates those parts of the data analysis process which are
already common knowledge, saving its users time that they can spend on the less well understood parts
of the analysis.

The development of an SG is not trivial but much simpler than that of a fully fledged PL.
Since the SG needs to read the instrument data, the SG needs to use the scriptable toolkit and
therefore should be coded in the scripting language which is bound to it.
A first version of the SG can already be produced before the instrument's first light.
As soon as real data is available, the SG is rapidly updated with new heuristics.
Unit tests of the SG can be constructed from test datasets which exhibit certain instrument
features and capabilities.

The execution time of an SG is typically much shorter than that of the subsequent execution of the DA script
which it produces because the SG mostly examines the {\it metadata} of the dataset, not the bulk data.
The automation of the scripting permits to produce fancy, standardized, well formated and marked-up
scripts for the end-users.

At the end of the initial commissioning phase, when the observatory starts to take its first science data,
the SG, like an expert system for writing data analysis scripts, encapsulates all the data processing knowledge
which was accumulated by the commissioning scientists up to this point.
The SG can then be used for standard data processing of the observatory's science data
while the development team can give full attention to the third stage of development.

{\bf 3 - Automated pipeline:}
The final stage of the first round of DA software development is
the complete automation of the data calibration and science product generation
(depending on the nature of the instrument this could, e.g., be imaging, the production of spectra etc.).
The fact that the SG is in place for the time being to provide a reasonably fast means
of processing data semi-automatically with the help of a number of data analysts,
buys time for the PL development team.

This is the most labour-intensive phase of the project w.r.t. data processing. Three teams work in parallel:
(1) The commissioning scientists continue to monitor the most recent raw data and find further improvements
to the processing heuristics quickly updating them in the SG.
(2) Data analysts use the SG to generate data processing scripts which they run to produce the calibrated data
  and derived data products for the observatory archive and its users.
(3) The PL development team works on automating the process in order to speed it up and
  reduce the number of necessary data analysts in preparation for the ramping up of the output data volume of the observatory.
  Thorough testing on a diverse selection of datasets is needed.

Once the first official data processing PL is released, the work of the data analysts begins to change
from mostly working with the SG to more and more double-checking the PL products.
As confidence in the PL grows, the fraction of the total data volume which is handled by it
approaches 100\%. Now the observatory can afford to further increase its output data volume.
The SG is only used in exceptional cases, {\it and} during commissioning of new capabilities.

\section{Gradual addition of further capabilities}
For ground-based observatories, the set of capabilities can be extended indefinitely. Every time a new capability is added,
the commissioning scientists can fall back to semi-automatic data processing with the help of the SG.
They develop new heuristics for the new type of data, and add them to the features of the SG.
Once the new capability is released to the observatory users, the observatory data analysts will at first use
the SG again to process this subset of the observatory output.
At the same time, the PL development team can start to enable the PL to handle the new type of data.
After one development cycle, the PL has learned to handle the new capability.

In this way, observatory users can be given access to well calibrated data and good derived data products from
recently added observatory upgrades with a minimum delay.

\section{The process in real life: ALMA}
The data analysis software development approach described above is the path which the ALMA observatory
followed on its way into science operations and through many capability expansions.
The raw data model and format chosen by ALMA
is the ALMA Science Data Model (ASDM, \citet{2006ASPC..351..627V}).
ALMA's scriptable data analysis toolkit is CASA \citep{2007ASPC..376..127M, 2020ASPC..527..267E} which
uses the scripting language Python. The ALMA raw data was calibrated during the first years of operations starting 2012
using exclusively the Calibration Script Generator (CSG)
 \citep[see][]{2014SPIE.9152E..0JP}, a tool also written in Python. Then, in 2014, the first ALMA calibration PL was released
(the documentation of this and subsequent PL versions can be found at
\url{https://almascience.org/processing/science-pipeline}, the latest is \citet{ALMAPipe2021}).
Since then, the CSG and the calibration PL have co-existed. ALMA data is calibrated
by the calibration PL whenever possible. For new observing modes, the CSG is upgraded first and used for ca.
one year of operations while the PL is being upgraded correspondingly. After that, the PL takes over the processing of the new mode.
Since ALMA also aims to offer high-quality images to its users, there is a second processing step after calibration to
produce these. This was initially done using imaging script templates. Then a second script generator was created,
the Imaging Script Generator (ISG). In 2016, the first ALMA PL with automated imaging was released.
Today, 95\% of the ALMA data are PL calibrated and imaged \citep{2020SPIE11449E..1TN}.
CSG and ISG have been released in 2021 for public use
(see \url{https://confluence.alma.cl/display/EAPR/ALMA+Data+Analysis+Utilities}).