\section{Introduction}
\label{1}

When analysing process behaviour, one of the early choices one has to make is between a linear and a branching view of time. In branching-time semantics, the choices a process has for proceeding from a particular state are taken into account when defining a notion of process equivalence (with bisimulation being the typical such equivalence), whereas in linear-time semantics such choices are abstracted away and the emphasis is on the individual executions that a process is able to exhibit. From a system verification perspective, one often chooses the linear-time view, as this not only leads to simpler specification logics and associated verification techniques, but also meets the practical need to verify all possible system executions.

While the theory of coalgebras has, from the outset, been able to provide a uniform account of various bisimulation-like observational equivalences (and later, of various simulation-like behavioural preorders), it has so far not been equally successful in giving a generic account of the linear-time behaviour of a state in a system whose type incorporates a notion of branching. For example, the generic trace theory of \cite{HasuoJS07} only applies to systems modelled as coalgebras of type ${\mathsf{T}} \circ F$, with the monad ${\mathsf{T}} : {\mathsf{Set}} \to {\mathsf{Set}}$ specifying a branching type (e.g.~non-deterministic or probabilistic), and the endofunctor $F : {\mathsf{Set}} \to {\mathsf{Set}}$ defining the structure of individual transitions (e.g.~labelled transitions or successful termination). The approach in loc.\,cit.~is complemented by that of \cite{JacobsSS12}, where traces are derived using a determinisation procedure similar to the one for non-deterministic automata. The latter approach applies to systems modelled as coalgebras of type $G \circ {\mathsf{T}}$, where again a monad ${\mathsf{T}} : {\mathsf{Set}} \to {\mathsf{Set}}$ is used to model branching behaviour, and an endofunctor $G$ specifies the transition structure. Neither of these approaches is able to account for potentially infinite traces, as typically employed in model-based formal verification. This limitation is partly addressed in \cite{cirstea-11}, but again, this only applies to coalgebras of type ${\mathsf{T}} \circ F$, albeit with more flexibility in the underlying category (which in particular allows a measure-theoretic account of infinite traces in probabilistic systems). Finally, none of the above-mentioned approaches exploits the compositionality that is intrinsic to the coalgebraic approach. In particular, coalgebras of type $G \circ {\mathsf{T}} \circ F$ (of which systems with both inputs and outputs are an example, see Example~\ref{input-output}) can not be accounted for by any of the existing approaches. This paper presents an attempt to address the above limitations concerning the types of coalgebras and the nature of traces that can be accounted for, by providing a \emph{uniform} and \emph{compositional} treatment of (possibly infinite) linear-time behaviour in systems with branching.

In our view, one of the reasons for only a partial success in developing a fully general coalgebraic theory of traces is the long-term aspiration within the coalgebra community to obtain a uniform characterisation of trace equivalence via a finality argument, in much the same way as is done for bisimulation (in the presence of a final coalgebra). This encountered difficulties, as a suitable category for carrying out such an argument proved difficult to find in the general case. In this paper, we tackle the problem of getting a handle on the linear-time behaviour of a state in a coalgebra with branching from a different angle: we do not attempt to directly define a notion of trace equivalence between two states (e.g.~via finality in some category), but focus on \emph{testing} whether a state is able to exhibit a particular trace, and on  measuring the extent of this ability. This "measuring" relates to the type of branching present in the system, and instantiates to familiar concepts such as the probability of exhibiting a given trace in probabilistic systems, the minimal cost of exhibiting a given trace in weighted computations, and simply the ability to exhibit a trace in non-deterministic systems.

The technical tool for achieving this goal is a generalisation of the notions of relation and relation lifting \cite{HermidaJ98}, which lie at the heart of the definition of coalgebraic bisimulation. Specifically, we employ relations valued in a partial semiring, and a corresponding generalised version of relation lifting. Our approach applies to coalgebras whose type is obtained as the composition of several endofunctors on ${\mathsf{Set}}$: one of these is a monad ${\mathsf{T}}$ that accounts for the presence of branching in the system, while the remaining endofunctors, assumed here to be polynomial, jointly determine the notion of linear-time behaviour. This strictly subsumes the types of systems considered in earlier work on coalgebraic traces \cite{HasuoJS07,cirstea-11,JacobsSS12}, while also providing compositionality in the system type.

Our main contribution, presented in Section~\ref{linear-time}, is a \emph{uniform} and \emph{compositional} account of linear-time behaviour in state-based systems with branching. A by-product of our work is an extension of the study of additive monads carried out in \cite{Kock2011,CoumansJ2011} to what we call \emph{partially additive monads} (Section~\ref{semiring}). Our approach can be summarised as follows:
\begin{itemize}
\item We move from two-valued to multi-valued relations, with the universe of truth values being induced by the choice of monad for modelling branching. This instantiates to relations valued in the interval $[0,1]$ in the case of probabilistic branching, the set $\mathbb N^\infty = \mathbb N \cup \{\infty\}$ in the case of weighted computations, and simply $\{\bot,\top\}$ in the case of non-deterministic branching. This reflects our view that the notion of truth used to reason about the observable behaviour of a system should be dependent on the branching behaviour present in that system. Such a dependency is also expected to result in temporal logics that are more natural and more expressive, and at the same time have a conceptually simpler semantics. In deriving a suitable structure on the universe of truth values, we generalise results on additive monads \cite{Kock2011,CoumansJ2011} to \emph{partially additive monads}. This allows us to incorporate probabilistic branching under our approach. We show that for a commutative, partially additive monad ${\mathsf{T}}$ on ${\mathsf{Set}}$, the set ${\mathsf{T}} 1$ carries a partial semiring structure with an induced preorder, which in turn makes ${\mathsf{T}} 1$ an appropriate choice of universe of truth values.
\item We generalise and adapt the notion of relation lifting used in the definition of coalgebraic bisimulation, in order to (i) support multi-valued relations, and (ii) abstract away branching. Specifically, we make use of the partial semiring structure carried by the universe of truth values to generalise relation lifting of polynomial endofunctors to multi-valued relations, and employ a canonical \emph{extension lifting} induced by the monad ${\mathsf{T}}$ to capture a move from branching to linear time. The use of this extension lifting allows us to make formal the idea of testing whether, and to what extent, a state in a coalgebra with branching can exhibit a particular \emph{linear-time} behaviour. Our approach resembles the idea employed by partition refinement algorithms for computing bisimulation on labelled transition systems with finite state spaces \cite{KS90}. There, one starts from a single partition of the state space, with all states related to each other, and repeatedly refines it through stepwise unfolding of the transition structure, until a fixpoint is reached. Similarly, we start by assuming that a state in a system with branching can exhibit any linear-time behaviour, and moreover, assign the maximum possible value to each pair consisting of a state and a linear-time behaviour. We then repeatedly refine the values associated to such pairs, through stepwise unfolding of the coalgebraic structure.
\end{itemize}

The present work is closely related to our earlier work on maximal traces and path-based logics \cite{cirstea-11}, which described a game-theoretic approach to testing if a system with non-deterministic branching is able to exhibit a particular trace. Here we consider arbitrary branching types, and while we do not emphasise the game-theoretic aspect, our use of greatest fixpoints has a very similar thrust.

\paragraph{Acknowledgements} Several fruitful discussions with participants at the 2012 Dagstuhl Seminar on Coalgebraic Logics helped refine the ideas presented here. Our use of relation lifting was inspired by the recent work on coinductive predicates \cite{Hasuo12}, itself based on the seminal work in \cite{HermidaJ98} on the use of predicate and relation lifting in the formalisation of induction and coinduction principles. Last but not least, the comments received from the anonymous reviewers contributed to improving the presentation of this work and to identifying new directions for future work.

\section{Preliminaries}

\subsection{Relation Lifting}
\label{rel-lifting}
The concepts of \emph{predicate lifting} and \emph{relation lifting}, to our knowledge first introduced in \cite{HermidaJ98}, are by now standard tools in the study of coalgebraic models, used e.g.~to provide an alternative definition of the notion of bisimulation (see e.g.~in \cite{JacobsBook}), or to describe the semantics of coalgebraic modal logics \cite{Pattinson03,Moss99}. While these concepts are very general, their use so far usually restricts this generality by viewing both predicates and relations as sub-objects in some category (possibly carrying additional structure). In this paper, we make use of the full generality of these concepts, and move from the standard view of relations as subsets to a setting where relations are valuations into a universe of truth values. This section recalls the definition of relation lifting in the standard setting where relations are given by monomorphic spans.

Throughout this section (only), {${\mathsf{Rel}}$} denotes the category whose objects are binary relations $(R,\langle r_1,r_2 \rangle)$ with $\langle r_1,r_2 \rangle : R \to X \times Y$ a monomorphic span, and whose arrows from $(R,\langle r_1,r_2 \rangle)$ to $(R',\langle r_1',r_2' \rangle)$ are given by pairs of functions $(f : X \to X'\,,\, g : Y \to Y')$ ~s.t.~ $(f \times g) \circ  \langle r_1,r_2 \rangle$ factors through $\langle r_1',r_2' \rangle$:
\[\UseComputerModernTips\xymatrix{
R \ar@{-->}[d] \ar@{>->}[r]^-{\langle r_1,r_2 \rangle} & X \times Y \ar[d]^-{f \times g} \\
R' \ar@{>->}[r]^-{\langle r_1',r_2' \rangle} & X' \times Y' }\]
In this setting, the \emph{relation lifting of a functor $F : {\mathsf{Set}} \to {\mathsf{Set}}$} is defined as a functor ${\mathsf{Rel}}(F) : {\mathsf{Rel}} \to {\mathsf{Rel}}$ taking a relation $\langle r_1,r_2 \rangle : R \to X \times Y$ to the relation defined by the span $\langle F(r_1),F(r_2) \rangle : F(R) \to F(X) \times F(Y)$, obtained via the unique epi-mono factorisation of $\langle F(r_1),F(r_2) \rangle$:
\[\UseComputerModernTips\xymatrix{
R \ar@{>->}[d]_-{\langle r_1,r_2 \rangle} & F(R) \ar[d]_-{\langle F(r_1),F(r_2) \rangle} \ar@{->>}[r] & {\mathsf{Rel}}(F)(R)\ar@{>->} [dl]\\
X \times Y & F(X) \times F(Y)
}\]
It follows easily that this construction is functorial, and in particular preserves the order $\le$ between relations on the same objects given by $(R,\langle r_1,r_2 \rangle) \le (S,\langle s_1,s_2 \rangle)$ if and only if $\langle r_1,r_2 \rangle$ factors through $\langle s_1,s_2 \rangle$:
\[\UseComputerModernTips\xymatrix@+0.3pc{
R \ar@{>-->}[r] \ar@{>->}@<-1.5ex>[rr]_-{\langle r_1,r_2 \rangle} & S \ar@{>->}[r]^-{\langle s_1,s_2 \rangle} & X \times Y}\]
An alternative definition of ${\mathsf{Rel}}(F)$ for $F$ a \emph{polynomial functor} (i.e.~constructed from the identity and constant functors using \emph{finite} products and set-indexed coproducts) can be given by induction on the structure of $F$. We refer the reader to \cite[Section~3.1]{JacobsBook} for details of this definition. An extension of this definition to a more general notion of relation will be given in Section~\ref{gen-rel-lifting}.

\subsection{Coalgebras}
\label{section-coalgebras}
We model state-based, dynamical systems as coalgebras over the category of sets. Given a functor $F : {\mathbb{C}} \to {\mathbb{C}}$ on an arbitrary category, an \emph{$F$-coalgebra} is given by a pair $(C,\gamma)$ with $C$ an object of ${\mathbb{C}}$, used to model the state space, and $\gamma : C \to F C$ a morphism in ${\mathbb{C}}$, describing the one-step evolution of the system states. Then, a canonical notion of observational equivalence between the states of two $F$-coalgebras is provided by the notion of bisimulation. Of the many, and under the assumption that $F$ preserves weak pullbacks, equivalent definitions of bisimulation (see \cite{JacobsBook} for a detailed account), we recall the one based on relation lifting. This applies to coalgebras over the category of sets (as described below), but also more generally to categories with logical factorisation systems (as described in \cite{JacobsBook}). According to this definition, an \emph{$F$-bisimulation} between coalgebras $(C,\gamma)$ and $(D,\delta)$ over ${\mathsf{Set}}$ is a ${\mathsf{Rel}}(F)$-coalgebra:
\[\UseComputerModernTips\xymatrix{
R \ar@{-->}[r] \ar@{>->}[d] & {\mathsf{Rel}}(F)(R)\ar@{>->}[d]\\
X \times Y \ar[r]_-{\gamma \times \delta} & F(X) \times F(Y)
}\]
In the remainder of this section we sketch a coalgebraic generalisation of a well-known partition refinement algorithm for computing \emph{bisimilarity} (i.e.~the largest bisimulation) on finite-state labelled transition systems \cite{KS90}. For an arbitrary endofunctor $F : {\mathsf{Set}} \to {\mathsf{Set}}$ and two finite-state $F$-coalgebras $(C,\gamma)$ and $(D,\delta)$, the generalised algorithm iteratively computes relations $\simeq_i \,\,\subseteq\,\, C \times D$ with $i = 0,1, \ldots$ as follows:
\begin{itemize}
\item $\sim_0 \,=\, C \times D$
\item $\sim_{i+1} \,=\, (\gamma \times \delta)^*({\mathsf{Rel}}(F)(\simeq_i))$ for $i = 0,1,\ldots$
\end{itemize}
where $(\gamma \times \delta)^*$ takes a relation $R \subseteq F C \times F D$ to the relation $\{(c,d) \in C \times D \mid (\gamma(c),\delta(d)) \in R \}$. Thus, in the initial approximation $\simeq_0$ of the bisimilarity relation, all states are related, whereas at step $i+1$ two states are related if and only if their one-step observations are suitably related using the relation $\simeq_i$. Bisimilarity between the coalgebras $(C,\gamma)$ and $(D,\delta)$ thus arises as the greatest fixpoint of a monotone operator on the complete lattice of relations between $C$ and $D$, which takes a relation $R \subseteq C \times D$ to the relation $(\gamma \times \delta)^*({\mathsf{Rel}}(F)(R))$. A similar characterisation of bisimilarity exists for coalgebras with infinite state spaces, but in this case the fixpoint can not, in general, be reached in a finite number of steps.

The above greatest fixpoint characterisation of bisimilarity is generalised and adapted in Section~\ref{linear-time}, in order to characterise the extent to which a state in a coalgebra with branching can exhibit a linear-time behaviour. There, the two coalgebras in question have different types: the former has branching behaviour and is used to model the system of interest, whereas the latter has linear behaviour only and describes the domain of possible traces.

\subsection{Monads}

In what follows, we use monads $({\mathsf{T}},\eta,\mu)$ on ${\mathsf{Set}}$ (where $\eta : {\mathsf{Id}} \Rightarrow {\mathsf{T}}$ and $\mu : {\mathsf{T}} \circ {\mathsf{T}} \Rightarrow {\mathsf{T}}$ are the \emph{unit} and \emph{multiplication} of ${\mathsf{T}}$) to capture branching in coalgebraic types. Moreover, we assume that these monads are \emph{strong} and \emph{commutative}, i.e.~they come equipped with a \emph{strength map} ${\mathsf{st}}_{X,Y} : X \times {\mathsf{T}} Y \to {\mathsf{T}}(X \times Y)$ as well as a \emph{double strength map} ${\mathsf{dst}}_{X,Y} : {\mathsf{T}} X \times {\mathsf{T}} Y \to {\mathsf{T}}(X \times Y)$ for each choice of sets $X,Y$; these maps are natural in $X$ and $Y$, and satisfy coherence conditions w.r.t.~the unit and multiplication of ${\mathsf{T}}$. We also make direct use of the \emph{swapped strength map} ${\mathsf{st}}'_{X,Y} : {\mathsf{T}} X \times Y \to {\mathsf{T}}(X \times Y)$, obtained from the strength via the \emph{twist map} ${\mathsf{tw}}_{X,Y} : X \times Y \to Y \times X$:
\[\UseComputerModernTips\xymatrix@+0.3pc{{\mathsf{T}} X \times Y \ar[r]^-{{\mathsf{tw}}_{{\mathsf{T}} X,Y}} & Y \times {\mathsf{T}} X \ar[r]^-{{\mathsf{st}}_{Y,X}} & {\mathsf{T}}(Y \times X) \ar[r]^-{{\mathsf{T}} {\mathsf{tw}}_{Y,X}} & {\mathsf{T}}(X \times Y)}\]
\begin{example}
\label{example-monads}
As examples of monads, we consider:
\begin{enumerate}
\item the \emph{powerset monad} ${\mathcal{P}} : {\mathsf{Set}} \to {\mathsf{Set}}$, modelling nondeterministic computations, with unit given by singletons and multiplication given by unions. Its strength and double strength are given by
\begin{align*}
{\mathsf{st}}_{X,Y}(x,V) = \{x\} \times V & &
{\mathsf{dst}}_{X,Y}(U,V) = U \times V
\end{align*}
for $x\in X$, $U \in {\mathcal{P}} X$ and $V \in {\mathcal{P}} Y$,
\item the \emph{semiring monad} ${\mathsf{T}}_S : {\mathsf{Set}} \to {\mathsf{Set}}$ with $(S,+,0,\bullet,1)$ a semiring, given by
\[{\mathsf{T}}_S(X) = \{ f : X \to S \mid \sup(f) \text{ is finite} \}\]
with $\sup(f) = \{ x \in X \mid f(x) \ne 0\}$ the \emph{support} of $f$. Its unit and multiplication are given by
\begin{align*}
\eta_X(x)(y) = \begin{cases}1 & \text{ if } y = x \\
0 & \text{ otherwise} \end{cases} & & \mu_X(f \in S^{(S^X)}) = \sum\limits_{g \in \sup(f)}\sum\limits_{x\in \sup(g)} f(g) \bullet g(x)
\end{align*}
while its strength and double strength are given by
\begin{align*}
{\mathsf{st}}_{X,Y}(x,g)(z,y) = \begin{cases} g(y) & \text{ if } z = x \\
0 & \text{ otherwise}
\end{cases} & & {\mathsf{dst}}_{X,Y}(f,g)(z,y) = f(z) \bullet g(y)
\end{align*}
for $x \in X$, $f\in {\mathsf{T}}_S(X)$, $g \in {\mathsf{T}}_S(Y)$, $z \in X$ and $y \in Y$.
As a concrete example, we will consider the semiring $W = (\mathbb N^\infty,\min,\infty,+,0)$, and use ${\mathsf{T}}_W$ to model weighted computations.
\item the \emph{sub-probability distribution monad} ${\mathcal{S}} : {\mathsf{Set}} \to {\mathsf{Set}}$, modelling probabilistic computations, with unit given by the Dirac distributions (i.e.~$\eta_{X}(x) = (x \mapsto 1)$), and multiplication given by $\mu_X(\Phi) = \sum\limits_{\varphi \in \sup(\Phi)}\sum\limits_{x \in \sup(\varphi)} \Phi(\varphi) * \varphi(x)$, with $*$ denoting multiplication on $[0,1]$. Its strength and double strength are given by
\begin{align*}
{\mathsf{st}}_{X,Y}(x,\psi)(z,y) = \begin{cases}\psi(y) & \text{if } z = x\\ 0 & \text{otherwise}
\end{cases}  & &
{\mathsf{dst}}_{X,Y}(\varphi,\psi)(z,y) = \varphi(z) * \psi(y)
\end{align*}
for $x \in X$, $\varphi \in {\mathcal{S}}(X)$, $\psi \in {\mathcal{S}}(Y)$, $z \in X$ and $y \in Y$.
\end{enumerate}
\end{example}

\section{From Partially Additive, Commutative Monads to Partial Commutative Semirings with Order}
\label{semiring}

Later in this paper we will consider coalgebras whose type is given by the composition of several endofunctors on ${\mathsf{Set}}$, one of which is a commutative monad ${\mathsf{T}} : {\mathsf{Set}} \to {\mathsf{Set}}$ accounting for the presence of branching in the systems of interest. This section extends results in \cite{Kock2011,CoumansJ2011} to show how to derive a universe of truth values from such a monad. The assumption of loc.\,cit.~concerning the \emph{additivity} of the monad under consideration is here weakened to \emph{partial additivity} (see Definition~\ref{additive}); this allows us to incorporate the sub-probability distribution monad (which is not additive) into our framework. Specifically, we show that any commutative, partially additive monad ${\mathsf{T}} : {\mathsf{Set}} \to {\mathsf{Set}}$ induces a partial commutative semiring structure on the set ${\mathsf{T}} 1$, with $1=\{*\}$ a final object in ${\mathsf{Set}}$. We recall that a \emph{commutative semiring} consists of a set $S$ carrying two commutative monoid structures $(+,0)$ and $(\bullet, 1)$, with the latter distributing over the former: $s \bullet 0 = 0$ and $s \bullet (t + u) = s \bullet t + s \bullet u$ for all $s,t,u \in S$. A \emph{partial commutative semiring} is defined similarly, except that $+$ is a partial operation subject to the condition that whenever $t + u$ is defined, so is $s \bullet t + s \bullet u$, and moreover $s \bullet (t + u) = s \bullet t + s \bullet u$. The relevance of a partial commutative semiring structure on the set of truth values will become clear in Sections~\ref{gen-rel-lifting} and \ref{linear-time}.

It follows from results in \cite{CoumansJ2011} that any commutative monad $({\mathsf{T}},\eta,\mu)$ on ${\mathsf{Set}}$ induces a commutative monoid $({\mathsf{T}}(1),\bullet,\eta_1(*))$, with multiplication $\bullet : {\mathsf{T}}(1) \times {\mathsf{T}}(1) \to {\mathsf{T}}(1)$ given by the composition
\[\UseComputerModernTips\xymatrix@+1pc{{\mathsf{T}} (1) \times {\mathsf{T}} (1) \ar[r]^-{{\mathsf{dst}}_{1,1}} & {\mathsf{T}}(1 \times 1) \ar[r]^-{{\mathsf{T}} \pi_2} & {\mathsf{T}} (1)}\]
Alternatively, this multiplication can be defined as the composition
\[\UseComputerModernTips\xymatrix{{\mathsf{T}}(1) \times {\mathsf{T}}(1) \ar[r]^-{{\mathsf{st}}'_{1,1}} & {\mathsf{T}}(1 \times {\mathsf{T}}(1)) \ar[r]^-{T \pi_2} & {\mathsf{T}}^2(1) \ar[r]^-{\mu_1} & {\mathsf{T}}(1)}\]
or as
\[\UseComputerModernTips\xymatrix{{\mathsf{T}}(1) \times {\mathsf{T}}(1) \ar[r]^-{{\mathsf{st}}_{1,1}} & {\mathsf{T}}({\mathsf{T}}(1) \times 1) \ar[r]^-{T \pi_1} & {\mathsf{T}}^2(1) \ar[r]^-{\mu_1} & {\mathsf{T}}(1)}\]
(While the previous two definitions coincide for commutative monads, this is not the case in general.)
\begin{remark}
\label{actions}
The following maps define left and right actions of $({\mathsf{T}}(1),\bullet)$ on ${\mathsf{T}}( X)$:
\[\UseComputerModernTips\xymatrix{
{\mathsf{T}}(1) \times {\mathsf{T}}(X) \ar[r]^-{{\mathsf{dst}}_{1,X}} & {\mathsf{T}}(1 \times X) \ar[r]^-{{\mathsf{T}} \pi_2} & {\mathsf{T}} (X)
} \qquad \qquad \UseComputerModernTips\xymatrix{
{\mathsf{T}}(X) \times {\mathsf{T}}(1) \ar[r]^-{{\mathsf{dst}}_{X,1}} & {\mathsf{T}}(X \times 1) \ar[r]^-{{\mathsf{T}} \pi_1} & {\mathsf{T}} (X)
}\]
\end{remark}

On the other hand, any monad ${\mathsf{T}} :  {\mathsf{Set}} \to {\mathsf{Set}}$ with {$ {\mathsf{T}} \emptyset = 1$} is such that, for any $X$, ${\mathsf{T}} X$ has a \emph{zero element} $0 \in {\mathsf{T}} X$, obtained as $({\mathsf{T}}!_X)(*)$. This yields a \emph{zero map $0 : Y \to {\mathsf{T}} X$} for any $X,Y$, obtained as the composition
\[\UseComputerModernTips\xymatrix{Y \ar[r]^-{!_Y} &  {\mathsf{T}} \emptyset \ar[r]^-{T !_X} & {\mathsf{T}} X}\]
with the maps $!_Y : Y \to {\mathsf{T}} \emptyset$ and $!_X : \emptyset \to X$ arising by finality and initiality, respectively. Now consider the following map:
\begin{equation}
\label{map}
\UseComputerModernTips\xymatrix@+4pc{T(X+Y) \ar[r]^-{\langle \mu_X \circ {\mathsf{T}} p_1,\mu_Y \circ {\mathsf{T}} p_2 \rangle} & {\mathsf{T}} X \times {\mathsf{T}} Y}
\end{equation}
where $p_1 = [\eta_X,0] : X + Y \to {\mathsf{T}} X$ and $p_2 = [0,\eta_Y] : X + Y \to {\mathsf{T}} Y$.
\begin{definition}
\label{additive}
A monad ${\mathsf{T}} : {\mathsf{Set}} \to {\mathsf{Set}}$ is called \emph{additive}\footnote{Additive monads were studied in \cite{Kock2011,CoumansJ2011}.} (\emph{partially additive}) if\, ${\mathsf{T}} \emptyset = 1$ and the map in (\ref{map}) is an isomorphism (respectively monomorphism).
\end{definition}
 The (partial) inverse of the map $\langle \mu_X \circ {\mathsf{T}} p_1,\mu_Y \circ {\mathsf{T}} p_2 \rangle$ can be used to define a (partial) addition on the set ${\mathsf{T}} X$, given by ${\mathsf{T}}[1_X,1_X] \circ q_{X,X}$, where $q_{X,X} : {\mathsf{T}} X \times {\mathsf{T}} X \to {\mathsf{T}}(X+X)$ is the (partial) left inverse of $\langle \mu_X \circ {\mathsf{T}} p_1,\mu_Y \circ {\mathsf{T}} p_2 \rangle$:
 \[\UseComputerModernTips\xymatrix@+1pc{{\mathsf{T}} X & {\mathsf{T}}(X+X) \ar@<+1ex>[rr]^-{\langle \mu_X \circ {\mathsf{T}} p_1,\mu_Y \circ {\mathsf{T}} p_2 \rangle} \ar[l]_-{{\mathsf{T}}[1_X,1_X]} & & {\mathsf{T}} X \times {\mathsf{T}} X \ar@<+1ex>@{-->}[ll]^-{q_{X,X}} \ar@<+1ex>@/^3ex/[lll]^-{{+}}}\]
That is, $a + b$ is defined if and only if $(a,b) \in \Im(\langle \mu_X \circ {\mathsf{T}} p_1,\mu_Y \circ {\mathsf{T}} p_2 \rangle)$\,\footnote{A similar, but \emph{total}, addition operation is defined in \cite{Kock2011,CoumansJ2011} for additive monads.}.

\cite[Section~5.2]{CoumansJ2011} explores the connection between additive, commutative monads and commutative semirings. The next result provides a generalisation to partially additive, commutative monads and partial commutative semirings.

The proof of Proposition~\ref{prop-partial} is a slight adaptation of the corresponding proofs in \cite[Section~5.2]{CoumansJ2011}.
\begin{proposition}
\label{prop-partial}
Let ${\mathsf{T}}$ be a commutative, (partially) additive monad. Then:
\begin{enumerate}
\item $({\mathsf{T}} 1,\bullet,\eta_1(*))$ is a commutative monoid.
\item $({\mathsf{T}} X,0,+)$ is a (partial) commutative monoid, for each set $X$.
\item\label{3}  $({\mathsf{T}} 1,0,+,\bullet,\eta_1(*))$ is a (partial) commutative semiring.
\end{enumerate}
\end{proposition}
\begin{proof}[Proof (Sketch)]
The commutativity of the following diagram lies at the heart of the proof of item \ref{3}:
 \[\UseComputerModernTips\xymatrix{
 {\mathsf{T}} 1 \times {\mathsf{T}} 1 \ar[dd]_-{\bullet} & & {\mathsf{T}} (1+1) \times {\mathsf{T}} 1 \ar[ll]_-{{\mathsf{T}}[1_X,1_X] \times 1_{{\mathsf{T}} 1}} \ar[dd]_-{a_{{\mathsf{T}}(1+1)}} \ar@<+1ex>[r]^-{\delta \times 1_{{\mathsf{T}} 1} }& ({\mathsf{T}} 1 \times {\mathsf{T}} 1) \times {\mathsf{T}} 1 \ar@<+1ex>@{-->}[l]^-{q_{1,1} \times 1_{{\mathsf{T}} 1}} \ar[d]^-{\langle \pi_1 \times \pi_2,\pi_2 \times \pi_2 \rangle} \\
 & & & ({\mathsf{T}} 1 \times {\mathsf{T}} 1) \times ({\mathsf{T}} 1 \times {\mathsf{T}} 1) \ar[d]^-{\bullet \,\times\, \bullet}\\
 {\mathsf{T}} 1 & & {\mathsf{T}}(1 + 1) \ar@<+1ex>[r]^-{\delta} \ar[ll]^-{{\mathsf{T}}[1_X,1_X]} & {\mathsf{T}} 1 \times {\mathsf{T}} 1\ar@<+1ex>@{-->}[l]^-{q_{1,1}}
 }\]
 where $a_{{\mathsf{T}} X} : {\mathsf{T}} X \times {\mathsf{T}} 1 \to {\mathsf{T}} X$ is the right action from Remark~\ref{actions}, and $\delta$ is the map $\langle \mu_1 \circ {\mathsf{T}} p_1,\mu_1 \circ {\mathsf{T}} p_2 \rangle$ used in the definition of $+$ on ${\mathsf{T}} 1$. The composition $\bullet \circ ({\mathsf{T}}[1_X,1_X] \times 1_{{\mathsf{T}} 1}) \circ (q_{1,1} \times 1_{{\mathsf{T}} 1})$ captures the computation of $(a + b) \bullet c$, whereas the composition ${\mathsf{T}}[1_X,1_X] \circ q_{1,1} \circ (\bullet \times \bullet) \circ \langle \pi_1 \times \pi_2,\pi_2 \times \pi_2 \rangle$ captures the computation $a \bullet c + b \bullet c$, with $a,b,c \in {\mathsf{T}} 1$. The fact that $\delta$ commutes with the strength map (by (iv) of \cite[Lemma~15]{CoumansJ2011}), together with $a_{{\mathsf{T}}(1+1)}$ and $\bullet$ being essentially given by the double strength maps ${\mathsf{dst}}_{1+1,1}$ and ${\mathsf{dst}}_{1,1}$, yields $(\bullet \times \bullet) \circ \langle \pi_1 \times \pi_2,\pi_2 \times \pi_2 \rangle \circ (\delta \times 1_{{\mathsf{T}} 1}) = \delta \circ a_{{\mathsf{T}}(1+1)}$, that is, commutativity (via the plain arrows) of the right side of the above diagram. This immediately results in $a \bullet c + b \bullet c$ being defined whenever $a + b$ is defined, and hence in the commutativity of the right side of the diagram also via the dashed arrows. This, combined with the commutativity of the left side of the diagram (which is simply naturality of the right action $a$), gives  $(a + b) \bullet c = a \bullet c + b \bullet c$ whenever $a+b$ is defined.
\end{proof}
\begin{example}
\label{example-semirings}
For the monads in Example~\ref{example-monads}, one obtains the commutative semirings $(\{\bot,\top\},\vee,\bot,\wedge,\top)$ when {${\mathsf{T}} = {\mathcal{P}}$}, $({\mathbb N}^\infty,\min,\infty,+,0)$ when {${\mathsf{T}} = {\T}_W$}\,\footnote{This is sometimes called the \emph{tropical semiring}.}, and the partial commutative semiring $([0,1],+,0,*,1)$ when ${\mathsf{T}} = {\mathcal{S}}$ (where in the latter case $a + b$ is defined if and only if $a + b \le 1$).
\end{example}

\section{Generalised Relations and Relation Lifting}
\label{gen-rel-lifting}

This section introduces generalised relations valued in a partial commutative semiring, and shows how to lift polynomial endofunctors on ${\mathsf{Set}}$ to the category of generalised relations. We begin by fixing a partial commutative semiring $(S,+,0,\bullet,1)$, and noting that the partial monoid $(S,+,0)$ can be used to define a preorder relation on $S$ as follows:
\[x \sqsubseteq y ~~~\text{if and only if}~~~ \text{there exists } z \in S \text{ such that }x + z = y\]
for $x,y \in S$. It is then straightforward to show (using the definition of a partial commutative semiring) that the preorder $\sqsubseteq$ has $0 \in S$ as bottom element, and is preserved by $\bullet$ in each argument. Proper (i.e.~not partial) semirings where the preorder $\sqsubseteq$ is a partial order are called \emph{naturally ordered} \cite{EsikK07}. We here extend this terminology to partial semirings.

\begin{example}
\label{example-orders}
For the monads in Example~\ref{example-monads}, the preorders associated to the induced partial semirings (see Example~\ref{example-semirings}) are all partial orders: $\le$ on $\{\bot,\top\}$ for ${\mathsf{T}} = {{\mathcal{P}}}$, $\le$ on $[0,1]$ for ${\mathsf{T}} = {{\mathcal{S}}}$, and $\ge$ on ${\mathbb N}^\infty$ for ${\mathsf{T}} = {\T}_W$.
\end{example}

We let ${\mathsf{Rel}}$ denote the category\footnote{To keep notation simple, the dependency on $S$ is left implicit.} with objects given by triples $(X,Y,R)$, where $R : X \times Y \to S$ is a function defining a \emph{multi-valued relation} (or \emph{$S$-relation}), and with arrows from $(X,Y,R)$ to $(X',Y',R')$ given by pairs of functions $(f,g)$ as below, such that $R \sqsubseteq R' \circ (f \times g)$:
\[\UseComputerModernTips\xymatrix{X \times Y \ar@{}[dr]|-{\sqsubseteq}\ar[r]^-{f \times g} \ar[d]_-{R} & X' \times Y' \ar[d]^-{R'}\\ S \ar@{=}[r] & S}\]
Here, the order $\sqsubseteq$ on $S$ has been extended pointwise to $S$-relations with the same carrier.

We write ${\mathsf{Rel}}_{X,Y}$ for the \emph{fibre over $(X,Y)$}, that is, the full subcategory of ${\mathsf{Rel}}$ whose objects are $S$-relations over $X \times Y$ and whose arrows are given by $(1_X,1_Y)$. It is straightforward to check that the functor $q : {\mathsf{Rel}} \to {\mathsf{Set}} \times {\mathsf{Set}}$ taking $(X,Y,R)$ to $(X,Y)$ defines a fibration: the reindexing functor $(f,g)^* : {\mathsf{Rel}}_{X',Y'} \to {\mathsf{Rel}}_{X,Y}$ takes $R' : X' \times Y' \to S$ to $R' \circ (f \times g) : X \times Y \to S$.

We now proceed to generalising relation lifting to $S$-relations.

\begin{definition}
\label{def-gen-rel-lifting}
Let $F : {\mathsf{Set}} \to {\mathsf{Set}}$. A \emph{relation lifting of $F$} is a functor\footnote{Given the definition of the fibration $q$, such a functor is automatically a morphism of fibrations.} $\Gamma : {\mathsf{Rel}} \to {\mathsf{Rel}}$ such that $q \circ \Gamma = (F \times F) \circ q$:
\[\UseComputerModernTips\xymatrix{
{\mathsf{Rel}} \ar[d]_-{q} \ar[r]^-{\Gamma} & {\mathsf{Rel}} \ar[d]^-{q} \\
{\mathsf{Set}} \times {\mathsf{Set}} \ar[r]_-{F \times F} & {\mathsf{Set}} \times {\mathsf{Set}}}\]
\end{definition}
We immediately note a fundamental difference compared to standard relation lifting as defined in Section~\ref{rel-lifting}. While in the case of standard relations each functor admits exactly one lifting, Definition~\ref{def-gen-rel-lifting} implies neither the existence nor the uniqueness of a lifting. We defer the study of a canonical lifting (similar to ${\mathsf{Rel}}(F)$ in the case of standard relations) to future work, and show how to define a relation lifting of $F$ in the case when $F$ is a polynomial functor. To this end, we make the additional assumption that the unit $1$ of the semiring multiplication is a top element (which we also write as $\top$) for the preorder $\sqsubseteq$. Recall that $\sqsubseteq$ also has a bottom element (which we will sometimes denote by $\bot$), given by the unit $0$ of the (partial) semiring addition. The definition of the relation lifting of a polynomial functor $F$ is by structural induction on $F$ and makes use of the semiring structure on $S$:
\begin{itemize}
\item If $F = {\mathsf{Id}}$, ${\mathsf{Rel}}(F)$ takes an $S$-relation to itself.
\item If $F = C$, ${\mathsf{Rel}}(F)$ takes an $S$-relation to the equality relation ${\mathsf{Eq}}(C) : C \times C \to S$ given by
\[{\mathsf{Eq}}_C(c,c') ~=~ \begin{cases} \top \text{ if }c = c' \\
\bot \text{ otherwise} \end{cases}\]
\item If $F = F_1 \times F_2$,  ${\mathsf{Rel}}(F)$ takes an $S$-relation $R : X \times Y \to S$ to:
\[\!\!\!\!\!\!\UseComputerModernTips\xymatrix@-1.1pc{(F_1 X \times F_2 X) \times (F_1 Y \times F_2 Y) \ar[rrr]^-{\langle \pi_1 \times \pi_1,\pi_2 \times \pi_2 \rangle} & & & (F_1 X \times F_1 Y) \times (F_2 X \times F_2 Y) \ar[rrrrr]^-{{\mathsf{Rel}}(F_1)(R) \times {\mathsf{Rel}}(F_2)(R)} & & & & & S \times S \ar[r]^-{\bullet} & S}\]
The functoriality of this definition follows from the preservation of $\sqsubseteq$ by $\bullet$ (see Section~\ref{semiring}).
\item if $F = F_1 + F_2$, ${\mathsf{Rel}}(F)(R) : (F_1 X + F_2 X) \times (F_1 Y + F_2 Y ) \to S$ is defined by case analysis:
\begin{align*}
{\mathsf{Rel}}(F)(R)(\iota_i(u),\iota_j(v)) & ~=~ \begin{cases} {\mathsf{Rel}}(F_i)(R)(u,v) & \text{ if } i = j\\
\bot & \text{ otherwise} \end{cases}
\end{align*}
for $i,j \in \{1,2\}$, $u \in F_i X$ and $v \in F_j Y$. This definition generalises straightforwardly from binary to set-indexed coproducts.
\end{itemize}

\begin{remark}
A more general definition of relation lifting, which applies to arbitrary functors on ${\mathsf{Set}}$, is outside the scope of this paper. We note in passing that such a relation lifting could be defined by starting from a \emph{generalised predicate lifting} $\delta : F \circ {\mathsf{P}}_0 \Rightarrow {\mathsf{P}}_0 \circ F$ for the functor $F$, similar to the predicate liftings used in the work on coalgebraic modal logic \cite{Pattinson03}. Here, the contravariant functor ${\mathsf{P}}_0 : {\mathsf{Set}} \to {\mathsf{Set}}^{\mathsf{op}}$ takes a set $X$ to the hom-set ${\mathsf{Set}}(X,S)$. Future work will also investigate the relevance of the results in \cite{Ghani2011,Ghani2012} to a general definition of relation lifting in our setting. Specifically, the work in loc.\,cit.~shows how to construct truth-preserving predicate liftings and equality-preserving relation liftings for arbitrary functors on the base category of a \emph{Lawvere fibration}, to the total category of that fibration.
\end{remark}
For the remainder of this paper, we take $(S,+,0,\bullet,1)$ to be the partial semiring derived in Section~\ref{semiring} from a commutative, partially additive monad ${\mathsf{T}}$, and we view $S$ as the set of truth values.
In the case of the powerset monad, this corresponds to the standard view of relations as subsets, whereas in the case of the sub-probability distribution monad, this results in relations given by valuations in the interval $[0,1]$.

\begin{example}
Let $F: {\mathsf{Set}} \to {\mathsf{Set}}$ be given by $F X = 1 + A \times X$, with $A$ a set (of labels), and let $(S,+,0,\bullet,1)$ be the partial semiring with carrier ${\mathsf{T}} 1$ defined in Section~\ref{semiring}.
\begin{itemize}
\item For ${\mathsf{T}} = {\mathcal{P}}$, ${\mathsf{Rel}}(F)$ takes a (standard) relation $R \subseteq X \times Y$ to the relation
\[\{(\iota_1(*),\iota_1(*)\} \cup \{((a,x),(a,y)) \mid a \in A, (x,y) \in R \}\]
\item For ${\mathsf{T}} = {\mathcal{S}}$, ${\mathsf{Rel}}(F)$ takes $R : X \times Y \to [0,1]$ to the relation $R' : F X \times F Y \to [0,1]$ given by
\[ R'(\iota_1(*),\iota_1(*)) = 1 ~\qquad~
R'((a,x),(a,y)) = R(x,y) ~\qquad~
R'(u,v) = 0 ~\text{ in all other cases}
\]
\item For ${\mathsf{T}} = {\T}_W$, ${\mathsf{Rel}}(F)$ takes $R : X \times Y \to \mathbb N^\infty$ to the relation $R' : F X \times F Y \to \mathbb N^\infty$ given by
\[
R'(\iota_1(*),\iota_1(*)) = 0 ~\qquad~
R'((a,x),(a,y)) = R(x,y) ~\qquad~
R'(u,v) = \infty ~\text{ in all other cases}
\]
\end{itemize}
\end{example}

\section{From Bisimulation to Traces}
\label{linear-time}

Throughout this section we fix a commutative, partially additive monad ${\mathsf{T}} : {\mathsf{Set}} \to {\mathsf{Set}}$ and assume, as in the previous section, that the natural preorder $\sqsubseteq$ induced by the partial commutative semiring obtained in Section~\ref{semiring} has the multiplication unit $\eta_1(*) \in {\mathsf{T}} 1$ as top element. Furthermore, we assume that this preorder is an \emph{$\omega^{{\mathsf{op}}}$-chain complete} partial order, where $\omega^{{\mathsf{op}}}$-chain completeness amounts to  any decreasing chain $x_1 \sqsupseteq x_2 \sqsupseteq \ldots$ having a greatest lower bound $\sqcap_{i \in \omega} x_i$. These assumptions are clearly satisfied by the orders in Example~\ref{example-orders}.

We now show how combining the liftings of polynomial functors to the category of generalised relations valued in the partial semiring ${\mathsf{T}} 1$ (as defined in Section~\ref{gen-rel-lifting}) with so-called \emph{extension liftings} which arise canonically from the monad ${\mathsf{T}}$, can be used to give an account of the linear-time behaviour of a state in a coalgebra with branching. The type of such a coalgebra can be any composition involving polynomial endofunctors and the branching monad ${\mathsf{T}}$, although compositions of type ${\mathsf{T}} \circ F$, $G \circ {\mathsf{T}}$ and $G \circ {\mathsf{T}} \circ F$ with $F$ and $G$ polynomial endofunctors are particularly emphasised in what follows.

We begin with some informal motivation. When ${\mathsf{Rel}}$ is the standard category of binary relations, recall from Section~\ref{section-coalgebras} that an $F$-bisimulation is simply a ${\mathsf{Rel}}(F)$-coalgebra, and that the largest $F$-bisimulation between two $F$-coalgebras $(C,\gamma)$ and $(D,\delta)$ can be obtained as the greatest fixpoint of the monotone operator  on ${\mathsf{Rel}}_{C \times D}$ which takes a relation $R$ to the relation $(\gamma \times \delta)^*( {\mathsf{Rel}}(F)(R))$. Generalising the notion of $F$-bisimulation from standard relations to ${\mathsf{T}} 1$-relations makes little sense when the systems of interest are $F$-coalgebras. However, when considering say, coalgebras of type ${\mathsf{T}} \circ F$, it turns out that liftings of $F$ to the category of ${\mathsf{T}} 1$-relations (as defined in Section~\ref{gen-rel-lifting}) can be used to describe the \emph{linear-time behaviour} of states in such a coalgebra, when combined with suitable liftings of ${\mathsf{T}}$ to the same category of relations. To see why, let us consider labelled transition systems viewed as coalgebras of type ${\mathcal{P}}(1 + A \times {\mathsf{Id}})$. In such a coalgebra $\gamma : C \to {\mathcal{P}}(1 + A \times C)$, explicit termination is modelled via transitions $c \to \iota_1(*)$, whereas deadlock (absence of a transition) is modelled as $\gamma(c) = \emptyset$. In this case, ${\mathsf{Rel}}({\mathcal{P}}) \circ {\mathsf{Rel}}(1 + A \times {\mathsf{Id}})$ is naturally isomorphic to ${\mathsf{Rel}}({\mathcal{P}}(1 + A \times {\mathsf{Id}}))$\,\footnote{A similar observation holds more generally for ${\mathcal{P}} \circ F$ with $F$ a polynomial endofunctor. In general, only a natural transformation ${\mathsf{Rel}}(F \circ G) \Rightarrow {\mathsf{Rel}}(F) \circ {\mathsf{Rel}}(G)$ exists, see \cite[Exercise~4.4.6]{JacobsBook}.}, and takes a relation $R \subseteq X \times Y$ to the relation $R' \subseteq {\mathcal{P}}(1 + A \times X) \times {\mathcal{P}}(1 + A \times Y)$ given by
\[(U,V) \in R' ~~~\text{ if and only if }~~~ \begin{cases}
\text{if } \iota_1(*) \in U \text{ then } \iota_1(*) \in V , \text{ and conversely}\\
\text{if } (a,x) \in U \text{ then there exists } (a,y) \in V \text{ with } (x,y) \in R, \text{ and conversely}
\end{cases}
\]
Thus, the largest ${\mathcal{P}}(1 + A \times {\mathsf{Id}})$-bisimulation between two coalgebras $(C,\gamma)$ and $(D,\delta)$ can be computed as the greatest fixpoint of the operator on ${\mathsf{Rel}}_{C,D}$ obtained as the composition
\begin{equation}
\label{el}
\UseComputerModernTips\xymatrix@+0.6pc{
R \subseteq C \times D \ar@{|->}[r]^-{{\mathsf{Rel}}(F)} & R_1 \subseteq F C \times F D \ar@{|->}[r]^-{{\mathsf{Rel}}({\mathcal{P}})} & R_2 \subseteq {\mathcal{P}}(F C) \times {\mathcal{P}} (F D) \ar@{|->}[r]^-{(\gamma \times \delta)^*} & R'\subseteq C \times D}
\end{equation}
where $F = 1 + A \times {\mathsf{Id}}$. Note first that ${\mathsf{Rel}}({\mathcal{P}})$ (defined in Section~\ref{rel-lifting} for an arbitrary endofunctor on ${\mathsf{Set}}$) takes a relation $R \subseteq X \times Y$ to the relation $R' \subseteq {\mathcal{P}}(X) \times {\mathcal{P}}(Y)$ given by
\[(U,V) \in R' \text{ ~if and only if~ for all } x \in U \text{ there exists } y \in V \text{ with } (x,y) \in R, \text{ and conversely}\]
Now consider the effect of replacing ${\mathsf{Rel}}({\mathcal{P}})$ in (\ref{el}) with the lifting $L : {\mathsf{Rel}} \to {\mathsf{Rel}}$ that takes a relation $R \subseteq X \times Y$ to the relation $R' \subseteq {\mathcal{P}}(X) \times Y$ given by
\[(U,y) \in R' \text{ ~if and only if~ there exists } x \in U \text{ with } (x,y) \in R\]
To do so, we must change the type of the coalgebra $(D,\delta)$ from ${\mathcal{P}} \circ F$ to just $F$. A closer look at the resulting operator on ${\mathsf{Rel}}_{C,D}$ reveals that it can be used to test for the existence of a matching trace: each state of the $F$-coalgebra $(D,\delta)$ can be associated a \emph{maximal trace}, i.e.~an element of the final $F$-coalgebra, by finality. In particular, when $F = 1 + A \times {\mathsf{Id}}$, maximal traces are either finite or infinite sequences of elements of $A$. Thus, the greatest fixpoint of the newly defined operator on ${\mathsf{Rel}}_{C \times D}$ corresponds to the relation on $C \times D$ given by
\begin{eqnarray*}c \ni_{\mathsf{tr}} d \text{ ~if and only if~  there exists a sequence of choices of transitions starting from } c \in C \text{ that leads to}\\
\qquad \qquad \quad \text{ exactly the same maximal trace (element of $A^* \cup A^\omega)$ as the single trace of } d \in D
\end{eqnarray*}
This relation models the ability of the state $c$ to exhibit the same trace as that of $d$.

The remainder of this section formalises the above intuitions, and generalises them to arbitrary monads ${\mathsf{T}}$ and polynomial endofunctors $F$, as well as to arbitrary compositions involving the monad ${\mathsf{T}}$ and polynomial endofunctors. We begin by restricting attention to coalgebras of type ${\mathsf{T}} \circ F$, with the monad ${\mathsf{T}}$ capturing branching and the endofunctor $F$ describing the structure of individual transitions. In this case it is natural to view the elements of the final $F$-coalgebra as possible \emph{linear-time} observable behaviours of states in ${\mathsf{T}} \circ F$-coalgebras. Similarly to the above discussion, we let $(C,\gamma)$ and $(D,\delta)$ denote a ${\mathsf{T}} \circ F$-coalgebra and respectively an $F$-coalgebra. The lifting of $F$ to ${\mathsf{T}} 1$-relations will be used as part of an operator on ${\mathsf{Rel}}_{C,D}$. In order to generalise the lifting $L$ above to arbitrary monads ${\mathsf{T}}$, we recall the following result from \cite{Kock12}, which assumes a strong monad ${\mathsf{T}}$ on a cartesian closed category.

\begin{proposition}[{\cite[Proposition~4.1]{Kock12}}]
\label{prop-kock}
Let $(B,\beta)$ be a ${\mathsf{T}}$-algebra. For any $f : X \times Y \to B$, there exists a unique $1$-linear $\overline{f} : {\mathsf{T}} X \times Y \to B$ making the following triangle commute:
\[\UseComputerModernTips\xymatrix{
{\mathsf{T}} X \times Y \ar[r]^-{\overline{f}} & B \\
X \times Y \ar[u]^-{\eta_X \times 1_Y} \ar[ur]_-{f}
}\]
\end{proposition}
In the above, \emph{$1$-linearity} is linearity in the first variable. More precisely, for ${\mathsf{T}}$-algebras $(A,\alpha)$ and $(B,\beta)$, a map $f : A \times Y \to B$ is called \emph{$1$-linear} if the following diagram commutes:
\[\UseComputerModernTips\xymatrix{
{\mathsf{T}}(A) \times Y \ar[r]^-{{\mathsf{st}}'_{A,Y}} \ar[d]_-{\alpha \times 1_Y} & {\mathsf{T}}(A \times Y) \ar[r]^-{{\mathsf{T}}(f)} & {\mathsf{T}}(B) \ar[d]^-{\beta}\\
A \times Y \ar[rr]_-{f} & & B
}\]
Clearly $1$-linearity should be expected of the lifting $L(R) : {\mathsf{T}} X \times Y \to \T1$ of a relation $R : X \times Y \to {\mathsf{T}} 1$, as this amounts to $L(R)$ commuting with the ${\mathsf{T}}$-algebra structures $({\mathsf{T}} X,\mu_X)$ and $({\mathsf{T}} 1,\mu_1)$. Given this, the diagram of Proposition~\ref{prop-kock} forces the definition of the generalised lifting.

\begin{definition}
The \emph{extension lifting} $L_{\mathsf{T}} : {\mathsf{Rel}} \to {\mathsf{Rel}}$ is the functor taking a relation $R : X \times Y \to \T1$ to its unique $1$-linear extension $\overline{R} : {\mathsf{T}} X \times Y \to \T1$.
\end{definition}

\begin{remark}
It follows from \cite{Kock12} that a direct definition of the relation $\overline{R} : {\mathsf{T}} X \times Y \to {\mathsf{T}} 1$ is as the composition
\[\UseComputerModernTips\xymatrix@+0.3pc{
{\mathsf{T}} X \times Y \ar[r]^-{{\mathsf{st}}'_{X,Y}} & {\mathsf{T}}(X \times Y) \ar[r]^-{{\mathsf{T}} (R)} & {\mathsf{T}}^2 1 \ar[r]^-{\mu_1} & {\mathsf{T}} 1
}\]
This also yields functoriality of $L_{\mathsf{T}}$, which follows from the functoriality of its restriction to each fibre category ${\mathsf{Rel}}_{X,Y}$, as proved next.
\end{remark}

\begin{proposition}
\label{prop-functoriality}
The mapping $R \in {\mathsf{Rel}}_{X,Y} \mapsto \overline{R} \in {\mathsf{Rel}}_{{\mathsf{T}} X,Y}$ is functorial.
\end{proposition}
\begin{proof}[Proof (Sketch)]
Let $R,R' \in {\mathsf{Rel}}_{X,Y}$ be such that $R \sqsubseteq R'$. Hence, there exists $S \in {\mathsf{Rel}}_{X,Y}$ such that $R + S = R'$ (pointwise). To show that $\overline{R} \sqsubseteq \overline{R'}$, it suffices to show that $\mu_1 \circ {\mathsf{T}}(R) \sqsubseteq \mu_1 \circ {\mathsf{T}}(R')$ (pointwise). To this end, we note that commutativity of the map $\delta$ with the monad multiplication, proved in \cite[Lemma~15\,(iii)]{CoumansJ2011} and captured by the commutativity of the lower diagram below (via the plain arrows)
\[\UseComputerModernTips\xymatrix{
{\mathsf{T}}^2 1 \ar[r]^-{\mu_1} & {\mathsf{T}} 1\\
{\mathsf{T}}^2(1 + 1) \ar[r]^-{\mu_{1 + 1}} \ar@<+1ex>[d]^-{{\mathsf{T}} \delta} \ar[u]^-{{\mathsf{T}}^2 !} & {\mathsf{T}}(1 + 1) \ar@<-1ex>[dd]_-{\delta} \ar[u]^-{{\mathsf{T}} !}\\
{\mathsf{T}}({\mathsf{T}} 1 \times {\mathsf{T}} 1) \ar[d]^-{\langle {\mathsf{T}} \pi_1,{\mathsf{T}} \pi_2 \rangle} \ar@<+1ex>@{-->}[u]^-{{\mathsf{T}} q_{1,1}}\\
{\mathsf{T}}^2 1 \times {\mathsf{T}}^2 1 \ar[r]_-{\mu_1 \times \mu_1}  & {\mathsf{T}} 1 \times {\mathsf{T}} 1 \ar@<-1ex>@{-->}[uu]_-{q_{1,1}}
}\]
also yields commutativity of the whole diagram (via the dashed arrows). This formalises the commutativity of $+$ (defined as ${\mathsf{T}} ! \circ q_{1,1}$) with the monad multiplication. Now pre-composing this commutative diagram (dashed arrows) with the map
\[\UseComputerModernTips\xymatrix{
{\mathsf{T}}(X \times Y) \ar[r] & {\mathsf{T}}({\mathsf{T}} 1 \times {\mathsf{T}} 1)
}\]
given by the image under ${\mathsf{T}}$ of the map $(x,y) \mapsto \langle R(x,y),S(x,y) \rangle$ yields
\[(\mu_1 \circ {\mathsf{T}}(R)) + (\mu_1 \circ {\mathsf{T}}(S)) = \mu_1 \circ {\mathsf{T}}(R+S) = \mu_1 \circ {\mathsf{T}} R'\]
and therefore, using the definition of $\sqsubseteq$, $\mu_1 \circ {\mathsf{T}}(R) \sqsubseteq \mu_1 \circ {\mathsf{T}}(R')$. This concludes the proof.
\end{proof}

Thus, $L_{\mathsf{T}}$ is a functor making the following diagram commute:
\[\UseComputerModernTips\xymatrix{
{\mathsf{Rel}} \ar[d]_-{q} \ar[r]^-{L_{\mathsf{T}}} & {\mathsf{Rel}} \ar[d]^-{q} \\
{\mathsf{Set}} \times {\mathsf{Set}} \ar[r]_-{{\mathsf{T}} \times {\mathsf{Id}}} & {\mathsf{Set}} \times {\mathsf{Set}}}\]

We are finally ready to give an alternative account of maximal traces of ${\mathsf{T}} \circ F$-coalgebras.

\begin{definition}
\label{max-trace-map}
Let $(C,\gamma)$ denote a ${\mathsf{T}} \circ F$-coalgebra, and let $(Z,\zeta)$ denote the final $F$-coalgebra. The \emph{maximal trace map ${\mathsf{tr}}_\gamma : C \to ({\mathsf{T}} 1)^Z$ of $\gamma$} is the exponential transpose of the greatest fixpoint $R : C \times Z \to \T1$ of the operator ${\mathcal{O}} : {\mathsf{Rel}}_{C,Z} \to {\mathsf{Rel}}_{C,Z}$ given by the composition
\[\UseComputerModernTips\xymatrix@+0.3pc{
{\mathsf{Rel}}_{C,Z} \ar[r]^-{{\mathsf{Rel}}(F)} & {\mathsf{Rel}}_{F C,F Z} \ar[r]^-{L_{\mathsf{T}}} & {\mathsf{Rel}}_{{\mathsf{T}}(F C), F Z} \ar[r]^-{(\gamma \times \zeta)^*} & {\mathsf{Rel}}_{C,Z}
}\]
\end{definition}
The above definition appeals to the existence of least fixpoints in chain-complete partial orders, as formalised in the following fixpoint theorem from \cite{Priestley2002}.
\begin{theorem}[{\cite[8.22]{Priestley2002}}]
Let $P$ be a complete partial order and let ${\mathcal{O}} : P \to P$ be order-preserving. Then ${\mathcal{O}}$ has a least fixpoint.
\end{theorem}
Definition~\ref{max-trace-map} makes use of this result applied to the \emph{dual} of the order $\sqsubseteq$. Our assumption that $\sqsubseteq$ is $\omega^{\mathsf{op}}$-chain complete makes the dual order a complete partial order. Monotonicity of the operator in Definition~\ref{max-trace-map} is an immediate consequence of the functoriality of ${\mathsf{Rel}}(F)$, $L_{\mathsf{T}}$ and $(\gamma \times \delta)^*$.

\cite{Priestley2002} also gives a construction for the least fixpoint of an order-preserving operator on a complete partial order, which involves taking a limit over an ordinal-indexed chain. Instantiating this construction to the dual of the order $\sqsubseteq$ yields an ordinal-indexed sequence of relations $(R_\alpha)$, where:
\begin{itemize}
\item $R_0 = \top$ (i.e.~the relation on $C \times D$ given by $(c,d) \mapsto 1$),
\item $R_{\alpha+1} = {\mathcal{O}}(R_\alpha)$,
\item $R_\alpha = \sqcap_{\beta < \alpha} R_{\beta}$, if $\alpha$ is a limit ordinal.
\end{itemize}
\begin{remark}
While in the case ${\mathsf{T}} = {\mathcal{P}}$, restricting to finite-state coalgebras $(C,\gamma)$ and $(D,\delta)$ results in the above sequence of relations stabilising in a finite number of steps, for ${\mathsf{T}} = {\mathcal{S}}$ or $T = {\T}_W$ this is not in general the case. However, for probabilistic or weighted computations, an approximation of the greatest fixpoint may be sufficient for verification purposes, since a threshold can be provided as part of a verification task.
\end{remark}

\begin{remark}
By replacing the $F$-coalgebra $(Z,\zeta)$ by $(I,\alpha^{-1})$ with $(I,\alpha)$ an \emph{initial} $F$-algebra, one obtains an alternative account of \emph{finite} traces of states in ${\mathsf{T}} \circ F$-coalgebras, with the \emph{finite trace map} ${\mathsf{ftr}}_\gamma : C \to ({\mathsf{T}} 1)^I$ of a ${\mathsf{T}}\circ F$-coalgebra $(C,\gamma)$ being obtained via the greatest fixpoint of essentially the same operator ${\mathcal{O}}$, but this time on ${\mathsf{Rel}}_{C,I}$. In fact, one can use any $F$-coalgebra in place of $(Z,\zeta)$, and for a specific verification task, a coalgebra with a finite state space, encoding a given linear-time behaviour, might be all that is required.
\end{remark}

\begin{remark}
The choice of functor $F$ directly impacts on the notion of linear-time behaviour. For example, by regarding labelled transition systems as coalgebras of type ${\mathcal{P}}(A \times {\mathsf{Id}})$ instead of ${\mathcal{P}}(1 + A \times {\mathsf{Id}})$ (i.e.~not modelling successful termination explicitly), finite traces are not anymore accounted for -- the elements of the final $F$-coalgebra are given by infinite sequences of elements of $A$. This should not be regarded as a drawback, in fact it illustrates the flexibility of our approach.
\end{remark}

\begin{example}
Let $F$ denote an arbitrary polynomial functor (e.g.~$1 + A \times {\mathsf{Id}}$).
\begin{itemize}
\item For $T = {\mathcal{P}}$, the extension lifting $L_{\mathcal{P}} : {\mathsf{Rel}} \to {\mathsf{Rel}}$ takes a (standard) relation $R \subseteq X \times Y$ to the relation $L_{\mathcal{P}}(R) \subseteq {\mathcal{P}} (X) \times Y$ given by
\[(U,y) \in L_{\mathcal{P}}(R) \text{ ~if and only if~ there exists } x \in U \text{ with } (x,y) \in R\]
As a result, the greatest fixpoint of ${\mathcal{O}}$ relates a state $c$ in a ${\mathcal{P}} \circ F$-coalgebra $(C,\gamma)$ with a state $z$ of the final $F$-coalgebra if and only if there exists a sequence of choices in the unfolding of $\gamma$ starting from $c$, that results in an $F$-behaviour bisimilar to $z$. This was made more precise in \cite{cirstea-11}, where infinite two-player games were developed for verifying whether a state of a ${\mathcal{P}} \circ F$-coalgebra has a certain maximal trace (element of the final $F$-coalgebra).
\item For $T = T_{\mathcal{S}}$, the extension lifting $L_{\mathcal{S}} : {\mathsf{Rel}} \to {\mathsf{Rel}}$ takes a valuation $R : X \times Y \to [0,1]$ to the valuation $L_{\mathcal{S}}(R) : {\mathcal{S}}(X) \times Y \to [0,1]$ given by
\[L_{\mathcal{S}}(R)(\varphi,y) = \sum\limits_{x \in \sup(\varphi)} \varphi(x) * R(x,y)\]
Thus, the greatest fixpoint of ${\mathcal{O}}$ yields, for each state in a ${\mathcal{S}} \circ F$-coalgebra and each potential maximal trace $z$, the probability of this trace being exhibited. As computing these probabilities amounts to multiplying infinitely-many probability values, the probability of an infinite trace will often turn out to be $0$ (unless from some point in the unfolding of a particular state, probability values of $1$ are associated to the individual transitions that match a particular infinite trace). This may appear as a deficiency of our framework, and one could argue that a measure-theoretic approach, whereby a probability measure is derived from the probabilities of finite prefixes of infinite traces, would be more appropriate. Future work will investigate the need for a measure-theoretic approach. At this point, we simply point out that in a future extension of the present approach to linear-time logics (where individual maximal traces are to be replaced by linear-time temporal logic formulas), this deficiency is expected to disappear.
\item For ${\mathsf{T}} = {\mathsf{T}}_W$, the extension lifting $L_W : {\mathsf{Rel}} \to {\mathsf{Rel}}$ takes a \emph{weighted relation} $R : X \times Y \to W$ to the relation $L_W(R) : {\mathsf{T}}_W(X) \times Y \to W$ given by
\[L_W(R)(f,y) = \min_{x \in \sup(f)} (f(x) + R(x,y))\]
for $f : X \to W$ and $y \in Y$. Thus, the greatest fixpoint of ${\mathcal{O}}$ maps a pair $(c,z)$, with $c$ a state in a ${\mathsf{T}}_W \circ F$-coalgebra and $z$ a maximal trace, to the \emph{cost} (computed via the $\min$ function) of exhibiting that trace. The case of weighted computations is somewhat different from our other two examples of branching types, in that the computation of the fixpoint starts from a relation that maps each pair of states $(c,z)$ to the value $0 \in \mathbb N^\infty$ (the top element for $\sqsubseteq$), and refines this down (w.r.t.~the $\sqsubseteq$ order) through stepwise unfolding of the coalgebra structures $\gamma$ and $\zeta$.
\end{itemize}
\end{example}
The approach presented above also applies to coalgebras of type $G \circ {\mathsf{T}}$ with $G$ a polynomial endofunctor, and more generally to coalgebras whose type is obtained as the composition of polynomial endofunctors and the monad ${\mathsf{T}}$, with possibly several occurrences of ${\mathsf{T}}$ in this composition. In the case of $G \circ {\mathsf{T}}$-coalgebras, instantiating our approach yields different results to the extension semantics proposed in \cite{JacobsSS12}. Specifically, the instantiation involves taking $(Z,\zeta)$ to be a final $G$-coalgebra and $(C,\gamma)$ to be an arbitrary $G \circ {\mathsf{T}}$-coalgebra, and considering the monotone operator on ${\mathsf{Rel}}_{C,Z}$ given by the composition
\begin{equation}
\label{o1}
\UseComputerModernTips\xymatrix@+0.3pc{
{\mathsf{Rel}}_{C,Z} \ar[r]^-{L_{\mathsf{T}}} & {\mathsf{Rel}}_{{\mathsf{T}} C,Z} \ar[r]^-{{\mathsf{Rel}}(G)} & {\mathsf{Rel}}_{G({\mathsf{T}} C), G Z} \ar[r]^-{(\gamma \times \zeta)^*} & {\mathsf{Rel}}_{C,Z}
}
\end{equation}
The following example illustrates the difference between our approach and that of \cite{JacobsSS12}.
\begin{example}
For $G = 2 \times {\mathsf{Id}}^A$ with $A$ a finite alphabet and ${\mathsf{T}} = {\mathcal{P}}$, $G \circ {\mathsf{T}}$-coalgebras are non-deterministic automata, whereas the elements of the final $G$-coalgebra are given by functions $z : A^* \to 2$ and correspond to languages over $A$. In this case, the greatest fixpoint of the operator in (\ref{o1}) maps a pair $(c,z)$, with $c$ a state of the automaton and $z$ a language over $A$, to $\top$ if and only if there exists a sequence of choices in the unfolding of the automaton starting from $c$ that results in a deterministic automaton which accepts the language denoted by $z$. Taking the union over all $z$ such that $(c,z)$ is mapped to $\top$ now gives the language accepted by the non-deterministic automaton with $c$ as initial state, but only under the assumption that for each $a \in A$, an $a$-labelled transition exists from any state of the automaton. This example points to the need to further generalise our approach, so that in particular it can also be applied to pairs consisting of a $G \circ {\mathsf{T}}$-coalgebra and a $G'$-coalgebra, with $G'$ different from $G$. This would involve considering relation liftings for pairs of (polynomial) endofunctors. We conjecture that taking $G$ and ${\mathsf{T}}$ as above and $G' = 1 + A \times {\mathsf{Id}}$ would allow us to recover the notion of acceptance of a finite word over $A$ by a non-deterministic automaton.
\end{example}
Finally, we sketch the general case of coalgebras whose type is obtained as the composition of several endofunctors on ${\mathsf{Set}}$, one of which is a monad ${\mathsf{T}}$ that accounts for the presence of branching in the system, while the remaining endofunctors are polynomial and jointly determine the notion of linear-time behaviour. For simplicity of presentation, we only consider coalgebras of type $G \circ {\mathsf{T}} \circ F$, with the final $G \circ F$-coalgebra $(Z,\zeta)$ providing the domain of possible linear-time behaviours.
\begin{definition}
\label{linear-time-beh}
The \emph{linear-time behaviour} of a state in a coalgebra $(C,\gamma)$ of type $G \circ {{\mathsf{T}}} \circ F$ is the greatest fixpoint of an operator ${\mathcal{O}}$ on ${\mathsf{Rel}}_{C,Z}$ defined by the composition:
\begin{equation}
\label{o2}
\UseComputerModernTips\xymatrix@+0.3pc{
{\mathsf{Rel}}_{C,Z} \ar[r]^-{{\mathsf{Rel}}(F)} & {\mathsf{Rel}}_{F C,F Z} \ar[r]^-{L_{\mathsf{T}}} & {\mathsf{Rel}}_{{\mathsf{T}}(F C), F Z}  \ar[r]^-{{\mathsf{Rel}}(G)} & {\mathsf{Rel}}_{G({\mathsf{T}} F C), G F Z} \ar[r]^-{(\gamma \times \zeta)^*} & {\mathsf{Rel}}_{C,Z}
}
\end{equation}
\end{definition}
The greatest fixpoint of ${\mathcal{O}}$ measures the extent with which a state in a $G \circ {\mathsf{T}} \circ F$-coalgebra can exhibit a given linear behaviour (element of the final $G \circ F$-coalgebra). Definition~\ref{linear-time-beh} generalises straightforwardly to coalgebraic types given by arbitrary compositions of polynomial endofunctors and the monad ${\mathsf{T}}$, with the extension lifting $L_{\mathsf{T}}$ being used once for each occurrence of ${\mathsf{T}}$ in such a composition.
\begin{example}
\label{input-output}
Coalgebras of type $G \circ {\mathsf{T}} \circ F$, where $G = (1 + {\mathsf{Id}})^A$ and $F = {\mathsf{Id}} \times B$,  model systems with branching, with both inputs (from a finite set $A$)  and outputs (in a set $B$). In this case, the possible linear behaviours are given by special trees, with both finite and infinite branches, whose edges are labelled by elements of $A$ (from each node, one outgoing edge for each $a \in A$), and whose nodes (with the exception of the root) are either labelled by $* \in 1$ (for leaves) or by an element of $B$ (for non-leaves). The linear-time behaviour of a state in a $G \circ {\mathsf{T}} \circ F$-coalgebra is then given by:
\begin{itemize}
\item the set of trees that can be exhibited from that state, when ${{\mathsf{T}} = {\mathcal{P}}}$,
\item the probability of exhibiting each tree (with the probabilities corresponding to different branches being \emph{multiplied} when computing this probability), when ${{\mathsf{T}} = {\mathcal{S}}}$, and
\item the minimum cost of exhibiting each tree (with the costs of different branches being \emph{added} when computing this cost), when ${{\mathsf{T}} = {\T}_W}$.
\end{itemize}
\end{example}
The precise connection between our approach and earlier work in \cite{HasuoJS07,cirstea-11,JacobsSS12} is yet to be explored. In particular, our assumptions are different from those of loc.\,cit., for example in \cite{HasuoJS07} the DCPO$_\bot$-enrichedness of the Kleisli category of ${\mathsf{T}}$ is required.

\begin{remark}
Our approach does not \emph{directly} apply to the probability distribution monad (defined similarly to the sub-probability distribution monad, but with probabilities adding up to exactly $1$), as this monad does not satisfy the condition ${\mathsf{T}} \emptyset = 1$ of Definition~\ref{additive}. However, systems where branching is described using probability distributions can still be dealt with, by regarding all probability distributions as sub-probability distributions.
\end{remark}

In the remainder of this section, we briefly explore the usefulness of an operator similar to ${\mathcal{O}}$, which employs a similar extension lifting arising from the \emph{double strength} of the monad ${\mathsf{T}}$. We begin by noting that a result similar to Proposition~\ref{prop-kock} is proved in \cite{Kock12} for a commutative monad on a cartesian closed category.
\begin{proposition}[{\cite[Proposition~9.3]{Kock12}}]
Let $(B,\beta)$ be a ${\mathsf{T}}$-algebra. Then any $f : X \times Y \to B$ extends uniquely along $\eta_X \times \eta_Y$ to a bilinear $\tilde{f} : {\mathsf{T}} X \times {\mathsf{T}} Y \to B$, making the following triangle commute:
\[\UseComputerModernTips\xymatrix{
{\mathsf{T}} X \times {\mathsf{T}} Y \ar[r]^-{\tilde{f}} & B \\
X \times Y \ar[u]^-{\eta_X \times \eta_Y} \ar[ur]_-{f}
}\]
\end{proposition}
Here, bilinearity amounts to linearity in each argument.
\begin{definition}
For a commutative monad ${\mathsf{T}} : {\mathsf{Set}} \to {\mathsf{Set}}$, the \emph{double extension lifting} $L_{\mathsf{T}}' : {\mathsf{Rel}} \to {\mathsf{Rel}}$ is the functor taking a relation $R : X \times Y \to \T1$ to its unique bilinear extension $\tilde{R} : {\mathsf{T}} X \times {\mathsf{T}} Y \to \T1$.
\end{definition}
\begin{remark}
\label{alt-lifting}
An alternative definition of $L_{\mathsf{T}}'$ is as the composition of $L_{\mathsf{T}}$ with a dual lifting, which takes a relation $R : X \times Y \to {\mathsf{T}} 1$ to its unique $2$-linear extension $\overline{R} : X \times {\mathsf{T}} Y \to {\mathsf{T}} 1$.
\end{remark}
\begin{remark}
Again, it can be shown that a direct definition of the relation $\tilde{R} : {\mathsf{T}} X \times {\mathsf{T}} Y \to {\mathsf{T}} 1$ is as the composition
\[\UseComputerModernTips\xymatrix@+0.3pc{
{\mathsf{T}} X \times {\mathsf{T}} Y \ar[r]^-{{\mathsf{dst}}_{X,Y}} & {\mathsf{T}}(X \times Y) \ar[r]^-{{\mathsf{T}} (R)} & {\mathsf{T}}^2 1 \ar[r]^-{\mu_1} & {\mathsf{T}} 1
}\]
\end{remark}

\begin{proposition}
\label{prop-functoriality-dual}
The mapping $R \in {\mathsf{Rel}}_{X,Y} \mapsto \overline{R} \in {\mathsf{Rel}}_{X,{\mathsf{T}} Y}$ is functorial.
\end{proposition}
We now fix \emph{two} ${\mathsf{T}} \circ F$-coalgebras $(C,\gamma)$ and $(D,\delta)$ and explore the greatest fixpoint of the operator ${\mathcal{O}}' : {\mathsf{Rel}}_{C,D} \to {\mathsf{Rel}}_{C,D}$ defined by the composition
\[\UseComputerModernTips\xymatrix@+0.3pc{
{\mathsf{Rel}}_{C,D} \ar[r]^-{{\mathsf{Rel}}(F)} & {\mathsf{Rel}}_{F C,F D} \ar[r]^-{L_{\mathsf{T}}'} & {\mathsf{Rel}}_{{\mathsf{T}}(F C), {\mathsf{T}}(F D)} \ar[r]^-{(\gamma \times \zeta)^*} & {\mathsf{Rel}}_{C,D}
}\]

As before, the operator ${\mathcal{O}}'$ is monotone and therefore admits a greatest fixpoint. We argue that this fixpoint also yields useful information regarding the linear-time behaviour of states in ${\mathsf{T}} \circ F$-coalgebras. Moreover, this generalises to coalgebras whose types are arbitrary compositions of polynomial functors and the branching monad ${\mathsf{T}}$. This is expected to be of relevance when extending the linear-time view presented here to linear-time logics and associated formal verification techniques. The connection to formal verification constitutes work in progress, but the following examples motivate our claim that the lifting $L_{\mathsf{T}}'$ is worth further exploration.

\begin{example}Let $F : {\mathsf{Set}} \to {\mathsf{Set}}$ be a polynomial endofunctor, describing some linear-type behaviour.
\begin{enumerate}
\item For non-deterministic systems (i.e.~${\mathcal{P}} \circ F$-coalgebras), the greatest fixpoint of ${\mathcal{O}}'$ relates two states if and only if they admit a common maximal trace.
\item For probabilistic systems (i.e.~${\mathcal{S}} \circ F$-coalgebras), the greatest fixpoint of ${\mathcal{O}}'$ measures the probability of two states exhibiting the same maximal trace.

\item For weighted systems (i.e.~${\mathsf{T}}_W \circ F$-coalgebras), the greatest fixpoint of ${\mathcal{O}}'$ measures the \emph{joint} minimal cost of two states exhibiting the same maximal trace. To see this, note that the lifting $L_W' : {\mathsf{Rel}} \to {\mathsf{Rel}}$ takes a weighted relation $R : X \times Y \to W$ to the relation $L_W'(R) : {\mathsf{T}}_W(X) \times {\mathsf{T}}_W(Y) \to W$ given by
\[L_W'(R)(f,g) = \min_{x \in \sup(f),y\in \sup(g)} (f(x) + g(y) + R(x,y))\]

\end{enumerate}
\end{example}

\section{Conclusions and Future Work}

We have provided a general and uniform account of the linear-time behaviour of a state in a coalgebra whose type incorporates some notion of branching (captured by a monad on ${\mathsf{Set}}$). Our approach is compositional, and so far applies to notions of linear behaviour specified by \emph{polynomial} endofunctors on ${\mathsf{Set}}$. The key ingredient of our approach is the notion of extension lifting, which allows the branching behaviour of a state to be abstracted away in a coinductive fashion.

Immediate future work will attempt to exploit the results of \cite{Ghani2011,Ghani2012} in order to define generalised relation liftings for \emph{arbitrary} endofunctors on ${\mathsf{Set}}$, and to extend our approach to other base categories. The work in loc.\,cit.~could also provide an alternative description for the greatest fixpoint used in Definition~\ref{linear-time-beh}.

The present work constitutes a stepping stone towards a coalgebraic approach to the formal verification of linear-time properties. This will employ linear-time coalgebraic temporal logics for the specification of system properties, and automata-based techniques for the verification of these properties, as outlined in \cite{Cirstea11} for the case of non-deterministic systems.


\section{Introduction:}

We consider Einstein's equation of general theory of relativity for a fluid with heat flow having the following energy-momentum tensor

\be T^{\alpha\beta} = (\rho + p) v^\alpha v^\beta - pg^{\alpha\beta} + q^\alpha v^\beta + q^\beta v^\alpha,\ee
where, $p$ and $\rho$ are the isotropic pressure and matter density of the fluid respectively, $q_\alpha$ is the heat flux in the radial direction, and $v_\alpha$ is the velocity vector. In the co-moving coordinate system, $v^\alpha = \delta_0^\alpha$, $v_\alpha v^\alpha = -1$ and $q_\alpha v^\alpha = 0$, along with the generalized Robertson-Walker line element

\be ds^2 = A^2 dt^2 - B^2 (dr^2 + r^2 d\theta^2 + r^2 \sin^2\theta d\phi^2),\ee
where $A$ and $B$ are functions of $r$ and $t$. Components of Einstein's equation $R_{\alpha\beta} - {1\over 2} g_{\alpha\beta} R = 8\pi G T_{\alpha\beta}$, had been reduced by Bergmann \cite{1} employing a technique formulated by Glass \cite{2}, to the following single equation,

\be A'' + 2{F'\over F} A' - {F''\over F} A = 0. \ee
In the above, prime denotes differentiation with respect to $x = r^2$, and $F = B^{-1}$. Clearly, one physically relevant assumption is required in order to solve the above differential equation containing a pair of variables $A$ and $F = B^{-1}$. However, a physically meaningful assumption on the metric coefficients $A$ and/ or $B$ is obscure. Bergmann \cite{1} therefore obtained a simple solution under the choice $A = 1$. In this paper, we opt for more general solutions. It is important to mention that once the forms of $A$ and $B$ are known, it is quite trivial to compute the radial component of heat flow, which is given by,

\be q = \left({4r\over G B^2}\right) \left(B\over AB\right)',\ee
where, $G$ is the Newtonian gravitational constant.

\section{Generating solutions:}

\textbf{Case:1.} ~~ $A'' = 0$.\\

\noindent
Under this choice, one obtains

\be A' = Q(t);~~~~~ \mathrm{and}~~~~~ A(x, t) = Q(t) x + P(t).\ee
Thus equation (3) reads as,

\be 2 Q F' - Q x F'' - PF'' = 0.\ee
Integrating the above equation and thereafter dividing throughout by $(Qx + P)^4$, one obtains

\be \left({F\over Q x + P}\right)'  + {h(t)\over (Q x + P)^4} = 0.\ee
Further integration yields,

\be F = {h\over 3 Q} + (Q x + P)^3 L,\ee
and thus,

\be B(x, t) = F^{-1} = \left[{h\over 3 Q} + (Q x + P)^3 L\right]^{-1},\ee
where, $h,~ Q,~ P,~, L$ are all functions of time. Equations (5) and (9) may be used to find explicit form of of the radial component of heat flow $q$, in view of the expression (4).\\

\noindent
\textbf{Case:2.}~~ $A'' \ne 0$.\\

\noindent
Under this choice, $F' \ne 0$, as may be seen from equation (3) and thus one can express $A$ as,

\be A = A(F, t);~~~A' = A_F F';~~~A'' = A_{FF} F'^2 + A_F F'',\ee
where, suffix stands for derivative. So, equation (3) in this case reduces to

\be {A_{FF} + 2{A_F\over F}\over A_F - {A\over F}} d F + {dF'\over F'} = 0.\ee
Integrating the above equation one obtains,

\be \int \left[{A_{FF} + 2{A_F\over F}\over A_F - {A\over F}}\right] d F + \ln{F'} = \ln{\alpha(t)},\ee
or,

\be \exp{\int \left[{A_{FF} + 2{A_F\over F}\over A_F - {A\over F}}~d F\right]} = \alpha(t) {dx\over dF}.\ee
Integrating yet again one obtains,

\be \int\left[\exp{\int \left({A_{FF} + 2{A_F\over F}\over A_F - {A\over F}}~d F\right)} \right] dF = \alpha(t) x + \beta(t).\ee
Therefore, if $A$ is given as a function of $F$ and $t$, then the above integral can be evaluated and hence the solutions may be obtained. Nevertheless, for a particular case, simple solutions may be obtained as follows.\\

Let us consider $F'' = m F$, where $m$ is a function of time alone. So equation (3) may be written as,

\be {U''\over U} = 2{F''\over F} = \pm k^2,~~i.e.,~~ U'' = \pm k^2 U\ee
where, $U = A F$, and $k$ is a function of time. Solutions of the above equation (16) may now be easily found as given below,

\be \begin{split} & U = C_1 e^{kx} + D_1 e^{-kx}, ~~~\mathrm{where,}~m ~\mathrm{is~ positive}~m = k^2,\\&
U = C_1 \cos{(kx)} + D_1 \sin{(kx)}, ~~~\mathrm{where,}~m ~\mathrm{is~ negative}~ m = - k^2,\\&
U = q x + r, ~~~\mathrm{where,}~m=0.\end{split}\ee

\noindent
Subcase-I: $m = k^2$:\\

\noindent
When $m > 0$, equation (15) may be solved to obtain

\be F = C_2 e^{kx\over \sqrt 2} + D_2 e^{-{kx\over \sqrt 2}}.\ee
Now since, $AF = U$ and $B = F^{-1}$, so

\be A = {C_1 e^{kx} + D_1 e^{-kx}\over C_2 e^{kx\over \sqrt 2} + D_2 e^{-{kx\over \sqrt 2}}}; ~~~~B = {1\over C_2 e^{kx\over \sqrt 2} + D_2 e^{-{kx\over \sqrt 2}}},\ee
where, $C_1$, $C_2$, $D_1$, $D_2$ and $k$ are all functions of time. Solution (18) may be used to evaluate $q$ from expression (4).\\

\noindent
Subcase-II: $m = -k^2$:\\

\noindent
When $m < 0$, equation (15) may be solved to obtain

\be F = C_3 \cos{\left(kx\over \sqrt 2\right)} + D_3 \sin{\left(kx\over \sqrt 2\right)},\ee
where, $C_3$ and $D_3$ are functions of time. As before, one can find $A$ and $B$ as,

\be A = {C_1 \cos{\left(kx\right)} + D_1 \sin{\left(kx\right)}\over C_3 \cos{\left(kx\over \sqrt 2\right)} + D_3 \sin{\left(kx\over \sqrt 2\right)}};~~~~B = {1\over C_3 \cos{\left(kx\over \sqrt 2\right)} + D_3 \sin{\left(kx\over \sqrt 2\right)}},\ee
and hence $q$ may be evaluated as well, from the expression (4).\\

\noindent
Subcase-III: $m = 0$:\\

\noindent
In this case $k^2 = 0$, and so equation (15) may be solved to obtain,

\be F = k(t) x + C(t),\ee
which when substituted in equation (3), one obtains

\be kx {d\over dx}\left({dA\over dx}\right) + C{d\over dx}\left({dA\over dx}\right) + 2k \left({dA\over dx}\right) = 0.\ee
Integration yields,

\be A = {f(t) x + g(t) \over k(t) x + C(t)};~~~~~ B = F^{-1} = {1\over k(t) x + C(t)}.\ee
Equation (23) may be used to find the expression for $q$ from equation (3).

\section{Conclusion:}

Summarily, the present paper gives the complete set of cosmological solutions of Einstein's equation with heat flow which was reduced by Bergman to equation (3), either explicitly or implicitly. For $A'' = 0$, solutions have been obtained explicitly and are presented in (5) and (9). For $A'' \ne 0$, on the contrary, solutions are given implicitly by (14). However, some explicit solutions can be obtained for $F'' = +{1\over 2} k^2 F$ as presented in equation (18), $F = - {1\over 2} k^2 F$ as in (20) and $F'' = 0$, as revealed in equation (23). \\

It has already been stated that the solution of equation (3) gives the solution of Einstein's equation for the metric (2) and the energy-momentum tensor (1), where $B = F^{-1}$ and $q$ is the heat flow given by equation (4). Having obtained these solutions, it remains to be shown that these are physically acceptable. Certain energy conditions have to be satisfied, particularly that the energy density is positive everywhere. \\

\noindent
\textbf{Acknowlwdgement:} The authors would like to thank Dr. A. Banerjee for bringing to their atention the work of Bergmann.


\section{Introduction}
\IEEEPARstart{H}{idden} markov models are very popular for modeling and simulating processes, when you do not observe...

\section{System Model}
Let $(S_n, X_n)$ be a two-component process, where $(S_n)$ is unobservable component and $(X_n)$ is observable one, $n \in \{1,2,\ldots, N\}$, $N\in\set{N}$; $(S_n)$ ``controls'' equation coefficients of $(X_n)$. Let $(S_n)$ be a stationary Markov chain with $M$ discrete states and transition matrix $\|p_{i,j}\|,\,p_{i,j}=\Pr(S_n = j \mid S_{n-1} = i)$. The process $(X_n)$ is described by the autoregressive model of order $p$:

\begin{equation}\label{observe_process}
X_n = \mu(S_n) + \sum\limits_{i=1}^p a_i(S_n)(X_{n-i} - \mu(S_n)) + b(S_n)\xi_n,
\end{equation}
where $\{\xi_n\}$ are i.i.d. random variables with the standard normal distribution, $\mu, a_i, b \in\set{R}$ are coefficients controlled by the process $(S_n)$.

As a quality measure for our methods we use mean risk $E(L(S_n, \hat{S}_n))$ with a simple loss function~$L$:

\begin{equation}\label{simple_loss_function}
L(S_n, \hat{S}_n) =
\begin{cases}
1, &S_n \ne \hat{S}_n,\\
0, &S_n = \hat{S}_n,
\end{cases}
\end{equation}
where $\hat{S}_n = \hat{S}_n(X_1^n)$ is an estimator of $S_n$ and $X_1^n = (X_1, X_2,\ldots,X_n)$.

As known, for this risk function with the loss function~(\ref{simple_loss_function}) the optimal estimator is
\begin{equation}\label{argmax_prob}
\hat{S}_n = \underset{m\in\{1,\ldots, M\}}{\operatorname{argmax}}\Pr(S_n = m \mid X_{1}^{n}),
\end{equation}
where $\Pr(S_n = m \mid X_{1}^{n})$ is a posterior probability with respect to a $\sigma$-algebra, generated by r.v. $X_1^n$. Its realization will be denoted by
\begin{equation}
P(S_n = m \mid X_1^n = x_1^n) = P(S_n = m \mid x_1^n),
\end{equation}
where we will write $x_1^n$ instead of $X_1^n = x_1^n$.

\subsection{Basic equations}
In this paper we consider methods of filtering and prediction in the case of unknown parametres (transition matrix) of process $(S_n)$ and known parametres (equation coefficients in~\eqref{observe_process}) of process $(X_n)$. For comparison with some standard we also consider optimal filtering and prediction, where all parametres are known.

Filtering is a problem to estimate $S_n$ by using $X_1^n$. Therefore basic equations for filtering
\begin{multline}\label{filtering_basic_1}
P(S_n=m \mid x_1^n)\\
= \frac{f(x_n\mid S_n=m, x_1^{n-1})}{f(x_n \mid x_1^{n-1})}P(S_n=m\mid x_1^{n-1}),
\end{multline}

\begin{multline}\label{filtering_basic_2}
f(x_n \mid x_1^{n-1})\\
= \sum\limits_{m = 1}^{M} f(x_n\mid S_n=m, x_1^{n-1}) P(S_n=m\mid x_1^{n-1}),
\end{multline}
can be obtained from the total probability formula. Since coefficients in~\eqref{observe_process} are known and $\xi_n \sim \mathcal{N}(0, 1)$ then
\begin{multline}
f(x_n\mid S_n=m, x_1^{n-1})\\
= f(x_n\mid S_n=m, x_{n-p}^{n-1}) = f_{m}(x_n),\label{cond_dens_norm_1}
\end{multline}
where
\begin{multline}
f_{m}(x_n)\\
= \phi\Big(x_n; \mu(m) + \sum\limits_{i=1}^p a_i(m)(x_{n-i} - \mu(m)), b^2(m)\Big)\label{density_f_s_n}
\end{multline}
with normal probability density function
\begin{gather}\label{norm_pdf}
\phi(x;\mu,\sigma^2) = \frac{1}{\sqrt{2 \pi}\sigma}\exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right),
\end{gather}
where $x, \mu \in \set{R},.\sigma \in\set{R}^{+}$.

\section{Optimal Filtering}
In the optimal filtering all parametres are known. We use~\eqref{cond_dens_norm_1} knowing coefficients in~\eqref{observe_process} and calculate $P(S_n=m\mid x_1^{n-1})$ in~\eqref{filtering_basic_1} knowing transition matrix:
\begin{gather}
P(S_n=m\mid x_1^{n-1}) = \sum\limits_{i=1}^{M}p_{i, m} P(S_{n-1}=i \mid x_1^{n-1}). \label{markov_chain_transition}
\end{gather}
Then the~\eqref{filtering_basic_1} is transformed to the evaluation equation~\cite{dobrovidov_2012}
\begin{gather*}
P(S_n=m\mid x_1^n) = \frac{f_{m}(x_n)\sum\limits_{i=1}^{M}p_{i, m} P(S_{n-1}=i \mid x_1^{n-1})}{\sum\limits_{j = 1}^{M} f_j(x_n)\sum\limits_{i=1}^{M}p_{i, j} P(S_{n-1}=i \mid x_1^{n-1})},\label{opt_filtering_basic_1}
\end{gather*}
which will be considered as the optimal standard.

\section{Non-parametric Filtering}
\subsection{Reducing to optimization problem}
In this section, the transition matrix $\|p_{i, j}\|$ is assumed unknown, therefore we can not use the equation~\eqref{markov_chain_transition}. To overcome this uncertainty we include formula~\eqref{cond_dens_norm_1} in equations~\eqref{filtering_basic_1},~\eqref{filtering_basic_2} and obtain
\begin{gather}
P(S_n=m\mid x_1^n) = \frac{f_m(x_n)}{f(x_n \mid x_1^{n-1})}u_n(m),\label{filtering_basic_u_1}\\
f(x_n \mid x_1^{n-1}) = \sum\limits_{m = 1}^{M} f_m(x_n)u_n(m),\label{filtering_basic_u_2}
\end{gather}
where
\begin{gather*}
u_n(m) = P(S_n=m \mid x_1^{n-1}),\quad \forall m=1, \ldots, M
\end{gather*}
are new variables, which do not depend on $x_n$ and
\begin{gather*}
\sum\limits_{i=1}^M u_i = 1,\quad u_m \ge 0,\quad \forall m=1, \ldots, M.
\end{gather*}
To calculate~\eqref{filtering_basic_u_1} and~\eqref{filtering_basic_u_2} it is neccessary to find all $u_n(m)$. We need to make the assumption. We suppose that process $(S_n, X_n)$ is $\alpha$-mixing, then
\begin{gather*}
f(x_n \mid x_1^{n-1}) \approx f(x_n \mid x_{n-\tau}^{n-1}),\quad \tau\in\{1,2,\ldots,  n-1\},
\end{gather*}
and estimate density $f(x_n \mid x_{n-\tau}^{n-1})$ using kernel density estimation and designate this estimator like $\hat{f}(x_n \mid x_{n-\tau}^{n-1})$.

Let us introduce vector $\mathbf{u}_n = (u_n(1), u_n(2), \ldots, u_n(M))$ with unknown elements $u_n(m),.m=1,\ldots, M$. Then for calculating $\mathbf{u}_n$ one proposes the following estimator
\begin{multline}\label{estimator_u_n}
\hat{\mathbf{u}}_n\\
=\underset{\mathbf{u} \in \mathrm{\Delta_M}}{\operatorname{argmin\,}}\int\limits_{-\infty}^{+\infty}|\hat{f}(z_n \mid x_{n-\tau}^{n-1})- \sum\limits_{m=1}^M f_m(z_n) u_m|^2 dz_n,
\end{multline}
where
\begin{multline*}
\mathrm{\Delta}_M = \Big\{(t_1, t_2, \ldots, t_M) \in \set{R}^{M}\\
\mid \sum\limits_{i=1}^M t_i = 1, t_i \ge 0, \forall i \in\{1,2,\ldots, M\}\Big\}\
\end{multline*}
is simplex. Let us rewrite estimator $\hat{\mathbf{u}}_n$ with more detailes:
\begin{gather*}
\hat{\mathbf{u}}_n=\underset{\mathbf{u} \in \mathrm{\Delta_M}}{\operatorname{argmin\,}}I_1 - 2I_2 + I_3,
\end{gather*}
where
\begin{align*}
I_1 &= \int\limits_{-\infty}^{+\infty}\hat{f}^2(z_n \mid x_{n-\tau}^{n-1})dz_n,\\
I_2 &= \int\limits_{-\infty}^{+\infty}\sum\limits_{m=1}^M\hat{f}(z_n \mid x_{n-\tau}^{n-1})f_m(z_n) u_m dz_n,\\
I_3 &= \int\limits_{-\infty}^{+\infty}\sum\limits_{i=1}^M\sum\limits_{j=1}^M f_i(z_n) f_j(z_n) u_i u_j dz_n.
\end{align*}
Since $I_1$ does not depend on $\mathbf{u}$, then reduce it, also transform $I_2$ and $I_3$, so $\hat{\mathbf{u}}_n$ has representation
\begin{align}\label{optimization_problem_2}
\hat{\mathbf{u}}_n&=\underset{\mathbf{u} \in \mathrm{\Delta_M}}{\operatorname{argmin\,}}I_3- 2I_2\notag\\
&=\underset{\mathbf{u} \in \mathrm{\Delta_M}}{\operatorname{argmin\,}}\sum\limits_{i=1}^M\sum\limits_{j=1}^M c_{ij} u_i u_j - 2 \sum\limits_{m=1}^M c_m u_m,
\end{align}
where
\begin{align}
c_{ij} &= \int\limits_{-\infty}^{+\infty}f_i(z_n) f_j(z_n)dz_n,\label{c_ij}\\
c_{m} &= \int\limits_{-\infty}^{+\infty}\hat{f}(z_n \mid x_{n-\tau}^{n-1})f_m(z_n)dz_n. \label{c_m}
\end{align}
To solve optimization problem~\eqref{optimization_problem_2}, primarily, it is necessary to calculate latter coefficients~\eqref{c_ij} and~\eqref{c_m}, which we will obtain using kernel density estimators. Therefore we introduce following chapter.

\subsection{Kernel density estimators}
In the general case kernel density estimator of density $f$ is
\begin{gather}\label{kde_estimator}
\hat{f}(\mathbf{y; H}) = \frac{1}{N}\sum\limits_{i=1}^{N}K_{\mathbf{H}}(\mathbf{y} - \mathbf{Y}_i),
\end{gather}
where $\mathbf{y}=(y_1, y_2, \ldots, y_d)^{T}$ is argument and $\mathbf{Y}_i = (Y_{i1}, Y_{i2}, \ldots, Y_{id})^{T}$, $i=1,2, \ldots, N$ are drawn from density $f$; $K_{\mathbf{H}}(\mathbf{y}) = |\mathbf{H}|^{-1/2}K(\mathbf{H}^{-1/2}\mathbf{y})$, where $K(\mathbf{y})$ is the multivariate kernel, which is probability density function; $\mathbf{H}\in\mathcal{H}$ is the bandwidth matrix and $\mathcal{H}$ is the set of $d\times d$, symmetric and positive-definite matrixes. We propose to use unbiased cross-validation (UCV) to find $\mathbf{H}$ (univariate case proposed in~\cite{rudemo_1982},~\cite{bowman_1984} and multivariate in~\cite{sain_1994},~\cite{duong_2005}). This is a popular and relevant method is aimed to estimate
\begin{gather*}
\mathrm{ISE} (\mathbf{H}) = \int\limits_{\set{R}^d}\left(\hat{f}(\mathbf{y; H}) - f(\mathbf{y})\right)^2d\mathbf{y}
\end{gather*}
and then minimize resulting function
\begin{multline}
\mathrm{UCV}(\mathbf{H})\\
= \frac{1}{N(N-1)}\sum\limits_{i=1}^{N}\sum\limits_{\begin{smallmatrix}j = 1,\\ j\neq i\end{smallmatrix}}^{N}(K_\mathbf{H}*K_\mathbf{H} - 2K_\mathbf{H})(\mathbf{Y}_i-\mathbf{Y}_j)\\
+ \frac{1}{N}R(K)|\mathbf{H}|^{-1/2},\label{ucv}
\end{multline}
\begin{gather*}
R(K)=\int\limits_{\set{R}^d}K(\mathbf{y})^2d\mathbf{y},
\end{gather*}
where $*$ denotes a convolution. Then the estimator of $\mathbf{H}$ is
\begin{gather}\label{ucv_rule}
\mathbf{H}_{\mathrm{UCV}} = \underset{\mathbf{H} \in \mathcal{H}}{\operatorname{argmin\,}} \mathrm{UCV(\mathbf{H})}.
\end{gather}

We suppose to generate components $Y_{ik}$ of vector $\mathbf{Y}_i$ from univariate sample $x_1, x_2, \ldots, x_n$ according to the rule
\begin{gather*}
Y_{ik}= x_{(i-1)l + k},\ k=1,2,\ldots, d
\end{gather*}
where $l\in\set{N}$ influences on stochastic dependence between vectors $\mathbf{Y}_i$ (for bigger $l$ less dependence). Then we suggest to simplify obtaining of estimator~\eqref{kde_estimator} and function~\eqref{ucv}. For this aim we:

\begin{itemize}
\item use normal kernel, it means that we set equal $\mathbf{H}$ to $d$-variate normal density with zero mean vector and identity covariance matrix $\phi$;
\item use scalar $h^2$ multiple of identity $d\times d$ matrix ($\mathbf{I}_d$) for bandwidth matrix: $$\mathbf{H} = h^2 \mathbf{I}_{d}.$$
\end{itemize}

Then the estimator~\eqref{kde_estimator} becomes
\begin{multline}\label{kde_estimator_h}
\hat{f}(\mathbf{y}; h)\\
= \frac{1}{N(2\pi)^{d/2}h^d}\sum\limits_{i=1}^{N}\exp\left(-\frac{\sum\limits_{j=1}^d (y_j - x_{(i-1)l+j}) ^ 2}{2h^2}\right),
\end{multline}
with $N = 1+ \lfloor\frac{n - d}{l}\rfloor$ and the estimator of $h$ is
\begin{gather}\label{estimator_of_h}
\hat{h}=\underset{h > 0}{\operatorname{argmin\,}}\mathrm{UCV}(h),
\end{gather}
\begin{multline*}
\mathrm{UCV}(h)\\
=\frac{1}{N(N-1)(2\pi)^{d/2}h^d}\sum\limits_{i=1}^{N}\sum\limits_{\begin{smallmatrix}j = 1,\\ j\neq i\end{smallmatrix}}^{N}\frac{1}{2^{d/2}}e^{-\frac{\Delta x_{ij}}{4h^2}}-2e^{-\frac{\Delta x_{ij}}{2h^2}}\\ + \frac{1}{N(4\pi)^{d/2}h^d},
\end{multline*}
\begin{gather*}
\Delta x_{ij} = \sum\limits_{k=1}^d \left(x_{(i-1)l+k} - x_{(j-1)l+k}\right) ^ 2.
\end{gather*}
Computing minima analytically is a challenge, so a numerical calculation is popular. The function $\mathrm{UCV}(h)$ often has multiple local minima, therefore more correct way is to use brute-force search to find $\hat{h}$, however it is a very slow algorithm. In~\cite{hall_1991} it was shown that spurios local minima are more likely at too small values of $h$, so we propose to use golden section search between 0 and $h^{+}$, where
\begin{gather*}
h^{+} = \left(\frac{4}{N(d + 2)}\right)^{\frac{1}{d + 4}}\underset{k\in\{1,\ldots, d\}}{\max\,}\hat{\sigma}_k,\\
\end{gather*}
where $\hat{\sigma}_k$ is the sample standard deviation of $k$-th elements of $\mathbf{Y}_i$. The parameter $h^{+}$ is an oversmoothed bandwidth. If the matrix $\mathbf{H}$ was an unconstrained then
\begin{gather*}
\mathbf{H}^{+}=\left(\frac{4}{N(d + 2)}\right)^{\frac{1}{d + 4}}\mathbf{S},
\end{gather*}
where $\mathbf{S}$ is a sample covariance matrix of $\mathbf{Y}_i$. The matrix $\mathbf{H}^{+}$ is oversmoothed bandwidth in the most cases. The latter estimator is proposed in~\cite{terrell_1990}. To calculate $\mathbf{H}_{\mathrm{UCV}}$ with unconstrained $\mathbf{H}$ you may use quasi-Newton minimization algorithm like in~\cite{duong_2005}.

\subsection{Calculation of coefficients $c_{ij}$ and $c_m$}
For calculating unknown coefficients $c_{ij}$ and $c_{m}$ in~\eqref{optimization_problem_2} we use formulas~\eqref{c_ij} and~\eqref{c_m}. Observe that for normal probability density function~\eqref{norm_pdf} following equation
\begin{multline*}
\int\limits_{-\infty}^{+\infty} \phi(x; \mu_1, \sigma_1^2) \phi(x; \mu_2, \sigma_2^2) dx\\
= \phi(\mu_1; \mu_2, \sigma_1 ^ 2 + \sigma_2 ^ 2) = \phi(\mu_2; \mu_1, \sigma_1 ^ 2 + \sigma_2 ^ 2)
\end{multline*}
is correct, therefore using it and~\eqref{density_f_s_n} we have
\begin{multline}\label{c_ij_finish}
c_{ij} = \int\limits_{-\infty}^{+\infty}\phi\Big(z_n; \mu(i) + \sum\limits_{k=1}^p a_k(i)(x_{n-k} - \mu(i)), b^2(i)\Big)\\
\cdot \phi\Big(z_n; \mu(j) + \sum\limits_{k=1}^p a_k(j)(x_{n-k} - \mu(j)), b^2(j)\Big)dz_n\\
=\phi\Big(\mu(i) + \sum\limits_{k=1}^p a_k(i)(x_{n-k} - \mu(i));\\
\mu(j) + \sum\limits_{k=1}^p a_k(j)(x_{n-k} - \mu(j), b^2(i) + b^2(j)\Big),
\end{multline}
also $c_{ij} = c_{j, i} > 0$. For calculating $c_{m}$ we estimate conditional density $\hat{f}(z_n \mid x_{n-\tau}^{n-1})$ applying~\eqref{kde_estimator_h}:
\begin{multline*}
\hat{f}(z_n \mid x_{n-\tau}^{n-1}) = \frac{\hat{f}(z_n, x_{n-\tau}^{n-1})}{\int\limits_{-\infty}^{+\infty} \hat{f}(z_n, x_{n-\tau}^{n-1})dz_n}\\
=\sum\limits_{i=1}^{N}\beta_{ni}(\tau)\phi(z_n; x_{(i-1)l + \tau + 1}, h^2),
\end{multline*}
\begin{gather*}
\beta_{ni}(\tau) = \frac{\exp\left(-\frac{\sum\limits_{j=-\tau}^{-1}(x_{n+j} - x_{(i-1)l+j + \tau + 1}) ^ 2}{2h^2}\right)}{\sum\limits_{k=1}^{N}\exp\left(-\frac{\sum\limits_{j=-\tau}^{-1}(x_{n+j} - x_{(k-1)l+j + \tau + 1}) ^ 2}{2h^2}\right)},
\end{gather*}
where $N = 1+ \lfloor\frac{n - 1 - d}{l}\rfloor$, bandwidth $h$ is estimated by~\eqref{estimator_of_h}. Remark that $\beta_{ni}(\tau)$ does not depend on $z_n$. Then we substitute latter estimator in~\eqref{c_m} and obtain
\begin{multline}\label{c_m_finish}
c_{m} = \int\limits_{-\infty}^{+\infty}\hat{f}(z_n \mid x_{n-\tau}^{n-1})f_m(z_n)dz_n \\
=\int\limits_{-\infty}^{+\infty}\sum\limits_{i=1}^{N}\beta_{ni}(\tau)\phi(z_n; x_{(i-1)l + \tau + 1}, h^2)\\
\cdot\phi\Big(z_n; \mu(m) + \sum\limits_{k=1}^p a_k(m)(x_{n-k} - \mu(m)), b^2(m)\Big)dz_n\\
=\sum\limits_{i=1}^{N}\beta_{ni}(\tau)\int\limits_{-\infty}^{+\infty}\phi(z_n; x_{(i-1)l + \tau + 1}, h^2)\\
\cdot\phi\Big(z_n; \mu(m) + \sum\limits_{k=1}^p a_k(m)(x_{n-k} - \mu(m)), b^2(m)\Big)dz_n\\
=\sum\limits_{i=1}^{N}\beta_{ni}(\tau)\phi\Big(x_{(i-1)l + \tau + 1};\\
\mu(m) + \sum\limits_{k=1}^p a_k(m)(x_{n-k} - \mu(m)), h^2 + b^2(m)\Big),
\end{multline}
also we remark that $c_m > 0$.

\subsection{Solution of optimization problem}
In the previous chapters we reduce main problem to optimization problem
\begin{gather*}
\hat{\mathbf{u}}_n=\underset{\mathbf{u} \in \mathrm{\Delta_M}}{\operatorname{argmin\,}}F_n(\mathbf{u}),\\
F_n(\mathbf{u})=\sum\limits_{i=1}^M\sum\limits_{j=1}^M c_{ij} u_i u_j - 2 \sum\limits_{m=1}^M c_m u_m,
\end{gather*}
where coefficients $c_{ij}$ and $c_m$ were calculated in~\eqref{c_ij_finish} and~\eqref{c_m_finish}. Let us consider kind of optimization. We have that $\mathrm{\Delta}_m$ is convex set and Hessian matrix of function $F_n(\mathbf{s})$ is
\begin{gather*}
\mathcal{L}''_{\mathbf{u}} = 2\cdot\begin{pmatrix}
c_{11} & c_{12} & \ldots & c_{1M}\\
c_{21} & c_{22} & \ldots & c_{2M}\\
\vdots & \vdots & \ddots & \vdots\\
c_{M1} & c_{M2} & \ldots & c_{MM}
\end{pmatrix}.
\end{gather*}
If $\mathcal{L}''_{\mathbf{u}}$ is positive defined, then $F_n(\mathbf{s})$ is convex, thus we have convex optimization. In this case we propose to use Karush--Kuhn--Tucker (KKT) conditions~\cite{kuhn_1951},~\cite{boyd_2004}, because of:
\begin{itemize}
\item our case is special because there is opportunity to solve KKT conditions analytically;
\item for convex optimization KKT conditions, which are primarily necessary, are also sufficient;
\end{itemize}
else you may apply methods of quadratic programming. Also we want remark that $\mathcal{L}''_{\mathbf{u}}$ does not depend on variables $u_i$ and coefficients $c_m$, which means that previous kernel density estiamtors have no influence on kind of optimization.

Let us consider KKT conditions, then Lagrangian is
\begin{gather*}
\mathcal{L} = \lambda_0 F_n(\mathbf{u}) + \sum\limits_{i=1}^{M}\lambda_{i} (-u_i) +  \lambda_{M+1}\left(\sum\limits_{i=1}^M u_i - 1\right),
\end{gather*}
where $\lambda^* = (\lambda_0^*, \lambda_1^*, \ldots, \lambda_{M+1}^*)\in\set{R}^{M+2}$. We need to find $\lambda^*$ and $\mathbf{u}^*$ such that stationary condition
\begin{gather*}
\mathcal{L}'_{u_i} = 2\lambda_0^*\left(\sum\limits_{j=1}^M c_{ij} u_j^* - c_i\right) - \lambda_{i}^* + \lambda_{M+1}^* = 0,\\
\forall i=1, \ldots, M
\end{gather*}
primal feasibility
\begin{gather*}
-u_i^* \le 0,\quad \forall i=1, \ldots, M\,\\
\sum\limits_{i=1}^M u_i^* - 1 = 0,
\end{gather*}
dual feasibility
\begin{gather*}
\lambda_{i}^* \ge 0,\quad \forall i=1, \ldots, M\
\end{gather*}
complementary slackness
\begin{gather*}
\lambda_{i}^* u_i^* = 0,\quad \forall i=1, \ldots, M
\end{gather*}
hold. Let $\lambda_0^*=0$ to check that the gradients of constraints are linearly independent at $\mathbf{u}^*$, so KKT conditions lead to system
\begin{gather*}
\begin{cases}
\lambda_1^* = \lambda_{2}^*=\ldots=\lambda_{M+1}^*,\\
\lambda_{i}^* u_i^* = 0,\ \lambda_{i}^* \ge 0, &\forall i=1, \ldots, M\\
\sum\limits_{i=1}^M u_i^* = 1,\ u_i^* \ge 0, &\forall i=1, \ldots, M
\end{cases}
\end{gather*}
which could be solved only with $\lambda^* = \vec{0}$, which means that gradients of constraints are linearly independent for any $\mathbf{u}^*$.
The vector $\lambda^*$ is defined with an accuracy of $\alpha > 0$, so we define $\lambda_0=1/2$, then KKT conditions lead to a system
\begin{gather*}
\mathbf{C}\cdot\vec{\rho} = \mathbf{c},
\end{gather*}
where
\begin{gather*}
\mathbf{C} = \begin{pmatrix}
c_{11} & c_{12} & \cdots & c_{1M} & -1 & 0 &\cdots &0 & 1\\
c_{21} & c_{22} & \cdots & c_{2M} & 0 & -1 & \cdots &0 & 1\\
\vdots & \vdots & \ddots & \vdots & \vdots& \vdots & \ddots & \vdots & \vdots\\
c_{M1} & c_{M2} & \cdots & c_{MM} & 0 & 0&\cdots & -1 & 1\\
1 & 1 & \cdots & 1 & 0 & 0 & \cdots & 0 & 0
\end{pmatrix},\\
\vec{\rho} = \begin{pmatrix}
u_1^*\\
\vdots\\
u_M^*\\
\lambda_1^*\\
\vdots\\
\lambda_{M+1}^*
\end{pmatrix},\ \mathbf{c} = \begin{pmatrix}
c_1\\
c_2\\
\vdots\\
c_M\\
1
\end{pmatrix},\\
\lambda_{i}^* u_i^* = 0,\ \lambda_{i}^* \ge 0,\ u_{i}^* \ge 0.\quad \forall i=1, \ldots, M
\end{gather*}
To solve last system it is necessary to consider all combinations of pairs $(u_i^*, \lambda_i^*),\ \forall i=1, \ldots, M$, where $u_i^*$ or $\lambda_i^*$ is equal to 0 (not both). Total amount of combinations is equal to $2^M$. If $u_i^* = 0$ then $i$-th column in the matrix $\mathbf{C}$ and $i$-th row in $\bar{\rho}$ are reduced, else $\lambda_i^*=0$ and $(M + i)$-th column in the matrix $\mathbf{C}$ and $(M + i)$-th row in $\bar{\rho}$ are reduced. After choosing zero element in each pair $(u_i^*, \lambda_i^*),\ \forall i=1, \ldots, M$ matrix $\mathbf{C}$ is reduced to an $(M + 1)\times (M + 1)$-matrix $\mathbf{C}_r$ and $\bar{\rho}$ to $(M + 1)\times 1$-matrix $\bar{\rho}_r$. Therefore for each combination it is necessary to calculate
\begin{gather*}
\bar{\rho}_r = \mathbf{C}_r^{-1}\cdot\mathbf{c}.
\end{gather*}
If the first $M$ elements in $\bar{\rho}_r$ are non-negative then obtained $\mathbf{u}^*$ is a solution ($\hat{\mathbf{u}}_n$) of optimization problem and there is no reason to calculate $\bar{\rho}_r$ for the next combination, because in convex optimization local minima is global minima.

As a result, we substitute estimator $\hat{\mathbf{u}}_n$ in~\eqref{filtering_basic_u_1} and~\eqref{filtering_basic_u_2} and problem of non-parametric filtering is solved.

\section{One-step Ahead Prediction}
We will consider one-step ahead prediction. Like for filtering we minimize mean risk $E(L(S_n, \hat{S}_n))$ with simple loss function~\eqref{simple_loss_function}. Therefore optimal estimator of $S_n$ is
\begin{gather*}
\hat{S}_n = \underset{m\in\{1,\ldots, M\}}{\operatorname{argmax}}\Pr(S_n = m \mid X_{1}^{n-1}).
\end{gather*}
We remark that probabilty $\Pr(S_n = m \mid X_{1}^{n-1})$ is already obtained in the considered approaches of filtering: for optimal prediction it is written in~\eqref{markov_chain_transition} and for non-parametric prediction accordingly in~\eqref{optimization_problem_2}. It means that we primarily solve problem of one-step ahead prediction and then filtering problem.

\section{Example}
Let the Markov chain $(S_n)$ has 3 states ($M=3$) and transition matrix
\begin{gather}
\|p_{i, j}\| = \begin{pmatrix}
0.8 & 0.1 & 0.1\\
0.05 & 0.9 & 0.05\\
0.1 & 0.05 & 0.85
\end{pmatrix}.
\end{gather}
Sample volume $n$ is changed from 500 to 600. Observable process $(X_n)$ is simulated like AR($2$) model with coefficients $\mu\in\{0, 0.5, 1\}$, $a_1\in\{0.3, 0.2, 0.1\}$, $a_2\in\{0.2, 0.3, 0.4\}$, $b\in\{0.1, 0.2, 0.1\}$. Also we take $\tau=2$ and $l=1$. The results are presented in \figurename~\ref{example_figures} and sample mean errors after 50 repeated experiments in Table~\ref{example_errors}.

\begin{table}[!t]
\renewcommand{\arraystretch}{1.3}
\caption{Sample Mean Errors}
\label{example_errors}
\centering
\begin{tabular}{ccc}
\hline
& Filtering error, \% & Prediction error, \%\\
\hline
Optimal & 16.4 & 26.6\\
Non-parametric & 22.7 & 37.6\\
\hline
\end{tabular}
\end{table}

\begin{figure}[!t]
\centering
\includegraphics[scale=1]{fig_1.eps}
\caption{From top to bottom: 1 --- unobservable $s_n$; 2 --- observable $x_n$;\quad\quad\quad 3, 4 --- optimal and non-parametric filtering; 5, 6 --- optimal and non-parametric prediction.}
\label{example_figures}
\end{figure}

\section{Conclusion}
Preparing...

\ifCLASSOPTIONcaptionsoff
  \newpage
\fi

\bibliographystyle{IEEEtran}