Suppose $G$ is a locally compact group and $H$ is an open subgroup for simplicity. Further suppose $\pi$ is a representation of $H$ on some Hilbert space $\mathcal{H}\_{\pi}$, i.e. $\pi(h)$ is unitary for all $h\in H$, $\pi$ is weakly continuous and $\pi$ is also a homomorphism.


To induce a representation on $G$ from $H$, it is natural to consider the usual action of $G$: as translation of functions on $G$. $H$ however acts naturally on $\mathcal{H}\_{\pi}$ by $\pi$. The way to marry these two notions in a self-consistent manner in order to induce a representation on $G$ is to consider those $f:G\to\mathcal{H}\_{\pi}$ such that $f(gh) = \pi(h^{-1})f(g)$. Such $f$ are "constant" on cosets of $H\_{\pi}$ in $G$ - more precisely, $\| f(\cdot)\|$ is constant on cosets.


From this, we can define a Hilbert space $\mathcal{H}(G,\pi)$:


$$\mathcal{H}(G,\pi) = \{f:G\to\mathcal{H}\_{\pi}:f(gh) = \pi(h^{-1})f(g),\; \sum\_{g'H\in G/H}\|f(g')\|^2<\infty\}$$


with the natural inner product given by


$$\langle f\_1,f\_2\rangle = \sum\_{gH\in G/H}\langle f\_1(g),f\_2(g)\rangle.$$


From here we can finally define the induced representation $\text{ind}\_H^G\pi:G\rightarrow U(\mathcal{H}(G,\pi))$:


$$\text{ind}\_H^G\pi(g)f(\cdot) = f(g^{-1}\cdot).$$


It is not hard to see that this is indeed a homomorphism and that it is unitary. However it is not necessary that $\pi$ be a homomorphism in order for $\text{ind}\_H^G\pi$ to be a homomorphism. This is not surprising since we are just acting by left translation. The weak continuity of $\pi$ seems necessary to keep in order for $\text{ind}\_H^G\pi$ to be weakly continuous since weak continuity is not guaranteed for a unitary homomorphism (pathological things may yet happen).


The only place in which $\pi$ being a homomorphism plays a role is in showing that $\text{ind}\_H^G\pi$ is in fact weakly continuous. The role is also quite minor in the proof. Is it possible to omit that the assumption that $\pi$ be a homomorphism and still have $\text{ind}\_H^G\pi$ be a representation? Just because it is *used* to prove that $\text{ind}\_H^G\pi$ is weakly continuous does not mean it is a necessary condition.



Is it necessary for $\pi:H\to U(\mathcal{H}_{\pi})$ to be a homomorphism in order for $\text{ind}_H^G\pi$ to be weakly continuous?
I don't believe that weak continuity is *"the only place in which $\pi$ being a homomorphism plays a role"*.


In fact, before you can even talk about weak continuity of the resulting representation, you want $\mathcal H(G, \pi)$ to be a vector subspace of $(\mathcal H\_\pi)^G$ and be $G$-invariant under left translation of functions, right?


But if so, then barring any stupid mistake on my part, one checks without trouble that 1º)
$$
\mathcal V\_\pi:=\{v\in \mathcal H\_\pi:\text{$v$ is the value of some $f\in\mathcal H(G, \pi)$ at some $g\in G$}\}
$$ is a vector subspace of $\mathcal H\_\pi$, and is invariant under $\pi(h)$ for all $h\in H$. And 2º) if $v\in\mathcal V\_\pi$, then choosing $f$ and $g$ such that $v=f(g)$ one can write:
\begin{align}
\pi(hh')v
&= \pi(hh')f(g) \\
&= f(gh'^{-1}h^{-1}) \\
&= \pi(h)f(gh'^{-1}) \\
&= \pi(h)\pi(h')f(g) \\
&= \pi(h)\pi(h')v.
\end{align}
So $\pi$ is a homomorphism *after restriction to $\mathcal V\_\pi$*. Which is obviously all that matters, as vectors in $\mathcal H\_\pi\setminus\mathcal V\_\pi$ play no role anywhere.


