So my question is somewhat similar to [Restriction from $\mathfrak{gl}\_{2n}$ to $\mathfrak{sp}\_{2n}$](https://mathoverflow.net/questions/50729/restriction-from-mathfrakgl-2n-to-mathfraksp-2n); but I was having difficulty understanding the formula given in reference (Harris & Fulton) mentioned there. 


Equation (25.37) in Harris & Fulton's "Representation Theory: A first course", says that if $m=2n$ or $m=2n+1$, $\lambda = (\lambda\_1 \geq \lambda\_2 \geq \cdots \geq \lambda\_m)$ is a $GL\_m$ highest weight, and $\bar{\lambda}=(\bar{\lambda\_1} \geq \cdots \geq \bar{\lambda}\_n)$ is an $O\_m$ highest weight, then the multiplicity of the irreducible highest-weight $O\_m$ representation $V\_{\bar{\lambda}}$ in the restriction of the irreducible highest-weight $GL\_m$ representation $V\_{\lambda}$ is equal to $\sum\_{\eta} N\_{\eta, \bar{\lambda}, \lambda}$; where the sum is over all partitions $\eta$ with all parts even, and $N\_{\lambda\_1, \lambda\_2, \lambda\_3}$ is the Littlewood-Richardson coefficient. I had some trouble getting this formula to work for $m=2$:


**Q1:** Since $O\_m$ is disconnected, an irreducible $O\_m$ representation is given by (a) if $m=2n+1$, a highest weight ${\lambda}=(\bar{\lambda\_1} \geq \cdots \geq \bar{\lambda}\_n)$ and a choice of $+$ or $-$ (b) if $m=2n$, a highest weight ${\lambda}=(\bar{\lambda\_1} \geq \cdots \geq \bar{\lambda}\_n)$ and, if $\bar{\lambda}\_n=0$, a choice of $+$ or $-$. [See pg 53 of <http://dspace.mit.edu/handle/1721.1/8642?show=fullfor> more details.] In the above formula, the book seems to be considering $\mathfrak{o}\_m$ representations instead, so how should the choices of $+$ or $-$ that arise be incorporated into the above formula?


**Q2:** Now take $m=2$. First let $\lambda=(1,1)$. If $\bar{\lambda} \neq 0$, then clearly $\sum\_{\eta} N\_{\eta, \bar{\lambda}, \lambda}=0$; if $\bar{\lambda}=0$, then $\sum\_{\eta} N\_{\eta, \bar{\lambda}, \lambda}=0$ also (since $N\_{2,0,(1,1)}=0$). So this seems to be saying that the restriction of $V\_{(1,1)}$ to $O(2)$ has no constituents, which is clearly false. The same problem occurs for $\lambda=(2k+1, 2k+1)$. Another example that was troubling me is $\lambda=(6,4), \bar{\lambda}=(4)$: then $N\_{6, 4, (6,4)}=N\_{(4,2), 4, (6,4)}=1$, so this says that the multiplicity of $V\_{4}$ in the restriction of $V\_{(6,4)}$ is $2$, which is also false. 



Restriction of representation from GL(n) to O(n)
For the exterior powers (highest weight $(1,1,\dots,1)$), the restriction to the orthogonal group is still irreducible -- I think you are forgetting to take $\overline{\lambda} = 0$. 


There are also issues with stable ranges in this formula (you have to assume that $\lambda$ doesn't have more than $n$ parts). If you want the formulas to work in all cases, then you have to use modification rules for transforming partitions of length more than $n$ into those with length at most $n$ (sometimes you have to introduce signs to cancel terms). Details of this modification rule can be found in several sources. One such that I like is Koike--Terada's paper: <http://www.sciencedirect.com/science/article/pii/0021869387900998> (Section 2.4)


