As the title says, How to find the probability of vectors a, b, c, on some unit sphere, all lies on same side of some hyperplane passing through the origin. Information present are the angles between vectors a and b, b and c, c and a.


I am trying to bound the area of sphere in which the normal of hyperplane can reside but I feel stuck. Any hints or directions?



Probability that three vectors of a unit sphere lie on one side of a hyperplane if angle between the vectors are given
$\newcommand\al\alpha\newcommand\be\beta\newcommand\ga\gamma$It appears that the question is as follows: Given unit vectors $a,b,c$ with angles
$$\al:=\cos^{-1}(b\cdot c),\quad \be:=\cos^{-1}(a\cdot c),\quad \ga:=\cos^{-1}(b\cdot a)$$
(where $\cdot$ denotes the dot product), find the probability, say $p$, that the vectors $a,b,c$ are to the same side of a hyperplane through the origin chosen uniformly at random.


The answer is
$$p=\frac{2\pi-\al-\be-\ga}{2\pi}. \tag{1}\label{1}$$


Indeed,
$$p=P(a\cdot U>0,b\cdot U>0,c\cdot U>0)
+P(a\cdot U<0,b\cdot U<0,c\cdot U<0)
=2P(a\cdot U>0,b\cdot U>0,c\cdot U>0),$$
where $U$ is a random vector uniformly distributed on the unit sphere. Next, the random vector $U$ equals $G/|G|$ in distribution, where $G$ is a standard Gaussian random vector and $|G|$ is the Euclidean norm of $G$. So,
$$p=2P(X>0,Y>0,Z>0),$$
where
$$X:=a\cdot G,\quad Y:=b\cdot G,\quad Z:=c\cdot G,$$
so that $X,Y,Z$ are zero-mean, unit-variance jointly normal random variables with correlations
$$\rho\_{X,Y}=a\cdot b=\cos\ga,\quad 
\rho\_{Y,Z}=b\cdot c=\cos\al,\quad 
\rho\_{X,Z}=a\cdot c=\cos\be.$$
Now \eqref{1} follows from the known formula
$$P(X>0,Y>0,Z>0)=\frac{\cos^{-1}(-\rho\_{X,Y})+\cos^{-1}(-\rho\_{Y,Z})+\cos^{-1}(-\rho\_{X,Z})-\pi}{4\pi}\tag{2}\label{2}$$
-- see e.g. the second display in Section 6 on p. 355 of [Plackett](https://www.jstor.org/stable/2332716).




---


Note that \eqref{1} holds for any dimensions $\ge3$.




---


Another way to derive \eqref{2} and hence \eqref{1} is, of course, to note that the numerator of the ratio in \eqref{2} is the area of the spherical triangle on the unit sphere with angles $\pi-\al,\pi-\be,\pi-\ga$. This area can be expressed as a double integral in spherical coordinates. Yet other ways to find this area can be found e.g. on [this page](https://math.stackexchange.com/questions/110075/deriving-the-surface-area-of-a-spherical-triangle). The formula for this area is [Girard's theorem](https://www.theoremoftheday.org/GeometryAndTrigonometry/Girard/TotDGirard.pdf).


