Can a system of three stars exist? Just like binary stars can a system of three stars mutually equidistant from each other?
What I mean is three stars at the vertices of a equilateral
If yes than what will be the orbit of a planet out there
If no what will be the problem for it's nonexistence?
It is possible in a Trojan configuration:



In the place of the "Planet" on the image, also a small star could exist. The third star would be at $L_4$ or at $L_5$. This configuration could be made stable.

However, as this link shows,


  In unnormalized units, this criterion becomes
  
  $$\frac{m_2}{m_1+ m_2} &lt; 0.0385$$
  
  We thus conclude that the $L_4$ and $L_5$ Lagrange points are stable
  equilibrium points, in the co-rotating frame, provided that mass $m_2$
  is less than about $4\%$ of mass $m_1$.


Thus, the mass of the second star should be at most 3.85% of the central star.

As far I know, no such known star system exists, but if it would, it would be stable.

Stable planetary orbit is possible to


either very close to one of the stars (compared to the size of the triangle)
in the other Lagrange point
or very far from all of them.


If the triangle is big, then a planet even in the habitable zone is possible.