Parabolic or hyperbolic trajectories According to Newton's laws, the trajectories for the two-body problem are conics: either ellipses, or parabolas or hyperbolas. Of course periodic motions require Ellipses and in the Solar system these elliptic curves are a pretty good approximation of the trajectory of each planet.

However, it is possible that the trajectory of a very fast comet could be an hyperbola or a parabola with the Sun at a focus; in that case the motion would not be periodic and the comet would be visible near the Sun only once.

My question: was there ever any observation of a  comet on a hyperbolic (or parabolic) trajectory (with the Sun at a focus)? Same question for different stellar objects. It would be certainly very difficult to observe, since the comet would be visible only one time at its perihelion and would go at infinity after this with no return.
Yes, and it is not uncommon for an orbit have an eccentricity close to one. The wikipedia site, linked in a comment above, notes C/1980 E1, which entered the inner solar system with an eccentricity close to one, but had a close encounter with jupiter and was accelerated. It left the inner solar system with a eccentricity of 1.05, and so is on a hyperbolic trajectory, and will escape from the sun's gravity

Orbits that are highly hyperbolic are very unlikely. Comets formed as part of the solar system.

They are not really harder to spot than any other comet. A comet takes many months to make its passage through the inner solar system. There is plenty of time for them to be spotted, especially if you have probes like SOHO or NEAT