Do all the stars visible to the naked eye belong to Milky Way? It is surprising that even late in 1920, people do not know there are galaxies besides the Milky Way. 

Does this mean that all the stars visible to the naked eye are near to us?
All stars visible individually to the naked eye belong to the Milky Way. A handful of galaxies are visible to the naked eye, but here you only see the combined light of millions to billions of stars.

The only exception is when a massive star ends its life, exploding as a supernova. This can be visible to the naked eye for a brief perdiod of time. Most notably, the gamma-ray burst GRB 080319B was visible to the naked eye for roughly 30 seconds in 2008, despite its distance from us being $\sim10^5$ times the size of the Milky Way.