I often feel self-conscious when I'm out in public
That is an unpleasant feeling and you feel like people are watching you
Exactly.  I don't know what to do to prevent it
It's very difficult and maybe therapy or medication. That kind of awareness can impair your ability to perform certain actions.
I wonder how many people it happens to regularly?
I'm sure many people feel the same way so you are not alone, especially people who are shy or introverted.
I'm not shy, but definitely introverted.  I have had issues with self-consciousness since I was in junior high school
Adolescence has been known to be a time of higher self-consciousness. Take one day at a time. I know this can be a problem at times.