Every time someone comes to my house without me inviting them, I have a panic attack or anxiety attack.

I have 4 other flatmates, so this becomes really hard, because people always show up without me knowing because someone else invited them.

I obviously can't ask them to not invite people over because it's their house too, but I don't know what to do about this. Even if the person who comes home is my best friend, I still panic if I didn't invite them  
It's been happening for almost 6 months now. Usually when it happens, my girlfriend helps me get through it and I feel better. But I can't always expect her to be there with me and I want to fix this.

If she is with me, I feel better in like 10-15 min but if she isn't there then the attack will last for as long as the people stay at home. Once they leave, I will feel better within the next 10 minutes.

I don't know why this is happening or how to fix it in any way. If anyone has any advice it would be really helpful.