What is the source of the very strong flavour in some bad fats (or dairy products?) that only some people can taste? It's a very, very strong flavour (like hot chili peppers in potency and ability to linger). 

It tastes nothing like any food I know - I've thought of it as kind of "chalky" (not sure why I know what chalk tastes like, maybe I ate some chalk as a child?).

It is very unpleasant and nauseating. I imagine no amount of willpower could keep down a mouthful of a food that has it (only managed to swallow small doses in company where it would be very embarassing to spit out food).

It seems to occur in fats, or maybe dairy products. I imagine it's some kind of contamination, maybe bacteria. I've tasted it in:


Blue cheese (don't think it's ever been absent the few times I've tried a bit)
The "creme" in little twinkie-style "swiss rolls" (occasionally)
Once in a home-made pizza (maybe in the cheese?)
Some indian sweets left out for too long (occasionally)


The weird thing is that often, I seem to be the only person that can taste it. This has often got me into trouble (I am not a fussy eater at all, and feel very embarrassed explaining I'm unable to eat something that everyone else thinks is fine).

Anyone have any idea what this flavour could be from?

I've wondered for years...

Edit:
Added some interesting points suggested by answers so far
Let me start by saying that I'm not familiar with the problem taste you're describing in dairy foods.
I am, however, quite sensitive to bitter tastes in many other foods, which is linked to a phenomenon often studied in genetics classes: The ability or inability to taste PTC, PROP or thiourea, chemical compounds associated with bitter flavors.
The ability to taste them or not, and whether you're a &quot;supertaster&quot; who's very sensitive, is genetic, and appears to have a significant impact on food preferences.
Here's an article that explains it better than I can: Supertaster
I don't know that this is associated with dairy products at all, but it's worth a look in. And for what it's worth as a strong--if not super--taster, I have a hard time dealing with some blue cheeses that my wife who is a weak to non-taster doesn't find strong at all. So maybe there's something in it.