Fish always giving bitter/unpleasent taste/smell When I make fish in a soup(no ingredients, just water, salt and fish) I notice the soup water tastes bitter/unpleasent and I stop eating it.  This can happen with any fish - it's the same taste and smell.

Sometimes when I cook whitebait from from frozen(which was apparently frozen at sea) that too can get this nasty smell.  I've recently been buying line caught mackeral which isnt old and it doesnt usually have the same problem(above smell and taste doesn't occur), note I usually eat it within 2-3 days.  However the other day on the second day I boiled the mackerel for a while and the same stink and taste happened.  I imagine the prolonged boiling caused the issue to prop up.  On the 3rd day from the same batch of mackerel I didnt overcook the mackeral and the smell didnt come.  This tells me it may be something in the way i'm cooking which is causing the issue.

Do you know what it might be and how I can prevent it?
Are frozen or small fish more likely to have this problem and if so why?

Do you know what the smel//taste might be and how I can prevent it?  I especially notice it consistently appear when I cooked sardines, whitebait, anchovies and other small fish though mackerel can get it as in the example I gave.

Thanks
Start with a fish stock instead of water. Cook that with whatever vegetables you want to put in it, add seasoning and get the broth to the flavor that you want. 

During the last 3 - 4 minutes of cooking, add your cut fish and just allow it to poach in the broth for a minute or two, cut the heat, let it sit for a few to finish poaching gently, then serve.

It sounds like you're trying to extract flavor from the fish as you cook it, that's not going to work, you'll just overcook the fish which is what's giving you that awful taste. 

Fish stock is made from the very gelatinous bones of the fish, skin, scales, heads, etc - then strained. You can buy it in most grocery stores, just read the label to see how much sodium it contains (go for the least amount, add salt yourself).