What happens to undated government stocks when interest rates dip below the coupon rate? What happens to undated government stocks (debt— known as "bonds" in the US) which pay a coupon and have an embedded call option, when interest rates dip below the coupon rate? Will the market value of the bond exceed its par value, or is the risk that the bond will be redeemed at any time liable to depress the price?
This is a pretty standard bond pricing issue. The short answer is yes, the market value of the bond can and often will exceed its par value if interest rates are below the coupon rate, just so long as the call option is structured such that the government must pay any interest that has accrued between the last coupon date and the time that the call option is exercised. However, the extent to which the bond will pay more than its face is a question of how the call option is structured: whether it can be called at any time, or whether it can only be called on coupon dates.

Consider an example where the government has issued a \$100 bond paying 4% per period and (for whatever reason) chooses not to exercise the option immediately, the prevailing interest rate is 3% per period, and the coupon rate is 4% per period:


Period 1: \$4 payment, discounted present value is \$3.88
Period 2: \$4 payment, DPV is \$3.77
Period 3: \$4 payment, DPV is \$3.66
Period 4: \$4 payment, call option exercised, \$100 principal returned, DPV is \$92.40


Total value: \$103.72 (difference due to rounding).

Now consider the case where the bond is outstanding, rates drop to 3%, and the bond is immediately called— the bond pays out \$100, plus whatever interest has accrued.

US government bonds are discretely callable— they can only be called on the coupon date, as in the first example. So in the case you give they'll generally trade above par at any date between coupon payments. UK bonds can be called at any time, so there's a lot more uncertainty— it's possible that you could buy the bond today and have it called immediately. However, given that there's little reinvestment risk and that in the event that they're called they will pay out the face value plus accrued interest, the lower bound of their value should be right around par. Thus, to the extent that there's uncertainty as to whether and when the call option will be exercised, it's reasonable to expect that they will trade strictly above par.

Interestingly, as the interest rate drops and the spread between a callable bond and market rates becomes larger, the likelihood that the bond will be called increases, so the value of the bond continues to increase, but at a decreasing rate. This is known as negative convexity. It's a common feature in US mortgage markets, because most borrowers in US mortgage markets (unlike mortgage markets in many other countries) have an implicit option to prepay at any time without penalty. As a result, mortgages and mortgage-backed securities display significant negative convexity, making hedging interest rate risk a big deal for mortgage lenders. For this reason, your question is actually closely related to the question, "Why do lenders dislike early loan repayments?".