Risk Premia in Continuous Time Take some state variable $X(t)$, which follows the law of motion

$$ \dot X(t) = f(t)X(t) $$

where $f(t)$ is a policy function, and determines the growth rate of $X(t)$. As a second shock, we have $\psi$, which is iid. The agent defaults whenever

$$ g(X(t), \psi) \leq 0$$

Allow the agent to borrow some money that he will have to repay continuously. Let's compute the risk premium. The probability of default at $t+\epsilon$ is

$$Prob(g(X(t+\epsilon), \psi) \leq 0) $$

As the lending has to be repaid continuously, the interest rate, given some risk-free interest rate $r^*$ and risk-neutral lenders, is given by

$$ r^* = r \cdot \lim_{\epsilon\to 0} \left(1 - Prob(g(X(t+\epsilon), \psi) \leq 0)\right)$$

However, as the law of motion for $X(t)$ is continuous, in the limit, this becomes

$$ r^* = r \cdot \left(1-Prob(g(X(t), \psi) \leq 0) \right)$$

This would mean that the agent's risk premium is independent of what he is doing: his policy $f(t)$ does not appear anymore.

But since $f(t)$ affects the state $X(t)$ and the latter the default probability, I feel it should. What's my mistake here? 

References are fine. Most of continuous time finance references I know are much too deep for this rather simple question.
The easiest way to model short-term but risky debt in continuous time is to have your $\psi$ be the increment of a compound Poisson process. 

Jumps in this process correspond to events that might or might not cause default; the size of the jump can enter together with the state $X(t)$ into a function like your $g(X(t),\psi)$ to determine whether or not default actually occurs. Using this formulation, one can circumvent your concern in the comment that "debt is risk-less, if it is almost immediately paid back". 

If, for instance (in a simple case), default occurs whenever there is a nonzero increment in the compound Poisson, regardless of the size of this increment, then if the arrival rate of these jumps is $\mu$, the interest rate will be $r=r^*+\mu$, the risk-free rate plus the flow default rate. If (in a more complicated case) default only occurs for some increment sizes, according to a threshold that depends on $X(t)$, then we can get $r$ varying with $X$.

In this environment, your observation that $f(t)$ does not affect $r(t)$ is completely correct. But this is because $f(t)$ does not contemporaneously affect $X(t)$ and hence does not affect whether or not there will be a default if the Poisson shock occurs at time $t$. $f$ does affect future $X$, and thus will potentially affect future default probabilities and interest rates.

The property we see here - where your policy today doesn't affect your interest cost today, because short-term creditors only care about the current probability of default and your policy can't affect that right away - is generically a strange feature of short-term debt models, and is brought into starkest relief in the continuous-time environment. This is why models of long-term debt are often more realistic, because the price of long-term debt incorporates future default decisions and therefore the effects of policy on future values of the state variable.