Modified core for cooperative games In the core of cooperative games, the total cost of forming a coalition is shared by all members in a coalition (efficiency). However for my problem I only require that all members are individually rational in the coalition - that is, if the utility of at least one member of the coalition is less than the utility he would gain by being alone, then he will break away from the coalition.

In other words, what is the solution concept that only requires stability and does not carry the notion of efficiency?
If I understand the question correctly, I believe the neatest way to put it is to simply call you solution concept "individual rationality". 

Individual rationality can easily be seen as a cooperative game theory solution concept in its own right. I am not aware of any other name for a solution concept consisting only of an individual rationality requirement.