Dr. Goldstein says a lot of his patients are hesitant to take the pill because, in their words, âitâs not natural.â. Itâs not the most common side effect, but itâs not unheard of either: For some women, birth-control pills dull their libido to the extent that they defeat the purpose of taking it in the first place.