 The problem with that is if you also are not taking out enough money from your 401ks or IRAs and you get to required minimum distributions, then you start paying a lot in taxes and you lose all control over how much you pay in taxes. And so there's a lot more of it than just thinking about Social Security in and of itself.