 without going all the way down the wood, I think you'll be fine. You get that off, you scrub it down real good. You've already dulled that surface because you've removed paint above it, and I'm sure you're going to scratch that surface of whatever paint was on there. And then a primer and then an acrylic or alkyd acrylic enamel on top of that, you should be good to go.