Oh, I always wonder about that, I never know whether it's correct I mean  it certainly sounds better as you said, she didn't take her hand away, but I always find that when I'm writing something down, I'm torn, if it's dialogue I want to say they didn't, couldn't and all things come in.