After college, Herman worked for Catholic Charities and other agencies advocating minority women employment. Jimmy Carter met Herman while campaigning in Atlanta, Georgia, and after becoming President in 1977, picked her to be Director of the Labor Department's Women's Bureau. At age 29, she was the youngest person to ever serve in that position.