Sanders was born in Brooklyn, New York to Eli Sanders and to Dorothy Glassberg. His father was a Jewish immigrant born in Słopnice, Poland in 1904. His mother was born to Jewish parents in New York City in 1912. He has an older brother, Larry. His grandparents were murdered in the Holocaust. His mother died in 1960 and his father died in 1962.