Charles was raised by his mother and by his maternal grandfather .
His mother worked as a maid and as a seamstress in a factory in New York 's Garment District .
Charles was raised by his mother , who worked as a maid and as a seamstress in a factory in New York 's Garment District , and by his maternal grandfather .