Gogarty's father, himself the son of a medical doctor, had been educated at Trinity College and owned two fashionable homes in Dublin, which set the Gogartys apart from other Irish Catholic families at that time and allowed them access to the same social circles as the Protestant Ascendancy.