Two decades later, after amassing a fortune in his association with Spreckels, Irwin moved away from the plantation activities and relocated to San Francisco, where he continued his affiliations with financial institutions.