Ballou lived in San Francisco, California for four years during the California Gold Rush, and was a member of the San Francisco Committee of Vigilance that used extralegal means to fight rampant crime and corruption.