Smith had built the Smith Opera House in downtown Geneva and the Smith Observatory on his property when he became interested in founding a college for women, a plan he pursued to the point of breaking ground before realizing it was beyond his means.