In 1870, the work of clearing the rubble started. The rebuilding of the fort was directed by General Quincy A. Gillmore. He had commanded the Union guns during the siege of the fort in 1861. But in 1876, the work stopped and the fort began to slowly deteriorate again. From 1876 to 1898, the fort served as a lighthouse.