Rubuta ci gaba don wannan sakin layi - Muhammad I, Sarkin Umayyad na Cordoba, ya gina fadar a tsakanin 860 da 880. Bayan da aka kori Moors daga Toledo a ƙarni na 11, fadar ta ci gaba da aikinta na tsaro. Henry III na Castile ya ƙara hasumiyoyi da yawa. Ɗansa John II ya yi amfani da shi a matsayin gidan sarauta. A lokacin yakin da Castilian Succession
(1476) sojojin Joanna la Beltraneja sun kewaye shi a cikin Alcázar, lokacin da ginin ya sha wahala sosai.