Qasriga waxaa markii hore dhisay Muhammad I, Amiirkii Umayyad ee Cordoba, intii u dhaxeysay 860 iyo 880. Kadib markii Moors laga saaray Toledo qarnigii 11aad, qalcaddii waxay sii haysatay shaqadeedii difaaca. Henry III ee Castile wuxuu ku daray dhowr munaaradood. Wiilkiisii John II wuxuu u isticmaalay guri boqortooyo. Intii lagu guda jiray Dagaalkii Dhaxalka Castilian
(1476) ciidamada Joanna la Beltraneja ayaa lagu hareereeyay Alcázar, xilligaas oo dhismaha uu waxyeelo weyn soo gaadhay.