Oral tradition says that the tribe would have left Arabia to go to Egypt. According to their legend they rebelled against the Caliph of Baghdad in the second half of the 10th century. In 1049, when the Zirid ruler of Tunisia rebelled against Egypt, nomads were sent to punish them. Ibn Khaldun, an early Tunisian historian, said they were like "a cloud of locusts" when they invaded Tunisia. The Arab poets told of their many conquests as they moved west. They were defeated in a series of battles during the 12th century by the Moroccan dynasty. Even though it was an oral history, the Banu Hilal were a real tribe. They did come from the Arabian Peninsula. And the Banu Hilal did control much of North Africa for over a century.