Patent foramen ovale is due to
The primitive atrium is divided into left and right atria by an interatrial septum which is formed by fusion of septum primum and septum secundum. The sequence of events are as follows

Septum primum arises from the roof of the common atrium (primitive atrium) and grows caudally towards septum intermedium (fused AV cushions).
Foramen primum is formed between the free border of growing septum primum and fused AV cushions (septum intermedium).
Shortly afterwards the septum primum fuses with septum intermedium (fused AV cushions), obliterating foramen primum
Foramen secundum is formed by degeneration of cranial part of septum primum.
Septum secundum grows caudally to the right of septum primum and overlaps the foramen secundum. The passage between septum primum and septum secundum is called foramen ovale. After birth, foramen ovale closes by fusion of septum primum with septum secundum. Failure of fusion of these two septa results in patent foramen ovale.
Failure of closure of sinoatrial orifice
Failure of fusion of septum primum and endocardial cushion
Failure of fusion of septum primum and septum intermedium
Failure of fusion of septum primum and septum secondum