Sports Minister Carál Nί Chuilίn said they stood down just before she was due to meet board representatives.
They were there to discuss a report by the Head of Internal Audit at the Department of Culture, Arts and Leisure and the Department of Education.
Ms Nί Chuilίn said she advised the remaining members at the meeting "of my deep concerns at the issues raised by their staff" with the internal auditor.
On Friday, the minister called for the meeting to consider the findings of the report into "serious allegations" about the "leadership, management and the overall culture" of the organisation.
After Monday's meeting, she said she had accepted the board members' resignations, and thanked them "for their efforts in promoting and fostering sport during their time in office".
In a statement, she said: "Given the ongoing challenges in Sport NI, I am taking steps to intervene directly to support both staff and the remaining board members.
"I also wish to advise staff that their concerns are being taken seriously and that they will be fully addressed."
Chair of the culture committee Nelson McCausland said the situation was "shocking and unprecedented".
"That is why we need to get Carál Nί Chuilίn in front of the committee as soon as possible," he said.
"It tells  us something about the department and the way that the minister has been running the department," Mr McCausland told BBC NI's Good Morning Ulster.
"We also want to talk to the people who resigned, they have a story as well.
"It is a whole quagmire, we need to get to the bottom of it. The minister owes that to the public, to Sport NI and to the  committee."
Ms Nί Chuilίn said she would meet the remaining board members later this week.
No-one from Sport NI was available for comment.

Jackie Powell was arrested last year on suspicion of preventing the lawful burial of Brady's victim Keith Bennett.
The Crown Prosecution Service said no action would be taken against Ms Powell, of Llanelli, Carmarthenshire.
Brady is being held at high security Ashworth Hospital on Merseyside.
Brady, now 74, and Myra Hindley murdered five children between 1963 and 1965 in Greater Manchester.
Keith Bennett is the only one of their victims never to be found.
Ms Powell was arrested after she told a television documentary Brady had given her a sealed envelope to pass to Winnie Johnson, Keith's mother, in the event of Brady's death.
The letter was never found and Mrs Johnson died last August still not knowing where her son was buried.
John Dilworth, head of the CPS North West Complex Case Unit, said: "After careful consideration, we have decided that Ms Powell should not be charged, as it cannot be established that she knew the contents of the letter referred to, that the letter in question existed or what information it might have contained.
"The only evidence of the letter's existence was in comments given by Ms Powell to an interviewer and she stated only that she believed it may contain information about Keith Bennett.
"Even if it could be proved that this letter existed, there is insufficient evidence to prove that she genuinely believed it contained the information in question."
Mr Dilworth said the CPS had written to Keith Bennett's brother to explain the decision and have offered a meeting to discuss it.
Martin Bottomley, head of investigative review in Greater Manchester Police's major and cold case crime unit, said: "As soon as we were made aware of the existence of this alleged letter, we made exhaustive attempts to obtain it to establish whether or not its contents would assist us in finding Keith's body.
"That resulted in the warrants we carried out in the South Wales area and at Ashworth Hospital in August 2012.
"However, despite seizing numerous documents and a search of Brady's cell, no such letter has been recovered."
He said the letter may have been destroyed, hidden elsewhere, or it may have been handed to someone else.
"It may simply never have existed in the first place and this has been yet more mind games by Brady," he said.
He added: "Keith still has surviving relatives who deserve to know the truth and Greater Manchester Police will continue to act on any credible evidence to bring this torment to an end."
Nine members of the Sport NI board have resigned with immediate effect.
[NEXT_CONCEPT]
Moors murderer Ian Brady's mental health advocate will not face charges over allegations she failed to disclose information about the location of one of his victims' remains.