Human Rights Watch believes Nato air strikes killed at least 72 civilians and says the organisation needs to bear responsibility where appropriate.
"We're calling for prompt, credible and thorough investigations," HRW's Fred Abrahams told BBC News.
Nato insists it took unprecedented care to minimise civilian casualties.
It argues that it cannot take responsibility because it has had no presence on the ground to confirm the deaths.
Aircraft from the US, the UK and France conducted most of the 9,658 strike sorties last year, targeting forces loyal to Libyan leader Col Muammar Gaddafi.
In March, another human rights organisation,  
          Amnesty International, said it had documented 55 cases
 of named civilians, including 16 children and 14 women, killed in air strikes.
It described Nato's failure to investigate these cases thoroughly as "deeply disappointing".
The point of the Nato air campaign in Libya last year was to protect civilians, so how many innocent people died is still a sensitive issue, BBC diplomatic correspondent Bridget Kendall reports.
In its report published on Monday, HRW said it had examined in detail evidence of claims of civilian deaths from eight separate Nato strikes.
In one instance, it said a first Nato bomb killed 14 people and a second bomb, moments later, killed 18 more who had rushed to help victims.
What concerns Mr Abrahams, the main author of the report, is that the deaths remain unacknowledged and the families have been offered no compensation.
"Until now, Nato has taken a position of denial," he said.
Source: Nato
"They refuse to give information about how they died and they refuse to investigate, and it is this lack of transparency that is deeply troubling.
"I think it will lead to unnecessary civilian deaths in the future if Nato refuses to look at what went wrong and make corrections."
Nato says it is ready to co-operate with the new Libyan authorities in assessing what further action is appropriate.
But so far the task force being set up in Tripoli seems to have made little headway, our diplomatic correspondent adds.

Neil Trotter admitted taking pictures of women at Edinburgh's Waverley Station and Glasgow's Buchanan Galleries between June and September 2014.
A sheriff told the 54-year-old, from Falkirk, that "women should be able to walk without fear" in public places.
He was also placed on the sex offenders register for 10 years.
A previous hearing at Edinburgh Sheriff Court was told that Trotter travelled into Edinburgh daily by train to go to work.
On Monday 29 September, a fellow passenger noted him walking in "a curious, a sinister manner" at Waverley Station, "clearly making an effort to get very close to a female ahead of him" on the escalator.
Fiscal depute Aidan Higgins, prosecuting, told the court that "it was clear he was using his phone to get under the lady's skirt, taking photographs".
The commuter reported his concerns to British Transport Police and Trotter was identified from CCTV footage. He was detained as he left the station the following day.
Examination of Trotter's phone and his home computer established that he had been using his mobile on various days between 11 June and 29 September, with 22 videos showing the camera had been directed under women's skirts.
Defence solicitor Neil Hay told the court Trotter was "most ashamed of his actions", and "well aware his behaviour is unacceptable".
He said his client had "lost his career, brought his family to the brink of financial ruin and brought shame and humiliation to himself and those very close to him".
Sentencing him to 18 months in prison, Sheriff Peter Braid told Trotter: "You have pled guilty to taking images of women in a clandestine manner, prompted by voyeuristic impulses for your own sexual gratification.
"This involved significant planning committed over a period of more than three months in public places where women should be able to walk without fear.
"The repugnance society must feel towards these offences is such that, in my view, there is no alternative to a custodial sentence, in retribution and to deter others."
A leading human rights organisation has urged Nato to investigate fully the deaths of civilians in air strikes in Libya last year.
[NEXT_CONCEPT]
A former senior information officer for the Scottish government has been jailed for 18 months for using his mobile phone to take photos up women's skirts.