He was speaking in an emergency debate at the European Parliament in the wake of last week's EU referendum.
The UK voted by 52% to 48% to leave the union after 43 years of membership.
Mr Smith used his speech in Brussels to urge the EU to respect the will of Scotland after its electors voted by 62% to 38% to remain in the EU.
He won applause form the chamber when he said he was "proudly Scottish and proudly European".
Mr Smith went on to say: "I want my country to be internationalist, cooperative, ecological, fair, European.
"And the people of Scotland, along with the people of Northern Ireland and the people of London, and lots and lots of people in Wales and England also, voted to remain within our family of nations.
"I demand that that status and that esprit European be respected.
"But colleagues, there are a lot of things to be negotiated.  We will need cool heads and warm hearts. But please, remember this: Scotland, did not let you down.
"Please, I beg you, chers collÃ¨gues, do not let Scotland down now."
After his short address he received a standing ovation. The leader of UKIP and pro-Leave backer Nigel Farage did not join in the applause.
MEP Mr Farage had earlier spoken to his fellow EU parliament colleagues.
He told them that "the little people" had rejected "big politics" in order "to get their country back".
Mr Farage believed the UK now offered "a beacon of hope" across Europe for other countries that wanted to leave the EU.
He said:  "When I came here 17 years ago and said I wanted to lead a campaign to get Britain to leave the European Union, you all laughed at me. Well, I have to say, you're not laughing now are you?"

The 36-year-old man, arrested over Penelope Davis's killing, can be held until about 19:30 BST.
Mrs Davis, 47, known as Pennie, was tending to her horses when she was attacked.
Her body was found in a field on Leygreen Farm, Beaulieu by her husband, Pete on 2 September.  A total of five people have been arrested in the case.
Hampshire Constabulary said Mrs Davis, a supermarket worker, may have been followed from Sainsbury's in Hampton Lane, Blackfield.
Officers have appealed for witnesses who saw Mrs Davis's dark green Toyota Land Cruiser travelling between the supermarket and the field at Leygreen Farm between 13:30 and 15:00 on 2 September.
A 28-year-old woman from Hythe, Hampshire, arrested on Thursday on suspicion of conspiracy to commit murder, has been released on bail until 3 November.
Leanne Doyle, 24, of Beech Crescent, Hythe, was charged on Monday with assisting an offender and perverting the course of justice.
She has been bailed and is due to appear at Winchester Crown Court on 1 October.
A 22-year-old man, previously arrested on suspicion of murder, has been released on bail, while a 37-year-old man held on suspicion of assisting an offender was also bailed.
Crimestoppers has offered a Â£5,000 reward for information leading to the conviction of Mrs Davis's killer.

Seven Shi Tzu puppies and two full-grown lurchers were taken from the property in Glendevon Park, Winchburgh, between 07:30 and 18:00 on Wednesday.
The Shi Tzu puppies are tan and white in colour and are worth about Â£650 each. The lurchers have pale gold rough coats and are working dogs.
Police officers are appealing for witnesses to come forward.
Con Rachel Royan, of Police Scotland, said: "At eight weeks old the puppies are too young to have left their mother, who was not taken, and the owners are extremely concerned for their welfare.
"We would urge anyone who saw anything suspicious in or around Glendevon Park on Wednesday to please get in touch as we are keen to reunite them with their owners."
SNP MEP Alyn Smith received a standing ovation from his European colleagues after he begged them not to "let Scotland down".
[NEXT_CONCEPT]
Police investigating the murder of a woman found dead in the New Forest have been given more time to question a man.
[NEXT_CONCEPT]
Nine dogs including seven puppies have been stolen from a garden in West Lothian.