Sherwood took over in February with Villa in the bottom three and led the club to Premier League safety and 17th.
"We gave the fans nothing to cheer about today but I promise them it will get better," Sherwood told BBC Sport.
"We've stayed in the division but we have a losing mentality. We don't want to be scraping relegation next season."
Villa were outclassed by Arsenal, whose victory at Wembley saw them become the most successful club in FA Cup history with a 12th title.
Sherwood acknowledged his side "came up short" against the Gunners, adding that his players "need to learn how to be winners".
Media playback is not supported on this device
"Of course we believed we could win this but I can't make any excuses," the 46-year-old added. "It's a learning curve and we need to build a new team.
"Some of the boys have already been told they won't be here next year or they can find other clubs. I want to be honest with players and don't want to be stringing experienced players along.
"But it's not just new additions to the squad but also getting some of the players we do have to play in a different way, to change the mentality of a few of them."
Villa captain Fabian Delph agreed Arsenal had been worthy winners and also stressed the importance of Villa finding a "winning mentality" next season.
"We were up against it, we tried to dig deep but their quality shone through and we struggled," the 25-year-old said.
"The manager said that we had it in us to cause an upset but we couldn't find the performance. But a few weeks ago we would have snapped your hand off to stay in the league and get to this final. Now it's important that we get a winning mentality.
"We have to forget about today, some players will come through the door to strengthen us and I am sure we will get better and better."

Evans took another step towards a possible world title shot in a scrappy fight that swung in favour of the champion in the later rounds.
Ormond was down twice at the end of the final round as two left hands to the body secured a 10-7 round for Evans.
The judges scored the contest 97-91, 99-89, 96-93 in favour of the Welshman.
Evans stated before the fight that he wants a shot at WBO lightweight champion Terry Flanagan and victory over Ormond will improve his WBO ranking of ninth.

The firm said it had signed a Letter of Intent to enter into exclusive negotiations with Liberty House Group.
More than 1,700 people are employed in the division, which has factories in Rotherham and Stocksbridge.
Steel union Community said it welcomed news of negotiations following "months of unnecessary stress and concern".
More on this and other South Yorkshire stories
The union's general secretary Roy Rickhuss said: "This is a positive step for the UK steel industry; however there remain huge challenges which government must address."
The union said it would be seeking urgent talks with Liberty House Group and would be asking what their plans were for investment, protecting jobs and providing decent pensions for members in retirement.
Tata Steel's UK boss Bimlendra Jha said the announcement was "an important step forward".
"We now look forward to working with Liberty on the due diligence and other work streams so that the sale can be successfully concluded," he said.
The Speciality Steels unit makes high-end components for the automotive, aerospace and oil industries.
In April, Tata sold its long-products division, based in Scunthorpe, to Greybull Capital, a UK-based investment firm.
Aston Villa manager Tim Sherwood promised to rid the club of its "losing mentality" after suffering a 4-0 defeat by Arsenal in the FA Cup final.
[NEXT_CONCEPT]
Welshman Craig Evans secured a unanimous points victory over Irishman Stephen Ormond to retain his WBO European lightweight title.
[NEXT_CONCEPT]
Tata Steel has announced its preferred bidder for the £100m sale of its Speciality Steels division based in South Yorkshire.