The Ibrox club opened their Premiership account with a win at Motherwell and followed that up with a 6-0 thumping of Dunfermline in the League Cup.
"It is always important to build on winning. They are believing in themselves as a team," Caixinha said ahead of Hibs' visit on Saturday.
"I am a guy who chases perfection, but when I get it I want more."
Alfredo Morelos netted twice in the demolition of Dunfermline, with Bruno Alves and Daniel Candeias also scoring maiden Rangers goals in a one-sided contest.
"One of the major parts of football is enjoyment," added Caixinha. "I felt freedom from the team. The early second goal helped, but the players worked hard to get it.
"It's just a good start. We want more. We want to keep progressing.
"I believe you need to be focused at all times, with a strong mentality and team spirit.
"The boys are doing well but we still have a long journey in front of us."
Rangers and Hibs last met in the top flight in 2012, while the most recent contest came in the 2016 Scottish Cup final.
A stoppage-time header from David Gray gave Hibs a dramatic 3-2 win at Hampden but Caixinha insists he is not interested in using previous pain as a motivation.
"Football is a moment, there are no two games played the same way," he explained.
"The contest is different, what was to be fought for is totally different, so we just want to focus on the three points that we want to take, that's all.
"My concern is to prepare the team, recover the players and make them ready for a strong start at 3pm on Saturday."
Meanwhile, the Rangers manager revealed that club captain Lee Wallace will have a scan on a leg injury.
Lee Hodson was drafted in as a late replacement for Wednesday's victory over Dunfermline at Ibrox.

The Cumbernauld-based firm, which also makes Rubicon and Tizer, said it would cut Irn Bru's sugar content from about 10g per 100ml to just below 5g.
It will reduce the calorie count per can from just under 140 to about 66.
AG Barr said the move was part of a "long-standing sugar reduction programme".
It confirmed in a stock market announcement on Wednesday that more than 90% of its portfolio would contain less than 5g of total sugar per 100ml by this autumn.
Tizer is already below that threshold, while the Rubicon carbonates range will all have less than 5g of sugar per 100ml by the middle of this year, the company said.
The UK government plans to introduce a levy on sugary drinks in April 2018.
The Soft Drinks Industry Levy (SDIL) will have two bands in the UK - one for soft drinks with more than 5g of sugar per 100ml and a higher one for drinks with more than 8g per 100ml.
Ministers hope the move will help tackle the nation's obesity problem.
A spokesman for the company said: "Irn Bru will still be made with sugar. We are not removing all the sugar, but we are reducing the amount.
"Having worked on this for some time, we are carefully blending and balancing a mix of sugar, sweeteners and our original and best flavour essence to deliver the great taste of Irn Bru but with less sugar.
"We're doing this because evidence shows that most consumers want to reduce their sugar intake while still enjoying great tasting drinks.
"86% of consumers are concerned about the amount of sugar in food and drink that they consume.
"That said, we have taken the soft drinks sugar tax thresholds into account when determining the levels of sugar reduction, but never compromising on taste."
In the past year, the company has introduced two new drinks - Irn Bru Xtra and Rubicon Spring - both of which contain no added sugar.
Chief executive Roger White said: "Evidence shows that consumers want to reduce their sugar intake while still enjoying great tasting drinks.
"We've responded by significantly reducing sugar across our portfolio in recent years, through reformulation and innovation.
"Today's announcement builds on this progress and we are now expanding our successful sugar reduction plans to include our iconic Irn Bru brand."
He added: "We will continue to respond to our consumers and adapt to their changing preferences, offering great tasting products that are right for this generation of consumers and the next."
Pedro Caixinha has urged his Rangers players to continue their winning start to the season against Hibernian.
[NEXT_CONCEPT]
AG Barr is to halve the amount of sugar in its leading Irn Bru brand, ahead of a government crackdown on the fizzy drinks industry.