Romelu Lukaku’s 89th-minute strike at Old Trafford, his fourth goal in two games, fired United back into the top four at the expense of Arsenal, who were held by Tottenham.
Solskjaer’s side will be aiming to overturn a 2-0 deficit in the Champions League last 16, second leg at PSG on Wednesday.
“The dressing room was bouncing, they were happy, looking forward to Wednesday,” said the Norwegian.
“We’re playing against one of the best teams in Europe with some absolute quality players but who knows? This has happened before, so many great comebacks.
“The Champions League has given quite a few of those comebacks, PSG against Barcelona for example. We have to do it away from home, but we have eight straight wins away from home.
“We’re confident away from home and if we’re in the game with 20 or 30 minutes to go, who knows?”
Solskjaer’s mentor Alex Ferguson became renowned for his team’s last-minute winners during a golden era that featured the current United interim manager, who scored a stoppage-time winner in the 1999 Champions League final against Bayern Munich.
And the Norwegian appeared to have summoned the spirits of the Ferguson era, after United claimed a crucial three points against lowly Southampton.
“It was like the old days,” beamed Solskjaer, whose side now lead Arsenal by one point in the race for Champions League qualification, with Chelsea four points further back with two games in hand.
“I’ve been part of that so many times myself. We spoke about it at half-time and said get the ball in the box and play more forward because the Stretford End will suck it in for you.
“I don’t want to take anything away from Andreas (Pereira) and Romelu, because they were great finishes, but that spirit is just part of this club’s history. We have done it so many times before.”
United appeared to be heading towards a disappointing home draw in the closing moments, when substitute Fred exchanged passes with Luke Shaw and found the in-form Lukaku on the edge of the area.
The Belgian striker did the rest with a pinpoint 18-yard finish into the bottom corner.
Solskjaer could even enjoy the luxury of a missed injury-time penalty when Paul Pogba’s effort was well saved by Angus Gunn following Stuart Armstrong’s trip on Marcus Rashford.
But the finale, which extended Solskjaer’s unbeaten run in domestic action with United to 15 games, was much-needed after it appeared his team would drop points in the chase for a top-four finish.
In a game of spectacular goals, Saints midfielder James Ward-Prowse struck a superb 25-yard free-kick after 74 minutes to make the score 2-2.
Pereira had equalised for United in the 53rd minute as substitute Diogo Dalot combined with Lukaku, forcing the ball through to the young Brazilian who took a touch before curling a magnificent finish into the Southampton goal.
After a lacklustre first-half display, United were now in an irrepressible mood and took the lead just six minutes later through Lukaku.
Pereira showed his strength to hold off challengers in midfield before slipping through an excellently-timed pass for Lukaku who finished clinically from 15 yards.
It was an impressive turnaround from Solskjaer’s team, who had been stunned by an incredible 26th-minute goal by Southampton wing-back Yan Valery.
But there were question marks over David De Gea as Charlie Austin passed wide to the overlapping 20-year-old and his immaculate 30-yard strike beat the United goalkeeper at his near post.
The dramatic conclusion was partly the result of some luck on Solskjaer’s part as an injury to the ineffective Alexis Sanchez early in the second half forced him to bring on the far more influential Dalot.