But this afternoon against a World XV at Newlands, the experienced lock forward will make his return for his beloved Springboks as the leader.
So how did all this come about for the man who retired three years ago as a 34-year-old, after the disappointment of the 2011 World Cup?
After hanging up his boots after South Africa were knocked out by Australia at the 2011 World Cup in New Zealand, he pursued a career as a television analyst for SuperSport and then later opted for a coaching role with the Bulls.
After making the decision to return to professional rugby, he started proving his critics wrong as he went from strength to strength with each match he played for the Bulls in Super Rugby.
It was all about Springbok coach Heyneke Meyer, who planted the initial seed, Matfield revealed earlier this week. The pair shared glorious moments at the Bulls together.
And then there’s the World Cup. Unfinished business.
“When Heyneke first tested the waters a year ago about a possible return, I told him my only motivation would be to realise my dream of playing in next year’s World Cup,” said Matfield.
“The way we bowed out in the 2011 World Cup was very disappointing and I truly believe Heyneke is busy putting together a strong Bok team who can realistically challenge for the World Cup title next year in England,” said Matfield.
Today at Newlands, Matfield will captain the side in the absence of the injured Jean de Villiers, but said he had his reservations about accepting the captain’s role.
“When Heyneke approached me I thought – am I doing the right thing?
“I realised there would be opinions about my age, but I enjoy being captain and I fulfilled that role at the Bulls.
“I also realise it is a big responsibility,” he said.
Matfield said if he had to be honest with himself, being back in the Bok jersey exceeded his wildest
expectations – and being captain on top of it was all a lot to take in.
But Meyer never had any doubt.
“I think because the majority of our team have already played under the captaincy of Victor, they only have the highest praise for him and would follow him blindly,” said Meyer, adding that captaincy in the side is crucial to the team’s success.
“I think when Jean returns as captain, Victor can continue as vice-captain and I believe the two of them will work very well together,” said Meyer.
However, the coach said before he considered Matfield as skipper he first of all had to make sure the lineout maestro was still a first-choice selection in the side.
“I think he is still without peer in world rugby. He has shown he is still the best in his position in the world, and he also knows what it takes being a captain.
“He is respected worldwide,” said Meyer.
One of the reasons Matfield is in such excellent shape, despite his 37 years of age, is because he always looked after himself, even in his time away from the field, said Meyer.
“I think the issue of becoming older is that you have to work so much harder than everyone else.”