Ardie Savea scored a pushover try in the 79th minute to level the scores and another replacement, Richie Mo’unga, slotted a simple conversion for a dramatic triumph.
After turning over level at 6-6, South Africa built a 30-13 lead on the hour only to be pipped by the resilient world champions before a capacity 52,000 crowd.
The result was a bitter blow for the Springboks, who won the possession and territory battles, as they sought successive wins over the All Blacks for the first time since 2009.
New Zealand were stunned 36-34 at home by South Africa last month, leading scrum-half TJ Perenara to say they wanted to “right a wrong” in Pretoria.
The outcome had no bearing on the southern hemisphere championship, with the All Blacks clinching first place after a win in Argentina last weekend and South Africa coming second.
All Blacks skipper Kieran Read said: “You have got to have belief and we had that deep down to come back. Both teams were going at it and the Springboks were outstanding.
“On the two games we were split on the scoreboard and tonight we were a bit fortunate to come out on top.
“We did not have the territory and possession, but we hung in there and, especially when they got momentum, we came back.
“I would like to thank the crowd as the atmosphere was fantastic. This is a great place to play.”
Springboks skipper Siya Kolisi conceded that “we let it get away. We knew it would be tough, but we let it go at the end. It was tough and I am really proud of the boys.
“It was much better than last week (against Australia), but there were still soft moments. We don’t need them at this level and we must learn and see how we can improve.”
New Zealand were forced to make a pre-match change with centre Ryan Crotty replacing ill Jack Goodhue.