Brathwaite made 65 not out and Samuels was unbeaten on 60 as the West Indies reached 144 for two in reply to South Africa's 417 for eight declared.
Morne Morkel was the only successful South African bowler, taking two wickets in two balls.
He ended a 55-run opening partnership between Brathwaite and Devon Smith by having Smith caught at first slip, with Leon Johnson edging the next delivery to third slip.
With most of the second day lost to rain, South Africa were hoping to score quick runs on Sunday morning before making inroads into a potentially fragile West Indian batting line-up. But they were frustrated on both counts.
The West Indies took four wickets before lunch, including the prize scalps of Amla and AB de Villiers, who were dismissed in successive overs.
Only 80 runs were scored during the morning but Dale Steyn went on the rampage after the interval, thrashing 58 off 28 balls with six fours and five sixes. Amla declared when Steyn was caught attempting another big hit off left-arm spinner Suleiman Benn.
New cap Temba Bavuma scored a boundary off his first ball, when a defensive shot flew to third man off a thick edge from Holder but De Villiers was out in the next over, bowled by Taylor.
