After 10 men quit in the first round, Jack Sock of the United States and Uzbekistan's Denis Istomin retired on Thursday.
Despite temperatures rocketing to over 30 degrees over the first four days, in harness with crushing levels of humidity at Flushing Meadows, Federer offered no sympathy.
"We've been here in North America for some time. It's not like, all of a sudden, hot. I mean, it was more on the warmer side, but it's not like impossible, to be quite honest. Really no excuse for that," said the 34-year-old.
"Maybe some guys already came in too tired, whatever it was. I think you have to analyze case by case. But I think other players should be so fit that heat really shouldn't matter at that point, the ones we've been playing in."
The 22-year-old Sock was leading Belgium's Ruben Bemelmans 6-4, 6-4, 3-6, 1-2 when he started cramping and was forced to retire.
Sock, the 28th seed, then collapsed on court, where he was treated with ice packs and cold towels by US Open medical staff before being carried off to the shade of the Grandstand Court.
Two women also retired in the first round, taking the total for this year's US Open to 14.
The carnage led again to suggestions that the men, who play the best of five sets, should be accorded the same protection as women players.
The WTA allows a 10-minute break between the second and third sets of women's matches when the mercury bursts through 30.1 degrees.
Third seed Andy Murray, who battled through five sets to beat France's Adrian Mannarino on Thursday, believes there is scope for men to have a similar time-out.
"I don't know exactly what it's for. But I guess you get the chance to sort of go off and change, get under a cold shower if you want to.
