In the tournament's biggest upset, the Swiss world number two had no answer to Italian Andreas Seppi, who he had conquered in their past 10 meetings.
"Just a bad day. I wish I could have played better but clearly it was tough losing the first two sets," Federer said after crashing out in the third round 6-4, 7-6 (7/5), 4-6, 7-6 (7/5).
"I had chances to get back into it but let it slip. It's a disappointing loss."
Federer was chasing his fifth Australian Open crown, but has now not won a Grand Slam title since Wimbledon in 2012.
He started sluggishly and when he was broken to love in the first set, giving Seppi a 5-4 lead, it was clear something was amiss with Federer's game.
"To beat Roger first time, especially in a Grand Slam, best-of-five, is a special moment for me," said the 46th ranked Seppi, who had only taken one set off him in their previous 10 matches.
"The other night was one of the toughest times I have spent on the court and my body wasn't very well and I felt very lucky to get through," he said.
"In general I think I was playing better than I did the other day, I was feeling better tonight."
Sharapova was also on fire, blitzing past Zarina Diyaz of Kazakhstan 6-1, 6-1 to erase memories of her big scare in the second round when she had to save two match points.
Murray was also in form, easily beating Portugal's Joao Sousa 6-1, 6-1, 7-5 and is yet to be seriously tested at Melbourne Park.
