Sent into bat, KKR were bowled out for 107 runs in 18.5 balls, and Mumbai chased down the target with 33 balls to spare at the Chinnaswamy Stadium.
Leg-spinner Karn Sharma was the star performer with the ball for Mumbai, returning excellent figures of 4/16 - the joint best effort by a spinner in an IPL knockout game, while Jasprit Bumrah provided fine support to pick 3/7 in three overs.
Krunal Pandya showed maturity, finishing as Mumbai's top scorer with 45 off 30 balls, while Rohit contributed 26. Pandya struck eight boundaries.
Mumbai finished the run chase by scoring 111 for four after losing three wickets quickly, including the in-form Parthiv Patel (14) and his opening partner Lendl Simmons (3).
At 34 for three, Krunal Pandya joined Rohit and brought Mumbai closer to victory with a 54-run partnership off 40 balls.
Mumbai knocked off 16 runs in the 11th over bowled by Piyush Chawla (2/34), and scored eight in the next over to inch closer to their target.
At that juncture, KKR did get the wicket of Rohit, but Pandya along with Kieron Pollard (9) steered Mumbai home.
Mumbai Indians exploited the conditions and rattled Kolkata batsmen from the very first over, having them on the mat at 31 for five in seven overs, after Rohit Sharma opted to field.
After Bumrah dismissed Chris Lynn for four in the second over, the 29-year-old Sharma emerged as the wrecker-in-chief, sending back KKR skipper Gautam Gambhir (12) and Colin de Grandhomme (0) off consecutive balls, besides castling Narine in the fifth over.
Ishank Jaggi (28) and Surayakumar Yadav (31) put on 56 runs for the sixth wicket in 47 balls, during which they belted five fours and one six, and saved KKR from further embarrassment.
KKR began the innings rather poorly, losing Lynn early, when he tried to loft Bumrah in the second over of the match.
Mumbai struck for the second time in the fifth over, removing Narine for 10, thanks to a superb googly by Sharma and wicketkeeper Parthiv Patel did the rest at the M Chinnaswamy Stadium.
In the very next over, Bumrah got his second wicket of the match, when he trapped Robin Uthappa plumb in front of the wicket.
Mumbai dealt KKR another body blow by sending back their captain Gautam Gambhir after the batsman had scored 12 off 15 balls.
The situation got worse for the Knight Riders as Sharma snapped up Colin de Grandhomme.
A face-saving partnership ensued before Mitchell Johnson claimed the wickets of Piyush Chawla and Nathan Coulter-Nile, leaving Kolkata at 101 for eight.
Bumrah and Lasith Malinga got the last two batsmen cheaply.