Kavinder notched up a remarkable 3-2 victory over Algeria's two-time world medallist Mohamed Flissi to make the last eight stage.
Gaurav Bidhuri (56kg) also advanced to the quarterfinals with an upset win over Ukrainian Mykola Butsenko. Both Kavinder and Gaurav are debutants at the mega-event.
However, Shiva, seeded fifth in the lightweight 60kg category, bowed out without exchanging a single blow, forced to give a walkover after being laid low by food poisoning and high fever.
"He was throwing up last night and this morning, he was running high fever as well. He couldn't have fought, his body was not holding up. We tried to do everything but he was very weak," an Indian team official told PTI.
Shiva had won a bronze medal at the 2015 World Championships in Doha.
Also making an exit from the tournament was Manoj Kumar (69kg).
The former Commonwealth Games gold-medallist went down in a split verdict to fourth-seeded Venezuelan Gabriel Maestre Perez.
The 31-year-old Manoj gave it his all and matched up to his higher-ranked rival quite well but could not get the judges' nod.
However, Kavinder and Gaurav ensured that India had something to celebrate.
With Amit Phangal (49kg) already through to quarters,
The Delhi-boxer, who got into the event on a wildcard after missing out on automatic qualification, held his nerves and matched his Butsenko punch for punch in a thoroughly engrossing contest.
The two boxers attacked each other relentlessly and expectedly the judges came out with a split verdict in favour of the Indian.
Also making an early exit was Asian silver-medallist Sumit Sangwan (91kg) who lost to Australia's Jason Whateley in a split decision.
Earlier, Gaurav came up with a determined performance against a rival representing one of international boxing's powerhouses.
Krishan, the two-time Asian Games medallist looked sluggish against his 19-year-old rival, who has been nicknamed 'The Future' in his country for the potential he has shown.
Whittaker's stronger legs helped him glide smoothly inside the ring, even as Vikas struggled to keep his balance after every attempted attack at the Englishman.
Sumit competed in the last bout of the evening and lost after a strong start. He dominated the opening round but Whateley fought back in the next two to clinch the issue.
Disclaimer: No Business Standard Journalist was involved in creation of this content
