The 47-year-old Rathore was till now serving as minister of state for information & broadcasting.
Rathore takes over from the pro-active Goel following a major cabinet reshuffle by Prime Minister Narendra Modi.
Colonel Rathore first stepped in a shooting range in the mid 1990s and some years later, he became India's first ever individual silver medal winner at the Olympic Games. He stood on the podium after finishing second in the men's double trap at the 2004 Athens Games.
A year before he created Olympic history, Rathore had won a silver medal at the 2003 World Championships in Sydney.
Prior to the Athens Games, Rathore was posted in Delhi with the army's marksmanship unit for two years to help him practise in the city's Tughlaqabad shooting range.
Rathore's recovery from a patella fracture and a prolapsed disc to an Olympic medal is well known. At Athens, he moved up from fifth in the qualifiers to win silver in the final.
Born into an army family, Rathore cleared the National Defence Academy examination and served in the Indian army, including fighting militants in Kashmir for two years, which, his mother believes, held him in good stead during the Olympic final.
A second-generation army officer, Rathore is attached to the Jaipur-based 9 Grenadiers. His father, Colonel (retd) Lakshman Singh Rathore, once commanded it.
After taking premature retirement from the Indian army, Rathore joined the Bharatiya Janata Party in 2013 and was sworn in as minister of state for information & broadcasting when the Modi government came to power in May, 2014.
Abhinav Bindra, who had bettered Rathore's Olympic feat with a first ever gold medal in 10m air rifle at Beijing Games, congratulated the minister.
"Delighted to see @Ra_THORe is the new sports minister. All the very very best!," Bindra wrote in his twitter handle.