He was administered the oath by Governor Ganga Prasad at Paljor Stadium.
Golay is not a member of the state assembly at present as he did not contest the polls.
Hundreds of SKM supporters who were at the stadium cheered the 51-year-old party chief when he was taking the oath in Nepali language.
Former chief minister Pawan Kumar Chamling and senior Sikkim Democratic Front (SDF) leaders were conspicuous by their absence at the swearing-in.
The SKM, founded in 2013, won a slender majority in the 32-member Sikkim Legislative Assembly by bagging 17 seats against 15 won by the SDF.
The Chamling government was ousted by the SKM after over 24 years.