The oath of office to Murmu, who will turn 60 next month, was administered by Jammu and Kashmir Chief Justice Gita Mittal at a simple function held at Raj Bhawan at the foothills of the Zaberwan range.
The warrant of appointment of Murmu, a 1985 batch IAS officer of Gujarat cadre, was read out by Chief Secretary B V R Subrahmanyam.
BJP leader Jugal Kishore and Rajya Sabha MP and PDP member Nazir Laway were among the over 250 guests present at the function.
Murmu, a native of Odisha, had worked closely with Prime Minister Narendra Modi in Gujarat as his additional principal secretary during his tenure as chief minister.
Murmu is an MBA in public service and also holds a post graduate in political science.
Satya Pal Malik, who was the last governor of Jammu and Kashmir, moved to Goa for the remainder of his tenure as governor.
Malik was first appointed governor of Bihar in September 2017 and moved to Jammu and Kashmir in August last year.
The two union territories -- Ladakh and Jammu and Kashmir -- came into existence on Thursday after the central government abrogated the special status of the state and bifurcated it on August 5.