"I do not want to act (as the court commissioner) and I cannot discuss the reasons in the open court," Najimi Waziri, standing counsel for the Delhi Government, told Justice Rajiv Shakdher.
Waziri said he may specify the reasons for not following the court's order if its proceedings are held in-camera.
The court then fixed the matter for hearing on October 15.
Justice Shakdher had earlier said, "In order to ascertain the correct position, I propose to appoint Najimi Waziri as a court commissioner who will visit the said site along with the MCD's City Zone Deputy Commissioner Subhash Chandra along with the ACP of Central District."
The court had also ordered demolition of the structures, if found illegal by the committee appointed by it.
Waziri expressed his inability to be part of the panel during hearing of a plea of Suhail Ahmed Khan, who had alleged that Shahi Imam of the 16th century historic mosque, Maulana Syed Ahmed Bhukhari, and his three brothers, Syed Hasan Bhukhari, Syed Yahia Bhukhari and Syed Tarikh Bhukhari have been raising illegal structures adjacent to the mosque.
Irked by the alleged illegal construction, the court had asked the authorities including Archaeological Survey of India (ASI) to stop dis-figuration of the site near the monument.
The court had also asked the city police, the DDA and the MCD to respond to petition and act on it.
The petition, filed through lawyers C Prakash and Rajesh Kumar Yadav, alleged Bhukhari brothers were illegally encroaching sites inside the mosque premises.
Such acts of the Imam and others are "earning disrepute to the nation from all over the world," it said.