Some 50,000 people stood for a minute of silence in Hiroshima's peace park near the epicentre of the early morning blast on August 6, 1945, that killed up to 140,000 people.
The bombing of Nagasaki three days later killed tens of thousands more, prompting Japan's surrender to the World War II Allies.
Prime Minister Shinzo Abe said that as the sole country to face nuclear attack, Japan has the duty to seek to wipe out nuclear weapons.
More than 100,000 people remain displaced because of radioactivity near the plant. Abe favours restarting plants under new safety guidelines, while many Japanese oppose such restarts.
In a "peace declaration" speech, Hiroshima's mayor, Kazumi Matsui, chided the government for its efforts to restart the nuclear plants and to export nuclear technology to other countries.
"This summer, eastern Japan is still suffering the aftermath of the great earthquake and the nuclear accident. The desperate struggle to recover hometowns continues. The people of Hiroshima know well the ordeal of recovery," Matsui said.
A recent agreement on discussing nuclear energy cooperation with India, he said, would likely hinder efforts to abolish nuclear weapons.
