Key's flag change proposal will be put to a vote later this year and the leader insisted there was still time to win over the public.
"It's always going to be a tough ask to change the flag by a public referendum," he told Radio New Zealand.
"You've got to engage people, you've got to get them to think through the issues... No one's arguing it's not a big challenge."
Only 25 per cent of 1,000 respondents wanted a new flag, with six per cent undecided.
The poll was taken after a government committee this month unveiled four potential designs for a new flag, to a lacklustre response from the public.
Three of the designs include a silver fern leaf, the informal national emblem which Key has said he wants on the flag.
The fourth depicts a spiralling black-and-white koru, or fern frond, a traditional Maori symbol of new life and creation.
Key has made the flag reform issue his pet project since his conservative government won a third term late last year, arguing the country needs a banner "that screams New Zealand".
He has also expressed frustration the existing flag -- which also features four red stars representing the Southern Cross on a dark blue background -- is frequently confused with Australia's banner.
Despite the poll, he said he still believed most people had open minds on the issue.
"In fact, the vast, overwhelming bulk want a change.
