Carter said he supports President Barack Obama's long-standing goal of closing the the site, but cautioned it remains unclear what to do with many of the inmates.
"Some of the people who are there at Guantanamo Bay have to be detained indefinitely, they've just got to be locked up," Carter told troops during a televised interview.
Currently, 116 inmates remain at the 13-year-old prison, which has drawn international condemnation for its treatment of prisoners and which critics say serves as a recruitment and propaganda tool for extremists.
"But there's another roughly half of them that are not safe to release, period."
The Pentagon is working with the Obama administration, including his top counterterror advisor Lisa Monaco, and the US Congress to come up with a plan to shutter Guantanamo and find alternative detention centers.
Officials are looking at military facilities like Fort Leavenworth, Kansas or the Navy Brig in Charleston, South Carolina as possible destinations for inmates. That may raise objections from hostile local politicians.
But he quickly became ensnared in legal and political wrangling over what to do with the inmates.
