The 29-year-old former judge of the singing reality show is resisting any attempts by ITV to woo her back in a bid to boost the struggling show, reported Sun online.
"Cheryl has no plans to return," her spokesperson said.
The Girls Aloud singer has also told her friends that she wants to concentrate on her music career for now.
"Cheryl really enjoyed her time on the show but she has moved on. She had three really good years and wants to leave it there," a source said.
Cole fell out with show supremo Simon Cowell last year after he ruthlessly dumped her from the US 'X Factor'.