The 22-year-old popstar could cut ties with Cowell and wants to "move as far away from the band (One Direction) as possible", reported Digital Spy.
Earlier this year, Styles broke off his relationship with Modest Management in favour of being handled by US agent Jeffrey Azoff, who is apparently good friends with the singer.
There's even talk of the pair investing in a Hollywood management company.