Cowell, 54, and Silverman, who are parents to three-month-old Eric, will apparently be spending more time in Manhattan and the Hamptons this summer, reported the New York Post.
While the media mogul last year rented a USD 26,500-a-month Upper East Side apartment, they are still looking to buy in the city.
Earlier, the couple were spotted searching for a place on the Upper East Side, close to Silverman's other son, Adam, seven, who lives with his father Andrew Silverman.
Cowell's representative have not yet commented.