The 55-year-old actor, who became popular for playing Lucius Malfoy in the "Harry Potter" series, made the claims in an interview with The Times, according to The Independent.
Isaacs said he is often asked the question that why did he not do anything against men who have indulged in sexual transgression if he knows them.
"The problem is that almost all of the most egregious things that people are being charged with went on behind closed doors," the actor said.
"I can think of half a dozen men I'd like to name, and who I think should probably be in prison. But they would just sue me and nothing bad would happen to them. All I can do is choose not to work with them," he added.
Isaacs, who promoting his Netflix show "OA" during the interview, said he is happy that the conversation around the issue is "going in the right direction".
"(It) is an incredibly healthy thing. I wish we could get to the end game very quickly," he added.
(This story has not been edited by Business Standard staff and is auto-generated from a syndicated feed.)