Detectives visited his home in Pendleton, Lancashire, yesterday following allegations by two men who were in their 20s at the time of the alleged attacks, which are said to have taken place at the property in July 2009 and March 2013.
The house was searched by officers and Deputy House of Commons speaker Evans was held in custody at Preston police station. He was subsequently released on bail till June 19.
Commons speaker John Bercow last night said: "It's been a total shock. Nigel is a popular and well-liked character. No one can quite believe what's happened."
Evans is openly gay, having come out as homosexual in 2010, the year he was elected to his current post.
In the 2010 general election Evans retained his seat with a majority of 14,769 over Labour, with Ukip making a then strong showing in fourth place with 3,496, or 6.7 per cent of the vote.
"The man will be interviewed at a police station in Lancashire during the course of the day. The offences are alleged to have been committed in Pendleton between July 2009 and March 2013. We take all allegations of a sexual nature extremely seriously and understand how difficult it can be for victims to have the confidence to come forward.
"As a constabulary, we are committed to investigating sexual offences sensitively but robustly, recognising the impact that these types of crimes have on victims. We would encourage anyone who has experienced sexual abuse, or who has information about it, to have the confidence to report it to us knowing that we will take it seriously, deal with it sensitively and investigate it thoroughly."
Evans is known as a rightwinger who voted against the minimum wage and is a strong Eurosceptic.
Originally from Swansea, he was John Major's spokesman on Welsh affairs. He held various positions as a parliamentary private secretary in Major's government including to Tony Baldry in the ministry of agriculture, food and fisheries.
In 1999 he was appointed as vice-chairman of the Conservative party, a post he held until 2001. At the time of his coming out as gay he said he was tired of living a lie and dealing with threats of blackmail from political enemies.
