Douglas Carswell, who defected from the Conservatives, won the seaside town of Clacton by 12,404 votes in the by-election, defeating Conservative Giles Watling who polled 8,709 votes.
UKIP was once dubbed by Prime Minister David Cameron as "fruitcakes, loonies and closet racists".
Its leader Nigel Farage hailed the triumph saying he had "shaken up British politics".
In the other by-election, Labour party held on to Heywood and Middleton seats but UKIP slashed its majority to 617.
Farage told BBC Breakfast that UKIP could hold the balance of power after next year's general election.
"The whole of British politics has been shaken up.
"Something big is happening here. People want change, they have had enough of career politicians in three parties," he said.
UKIP is the one which wants Britain to leave the European Union and to restrict immigration severely. People's support for the party has notably increased in the last two years amid soaring disenchantment with mainstream political parties.
