Opponents said the plan would have robbed business of skilled workers and torpedoed accords that enhance non-EU member Switzerland's access to the crucial EU single market.
Under Switzerland's system of direct democracy, the referendum could have forced the government to annul the EU agreement if negotiations did not produce a deal on ending the pact voluntarily, something Brussels has ruled out.
A "guillotine clause" meant that ending free movement would have toppled other bilateral pacts on land and air transport, procurement, technical barriers to trade, and research.
Around two-thirds of the 2.1 million foreigners living in Switzerland in 2019 were citizens of the EU, as well as Iceland, Norway and Liechtenstein, which with Switzerland are members of the European Free Trade Association (EFTA).More than 450,000 Swiss live in the EU.