The 36-year-old is wanted by Tunisian authorities on suspicion of "participating in planning and carrying out" the attack.
He is also accused of involvement in the deadly jihadist assault on the border town of Ben Guerdane last March, the prosecutor's office in the western state of Hesse said in a statement.
Tunisian authorities issued a warrant for his arrest in June 2016, it added.
The suspect was taken into custody early today as police carried out sweeping anti-terror raids in Frankfurt and nearby towns.
In Germany, the Tunisian national is suspected of recruiting for the IS jihadist group and of building a network of IS supporters with the goal of staging an attack in the country, he added.
Plans for the attack were still "at a very early stage", he said. "There was no concrete danger of an attack."
The suspect arrived in Germany as an asylum seeker in August 2015, the prosecutors said, after already living in the country for a decade some years earlier.
He was arrested the following August on an outstanding 2008 conviction for causing bodily harm.
"As the Tunisian authorities, despite repeated reminders from the German authorities, failed to supply the necessary deportation documents within the 40-day period, the suspect was released on November 4, 2016," the statement said.
He was kept under surveillance from the day of his release until his arrest today, it added.
IS claimed responsibility for the Bardo attack during which two gunmen opened fire at the museum, killing 21 foreign tourists and a police officer.
The Ben Guerdane attack last March saw dozens of heavily armed jihadists cross into the frontier town from Libya to launch coordinated attacks on police and army posts, killing seven civilians and 13 security personnel.
Disclaimer: No Business Standard Journalist was involved in creation of this content
