The 26-year-old actress and the actor, 34, were first rumoured to be dating in May, and over the summer, they were photographed together a few more times.
When asked if she is dating Hiddleston, Olsen said, "No."
"I mean, people can think what they want to think," she added.
Olsen, however, said she shares a great friendship with Hiddleston.
"I mean, we definitely are friends and we've known each other for about four years. And we happened to be at a restaurant at a wrong time having dinner," she said at the Toronto International Film Festival, where she promoted "I Saw the Light". "We all go out to dinner."