The study found that people find potential dates more attractive after having a positive face-to-face interaction with them.
According to the researchers from the University of Kansas in the US, evaluating someone based on physical attractiveness alone tells you little about what you will think about them once you meet face to face.
Rating someone's attractiveness before meeting them diminishes the rater's evaluation of that person afterwards, probably because the rater is comparing their conversation partner to all the other potential partners they saw online.
"With Tinder and user-directed online dating services, people try to manage the vast number of profiles by picking the most attractive people," said Jeffrey Hall, Associate Professor at the University of Kansas.
The researchers randomly divided participants into three groups. In the first group, heterosexual undergrads viewed photos of 10 opposite-sex individuals and rated each from one to 10 on physical attractiveness.
Then the participants were ushered into a lab, where they met for the first time one of those 10 people they had just rated and had a 10-minute conversation.
Afterwards, they were asked questions like 'Did you think this person was likeable, charismatic, responsive, smart and funny?'.
"Then we had them evaluate the same 10 photos again," Hall said.
"There was a significant difference in the attractiveness rating only for the person participants had just interacted with," he said.
"Two characteristics played an important role in whether the rating changed," Hall said.
"One was social attractiveness, which is whether we think we could be friends. It's not sexual attractiveness or romance, but likability," he said.
"The other was combined sense-of-humour or being-a-fun- person measure. If you're rated more highly in those two things, your appearance rating goes up more," he added.
Moreover, Hall said, "those who benefited the most were rated mid to low attractiveness, to begin with."
The other two groups in the study were participants who did not rate their partner before interacting.
One group pre-rated 10 photos, not including their partner, and the other group did not pre-rate anyone at all.
The results showed that if you rated your partner beforehand, it decreased your enjoyment of the interaction and your perceptions of your partner's personality.
"Online daters understandably become overwhelmed with the number of choices," Hall said.
"You reduce your evaluation of your conversation partner because you saw more attractive choices that you think you could have had. It makes your in-person experience worse," he said.
Researchers suggest that using physical attractiveness to sort people to date is a bad strategy.
It misses a lot of what makes for a good conversation, and the characteristics of a good conversational partner change how attractive they are in your eyes, Hall said.