Health trends captured by categorising Twitter banter

By sifting through the busy flood of banter on Twitter, is it possible to also track important public health trends? Two Johns Hopkins University computer scientists would respond with a one-word tweet: “Yes!”

Mark Dredze and Michael J Paul fed two billion public tweets posted between May 2009 and October 2010 into computers. They then used software to filter out the 1.5 million messages that referred to health matters. Identities of the tweeters were not collected by Dredze, a researcher at the university’s Human Language Technology Center of Excellence and an assistant research professor of computer science, and Paul, a doctoral student.

To find the health-related posts among the billions of messages in their original pool, the researchers applied a filtering and categorisation system that they devised. With this tool, computers can be taught to disregard phrases that do not really relate to one’s health, even though they contain a word commonly used in a health context. Once the unrelated tweets were removed, the remaining results provided some surprising findings. In about 200,000 of the health-related tweets, the researchers were able to draw on user-provided public information to identify the geographic state from which the message was sent. That allowed them to track some trends by time and place, such as when the allergy and flu seasons peaked in various parts of the country.

In some cases, the researchers said they probably leart some things even the tweeters’ doctors were not aware of. These included which over-the-counter medicines the posters were using to treat their symptoms at home. By sorting these health-related tweets into electronic ‘piles’, Dredze and Paul uncovered intriguing patterns about allergies, flu cases, insomnia, cancer, obesity, depression, pain and other ailments. They would present their complete study at the International Conference on Weblogs and Social Media on July 18 in Barcelona.

Dredze and Paul have already begun talking to public health scientists, including some affiliated with Johns Hopkins, who say future studies of tweets could uncover even more useful data, not only about posters’ medical problems, but also about public perceptions concerning illnesses, medications and other health issues.