How tech companies scour our digital lives for clues to our health

Your digital footprint — how often you post on social media, how quickly you scroll through your contacts, how frequently you check your phone late at night — could hold clues to your physical and mental health.

That at least is the theory behind an emerging field, digital phenotyping, that is trying to assess people’s well-being based on their interactions with digital devices. Researchers and technology companies are tracking users’ social media posts, calls, scrolls and clicks in search of behavior changes that could correlate with disease symptoms. Some of these services are opt-in. At least one is not. People typically touch their phones 2,617 per day, according to one study — leaving a particularly enticing trail of data to mine.

“Our interactions with the digital world could actually unlock secrets of disease,” said Dr Sachin H Jain, chief executive of CareMore Health, a health system, who has helped study Twitter posts for signs of sleep problems. Similar approaches, he said, might someday help gauge whether patients’ medicines are working. “It could help with understanding the effectiveness of treatments,” he said.

The field is so new and so little studied, however, that even proponents warn that some digital phenotyping may be no better at detecting health problems than a crystal ball.

If a sociable person suddenly stopped texting friends, for instance, it might indicate that he or she had become depressed, said Dr Steve Steinhubl, director of digital medicine at the Scripps Translational Science Institute in San Diego. Or “it could mean that somebody’s just going on a camping trip and has changed their normal behavior,” he said. “It’s this whole new potential for snake oil,” Dr Steinhubl said. That is not stopping the rush into the field — by start-ups and giants like Facebook — despite questions about efficacy and data privacy.

One of the most ambitious efforts is being conducted by Facebook. The company recently announced that it was using artificial intelligence to scan posts and live video streams on its social network for signs of possible suicidal thoughts. If the system detects certain language patterns — such as friends posting comments like “Can I help?” or “Are you OK?” — it may assign a certain algorithmic score to the post and alert a Facebook review team.

In some cases, Facebook sends users a supportive notice with suggestions like “Call a helpline.” In urgent cases, Facebook has worked with local authorities to dispatch help to the user’s location. The company said that, over a month, its response team had worked with emergency workers more than 100 times. Some health researchers applauded Facebook’s effort, which wades into the complex and fraught realm of mental health, as well-intentioned. But they also raised concerns. For one thing, Facebook has not published a study of the system’s accuracy and potential risks, such as inadvertently increasing user distress.
 

“It’s a great idea and a huge unmet need,” Dr Steinhubl said. Even so, he added, Facebook is “certainly right up to that line of practicing medicine not only without a license, but maybe without proof that what they are doing provides more benefit that harm.”

For another thing, Facebook is scanning user posts in the US and some other countries for signs of possible suicidal thoughts without giving users a choice of opting out of the scans. “Once you are characterised as suicidal, is that forever associated with your name?” said Frank Pasquale, a law professor at the University of Maryland who studies emerging health technologies. “Who has access to that information?” Will Nevius, a Facebook spokesman, said Facebook deleted the algorithmic scores associated with posts after 30 days. The cases involving emergency responders are kept in a separate system that is not tied to users’ profiles, he said.

Facebook said it had worked with suicide prevention groups when developing the effort. Nevius added that publishing a useful study would be complex because of the difficulty in removing personal data and “the delicate nature of the posts.”

Therapists traditionally diagnose depression by observing patients and asking them how they feel. Mindstrong Health, a mental health start-up in Palo Alto, Calif., is observing people’s smartphone use.

The company has developed a research platform to continuously monitor users’ phone habits, looking at changes in taps and clicks for hints about mood and memory changes associated with depression.

“We are building digital smoke alarms for people with mental illness,” said Dr Thomas R Insel, a Mindstrong co-founder and a former director of the National Institute of Mental Health.

Mindstrong’s research app tracks 1,000 smartphone-related data points — like how long it takes someone to scroll through a contact list and click on a name. The start-up recruited 200 volunteers to participate in pilot studies. 

Insel said a few of the signals, like changes in users’ keyboard accuracy and speed, correlated with similar motor skills changes that researchers could measure in lab tests.

Now the company is participating in a large government-funded study of trauma patients. Part of it involves using the Mindstrong platform to study whether patients who go on to develop post-traumatic stress disorder also develop corresponding changes in their smartphone use.

“We’ve got these really interesting statistical signals with very high correlations,” Insel said. “But whether that’s going to work in the real world of clinical care is something we’re looking at right now.”

He added that Mindstrong had tapped law and ethics experts to help examine the implications of its technology and develop ethical frameworks for using it.

“You want to think through all the unintended consequences early on,” Insel said, “so they don’t come back to bite you.”

Vetting Calls for Signs of Stress

The traditional use of a phone — talking — is also being examined for health clues. Sharecare, a digital health company based in Atlanta, offers a wellness app with an optional feature that analyzes users’ stress levels during phone calls.

The system uses pattern recognition technology to categorize users’ speech, the company said. After each call, the system delivers reports like “you seemed anxious” or “you seemed balanced.” It also characterizes users’ relationships with the people they call in terms of attitudes like “dominance” or “affection.”

Jeff Arnold, a co-founder of Sharecare, described the voice scan as “an emotional selfie.”

“If I can tell you your stress level in real time, it will in itself change your behavior,” said Arnold, who previously founded WebMD.

Health insurers and self-insured companies use Sharecare to promote wellness and manage health care costs. Sharecare is working with the Georgia Institute of Technology to study the effectiveness of its voice analysis service.

Jiten Chhabra, a health tech researcher at the university’s Interactive Media Technology Center, said volunteers who tried the voice-scanning feature reported feeling less stressed afterward. But he said it was too soon to tell whether the stress analysis itself directly caused the change — or whether volunteers had simply become more relaxed in their daily lives.

The company does not record the content of the calls it scans, it said. But the app did collect phone numbers for people on the other side of calls from Sharecare users, according to an analysis by The New York Times. The service did not inform people on the phone with Sharecare users that their relationships were being analyzed.

Jennifer Martin Hall, a spokeswoman for Sharecare, said the way the company protected data “makes it practically impossible for any Sharecare employees to access a phone number in the call information.” She added that characterizing users’ voice “analysis in terms of ‘relationships’ helps contextualize the relevance of their stress and enables them to be more mindful day to day.”

Other researchers said such pervasive scanning could also have the opposite effect — increasing stress on otherwise healthy people.

“It’s like we’re in school forever,” Professor Pasquale said, “and we’re being graded in all these ways forever by all the companies that have the most data about us.”

How they do it

Researchers and technology companies are tracking users’ social media posts, calls, scrolls and clicks in search of behaviour changes that could correlate with disease symptoms
 

Facebook recently announced that it was using artificial intelligence to scan posts and live video streams for signs of possible suicidal thoughts

Mindstrong Health, a mental health start-up, has developed a research platform to monitor users’ phone habits, looking at changes in taps and clicks for hints about mood and memory changes associated with depression.

© 2018 The New York Times News Service