Monday, February 17, 2025 | 11:22 PM ISTहिंदी में पढें
Business Standard
Notification Icon
userprofile IconSearch

<b>Steve Lohr:</b> If algorithms know all, how much should humans help?

The application of data science to fields like medical diagnosis raises questions of when human supervision of an algorithm's results is needed

Image

Steve Lohr
Armies of the finest minds in computer science have dedicated themselves to improving the odds of making a sale. The internet-era abundance of data and clever software has opened the door to tailored marketing, targeted advertising and personalised product recommendations.

Shake your head if you like, but that's no small thing. Just look at the technology-driven shake-up in the advertising, media and retail industries.

This automated decision-making is designed to take the human out of the equation, but it is an all-too-human impulse to want someone looking over the result spewed out of the computer. Many data quants see marketing as a low-risk - and, yes, lucrative - petri dish in which to hone the tools of an emerging science. "What happens if my algorithm is wrong? Someone sees the wrong ad," said Claudia Perlich, a data scientist who works for an ad-targeting start-up. "What's the harm? It's not a false positive for breast cancer."

But the stakes are rising as the methods and mind-set of data science spread across the economy and society. Big companies and start-ups are beginning to use the technology in decisions like medical diagnosis, crime prevention and loan approvals. The application of data science to such fields raises questions of when close human supervision of an algorithm's results is needed.

These questions are spurring a branch of academic study known as algorithmic accountability. Public interest and civil rights organisations are scrutinising the implications of data science, both the pitfalls and the potential.

Take consumer lending, a market with several big data start-ups. Its methods amount to a digital-age twist on the most basic tenet of banking: Know your customer. By harvesting data sources like social network connections, or even by looking at how an applicant fills out online forms, the new data lenders say they can know borrowers as never before, and more accurately predict whether they will repay than they could have by simply looking at a person's credit history.

The promise is more efficient loan underwriting and pricing, saving millions of people billions of dollars. But big data lending depends on software algorithms poring through mountains of data, learning as they go. It is a highly complex, automated system - and even enthusiasts have qualms.

"A decision is made about you, and you have no idea why it was done," said Rajeev Date, an investor in data-science lenders and a former deputy director of Consumer Financial Protection Bureau. "That is disquieting."

The concern is similar in other fields. Since its Watson computer beat human "Jeopardy" champions four years ago, IBM has taken its data-driven artificial intelligence technology well beyond brainy games. Health care has been a major initiative. The history of "expert" decision-support technology in medicine has been disappointing; the systems have not been smart or fast enough to really help doctors in day-to-day practice.

But IBM scientists in collaboration with researchers at leading medical groups are making progress. Watson can read through medical documents at a pace incomprehensible to humans: many thousands per second, searching for clues, correlations and insights.

The software has been used to help train medical students and is starting to be deployed in clinical settings in oncology, offering diagnostic and treatment recommendations as a kind of quick-witted digital assistant.

IBM has also developed a software program called Watson Paths, which is a visual tool that allows a doctor to see the underlying evidence and inference paths Watson took in making a recommendation.

"It's not sufficient to give a black-box answer," said Eric Brown, IBM's director of Watson technologies.

Watson Paths points to the need for some machine-to-man translation as data science advances. As Danny Hillis, an artificial intelligence expert, put it, "The key thing that will make it work and make it acceptable to society is storytelling." Not so much literal storytelling, but more an understandable audit trail that explains how an automated decision was made. "How does it relate to us?" Hillis said. "How much of this decision is the machine and how much is human?"

Keeping a human in the loop is one approach. The new data-science lenders are animated by data and software. But one of the start-ups in San Francisco, Earnest, has at least one of its staff members review the predictive recommendations of its software, even if the algorithms are rarely overruled. "We think the human element will always be an important piece in our process to make sure we're getting it right," said Louis Beryl, co-founder and chief executive of Earnest.

But such a stance, others say, amounts to a comforting illusion - good marketing perhaps, but not necessarily good data science. Giving a person veto power in algorithmic systems, they say, introduces human bias. The promise of big data decision-making, after all, is that decisions based on data and analysis - more science, less gut feel and rule of thumb - will yield better outcomes.

Yet even if optimism is justified, there is a serious challenge, given the complexity and opacity of data science. Will a technology that promises large benefits on average sufficiently protect the individual from a mysterious and wayward decision that might have a lasting effect on a person's life?

One solution, according to Gary King, director of Harvard's Institute for Quantitative Social Science, may be for the human creators of the scoring algorithms to tweak them not so much for maximum efficiency or profit but to give somewhat greater weight to the individual, reducing the risk of getting it wrong.

In banking, for example, an algorithm might be tuned to reduce the probability of misclassifying a loan applicant as a deadbeat, even if the trade-off is a few more delinquent loans for the lender.

"The goal," King said, "is not necessarily to have a human look at the outcome afterwards, but to improve the quality of the classification for the individual."

Steve Lohr, a technology reporter for The Times, is the author of "Data-ism."
© 2015 The New York Times News Service
 
Disclaimer: These are personal views of the writer. They do not necessarily reflect the opinion of www.business-standard.com or the Business Standard newspaper

Don't miss the most important news and views of the day. Get them on our Telegram channel

First Published: Apr 07 2015 | 9:44 PM IST

Explore News