Wall Street's ChatGPT Nightmare Is Over Before It Starts as Banks Crack Down

News February 24, 2023 at 04:13 PM
Share & Print

In a dark future for humans on Wall Street, banks fire traders en masse as artificial intelligence models like ChatGPT take over bond and commodities markets that were once too tough to automate.

But not yet. And maybe not ever.

Wall Street banks are taking steps to limit or prohibit its use, including JPMorgan Chase & Co., Bank of America Corp. and Citigroup Inc. And the traders and other professionals that are dabbling in the technology are quickly finding that, while it could make some of their most mundane tasks faster, the process is hardly seamless.

A salesperson at a US bank used ChatGPT's search engine on his personal device to get an overview of a client. The task was completed in less time than it would take to scour the internet, but the person said it couldn't be used in an internal report and had to be cross-checked for accuracy.

An oil trader used a version of ChatGPT to write a research note on the outlook for crude. It read well, she said, but the information was out-of-date and had to be fixed.

A stock trader in Taipei used it to compile key takeaways from U.S. earnings, sparing himself tedious copying and pasting between documents. Still, he based investment decisions on his own notes.

And a bond trader in mainland China wrote routine reports on policy analysis using AI to save time — part of which she then spent carefully fact-checking.

"It may save time, but we don't know if it's true, which is the biggest downside of the tool," Oded Netzer, a professor at Columbia Business School who researches data and technology, said in an interview. "It can be used like an intelligent colleague in the office, going over your work and improving it."

For all of the hoopla over the new breed of AI platforms, parts of the financial world are mostly using them like a teenager the night before an essay is due: generating text, and then — hopefully — making sure it's right. One credit analyst said he recently discovered a colleague was trying to use ChatGPT to draft earnings reports, but they were so disastrously flawed that he told him to knock it off.

Still, the analyst predicted, such technology will someday replace him.

"Technology continues to improve, and one of these days we will be at a point where the machines will out-think people," said Larry Tabb, an analyst at Bloomberg Intelligence.

But he said regulatory concerns, including Securities and Exchange Commission rules, will have the final word.

"When the SEC knocks on your door and asks why did you execute that transaction, you have to have a better answer than, 'Well the machine told me to,'" Tabb said. "You can gather insights and analysis from AI and then program your computers, but black-box trading models are generally not sanctioned on Wall Street because you don't know why the trading decision was made."

Nothing Afoot

The head of trading at one of the world's largest banks — speaking, like many others for this story, on the condition of anonymity — outlined some fairly obvious limitations for using the new AI to replace humans. For starters, in areas like stock trading, they've already been replaced.

While it's possible that ChatGPT and rival platforms may help banks realize their longtime goal of automating other areas such as fixed income markets, the executive said there's no project afoot to use them.

Perhaps someday.

The world has been buzzing over OpenIA's ChatGPT since its public unveiling late last year. The technology, using word prediction, can formulate essays, letters, lyrics and poetry practically instantaneously. And because the results are probabilistic, it gradually gets smarter and more nuanced.

But for now, one problem for Wall Street is that ChatGPT struggles with math.

The system scored slightly below 60% when an associate professor at Arizona State University fed it 1,000 mathematical word problems in early January. An update to the model later that month aimed at improving that.

Corporate earnings also seem to bedevil chatbots. Microsoft Corp.'s Bing chat, a cousin of ChatGPT, spit out incorrect earnings data in a demonstration at Microsoft's campus when the company unveiled the new product earlier this month.

Copyright 2023 Bloomberg. All rights reserved. This material may not be published, broadcast, rewritten, or redistributed.

Related Stories

Resource Center