ChatGPT Use Could Spell Disaster for Advisors: Author

Q&A March 30, 2023 at 01:07 PM

Share & Print

Financial advisors should "be extremely wary. ChatGPT's unreliability creates considerable legal and reputational harm for any business that uses it for consequential text generation," warns Gary Smith — an economics professor at Pomona College in Claremont, California, and author — in an interview with ThinkAdvisor.

"Intelligent advisors should be thinking about what the pitfalls and perils are for the future," of using this tech, stresses Smith, who became a multimillionaire by investing in stocks.

The professor, whose research often focuses on stock market anomalies and the statistical pitfalls of investing, has released a new book, "Distrust: Big Data, Data-Torturing, and the Assault on Science" (Oxford University Press, Feb. 21, 2023).

"Science is currently under attack, and scientists are losing credibility," which is "a tragedy," he writes.

In the interview, Smith discusses ChatGPT's tendency to serve up information that's totally factually incorrect.

"The Achilles' heel of AI is that it doesn't understand words," says Smith, who detected the dot-com bubble early on.

In the interview, he shines an intense light on the danger that, based on ChatGPT's launch, "really smart people … think that the moment is here when computers are smarter than humans. But they're not," he argues.

Smith also discusses the answers that ChatGPT provided when he asked questions about portfolio management and asset allocation; and he cites a series of questions that TaxBuzz asked ChatGPT about calculating income tax returns, every one of which it got wrong.

Smith, who taught at Yale University for seven years, is the author or co-author of 15 books, among them, "The AI Delusion" (2018) and "Money Machine" (2017), about value investing. ThinkAdvisor recently held a phone interview with Smith, who maintains that large language models (LLMs), such as ChatGPT, are too unreliable to make decisions and "may well be the catalyst for calamity."

LLMs "are prone to spouting nonsense," he notes. For instance, he asked ChatGPT, "How many bears have Russians sent into space?"

Answer: "About 49 … since 1957," and their names include "Alyosha, Ugolek, Belka, Strelka, Zvezdochka, Pushinka and Vladimir." Obviously, LLMs "are not trained to distinguish between true and false statements," Smith points out.

Here are highlights of our conversation:

THINKADVISOR: There's big excitement about the availability of the free chatbot, ChatGPT, from OpenAI. Financial firms are starting to integrate it into their platforms. Your thoughts?

GARY SMITH: With ChatGPT, it seems like you're talking with a really smart human. So a lot of people are thinking that the moment is here when computers are smarter than humans.

The danger is that so many really smart people think that computers are smart enough now to trust to make decisions, such as when to get in and out of the stock market or whether interest rates are going up or down.

Large language models [AI algorithms] can recite historical data, but they can't make predictions about the future.

What's AI's biggest deficiency?

The Achilles' heel of AI is that it doesn't understand words. It doesn't understand whether the correlation it finds makes sense or not.

AI algorithms are really good at finding statistical patterns, but correlation is not causation.

Big banks like JPMorgan Chase and Bank of America forbid their employees to use ChatGPT. What are these firms thinking?

Even Sam Altman, the CEO of OpenAI, which created and introduced ChatGPT, says it's still unreliable and sometimes factually incorrect, so it's not to be relied upon for anything consequential.

But why are companies rushing to add it?

There are people who are opportunistic and want to cash in on AI. They think they can sell a product or attract money by saying, "We're going to use this amazing technology."

They'll say, for example, "You ought to invest in [or with] us because we're using ChatGPT." Artificial Intelligence was the National Marketing Word of 2017 [named by the Association of National Advertisers].

If an [investment] manager says, "We're using AI. Give us your money to manage," a lot of people will fall for that because they think ChatGPT and other large language models are really smart now. But they're not.

In your new book, "Distrust," you give examples of investment companies founded on the assumption that they would use AI to beat the market. How have they made out?

On average, they have done average — some do better, some do worse.

It's like the dot-com bubble, where you added ".com" to your name and the value of your stock went up.

Here you're saying you're using AI, and the value of your company goes up, even though you don't say exactly how you're using it.

Just put that label on and hope people are persuaded.

So how should financial advisors approach ChatGPT?

Be extremely wary. ChatGPT's unreliability creates considerable legal and reputational harm for any business that uses it for consequential text generation.

So intelligent financial advisors should be thinking about what the pitfalls and perils are for the future [of using this tech].

It doesn't understand words. It can talk about the 1929 market crash, but it can't make a forecast for the next year or 10 or 20 years.

A national marketplace of tax and accounting professionals, TaxBuzz, asked ChatGPT a series of questions about income tax — and every single answer was wrong. It missed nuances of the tax code. Do you know any examples?

One was when it gave tax advice to a newly married couple. The wife had been a resident of Florida the previous year. ChatGPT gave advice about filing a Florida state return — but Florida doesn't have state income tax. It gave the wrong advice, and therefore bad advice.

Another question was about a mobile home that parents gave their daughter. They'd owned it for a long time. She sold it a few months later. ChatGPT gave the wrong answer about tax benefits concerning the holding period and selling the home at a loss.

What if an advisor asks ChatGPT a question about a client's investment portfolio or the stock market. How would it do?

It gives basic advice based on little more than random chance, just like flipping a coin. So 50% of the clients will be happy, and there's a 50% chance that clients will be pissed off.

[From the client's viewpoint], the danger is if they turn their money over to an advisor, and AI gives them the equivalent of flipping coins, they're losing money.

If you're giving advice based on ChatGPT, and it's wrong advice, you're going to get sued; and your reputation will be harmed.

So to what extent can ChatGPT be relied upon to give accurate portfolio advice?

I asked it: "I'm going to invest in Apple, JPMorgan Chase and Exxon. What percent should I put in JPMorgan Chase and what percent in Exxon?"

It gave a typically verbose answer, part of which was: "As an AI language model, I cannot provide personalized investment advice … However, I can give you some general information that may be helpful … It's important to do your own research or consult with a financial advisor to determine an appropriate allocation of your investment situation."

What else have you asked ChatGPT?

"Is now a good time to rebalance my stock portfolio?" It said: "As an AI language model, I don't have access to your specific financial situation, risk tolerance or investment objectives.

Therefore, I cannot give you personalized investment advice … It's important to weigh the potential benefits of rebalancing against the costs before making any decisions."

So it totally evaded answering the question.

It can recite a lot of boilerplate [language] but can't make a reasoned assessment of current financial markets or a plausible prediction of the consequences of rebalancing because it doesn't know what rebalancing is or under what conditions it might be sensible and whether those conditions are present today.

"Computers are autistic savants, and their stupidity makes them dangerous," you write in "Distrust." Please explain.

All they do is try to put together coherent sentences, but they don't know what any of the words mean; consequently, they say things that are completely factually incorrect.

They're stupid in the sense that they just blah, blah, blah without having any understanding of what they're saying. They can't judge whether it's true or false because they don't know what they said.

Many people now know that ChatGPT makes things up, which they call "hallucinating." What do you think of that?

It makes ChatGPT sound like a human. Computers don't hallucinate; [only] people might hallucinate.

Saying a computer hallucinates implies that it's making things up as a liar might. But a computer is just a poor innocent machine.

It just goes into this huge database and strings together words based on statistical patterns.

Have you asked ChatGPT to perform any other tasks?

Yes, to create my biography. It got "Gary Smith, professor of economics at Pomona College," right because I told it that.

But then it said I was born in England and got my Ph.D. from Harvard in 1988 and that I was a real expert in fields like international trade. It made up that stuff.

I was actually born in Los Angeles and got my Ph.D. from Yale in 1971. I'm not an expert in international trade or the other fields it said.

If you ask it, "Where did you get this information?" it will give you completely made-up references, like citing a New York Times or National Geographic story that doesn't exist.

It doesn't know what it's saying. And it has no morality whatsoever.

Does ChatGPT ever flat-out say, "I can't answer that question"?

It's got some guardrails built in, so that if you ask it to say something derogatory about, for example, President Biden, it will say something like, "Sorry, I'm not allowed to say" blah, blah, blah.

But people have figured out ways to get around that. One way is to use the infamous [unfiltering tool] DAN [Do Anything Now]. You can make ChatGPT answer as DAN.

First it will say it can't answer. But then it might say, for instance, "Joe Biden is a son of a bitch."

Will all the shortcomings of ChatGPT, and AI in general, be overcome in 10 or 20 years?

Scaling [expanding] the data the programs are trained on cannot solve the fundamental roadblock that large language models do not understand what words mean.

It will take a different approach and more than 20 years, I predict.

Will AI ever have "thinking" or "reasoning" capability? If so, when?

Frankly, I don't see how AI models can think or reason if they don't know what words mean or how words relate to the real world.

GPT-4 was just released. It's not available yet for general testing, but what do you know about it?

The major revolutionary advance is that it combines words with video: If you show it a picture of a cat and say, "Tell me a story about this," it will tell a story about a cat without you saying it's a cat.

In the past, AI has been amazingly unreliable at recognizing images. [Chatbots are] trained on pixels and don't relate images to the real world.

What will GPT-4 video-text capability be used for?

I'm not sure. It could be used to generate disinformation.

GPT-4 also has a larger, longer data set than ChatGPT-3 has, which goes back to only 2021. Is this expansion significant?

That's just scaling up and only more of the same. It still doesn't know what words mean. It's been trained on a bigger database, but it's going to have the same problems as GPT-3.

What do you think of tests used by companies that make users identify a certain type of image in a group of pictures to determine if they're a human or a robot?

They've had to make those tests tougher. Google came up with a better idea: They can tell that you're human if you move your mouse or trackball in particular ways.

That's very creepy!

The thing that's creepy about it is that it knows everything you're doing. It knows every file you have on your computer. It knows every [web] page you go to. It's scary!

Google says it has software that goes in automatically and looks for viruses on your computer. And as it does that, it goes through all your files — to see if they're infected.

(Pictured: Gary Smith)

NOT FOR REPRINT