A recent study released by Apple has sparked discussions about the effectiveness of generative artificial intelligence (AI) in financial advice. Surveys indicate that an increasing number of American consumers are using generative AI tools, such as ChatGPT, to obtain financial advice, a trend particularly pronounced among younger individuals. According to a Motley Fool survey, 54% of Americans have sought financial product recommendations through ChatGPT, with higher usage among younger generations.
Image source: Image generated by AI, provided by Midjourney, an image licensing service
The survey results show that while half of the consumers are willing to use ChatGPT for recommendations, interest in specific financial products is relatively low. For example, only 25% of respondents want ChatGPT to recommend credit cards. Additionally, respondents generally have a "somewhat satisfied" view of ChatGPT's recommendations, with an average satisfaction rating of 3.7 out of 5, indicating a degree of acceptance.
However, Apple's research points out that current large language models (LLMs) have significant flaws in logical reasoning, especially in mathematical reasoning. Researchers have found that these models perform poorly when faced with complex mathematical problems, often failing to correctly understand or solve simple mathematical calculations. As the complexity of the problems increases, the model's performance further declines, indicating deeper issues in its reasoning process.
An article from TechCrunch lists several examples of generative AI making mistakes in mathematical calculations, illustrating their shortcomings in handling basic mathematical problems. The report mentions that the "chunking" technique used by AI models when dealing with numbers often disrupts the relationships between numbers, leading to calculation errors.
Furthermore, machine learning faces challenges in providing financial advice. Although some confuse machine learning with statistical analyses like regression analysis, machine learning actually requires a decision-making process, an error evaluation function, and a model optimization process. This makes generative AI in financial advice potentially ineffective in meeting user needs.
Apple's research suggests that banks and credit unions should not rely on AI for financial advice at this stage. Although improvements may be possible in the future, in the foreseeable future, generative AI will still struggle to handle complex financial consulting tasks.
Key Points:
🧠 54% of Americans have obtained financial advice through ChatGPT, with higher usage among younger generations.
📉 Apple's research indicates that generative AI has significant flaws in mathematical reasoning, especially in handling complex problems.
💡 Currently, banks and credit unions should not rely on AI for financial advice; improvements may take 5 to 10 years.