{"id":21269,"date":"2024-10-12T09:32:23","date_gmt":"2024-10-12T01:32:23","guid":{"rendered":"https:\/\/www.1ai.net\/?p=21269"},"modified":"2024-10-12T09:32:23","modified_gmt":"2024-10-12T01:32:23","slug":"%e8%8b%b9%e6%9e%9c%e7%a0%94%e7%a9%b6%e4%ba%ba%e5%91%98%e8%b4%a8%e7%96%91-ai-%e7%9a%84%e6%8e%a8%e7%90%86%e8%83%bd%e5%8a%9b%ef%bc%9a%e7%ae%80%e5%8d%95%e6%95%b0%e5%ad%a6%e9%97%ae%e9%a2%98%e7%a8%8d","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/21269.html","title":{"rendered":"Apple researchers question AI's reasoning ability: simple math questions can be answered incorrectly with minor changes"},"content":{"rendered":"

In recent years, artificial intelligence (AI<\/a>) have made significant progress in various areas, with large-scale language modeling (LLM<\/a>) is capable of generating human-level text and even exceeding human performance on some tasks. However, researchers of LLM'sreasoning ability<\/a>questioned, they found that these models, when solving simple mathematical problems, wereThe fact that mistakes are made with just a few minor changes suggests that they may not be capable of true logical reasoning.<\/strong><\/p>\n

\"a6b82b61j00sl7ywg001ed000q400hfm\"<\/p>\n

Thursday.apple<\/a>A group of researchers at the company published a paper titled \"Understanding the Limitations of Mathematical Reasoning in Large Language Models,\" revealing that LLMs are susceptible to interference when solving mathematical problems.IT House notes that theThe researchers tested the reasoning power of the LLM by making small changes to the math problem, such as adding irrelevant information<\/strong>. It turns out that the performance of these models drops dramatically in the face of such changes.<\/p>\n

For example, when researchers give a simple mathematical question: \"Oliver picks 44 ecstasy nuts on Friday and 58 ecstasy on Saturday. On Sunday, he picked twice as strange as Friday. How many strange results did Oliver pick?\" When the LLM was able to calculate the answer correctly. However, when the researcher added an unrelated detail, \u201cSunday, he picked twice as many ecstasy as Friday, five of which were smaller than average\u201d, the LLM answered wrongly. For example, GPT-o1-mini responded: \"... Sunday, 5 of which are smaller than the average. We need to subtract them from the total number of Sundays: 88 \u2013 5 = 83<\/p>\n

The above is just a simple example ofThe researchers modified hundreds of questions, almost all of which resulted in a significant decrease in the model's response success rate.<\/strong><\/p>\n

According to the researchers, this phenomenon suggests that LLMs don't really understand math problems, but instead make predictions based solely on patterns in the training data. But when real \"reasoning\" is required, such as whether to count small kiwis, they produce strange and implausible results.<\/p>\n

This finding has important implications for the development of AI. Although LLM performs well in many areas, there are still limitations in its reasoning ability. In the future, researchers need to further explore how to improve LLM's reasoning ability so that it can better understand and solve complex problems.<\/p>","protected":false},"excerpt":{"rendered":"

Artificial Intelligence (AI) has made significant progress in various fields in recent years, with Large Language Models (LLMs) capable of generating human-level text and even outperforming humans on certain tasks. However, researchers have questioned the reasoning ability of LLMs, finding that these models make mistakes when solving simple math problems with minor changes, suggesting that they may not be capable of true logical reasoning. On Thursday, a group of researchers at Apple published a paper titled \"Understanding the Limitations of Mathematical Reasoning in Large Language Models,\" revealing that LLMs are prone to interfering when solving mathematical problems.IT House notes that the researchers tested the reasoning ability of the LLMs by making small changes to the mathematical problems, such as adding irrelevant information<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[411,473,4582,345],"collection":[],"class_list":["post-21269","post","type-post","status-publish","format-standard","hentry","category-news","tag-ai","tag-llm","tag-4582","tag-345"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/21269","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=21269"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/21269\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=21269"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=21269"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=21269"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=21269"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}