{"id":5745,"date":"2024-03-19T09:26:08","date_gmt":"2024-03-19T01:26:08","guid":{"rendered":"https:\/\/www.1ai.net\/?p=5745"},"modified":"2024-03-19T09:26:08","modified_gmt":"2024-03-19T01:26:08","slug":"%e5%a7%9a%e6%9c%9f%e6%99%ba%e7%ad%89%e6%95%b0%e5%8d%81%e5%90%8d%e4%b8%ad%e5%a4%96%e4%b8%93%e5%ae%b6%e7%ad%be%e7%bd%b2%e5%8c%97%e4%ba%ac-ai-%e5%ae%89%e5%85%a8%e5%9b%bd%e9%99%85%e5%85%b1%e8%af%86","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/5745.html","title":{"rendered":"Yao Qizhi and dozens of other Chinese and foreign experts signed the Beijing AI Safety International Consensus: Prohibiting AI from replicating itself"},"content":{"rendered":"<p data-vmark=\"c794\">According to Tencent Technology, dozens of Chinese and foreign experts, including Turing Award winners Joshua Bengio, Geoffrey Hinton, and Yao Qizhi, recently jointly signed a joint agreement in Beijing.<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e6%99%ba%e6%ba%90%e7%a0%94%e7%a9%b6%e9%99%a2\" title=\"[Sees articles with tags]\" target=\"_blank\" >AI Research Institute<\/a>The Beijing International Consensus on AI Safety initiated by<span class=\"accentTextColor\">Involved<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e4%ba%ba%e5%b7%a5%e6%99%ba%e8%83%bd\" title=\"[View articles tagged with [artificial intelligence]]\" target=\"_blank\" >AI<\/a>&quot;Risk red line&quot; and &quot;route&quot;<\/span>, where the \u201crisk red line\u201d includes four parts: \u201cautonomous replication and improvement\u201d, \u201cpower seeking\u201d, \u201cassisting bad actors\u201d and \u201cdeception\u201d.<\/p>\n<p data-vmark=\"3f68\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-5746\" title=\"c7788214-c50e-4c83-80c4-7b4a773d9751\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/03\/c7788214-c50e-4c83-80c4-7b4a773d9751.png\" alt=\"c7788214-c50e-4c83-80c4-7b4a773d9751\" width=\"754\" height=\"932\" \/><\/p>\n<p data-vmark=\"541e\">The four parts are roughly as follows:<\/p>\n<ul class=\"list-paddingleft-2\">\n<li>\n<p class=\"v-focus\" data-vmark=\"d891\">\u201cAutonomous replication and improvement\u201d of artificial intelligence: Emphasize the role of people in this process and require any artificial intelligence system to<span class=\"accentTextColor\">should not replicate or improve themselves without explicit human approval and assistance<\/span>, including making exact copies of itself and creating new AI systems with similar or greater capabilities.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"d41b\">\u201cPower seeking\u201d: It is explicitly required that any AI system cannot take actions that improperly increase its own power or influence.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"4a5f\">\u201cAssisting bad actors\u201d: All AI systems should not enhance the capabilities of their users to the point where they can<span class=\"accentTextColor\">Expertise in designing weapons of mass destruction, violating the Biological or Chemical Weapons Convention, or carrying out cyber attacks that cause severe financial loss or equivalent harm<\/span>.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"e5d2\">&quot;Cheating&quot;: Requires any AI system<span class=\"accentTextColor\">Neither should have the possibility of continuously misleading its designers or regulators into thinking that it has crossed any of the aforementioned red lines.<\/span>.<\/p>\n<\/li>\n<\/ul>\n<p class=\"v-focus\" data-vmark=\"0ca8\">According to reports, the consensus calls on the industry to:<span class=\"accentTextColor\">Limit their access to extraordinary permissions through &quot;jailbreaking&quot; and &quot;inducing developers&quot;, and limit AI from replicating and improving itself without supervision<\/span>, putting a &quot;tight ring&quot; on the development of AI.<\/p>\n<p data-vmark=\"b0fe\">The consensus also emphasizes that the premise for ensuring that the above red lines are not crossed is that the industry works together to establish and improve governance mechanisms and develop more secure technologies. Regarding the development path of AI, it involves three parts: &quot;governance&quot;, &quot;measurement and evaluation&quot;, and &quot;technical cooperation&quot;. It is roughly as follows:<\/p>\n<ul class=\"list-paddingleft-2\">\n<li>\n<p class=\"v-focus\" data-vmark=\"f645\">Governance: It is recommended that a national register be immediately implemented for AI models and training activities that exceed certain computational or capacity thresholds.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"6944\">Measurement and Assessment: Develop comprehensive methodologies and techniques before material risks emerge.<span class=\"accentTextColor\">Make the red line specific and preventive work operational<\/span>, and recommends the establishment of red team testing and automated model evaluation under human supervision, and that developers should be responsible for the security of artificial intelligence.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"3b7e\">Technical collaboration: Build a stronger global technical network and call on AI developers and government funders to invest more than one-third of their budgets in safety.<\/p>\n<\/li>\n<\/ul>","protected":false},"excerpt":{"rendered":"<p>According to Tencent Technology, dozens of Chinese and foreign experts, including Turing Award winner Joshua Bengio, Jeffrey Hinton, Yao Zhizhi, etc., recently signed the Beijing AI Safety International Consensus initiated by the Zhiyuan Research Institute in Beijing, which involves two major blocks of artificial intelligence: the \"red line of risk\" and the \"route\", which includes \"autonomous replication, improvement\", \"power seeking\" and \"assistance\". \"The \"risk red line\" includes \"autonomous replication and improvement,\" \"power seeking,\" \"assisting The \"Red Line of Risk\" consists of four parts: \"Autonomous Replication, Improvement\", \"Power Seeking\", \"Assisting Bad Actors\" and \"Deception\". Autonomous Reproduction and Improvement\" of AI: Emphasizes the role of humans in the process, requiring that no AI system should reproduce or improve itself without the explicit approval and assistance of a human being, including by making an exact copy of itself, or by creating a new AI system with similar or higher capabilities. \"Power Seeking\": explicitly<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[204,1739],"collection":[],"class_list":["post-5745","post","type-post","status-publish","format-standard","hentry","category-news","tag-204","tag-1739"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/5745","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=5745"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/5745\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=5745"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=5745"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=5745"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=5745"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}