{"id":13573,"date":"2024-06-20T09:28:05","date_gmt":"2024-06-20T01:28:05","guid":{"rendered":"https:\/\/www.1ai.net\/?p=13573"},"modified":"2024-06-20T09:28:11","modified_gmt":"2024-06-20T01:28:11","slug":"b%e7%ab%99%e5%bc%80%e6%ba%90%e8%bd%bb%e9%87%8f%e7%ba%a7-index-1-9b-%e7%b3%bb%e5%88%97%e6%a8%a1%e5%9e%8b%ef%bc%9a2-8t-%e8%ae%ad%e7%bb%83%e6%95%b0%e6%8d%ae%ef%bc%8c%e6%94%af%e6%8c%81%e8%a7%92%e8%89%b2","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/13573.html","title":{"rendered":"B station open source lightweight Index-1.9B series model: 2.8T training data, support role-playing"},"content":{"rendered":"<p data-vmark=\"0fbe\"><a href=\"https:\/\/www.1ai.net\/en\/tag\/b%e7%ab%99\" title=\"[SEE ARTICLE WITH [B] LABEL]\" target=\"_blank\" >Station B<\/a>Yesterday<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e5%bc%80%e6%ba%90\" title=\"[View articles tagged with [open source]]\" target=\"_blank\" >Open Source<\/a>The lightweight Index-1.9B series<a href=\"https:\/\/www.1ai.net\/en\/tag\/%e6%a8%a1%e5%9e%8b\" title=\"_Other Organiser\" target=\"_blank\" >Model<\/a>, including multiple versions such as base model, control group model, dialogue model, role-playing model, etc.<\/p>\n<p data-vmark=\"12ce\">Official Introduction:<\/p>\n<ul class=\"list-paddingleft-2\">\n<li>\n<p data-vmark=\"08d1\"><strong>Index - 1.9B base:\u00a0<\/strong>The base model has 1.9 billion non-word embedding parameters and is pre-trained on 2.8T of Chinese and English-based corpora. It is ahead of models of the same level on multiple evaluation benchmarks.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"cad1\"><strong>Index-1.9B pure :<\/strong>\u00a0The control group of the base model has the same parameters and training strategy as the base, but the difference is that all instruction-related data in the corpus of this version is strictly filtered to verify the impact of instructions on the benchmark.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"58ea\"><strong>Index-1.9B chat:<\/strong>Based on the index-1.9B base, the dialogue model is aligned through SFT and DPO. As more Internet community corpus is introduced in pre-training, the chat is obviously more interesting.<\/p>\n<\/li>\n<li>\n<p data-vmark=\"9ed4\"><strong>Index-1.9B character:<\/strong>RAG was introduced based on SFT and DPO to achieve fewshots role-playing customization.<\/p>\n<\/li>\n<\/ul>\n<p data-vmark=\"80bf\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-13574\" title=\"5964f462-ab56-41eb-bd96-28649efab4de\" src=\"https:\/\/www.1ai.net\/wp-content\/uploads\/2024\/06\/5964f462-ab56-41eb-bd96-28649efab4de.png\" alt=\"5964f462-ab56-41eb-bd96-28649efab4de\" width=\"1440\" height=\"563\" \/><\/p>\n<p data-vmark=\"e582\">According to reports, the model used 2.8T of data in the pre-training stage, with a Chinese-English ratio of 4:5 and 6% of code. Currently, the role-playing model has a built-in character &quot;San San&quot;, and users can also create their own characters on demand.<\/p>\n<p data-vmark=\"132e\"><strong>Project address:<\/strong><a title=\"https:\/\/github.com\/bilibili\/Index-1.9B\/blob\/main\/README.md\" href=\"https:\/\/github.com\/bilibili\/Index-1.9B\/blob\/main\/README.md\" target=\"_blank\" rel=\"noopener\"><span class=\"link-text-start-with-http\">https:\/\/github.com\/bilibili\/Index-1.9B\/blob\/main\/README.md<\/span><\/a><\/p>","protected":false},"excerpt":{"rendered":"<p>Yesterday, B.com open-sourced the lightweight Index-1.9B series of models, including the base model, control group, dialog model, role-playing model and other versions. Official introduction: Index-1.9B base: base model, with 1.9 billion non-word embedded parameters, pre-trained on 2.8T Chinese and English language-based corpus, and leading with the same level of models on multiple evaluation benchmarks. Index-1.9B pure: the control group of the base model, with the same parameters and training strategy as base, but with the difference of strictly filtering all command-related data in this version of the corpus, in order to verify the effect of commands on the benchmark. Index-1.9B chat : A control group of the base model with the same parameters and training strategy as base.<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[146],"tags":[938,219,1489],"collection":[],"class_list":["post-13573","post","type-post","status-publish","format-standard","hentry","category-news","tag-b","tag-219","tag-1489"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/13573","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=13573"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/13573\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=13573"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=13573"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=13573"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=13573"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}