Wall-facing intelligencereleaseOpen Source Big Model Eurux-8x22B, including Eurux-8x22B-NCA and Eurux-8x22B-KTO, focuses on inference capability.
Eurux-8x22B in official tests Tested on LeetCode (180 LeetCode programming questions) and TheoremQA (IT Home Note: U.S. college-level STEM questions).first (of multiple parts)Outperforms Llama3-70B and outperforms closed-source GPT-3.5-Turbo on LeetCode tests.
According to the introduction,Eurux-8x22B model activation parameter 39B, supports 64k contexts, which is aligned from the Mixtral-8x22B model, trained on the UltraInteract aligned dataset.
UltraInteract is a large-scale, high-quality alignment dataset designed specifically to improve large model inference, containing 86K instructions and 220K preference pairs from 12 open-source datasets covering mathematical, coding, and logical inference problems, for a total of half a million or so numbers of