April 19, 2011 - Technology media outlet The Information released a blog post yesterday (April 18) reporting that Meta Platforms in AI Competitionin the face of significant cost pressures, 2024 year had sought financial support from competitors such as Microsoft and Amazon for training its flagship large language model, Llama.
1AICiting the outlet's report, Meta internally initiated a proposal for a "Llama Consortium" due to concerns about resource needs, and Meta even offered to involve potential investors in the development of future Llama features in order to attract partnerships.
The initial response has been lukewarm, and it remains unclear whether a formal funding deal will be reached, the sources said. The attempt highlights the huge cost of building leading AI systems, and even Meta, with its deep pockets, is feeling the pinch, demonstrating the high stakes in the generative AI space.
Meta's recently announced Llama 4 family of models further emphasizes its technical ambitions and cost challenges. The Llama 4 Scout (109B total parameters, 17B active parameters) is designed for single GPU use, with a large context window of 10 million tokens and the ability to process approximately 7.5 million words at once. Llama 4 Maverick (400B total parameters, 17B active parameters, 128 experts) is aimed at larger scale tasks.
Both models use the Mixture-of-Experts (MoE) architecture, which improves efficiency by activating specific sub-networks. In addition, Llama 4 Behemoth, with parameters up to 2 trillion, was developed in-house at Meta to train smaller models, utilizing up to 32,000 GPUs.
Meta's funding quest is closely tied to the centrality of the Llama model to its business.Llama 4 was released and quickly integrated into Meta AI functionality on WhatsApp, Instagram, and Facebook, and is available for licensing through cloud platforms such as Amazon SageMaker JumpStart and Microsoft Azure AI Foundry. commercial licenses.
Meanwhile, Meta is taking a strong stance against the competition by blocking Apple's Apple Intelligence feature from its iOS apps, forcing users to turn to Llama-based AI tools.