{"id":11429,"date":"2024-05-27T12:36:00","date_gmt":"2024-05-27T04:36:00","guid":{"rendered":"https:\/\/www.1ai.net\/?p=11429"},"modified":"2024-05-27T12:36:00","modified_gmt":"2024-05-27T04:36:00","slug":"%e5%bc%80%e6%ba%90%e5%a4%a7%e6%a8%a1%e5%9e%8bllama3%e6%9c%ac%e5%9c%b0%e9%83%a8%e7%bd%b2%e6%95%99%e7%a8%8b%ef%bc%8c%e6%89%93%e9%80%a0%e4%b8%aa%e4%ba%ba%e7%9f%a5%e8%af%86%e5%ba%93","status":"publish","type":"post","link":"https:\/\/www.1ai.net\/en\/11429.html","title":{"rendered":"Open source large model Llama3 local deployment tutorial, build a personal knowledge base"},"content":{"rendered":"


\nOllama<\/a><\/strong><\/p>\n

\n

Official Website<\/strong>: https:\/\/ollama.com\/<\/p>\n

Github<\/strong>\uff1ahttps:\/\/github.com\/ollama\/ollama<\/p>\n

\n<\/blockquote>\n

Ollama is aOpen Source<\/a>Platform for managing and running various large language models (LLMs), such as Llama3<\/a>, gemma and qwen.<\/p>\n

It provides a command line interface (CLI) for installation, model management, and interaction.<\/p>\n

You can use Ollama to download, load, and run different LLM models based on your needs.<\/p>\n

# Windows Installation<\/strong><\/p>\n

\n

Requirements: Windows 10 or above operating system<\/p>\n

Installation package download address: https:\/\/ollama.com\/download\/windows<\/p>\n

\n<\/blockquote>\n

Download the corresponding system installation package<\/p>\n

\"\"<\/div>\n

After the download is complete, double-click the downloaded installer<\/p>\n

\"\"<\/div>\n

Click Install to install<\/p>\n

\"\"<\/div>\n

After the installation is complete, open a terminal and enter the following command in the terminal to run a large language model for testing. Here we take Qianwen, which performs relatively well in Chinese, as an example:<\/p>\n

ollama run qwen<\/code><\/pre>\n
The model will be downloaded first when it is run for the first time. After the download is complete, you can ask questions<\/p>\n
\"\"<\/div>\n
Once the model is downloaded, we can use it by entering questions in the terminal:<\/p>\n
\"\"<\/div>\n
So far, we ordinary people can also use AILarge Model<\/a>Question and answer session!!!<\/strong><\/p>\n
#Docker installation<\/strong><\/p>\n
\u2460Docker deployment<\/strong><\/p>\n
# Only CPU uses this docker run -itd --name ollama -v ollama:\/root\/.ollama -p 11434:11434 ollama\/ollama # There is a GPU and use this docker run -itd --name ollama --restart always --gpus=all -v \/home\/suxy\/ollama:\/root\/.ollama -p 11434:11434 ollama\/ollama<\/code><\/pre>\n
\u2461 Pull the model and run it<\/strong><\/p>\n
# Qianwen 4b docker exec -it ollama ollama run qwen #llama3 docker exec -it ollama ollama run llama3 #gemma docker exec -it ollama ollama run gemma<\/code><\/pre>\n
When you pull a model for the first time, you will first download the model. After the model is downloaded, you can start a conversation. The example is as follows:<\/p>\n
\"\"<\/div>\n
\u2462Other models<\/strong><\/p>\n
You can view:https:\/\/ollama.ai\/library<\/u><\/p>\n
\"\"<\/div>\n
You can also download models with different parameter levels according to your needs.<\/p>\n
\"\"<\/div>\n
Open WebUI<\/strong><\/p>\n
The above can only be used in the terminal, and the operation interface is not as beautiful as ChatGPT. If you can use the web page to interact, the user experience will be better, and you can also keep the previous chat records and look through them for inquiries.<\/p>\n
In this case, deploying the Open WebUI project can implement an interactive interface similar to chatgpt.<\/p>\n
This tutorial takes the Open WebUI project as an example. It was formerly known as Formerly Ollama WebUI and is specifically adapted for Ollama's WebUI.<\/p>\n
\n
Official documentation: https:\/\/docs.openwebui.com\/getting-started\/<\/p>\n
Github: https:\/\/github.com\/open-webui\/open-webui<\/p>\n<\/blockquote>\n
#docker quick deployment<\/strong><\/p>\n
Execute the following command<\/p>\n
The # example uses ollama-webui docker run -itd --name ollama-webui --restart always -p 3500:8080 --add-host=host.docker.internal:host-gateway ghcr.io\/ollama-webui\/ ollama-webui:latest # or docker run -itd --name open-webui --restart always -p 3500:8080 --add-host=host.docker.internal:host-gateway -v open-webui:\/app\/ backend\/data ghcr.io\/open-webui\/open-webui:latest<\/code><\/pre>\n
After successful deployment, the browser opens the connection:http:\/\/127.0.0.1:3500<\/u><\/p>\n
First time login, create an account first<\/p>\n
\"\"<\/div>\n
Click sign up to register, and remember your account, email address, and password. You will need to use your email address and password to log in next time:<\/p>\n
\"\"<\/div>\n
Then click create account to create an account, and then you can use the Open WebUI similar to the chatgpt interface in the browser!<\/p>\n
\"\"<\/div>\n
Add and click on the settings in the upper right corner to set the ollama service installed above, for example:http:\/\/localhost:11434\/api<\/u><\/p>\n
\"\"<\/div>\n
At the same time, you can download the model you want to install on the page. After clicking the download button, you can see the download progress<\/p>\n
\"\"<\/div>\n
Click select a model to select which installed model to use<\/p>\n
\"\"<\/div>\n
Next, you can chat happily with AI!<\/p>\n
\"\"<\/div>\n
AnythingLLM<\/strong><\/p>\n
\n
Official documentation: https:\/\/docs.useanything.com\/<\/p>\n
Github: https:\/\/github.com\/Mintplex-Labs\/anything-llm<\/p>\n<\/blockquote>\n
AnythingLLM is an open source, efficient, and customizable private knowledge base solution built on the RAG (Retrieval-Augmented Generation) solution. You can use commercial off-the-shelf LLM or popular open source LLM and vectorDB solutions to build private ChatGPT<\/p>\n
# download and install AnythingLLM<\/strong><\/p>\n
\n
Official installation tutorial: https:\/\/docs.useanything.com\/anythingllm-desktop\/windows-instructions<\/p>\n
Installation package download address: https:\/\/useanything.com\/download<\/p>\n<\/blockquote>\n
This example takes Windows as an example:<\/p>\n
\"\"<\/div>\n
After the installation package is downloaded, double-click the installation package to install it.<\/p>\n
Because I have already configured it, it is difficult to take a screenshot of the initial configuration interface, but you can find it in the settings<\/p>\n
# configuration LLM<\/strong><\/p>\n
AnythingLLM supports LLMs such as OpenAI, LocalAi, Ollama, etc.<\/p>\n
\"\"<\/div>\n
Select Ollama as the background service here, and fill in the URL herehttp:\/\/127.0.0.1:11434<\/u>, which is the service port started by Ollama above. After filling in the LLM model, select llama3<\/p>\n
\"\"<\/div>\n
# configuration Embedding Model<\/strong><\/p>\n
The Embedding Models supported by AnythingLLM include: AnythingLLM Native Embedder (default), OpenAi, LocalAi, Ollama, etc.<\/p>\n
Here we also select Ollama as the background service and fill in the URL herehttp:\/\/127.0.0.1:11434<\/u>After filling in the Embedding Model, select nomic-embed-text:latest:<\/p>\n
\"\"<\/div>\n
# Configure Vector Datebase<\/strong><\/p>\n
The vector databases supported by AnythingLLM are: LanceDB (default), Astra DB, Pinecone, Chroma, Weaviate, QDrant, Milvus, Zilliz, etc.<\/p>\n
\"\"<\/div>\n
After completing the above three key configurations, you can start using AnythingLLM!!!<\/strong><\/p>\n
Click New Workspace to create a new document library and fill in the name<\/p>\n
\"\"<\/div>\n
Click the Upload Files button to start adding documents:<\/p>\n
\"\"<\/div>\n
Now you can ask questions based on the content of the document.<\/p>\n
\"\"<\/div>\n
 <\/p>","protected":false},"excerpt":{"rendered":"
Ollama Official Website: https:\/\/ollama.com\/ Github: https:\/\/github.com\/ollama\/ollama Ollama is an open source platform for managing and running a variety of Large Language Models (LLMs), such as Llama3, gemma, and qwen. It provides a command line interface (CLI) for installation, model management, and interaction. You can use Ollama to download, load and run different LLM models according to your needs. #Windows Installation Requirements: Win10 or higher Operating System Installation package download at: https:\/\/ollama.com\/download<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[149,144],"tags":[2403,2405,216,219],"collection":[],"class_list":["post-11429","post","type-post","status-publish","format-standard","hentry","category-jiaocheng","category-baike","tag-llama3","tag-ollama","tag-216","tag-219"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/11429","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/comments?post=11429"}],"version-history":[{"count":0,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/posts\/11429\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/media?parent=11429"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/categories?post=11429"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/tags?post=11429"},{"taxonomy":"collection","embeddable":true,"href":"https:\/\/www.1ai.net\/en\/wp-json\/wp\/v2\/collection?post=11429"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}