[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71274":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":16,"stars30d":16,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":17,"rankGlobal":10,"rankLanguage":10,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":10,"pushedAt":10,"updatedAt":39,"readmeContent":40,"aiSummary":41,"trendingCount":16,"starSnapshotCount":16,"syncStatus":42,"lastSyncTime":43,"discoverSource":44},71274,"llama-gpt","getumbrel\u002Fllama-gpt","getumbrel","A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!","https:\u002F\u002Fapps.umbrel.com\u002Fapp\u002Fllama-gpt",null,"TypeScript",10950,708,76,84,0,43.55,"MIT License",false,"master",true,[23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38],"ai","chatgpt","code-llama","codellama","gpt","gpt-4","gpt4all","llama","llama-2","llama-cpp","llama2","llamacpp","llm","localai","openai","self-hosted","2026-06-12 02:02:50","\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fapps.umbrel.com\u002Fapp\u002Fllama-gpt\">\n    \u003Cimg width=\"150\" height=\"150\" src=\"https:\u002F\u002Fi.imgur.com\u002FLI59cui.png\" alt=\"LlamaGPT\" width=\"200\" \u002F>\n  \u003C\u002Fa>\n\u003C\u002Fp>\n\u003Cp align=\"center\">\n  \u003Ch1 align=\"center\">LlamaGPT\u003C\u002Fh1>\n  \u003Cp align=\"center\">\n    A self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.\n    \u003Cbr\u002F>\n    \u003Cstrong>New: Support for Code Llama models and Nvidia GPUs.\u003C\u002Fstrong>\n    \u003Cbr \u002F>\n    \u003Cbr \u002F>\n    \u003Ca href=\"https:\u002F\u002Fumbrel.com\">\u003Cstrong>umbrel.com (we're hiring) »\u003C\u002Fstrong>\u003C\u002Fa>\n    \u003Cbr \u002F>\n    \u003Cbr \u002F>\n    \u003Ca href=\"https:\u002F\u002Ftwitter.com\u002Fumbrel\">\n      \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Ftwitter\u002Ffollow\u002Fumbrel?style=social\" \u002F>\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Ft.me\u002Fgetumbrel\">\n      \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcommunity-chat-%235351FB\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Freddit.com\u002Fr\u002Fgetumbrel\">\n      \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Freddit\u002Fsubreddit-subscribers\u002Fgetumbrel?style=social\">\n    \u003C\u002Fa>\n    \u003Ca href=\"https:\u002F\u002Fcommunity.umbrel.com\">\n      \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcommunity-forum-%235351FB\">\n    \u003C\u002Fa>\n  \u003C\u002Fp>\n\u003C\u002Fp>\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fumbrel.com\u002F#start\">\n    \u003Cimg src=\"https:\u002F\u002Fi.imgur.com\u002Fsj5vqEG.jpg\" width=\"100%\" \u002F>\n  \u003C\u002Fa>\n\u003C\u002Fp>\n\n## Contents\n\n1. [Demo](#demo)\n2. [Supported Models](#supported-models)\n3. [How to install](#how-to-install)\n   - [On umbrelOS home server](#install-llamagpt-on-your-umbrelos-home-server)\n   - [On M1\u002FM2 Mac](#install-llamagpt-on-m1m2-mac)\n   - [Anywhere else with Docker](#install-llamagpt-anywhere-else-with-docker)\n   - [Kubernetes](#install-llamagpt-with-kubernetes)\n4. [OpenAI-compatible API](#openai-compatible-api)\n5. [Benchmarks](#benchmarks)\n6. [Roadmap and contributing](#roadmap-and-contributing)\n7. [Acknowledgements](#acknowledgements)\n\n## Demo\n\nhttps:\u002F\u002Fgithub.com\u002Fgetumbrel\u002Fllama-gpt\u002Fassets\u002F10330103\u002F5d1a76b8-ed03-4a51-90bd-12ebfaf1e6cd\n\n## Supported models\n\nCurrently, LlamaGPT supports the following models. Support for running custom models is on the roadmap.\n\n| Model name                               | Model size | Model download size | Memory required |\n| ---------------------------------------- | ---------- | ------------------- | --------------- |\n| Nous Hermes Llama 2 7B Chat (GGML q4_0)  | 7B         | 3.79GB              | 6.29GB          |\n| Nous Hermes Llama 2 13B Chat (GGML q4_0) | 13B        | 7.32GB              | 9.82GB          |\n| Nous Hermes Llama 2 70B Chat (GGML q4_0) | 70B        | 38.87GB             | 41.37GB         |\n| Code Llama 7B Chat (GGUF Q4_K_M)         | 7B         | 4.24GB              | 6.74GB          |\n| Code Llama 13B Chat (GGUF Q4_K_M)        | 13B        | 8.06GB              | 10.56GB         |\n| Phind Code Llama 34B Chat (GGUF Q4_K_M)  | 34B        | 20.22GB             | 22.72GB         |\n\n## How to install\n\n### Install LlamaGPT on your umbrelOS home server\n\nRunning LlamaGPT on an [umbrelOS](https:\u002F\u002Fumbrel.com) home server is one click. Simply install it from the [Umbrel App Store](https:\u002F\u002Fapps.umbrel.com\u002Fapp\u002Fllama-gpt).\n\n[![LlamaGPT on Umbrel App Store](https:\u002F\u002Fapps.umbrel.com\u002Fapp\u002Fllama-gpt\u002Fbadge-light.svg)](https:\u002F\u002Fapps.umbrel.com\u002Fapp\u002Fllama-gpt)\n\n### Install LlamaGPT on M1\u002FM2 Mac\n\nMake sure your have Docker and Xcode installed.\n\nThen, clone this repo and `cd` into it:\n\n```\ngit clone https:\u002F\u002Fgithub.com\u002Fgetumbrel\u002Fllama-gpt.git\ncd llama-gpt\n```\n\nRun LlamaGPT with the following command:\n\n```\n.\u002Frun-mac.sh --model 7b\n```\n\nYou can access LlamaGPT at http:\u002F\u002Flocalhost:3000.\n\n> To run 13B or 70B chat models, replace `7b` with `13b` or `70b` respectively.\n> To run 7B, 13B or 34B Code Llama models, replace `7b` with `code-7b`, `code-13b` or `code-34b` respectively.\n\nTo stop LlamaGPT, do `Ctrl + C` in Terminal.\n\n### Install LlamaGPT anywhere else with Docker\n\nYou can run LlamaGPT on any x86 or arm64 system. Make sure you have Docker installed.\n\nThen, clone this repo and `cd` into it:\n\n```\ngit clone https:\u002F\u002Fgithub.com\u002Fgetumbrel\u002Fllama-gpt.git\ncd llama-gpt\n```\n\nRun LlamaGPT with the following command:\n\n```\n.\u002Frun.sh --model 7b\n```\n\nOr if you have an Nvidia GPU, you can run LlamaGPT with CUDA support using the `--with-cuda` flag, like:\n\n```\n.\u002Frun.sh --model 7b --with-cuda\n```\n\nYou can access LlamaGPT at `http:\u002F\u002Flocalhost:3000`.\n\n> To run 13B or 70B chat models, replace `7b` with `13b` or `70b` respectively.\n> To run Code Llama 7B, 13B or 34B models, replace `7b` with `code-7b`, `code-13b` or `code-34b` respectively.\n\nTo stop LlamaGPT, do `Ctrl + C` in Terminal.\n\n> Note: On the first run, it may take a while for the model to be downloaded to the `\u002Fmodels` directory. You may also see lots of output like this for a few minutes, which is normal:\n>\n> ```\n> llama-gpt-llama-gpt-ui-1       | [INFO  wait] Host [llama-gpt-api-13b:8000] not yet available...\n> ```\n>\n> After the model has been automatically downloaded and loaded, and the API server is running, you'll see an output like:\n>\n> ```\n> llama-gpt-ui_1   | ready - started server on 0.0.0.0:3000, url: http:\u002F\u002Flocalhost:3000\n> ```\n>\n> You can then access LlamaGPT at http:\u002F\u002Flocalhost:3000.\n\n---\n\n### Install LlamaGPT with Kubernetes\n\nFirst, make sure you have a running Kubernetes cluster and `kubectl` is configured to interact with it.\n\nThen, clone this repo and `cd` into it.\n\nTo deploy to Kubernetes first create a namespace:\n\n```bash\nkubectl create ns llama\n```\n\nThen apply the manifests under the `\u002Fdeploy\u002Fkubernetes` directory with\n\n```bash\nkubectl apply -k deploy\u002Fkubernetes\u002F. -n llama\n```\n\nExpose your service however you would normally do that.\n\n## OpenAI compatible API\n\nThanks to llama-cpp-python, a drop-in replacement for OpenAI API is available at `http:\u002F\u002Flocalhost:3001`. Open http:\u002F\u002Flocalhost:3001\u002Fdocs to see the API documentation.\n\n## Benchmarks\n\nWe've tested LlamaGPT models on the following hardware with the default system prompt, and user prompt: \"How does the universe expand?\" at temperature 0 to guarantee deterministic results. Generation speed is averaged over the first 10 generations.\n\nFeel free to add your own benchmarks to this table by opening a pull request.\n\n#### Nous Hermes Llama 2 7B Chat (GGML q4_0)\n\n| Device                              | Generation speed |\n| ----------------------------------- | ---------------- |\n| M1 Max MacBook Pro (64GB RAM)       | 54 tokens\u002Fsec    |\n| GCP c2-standard-16 vCPU (64 GB RAM) | 16.7 tokens\u002Fsec  |\n| Ryzen 5700G 4.4GHz 4c (16 GB RAM)   | 11.50 tokens\u002Fsec |\n| GCP c2-standard-4 vCPU (16 GB RAM)  | 4.3 tokens\u002Fsec   |\n| Umbrel Home (16GB RAM)              | 2.7 tokens\u002Fsec   |\n| Raspberry Pi 4 (8GB RAM)            | 0.9 tokens\u002Fsec   |\n\n#### Nous Hermes Llama 2 13B Chat (GGML q4_0)\n\n| Device                              | Generation speed |\n| ----------------------------------- | ---------------- |\n| M1 Max MacBook Pro (64GB RAM)       | 20 tokens\u002Fsec    |\n| GCP c2-standard-16 vCPU (64 GB RAM) | 8.6 tokens\u002Fsec   |\n| GCP c2-standard-4 vCPU (16 GB RAM)  | 2.2 tokens\u002Fsec   |\n| Umbrel Home (16GB RAM)              | 1.5 tokens\u002Fsec   |\n\n#### Nous Hermes Llama 2 70B Chat (GGML q4_0)\n\n| Device                              | Generation speed |\n| ----------------------------------- | ---------------- |\n| M1 Max MacBook Pro (64GB RAM)       | 4.8 tokens\u002Fsec   |\n| GCP e2-standard-16 vCPU (64 GB RAM) | 1.75 tokens\u002Fsec  |\n| GCP c2-standard-16 vCPU (64 GB RAM) | 1.62 tokens\u002Fsec  |\n\n#### Code Llama 7B Chat (GGUF Q4_K_M)\n\n| Device                        | Generation speed |\n| ----------------------------- | ---------------- |\n| M1 Max MacBook Pro (64GB RAM) | 41 tokens\u002Fsec    |\n\n#### Code Llama 13B Chat (GGUF Q4_K_M)\n\n| Device                        | Generation speed |\n| ----------------------------- | ---------------- |\n| M1 Max MacBook Pro (64GB RAM) | 25 tokens\u002Fsec    |\n\n#### Phind Code Llama 34B Chat (GGUF Q4_K_M)\n\n| Device                        | Generation speed |\n| ----------------------------- | ---------------- |\n| M1 Max MacBook Pro (64GB RAM) | 10.26 tokens\u002Fsec |\n\n## Roadmap and contributing\n\nWe're looking to add more features to LlamaGPT. You can see the roadmap [here](https:\u002F\u002Fgithub.com\u002Fgetumbrel\u002Fllama-gpt\u002Fissues\u002F8#issuecomment-1681321145). The highest priorities are:\n\n- [x] Moving the model out of the Docker image and into a separate volume.\n- [x] Add Metal support for M1\u002FM2 Macs.\n- [x] Add support for Code Llama models.\n- [x] Add CUDA support for NVIDIA GPUs.\n- [ ] Add ability to load custom models.\n- [ ] Allow users to switch between models.\n\nIf you're a developer who'd like to help with any of these, please open an issue to discuss the best way to tackle the challenge. If you're looking to help but not sure where to begin, check out [these issues](https:\u002F\u002Fgithub.com\u002Fgetumbrel\u002Fllama-gpt\u002Flabels\u002Fgood%20first%20issue) that have specifically been marked as being friendly to new contributors.\n\n## Acknowledgements\n\nA massive thank you to the following developers and teams for making LlamaGPT possible:\n\n- [Mckay Wrigley](https:\u002F\u002Fgithub.com\u002Fmckaywrigley) for building [Chatbot UI](https:\u002F\u002Fgithub.com\u002Fmckaywrigley).\n- [Georgi Gerganov](https:\u002F\u002Fgithub.com\u002Fggerganov) for implementing [llama.cpp](https:\u002F\u002Fgithub.com\u002Fggerganov\u002Fllama.cpp).\n- [Andrei](https:\u002F\u002Fgithub.com\u002Fabetlen) for building the [Python bindings for llama.cpp](https:\u002F\u002Fgithub.com\u002Fabetlen\u002Fllama-cpp-python).\n- [NousResearch](https:\u002F\u002Fnousresearch.com) for [fine-tuning the Llama 2 7B and 13B models](https:\u002F\u002Fhuggingface.co\u002FNousResearch).\n- [Phind](https:\u002F\u002Fwww.phind.com\u002F) for [fine-tuning the Code Llama 34B model](https:\u002F\u002Fwww.phind.com\u002Fblog\u002Fcode-llama-beats-gpt4).\n- [Tom Jobbins](https:\u002F\u002Fhuggingface.co\u002FTheBloke) for [quantizing the Llama 2 models](https:\u002F\u002Fhuggingface.co\u002FTheBloke\u002FNous-Hermes-Llama-2-7B-GGML).\n- [Meta](https:\u002F\u002Fai.meta.com\u002Fllama) for releasing Llama 2 and Code Llama under a permissive license.\n\n---\n\n[![License](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002Fgetumbrel\u002Fllama-gpt?color=%235351FB)](https:\u002F\u002Fgithub.com\u002Fgetumbrel\u002Fllama-gpt\u002Fblob\u002Fmaster\u002FLICENSE.md)\n\n[umbrel.com](https:\u002F\u002Fumbrel.com)\n","LlamaGPT是一个自托管、离线的类ChatGPT聊天机器人，基于Llama 2模型构建，确保100%的数据隐私，所有数据均不会离开你的设备。该项目支持Code Llama模型及Nvidia GPU，提供多种预训练模型选项，包括不同大小的对话模型和代码生成模型，满足多样化的应用需求。用户可以根据自身硬件条件选择合适的模型进行部署。LlamaGPT适合需要在本地环境中使用高级语言模型处理文本生成或代码辅助任务的个人开发者或团队，特别是对数据安全有严格要求的场景。项目采用TypeScript编写，并遵循MIT许可协议开放源码，便于二次开发与定制。",2,"2026-06-11 03:36:56","high_star"]