[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72887":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":29,"readmeContent":30,"aiSummary":31,"trendingCount":16,"starSnapshotCount":16,"syncStatus":32,"lastSyncTime":33,"discoverSource":34},72887,"mistral-inference","mistralai\u002Fmistral-inference","mistralai","Official inference library for Mistral models","https:\u002F\u002Fmistral.ai\u002F",null,"Jupyter Notebook",10813,1052,124,133,0,3,4,15,9,78.57,"Apache License 2.0",false,"main",true,[27,28,7],"llm","llm-inference","2026-06-12 04:01:07","# Mistral Inference\n\u003Ca target=\"_blank\" href=\"https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Fmistralai\u002Fmistral-inference\u002Fblob\u002Fmain\u002Ftutorials\u002Fgetting_started.ipynb\">\n  \u003Cimg src=\"https:\u002F\u002Fcolab.research.google.com\u002Fassets\u002Fcolab-badge.svg\" alt=\"Open In Colab\"\u002F>\n\u003C\u002Fa>\n\n\nThis repository contains minimal code to run Mistral models.\n\nBlog 7B: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fannouncing-mistral-7b\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fannouncing-mistral-7b\u002F)\\\nBlog 8x7B: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-of-experts\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-of-experts\u002F)\\\nBlog 8x22B: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-8x22b\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmixtral-8x22b\u002F)\\\nBlog Codestral 22B: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fcodestral](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fcodestral\u002F) \\\nBlog Codestral Mamba 7B: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fcodestral-mamba\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fcodestral-mamba\u002F) \\\nBlog Mathstral 7B: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmathstral\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmathstral\u002F) \\\nBlog Nemo: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-nemo\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-nemo\u002F) \\\nBlog Mistral Large 2: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-large-2407\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-large-2407\u002F) \\\nBlog Pixtral 12B: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fpixtral-12b\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fpixtral-12b\u002F)\nBlog Mistral Small 3.1: [https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-small-3-1\u002F](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-small-3-1\u002F)\n\nDiscord: [https:\u002F\u002Fdiscord.com\u002Finvite\u002Fmistralai](https:\u002F\u002Fdiscord.com\u002Finvite\u002Fmistralai)\\\nDocumentation: [https:\u002F\u002Fdocs.mistral.ai\u002F](https:\u002F\u002Fdocs.mistral.ai\u002F)\\\nGuardrailing: [https:\u002F\u002Fdocs.mistral.ai\u002Fusage\u002Fguardrailing](https:\u002F\u002Fdocs.mistral.ai\u002Fusage\u002Fguardrailing)\n\n## Installation\n\nNote: You will use a GPU to install `mistral-inference`, as it currently requires `xformers` to be installed and `xformers` itself needs a GPU for installation.\n\n### PyPI\n\n```\npip install mistral-inference\n```\n\n### Local\n\n```\ncd $HOME && git clone https:\u002F\u002Fgithub.com\u002Fmistralai\u002Fmistral-inference\ncd $HOME\u002Fmistral-inference && poetry install .\n```\n\n## Model download\n\n### Direct links\n\n| Name        | Download | md5sum |\n|-------------|-------|-------|\n| 7B Instruct | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmistral-7b-v0-3\u002Fmistral-7B-Instruct-v0.3.tar | `80b71fcb6416085bcb4efad86dfb4d52` |\n| 8x7B Instruct | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmixtral-8x7b-v0-1\u002FMixtral-8x7B-v0.1-Instruct.tar (**Updated model coming soon!**) | `8e2d3930145dc43d3084396f49d38a3f` |\n| 8x22 Instruct | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmixtral-8x22b-v0-3\u002Fmixtral-8x22B-Instruct-v0.3.tar | `471a02a6902706a2f1e44a693813855b` |\n| 7B Base | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmistral-7b-v0-3\u002Fmistral-7B-v0.3.tar | `0663b293810d7571dad25dae2f2a5806` |\n| 8x7B |     **Updated model coming soon!**       | - |\n| 8x22B | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmixtral-8x22b-v0-3\u002Fmixtral-8x22B-v0.3.tar | `a2fa75117174f87d1197e3a4eb50371a` |\n| Codestral 22B | https:\u002F\u002Fmodels.mistralcdn.com\u002Fcodestral-22b-v0-1\u002Fcodestral-22B-v0.1.tar | `1ea95d474a1d374b1d1b20a8e0159de3` |\n| Mathstral 7B | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmathstral-7b-v0-1\u002Fmathstral-7B-v0.1.tar | `5f05443e94489c261462794b1016f10b` |\n| Codestral-Mamba 7B | https:\u002F\u002Fmodels.mistralcdn.com\u002Fcodestral-mamba-7b-v0-1\u002Fcodestral-mamba-7B-v0.1.tar | `d3993e4024d1395910c55db0d11db163` |\n| Nemo Base | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmistral-nemo-2407\u002Fmistral-nemo-base-2407.tar | `c5d079ac4b55fc1ae35f51f0a3c0eb83` |\n| Nemo Instruct | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmistral-nemo-2407\u002Fmistral-nemo-instruct-2407.tar | `296fbdf911cb88e6f0be74cd04827fe7` |\n| Mistral Large 2 | https:\u002F\u002Fmodels.mistralcdn.com\u002Fmistral-large-2407\u002Fmistral-large-instruct-2407.tar | `fc602155f9e39151fba81fcaab2fa7c4` |\n\nNote:\n- **Important**:\n  - `mixtral-8x22B-Instruct-v0.3.tar` is exactly the same as [Mixtral-8x22B-Instruct-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x22B-Instruct-v0.1), only stored in `.safetensors` format\n  - `mixtral-8x22B-v0.3.tar` is the same as [Mixtral-8x22B-v0.1](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMixtral-8x22B-v0.1), but has an extended vocabulary of 32768 tokens.\n  - `codestral-22B-v0.1.tar` has a custom non-commercial license, called [Mistral AI Non-Production (MNPL) License](https:\u002F\u002Fmistral.ai\u002Flicenses\u002FMNPL-0.1.md)\n  - `mistral-large-instruct-2407.tar` has a custom non-commercial license, called [Mistral AI Research (MRL) License](https:\u002F\u002Fmistral.ai\u002Flicenses\u002FMRL-0.1.md)\n- All of the listed models above support function calling. For example, Mistral 7B Base\u002FInstruct v3 is a minor update to Mistral 7B Base\u002FInstruct v2,  with the addition of function calling capabilities.\n- The \"coming soon\" models will include function calling as well.\n- You can download the previous versions of our models from our [docs](https:\u002F\u002Fdocs.mistral.ai\u002Fgetting-started\u002Fopen_weight_models\u002F#downloading).\n\n### From Hugging Face Hub\n\n| Name        | ID | URL |\n|-------------|-------|-------|\n| Pixtral Large Instruct | mistralai\u002FPixtral-Large-Instruct-2411 | https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FPixtral-Large-Instruct-2411 |\n| Pixtral 12B Base | mistralai\u002FPixtral-12B-Base-2409 | https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FPixtral-12B-Base-2409 |\n| Pixtral 12B | mistralai\u002FPixtral-12B-2409 | https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FPixtral-12B-2409 |\n| Mistral Small 3.1 24B Base | mistralai\u002FMistral-Small-3.1-24B-Base-2503 | https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-Small-3.1-24B-Base-2503\n| Mistral Small 3.1 24B Instruct | mistralai\u002FMistral-Small-3.1-24B-Instruct-2503 | https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-Small-3.1-24B-Instruct-2503 |\n\n\n### Usage\n\n**News!!!**: Mistral Large 2 is out. Read more about its capabilities [here](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-large-2407\u002F).\n\nCreate a local folder to store models\n```sh\nexport MISTRAL_MODEL=$HOME\u002Fmistral_models\nmkdir -p $MISTRAL_MODEL\n```\n\nDownload any of the above links and extract the content, *e.g.*:\n\n```sh\nexport 12B_DIR=$MISTRAL_MODEL\u002F12B_Nemo\nwget https:\u002F\u002Fmodels.mistralcdn.com\u002Fmistral-nemo-2407\u002Fmistral-nemo-instruct-2407.tar\nmkdir -p $12B_DIR\ntar -xf mistral-nemo-instruct-2407.tar -C $12B_DIR\n```\n\nor\n\n```sh\nexport M8x7B_DIR=$MISTRAL_MODEL\u002F8x7b_instruct\nwget https:\u002F\u002Fmodels.mistralcdn.com\u002Fmixtral-8x7b-v0-1\u002FMixtral-8x7B-v0.1-Instruct.tar\nmkdir -p $M8x7B_DIR\ntar -xf Mixtral-8x7B-v0.1-Instruct.tar -C $M8x7B_DIR\n```\n\nFor Hugging Face models' weights, here is an example to download [Mistral Small 3.1 24B Instruct](https:\u002F\u002Fhuggingface.co\u002Fmistralai\u002FMistral-Small-3.1-24B-Instruct-2503):\n\n```python\nfrom pathlib import Path\nfrom huggingface_hub import snapshot_download\n\n\nmistral_models_path = Path.home().joinpath(\"mistral_models\")\n\nmodel_path = mistral_models_path \u002F \"mistral-small-3.1-instruct\"\nmodel_path.mkdir(parents=True, exist_ok=True)\n\nrepo_id = \"mistralai\u002FMistral-Small-3.1-24B-Instruct-2503\"\n\nsnapshot_download(\n    repo_id=repo_id,\n    allow_patterns=[\"params.json\", \"consolidated.safetensors\", \"tekken.json\"],\n    local_dir=model_path,\n)\n```\n\n## Usage\n\nThe following sections give an overview of how to run the model from the Command-line interface (CLI) or directly within Python.\n\n### CLI\n\n- **Demo**\n\nTo test that a model works in your setup, you can run the `mistral-demo` command.\n*E.g.* the 12B Mistral-Nemo model can be tested on a single GPU as follows:\n\n```sh\nmistral-demo $12B_DIR\n```\n\nLarge models, such **8x7B** and **8x22B** have to be run in a multi-GPU setup.\nFor these models, you can use the following command:\n\n```sh\ntorchrun --nproc-per-node 2 --no-python mistral-demo $M8x7B_DIR\n```\n\n*Note*: Change `--nproc-per-node` to more GPUs if available.\n\n- **Chat**\n\nTo interactively chat with the models, you can make use of the `mistral-chat` command.\n\n```sh\nmistral-chat $12B_DIR --instruct --max_tokens 1024 --temperature 0.35\n```\n\nFor large models, you can make use of `torchrun`.\n\n```sh\ntorchrun --nproc-per-node 2 --no-python mistral-chat $M8x7B_DIR --instruct\n```\n\n*Note*: Change `--nproc-per-node` to more GPUs if necessary (*e.g.* for 8x22B).\n\n- **Chat with Codestral**\n\nTo use [Codestral](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fcodestral\u002F) as a coding assistant you can run the following command using `mistral-chat`.\nMake sure `$M22B_CODESTRAL` is set to a valid path to the downloaded codestral folder, e.g. `$HOME\u002Fmistral_models\u002FCodestral-22B-v0.1`\n\n```sh\nmistral-chat $M22B_CODESTRAL --instruct --max_tokens 256\n```\n\nIf you prompt it with *\"Write me a function that computes fibonacci in Rust\"*, the model should generate something along the following lines:\n\n```sh\nSure, here's a simple implementation of a function that computes the Fibonacci sequence in Rust. This function takes an integer `n` as an argument and returns the `n`th Fibonacci number.\n\nfn fibonacci(n: u32) -> u32 {\n    match n {\n        0 => 0,\n        1 => 1,\n        _ => fibonacci(n - 1) + fibonacci(n - 2),\n    }\n}\n\nfn main() {\n    let n = 10;\n    println!(\"The {}th Fibonacci number is: {}\", n, fibonacci(n));\n}\n\nThis function uses recursion to calculate the Fibonacci number. However, it's not the most efficient solution because it performs a lot of redundant calculations. A more efficient solution would use a loop to iteratively calculate the Fibonacci numbers.\n```\n\nYou can continue chatting afterwards, *e.g.* with *\"Translate it to Python\"*.\n\n- **Chat with Codestral-Mamba**\n\nTo use [Codestral-Mamba](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fcodestral-mamba\u002F) as a coding assistant you can run the following command using `mistral-chat`.\nMake sure `$7B_CODESTRAL_MAMBA` is set to a valid path to the downloaded codestral-mamba folder, e.g. `$HOME\u002Fmistral_models\u002Fmamba-codestral-7B-v0.1`.\n\nYou then need to additionally install the following packages:\n\n```\npip install packaging mamba-ssm causal-conv1d transformers\n```\n\nbefore you can start chatting:\n\n```sh\nmistral-chat $7B_CODESTRAL_MAMBA --instruct --max_tokens 256\n```\n\n- **Chat with Mathstral**\n\nTo use [Mathstral](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmathstral\u002F) as an assistant you can run the following command using `mistral-chat`.\nMake sure `$7B_MATHSTRAL` is set to a valid path to the downloaded codestral folder, e.g. `$HOME\u002Fmistral_models\u002Fmathstral-7B-v0.1`\n\n```sh\nmistral-chat $7B_MATHSTRAL --instruct --max_tokens 256\n```\n\nIf you prompt it with *\"Albert likes to surf every week. Each surfing session lasts for 4 hours and costs $20 per hour. How much would Albert spend in 5 weeks?\"*, the model should answer with the correct calculation.\n\nYou can then continue chatting afterwards, *e.g.* with *\"How much would he spend in a year?\"*.\n\n- **Chat with Mistral Small 3.1 24B Instruct**\n\nTo use [Mistral Small 3.1 24B Instruct](https:\u002F\u002Fmistral.ai\u002Fnews\u002Fmistral-small-3-1\u002F) as an assistant you can run the following command using `mistral-chat`.\nMake sure `$MISTRAL_SMALL_3_1_INSTRUCT` is set to a valid path to the downloaded mistral small folder, e.g. `$HOME\u002Fmistral_models\u002Fmistral-small-3.1-instruct`\n\n```sh\n    mistral-chat $MISTRAL_SMALL_3_1_INSTRUCT --instruct --max_tokens 256\n```\n\nIf you prompt it with *\"The above image presents an image of which park ? Please give the hints to identify the park.\"* with the following image URL *https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fpatrickvonplaten\u002Frandom_img\u002Fresolve\u002Fmain\u002Fyosemite.png*, the model should answer with the Yosemite park and give hints to identify it.\n\nYou can then continue chatting afterwards, *e.g.* with *\"What is the name of the lake in the image?\"*. The model should respond that it is not a lake but a river.\n\n### Python\n\n- *Instruction Following*:\n\n```py\nfrom mistral_inference.transformer import Transformer\nfrom mistral_inference.generate import generate\n\nfrom mistral_common.tokens.tokenizers.mistral import MistralTokenizer\nfrom mistral_common.protocol.instruct.messages import UserMessage\nfrom mistral_common.protocol.instruct.request import ChatCompletionRequest\n\n\ntokenizer = MistralTokenizer.from_file(\".\u002Fmistral-nemo-instruct-v0.1\u002Ftekken.json\")  # change to extracted tokenizer file\nmodel = Transformer.from_folder(\".\u002Fmistral-nemo-instruct-v0.1\")  # change to extracted model dir\n\nprompt = \"How expensive would it be to ask a window cleaner to clean all windows in Paris. Make a reasonable guess in US Dollar.\"\n\ncompletion_request = ChatCompletionRequest(messages=[UserMessage(content=prompt)])\n\ntokens = tokenizer.encode_chat_completion(completion_request).tokens\n\nout_tokens, _ = generate([tokens], model, max_tokens=1024, temperature=0.35, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)\nresult = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])\n\nprint(result)\n```\n\n- *Multimodal Instruction Following*:\n\n\n```python\nfrom pathlib import Path\n\nfrom huggingface_hub import snapshot_download\nfrom mistral_common.protocol.instruct.messages import ImageURLChunk, TextChunk\nfrom mistral_common.tokens.tokenizers.mistral import MistralTokenizer\nfrom mistral_inference.generate import generate\nfrom mistral_inference.transformer import Transformer\n\nmodel_path = Path.home().joinpath(\"mistral_models\") \u002F \"mistral-small-3.1-instruct\" # change to extracted model\n\ntokenizer = MistralTokenizer.from_file(model_path \u002F \"tekken.json\")\nmodel = Transformer.from_folder(model_path)\n\nurl = \"https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fpatrickvonplaten\u002Frandom_img\u002Fresolve\u002Fmain\u002Fyosemite.png\"\nprompt = \"The above image presents an image of which park ? Please give the hints to identify the park.\"\n\nuser_content = [ImageURLChunk(image_url=url), TextChunk(text=prompt)]\n\ntokens, images = tokenizer.instruct_tokenizer.encode_user_content(user_content, False)\n\nout_tokens, _ = generate(\n    [tokens],\n    model,\n    images=[images],\n    max_tokens=256,\n    temperature=0.15,\n    eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id,\n)\nresult = tokenizer.decode(out_tokens[0])\n\nprint(\"Prompt:\", prompt)\nprint(\"Completion:\", result)\n```\n\n- *Function Calling*:\n\n```py\nfrom mistral_common.protocol.instruct.tool_calls import Function, Tool\n\ncompletion_request = ChatCompletionRequest(\n    tools=[\n        Tool(\n            function=Function(\n                name=\"get_current_weather\",\n                description=\"Get the current weather\",\n                parameters={\n                    \"type\": \"object\",\n                    \"properties\": {\n                        \"location\": {\n                            \"type\": \"string\",\n                            \"description\": \"The city and state, e.g. San Francisco, CA\",\n                        },\n                        \"format\": {\n                            \"type\": \"string\",\n                            \"enum\": [\"celsius\", \"fahrenheit\"],\n                            \"description\": \"The temperature unit to use. Infer this from the users location.\",\n                        },\n                    },\n                    \"required\": [\"location\", \"format\"],\n                },\n            )\n        )\n    ],\n    messages=[\n        UserMessage(content=\"What's the weather like today in Paris?\"),\n        ],\n)\n\ntokens = tokenizer.encode_chat_completion(completion_request).tokens\n\nout_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)\nresult = tokenizer.instruct_tokenizer.tokenizer.decode(out_tokens[0])\n\nprint(result)\n```\n\n- *Fill-in-the-middle (FIM)*:\n\nMake sure to have `mistral-common >= 1.2.0` installed:\n```\npip install --upgrade mistral-common\n```\n\nYou can simulate a code completion in-filling as follows.\n\n```py\nfrom mistral_inference.transformer import Transformer\nfrom mistral_inference.generate import generate\nfrom mistral_common.tokens.tokenizers.mistral import MistralTokenizer\nfrom mistral_common.tokens.instruct.request import FIMRequest\n\ntokenizer = MistralTokenizer.from_model(\"codestral-22b\")\nmodel = Transformer.from_folder(\".\u002Fmistral_22b_codestral\")\n\nprefix = \"\"\"def add(\"\"\"\nsuffix = \"\"\"    return sum\"\"\"\n\nrequest = FIMRequest(prompt=prefix, suffix=suffix)\n\ntokens = tokenizer.encode_fim(request).tokens\n\nout_tokens, _ = generate([tokens], model, max_tokens=256, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)\nresult = tokenizer.decode(out_tokens[0])\n\nmiddle = result.split(suffix)[0].strip()\nprint(middle)\n```\n\n### Test\n\nTo run logits equivalence:\n```\npython -m pytest tests\n```\n\n## Deployment\n\nThe `deploy` folder contains code to build a [vLLM](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm) image with the required dependencies to serve the Mistral AI model. In the image, the [transformers](https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Ftransformers\u002F) library is used instead of the reference implementation. To build it:\n\n```bash\ndocker build deploy --build-arg MAX_JOBS=8\n```\n\nInstructions to run the image can be found in the [official documentation](https:\u002F\u002Fdocs.mistral.ai\u002Fquickstart).\n\n\n## Model platforms\n\n- Use Mistral models on [Mistral AI official API](https:\u002F\u002Fconsole.mistral.ai\u002F) (La Plateforme)\n- Use Mistral models via [cloud providers](https:\u002F\u002Fdocs.mistral.ai\u002Fdeployment\u002Fcloud\u002Foverview\u002F)\n\n## References\n\n[1]: [LoRA](https:\u002F\u002Farxiv.org\u002Fabs\u002F2106.09685): Low-Rank Adaptation of Large Language Models, Hu et al. 2021\n\n## License\n\nThis library is licensed under the Apache 2.0 License. See the [LICENCE](.\u002FLICENCE) file for more information.\n\n*You must not use this library or our models in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.*\n","Mistral Inference 是官方提供的用于运行 Mistral 模型的推理库。该项目支持多种版本和规模的 Mistral 模型，包括 7B、8x7B、8x22B 等，并提供了详细的安装指南和模型下载链接。其核心功能在于简化了 Mistral 模型的部署过程，通过使用 xformers 库来加速 GPU 上的推理操作。适用于需要快速集成大语言模型以进行自然语言处理任务的场景，如文本生成、代码补全和数学问题解答等。此外，项目还提供了 Jupyter Notebook 格式的教程，便于用户快速上手。",2,"2026-06-11 03:43:54","high_star"]