[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-2190":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":10,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":21,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":24,"readmeContent":25,"aiSummary":26,"trendingCount":16,"starSnapshotCount":16,"syncStatus":27,"lastSyncTime":28,"discoverSource":29},2190,"Qwen3","QwenLM\u002FQwen3","QwenLM","Qwen3 is the large language model series developed by Qwen team, Alibaba Cloud.","",null,"Python",27299,1993,165,43,0,21,90,11,86.4,false,"main",[],"2026-06-12 04:00:13","# Qwen3\n\n\u003Cp align=\"center\">\n    \u003Cimg src=\"https:\u002F\u002Fqianwen-res.oss-accelerate-overseas.aliyuncs.com\u002Flogo_qwen3.png\" width=\"400\"\u002F>\n\u003Cp>\n\n\u003Cp align=\"center\">\n          💜 \u003Ca href=\"https:\u002F\u002Fchat.qwen.ai\u002F\">\u003Cb>Qwen Chat\u003C\u002Fb>\u003C\u002Fa>&nbsp&nbsp | &nbsp&nbsp🤗 \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002FQwen\">Hugging Face\u003C\u002Fa>&nbsp&nbsp | &nbsp&nbsp🤖 \u003Ca href=\"https:\u002F\u002Fmodelscope.cn\u002Forganization\u002Fqwen\">ModelScope\u003C\u002Fa>&nbsp&nbsp | &nbsp&nbsp 📑 \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.09388\">Paper\u003C\u002Fa> &nbsp&nbsp | &nbsp&nbsp 📑 \u003Ca href=\"https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen3\u002F\">Blog\u003C\u002Fa> &nbsp&nbsp ｜ &nbsp&nbsp📖 \u003Ca href=\"https:\u002F\u002Fqwen.readthedocs.io\u002F\">Documentation\u003C\u002Fa>\n\u003Cbr>\n🖥️ \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FQwen\u002FQwen3-Demo\">Demo\u003C\u002Fa>&nbsp&nbsp | &nbsp&nbsp💬 \u003Ca href=\"https:\u002F\u002Fgithub.com\u002FQwenLM\u002FQwen\u002Fblob\u002Fmain\u002Fassets\u002Fwechat.png\">WeChat (微信)\u003C\u002Fa>&nbsp&nbsp | &nbsp&nbsp🫨 \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FCV4E9rpNSD\">Discord\u003C\u002Fa>&nbsp&nbsp\n\u003C\u002Fp>\n\n\nVisit our Hugging Face or ModelScope organization (click links above), search checkpoints with names starting with `Qwen3-` or visit the [Qwen3 collection](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002FQwen\u002Fqwen3-67dd247413f0e2e4f653967f), and you will find all you need! Enjoy!\n\nTo learn more about Qwen3, feel free to read our documentation \\[[EN](https:\u002F\u002Fqwen.readthedocs.io\u002Fen\u002Flatest\u002F)|[ZH](https:\u002F\u002Fqwen.readthedocs.io\u002Fzh-cn\u002Flatest\u002F)\\]. Our documentation consists of the following sections:\n\n- Quickstart: the basic usages and demonstrations;\n- Inference: the guidance for the inference with Transformers, including batch inference, streaming, etc.;\n- Run Locally: the instructions for running LLM locally on CPU and GPU, with frameworks like llama.cpp, Ollama, and LM Studio;\n- Deployment: the demonstration of how to deploy Qwen for large-scale inference with frameworks like SGLang, vLLM, TGI, etc.;\n- Quantization: the practice of quantizing LLMs with GPTQ, AWQ, as well as the guidance for how to make high-quality quantized GGUF files;\n- Training: the instructions for post-training, including SFT and RLHF (TODO) with frameworks like Axolotl, LLaMA-Factory, etc.\n- Framework: the usage of Qwen with frameworks for application, e.g., RAG, Agent, etc.\n\n## Introduction\n\n### Qwen3-2507\n\nOver the past three months, we continued to explore the potential of the Qwen3 families and we are excited to introduce the updated **Qwen3-2507** in two variants, Qwen3-Instruct-2507 and Qwen3-Thinking-2507, and three sizes, 235B-A22B, 30B-A3B, and 4B.\n\n**Qwen3-Instruct-2507** is the updated version of the previous Qwen3 non-thinking mode, featuring the following key enhancements:  \n\n- **Significant improvements** in general capabilities, including **instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage**.  \n- **Substantial gains** in long-tail knowledge coverage across **multiple languages**.  \n- **Markedly better alignment** with user preferences in **subjective and open-ended tasks**, enabling more helpful responses and higher-quality text generation.  \n- **Enhanced capabilities** in **256K-token long-context understanding**, extendable up to **1 million tokens**.\n\n**Qwen3-Thinking-2507** is the continuation of Qwen3 thinking model, with improved quality and depth of reasoning, featuring the following key enhancements:\n- **Significantly improved performance** on reasoning tasks, including logical reasoning, mathematics, science, coding, and academic benchmarks that typically require human expertise — achieving **state-of-the-art results among open-weight thinking models**.\n- **Markedly better general capabilities**, such as instruction following, tool usage, text generation, and alignment with human preferences.\n- **Enhanced 256K long-context understanding** capabilities, extendable up to **1 million tokens**.\n\n\n\u003Cdetails>\n    \u003Csummary>\u003Cb>Previous Qwen3 Release\u003C\u002Fb>\u003C\u002Fsummary>\n    \u003Ch3>Qwen3 (aka Qwen3-2504)\u003C\u002Fh3>\n    \u003Cp>\n    We are excited to announce the release of Qwen3, the latest addition to the Qwen family of large language models. \n    These models represent our most advanced and intelligent systems to date, improving from our experience in building QwQ and Qwen2.5.\n    We are making the weights of Qwen3 available to the public, including both dense and Mixture-of-Expert (MoE) models. \n    \u003Cbr>\u003Cbr>\n    The highlights from Qwen3 include:\n        \u003Cul>\n            \u003Cli>\u003Cb>Dense and Mixture-of-Experts (MoE) models of various sizes\u003C\u002Fb>, available in 0.6B, 1.7B, 4B, 8B, 14B, 32B and 30B-A3B, 235B-A22B.\u003C\u002Fli>\n            \u003Cli>\u003Cb>Seamless switching between thinking mode\u003C\u002Fb> (for complex logical reasoning, math, and coding) and \u003Cb>non-thinking mode\u003C\u002Fb> (for efficient, general-purpose chat), ensuring optimal performance across various scenarios.\u003C\u002Fli>\n            \u003Cli>\u003Cb>Significantly enhancement in reasoning capabilities\u003C\u002Fb>, surpassing previous QwQ (in thinking mode) and Qwen2.5 instruct models (in non-thinking mode) on mathematics, code generation, and commonsense logical reasoning.\u003C\u002Fli>\n            \u003Cli>\u003Cb>Superior human preference alignment\u003C\u002Fb>, excelling in creative writing, role-playing, multi-turn dialogues, and instruction following, to deliver a more natural, engaging, and immersive conversational experience.\u003C\u002Fli>\n            \u003Cli>\u003Cb>Expertise in agent capabilities\u003C\u002Fb>, enabling precise integration with external tools in both thinking and unthinking modes and achieving leading performance among open-source models in complex agent-based tasks.\u003C\u002Fli>\n            \u003Cli>\u003Cb>Support of 100+ languages and dialects\u003C\u002Fb> with strong capabilities for \u003Cb>multilingual instruction following\u003C\u002Fb> and \u003Cb>translation\u003C\u002Fb>.\u003C\u002Fli>\n        \u003C\u002Ful>\n    \u003C\u002Fp>\n\u003C\u002Fdetails>\n\n\n## News\n- 2025.08.08: You can now use Qwen3-2507 to handle ultra-long inputs of **1 million tokens**! See the update modelcards ([235B-A22B-Instruct-2507](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-235B-A22B-Instruct-2507), [235B-A22B-Thinking-2507](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-235B-A22B-Thinking-2507), [A30B-A3B-Instruct-2507](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-30B-A3B-Instruct-2507), [A30B-A3B-Thinking-2507](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-30B-A3B-Thinking-2507)) for how to enable this feature.\n- 2025.08.06: The final open release of Qwen3-2507, [Qwen3-4B-Instruct-2507](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-4B-Instruct-2507) and [Qwen3-4B-Thinking-2507](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-4B-Thinking-2507), is out!\n- 2025.07.31: Qwen3-30B-A3B-Thinking-2507 is released. Check out the [modelcard](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-30B-A3B-Thinking-2507) for more details!\n- 2025.07.30: Qwen3-30B-A3B-Instruct-2507 is released. Check out the [modelcard](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-30B-A3B-Instruct-2507) for more details!\n- 2025.07.25: We released the updated version of Qwen3-235B-A22B thinking mode, named Qwen3-235B-A22B-Thinking-2507. Check out the [modelcard](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-235B-A22B-Thinking-2507) for more details!\n- 2025.07.21: We released the updated version of Qwen3-235B-A22B non-thinking mode, named Qwen3-235B-A22B-Instruct-2507, featuring significant enhancements over the previous version and supporting 256K-token long-context understanding. Check our [modelcard](https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-235B-A22B-Instruct-2507) for more details!\n- 2025.04.29: We released the Qwen3 series. Check our [blog](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen3) for more details!\n- 2024.09.19: We released the Qwen2.5 series. This time there are 3 extra model sizes: 3B, 14B, and 32B for more possibilities. Check our [blog](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen2.5) for more!\n- 2024.06.06: We released the Qwen2 series. Check our [blog](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen2\u002F)!\n- 2024.03.28: We released the first MoE model of Qwen: Qwen1.5-MoE-A2.7B! Temporarily, only HF transformers and vLLM support the model. We will soon add the support of llama.cpp, mlx-lm, etc. Check our [blog](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen-moe\u002F) for more information!\n- 2024.02.05: We released the Qwen1.5 series.\n\n## Performance\n\nDetailed evaluation results are reported in this [📑 blog (Qwen3-2504)](https:\u002F\u002Fqwenlm.github.io\u002Fblog\u002Fqwen3\u002F) and this [📑 blog (Qwen3-2507) \\[coming soon\\]]().\n\nFor requirements on GPU memory and the respective throughput, see results [here](https:\u002F\u002Fqwen.readthedocs.io\u002Fen\u002Flatest\u002Fgetting_started\u002Fspeed_benchmark.html).\n\n## Run Qwen3\n\n### 🤗 Transformers\n\nTransformers is a library of pretrained natural language processing for inference and training. \nThe latest version of `transformers` is recommended and `transformers>=4.51.0` is required.\n\n#### Qwen3-Instruct-2507\n\nThe following contains a code snippet illustrating how to use Qwen3-30B-A3B-Instruct-2507 to generate content based on given inputs. \n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_name = \"Qwen\u002FQwen3-30B-A3B-Instruct-2507\"\n\n# load the tokenizer and the model\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    torch_dtype=\"auto\",\n    device_map=\"auto\"\n)\n\n# prepare the model input\nprompt = \"Give me a short introduction to large language model.\"\nmessages = [\n    {\"role\": \"user\", \"content\": prompt}\n]\ntext = tokenizer.apply_chat_template(\n    messages,\n    tokenize=False,\n    add_generation_prompt=True,\n)\nmodel_inputs = tokenizer([text], return_tensors=\"pt\").to(model.device)\n\n# conduct text completion\ngenerated_ids = model.generate(\n    **model_inputs,\n    max_new_tokens=16384\n)\noutput_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() \n\ncontent = tokenizer.decode(output_ids, skip_special_tokens=True)\n\nprint(\"content:\", content)\n```\n\n> [!Note]\n> Qwen3-Instruct-2507 supports only non-thinking mode and does not generate ``\u003Cthink>\u003C\u002Fthink>`` blocks in its output. Meanwhile, specifying `enable_thinking=False` is no longer required.\n\n\n#### Qwen3-Thinking-2507\n\nThe following contains a code snippet illustrating how to use Qwen3-30B-A3B-Thinking-2507 to generate content based on given inputs. \n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_name = \"Qwen\u002FQwen3-30B-A3B-Thinking-2507\"\n\n# load the tokenizer and the model\ntokenizer = AutoTokenizer.from_pretrained(model_name)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_name,\n    torch_dtype=\"auto\",\n    device_map=\"auto\"\n)\n\n# prepare the model input\nprompt = \"Give me a short introduction to large language model.\"\nmessages = [\n    {\"role\": \"user\", \"content\": prompt}\n]\ntext = tokenizer.apply_chat_template(\n    messages,\n    tokenize=False,\n    add_generation_prompt=True,\n)\nmodel_inputs = tokenizer([text], return_tensors=\"pt\").to(model.device)\n\n# conduct text completion\ngenerated_ids = model.generate(\n    **model_inputs,\n    max_new_tokens=32768\n)\noutput_ids = generated_ids[0][len(model_inputs.input_ids[0]):].tolist() \n\n# parsing thinking content\ntry:\n    # rindex finding 151668 (\u003C\u002Fthink>)\n    index = len(output_ids) - output_ids[::-1].index(151668)\nexcept ValueError:\n    index = 0\n\nthinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip(\"\\n\")\ncontent = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip(\"\\n\")\n\nprint(\"thinking content:\", thinking_content)  # no opening \u003Cthink> tag\nprint(\"content:\", content)\n\n```\n\n> [!Note]\n> Qwen3-Thinking-2507 supports only thinking mode.\n> Additionally, to enforce model thinking, the default chat template automatically includes `\u003Cthink>`. Therefore, it is normal for the model's output to contain only `\u003C\u002Fthink>` without an explicit opening `\u003Cthink>` tag.\n> \n> Qwen3-Thinking-2507 also features an increased thinking length. We strongly recommend its use in highly complex reasoning tasks with adequate maximum generation length.\n\n\n\n\u003Cdetails>\n    \u003Csummary>\u003Cb>Switching Thinking\u002FNon-thinking Modes for Previous Qwen3  Models\u003C\u002Fb>\u003C\u002Fsummary>\n    \u003Cp>\n    By default, Qwen3 models will think before response.\n    This could be controlled by\n        \u003Cul>\n            \u003Cli>\u003Ccode>enable_thinking=False\u003C\u002Fcode>: Passing \u003Ccode>enable_thinking=False\u003C\u002Fcode> to `tokenizer.apply_chat_template` will strictly prevent the model from generating thinking content.\u003C\u002Fli>\n            \u003Cli>\u003Ccode>\u002Fthink\u003C\u002Fcode> and \u003Ccode>\u002Fno_think\u003C\u002Fcode> instructions: Use those words in the system or user message to signify whether Qwen3 should think. In multi-turn conversations, the latest instruction is followed.\u003C\u002Fli>\n        \u003C\u002Ful>\n    \u003C\u002Fp>\n\u003C\u002Fdetails>\n\n\n### ModelScope\n\nWe strongly advise users especially those in mainland China to use ModelScope. \nModelScope adopts a Python API similar to Transformers.\nThe CLI tool `modelscope download` can help you solve issues concerning downloading checkpoints.\nFor vLLM and SGLang, the environment variable `VLLM_USE_MODELSCOPE=true` and `SGLANG_USE_MODELSCOPE=true` can be used respectively.\n\n\n### llama.cpp\n\n[`llama.cpp`](https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp) enables LLM inference with minimal setup and state-of-the-art performance on a wide range of hardware.\n`llama.cpp>=b5401` is recommended for the full support of Qwen3.\n\nTo use the CLI, run the following in a terminal:\n```shell\n.\u002Fllama-cli -hf Qwen\u002FQwen3-8B-GGUF:Q8_0 --jinja --color -ngl 99 -fa -sm row --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 -c 40960 -n 32768 --no-context-shift\n# CTRL+C to exit\n```\n\nTo use the API server, run the following in a terminal:\n```shell\n.\u002Fllama-server -hf Qwen\u002FQwen3-8B-GGUF:Q8_0 --jinja --reasoning-format deepseek -ngl 99 -fa -sm row --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 -c 40960 -n 32768 --no-context-shift --port 8080\n```\nA simple web front end will be at `http:\u002F\u002Flocalhost:8080` and an OpenAI-compatible API will be at `http:\u002F\u002Flocalhost:8080\u002Fv1`.\n\nFor additional guides, please refer to [our documentation](https:\u002F\u002Fqwen.readthedocs.io\u002Fen\u002Flatest\u002Frun_locally\u002Fllama.cpp.html).\n\n> [!Note]\n> llama.cpp adopts \"rotating context management\" and infinite generation is made possible by evicting earlier tokens.\n> It could configured by parameters and the commands above effectively disable it.\n> For more details, please refer to [our documentation](https:\u002F\u002Fqwen.readthedocs.io\u002Fen\u002Flatest\u002Frun_locally\u002Fllama.cpp.html#llama-cli).\n\n### Ollama\n\nAfter [installing Ollama](https:\u002F\u002Follama.com\u002F), you can initiate the Ollama service with the following command (Ollama v0.9.0 or higher is recommended):\n```shell\nollama serve\n# You need to keep this service running whenever you are using ollama\n```\n\nTo pull a model checkpoint and run the model, use the `ollama run` command. You can specify a model size by adding a suffix to `qwen3`, such as `:8b` or `:30b-a3b`:\n```shell\nollama run qwen3:8b\n# Setting parameters, type \"\u002Fset parameter num_ctx 40960\" and \"\u002Fset parameter num_predict 32768\"\n# To exit, type \"\u002Fbye\" and press ENTER\n# For Qwen3-2504 models,\n# - To enable thinking, which is the default, type \"\u002Fset think\"\n# - To disable thinking, type \"\u002Fset nothink\"\n```\n\nYou can also access the Ollama service via its OpenAI-compatible API. \nPlease note that you need to (1) keep `ollama serve` running while using the API, and (2) execute `ollama run qwen3:8b` before utilizing this API to ensure that the model checkpoint is prepared.\nThe API is at `http:\u002F\u002Flocalhost:11434\u002Fv1\u002F` by default.\n\nFor additional details, please visit [ollama.ai](https:\u002F\u002Follama.com\u002F).\n\n> [!Note]\n> Ollama's naming may not be consistent with the Qwen's original naming.\n> For example, `qwen3:30b-a3b` in Ollama points to `qwen3:30b-a3b-thinking-2507-q4_K_M` as of August 2025.\n> Please check \u003Chttps:\u002F\u002Follama.com\u002Flibrary\u002Fqwen3\u002Ftags> before use.\n\n\n> [!Note]\n> Ollama adopts the same \"rotating context management\" with llama.cpp.\n> However, its default settings (`num_ctx` 2048 and `num_predict` -1), suggesting infinite generation with a 2048-token context,\n> could lead to trouble for Qwen3 models.\n> We recommend setting `num_ctx` and `num_predict` properly.\n\n### LMStudio\n\nQwen3 has already been supported by [lmstudio.ai](https:\u002F\u002Flmstudio.ai\u002F). You can directly use LMStudio with our GGUF files.\n\n### ExecuTorch\n\nTo export and run on ExecuTorch (iOS, Android, Mac, Linux, and more), please follow this [example](https:\u002F\u002Fgithub.com\u002Fpytorch\u002Fexecutorch\u002Fblob\u002Fmain\u002Fexamples\u002Fmodels\u002Fqwen3\u002FREADME.md).\n\n### MNN\n\nTo export and run on MNN, which supports Qwen3 on mobile devices, please visit [Alibaba MNN](https:\u002F\u002Fgithub.com\u002Falibaba\u002FMNN).\n\n### MLX LM\n\nIf you are running on Apple Silicon, [`mlx-lm`](https:\u002F\u002Fgithub.com\u002Fml-explore\u002Fmlx-lm) also supports Qwen3 (`mlx-lm>=0.24.0`). \nLook for models ending with MLX on Hugging Face Hub.\n\n\n### OpenVINO\n\nIf you are running on Intel CPU or GPU, [OpenVINO toolkit](https:\u002F\u002Fgithub.com\u002Fopenvinotoolkit) supports Qwen3.\nYou can follow this [chatbot example](https:\u002F\u002Fgithub.com\u002Fopenvinotoolkit\u002Fopenvino_notebooks\u002Fblob\u002Flatest\u002Fnotebooks\u002Fllm-chatbot\u002Fllm-chatbot.ipynb).\n\n\n## Deploy Qwen3\n\nQwen3 is supported by multiple inference frameworks. \nHere we demonstrate the usage of `SGLang`, `vLLM` and `TensorRT-LLM`.\nYou can also find Qwen3 models from various inference providers, e.g., [Alibaba Cloud Model Studio](https:\u002F\u002Fwww.alibabacloud.com\u002Fen\u002Fproduct\u002Fmodelstudio).\n\n\n### SGLang\n\n[SGLang](https:\u002F\u002Fgithub.com\u002Fsgl-project\u002Fsglang) is a fast serving framework for large language models and vision language models.\nSGLang could be used to launch a server with OpenAI-compatible API service. \n`sglang>=0.4.6.post1` is required.\n\nFor Qwen3-Instruct-2507, \n```shell\npython -m sglang.launch_server --model-path Qwen\u002FQwen3-30B-A3B-Instruct-2507 --port 30000 --context-length 262144\n```\n\nFor Qwen3-Thinking-2507,\n```shell\npython -m sglang.launch_server --model-path Qwen\u002FQwen3-30B-A3B-Thinking-2507 --port 30000 --context-length 262144 --reasoning-parser deepseek-r1\n```\n\nFor Qwen3, it is\n```shell\npython -m sglang.launch_server --model-path Qwen\u002FQwen3-8B --port 30000 --context-length 131072 --reasoning-parser qwen3\n```\nAn OpenAI-compatible API will be available at `http:\u002F\u002Flocalhost:30000\u002Fv1`.\n\n> [!Note]\n> Due to the preprocessing of API requests in SGLang, which drops all `reasoning_content` fields, the quality of **multi-step tool use with Qwen3 thinking models** may be suboptimal, which requires the existence of the related thinking content. While the fixes are being worked on, as a workdaround, we recommend passing the content as it is, without extracting thinking content, and the chat template will correctly handle the processing.\n\n\n### vLLM\n\n[vLLM](https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fvllm) is a high-throughput and memory-efficient inference and serving engine for LLMs.\n`vllm>=0.9.0` is recommended.\n\nFor Qwen3-Instruct-2507, \n```shell\nvllm serve Qwen\u002FQwen3-30B-A3B-Instruct-2507 --port 8000 --max-model-len 262144\n```\n\nFor Qwen3-Thinking-2507,\n```shell\nvllm serve Qwen\u002FQwen3-30B-A3B-Thinking-2507 --port 8000 --max-model-len 262144 --enable-reasoning --reasoning-parser deepseek_r1\n```\n\nFor Qwen3, it is\n```shell\nvllm serve Qwen\u002FQwen3-8B --port 8000 --max-model-len 131072 --enable-reasoning --reasoning-parser qwen3\n```\nAn OpenAI-compatible API will be available at `http:\u002F\u002Flocalhost:8000\u002Fv1`.\n\n> [!Note]\n> Due to the preprocessing of API requests in vLLM, which drops all `reasoning_content` fields, the quality of **multi-step tool use with Qwen3 thinking models** may be suboptimal, which requires the existence of the related thinking content. While the fixes are being worked on, as a workdaround, we recommend passing the content as it is, without extracting thinking content, and the chat template will correctly handle the processing.\n\n### TensorRT-LLM\n\n[TensorRT-LLM](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FTensorRT-LLM) is an open-source LLM inference engine from NVIDIA, which provides optimizations including custom attention kernels, quantization and more on NVIDIA GPUs. Qwen3 is supported in its re-architected [PyTorch backend](https:\u002F\u002Fnvidia.github.io\u002FTensorRT-LLM\u002Ftorch.html). `tensorrt_llm>=0.20.0rc3` is recommended. Please refer to the [README](https:\u002F\u002Fgithub.com\u002FNVIDIA\u002FTensorRT-LLM\u002Fblob\u002Fmain\u002Fexamples\u002Fmodels\u002Fcore\u002Fqwen\u002FREADME.md#qwen3) page for more details.\n\n```shell\ntrtllm-serve Qwen\u002FQwen3-8B --host localhost --port 8000 --backend pytorch\n```\nAn OpenAI-compatible API will be available at `http:\u002F\u002Flocalhost:8000\u002Fv1`.\n\n### MindIE\n\nFor deployment on Ascend NPUs, please visit [Modelers](https:\u002F\u002Fmodelers.cn\u002F) and search for Qwen3.\n\n\u003C!-- \n### OpenLLM\n\n[OpenLLM](https:\u002F\u002Fgithub.com\u002Fbentoml\u002FOpenLLM) allows you to easily run Qwen2.5 as OpenAI-compatible APIs. You can start a model server using `openllm serve`. For example:\n\n```bash\nopenllm serve qwen2.5:7b\n```\n\nThe server is active at `http:\u002F\u002Flocalhost:3000\u002F`, providing OpenAI-compatible APIs. You can create an OpenAI client to call its chat API. For more information, refer to [our documentation](https:\u002F\u002Fqwen.readthedocs.io\u002Fen\u002Flatest\u002Fdeployment\u002Fopenllm.html). -->\n\n\n## Build with Qwen3\n\n### Tool Use\n\nFor tool use capabilities, we recommend taking a look at [Qwen-Agent](https:\u002F\u002Fgithub.com\u002FQwenLM\u002FQwen-Agent), which provides a wrapper around these APIs to support tool use or function calling with MCP support.\nTool use with Qwen3 can also be conducted with SGLang, vLLM, Transformers, llama.cpp, Ollama, etc.\nFollow guides in our documentation to see how to enable the support.\n\n\n### Finetuning\n\nWe advise you to use training frameworks, including [Axolotl](https:\u002F\u002Fgithub.com\u002FOpenAccess-AI-Collective\u002Faxolotl), [UnSloth](https:\u002F\u002Fgithub.com\u002Funslothai\u002Funsloth), [Swift](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002Fswift), [Llama-Factory](https:\u002F\u002Fgithub.com\u002Fhiyouga\u002FLLaMA-Factory), etc., to finetune your models with SFT, DPO, GRPO, etc.\n\n\n## License Agreement\n\nAll our open-weight models are licensed under Apache 2.0. \nYou can find the license files in the respective Hugging Face repositories.\n\n## Citation\n\nIf you find our work helpful, feel free to give us a cite.\n\n```bibtex\n@article{qwen3,\n    title={Qwen3 Technical Report}, \n    author={An Yang and Anfeng Li and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Gao and Chengen Huang and Chenxu Lv and Chujie Zheng and Dayiheng Liu and Fan Zhou and Fei Huang and Feng Hu and Hao Ge and Haoran Wei and Huan Lin and Jialong Tang and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jing Zhou and Jingren Zhou and Junyang Lin and Kai Dang and Keqin Bao and Kexin Yang and Le Yu and Lianghao Deng and Mei Li and Mingfeng Xue and Mingze Li and Pei Zhang and Peng Wang and Qin Zhu and Rui Men and Ruize Gao and Shixuan Liu and Shuang Luo and Tianhao Li and Tianyi Tang and Wenbiao Yin and Xingzhang Ren and Xinyu Wang and Xinyu Zhang and Xuancheng Ren and Yang Fan and Yang Su and Yichang Zhang and Yinger Zhang and Yu Wan and Yuqiong Liu and Zekun Wang and Zeyu Cui and Zhenru Zhang and Zhipeng Zhou and Zihan Qiu},\n    journal = {arXiv preprint arXiv:2505.09388},\n    year={2025}\n}\n\n@article{qwen2.5,\n    title   = {Qwen2.5 Technical Report}, \n    author  = {An Yang and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chengyuan Li and Dayiheng Liu and Fei Huang and Haoran Wei and Huan Lin and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Yang and Jiaxi Yang and Jingren Zhou and Junyang Lin and Kai Dang and Keming Lu and Keqin Bao and Kexin Yang and Le Yu and Mei Li and Mingfeng Xue and Pei Zhang and Qin Zhu and Rui Men and Runji Lin and Tianhao Li and Tingyu Xia and Xingzhang Ren and Xuancheng Ren and Yang Fan and Yang Su and Yichang Zhang and Yu Wan and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zihan Qiu},\n    journal = {arXiv preprint arXiv:2412.15115},\n    year    = {2024}\n}\n\n@article{qwen2,\n    title   = {Qwen2 Technical Report}, \n    author  = {An Yang and Baosong Yang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Zhou and Chengpeng Li and Chengyuan Li and Dayiheng Liu and Fei Huang and Guanting Dong and Haoran Wei and Huan Lin and Jialong Tang and Jialin Wang and Jian Yang and Jianhong Tu and Jianwei Zhang and Jianxin Ma and Jin Xu and Jingren Zhou and Jinze Bai and Jinzheng He and Junyang Lin and Kai Dang and Keming Lu and Keqin Chen and Kexin Yang and Mei Li and Mingfeng Xue and Na Ni and Pei Zhang and Peng Wang and Ru Peng and Rui Men and Ruize Gao and Runji Lin and Shijie Wang and Shuai Bai and Sinan Tan and Tianhang Zhu and Tianhao Li and Tianyu Liu and Wenbin Ge and Xiaodong Deng and Xiaohuan Zhou and Xingzhang Ren and Xinyu Zhang and Xipin Wei and Xuancheng Ren and Yang Fan and Yang Yao and Yichang Zhang and Yu Wan and Yunfei Chu and Yuqiong Liu and Zeyu Cui and Zhenru Zhang and Zhihao Fan},\n    journal = {arXiv preprint arXiv:2407.10671},\n    year    = {2024}\n}\n```\n\n## Contact Us\nIf you are interested to leave a message to either our research team or product team, join our [Discord](https:\u002F\u002Fdiscord.gg\u002Fz3GAxXZ9Ce) or [WeChat groups](assets\u002Fwechat.png)!\n","Qwen3是由阿里云Qwen团队开发的大型语言模型系列。其核心功能包括指令执行、逻辑推理、文本理解、数学计算、科学知识、编程及工具使用等方面的显著提升，同时在多语言长尾知识覆盖上也取得了实质性进展，并且在主观和开放式任务中更好地符合用户偏好。该模型适合需要高质量自然语言处理能力的应用场景，例如智能客服、内容生成、代码辅助等。此外，Qwen3支持多种部署方式，包括本地运行（CPU\u002FGPU）、大规模推理部署以及量化处理，为不同需求提供了灵活的选择。",2,"2026-06-11 02:48:42","top_language"]