[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-10770":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":17,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":23,"hasPages":21,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":33,"readmeContent":34,"aiSummary":35,"trendingCount":16,"starSnapshotCount":16,"syncStatus":36,"lastSyncTime":37,"discoverSource":38},10770,"Ovis","AIDC-AI\u002FOvis","AIDC-AI","A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.","https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis2.5-9B",null,"Python",1452,83,19,79,0,1,7,54.97,"Apache License 2.0",false,"main",true,[25,26,27,28,29,30,31,32],"chatbot","llama3","multimodal","multimodal-large-language-models","multimodality","qwen","vision-language-learning","vision-language-model","2026-06-12 04:00:52","# Ovis\n\u003Cdiv align=\"center\">\n  \u003Cimg src=docs\u002Fovis_logo.png width=\"30%\"\u002F>\n\u003C\u002Fdiv>\n\u003Cbr>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2508.11737\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F📖_Technical_Report-Ovis2.5-b31b1b.svg\" alt=\"technical report\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FAIDC-AI\u002FOvis2.5-9B\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🎨_HF_Spaces-AIDC--AI\u002FOvis2.5--9B-lightblack\" alt=\"demo\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fhuggingface.co\u002Fcollections\u002FAIDC-AI\u002Fovis25-689ec1474633b2aab8809335\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🤗_Models-AIDC--AI\u002FOvis2.5-yellow\" alt=\"models\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n## Introduction\n\nOvis (Open VISion) is a novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.\n\n\u003Cdiv style=\"text-align: center;\">\n  \u003Cimg style=\"max-width: 100%;\" src=\"docs\u002FOvis25_arch.png\" alt=\"Ovis Illustration\"\u002F>\n\u003C\u002Fdiv>\n\n## 🔥 We are hiring!\nWe are looking for both interns and full-time researchers to join our team, focusing on multimodal understanding, generation, reasoning, AI agents, and unified multimodal models. If you are interested in exploring these exciting areas, please reach out to us at qingguo.cqg@alibaba-inc.com.\n\n## Release\n- [25\u002F08\u002F15] 🔥 Launch of [Ovis2.5-2B\u002F9B](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis2.5-9B), featuring native-resolution visual perception, enhanced reflective reasoning (*thinking mode*), and leading performance across STEM, chart analysis, grounding, and video understanding.\n- [25\u002F03\u002F25] 🔥 Announcing quantized versions of Ovis2 series, covering [Ovis2-2\u002F4\u002F8\u002F16\u002F34B](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis2-34B-GPTQ-Int4)!\n- [25\u002F01\u002F26] 🔥 Launch of [Ovis2-1\u002F2\u002F4\u002F8\u002F16\u002F34B](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis2-34B), the latest version of Ovis models, featuring breakthrough small-model performance, enhanced reasoning capabilities, advanced video and multi-image processing, expanded multilingual OCR support, and improved high-resolution image handling.\n- [24\u002F11\u002F26] 🔥 Announcing [Ovis1.6-Gemma2-27B](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis1.6-Gemma2-27B)!\n- [24\u002F11\u002F04] 🔥 Announcing quantized versions of Ovis1.6: [Ovis1.6-Gemma2-9B-GPTQ-Int4](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis1.6-Gemma2-9B-GPTQ-Int4) and [Ovis1.6-Llama3.2-3B-GPTQ-Int4](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis1.6-Llama3.2-3B-GPTQ-Int4)!\n- [24\u002F10\u002F22] 🔥 Announcing Ovis1.6-Llama3.2-3B ([Model](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis1.6-Llama3.2-3B), [Demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FAIDC-AI\u002FOvis1.6-Llama3.2-3B))!\n- [24\u002F09\u002F19] 🔥 Announcing Ovis1.6-Gemma2-9B ([Model](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis1.6-Gemma2-9B), [Demo](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FAIDC-AI\u002FOvis1.6-Gemma2-9B))! This release further enhances high-resolution image processing, is trained on a larger, more diverse, and higher-quality dataset, and refines the training process with DPO training following instruction-tuning.\n- [24\u002F07\u002F24] 🔥 Introducing Ovis1.5, featuring improved high-resolution image processing and optimized training data for enhanced performance.\n- [24\u002F06\u002F14] 🔥 Launch of Ovis1.0, the inaugural version of the Ovis model.\n\n## Contents\n- [Ovis: Structural Embedding Alignment for Multimodal Large Language Model](#ovis-structural-embedding-alignment-for-multimodal-large-language-model)\n  - [Release](#release)\n  - [Contents](#contents)\n  - [Model](#model)\n  - [Performance](#performance)\n  - [Install](#install)\n  - [Inference](#inference)\n  - [Model Fine-tuning](#model-fine-tuning)\n  - [Citation](#citation)\n  - [Team](#team)\n  - [License](#license)\n  - [Disclaimer](#disclaimer)\n\n## Model\nOvis can be instantiated with popular LLMs. We provide the following Ovis MLLMs:\n\n| Ovis MLLMs |           ViT           |          LLM          |                      Model Weights                      |                           Demo                           |\n|:-----------|:-----------------------:|:---------------------:|:-------------------------------------------------------:|:--------------------------------------------------------:|\n| Ovis2.5-2B   | siglip2-so400m-patch16-512 | Qwen3-1.7B | [Huggingface](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis2.5-2B)  | [Space](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FAIDC-AI\u002FOvis2.5-2B) |\n| Ovis2.5-9B   | siglip2-so400m-patch16-512  |  Qwen3-8B  | [Huggingface](https:\u002F\u002Fhuggingface.co\u002FAIDC-AI\u002FOvis2.5-9B)  | [Space](https:\u002F\u002Fhuggingface.co\u002Fspaces\u002FAIDC-AI\u002FOvis2.5-9B) |\n\n\n## Performance\nOvis2.5 demonstrates strong results on general multimodal benchmarks, complex chart analysis, and reasoning tasks, achieving leading performance among open-source models under 40B parameters.\n\n\n![performance-Ovis2_5](docs\u002Fperformance\u002FOvis2_5_performance.png)\n\n\n![OC-Ovis2_5](docs\u002Fperformance\u002FOvis2_5_OC.png)\n\n![REASON-Ovis2_5](docs\u002Fperformance\u002FOvis2_5_reason.png)\n\n## Install\nOvis has been tested with Python 3.10, Torch 2.4.0, Transformers 4.51.3, and DeepSpeed 0.15.4. For a comprehensive list of package dependencies, please consult the `requirements.txt` file.\n```bash\ngit clone git@github.com:AIDC-AI\u002FOvis.git\nconda create -n ovis python=3.10 -y\nconda activate ovis\ncd Ovis\npip install -r requirements.txt\npip install -e .\n```\n\nFor `vLLM`:\n\n```bash\npip install vllm==0.10.2 --extra-index-url https:\u002F\u002Fwheels.vllm.ai\u002F0.10.2\u002F\n```\n\n## Inference\n\nWe provide inference examples using both **transformers** and **vLLM**.\n\n### transformers\n\nIn `ovis\u002Fserve` we provide three example files:\n\n* **`ovis\u002Fserve\u002Finfer_think_demo.py`**  \n  Demonstrates how to enable the model’s *reflective reasoning* via  \n  `enable_thinking` and to control the reasoning phase length with `thinking_budget`.\n\n* **`ovis\u002Fserve\u002Finfer_basic_demo.py`**  \n  Provides inference examples for single-image, multi-image, video, and pure-text inputs.\n\n* **`ovis\u002Fserve\u002Fweb_ui.py`**\n  Provides a **Gradio-based Web UI** demo.\n  Example run:\n\n  ```bash\n  python ovis\u002Fserve\u002Fweb_ui.py --model-path AIDC-AI\u002FOvis2.5-9B --port 8001\n  ```\n\n### vLLM\n\nStart the vLLM server:\n\n```bash\nvllm serve AIDC-AI\u002FOvis2.5-9B \\\n     --trust-remote-code \\\n     --port 8000\n```\n\nCall the model using the **OpenAI Python SDK**:\n\n```python\nfrom openai import OpenAI\n\nopenai_api_key = \"EMPTY\"\nopenai_api_base = \"http:\u002F\u002Flocalhost:8000\u002Fv1\"\n\nclient = OpenAI(\n    api_key=openai_api_key,\n    base_url=openai_api_base,\n)\n\nchat_response = client.chat.completions.create(\n    model=\"AIDC-AI\u002FOvis2.5-9B\",\n    messages=[\n        {\n            \"role\": \"user\",\n            \"content\": [\n                {\n                    \"type\": \"image_url\",\n                    \"image_url\": {\n                        \"url\": \"https:\u002F\u002Fcdn-uploads.huggingface.co\u002Fproduction\u002Fuploads\u002F637aebed7ce76c3b834cea37\u002Fkh-1dhZRAduP-P4SkIhXr.png\"\n                    },\n                },\n                {\"type\": \"text\", \"text\": \"Recognize the table content\"},\n            ],\n        },\n    ],    \n    extra_body={\n        \"chat_template_kwargs\": {\n            \"enable_thinking\": True,\n        },\n        \"mm_processor_kwargs\": {\n            \"images_kwargs\": {\n                \"min_pixels\": 1048576,   # 1024 * 1024\n                \"max_pixels\": 3211264    # 1792 * 1792\n            }\n        }\n    }\n)\n\nprint(\"Chat response:\\n\", chat_response.choices[0].message.content)\n```\n\n#### Explanation of `extra_body` parameters:\n\n* **`chat_template_kwargs.enable_thinking`**\n  Enables *thinking mode* (reflective reasoning).\n\n* **`mm_processor_kwargs.images_kwargs.min_pixels \u002F max_pixels`**\n  Controls the resolution range of input images (in total pixel count), balancing accuracy and GPU memory usage.\n\n\n## Model Fine-tuning\n\nOvis can be fine-tuned using either the provided training code in this repository or via [ms-swift](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002Fms-swift).\n\n\n### 1. Fine-tuning with in-repo code\n\n#### Data Format\n\nThe training dataset is stored as a **JSON list**, where each element corresponds to a single sample.\nExample dataset JSON:\n\n```jsonc\n[\n    {\n        \"id\": 1354,\n        \"image\": \"1354.png\",\n        \"conversations\": [\n            {\n                \"from\": \"human\",\n                \"value\": \"\u003Cimage>\\nIn the figure, the vertices of quadrilateral ABCD intersect square EFGH and divide its sides into segments with measures that have a ratio of 1:2. Find the ratio between the areas of ABCD and EFGH.\"\n            },\n            {\n                \"from\": \"gpt\",\n                \"value\": \"5:9\"\n            }\n        ]\n    }\n]\n```\n\n#### Dataset Information\n\nDatasets are referenced via **datainfo JSON files**, e.g. `ovis\u002Ftrain\u002Fdataset\u002Fovis2_5_sft_datainfo.json`:\n\n```json\n{\n    \"geometry3k_local\": {\n        \"meta_file\": \"path\u002Fto\u002Fgeometry3k_local.json\",\n        \"storage_type\": \"hybrid\",\n        \"data_format\": \"conversation\",\n        \"image_dir\": \"path\u002Fto\u002Fimages\u002F\"\n    }\n}\n```\n\n* `meta_file`: path to the converted dataset JSON file (a list of samples).\n* `storage_type`: usually set to `\"hybrid\"`.\n* `data_format`: usually set to `\"conversation\"`.\n* `image_dir`: directory path containing the referenced images.\n\n#### Training Script\n\nWe provide example training scripts under `scripts\u002F`.\nFor instance, to fine-tune Ovis2.5 with SFT:\n\n```bash\nbash scripts\u002Frun_ovis2_5_sft.sh\n```\n\nThis script configures the DeepSpeed engine, dataset paths, and model checkpoint initialization. Modify it to match your own dataset and environment.\n\n### 2. Fine-tuning with ms-swift\n\nAlternatively, Ovis models can be fine-tuned using [ms-swift](https:\u002F\u002Fgithub.com\u002Fmodelscope\u002Fms-swift), a flexible training framework for LLMs.\n\n\n## Citation\nIf you find Ovis useful, please cite the paper\n```\n@article{lu2025ovis25technicalreport,\n  title={Ovis2.5 Technical Report}, \n  author={Shiyin Lu and Yang Li and Yu Xia and Yuwei Hu and Shanshan Zhao and Yanqing Ma and Zhichao Wei and Yinglun Li and Lunhao Duan and Jianshan Zhao and Yuxuan Han and Haijun Li and Wanying Chen and Junke Tang and Chengkun Hou and Zhixing Du and Tianli Zhou and Wenjie Zhang and Huping Ding and Jiahe Li and Wen Li and Gui Hu and Yiliang Gu and Siran Yang and Jiamang Wang and Hailong Sun and Yibo Wang and Hui Sun and Jinlong Huang and Yuping He and Shengze Shi and Weihong Zhang and Guodong Zheng and Junpeng Jiang and Sensen Gao and Yi-Feng Wu and Sijia Chen and Yuhui Chen and Qing-Guo Chen and Zhao Xu and Weihua Luo and Kaifu Zhang},\n  year={2025},\n  journal={arXiv:2508.11737}\n}\n\n@article{lu2024ovis,\n  title={Ovis: Structural Embedding Alignment for Multimodal Large Language Model}, \n  author={Shiyin Lu and Yang Li and Qing-Guo Chen and Zhao Xu and Weihua Luo and Kaifu Zhang and Han-Jia Ye},\n  year={2024},\n  journal={arXiv:2405.20797}\n}\n```\n\n## Team\nThis work is a collaborative effort by the Alibaba Ovis team. We would also like to provide links to the following MLLM papers from our team:\n- [Parrot: Multilingual Visual Instruction Tuning](https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.02539)\n- [Wings: Learning Multimodal LLMs without Text-only Forgetting](https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.03496)\n\n## License\nThis project is licensed under the [Apache License, Version 2.0](https:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0.txt) (SPDX-License-Identifier: Apache-2.0).\n\n## Disclaimer\nWe used compliance-checking algorithms during the training process, to ensure the compliance of the trained model to the best of our ability. Due to the complexity of the data and the diversity of language model usage scenarios, we cannot guarantee that the model is completely free of copyright issues or improper content. If you believe anything infringes on your rights or generates improper content, please contact us, and we will promptly address the matter.\n","Ovis是一个新颖的多模态大型语言模型（MLLM）架构，旨在结构化对齐视觉和文本嵌入。该项目通过结合视觉与文本信息，提供了一种能够理解和生成跨模态内容的强大工具。其核心功能包括原生分辨率的视觉感知、增强的反思推理能力以及在STEM、图表分析、基础化和视频理解等方面的领先性能。Ovis适合需要处理复杂多媒体数据的应用场景，如教育辅助、科研支持、智能客服等，特别是在需要同时解析图像和文本信息的情况下表现尤为出色。基于Python开发，并采用Apache License 2.0开源许可协议，使得开发者可以轻松地集成到现有系统中或进行二次开发。",2,"2026-06-11 03:30:05","top_topic"]