[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71180":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":15,"stars30d":16,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":17,"rankGlobal":10,"rankLanguage":10,"license":18,"archived":19,"fork":19,"defaultBranch":20,"hasWiki":21,"hasPages":19,"topics":22,"createdAt":10,"pushedAt":10,"updatedAt":31,"readmeContent":32,"aiSummary":33,"trendingCount":15,"starSnapshotCount":15,"syncStatus":34,"lastSyncTime":35,"discoverSource":36},71180,"sparrow","katanaml\u002Fsparrow","katanaml","Structured data extraction, instruction calling and agentic workflows with ML, LLM and Vision LLM","https:\u002F\u002Fsparrow.katanaml.io",null,"Python",5160,518,58,0,5,64.65,"GNU General Public License v3.0",false,"main",true,[23,24,25,26,27,28,29,30],"computer-vision","gpt","huggingface-transformers","llm","machinelearning","nlp-machine-learning","rag","vllm","2026-06-12 04:00:59","# Sparrow\n\n[![PyPI - Python](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-v3.12+-blue.svg)](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow)\n[![GitHub Stars](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fstars\u002Fkatanaml\u002Fsparrow.svg)](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fstargazers)\n[![GitHub Issues](https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fissues\u002Fkatanaml\u002Fsparrow.svg)](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fissues)\n[![Current Version](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fversion-0.4.4-green.svg)](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow)\n[![License: GPL v3](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-GPLv3-blue.svg)](https:\u002F\u002Fwww.gnu.org\u002Flicenses\u002Fgpl-3.0)\n\n**Structured data extraction and instruction calling with ML, LLM and Vision LLM**\n\n\u003Cp align=\"center\">\n  \u003Cimg width=\"300\" height=\"300\" src=\"https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fsparrow_logo_5.png\">\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cstrong>🚀 \u003Ca href=\"https:\u002F\u002Fsparrow.katanaml.io\">Try Sparrow Online\u003C\u002Fa> | 📖 \u003Ca href=\"#-quickstart\">Quick Start\u003C\u002Fa> | 🛠️ \u003Ca href=\"#️-installation\">Installation\u003C\u002Fa> | 📚 \u003Ca href=\"#-examples\">Examples\u003C\u002Fa> | 🤖 \u003Ca href=\"#-sparrow-agent\">Agents\u003C\u002Fa>\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n---\n\n## 🌟 Sparrow\n\n[🚀 Try Sparrow Online](https:\u002F\u002Fsparrow.katanaml.io)\n\nProduction-ready structured data extraction powered by ML, LLMs & Vision LLMs.\nTurn invoices, receipts, statements, forms and images into clean structured data.\n\nSparrow is an **API-first platform** built for enterprise document intelligence. It provides RESTful APIs for structured data extraction, instruction processing, and multi-agent workflow orchestration — all running on your own infrastructure with no external API calls or cloud dependencies.\n\n**Platform capabilities:**\n- **Structured Extraction API**: Submit documents via REST and receive validated JSON — integrate directly into any backend or data pipeline\n- **Instruction Processing**: Beyond document extraction — text processing, validation, and decision making via the instruction inference API\n- **Agent Framework**: Orchestrate multi-step workflows with custom agents, visual monitoring via Prefect, and robust error handling\n- **Pluggable Pipelines**: Mix and match Vision LLM (Sparrow Parse), Text LLM (Sparrow Instructor), and Agent pipelines depending on the task\n- **Multiple Backends**: MLX on Apple Silicon, vLLM on NVIDIA, Ollama, Hugging Face — same API surface across all\n\n![Sparrow UI](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fsparrow_ui.png)\n\n### Sparrow UI Features\n\nThe web UI provides a visual interface on top of the same API:\n\n- **Drag & Drop**: Upload documents directly\n- **Real-time Processing**: See results instantly\n- **Data Query**: JSON based schema for data query\n- **Structured Output**: JSON structured output\n- **Result Annotation**: View bounding boxes\n\n## 📑 Table of Contents\n\n- [✨ Key Features](#-key-features)\n- [🏗️ Architecture](#️-architecture)\n- [🚀 Quickstart](#-quickstart)\n- [🛠️ Installation](#️-installation)\n- [📚 Examples](#-examples)\n- [💻 CLI Usage](#-cli-usage)\n- [🌐 API Usage](#-api-usage)\n- [🤖 Sparrow Agent](#-sparrow-agent)\n- [📊 Dashboard](#-dashboard)\n- [🔧 Pipeline Comparison](#-pipeline-comparison)\n- [⚡ Performance Tips](#-performance-tips)\n- [🔍 Troubleshooting](#-troubleshooting)\n- [⭐ Star History](#-star-history)\n- [📜 License](#-license)\n\n## ✨ Key Features\n\n🎯 **Universal Document Processing**: Handle invoices, receipts, forms, bank statements, tables    \n🔧 **Pluggable Architecture**: Mix and match different pipelines (Sparrow Parse, Instructor, Agents)  \n🖥️ **Multiple Backends**: MLX (Apple Silicon), Ollama, vLLM, Docker, Hugging Face Cloud GPU  \n📱 **Multi-format Support**: Images (PNG, JPG) and multi-page PDFs  \n🎨 **Schema Validation**: JSON schema-based extraction with automatic validation  \n🌐 **API-First Design**: RESTful APIs for easy integration  \n💬 **Instruction Calling**: Text processing, validation, decision making with GPT-OSS, Mistral, Qwen 3.6, etc.  \n📊 **Visual Monitoring**: Built-in dashboard and agent workflow tracking  \n🔒 **Enterprise Ready**: Rate limiting, usage analytics, commercial licensing available  \n🚀 **Local Vision LLMs**: Mistral, Qwen 3.6, DeepSeek OCR, dots.ocr, Gemma 4, etc.  \n\n## 🏗️ Architecture\n\n![Sparrow Architecture](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fsparrow_architecture.jpeg)\n\n### Core Components\n\n| Component | Purpose | Use Case |\n|-----------|---------|----------|\n| **[Sparrow ML LLM](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Ftree\u002Fmain\u002Fsparrow-ml\u002Fllm)** | Main API engine | Document processing pipelines |\n| **[Sparrow Parse](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Ftree\u002Fmain\u002Fsparrow-data\u002Fparse)** | Vision LLM library | Structured JSON extraction |\n| **[Sparrow Agents](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Ftree\u002Fmain\u002Fsparrow-ml\u002Fagents)** | Workflow orchestration | Complex multi-step processing |\n| **[Sparrow OCR](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Ftree\u002Fmain\u002Fsparrow-data\u002Focr)** | Text recognition | OCR preprocessing |\n| **[Sparrow UI](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Ftree\u002Fmain\u002Fsparrow-ui\u002F)** | Web interface | Interactive document processing |\n\n## 🚀 Quickstart\n\n### Prerequisites\n- **Python 3.12.10+** (use `pyenv` for version management)\n- **macOS** (for MLX backend) or **Linux\u002FWindows** (for other backends)\n- **GPU** (make sure GPU have enough memory to run selected Vision LLM)\n\n### 30-Second Setup\n\n```bash\n# 1. Install pyenv and Python 3.12.10\npyenv install 3.12.10\npyenv global 3.12.10\n\n# 2. Create virtual environment\npython -m venv .env_sparrow_parse\nsource .env_sparrow_parse\u002Fbin\u002Factivate  # Linux\u002FMac\n# or .env_sparrow_parse\\Scripts\\activate  # Windows\n\n# 3. Install Sparrow Parse pipeline\ngit clone https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow.git\ncd sparrow\u002Fsparrow-ml\u002Fllm\npip install -r requirements_sparrow_parse.txt\n\n# 4. For macOS: Install poppler for PDF processing\nbrew install poppler\n\n# 5. Start the API server\npython api.py\n```\n\nBefore running `pip install -r requirements_sparrow_parse.txt`, check your platform. If you are on macOS and want to run MLX backend, go to `requirements_sparrow_parse.txt` and make sure `sparrow-parse[mlx]` libary reference is defined. If you are running Sparrow on Linux\u002FWindows, make sure to use `sparrow-parse` library reference, this will skip MLX related libraries.\n\n### First Document Extraction\n\n```bash\n# Extract data from a bonds table\n.\u002Fsparrow.sh '[{\"instrument_name\":\"str\", \"valuation\":0}]' \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen2.5-VL-72B-Instruct-4bit \\\n  --file-path \"data\u002Fbonds_table.png\"\n```\n\n**Result:**\n```json\n{\n  \"data\": [\n    {\"instrument_name\": \"UNITS BLACKROCK...\", \"valuation\": 19049},\n    {\"instrument_name\": \"UNITS ISHARES...\", \"valuation\": 83488}\n  ],\n  \"valid\": \"true\"\n}\n```\n\nUse `--options mlx` for MLX backend, `--options ollama` for Ollama backend, `--options vllm` for vLLM backend. Make sure to provide correct Vision LLM model name, download model first separately with MLX, vLLM or Ollama.\n\n## 🛠️ Installation\n\n### Quick Setup\n\n```bash\n# 1. Clone repository\ngit clone https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow.git\ncd sparrow\n```\n\n📖 **For complete installation instructions**, see our [detailed environment setup guide](environment_setup.md).\n\n### Essential Steps Summary\n\n1. **Python Environment**: Install Python 3.12.10 using pyenv\n2. **Virtual Environments**: Create separate environments for different pipelines:\n   - `.env_sparrow_parse` - for Sparrow Parse (Vision LLM)\n   - `.env_instructor` - for Instructor (Text LLM) \n   - `.env_ocr` - for OCR service (optional)\n3. **System Dependencies**: Install poppler for PDF processing\n4. **Requirements**: Install pipeline-specific dependencies, for example:\n\n`pip install -r requirements_sparrow_parse.txt`\n\n### Platform-Specific Notes\n\n**macOS:**\n```bash\nbrew install poppler  # Required for PDF processing\n```\n\n**Ubuntu\u002FDebian:**\n```bash\nsudo apt-get install poppler-utils libpoppler-cpp-dev\n```\n\n**Apple Silicon**: MLX backend available for optimal performance  \n**NVIDIA\u002FAMD GPU**: Use vLLM or Ollama backend  \n**CPU Only**: Use smaller models or Hugging Face cloud backend  \n\n### Verification\n\n```bash\n# Test installation\npython api.py --port 8002\n# Visit http:\u002F\u002Flocalhost:8002\u002Fapi\u002Fv1\u002Fsparrow-llm\u002Fdocs\n```\n\n## 📚 Examples\n\n### 🏦 Bank Statement Processing\n\n![Bank Statement](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fbank_statement.png)\n\n```bash\n# Extract all data from bank statement\n.\u002Fsparrow.sh \"*\" \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen2.5-VL-72B-Instruct-4bit \\\n  --file-path \"data\u002Fbank_statement.pdf\"\n```\n\n\u003Cdetails>\n\u003Csummary>\u003Cstrong>📄 View Complete JSON Output\u003C\u002Fstrong>\u003C\u002Fsummary>\n\n```json\n{\n  \"bank\": \"First Platypus Bank\",\n  \"address\": \"1234 Kings St., New York, NY 12123\",\n  \"account_holder\": \"Mary G. Orta\",\n  \"account_number\": \"1234567890123\",\n  \"statement_date\": \"3\u002F1\u002F2022\",\n  \"period_covered\": \"2\u002F1\u002F2022 - 3\u002F1\u002F2022\",\n  \"account_summary\": {\n    \"balance_on_march_1\": \"$25,032.23\",\n    \"total_money_in\": \"$10,234.23\",\n    \"total_money_out\": \"$10,532.51\"\n  },\n  \"transactions\": [\n    {\n      \"date\": \"02\u002F01\",\n      \"description\": \"PGD EasyPay Debit\",\n      \"withdrawal\": \"203.24\",\n      \"deposit\": \"\",\n      \"balance\": \"22,098.23\"\n    },\n    {\n      \"date\": \"02\u002F02\",\n      \"description\": \"AB&B Online Payment*****\",\n      \"withdrawal\": \"71.23\",\n      \"deposit\": \"\",\n      \"balance\": \"22,027.00\"\n    },\n    {\n      \"date\": \"02\u002F04\",\n      \"description\": \"Check No. 2345\",\n      \"withdrawal\": \"\",\n      \"deposit\": \"450.00\",\n      \"balance\": \"22,477.00\"\n    },\n    {\n      \"date\": \"02\u002F05\",\n      \"description\": \"Payroll Direct Dep 23422342 Giants\",\n      \"withdrawal\": \"\",\n      \"deposit\": \"2,534.65\",\n      \"balance\": \"25,011.65\"\n    },\n    {\n      \"date\": \"02\u002F06\",\n      \"description\": \"Signature POS Debit - TJP\",\n      \"withdrawal\": \"84.50\",\n      \"deposit\": \"\",\n      \"balance\": \"24,927.15\"\n    },\n    {\n      \"date\": \"02\u002F07\",\n      \"description\": \"Check No. 234\",\n      \"withdrawal\": \"1,400.00\",\n      \"deposit\": \"\",\n      \"balance\": \"23,527.15\"\n    },\n    {\n      \"date\": \"02\u002F08\",\n      \"description\": \"Check No. 342\",\n      \"withdrawal\": \"\",\n      \"deposit\": \"25.00\",\n      \"balance\": \"23,552.15\"\n    },\n    {\n      \"date\": \"02\u002F09\",\n      \"description\": \"FPB AutoPay***** Credit Card\",\n      \"withdrawal\": \"456.02\",\n      \"deposit\": \"\",\n      \"balance\": \"23,096.13\"\n    },\n    {\n      \"date\": \"02\u002F08\",\n      \"description\": \"Check No. 123\",\n      \"withdrawal\": \"\",\n      \"deposit\": \"25.00\",\n      \"balance\": \"23,552.15\"\n    },\n    {\n      \"date\": \"02\u002F09\",\n      \"description\": \"FPB AutoPay***** Credit Card\",\n      \"withdrawal\": \"156.02\",\n      \"deposit\": \"\",\n      \"balance\": \"23,096.13\"\n    },\n    {\n      \"date\": \"02\u002F08\",\n      \"description\": \"Cash Deposit\",\n      \"withdrawal\": \"\",\n      \"deposit\": \"25.00\",\n      \"balance\": \"23,552.15\"\n    }\n  ],\n  \"valid\": \"true\"\n}\n```\n\n\u003C\u002Fdetails>\n\n### 📊 Financial Tables\n\n![Bonds Table](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fbonds_table.png)\n\n```bash\n# Extract structured data from financial table\n.\u002Fsparrow.sh '[{\"instrument_name\":\"str\", \"valuation\":0}]' \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen2.5-VL-72B-Instruct-4bit \\\n  --file-path \"data\u002Fbonds_table.png\"\n```\n\n\u003Cdetails>\n\u003Csummary>\u003Cstrong>📄 View JSON Output\u003C\u002Fstrong>\u003C\u002Fsummary>\n\n```json\n{\n  \"data\": [\n    {\n      \"instrument_name\": \"UNITS BLACKROCK FIX INC DUB FDS PLC ISHS EUR INV GRD CP BD IDX\u002FINST\u002FE\",\n      \"valuation\": 19049\n    },\n    {\n      \"instrument_name\": \"UNITS ISHARES III PLC CORE EUR GOVT BOND UCITS ETF\u002FEUR\",\n      \"valuation\": 83488\n    },\n    {\n      \"instrument_name\": \"UNITS ISHARES III PLC EUR CORP BOND 1-5YR UCITS ETF\u002FEUR\",\n      \"valuation\": 213030\n    },\n    {\n      \"instrument_name\": \"UNIT ISHARES VI PLC\u002FJP MORGAN USD E BOND EUR HED UCITS ETF DIST\u002FHDGD\u002F\",\n      \"valuation\": 32774\n    },\n    {\n      \"instrument_name\": \"UNITS XTRACKERS II SICAV\u002FEUR HY CORP BOND UCITS ETF\u002F-1D-\u002FDISTR.\",\n      \"valuation\": 23643\n    }\n  ],\n  \"valid\": \"true\"\n}\n```\n\n\u003C\u002Fdetails>\n\n### 🧾 Invoice Processing\n\n```bash\n# Extract invoice with cropping for better accuracy\n.\u002Fsparrow.sh \"*\" \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen2.5-VL-72B-Instruct-4bit \\\n  --crop-size 60 \\\n  --file-path \"data\u002Finvoice.pdf\"\n```\n\n\u003Cdetails>\n\u003Csummary>\u003Cstrong>📄 View Complete JSON Output\u003C\u002Fstrong>\u003C\u002Fsummary>\n\n```json\n{\n  \"invoice_number\": \"61356291\",\n  \"date_of_issue\": \"09\u002F06\u002F2012\",\n  \"seller\": {\n    \"name\": \"Chapman, Kim and Green\",\n    \"address\": \"64731 James Branch, Smithmouth, NC 26872\",\n    \"tax_id\": \"949-84-9105\",\n    \"iban\": \"GB50ACIE59715038217063\"\n  },\n  \"client\": {\n    \"name\": \"Rodriguez-Stevens\",\n    \"address\": \"2280 Angela Plain, Hortonshire, MS 93248\",\n    \"tax_id\": \"939-98-8477\"\n  },\n  \"items\": [\n    {\n      \"description\": \"Wine Glasses Goblets Pair Clear\",\n      \"quantity\": 5,\n      \"unit\": \"each\",\n      \"net_price\": 12.0,\n      \"net_worth\": 60.0,\n      \"vat_percentage\": 10,\n      \"gross_worth\": 66.0\n    },\n    {\n      \"description\": \"With Hooks Stemware Storage Multiple Uses Iron Wine Rack Hanging\",\n      \"quantity\": 4,\n      \"unit\": \"each\", \n      \"net_price\": 28.08,\n      \"net_worth\": 112.32,\n      \"vat_percentage\": 10,\n      \"gross_worth\": 123.55\n    },\n    {\n      \"description\": \"Replacement Corkscrew Parts Spiral Worm Wine Opener Bottle Houdini\",\n      \"quantity\": 1,\n      \"unit\": \"each\",\n      \"net_price\": 7.5,\n      \"net_worth\": 7.5,\n      \"vat_percentage\": 10,\n      \"gross_worth\": 8.25\n    },\n    {\n      \"description\": \"HOME ESSENTIALS GRADIENT STEMLESS WINE GLASSES SET OF 4 20 FL OZ (591 ml) NEW\",\n      \"quantity\": 1,\n      \"unit\": \"each\",\n      \"net_price\": 12.99,\n      \"net_worth\": 12.99,\n      \"vat_percentage\": 10,\n      \"gross_worth\": 14.29\n    }\n  ],\n  \"summary\": {\n    \"total_net_worth\": 192.81,\n    \"total_vat\": 19.28,\n    \"total_gross_worth\": 212.09\n  }\n}\n```\n\n\u003C\u002Fdetails>\n\n### 📄 Multi-page PDF Processing\n\n```bash\n# Process multi-page PDF with structured output per page\n.\u002Fsparrow.sh '{\"table\": [{\"description\": \"str\", \"latest_amount\": 0, \"previous_amount\": 0}]}' \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen2.5-VL-72B-Instruct-4bit \\\n  --file-path \"data\u002Ffinancial_report.pdf\" \\\n  --debug-dir \"debug\u002F\"\n```\n\n\u003Cdetails>\n\u003Csummary>\u003Cstrong>📄 View JSON Output\u003C\u002Fstrong>\u003C\u002Fsummary>\n\n```json\n[\n    {\n        \"table\": [\n            {\n                \"description\": \"Revenues\",\n                \"latest_amount\": 12453,\n                \"previous_amount\": 11445\n            },\n            {\n                \"description\": \"Operating expenses\",\n                \"latest_amount\": 9157,\n                \"previous_amount\": 8822\n            }\n        ],\n        \"valid\": \"true\",\n        \"page\": 1\n    },\n    {\n        \"table\": [\n            {\n                \"description\": \"Revenues\", \n                \"latest_amount\": 12453,\n                \"previous_amount\": 11445\n            },\n            {\n                \"description\": \"Operating expenses\",\n                \"latest_amount\": 9157,\n                \"previous_amount\": 8822\n            }\n        ],\n        \"valid\": \"true\",\n        \"page\": 2\n    }\n]\n```\n\n\u003C\u002Fdetails>\n\n### 💬 Text Instruction Processing\n\n```bash\n# Instruction-based processing\n.\u002Fsparrow.sh \"instruction: do arithmetic operation, payload: 2+2=\" \\\n  --pipeline \"sparrow-instructor\" \\\n  --options mlx \\\n  --options lmstudio-community\u002FMistral-Small-3.2-24B-Instruct-2506-8bit\n\n# Instruction processing with document input\n.\u002Fsparrow.sh \"check if business entity Chapman, Kim and Green is invoice issuing party\" \n  --pipeline \"sparrow-parse\" \n  --instruction \n  --options mlx --options lmstudio-community\u002FMistral-Small-3.2-24B-Instruct-2506-8bit \n  --file-path \"invoice_1.jpg\"\n```\n\n\n**JSON Output:**\n```\nThe result of 2 + 2 is:\n\n4\n```\n\n\n### 📈 Stock Data Function Calling\n\n```bash\n# Function calling example\n.\u002Fsparrow.sh assistant --pipeline \"stocks\" --query \"Oracle\"\n```\n\n**JSON Output:**\n```json\n{\n  \"company\": \"Oracle Corporation\",\n  \"ticker\": \"ORCL\"\n}\n```\n\n**Additional Output:**\n```\nThe stock price of the Oracle Corporation is 186.3699951171875. USD\n```\n\n### 🧾 Table\u002FForm Processing with Sparrow Template\n\n```bash\n.\u002Fsparrow.sh \"*\" --pipeline \"sparrow-parse\" \\\n  --debug --table --table-template \"sparrow_generic_table\" \\\n  --options mlx --options mlx-community\u002FQwen3.6-35B-A3B-8bit \\\n  --options mlx --options mlx-community\u002Fdots.ocr-bf16 --file-path \"data\u002Fwell_report.jpg\"   \n```\n\n### 🧾 Query Hints\n\n```bash\n.\u002Fsparrow.sh \"[{\\\"instrument_name\\\":\\\"str\\\", \\\"valuation\\\":\\\"int\\\"}]\" \\\n  --pipeline \"sparrow-parse\" --debug --options mlx \\\n  --options mlx-community\u002Fgemma-4-31b-it-8bit \\\n  --file-path \"data\u002Fbonds_table.png\" --hints-file-path \"data\u002Fllm_hints_eu.json\"  \n```\n\n## 💻 CLI Usage\n\n### Basic Syntax\n\n```bash\n.\u002Fsparrow.sh \"\u003CJSON_SCHEMA>\" --pipeline \"\u003CPIPELINE>\" [OPTIONS] --file-path \"\u003CFILE>\"\n```\n\n### Command Line Arguments\n\n| Argument | Type | Description | Example |\n|----------|------|-------------|---------|\n| `query` | JSON\u002FString | Schema or instruction | `'[{\"field\":\"str\"}]'` |\n| `--pipeline` | String | Pipeline to use | `sparrow-parse` |\n| `--file-path` | Path | Input document | `data\u002Finvoice.pdf` |\n| `--hints-file-path` | Path | Query hints | `data\u002Fhints.json` |\n| `--options` | String | Backend configuration | `mlx,model-name` |\n| `--instruction` | Boolean | Sparrow query will be used as instruction | `--instruction` |\n| `--validation` | Boolean | Sparrow query will be used for field validation | `--validation` |\n| `--markdown` | Boolean | Markdown pre-processing | `--markdown` |\n| `--ocr` | Boolean | Experimental functionality | `--ocr` |\n| `--table` | Boolean | Experimental functionality | `--table` |\n| `--table-template` | String | Experimental functionality | `--name` |\n| `--crop-size` | Integer | Border cropping pixels | `60` |\n| `--page-type` | String | Page classification | `financial_table` \n| `--debug` | Boolean | Enable debug mode | `--debug` |\n| `--debug-dir` | Path | Debug output folder | `.\u002Fdebug\u002F` |\n\n### Pipeline Options\n\n#### Sparrow Parse (Vision LLM)\n```bash\n# MLX Backend (Apple Silicon)\n.\u002Fsparrow.sh '[{\"instrument_name\":\"str\", \"valuation\":0}]' \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen3.6-35B-A3B-8bit \\\n  --file-path \"data\u002Fbonds_table.png\"\n\n# Hugging Face Cloud GPU\n--options huggingface --options your-space\u002Fmodel-name\n\n# Additional flags\n--options tables_only        # Extract only tables\n--options validation_off     # Disable schema validation\n--options apply_annotation   # Include bounding boxes\n--page-type financial_table  # Classify page type\n```\n\n#### Sparrow Instructor (Text LLM)\n```bash\n# Instruction-based processing\n.\u002Fsparrow.sh \"instruction: do arithmetic operation, payload: 2+2=\" \\\n  --pipeline \"sparrow-instructor\" \\\n  --options mlx \\\n  --options lmstudio-community\u002FMistral-Small-3.2-24B-Instruct-2506-8bit\n```\n\n### Advanced Examples\n\n```bash\n# Multi-page PDF with page classification\n.\u002Fsparrow.sh \"*\" \\\n  --page-type invoice \\\n  --page-type table \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen3.6-35B-A3B-8bit \\\n  --file-path \"multi_page.pdf\"\n\n# Handle missing fields with null values\n.\u002Fsparrow.sh '[{\"required_field\":\"str\", \"optional_field\":\"str or null\"}]' \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen3.6-35B-A3B-8bit \\\n  --file-path \"document.png\"\n\n# Table extraction with cropping\n.\u002Fsparrow.sh '*' \\\n  --pipeline \"sparrow-parse\" \\\n  --options mlx \\\n  --options mlx-community\u002FQwen3.6-35B-A3B-8bit \\\n  --options tables_only \\\n  --crop-size 100 \\\n  --file-path \"scan.pdf\"\n\n# Instruction execution\n.\u002Fsparrow.sh \"check if business entity Chapman, Kim and Green is invoice issuing party\" \n  --pipeline \"sparrow-parse\" \n  --instruction \n  --options mlx --options lmstudio-community\u002FMistral-Small-3.2-24B-Instruct-2506-8bit \n  --file-path \"invoice_1.jpg\"\n\n# Field validation\n.\u002Fsparrow.sh \"tax_id,shipment_code,total_gross_worth\" \n  --pipeline \"sparrow-parse\" \n  --validation \n  --options mlx --options lmstudio-community\u002FMistral-Small-3.2-24B-Instruct-2506-8bit \n  --file-path \"invoice_1.jpg\"\n\n{\n  \"tax_id\": true,\n  \"shipment_code\": false,\n  \"total_gross_worth\": true\n}\n```\n\n## 🌐 API Usage\n\n### Starting the Server\n\n```bash\n# Default port (8002)\npython api.py\n\n# Custom port\npython api.py --port 8001\n\n# Multiple instances\npython api.py --port 8002 &  # Sparrow Parse\npython api.py --port 8003 &  # Instructor\n```\n\n### API Endpoints\n\n#### Document Extraction (`\u002Finference`)\n\n```bash\ncurl -X POST 'http:\u002F\u002Flocalhost:8002\u002Fapi\u002Fv1\u002Fsparrow-llm\u002Finference' \\\n  -H 'Content-Type: multipart\u002Fform-data' \\\n  -F 'query=[{\"field_name\":\"str\", \"amount\":0}]' \\\n  -F 'pipeline=sparrow-parse' \\\n  -F 'options=mlx,mlx-community\u002FQwen2.5-VL-72B-Instruct-4bit' \\\n  -F 'file=@document.pdf'\n```\n\n#### Text Instructions (`\u002Finstruction-inference`)\n\n```bash\ncurl -X POST 'http:\u002F\u002Flocalhost:8002\u002Fapi\u002Fv1\u002Fsparrow-llm\u002Finstruction-inference' \\\n  -H 'Content-Type: application\u002Fx-www-form-urlencoded' \\\n  -d 'query=instruction: analyze data, payload: {...}' \\\n  -d 'pipeline=sparrow-instructor' \\\n  -d 'options=mlx,mlx-community\u002FQwen3.6-35B-A3B-8bit'\n```\n\n### API Documentation\n\nVisit `http:\u002F\u002Flocalhost:8002\u002Fapi\u002Fv1\u002Fsparrow-llm\u002Fdocs` for interactive Swagger documentation.\n\n![API Documentation](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fsparrow_api.png)\n\n## 🤖 Sparrow Agent\n\n![Sparrow Agents](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fsparrow_agent.png)\n\nOrchestrate complex document processing workflows with visual monitoring powered by Prefect.\n\n### Features\n- **Multi-step Workflows**: Chain classification, extraction, and validation\n- **Visual Monitoring**: Real-time pipeline tracking\n- **Error Handling**: Robust failure recovery\n- **Extensible**: Custom agents for specific use cases\n\n### Usage\n\n```bash\n# Start agent server\ncd sparrow-ml\u002Fagents\npython api.py --port 8001\n\n# Process medical prescriptions\ncurl -X POST 'http:\u002F\u002Flocalhost:8001\u002Fapi\u002Fv1\u002Fsparrow-agents\u002Fexecute\u002Ffile' \\\n  -F 'agent_name=medical_prescriptions' \\\n  -F 'extraction_params={\"sparrow_key\":\"123456\"}' \\\n  -F 'file=@prescription.pdf'\n```\n\n## 📊 Dashboard\n\nBuilt-in analytics and monitoring dashboard at [sparrow.katanaml.io](https:\u002F\u002Fsparrow.katanaml.io). This is part of Sparrow UI, requires local Oracle Database 23ai Free. \n\n![Dashboard](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fblob\u002Fmain\u002Fsparrow-ui\u002Fassets\u002Fsparrow_ui_3.png)\n\n### Features\n- **Usage Analytics**: Track API calls, success rates, performance\n- **Geographic Distribution**: See usage by country\n- **Model Performance**: Compare different model performance\n- **Real-time Monitoring**: Live processing statistics\n\n## 🔧 Pipeline Comparison\n\n| Feature | Sparrow Parse | Sparrow Instructor | Sparrow Agents |\n|---------|---------------|-------------------|----------------|\n| **Input** | Documents + JSON schema | Text instructions | Complex workflows |\n| **Output** | Structured JSON | Free-form text | Multi-step results |\n| **Use Cases** | Data extraction, forms | Summarization, analysis | Enterprise workflows |\n| **Validation** | Schema-based | Manual | Custom rules |\n| **Complexity** | Simple | Medium | High |\n| **Best For** | Invoices, tables, forms | Text processing | Multi-document flows |\n\n### When to Use What\n\n**Sparrow Parse**: Use for structured data extraction from documents  \n**Sparrow Instructor**: Use for text analysis, summarization, Q&A  \n**Sparrow Agents**: Use for complex multi-step document processing workflows  \n\n\n## ⚡ Performance Tips\n\n### Hardware Optimization\n\n**Apple Silicon (MLX)**\n- ✅ Best performance with unified memory\n- ✅ Models: Mistral Small 3.2 24B, Qwen3.6 27B Dense, Qwen3.6 35B MoE, Gemma 4 31B Dense, Gemma 4 26B MoE\n- ⚠️ Requires macOS with Apple Silicon\n\n**NVIDIA GPU (vLLM)**\n- ✅ Production inference via vLLM backend\n- ✅ Models: Mistral Small 3.2 24B full precision (primary), dots.ocr for large table pipelines\n- ✅ Recommended: 96GB VRAM for full precision models\n- ⚠️ Requires CUDA setup\n\n**CPU Only**\n- ⚠️ Significantly slower\n- ✅ Use smaller models (7B parameters max)\n- ✅ Consider Hugging Face cloud backend\n\n### Table Extraction\n\nFor large or complex tables, use the dots.ocr → Sparrow Templates pipeline instead of Vision LLM direct extraction:\n\n```bash\n.\u002Fsparrow.sh \"*\" --pipeline \"sparrow-parse\" \\\n  --debug --table --table-template \"sparrow_generic_table\" \\\n  --options mlx --options mlx-community\u002FQwen3.6-35B-A3B-8bit \\\n  --options mlx --options mlx-community\u002Fdots.ocr-bf16 --file-path \"data\u002Fwell_report.jpg\"\n```\n\n- **dots.ocr**: Handles large tables with high accuracy via HTML intermediate output\n- **Sparrow Templates**: Maps extracted HTML table structure to JSON schema\n- Recommended for financial statements, multi-column invoices, and structured reports\n\n### Extraction Hints\n\nUse Sparrow hints to improve accuracy on complex documents — steer model attention to footers and fine print, disambiguate structurally similar fields (e.g., supplier vs. recipient VAT), normalize date and number formats, and resolve priority ordering for ambiguous fields:\n\n```bash\n.\u002Fsparrow.sh \"[{\\\"instrument_name\\\":\\\"str\\\", \\\"valuation\\\":\\\"int\\\"}]\" \\\n  --pipeline \"sparrow-parse\" --debug --options mlx \\\n  --options mlx-community\u002Fgemma-4-31b-it-8bit \\\n  --file-path \"data\u002Fbonds_table.png\" --hints-file-path \"data\u002Fllm_hints_eu.json\"\n```\n\n### Model Selection\n\n| Use Case | Recommended Model | Backend | Notes |\n|----------|------------------|---------|-------|\n| **Invoices \u002F Forms (EU)** | Mistral Small 3.2 24B | vLLM \u002F MLX | Primary production model |\n| **Invoices \u002F Forms (US)** | Gemma 4 31B Dense | MLX | Strong on English documents |\n| **Large Tables** | dots.ocr | vLLM | Via Sparrow Templates pipeline |\n| **Quick Testing** | Qwen3.6 27B Dense | MLX | Fast, good general accuracy |\n| **Low Memory** | Qwen3.6 35B MoE \u002F Gemma 4 26B MoE | MLX | Reduced memory footprint |\n\n## 🔍 Troubleshooting\n\n### Common Issues\n\n\u003Cdetails>\n\u003Csummary>🚫 Installation Problems\u003C\u002Fsummary>\n\n**Python Version Issues:**\n```bash\n# Verify Python version\npython --version  # Should be 3.12.10+\n\n# Fix with pyenv\npyenv install 3.12.10\npyenv global 3.12.10\n```\n\n**MLX Installation (Apple Silicon):**\n```bash\n# If MLX fails to install\npip install --upgrade pip\npip install mlx-vlm --no-cache-dir\n```\n\n```bash\n# If pip install command throws AttributeError: 'NoneType' object has no attribute 'get'\n# POTENTIAL SECURITY RISK - SSL verification is bypassed. Apply if you know what you are doing\npip install mlx-vlm --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org\n```\n\n**Poppler Missing:**\n```bash\n# macOS\nbrew install poppler\n\n# Ubuntu\u002FDebian\nsudo apt-get install poppler-utils\n\n# Verify installation\npdftoppm -h\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>🔧 Runtime Issues\u003C\u002Fsummary>\n\n**Memory Errors:**\n- Use smaller or MoE models to reduce VRAM footprint\n- Enable image cropping: `--crop-size 100`\n- Process single pages instead of entire PDFs\n\n**Model Loading Fails:**\n```bash\n# Clear model cache\nrm -rf ~\u002F.cache\u002Fhuggingface\u002F\nrm -rf ~\u002F.mlx\u002F\n\n# Redownload models\npython -c \"from mlx_vlm import load; load('model-name')\"\n```\n\n**API Connection Issues:**\n```bash\n# Check if server is running\ncurl http:\u002F\u002Flocalhost:8002\u002Fhealth\n\n# Check logs\npython api.py --debug\n```\n\n\u003C\u002Fdetails>\n\n\u003Cdetails>\n\u003Csummary>📄 Document Processing Issues\u003C\u002Fsummary>\n\n**Poor Extraction Quality:**\n- Add extraction hints to steer model attention to problem fields\n- Try image cropping: `--crop-size 60`\n- Use `--table --table-template` with dots.ocr for table-heavy documents\n- Ensure image resolution is adequate (300+ DPI)\n- Use schema validation: avoid `--options validation_off`\n\n**PDF Processing Fails:**\n```bash\n# Test PDF manually\npdftoppm -png input.pdf output\n\n# Check page count\npython -c \"\nimport pypdf\nwith open('file.pdf', 'rb') as f:\n    reader = pypdf.PdfReader(f)\n    print(f'Pages: {len(reader.pages)}')\n\"\n```\n\n**JSON Schema Errors:**\n- Validate JSON syntax: Use [jsonlint.com](https:\u002F\u002Fjsonlint.com)\n- Use proper field types: `\"str\"`, `0`, `0.0`, `\"str or null\"`\n- Test with simple schema first\n\n\u003C\u002Fdetails>\n\n\n### Getting Help\n\n1. **📖 Check Documentation**: Review this README and component docs\n2. **🐛 Search Issues**: [GitHub Issues](https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\u002Fissues)  \n3. **💬 Create Issue**: Provide logs, system info, minimal example\n4. **📧 Commercial Support**: [abaranovskis@redsamuraiconsulting.com](mailto:abaranovskis@redsamuraiconsulting.com)\n\n## ⭐ Star History\n\n[![Star History Chart](https:\u002F\u002Fapi.star-history.com\u002Fsvg?repos=katanaml\u002Fsparrow&type=Date)](https:\u002F\u002Fstar-history.com\u002F#katanaml\u002Fsparrow&Date)\n\n## 📜 License\n\n**Open Source**: Licensed under GPL 3.0. Free for open source projects and organizations under $5M revenue.\n\n**Commercial**: Dual licensing available for proprietary use, enterprise features, and dedicated support.\n\n**Contact**: [abaranovskis@redsamuraiconsulting.com](mailto:abaranovskis@redsamuraiconsulting.com) for commercial licensing and consulting.\n\n## 👥 Authors\n\n- **[Katana ML](https:\u002F\u002Fkatanaml.io)** - AI\u002FML consulting and solutions\n- **[Andrej Baranovskij](https:\u002F\u002Fgithub.com\u002Fabaranovskis-redsamurai)** - Lead developer\n\n---\n\n\u003Cp align=\"center\">\n  \u003Cstrong>⭐ Star us on GitHub if Sparrow is useful for your projects!\u003C\u002Fstrong>\u003Cbr>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fkatanaml\u002Fsparrow\">github.com\u002Fkatanaml\u002Fsparrow\u003C\u002Fa>\n\u003C\u002Fp>\n","Sparrow 是一个用于结构化数据提取和指令处理的平台，结合了机器学习（ML）、大语言模型（LLM）和视觉大语言模型（Vision LLM）。其核心功能包括通过RESTful API提交文档并接收验证过的JSON数据、文本处理与决策制定以及多代理工作流编排。技术特点上，Sparrow支持多种后端如Apple Silicon上的MLX和NVIDIA上的vLLM，并提供统一的API接口。此外，它还拥有一个直观易用的Web界面，支持拖拽上传文件、实时处理结果展示等功能。适用于需要从发票、收据、报表等文档中高效准确地抽取结构化信息的企业场景。",2,"2026-06-11 03:36:28","high_star"]