[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-3859":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},3859,"data-formulator","microsoft\u002Fdata-formulator","microsoft","🪄 Create rich visualizations with AI ","https:\u002F\u002Farxiv.org\u002Fabs\u002F2408.16119",null,"TypeScript",15818,1486,106,50,0,4,28,338,19,44.52,"MIT License",false,"main",true,[],"2026-06-12 02:00:55","\u003Ch1 align=\"center\">\n  \u003Cimg src=\".\u002Fpublic\u002Ffavicon.ico\" alt=\"Data Formulator icon\" width=\"28\"> \n  Data Formulator: AI-powered Data Visualization\n\u003C\u002Fh1>\n\n\n\u003Cp align=\"center\">\n  🪄 Explore data with visualizations, powered by AI agents.\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fdata-formulator.ai\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F🚀_Try_Online_Demo-data--formulator.ai-F59E0B?style=for-the-badge\" alt=\"Try Online Demo\">\u003C\u002Fa>\n   \n  \u003Ca href=\"#get-started\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002F💻_Install_Locally-uvx_|_pip-3776AB?style=for-the-badge\" alt=\"Install Locally\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2408.16119\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FPaper-arXiv:2408.16119-b31b1b.svg\" alt=\"arXiv\">\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Fopensource.org\u002Flicenses\u002FMIT\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FLicense-MIT-yellow.svg\" alt=\"License: MIT\">\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=GfTE2FLyMrs\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FYouTube-white?logo=youtube&logoColor=%23FF0000\" alt=\"YouTube\">\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Factions\u002Fworkflows\u002Fpython-build.yml\">\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Factions\u002Fworkflows\u002Fpython-build.yml\u002Fbadge.svg\" alt=\"build\">\u003C\u002Fa> \n  \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FmYCZMQKYZb\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fdiscord-chat-green?logo=discord\" alt=\"Discord\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003C!-- [![Open in GitHub Codespaces](https:\u002F\u002Fgithub.com\u002Fcodespaces\u002Fbadge.svg)](https:\u002F\u002Fcodespaces.new\u002Fmicrosoft\u002Fdata-formulator?quickstart=1) -->\n\u003C!-- \nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F8ca57b68-4d7a-42cb-bcce-43f8b1681ce2 -->\n\n\u003Ckbd>\n  \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F3ffb15aa-93ce-42b8-92cf-aaf321f9a06a\">\n\u003C\u002Fkbd>\n\n\n## News 🔥🔥🔥\n\n[03-02-2026] **Data Formulator 0.7 (alpha)** — More charts, new experience, enterprise-ready\n-  📊 **30 chart types** with a new semantic chart engine (area, streamgraph, candlestick, pie, radar, maps, and more).\n-  💬 **Hybrid chat + data thread** — chat woven into the exploration timeline with lineage, previews, and reasoning.\n-  🤖 **Unified `DataAgent`** replacing four separate agents, plus new recommendation and insight agents.\n-  🏗️ **Workspace \u002F Data Lake** — persistent, identity-based data management with local and Azure Blob backends.\n-  🔒 **Security hardening** — code signing, sandboxed execution, authentication, and rate limiting.\n-  📦 **UV-first build** — reproducible builds via `uv.lock`; `uv sync` + `uv run data_formulator`.\n-  📝 Detailed writeup on the new architecture coming soon — stay tuned!\n\n## Previous Updates\n\nHere are milestones that lead to the current design:\n- **v0.6** ([Demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.6)): Real-time insights from live data — connect to URLs and databases with automatic refresh\n- **uv support**: Faster installation with [uv](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F) — `uvx data_formulator` or `uv pip install data_formulator`\n- **v0.5.1** ([Demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fpull\u002F200#issue-3635408217)): Community data loaders, US Map & Pie Chart, editable reports, snappier UI\n- **v0.5**: Vibe with your data, in control — agent mode, data extraction, reports\n- **v0.2.2** ([Demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fpull\u002F176)): Goal-driven exploration with agent recommendations and performance improvements\n- **v0.2.1.3\u002F4** ([Readme](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Ftree\u002Fmain\u002Fpy-src\u002Fdata_formulator\u002Fdata_loader) | [Demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fpull\u002F155)): External data loaders (MySQL, PostgreSQL, MSSQL, Azure Data Explorer, S3, Azure Blob)\n- **v0.2** ([Demos](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.2)): Large data support with DuckDB integration\n- **v0.1.7** ([Demos](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.1.7)): Dataset anchoring for cleaner workflows\n- **v0.1.6** ([Demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.1.6)): Multi-table support with automatic joins\n- **Model Support**: OpenAI, Azure, Ollama, Anthropic via [LiteLLM](https:\u002F\u002Fgithub.com\u002FBerriAI\u002Flitellm) ([feedback](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fissues\u002F49))\n- **Python Package**: Easy local installation ([try it](#get-started))\n- **Visualization Challenges**: Test your skills ([challenges](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fissues\u002F53))\n- **Data Extraction**: Parse data from images and text ([demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fpull\u002F31#issuecomment-2403652717))\n- **Initial Release**: [Blog](https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Fresearch\u002Fblog\u002Fdata-formulator-exploring-how-ai-can-help-analysts-create-rich-data-visualizations\u002F) | [Video](https:\u002F\u002Fyoutu.be\u002F3ndlwt0Wi3c)\n\n\u003Cdetails>\n\u003Csummary>\u003Cb>View detailed update history\u003C\u002Fb>\u003C\u002Fsummary>\n\n- [07-10-2025] Data Formulator 0.2.2: Start with an analysis goal\n  - Some key frontend performance updates. \n  - You can start your exploration with a goal, or, tab and see if the agent can recommend some good exploration ideas for you. [Demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fpull\u002F176)\n\n- [05-13-2025] Data Formulator 0.2.1.3\u002F4: External Data Loader \n  - We introduced external data loader class to make import data easier. [Readme](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Ftree\u002Fmain\u002Fpy-src\u002Fdata_formulator\u002Fdata_loader) and [Demo](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fpull\u002F155)\n    - Current data loaders: MySQL, Azure Data Explorer (Kusto), Azure Blob and Amazon S3 (json, parquet, csv).\n    - [07-01-2025] Updated with: Postgresql, mssql.\n  - Call for action [link](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fissues\u002F156):\n    - Users: let us know which data source you'd like to load data from.\n    - Developers: let's build more data loaders.\n\n- [04-23-2025] Data Formulator 0.2: working with *large* data 📦📦📦\n  - Explore large data by:\n    1. Upload large data file to the local database (powered by [DuckDB](https:\u002F\u002Fgithub.com\u002Fduckdb\u002Fduckdb)).\n    2. Use drag-and-drop to specify charts, and Data Formulator dynamically fetches data from the database to create visualizations (with ⚡️⚡️⚡️ speeds).\n    3. Work with AI agents: they generate SQL queries to transform the data to create rich visualizations!\n    4. Anchor the result \u002F follow up \u002F create a new branch \u002F join tables; let's dive deeper. \n  - Checkout the demos at [[https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.2]](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.2)\n  - Improved overall system performance, and enjoy the updated derive concept functionality.\n\n- [03-20-2025] Data Formulator 0.1.7: Anchoring ⚓︎\n  - Anchor an intermediate dataset, so that followup data analysis are built on top of the anchored data, not the original one.\n  - Clean a data and work with only the cleaned data; create a subset from the original data or join multiple data, and then go from there. AI agents will be less likely to get confused and work faster. ⚡️⚡️\n  - Check out the demos at [[https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.1.7]](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.1.7)\n  - Don't forget to update Data Formulator to test it out!\n\n- [02-20-2025] Data Formulator 0.1.6 released! \n  - Now supports working with multiple datasets at once! Tell Data Formulator which data tables you would like to use in the encoding shelf, and it will figure out how to join the tables to create a visualization to answer your question. 🪄\n  - Checkout the demo at [[https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.1.6]](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Freleases\u002Ftag\u002F0.1.6).\n  - Update your Data Formulator to the latest version to play with the new features.\n\n- [02-12-2025] More models supported now!\n  - Now supports OpenAI, Azure, Ollama, and Anthropic models (and more powered by [LiteLLM](https:\u002F\u002Fgithub.com\u002FBerriAI\u002Flitellm));\n  - Models with strong code generation and instruction following capabilities are recommended (gpt-4o, claude-3-5-sonnet etc.);\n  - You can store API keys in `.env` to avoid typing them every time (copy `.env.template` to `.env` and fill in your keys).\n  - Let us know which models you have good\u002Fbad experiences with, and what models you would like to see supported! [[comment here]](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fissues\u002F49)\n\n- [11-07-2024] Minor fun update: data visualization challenges!\n  - We added a few visualization challenges with the sample datasets. Can you complete them all? [[try them out!]](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fissues\u002F53#issue-2641841252)\n  - Comment in the issue when you did, or share your results\u002Fquestions with others! [[comment here]](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fissues\u002F53)\n\n- [10-11-2024] Data Formulator python package released! \n  - You can now install Data Formulator using Python and run it locally, easily. [[check it out]](#get-started).\n  - Our Codespaces configuration is also updated for fast start up ⚡️. [[try it now!]](https:\u002F\u002Fcodespaces.new\u002Fmicrosoft\u002Fdata-formulator?quickstart=1)\n  - New experimental feature: load an image or a messy text, and ask AI to parse and clean it for you(!). [[demo]](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fpull\u002F31#issuecomment-2403652717)\n  \n- [10-01-2024] Initial release of Data Formulator, check out our [[blog]](https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Fresearch\u002Fblog\u002Fdata-formulator-exploring-how-ai-can-help-analysts-create-rich-data-visualizations\u002F) and [[video]](https:\u002F\u002Fyoutu.be\u002F3ndlwt0Wi3c)!\n\n\u003C\u002Fdetails>\n\n## Overview\n\n**Data Formulator** is a Microsoft Research prototype for data exploration with visualizations powered by AI agents.\n\nData Formulator enables analysts to iteratively explore and visualize data. Started with data in any format (screenshot, text, csv, or database), users can work with AI agents with a novel blended interface that combines *user interface interactions (UI)* and *natural language (NL) inputs* to communicate their intents, control branching exploration directions, and create reports to share their insights. \n\n## Get Started\n\nPlay with Data Formulator with one of the following options:\n\n- **Option 1: Install via uv (recommended)**\n  \n  [uv](https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002F) is an extremely fast Python package manager. If you have uv installed, you can run Data Formulator directly without any setup:\n  \n  ```bash\n  # Run data formulator directly (no install needed)\n  uvx data_formulator\n  ```\n\n  Or install it in a project\u002Fvirtual environment:\n  \n  ```bash\n  # Install data_formulator\n  uv pip install data_formulator\n\n  # Run data formulator\n  python -m data_formulator\n  ```\n\n  Data Formulator will be automatically opened in the browser at [http:\u002F\u002Flocalhost:5567](http:\u002F\u002Flocalhost:5567).\n\n- **Option 2: Install via pip**\n  \n  Use pip for installation (recommend: install it in a virtual environment).\n  \n  ```bash\n  # install data_formulator\n  pip install data_formulator\n\n  # Run data formulator with this command\n  python -m data_formulator\n  ```\n\n  Data Formulator will be automatically opened in the browser at [http:\u002F\u002Flocalhost:5567](http:\u002F\u002Flocalhost:5567).\n\n  *you can specify the port number (e.g., 8080) by `python -m data_formulator --port 8080` if the default port is occupied.*\n\n- **Option 3: Codespaces (5 minutes)**\n  \n  You can also run Data Formulator in Codespaces; we have everything pre-configured. For more details, see [CODESPACES.md](CODESPACES.md).\n  \n  [![Open in GitHub Codespaces](https:\u002F\u002Fgithub.com\u002Fcodespaces\u002Fbadge.svg)](https:\u002F\u002Fcodespaces.new\u002Fmicrosoft\u002Fdata-formulator?quickstart=1)\n\n- **Option 4: Working in the developer mode**\n  \n  You can build Data Formulator locally if you prefer full control over your development environment and develop your own version on top. For detailed instructions, refer to [DEVELOPMENT.md](DEVELOPMENT.md).\n\n\n## Using Data Formulator\n\n### Load Data\n\nBesides uploading csv, tsv or xlsx files that contain structured data, you can ask Data Formulator to extract data from screenshots, text blocks or websites, or load data from databases use connectors. Then you are ready to explore.\n\n\u003Cimg width=\"1920\" alt=\"image\" src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fe23cdb47-984c-4ce4-a014-8f36e025e393\" \u002F>\n\n### Explore Data\n\nThere are four levels to explore data based depending on whether you want more vibe or more control:\n\n- Level 1 (most control): Create charts with UI via drag-and-drop, if all fields to be visualized are already in the data.\n- Level 2: Specify chart designs with natural language + NL. Describe how new fields should be visualized in your chart, AI will automatically transform data to realize the design.\n- Level 3: Get recommendations: Ask AI agents to recommend charts directly from NL descriptions, or even directly ask for exploration ideas.\n- Level 4 (most vibe): In agent mode, provide a high-level goal and let AI agents automatically plan and explore data in multiple turns. Exploration threads will be created automatically.\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F164aff58-9f93-4792-b8ed-9944578fbb72\n\n- Level 5: In practice, leverage all of them to keep up with both vibe and control!\n\n### Create Reports\n\nUse the report builder to compose a report of the style you like, based on selected charts. Then share the reports to others!\n\n\u003C!-- \n### The basics of data visualization\n* Set up model provider, for agentic experience, model with reasoning and strong code generation ablity is recommended.\n* Describe the exploration \n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F0fbea012-1d2d-46c3-a923-b1fc5eb5e5b8\n\n\n### Create visualization beyond the initial dataset (powered by 🤖)\n* You can type names of **fields that do not exist in current data** in the encoding shelf:\n    - this tells Data Formulator that you want to create visualizations that require computation or transformation from existing data,\n    - you can optionally provide a natural language prompt to explain and clarify your intent (not necessary when field names are self-explanatory).\n* Click the **Formulate** button.\n    - Data Formulator will transform data and instantiate the visualization based on the encoding and prompt.\n* Inspect the data, chart and code.\n* To create a new chart based on existing ones, follow up in natural language:\n    - provide a follow up prompt (e.g., *``show only top 5!''*),\n    - you may also update visual encodings for the new chart.\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F160c69d2-f42d-435c-9ff3-b1229b5bddba\n\nhttps:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fc93b3e84-8ca8-49ae-80ea-f91ceef34acb\n\nRepeat this process as needed to explore and understand your data. Your explorations are trackable in the **Data Threads** panel.  -->\n\n## Developers' Guide\n\nFollow the [developers' instructions](DEVELOPMENT.md) to build your new data analysis tools on top of Data Formulator.\n\nHelp wanted:\n\n* Add more database connectors (https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002Fdata-formulator\u002Fissues\u002F156)\n* Scaling up messy data extractor: more document types and larger files.\n* Adding more chart templates (e.g., maps).\n* other ideas?\n\n## Research Papers\n* [Data Formulator 2: Iteratively Creating Rich Visualizations with AI](https:\u002F\u002Farxiv.org\u002Fabs\u002F2408.16119)\n\n```\n@article{wang2024dataformulator2iteratively,\n      title={Data Formulator 2: Iteratively Creating Rich Visualizations with AI}, \n      author={Chenglong Wang and Bongshin Lee and Steven Drucker and Dan Marshall and Jianfeng Gao},\n      year={2024},\n      booktitle={ArXiv preprint arXiv:2408.16119},\n}\n```\n\n* [Data Formulator: AI-powered Concept-driven Visualization Authoring](https:\u002F\u002Farxiv.org\u002Fabs\u002F2309.10094)\n\n```\n@article{wang2023data,\n  title={Data Formulator: AI-powered Concept-driven Visualization Authoring},\n  author={Wang, Chenglong and Thompson, John and Lee, Bongshin},\n  journal={IEEE Transactions on Visualization and Computer Graphics},\n  year={2023},\n  publisher={IEEE}\n}\n```\n\n\n## Contributing\n\nThis project welcomes contributions and suggestions. Most contributions require you to\nagree to a Contributor License Agreement (CLA) declaring that you have the right to,\nand actually do, grant us the rights to use your contribution. For details, visit\nhttps:\u002F\u002Fcla.microsoft.com.\n\nWhen you submit a pull request, a CLA-bot will automatically determine whether you need\nto provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the\ninstructions provided by the bot. You will only need to do this once across all repositories using our CLA.\n\nThis project has adopted the [Microsoft Open Source Code of Conduct](https:\u002F\u002Fopensource.microsoft.com\u002Fcodeofconduct\u002F).\nFor more information see the [Code of Conduct FAQ](https:\u002F\u002Fopensource.microsoft.com\u002Fcodeofconduct\u002Ffaq\u002F)\nor contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.\n\n## Trademarks\n\nThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft \ntrademarks or logos is subject to and must follow \n[Microsoft's Trademark & Brand Guidelines](https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Flegal\u002Fintellectualproperty\u002Ftrademarks\u002Fusage\u002Fgeneral).\nUse of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.\nAny use of third-party trademarks or logos are subject to those third-party's policies.\n","Data Formulator 是一个由AI驱动的数据可视化工具，旨在帮助用户通过丰富的图表探索数据。该项目利用AI代理生成多样化的视觉展示，支持30种图表类型，包括面积图、流图、蜡烛图等，并提供统一的DataAgent进行数据处理与分析建议。此外，它还具备混合聊天和数据线程功能，能够在数据探索过程中融入对话记录、预览及推理过程。Data Formulator适用于需要高效数据分析与展示的企业级应用场景，如商业智能报告制作、科学研究成果展示等。项目采用TypeScript编写，遵循MIT许可协议开放源代码，便于开发者根据需求定制或扩展功能。",2,"2026-06-11 02:56:43","top_language"]