[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72688":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":25,"hasPages":23,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},72688,"guidance","guidance-ai\u002Fguidance","guidance-ai","A guidance language for controlling large language models.","",null,"Jupyter Notebook",21493,1169,120,254,0,7,9,39,21,91.6,"MIT License",false,"main",true,[],"2026-06-12 04:01:06","\u003Cdiv align=\"right\">\n  \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FcjPfAK43dz\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDiscord-Join%20Us-5865F2?logo=discord&logoColor=white\" alt=\"Discord\">\u003C\u002Fa>\n  \u003Ca href=\"mailto:guidanceai@microsoft.com\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FEmail-guidanceai%40microsoft.com-0078D4?logo=microsoft-outlook&logoColor=white\" alt=\"Email\">\u003C\u002Fa>\n  \u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHours-10am--2pm%20Pacific-gray\" alt=\"Hours\">\n\u003C\u002Fdiv>\n\u003Cdiv align=\"center\">\u003Cpicture>\n  \u003Csource media=\"(prefers-color-scheme: dark)\" srcset=\"docs\u002Ffigures\u002Fguidance_logo_blue_dark.svg\">\n  \u003Cimg alt=\"guidance\" src=\"docs\u002Ffigures\u002Fguidance_logo_blue.svg\" width=300\">\n\u003C\u002Fpicture>\u003C\u002Fdiv>\n\u003Cbr\u002F>\n\n**Guidance is an efficient programming paradigm for steering language models.** With Guidance, you can control how output is structured and get high-quality output for your use case—*while reducing latency and cost vs. conventional prompting or fine-tuning.* It allows users to constrain generation (e.g. with regex and CFGs) as well as to interleave control (conditionals, loops, tool use) and generation seamlessly.\n\n   * [Install](#install)\n   * [Features](#features)\n\n\n## Install\nGuidance is available through PyPI and supports a variety of backends (Transformers, llama.cpp, OpenAI, etc.).\nIf you already have the backend required for your model, you can simply run\n```bash\npip install guidance\n```\n\n## Features\n\n### A Pythonic interface for language models\n\nWhen using Guidance, you can work with large language models using common Python idioms:\n\n```python\nfrom guidance import system, user, assistant, gen\nfrom guidance.models import Transformers\n\n# Could also do LlamaCpp or many other models\nphi_lm = Transformers(\"microsoft\u002FPhi-4-mini-instruct\")\n\n# Model objects are immutable, so this is a copy\nlm = phi_lm\n\nwith system():\n    lm += \"You are a helpful assistant\"\n\nwith user():\n    lm += \"Hello. What is your name?\"\n\nwith assistant():\n    lm += gen(max_tokens=20)\n\nprint(lm)\n```\nIf run at the command line, this will produce output like:\n```\n\u003C|system|>You are a helpful assistant\u003C|end|>\u003C|user|>Hello. What is your name?\u003C|end|>\u003C|assistant|>I am Phi, an AI developed by Microsoft. How can I help you today?\n```\nHowever, if running in a Jupyter notebook, then Guidance provides a widget for a richer user experience:\n\n\u003Cimg src=\"docs\u002Ffigures\u002Fwidget_basic_example_20250703.png\" alt=\"Guidance widget showing HTML generation\" \u002F>\n\nWith Guidance, it's really easy to capture generated text:\n\n```python\n# Get a new copy of the Model\nlm = phi_lm\n\nwith system():\n    lm += \"You are a helpful assistant\"\n\nwith user():\n    lm += \"Hello. What is your name?\"\n\nwith assistant():\n    lm += gen(name=\"lm_response\", max_tokens=20)\n\nprint(f\"{lm['lm_response']=}\")\n```\n\n```\nlm['lm_response']='I am Phi, an AI developed by Microsoft. How can I help you today?'\n```\n\n### Guarantee output syntax with constrained generation\n\nGuidance provides an easy to use, yet immensely powerful syntax for constraining the output of a language model.\nFor example, a `gen()` call can be constrained to match a regular expression:\n\n```python\nlm = phi_lm\n\nwith system():\n    lm += \"You are a teenager\"\n\nwith user():\n    lm += \"How old are you?\"\n\nwith assistant():\n    lm += gen(\"lm_age\", regex=r\"\\d+\", temperature=0.8)\n\nprint(f\"The language model is {lm['lm_age']} years old\")\n```\n\n```\nThe language model is 13 years old\n```\n\nOften, we know that the output has to be an item from a list we know in advance.\nGuidance provides a `select()` function for this scenario:\n\n```python\nfrom guidance import select\n\nlm = phi_lm\n\nwith system():\n    lm += \"You are a geography expert\"\n\nwith user():\n    lm += \"\"\"What is the capital of Sweden? Answer with the correct letter.\n\n    A) Helsinki\n    B) Reykjavík \n    C) Stockholm\n    D) Oslo\n    \"\"\"\n\nwith assistant():\n    lm += select([\"A\", \"B\", \"C\", \"D\"], name=\"model_selection\")\n\nprint(f\"The model selected {lm['model_selection']}\")\n```\n\n```\nThe model selected C\n```\n\nThe constraint system offered by Guidance is extremely powerful.\nIt can ensure that the output conforms to any context free grammar (so long as the backend LLM has full support for Guidance).\nMore on this below.\n\n### Debug grammars offline (no model API calls)\n\nWhen iterating on constraints, you can validate candidate strings locally and test a full run with the `Mock` model.\n\n```python\nfrom guidance import gen\nfrom guidance.models import Mock\n\ngrammar = \"expr=\" + gen(regex=r\"\\d+([+*]\\d+)*\", name=\"expr\")\n\n# 1) Validate strings directly against the grammar\nassert grammar.match(\"expr=12+7*3\") is not None\nassert grammar.match(\"expr=12+*3\") is None\n\n# 2) Run the same grammar with a local mock model\nlm = Mock(b\"\u003Cs>expr=12+7*3\")\nlm += grammar\nprint(lm[\"expr\"])  # 12+7*3\n```\n\n### Create your own Guidance functions\n\nWith Guidance, you can create your own Guidance functions which can interact with language models.\nThese are marked using the `@guidance` decorator.\nSuppose we wanted to answer lots of multiple choice questions.\nWe could do something like the following:\n\n```python\nimport guidance\n\nfrom guidance.models import Model\n\nASCII_OFFSET = ord(\"a\")\n\n@guidance\ndef zero_shot_multiple_choice(\n    language_model: Model,\n    question: str,\n    choices: list[str],\n):\n    with user():\n        language_model += question + \"\\n\"\n        for i, choice in enumerate(choices):\n            language_model += f\"{chr(i+ASCII_OFFSET)} : {choice}\\n\"\n\n    with assistant():\n        language_model += select(\n            [chr(i + ASCII_OFFSET) for i in range(len(choices))], name=\"string_choice\"\n        )\n\n    return language_model\n```\nNow, define some questions:\n```python\nquestions = [\n    {\n        \"question\" : \"Which state has the northernmost capital?\",\n        \"choices\" : [\n            \"New South Wales\",\n            \"Northern Territory\",\n            \"Queensland\",\n            \"South Australia\",\n            \"Tasmania\",\n            \"Victoria\",\n            \"Western Australia\",\n        ],\n        \"answer\" : 1,\n    },\n    {\n        \"question\" : \"Which of the following is venomous?\",\n        \"choices\" : [\n            \"Kangaroo\",\n            \"Koala Bear\",\n            \"Platypus\",\n        ],\n        \"answer\" : 2,\n    }\n]\n```\nWe can use our decorated function like `gen()` or `select()`.\nThe `language_model` argument will be filled in for us automatically:\n```python\nlm = phi_lm\n\nwith system():\n    lm += \"You are a student taking a multiple choice test.\"\n\nfor mcq in questions:\n    lm_temp = lm + zero_shot_multiple_choice(question=mcq[\"question\"], choices=mcq[\"choices\"])\n    converted_answer = ord(lm_temp[\"string_choice\"]) - ASCII_OFFSET\n    print(lm_temp)\n    print(f\"LM Answer: {converted_answer},  Correct Answer: {mcq['answer']}\")\n```\n\n```\n\u003C|system|>You are a student taking a multiple choice test.\u003C|end|>\u003C|user|>Which state has the northernmost capital?\na : New South Wales\nb : Northern Territory\nc : Queensland\nd : South Australia\ne : Tasmania\nf : Victoria\ng : Western Australia\n\u003C|end|>\u003C|assistant|>b\nLM Answer: 1,  Correct Answer: 1\n\u003C|system|>You are a student taking a multiple choice test.\u003C|end|>\u003C|user|>Which of the following is venomous?\na : Kangaroo\nb : Koala Bear\nc : Platypus\n\u003C|end|>\u003C|assistant|>c\nLM Answer: 2,  Correct Answer: 2\n```\n\nGuidance functions can be composed, in order to construct a full context free grammar.\nFor example, we can create Guidance functions to build a simple HTML webpage (note that this is _not_ a full implementation of HTML).\nWe start with a simple function which will generate text which does not contain any HTML tags.\nThe function is marked as `stateless` to indicate that we intend to use it for composing a grammar:\n\n```python\n@guidance(stateless=True)\ndef _gen_text(lm: Model):\n    return lm + gen(regex=\"[^\u003C>]+\") \n```\n\nWe can then use this function to generate text within an arbitrary HTML tag:\n```python\n@guidance(stateless=True)\ndef _gen_text_in_tag(lm: Model, tag: str):\n    lm += f\"\u003C{tag}>\"\n    lm += _gen_text()\n    lm += f\"\u003C\u002F{tag}>\"\n    return lm\n```\nNow, let us create the page header. As part of this, we need to generate a page title:\n```python\n@guidance(stateless=True)\ndef _gen_header(lm: Model):\n    lm += \"\u003Chead>\\n\"\n    lm += _gen_text_in_tag(\"title\") + \"\\n\"\n    lm += \"\u003C\u002Fhead>\\n\"\n    return lm\n```\nThe body of the HTML page is going to be filled with headings and paragraphs.\nWe can define a function to do each:\n```python\nfrom guidance.library import one_or_more\n\n@guidance(stateless=True)\ndef _gen_heading(lm: Model):\n    lm += select(\n        options=[_gen_text_in_tag(\"h1\"), _gen_text_in_tag(\"h2\"), _gen_text_in_tag(\"h3\")]\n    )\n    lm += \"\\n\"\n    return lm\n\n@guidance(stateless=True)\ndef _gen_para(lm: Model):\n    lm += \"\u003Cp>\"\n    lm += one_or_more(\n        select(\n            options=[\n                _gen_text(),\n                _gen_text_in_tag(\"em\"),\n                _gen_text_in_tag(\"strong\"),\n                \"\u003Cbr \u002F>\",\n            ],\n        )\n    )\n    lm += \"\u003C\u002Fp>\\n\"\n    return lm\n```\nNow, the function to define the body of the HTML itself:\n```python\n@guidance(stateless=True)\ndef _gen_body(lm: Model):\n    lm += \"\u003Cbody>\\n\"\n    lm += one_or_more(select(options=[_gen_heading(), one_or_more(_gen_para())]))\n    lm += \"\u003C\u002Fbody>\\n\"\n    return lm\n```\nNext, we come to the function which generates the complete HTML page.\nWe add the HTML start tag, then generate the header, then body, and then append the ending HTML tag:\n```python\n@guidance(stateless=True)\ndef _gen_html(lm: Model):\n    lm += \"\u003Chtml>\\n\"\n    lm += _gen_header()\n    lm += _gen_body()\n    lm += \"\u003C\u002Fhtml>\\n\"\n    return lm\n```\nFinally, we provide a user-friendly wrapper, which will allow us to:\n- Set the temperature of the generation\n- Capture the generated page from the Model object\n```python\nfrom guidance.library import capture, with_temperature\n\n@guidance(stateless=True)\ndef make_html(\n    lm,\n    name: str | None = None,\n    *,\n    temperature: float = 0.0,\n):\n    return lm + capture(\n        with_temperature(_gen_html(), temperature=temperature),\n        name=name,\n    )\n```\nNow, use this to generate a simple webpage:\n```python\nlm = phi_lm\n\nwith system():\n    lm += \"You are an expert in HTML\"\n\nwith user():\n    lm += \"Create a simple and short web page about your life story.\"\n\nwith assistant():\n    lm += make_html(name=\"html_text\", temperature=0.7)\n```\n\nWhen running in a Jupyter Notebook so that the widget is active, we get the following output:\n\n\u003Cimg src=\"docs\u002Ffigures\u002Fwidget_make_html_20250703.png\" alt=\"Guidance widget showing HTML generation with token fast-forwarding\" \u002F>\n\nNote the varying highlighting of the generation.\nThis is showing another of Guidance's capabilities: fast-forwarding of tokens.\nThe constraints imposed by a grammar often mean that some tokens are known in advance.\nGuidance doesn't need the model to generate these; instead it can insert them into the generation.\nThis saves forward passes through the model, and hence reduces GPU usage.\nFor example, in the above HTML generation, Guidance always knows the last opening tag.\nIf the last opened tag was `\u003Ch1>` (for example), then as soon as the model generates `\u003C\u002F`, Guidance can fill in `h1>` without needing the model to perform a forward pass.\n\n### Generating JSON\n\nA JSON schema is actually a context free grammar, and hence it can be used to constrain an LLM using Guidance.\nThis is a common enough case that Guidance provides special support for it.\nA quick sample, based on a Pydantic model:\n```python\nimport json\nfrom pydantic import BaseModel, Field\n\nfrom guidance import json as gen_json\n\nclass BloodPressure(BaseModel):\n    systolic: int = Field(gt=300, le=400)\n    diastolic: int = Field(gt=0, le=20)\n    location: str = Field(max_length=50)\n    model_config = dict(extra=\"forbid\")\n\nlm = phi_lm\n\nwith system():\n    lm += \"You are a doctor taking a patient's blood pressure taken from their arm\"\n\nwith user():\n    lm += \"Report the blood pressure\"\n\nwith assistant():\n    lm += gen_json(name=\"bp\", schema=BloodPressure)\n\nprint(f\"{lm['bp']=}\")\n\n# Use Python's JSON library\nloaded_json = json.loads(lm[\"bp\"])\nprint(json.dumps(loaded_json, indent=4))\n\n# Use Pydantic\nresult = BloodPressure.model_validate_json(lm[\"bp\"])\nprint(result.model_dump_json(indent=8))\n```\n```\nlm['bp']='{\"systolic\": 301, \"diastolic\": 15, \"location\": \"arm\"}'\n{\n    \"systolic\": 301,\n    \"diastolic\": 15,\n    \"location\": \"arm\"\n}\n{\n        \"systolic\": 301,\n        \"diastolic\": 15,\n        \"location\": \"arm\"\n}\n```\nNote that the generated blood pressure is not one the model will have seen for a human.\nWhen generating JSON, a substantial number of tokens can often be fast-forwarded, due to the structural constraints imposed by the schema.\n","Guidance 是一种用于控制大型语言模型的高效编程范式。它通过提供结构化输出控制、减少延迟和成本，以及支持正则表达式和上下文无关文法等约束生成，使用户能够无缝地混合控制（如条件语句、循环、工具使用）与生成过程。此外，Guidance 提供了一个Python式的接口来操作语言模型，并且可以在Jupyter Notebook中以更丰富的形式展示交互结果。该项目适合需要精确控制AI生成内容格式及质量的应用场景，例如构建对话系统或定制化的文本生成服务。",2,"2026-06-11 03:43:11","high_star"]