[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71104":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":15,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":16,"starSnapshotCount":16,"syncStatus":29,"lastSyncTime":30,"discoverSource":31},71104,"OLMo","allenai\u002FOLMo","allenai","Modeling, training, eval, and inference code for OLMo","https:\u002F\u002Fallenai.org\u002Folmo",null,"Python",6534,767,58,20,0,11,28,33,97.46,"Apache License 2.0",false,"main",true,[],"2026-06-12 04:00:59","\u003Cdiv align=\"center\">\n  \u003C!-- \u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fassets\u002F8812459\u002F774ac485-a535-4768-8f7c-db7be20f5cc3\" width=\"300\"\u002F> -->\n  \u003Cbr>\n  \u003Cbr>\n  \u003Ch1>OLMo: Open Language Model\u003C\u002Fh1>\n\u003C\u002Fdiv>\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fblob\u002Fmain\u002FLICENSE\">\n    \u003Cimg alt=\"GitHub License\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Flicense\u002Fallenai\u002FOLMo\">\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Freleases\">\n    \u003Cimg alt=\"GitHub release\" src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Frelease\u002Fallenai\u002FOLMo.svg\">\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fpdf\u002F2501.00656.pdf\">\n    \u003Cimg alt=\"Paper URL\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Farxiv-2402.00838-blue\">\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fplayground.allenai.org\">\n    \u003Cimg alt=\"Playground\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FAi2-Playground-F0529C\">\n  \u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fdiscord.gg\u002FsZq3jTNVNG\">\n    \u003Cimg alt=\"Discord\" src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FDiscord%20-%20blue?style=flat&logo=discord&label=Ai2&color=%235B65E9\">\n  \u003C\u002Fa>\n\u003C\u002Fp>\n\n# ⚠️ NOTICE ⚠️ This repository is out of date with our more recent releases and is no longer active. For the latest Olmo releases and updates, please visit: https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo-core\u002F \n\nOLMo is a repository for training and using AI2's state-of-the-art open language models. It is designed by scientists, for scientists.\n\n## Installation\n\nFirst, install [PyTorch](https:\u002F\u002Fpytorch.org) following the instructions specific to your operating system.\n\nFor training and fine-tuning, we recommend installing from source:\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo.git\ncd OLMo\npip install -e .[all]\n```\nYou can also install from PyPI with:\n```bash\npip install ai2-olmo\n```\n\n## Pretraining\n\nOLMo pretraining follows a two-stage training procedure.\nIn the first stage, we train on large amounts of mostly web-based data: [OLMo-mix-1124](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fallenai\u002Folmo-mix-1124)\nIn the second stage, we train on a smaller amount of high-quality, targeted data: [Dolmino-mix-1124](https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002Fallenai\u002Fdolmino-mix-1124)\n\nYou can find *all* the checkpoints, at minimum every 1000 training steps in OLMo core and Hugging Face format:\n\n\n| Variant         | OLMo Format (Stage 1)                                                                                         | OLMo Format (Stage 2) | Hugging Face Format                                                               |\n|----------------|-----------------------------------------------------------------------------------------------------|--------|----------------------------------------------------------------------------------|\n| **OLMo-2 1B**  | [OLMo-2 1B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fblob\u002Fmain\u002Fconfigs\u002Fofficial-0425\u002FOLMo-2-0425-1B.csv)     | [OLMo-2 1B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fblob\u002Fmain\u002Fconfigs\u002Fofficial-0425\u002FOLMo-2-0425-1B-stage2.csv)      | [Hugging Face for the 1B variant](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0425-1B)  |\n| **OLMo-2 7B**  | [OLMo-2 7B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fblob\u002Fmain\u002Fconfigs\u002Fofficial-1124\u002FOLMo-2-1124-7B.csv)     | [OLMo-2 7B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fblob\u002Fmain\u002Fconfigs\u002Fofficial-1124\u002FOLMo-2-1124-7B-stage2.csv)      | [Hugging Face for the 7B variant](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-7B)  |\n| **OLMo-2 13B** | [OLMo-2 13B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fblob\u002Fmain\u002Fconfigs\u002Fofficial-1124\u002FOLMo-2-1124-13B.csv)   | [OLMo-2 13B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo\u002Fblob\u002Fmain\u002Fconfigs\u002Fofficial-1124\u002FOLMo-2-1124-13B-stage2.csv)       | [Hugging Face for the 13B variant](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B) |\n| **OLMo-2 32B** | [OLMo-2 32B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo-core\u002Fblob\u002Fmain\u002Fsrc\u002Fscripts\u002Fofficial\u002FOLMo2-0325-32B.csv)   | [OLMo-2 32B](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo-core\u002Fblob\u002Fmain\u002Fsrc\u002Fscripts\u002Fofficial\u002FOLMo-2-0325-32B-stage2.csv) | [Hugging Face for the 32B variant](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0325-32B) |\n\n> Note: The 32B variant was trained on our new trainer. To train or fine-tune OLMo-2 32B, visit [OLMo-core](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo-core).\n\n### Steps to reproduce\n\nTo reproduce any of the training processes described below, run this:\n\n```bash\ntorchrun --nproc_per_node=8 scripts\u002Ftrain.py {path_to_train_config}\n```\n\nFor the training config, use any of the configs listed below.\n\nIf you want to override any of the settings in the training config without having to write a new config every time,\nyou can do this:\n\n```bash\ntorchrun --nproc_per_node=8 scripts\u002Ftrain.py {path_to_train_config} \\\n  --setting1=value \\\n  --setting2=value \\\n  --setting3.subsetting1=value\n```\n\nThe training configs below refer to training data that gets streamed in live over HTTP.\nTo reproduce at large scale, we recommend downloading the files locally and changing the paths to point to your\nlocal file system.\n\n#### To run on Mac silicon devices:\n```bash\npython scripts\u002Ftrain.py {path_to_train_config}\n```\nExample:\n```bash\npython scripts\u002Ftrain.py configs\u002Ftiny\u002FOLMo-20M.yaml --save_overwrite\n```\n> Note: You need to upgrade PyTorch to 2.5.x to run.\n\n### Stage 1\n\nStage 1 is the biggest stage, where we train on 4T or 5T tokens on largely web-based data. \n\n|                 | OLMo2 1B                                                                                                          | OLMo2 7B                                                                                                          | OLMo2 13B                                                                                                          |\n|-----------------|-----------------|-------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|\n| Number of tokens| 4 Trillion | 4 Trillion                                                                                                        | 5 Trillion                                                                                                         |\n| Checkpoint      |[stage1-step1907359-tokens4001B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0425-1B\u002Ftree\u002Fstage1-step1907359-tokens4001B)| [stage1-step928646-tokens3896B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-7B\u002Ftree\u002Fstage1-step928646-tokens3896B) | [stage1-step596057-tokens5001B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B\u002Ftree\u002Fstage1-step596057-tokens5001B) |\n| Training config | [OLMo2-1B-stage1.yaml](configs\u002Fofficial-0425\u002FOLMo2-1B-stage1.yaml) |[OLMo2-7B-stage1.yaml](configs\u002Fofficial-1124\u002FOLMo2-7B-stage1.yaml)                                                | [OLMo2-13B-stage1.yaml](configs\u002Fofficial-1124\u002FOLMo2-13B-stage1.yaml)                                               |                                              |\n| WandB           | [wandb.ai\u002FOLMo2-1B](https:\u002F\u002Fapi.wandb.ai\u002Flinks\u002Fai2-llm\u002Fizdtrtu0)|[wandb.ai\u002FOLMo2-7B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-7B\u002Freports\u002FOLMo-2-7B-Nov-2024--VmlldzoxMDUzMzE1OA)       | [wandb.ai\u002FOLMo2-13B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-13B\u002Freports\u002FOLMo-2-13B-Nov-2024--VmlldzoxMDUzMjQxNg) |\n\nYou can find the .csv.gz files containing the training data [here](configs\u002Fofficial-1124\u002Fprovenance.csv).\n\n### Stage 2 for the 1B\n\nFor the 1B model, we have trained three times with different data order on 50B high quality tokens, used last checkpoint of seed 42 as final checkpoint.\n\n|                        | Checkpoint                                                                                                                          | Training config                                                                        | WandB       |\n|------------------------|-------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|-------------|\n| random seed 42069         | [stage2-ingredient1-step23852-tokens51B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0425-1B\u002Ftree\u002Fstage2-ingredient1-step23852-tokens51B) | [OLMo2-1B-stage2-seed42069.yaml](configs\u002Fofficial-0425\u002FOLMo2-1B-stage2-seed42069.yaml)       | [wandb.ai\u002FOLMo2-1B](https:\u002F\u002Fapi.wandb.ai\u002Flinks\u002Fai2-llm\u002Fizdtrtu0) |\n| random seed 666      | [stage2-ingredient2-step23852-tokens51B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0425-1B\u002Ftree\u002Fstage2-ingredient2-step23852-tokens51B) | [OLMo2-1B-stage2-seed666.yaml](configs\u002Fofficial-0425\u002FOLMo2-1B-stage2-seed666.yaml) | [wandb.ai\u002FOLMo2-1B](https:\u002F\u002Fapi.wandb.ai\u002Flinks\u002Fai2-llm\u002Fizdtrtu0) |\n| random seed 42  (main)      | [stage2-ingredient3-step23852-tokens51B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0425-1B\u002Ftree\u002Fstage2-ingredient3-step23852-tokens51B) | [OLMo2-1B-stage2-seed42.yaml](configs\u002Fofficial-0425\u002FOLMo2-1B-stage2-seed42.yaml)     | [wandb.ai\u002FOLMo2-1B](https:\u002F\u002Fapi.wandb.ai\u002Flinks\u002Fai2-llm\u002Fizdtrtu0) |\n\n\n### Stage 2 for the 7B\n\nFor the 7B model, we train three times with different data order on 50B high quality tokens, and then average (\"soup\") the models.\n\n|                        | Checkpoint                                                                                                                          | Training config                                                                        | WandB       |\n|------------------------|-------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|-------------|\n| random seed 42         | [stage2-ingredient1-step11931-tokens50B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-7B\u002Ftree\u002Fstage2-ingredient1-step11931-tokens50B) | [OLMo2-7B-stage2-seed42.yaml](configs\u002Fofficial-1124\u002FOLMo2-7B-stage2-seed42.yaml)       | [wandb.ai\u002FOLMo2-7B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-7B\u002Freports\u002F) |\n| random seed 42069      | [stage2-ingredient2-step11931-tokens50B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-7B\u002Ftree\u002Fstage2-ingredient2-step11931-tokens50B) | [OLMo2-7B-stage2-seed42069.yaml](configs\u002Fofficial-1124\u002FOLMo2-7B-stage2-seed42069.yaml) | [wandb.ai\u002FOLMo2-7B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-7B\u002Freports\u002F) |\n| random seed 666        | [stage2-ingredient3-step11931-tokens50B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-7B\u002Ftree\u002Fstage2-ingredient3-step11931-tokens50B) | [OLMo2-7B-stage2-seed666.yaml](configs\u002Fofficial-1124\u002FOLMo2-7B-stage2-seed666.yaml)     | [wandb.ai\u002FOLMo2-7B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-7B\u002Freports\u002F) |\n| **final souped model** | [main](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-7B\u002Ftree\u002Fmain) | no config, we just averaged the weights in Python                                      | |\n\nThe training configs linked here are set up to download the latest checkpoint after stage 1, and start training from there.\n\n### Stage 2 for the 13B\n\nFor the 13B model, we train three times with different data order on 100B high quality tokens, and one more time\non 300B high quality tokens. Then we average (\"soup\") the models.\n\n|                        | Checkpoint                                                                                                                             | Training config                                                                                  | WandB       |\n|------------------------|----------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------|-------------|\n| random seed 1110, 100B | [stage2-ingredient1-step11931-tokens100B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B\u002Ftree\u002Fstage2-ingredient1-step11931-tokens100B) | [OLMo2-13B-stage2-seed1110-100B.yaml](configs\u002Fofficial-1124\u002FOLMo2-13B-stage2-seed1110-100B.yaml) | [wandb.ai\u002FOLMo2-13B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-13B\u002Freports\u002FOLMo-2-13B-Nov-2024--VmlldzoxMDUzMjQxNg) |\n| random seed 2662, 100B | [stage2-ingredient2-step11931-tokens100B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B\u002Ftree\u002Fstage2-ingredient2-step11931-tokens100B) | [OLMo2-13B-stage2-seed2662-100B.yaml](configs\u002Fofficial-1124\u002FOLMo2-13B-stage2-seed2662-100B.yaml) | [wandb.ai\u002FOLMo2-13B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-13B\u002Freports\u002FOLMo-2-13B-Nov-2024--VmlldzoxMDUzMjQxNg) |\n| random seed 6209, 100B | [stage2-ingredient3-step11931-tokens100B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B\u002Ftree\u002Fstage2-ingredient3-step11931-tokens100B) | [OLMo2-13B-stage2-seed6209-100B.yaml](configs\u002Fofficial-1124\u002FOLMo2-13B-stage2-seed6209-100B.yaml) | [wandb.ai\u002FOLMo2-13B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-13B\u002Freports\u002FOLMo-2-13B-Nov-2024--VmlldzoxMDUzMjQxNg) |\n| random seed 2662, 300B | [stage2-ingredient4-step11931-tokens300B](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B\u002Ftree\u002Fstage2-ingredient4-step35773-tokens300B) | [OLMo2-13B-stage2-seed2662-300B.yaml](configs\u002Fofficial-1124\u002FOLMo2-13B-stage2-seed2662-300B.yaml) | [wandb.ai\u002FOLMo2-13B](https:\u002F\u002Fwandb.ai\u002Fai2-llm\u002FOLMo-2-1124-13B\u002Freports\u002FOLMo-2-13B-Nov-2024--VmlldzoxMDUzMjQxNg) |\n| **final souped model** | [main](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B\u002Ftree\u002Fmain)                                                                       | no config, we just averaged the weights in Python                                                | |\n\nThe training configs linked here are set up to download the latest checkpoints after stage 1, and start training from there.\n\n> Note: You can find all the information about the 32B in the [OLMo-core](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo-core) repository.\n\n## Instruction tuned variants\n\nFor instruction tuned variants of these models, go to\n * [OLMo2 1B Instruct](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0425-1B-Instruct)\n * [OLMo2 7B Instruct](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-7B-Instruct)\n * [OLMo2 13B Instruct](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-1124-13B-Instruct)\n * [OLMo2 32B Instruct](https:\u002F\u002Fhuggingface.co\u002Fallenai\u002FOLMo-2-0325-32B-Instruct)\n\n## Inference\n\nYou can use our Hugging Face integration to run inference on the OLMo Transformers checkpoints:\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nolmo = AutoModelForCausalLM.from_pretrained(\"allenai\u002FOLMo-2-0425-1B\")\ntokenizer = AutoTokenizer.from_pretrained(\"allenai\u002FOLMo-2-0425-1B\")\nmessage = [\"Language modeling is \"]\ninputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)\n# optional verifying cuda\n# inputs = {k: v.to('cuda') for k,v in inputs.items()}\n# olmo = olmo.to('cuda')\nresponse = olmo.generate(**inputs, max_new_tokens=100, do_sample=True, top_k=50, top_p=0.95)\nprint(tokenizer.batch_decode(response, skip_special_tokens=True)[0])\n```\n\nAlternatively, with the Hugging Face pipeline abstraction:\n\n```python\nfrom transformers import pipeline\nolmo_pipe = pipeline(\"text-generation\", model=\"allenai\u002FOLMo-2-0425-1B\")\nprint(olmo_pipe(\"Language modeling is\"))\n```\n\n### Quantization\n\n```python\nolmo = AutoModelForCausalLM.from_pretrained(\"allenai\u002FOLMo-2-0425-1B\", torch_dtype=torch.float16, load_in_8bit=True)  # requires bitsandbytes\n```\n\nThe quantized model is sensitive to input types and CUDA handling. To avoid potential issues, we recommend explicitly converting input IDs to CUDA using: `inputs.input_ids.to('cuda')`\n\n## Evaluation\n\nAdditional tools for evaluating OLMo models are available at the [OLMo Eval](https:\u002F\u002Fgithub.com\u002Fallenai\u002FOLMo-eval) and [olmes](https:\u002F\u002Fgithub.com\u002Fallenai\u002Folmes) repositories.\n\n## Modal.com Hosting\n\nAn example script is provided for hosting an OLMo 2 model on Modal.com using the OpenAI API in `.\u002Fscripts\u002Folmo2_modal_openai.py`.\nTo run that:\n\n1. Follow the instructions under Getting Started in [the Modal.com Guide](https:\u002F\u002Fmodal.com\u002Fdocs\u002Fguide) to install\nthe Modal library and command line tools.\u003C\u002Fli>\n2. Follow the instructions under [Secrets](https:\u002F\u002Fmodal.com\u002Fdocs\u002Fguide\u002Fsecrets) in the Modal.com Guide to create a Modal secret named \"example-secret-token\"\nthat defines a value for the variable MODAL_TOKEN for your server.\u003C\u002Fli>\n3. Then run\n```bash\nmodal deploy .\u002Fscripts\u002Folmo2_modal_openai.py\n```\n\nYou can check your endpoint using curl similar to the following:\n```bash\ncurl -X POST \\\n  -H \"Authorization: Bearer [the secret token from above]\" \\\n  -H \"Content-Type: application\u002Fjson\" \\\n  -d @body.json \\\n  https:\u002F\u002F[the web endpoint modal creates above]\u002Fv1\u002Fchat\u002Fcompletions\n```\n\nwhere `body.json` is of the form:\n```\n{\n    \"model\": \"OLMo-2-1124-13B-Instruct\",\n    \"messages\": [\n        {\n            \"role\": \"user\",\n            \"content\": \"Who was Alan Turing?\"\n        }\n      ],\n    \"max_tokens\": 100,\n    \"temperature\": 0.9,\n    \"stream\": true\n}\n```\n\n\n## Citing\n\n```bibtex\n@misc{olmo20242olmo2furious,\n      title={2 OLMo 2 Furious}, \n      author={Team OLMo and Pete Walsh and Luca Soldaini and Dirk Groeneveld and Kyle Lo and Shane Arora and Akshita Bhagia and Yuling Gu and Shengyi Huang and Matt Jordan and Nathan Lambert and Dustin Schwenk and Oyvind Tafjord and Taira Anderson and David Atkinson and Faeze Brahman and Christopher Clark and Pradeep Dasigi and Nouha Dziri and Michal Guerquin and Hamish Ivison and Pang Wei Koh and Jiacheng Liu and Saumya Malik and William Merrill and Lester James V. Miranda and Jacob Morrison and Tyler Murray and Crystal Nam and Valentina Pyatkin and Aman Rangapur and Michael Schmitz and Sam Skjonsberg and David Wadden and Christopher Wilhelm and Michael Wilson and Luke Zettlemoyer and Ali Farhadi and Noah A. Smith and Hannaneh Hajishirzi},\n      year={2024},\n      eprint={2501.00656},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.00656}, \n}\n```\n","OLMo是一个用于训练和使用AI2最先进的开放语言模型的项目。该项目提供了从建模、训练到评估和推理的全流程代码支持，采用Python语言编写，并基于PyTorch框架实现。其核心特色在于采用了两阶段训练方法：首先利用大量网络数据进行初步训练，随后在高质量目标数据上进行精细化调整，从而确保模型既能广泛吸收信息又能精准适应特定任务需求。适用于需要定制化语言模型的研究人员或开发者，特别是在追求模型性能与灵活性之间平衡的应用场景下。",2,"2026-06-11 03:35:54","high_star"]