[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-81812":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":8,"htmlUrl":8,"language":9,"languages":8,"totalLinesOfCode":8,"stars":10,"forks":11,"watchers":12,"openIssues":13,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":14,"forks30d":14,"starsTrendScore":11,"compositeScore":18,"rankGlobal":8,"rankLanguage":8,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":20,"hasPages":20,"topics":22,"createdAt":8,"pushedAt":8,"updatedAt":23,"readmeContent":24,"aiSummary":25,"trendingCount":14,"starSnapshotCount":14,"syncStatus":15,"lastSyncTime":26,"discoverSource":27},81812,"codag-drain","codag-megalith\u002Fcodag-drain","codag-megalith",null,"Rust",41,6,34,1,0,2,5,7,46.74,"Other",false,"master",[],"2026-06-12 04:01:35","# codag-drain\n\nReal-time log compression for agents.\n\n`codag-drain` adapts [Drain3](https:\u002F\u002Fgithub.com\u002Flogpai\u002FDrain3) into a streaming\nlog-templating engine. It collapses high-volume, repetitive log lines into a\ncompact set of template groups — each with a few raw samples and slot summaries —\nso an agent can take in a large, noisy log window without spending its context\nbudget on near-duplicate lines.\n\nThat is the whole job. It is the compression layer behind `codag wrap`: turn a\nraw log stream into a small, agent-readable artifact, in real time, as lines\narrive. What reads that artifact decides what it means.\n\n## Workspace\n\n```text\ndrain3_rust\u002F     Rust Drain3 implementation used as the base algorithm\ncodag-drain\u002F     deterministic templating library and CLI\nexamples\u002Fserver\u002F reference HTTP host for long-lived wrapping sessions\ndocs\u002F            design and evaluation notes\n```\n\n## Default Algorithm\n\nThe default `GrouperKind::Drain` is Drain-style positional similarity with one\ncodag adaptation:\n\n- normal logs use Drain3-compatible whitespace tokenization and default masking;\n- compact punctuation-heavy one-token logs, such as compact JSON, use a generic\n  character-class tokenizer so Drain still has token positions to compare;\n- output rendering is codag-specific: template count, samples, and slot\n  summaries.\n\nAdditional deterministic arms are available for evaluation:\n\n- `drain-stock`\n- `drain-delimited`\n- `drain-fullsearch`\n- `statistical`\n\nSee [docs\u002FPUBLIC_BENCHMARKS.md](docs\u002FPUBLIC_BENCHMARKS.md) for reproducible\nLogHub parser benchmarks and\n[docs\u002FAGENT_SERVING_EVAL.md](docs\u002FAGENT_SERVING_EVAL.md) for the downstream\nblind-judge evidence.\n\n## CLI\n\n```bash\necho 'worker ready shard=1\nworker ready shard=2' | cargo run -p codag-drain\n```\n\nJSON output:\n\n```bash\necho 'worker ready shard=1' \\\n  | cargo run -p codag-drain -- --format json\n```\n\nSelect a grouper:\n\n```bash\ncargo run -p codag-drain -- --grouper drain-stock\n```\n\nPrint CLI compression stats on stderr:\n\n```bash\ncargo run -p codag-drain -- --stats\n```\n\n## Reference Server\n\nThe `examples\u002Fserver` crate is a thin host around `TemplateIndex`. It is useful\nfor local integration tests and as a deployment reference; production auth,\ntenancy, persistence, and routing should live in the production service layer.\n\n```bash\ncargo run -p codag-drain-server\n```\n\nRoutes:\n\n```text\nGET  \u002Fhealth\nPOST \u002Fv1\u002Ftemplate\nPOST \u002Fv1\u002Fsession\u002F:id\u002Fingest\nGET  \u002Fv1\u002Fsession\u002F:id\u002Ftemplates\n```\n\nQuery parameters:\n\n```text\ngrouper=drain|drain-stock|drain-delimited|drain-fullsearch|statistical\nsamples=N\nformat=text|json\nbody=text|ndjson\n```\n\nThe hosted production instance is a separate Railway service:\n\n```text\nRailway project: codag-drain\nRailway service: codag-drain\nProduction URL: https:\u002F\u002Fcodag-drain-production.up.railway.app\nBackend env: CODAG_DRAIN_URL=https:\u002F\u002Fcodag-drain-production.up.railway.app\n```\n\nAll `\u002Fv1\u002F*` routes require `Authorization: Bearer \u003Ctoken>`. Configure the\nsame secret value on both services:\n\n```text\ncodag-drain: CODAG_DRAIN_AUTH_TOKEN=\u003Crandom secret>\nbackend:     CODAG_DRAIN_AUTH_TOKEN=\u003Csame random secret>\n```\n\nDeploy it from this repo root:\n\n```bash\nrailway up --service codag-drain --environment production --detach\n```\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md). Benchmark claims must be scoped and\npaired against raw logs and Drain3; do not claim a win from compression alone.\n\n## Tests\n\n```bash\nCARGO_TARGET_DIR=\u002Fprivate\u002Ftmp\u002Fcodag-drain-target cargo test\nCARGO_TARGET_DIR=\u002Fprivate\u002Ftmp\u002Fcodag-drain-target cargo clippy --all-targets --all-features -- -D warnings\n```\n\nHeld-out evals are ignored by default because they need local data:\n\n```bash\nLOGHUB_DIR=\u002Fpath\u002Fto\u002Floghub2 \\\nCARGO_TARGET_DIR=\u002Fprivate\u002Ftmp\u002Fcodag-drain-target cargo test -p codag-drain --test eval_loghub grouping_loghub -- --ignored --nocapture\nGITHUB_JSONL=\u002Fpath\u002Fto\u002Fgithub.jsonl \\\nCARGO_TARGET_DIR=\u002Fprivate\u002Ftmp\u002Fcodag-drain-target cargo test -p codag-drain --test eval_loghub grouping_github_lora -- --ignored --nocapture\n```\n","codag-drain 是一个实时日志压缩工具，主要用于代理服务器。它基于 Drain3 算法实现了一个流式日志模板引擎，能够将大量重复的日志行压缩成一组紧凑的模板，并附带少量原始样本和槽位摘要，从而帮助代理在处理大规模、高噪声的日志窗口时有效节省上下文预算。项目使用 Rust 语言编写，确保了高性能与低资源消耗。适用于需要高效处理日志数据的场景，如监控系统、故障排查等，通过减少冗余信息来提高日志分析效率。","2026-06-11 04:06:48","CREATED_QUERY"]