[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80527":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":24,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":16,"starSnapshotCount":16,"syncStatus":29,"lastSyncTime":30,"discoverSource":31},80527,"rivers","ion-elgreco\u002Frivers","ion-elgreco","Rivers is an orchestration platform for data and ML pipelines, written in Rust for native performance with a Python-first development experience.","https:\u002F\u002Fion-elgreco.github.io\u002Frivers\u002F",null,"Rust",72,5,64,32,0,1,8,3,45.63,"GNU Affero General Public License v3.0",false,"main",true,[],"2026-06-12 04:01:28","\u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Fion-elgreco\u002Frivers\u002Fmain\u002Fassets\u002Flogo_wordmark.png\" alt=\"rivers\" width=\"320\" \u002F>\n\n**Orchestration platform for tasks and assets, fully backed by Rust.**\n\nrivers is a Rust-powered orchestration platform built around data assets. Define pipelines in Python; rivers resolves the graph, plans execution - no Python interpreter on the control plane.\n\n[Documentation](https:\u002F\u002Fion-elgreco.github.io\u002Frivers\u002F) · [Issues](https:\u002F\u002Fgithub.com\u002Fion-elgreco\u002Frivers\u002Fissues) · [Discussions](https:\u002F\u002Fgithub.com\u002Fion-elgreco\u002Frivers\u002Fdiscussions)\n\n## Key features\n\n- **Asset-based orchestration** — define data assets as Python functions; rivers resolves the dependency graph automatically.\n- **Rust core** — graph resolution, execution planning, partition logic, and the scheduler all run in compiled Rust.\n- **Multiple asset types** — single, multi-output, graph (composing `Task`s into sub-DAGs), and external assets.\n- **Partitioning** — static, time-window (daily\u002Fhourly\u002Fcustom cron), multi-dimensional, and runtime-extensible dynamic partitions.\n- **Pluggable IO** — built-in handlers for in-memory, pickle (any object store), and Delta Lake with merge support.\n- **Parallel & distributed execution** — `Executor.parallel()` for concurrent subprocess workers, `Executor.kubernetes()` for one-pod-per-step on K8s.\n- **Schedules, sensors, and automation conditions** — declarative triggers (cron, event-driven, dep-aware) executed by the rivers daemon.\n- **Backfills** — partition-range execution with multi-run, single-run, and per-dimension strategies.\n- **Persistent storage** — embedded SurrealDB + RocksDB for local dev, SurrealDB server for production.\n- **Concurrency control** — run-queue limits, tag concurrency, and step-level concurrency pools.\n- **Single-binary dev experience** — `rivers dev \u003Cmodule>` boots SurrealDB (embedded RocksDB), the scheduler, and the web UI on `:3000` in one process.\n\n## Performance\n\nHot paths run in compiled Rust: graph resolution, partition mapping, execution planning, the scheduler. Python is the API surface only. Plan times stay sub-millisecond on graphs with thousands of nodes. The UI is Rust too — Leptos SSR + WASM on `axum`, state read straight from SurrealDB and pushed to the browser via Server-Sent Events.\n\n## Kubernetes-native\n\nrivers ships with a Kubernetes operator and CRDs. Declare a repo as a `CodeLocation`:\n\n```yaml\napiVersion: rivers.io\u002Fv1alpha1\nkind: CodeLocation\nmetadata:\n  name: analytics\nspec:\n  image: ghcr.io\u002Facme\u002Fpipelines\n  tag: v0.2.0\n  module: pipelines.analytics\n```\n\nThe operator resolves the image to a digest, reconciles a `Deployment` + `Service` running `rivers serve`, registers it with the UI's discovery registry, and re-polls the registry to keep the digest fresh. Multi-arch images (`linux\u002Famd64`, `linux\u002Farm64`) and Helm charts are published to `ghcr.io` on every release with SLSA build-provenance attestations.\n\nSee the [installation guide](https:\u002F\u002Fion-elgreco.github.io\u002Frivers\u002Finstallation\u002Fkubernetes\u002F) for the full setup — helm install commands, common values, and an [architecture overview](https:\u002F\u002Fion-elgreco.github.io\u002Frivers\u002Finstallation\u002Foverview\u002F) with the reconciliation and run sequence diagrams.\n\n## Install\n\n```bash\npip install rivers\n```\n\nOptional extras for IO handlers:\n\n```bash\npip install rivers[delta]     # Delta Lake support\npip install rivers[pyarrow]   # PyArrow table support\npip install rivers[polars]    # Polars DataFrame support\n```\n\n## Quick example\n\n```python\nimport rivers as rs\n\n@rs.Asset\ndef raw_data():\n    return {\"users\": 100, \"events\": 5000}\n\n@rs.Asset\ndef summary(raw_data: dict):\n    return f\"{raw_data['users']} users, {raw_data['events']} events\"\n\nrepo = rs.CodeRepository(assets=[raw_data, summary])\nresult = repo.materialize()\n\nprint(repo.load_node(\"summary\"))  # \"100 users, 5000 events\"\n```\n\nSee the [Getting Started guide](https:\u002F\u002Fion-elgreco.github.io\u002Frivers\u002Flatest\u002Fgetting-started\u002F) for partitioning, jobs, IO handlers, and the K8s executor.\n\n## Contributing\n\nContributions are welcome. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for development setup (`just develop`, `just test`, `just pre-commit`), code conventions, and the test matrix. The [`docs\u002F`](docs\u002F) directory hosts both the user-facing guides and architectural notes for contributors.\n","Rivers 是一个用于数据和机器学习流水线的编排平台，使用 Rust 编写以实现原生性能，并提供 Python 优先的开发体验。其核心功能包括基于资产的编排、自动解析依赖图、以及通过 Rust 核心进行图解析、执行计划和调度等，支持多种资产类型和灵活的分区策略。此外，它还具备并行与分布式执行能力，支持 Kubernetes 郲署，并内置了持久化存储解决方案。Rivers 适用于需要高性能数据处理管道和机器学习工作流管理的企业级应用场景，特别是在对执行效率和资源利用率有较高要求的情况下。",2,"2026-06-11 04:01:06","CREATED_QUERY"]