[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-5697":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":21,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":24,"readmeContent":25,"aiSummary":26,"trendingCount":16,"starSnapshotCount":16,"syncStatus":27,"lastSyncTime":28,"discoverSource":29},5697,"noria","mit-pdos\u002Fnoria","mit-pdos","Fast web applications through dynamic, partially-stateful dataflow","",null,"Rust",5237,248,110,43,0,1,9,38.19,"Apache License 2.0",false,"master",[],"2026-06-12 02:01:14","# Noria: data-flow for high-performance web applications\n\n[![noria on crates.io](https:\u002F\u002Fimg.shields.io\u002Fcrates\u002Fv\u002Fnoria.svg)](https:\u002F\u002Fcrates.io\u002Fcrates\u002Fnoria)\n[![noria on docs.rs](https:\u002F\u002Fdocs.rs\u002Fnoria\u002Fbadge.svg)](https:\u002F\u002Fjon.thesquareplanet.com\u002Fcrates\u002Fnoria\u002F)\n[![noria-server on crates.io](https:\u002F\u002Fimg.shields.io\u002Fcrates\u002Fv\u002Fnoria-server.svg)](https:\u002F\u002Fcrates.io\u002Fcrates\u002Fnoria-server)\n[![noria-server on docs.rs](https:\u002F\u002Fdocs.rs\u002Fnoria-server\u002Fbadge.svg)](https:\u002F\u002Fjon.thesquareplanet.com\u002Fcrates\u002Fnoria-server\u002F)\n[![Azure Status](https:\u002F\u002Fdev.azure.com\u002Fmit-pdos\u002Fmit-pdos\u002F_apis\u002Fbuild\u002Fstatus\u002Fnoria?branchName=master)](https:\u002F\u002Fdev.azure.com\u002Fmit-pdos\u002Fmit-pdos\u002F_build\u002Flatest?definitionId=1&branchName=master)\n\nNoria is a new streaming data-flow system designed to act as a fast\nstorage backend for read-heavy web applications based on [Jon Gjengset's\nPhd Thesis](https:\u002F\u002Fjon.thesquareplanet.com\u002Fpapers\u002Fphd-thesis.pdf), as\nwell as [this paper](https:\u002F\u002Fjon.tsp.io\u002Fpapers\u002Fosdi18-noria.pdf) from\n[OSDI'18](https:\u002F\u002Fwww.usenix.org\u002Fconference\u002Fosdi18\u002Fpresentation\u002Fgjengset).\nIt acts like a database, but precomputes and caches relational query\nresults so that reads are blazingly fast. Noria automatically keeps cached\nresults up-to-date as the underlying data, stored in persistent _base\ntables_, change. Noria uses partially-stateful data-flow to reduce memory\noverhead, and supports dynamic, runtime data-flow and query change.\n\nNoria comes with [a MySQL\nadapter](https:\u002F\u002Fgithub.com\u002Fmit-pdos\u002Fnoria-mysql) that implements the\nbinary MySQL protocol. This lets any application that currently talks to\nMySQL or MariaDB switch to Noria with minimal effort. For example,\nrunning a [Lobsters-like workload](https:\u002F\u002Fgithub.com\u002Fjonhoo\u002Ftrawler)\nthat issues the [equivalent SQL\nqueries](https:\u002F\u002Fgithub.com\u002Fmit-pdos\u002Fnoria\u002Ftree\u002Fmaster\u002Fapplications\u002Flobsters\u002Fsrc\u002Fendpoints\u002Fnatural)\nto the real [Lobsters website](https:\u002F\u002Flobste.rs), Noria improves\nthroughput supported by 5x:\n\n![Noria speeds up Lobsters queries by 5x](https:\u002F\u002Fpeople.csail.mit.edu\u002Fmalte\u002Fprojects\u002Fnoria\u002Flobsters-perf.svg)\n\nAt a high level, Noria takes a set of parameterized SQL queries (think\n[prepared\nstatements](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPrepared_statement)), and\nproduces a [data-flow\nprogram](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FStream_processing) that maintains\n[materialized views](https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMaterialized_view)\nfor the output of those queries. Reads now become fast lookups directly\ninto these materialized views, as if the value had been directly cached\nin memcached. The views are then kept up-to-date incrementally through\nthe data-flow, which yields high write throughput.\n\n## Running Noria\n\nLike most databases, Noria follows a server-client model where many\nclients connect to a (potentially distributed) server. The server in\nthis case is the `noria-server` binary, and must be started before\nclients can connect. Noria also uses [Apache\nZooKeeper](https:\u002F\u002Fzookeeper.apache.org\u002F) to announce the location of\nits servers, so ZooKeeper must be running.\n\nYou (currently) need nightly Rust to build `noria-server`. This will be\narranged for\n[automatically](https:\u002F\u002Fgithub.com\u002Frust-lang-nursery\u002Frustup.rs#the-toolchain-file)\nif you're using [`rustup.rs`](https:\u002F\u002Frustup.rs\u002F). To build\n`noria-server`, run\n\n```console\n$ cargo build --release --bin noria-server\n```\n\nYou may need to install some dependencies for the above to work:\n\n - clang\n - libclang-dev\n - libssl-dev\n - liblz4-dev\n - build-essential\n\nTo start a long-running `noria-server` instance, ensure that ZooKeeper\nis running, and then run:\n\n```console\n$ cargo r --release --bin noria-server -- --deployment myapp --no-reuse --address 172.16.0.19 --shards 0\n```\n\n`myapp` here is a _deployment_. Many `noria-server` instances can\noperate in a single deployment at the same time, and will share the\nworkload between them. Workers in the same deployment automatically\nelect a leader and discovery each other via\n[ZooKeeper](http:\u002F\u002Fzookeeper.apache.org\u002F).\n\n## Interacting with Noria\n\nThere are two primary ways to interact with Noria: through the [Rust\nbindings](https:\u002F\u002Fcrates.io\u002Fcrates\u002Fnoria) or through the [MySQL\nadapter](https:\u002F\u002Fgithub.com\u002Fmit-pdos\u002Fnoria-mysql). They both\nautomatically locate the running worker through ZooKeeper (use `-z` if\nZooKeeper is not running on `localhost:2181`).\n\n### Rust bindings\n\nThe [`noria` crate](https:\u002F\u002Fcrates.io\u002Fcrates\u002Fnoria) provides native Rust\nbindings to interact with `noria-server`. See the [`noria`\ndocumentation](https:\u002F\u002Fjon.thesquareplanet.com\u002Fcrates\u002Fnoria\u002F) for detailed\ninstructions on how to use the library. You can also take a look at the\n[example Noria program](noria\u002Fexamples\u002Fquickstart.rs) using Noria's\nclient API. You can also see a self-contained version that embeds\n`noria-server` (and doesn't require ZooKeeper) in [this\nexample](server\u002Fexamples\u002Flocal-server.rs).\n\n### MySQL adapter\n\nWe have built a [MySQL\nadapter](https:\u002F\u002Fgithub.com\u002Fmit-pdos\u002Fnoria-mysql) for Noria that accepts\nstandard MySQL queries and speaks the MySQL protocol to make it easy to\ntry Noria out for existing applications. Once the adapter is running\n(see its `README`), you should be able to point your application at\n`localhost:3306` to send queries to Noria. If your application crashes,\nthis is a bug, and we would appreciate it if you [open an\nissue](https:\u002F\u002Fgithub.com\u002Fmit-pdos\u002Fnoria\u002Fissues). You may also want to\ntry to disable automatic re-use (with `--no-reuse`) or sharding (with\n`--shards 0`) in case those are misbehaving.\n\n## CLI and Web UI\n\nYou can manually inspect the data stored in Noria using any MySQL client\n(e.g., the `mysql` CLI), or use [Noria's own web\ninterface](https:\u002F\u002Fgithub.com\u002Fmit-pdos\u002Fnoria-ui).\n\n## Noria development\n\nNoria is a large piece of software that spans many sub-crates and\nexternal tools (see links in the text above). Each sub-crate is\nresponsible for a component of Noria's architecture, such as external\nAPI (`noria`), mapping SQL to data-flow (`server\u002Fmir`), and\nexecuting data-flow operators (`server\u002Fdataflow`). The code in\n`server\u002Fsrc\u002F` is the glue that ties these pieces together by\nestablishing materializations, scheduling data-flow work, orchestrating\nNoria program changes, handling failovers, etc.\n\n[`server\u002Fsrc\u002Flib.rs`](server\u002Fsrc\u002Flib.rs) has a pretty extensive comment at\nthe top of it that goes through how the Noria internals fit together at\nan implementation level. While it occasionally lags behind, especially\nfollowing larger changes, it should serve to get you familiarized with\nthe basic building blocks relatively quickly.\n\nThe sub-crates each serve a distinct role:\n\n - [`noria\u002F`](noria\u002F): everything that an external program communicating\n   with Noria needs. This includes types used in RPCs as\n   arguments\u002Freturn types, as well as code for discovering Noria workers\n   through ZooKeeper, establishing a connection to Noria through\n   ZooKeeper, and invoking the various RPC exposed by the Noria\n   controller ([`server\u002Fsrc\u002Fcontroller.rs`](server\u002Fsrc\u002Fcontroller\u002Finner.rs)).\n   The `noria` sub-crate also contains a number of internal\n   data-structures that must be shared between the client and the\n   server like [`DataType`](noria\u002Fsrc\u002Fdata.rs) (Noria's \"value\"\n   type). These are annotated with `#[doc(hidden)]`, and should be easy\n   to spot in `noria\u002Fsrc\u002Flib.rs`.\n - [`applications\u002F`](applications\u002F): a collection of various\n   Noria benchmarks. The most frequently used one is `vote`, which runs\n   the vote benchmark from §8.2 of the OSDI paper. You can run it in a\n   bunch of different ways (`--help` should be useful), and with many\n   different backends. The `localsoup` backend is the one that's easiest\n   to get up and running with.\n - [`server\u002Fsrc\u002F`](server\u002Fsrc\u002F): the Noria server, including\n   high-level components such as RPC handling, domain scheduling,\n   connection management, and all the controller operations (listening\n   for heartbeats, handling failed workers, etc.). It contains two\n   notable sub-crates:\n\n   - [`dataflow\u002F`](server\u002Fdataflow\u002F): the code that implements the\n     internals of the data-flow graph. This includes implementations of\n     the different operators ([`ops\u002F`](server\u002Fdataflow\u002Fsrc\u002Fops\u002F)),\n     \"special\" operators like leaf views and sharders\n     ([`node\u002Fspecial\u002F`](server\u002Fdataflow\u002Fsrc\u002Fnode\u002Fspecial\u002F)),\n     implementations of view storage ([`state\u002F`](server\u002Fdataflow\u002Fsrc\u002Fstate\u002F)),\n     and the code that coordinates execution of control, data, and\n     backfill messages within a thread domain\n     ([`domain\u002F`](server\u002Fdataflow\u002Fsrc\u002Fdomain\u002F)).\n   - [`mir\u002F`](server\u002Fmir\u002F): the code that implements Noria's\n     SQL-to-dataflow mapping. This includes resolving columns and keys,\n     creating dataflow operators, and detecting reuse opportunities, and\n     triggering migrations to make changes after new SQL queries have\n     been added. @ms705 is the primary author of this particular\n     subcrate, and it builds largely upon\n     [`nom-sql`](https:\u002F\u002Fdocs.rs\u002Fnom-sql\u002F).\n   - [`common\u002F`](server\u002Fcommon\u002F): data-structures that are shared\n     between the various `server` sub-crates.\n\nTo run the test suite, use:\n```console\n$ cargo test\n```\n\nBuild and open the documentation with:\n```console\n$ cargo doc --open\n```\n\nOnce `noria-server` is running, its API is available on port 6033 at the\nspecified listen address.\n\nAlternatively, you can discover Noria's REST API listen address and port\nthrough ZooKeeper via this command:\n\n```console\n$ cargo run --bin noria-zk -- \\\n    --show --deployment myapp\n    | grep external | cut -d' ' -f4\n```\n\nA basic graphical UI runs at `http:\u002F\u002FIP:PORT\u002Fgraph.html` and shows\nthe running data-flow graph. You can also deploy Noria's\n[more advanced web UI](https:\u002F\u002Fgithub.com\u002Fmit-pdos\u002Fnoria-ui) that serves\nthe REST API endpoints in a human-digestible form and includes the\ngraph visualization.\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0\n   ([LICENSE-APACHE](LICENSE-APACHE) or http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0)\n * MIT license\n   ([LICENSE-MIT](LICENSE-MIT) or http:\u002F\u002Fopensource.org\u002Flicenses\u002FMIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n","Noria 是一个为读取密集型 Web 应用程序设计的快速数据流系统。它通过预计算和缓存关系查询结果来加速读取操作，并自动保持这些缓存结果与底层数据的一致性。Noria 利用部分状态的数据流技术减少内存开销，支持动态调整数据流和查询变更。该项目使用 Rust 语言开发，提供了一个 MySQL 适配器，使得现有基于 MySQL 或 MariaDB 的应用程序可以轻松迁移到 Noria。适用于需要高吞吐量、低延迟响应的在线服务场景，如社交网络、新闻网站等，能够显著提升应用性能。",2,"2026-06-11 03:04:45","top_language"]