[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-73188":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":23,"defaultBranch":24,"hasWiki":23,"hasPages":25,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},73188,"openzl","facebook\u002Fopenzl","facebook","A novel data compression framework","https:\u002F\u002Fopenzl.org",null,"C",3088,150,18,27,0,5,7,35,15,28.54,"Other",false,"dev",true,[],"2026-06-12 02:03:10","# OpenZL\n\nOpenZL delivers high compression ratios _while preserving high speed_, a level of performance that is out of reach for generic compressors. **Check out the [blog post](https:\u002F\u002Fengineering.fb.com\u002F2025\u002F10\u002F06\u002Fdeveloper-tools\u002Fopenzl-open-source-format-aware-compression-framework\u002F) and [whitepaper](https:\u002F\u002Farxiv.org\u002Fabs\u002F2510.03203) for a breakdown of how it works.**\n\nOpenZL takes a description of your data and builds from it a specialized compressor optimized for your specific format. [Learn how it works →](https:\u002F\u002Ffacebook.github.io\u002Fopenzl\u002Fgetting-started\u002Fintroduction\u002F)\n\nOpenZL consists of a core library and tools to generate specialized compressors —\nall compatible with a single universal decompressor.\nIt is designed for engineers that deal with large quantities of specialized datasets (like AI workloads for example) and require high speed for their processing pipelines.\n\nSee our [docs](https:\u002F\u002Ffacebook.github.io\u002Fopenzl) for more information and our [quickstart guide](https:\u002F\u002Ffacebook.github.io\u002Fopenzl\u002Fgetting-started\u002Fquick-start) to get started with a guided tutorial.\n\n## Project Status\n\nThis project is under active development. The API, the compressed format, and the set of codecs and graphs included in OpenZL are all subject to (and will!) change as the project matures.\n\nHowever, we intend to maintain some stability guarantees in the face of that evolution. In particular, payloads compressed with any release-tagged version of the library will remain decompressible by new releases of the library for at least the next several years. And new releases of the library will be able to generate frames compatible with at least the previous release.\n\n(Commits on the `dev` branch offer no guarantees whatsoever. Use only release-tagged commits for any non-experimental deployments.)\n\nDespite the big scary warnings above, we consider the core to have reached production-readiness, and OpenZL is used extensively in production at Meta.\n\n## Building OpenZL\n\n### Prerequisites\nOpenZL requires a compiler that supports C11 and C++17. When building with `cmake`, `cmake 3.20.2` or newer is required. There is ongoing work to relax these restrictions. As that happens, this section will be updated.\n\n### Build with `make`\n\nThe OpenZL library and essential tools can be built using `make`:\n\n```sh\nmake\n```\n\n#### Build Options\n\nThe `Makefile` supports all standard build variables, such as `CC`, `CFLAGS`, `CPPFLAGS`, `LDFLAGS`, `LDLIBS`, etc.\n\nIt builds with multi-threading by default, auto-detecting the local number of cores, and can be overridden using standard `-j#` flag (ex: `make -j8`).\n\n#### Build Types\n\nBinary generation can be altered by explicitly requesting a build type:\n\nExample:\n```sh\nmake lib BUILD_TYPE=DEV\n```\n\nBuild types are documented in `make help`, and their exact flags are detailed with `make show-config`.\n\nUsual ones are:\n\n* `BUILD_TYPE=DEV`: debug build with asserts enabled and ASAN \u002F UBSAN enabled\n* `BUILD_TYPE=OPT`: optimized build with asserts disabled (default)\n\n### Build with `cmake`\n\nOpenZL can be built using `cmake`. Basic usage is as follows:\n\n```sh\nmkdir build\ncd build\ncmake -DCMAKE_BUILD_TYPE=Release -DOPENZL_BUILD_TESTS=ON ..\nmake -j\nmake -j test\n```\n\nDetails on setting CMake variables is below.\n\n#### Build Modes\n\nBy default, we ship several different predefined build modes which can be set with the `OPENZL_BUILD_MODE` variable:\n\n* `none` (default): CMake default build mode controlled by `CMAKE_BUILD_TYPE`\n* `dev`: debug build with asserts enabled and ASAN \u002F UBSAN enabled\n* `dev-nosan`: debug build with asserts enabled\n* `opt`: optimized build with asserts disabled\n* `opt-asan`: optimized build with asserts disabled and ASAN \u002F UBSAN enabled\n* `dbgo`: optimized build with asserts enabled\n* `dbgo-asan`: optimized build with asserts enabled and ASAN \u002F UBSAN enabled\n\n> [!CAUTION]\n> When switching between build modes, make sure to purge the CMake cache and re-configure the build. For instance,\n> `cmake --fresh -DOPENZL_BUILD_MODE=dev-nosan ..`\n\nFor ASAN \u002F UBSAN, ensure that `libasan` and `libubsan` are installed on the machine.\n\n#### Editor Integration\n\nOpenZL ships with settings to configure VSCode to work with the CMake build system. To enable it install two extensions:\n\n1. `cmake-tools`\n2. `clangd` (or any other C++ language server that works with `compile_commands.json`)\n\n**Important:** For proper C++ language server support, you need to generate `compile_commands.json`:\n\nThe preferred method is to use the CMake Tools extension command \"`CMake: Configure`\".\n\nIf it doesn't work, or is too difficult to setup, you can use the manual setup:\n\n```bash\nmkdir -p cmakebuild\ncmake -B cmakebuild -DOPENZL_BUILD_TESTS=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON .\ncp cmakebuild\u002Fcompile_commands.json .\n```\n\n**When to regenerate:**\n\n* After cloning the repository (first-time setup)\n* When adding\u002Fremoving source files\n* When modifying `CMakeLists.txt`\n\n#### CMake Variables\n\n* `CMAKE_C_COMPILER` = Set the C compiler for OpenZL & dependency builds\n* `CMAKE_CXX_COMPILER` = Set the C++ compiler for OpenZL & dependency builds\n* `CMAKE_C_FLAGS` = C flags for OpenZL & dependency builds\n* `CMAKE_CXX_FLAGS` = C++ flags for OpenZL & dependency builds\n* `OPENZL_BUILD_TESTS=ON` = pull in testing deps and build the unit\u002Fintegration tests\n* `OPENZL_BUILD_BENCHMARKS=ON` = pull in benchmarking deps and build the benchmark executable\n* `OPENZL_BUILD_MODE` = Sets the build mode for OpenZL and dependencies\n* `OPENZL_SANITIZE_ADDRESS=ON` = Enable ASAN & UBSAN for OpenZL (but not dependencies)\n* `OPENZL_COMMON_COMPILE_OPTIONS` = Shared C\u002FC++ compiler options for OpenZL only\n* `OPENZL_C_COMPILE_OPTIONS` = C compiler options for OpenZL only\n* `OPENZL_CXX_COMPILE_OPTIONS` = C++ compiler options for OpenZL only\n* `OPENZL_COMMON_COMPILE_DEFINITIONS` = Shared C\u002FC++ compiler definitions (-D) for OpenZL only\n* `OPENZL_C_COMPILE_DEFINITIONS` = C compiler definitions (-D) for OpenZL only\n* `OPENZL_CXX_COMPILE_DEFINITIONS` = C++ compiler definitions (-D) for OpenZL only\n* `OPENZL_COMMON_FLAGS` = extra compiler flags used in all targets\n\n### Windows Build\n\nOpenZL uses modern C11 features that may not be fully supported by MSVC. For Windows builds, we recommend using **clang-cl** for the best compatibility.\n\n#### Quick Start (Windows)\n\n1. **Recommended**: Use `clang-cl` for full C11 support\n```cmd\ncmake -S . -B build -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl\ncmake --build build --config Release\n```\n\n2. **Alternative**: Use MinGW-w64 for GNU toolchain compatibility.\n```cmd\ncmake -S . -B build -G \"MinGW Makefiles\"\ncmake --build build --config Release\n```\n\n3. **Limited Support**: MSVC may produce C2099 errors due to limited C11 support.\n\n#### Compiler Detection\n\nRun our detection script to check available compilers and get recommendations:\n\n```cmd\n# PowerShell\n.\u002Fbuild-scripts\u002Fcmake\u002Fdetect_windows_compiler.ps1\n\n# Command Prompt\n.\u002Fbuild-scripts\u002Fcmake\u002Fdetect_windows_compiler.bat\n```\n\nFor detailed Windows build instructions, troubleshooting, and installation guides, see [build-scripts\u002Fcmake\u002FWINDOWS_BUILD.md](build-scripts\u002Fcmake\u002FWINDOWS_BUILD.md).\n\n## License\n\nOpenZL is BSD licensed, as found in the [LICENSE](LICENSE) file.\n","OpenZL 是一个创新的数据压缩框架，旨在为特定数据格式提供高效的压缩比同时保持高速度。其核心功能包括根据用户提供的数据描述自动生成针对特定格式优化的压缩器，并且所有生成的压缩文件都可以通过统一的解压器进行解压。技术上，OpenZL 支持 C11 和 C++17 标准，并使用 CMake 进行构建管理。该项目特别适合处理大规模专业化数据集（如 AI 负载）的工程师，能够显著提升数据处理管道的速度和效率。尽管项目仍在积极开发中，但已达到生产就绪状态，并在 Meta 内部广泛使用。",2,"2026-06-11 03:44:25","high_star"]