[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-92391":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":13,"contributorsCount":13,"subscribersCount":13,"size":13,"stars1d":13,"stars7d":13,"stars30d":15,"stars90d":13,"forks30d":13,"starsTrendScore":13,"compositeScore":16,"rankGlobal":10,"rankLanguage":10,"license":17,"archived":18,"fork":18,"defaultBranch":19,"hasWiki":20,"hasPages":18,"topics":21,"createdAt":10,"pushedAt":10,"updatedAt":22,"readmeContent":23,"aiSummary":24,"trendingCount":13,"starSnapshotCount":13,"syncStatus":25,"lastSyncTime":26,"discoverSource":27},92391,"qv-lite","mababaNiubi\u002Fqv-lite","mababaNiubi","Lightweight Edge KV Time-Series Database Engine.","",null,"Go",64,0,51,13,41.3,"MIT License",false,"main",true,[],"2026-07-22 04:02:06","\u003Cp align=\"center\">\n  \u003Cimg src=\".\u002Flogo.png\" alt=\"qv-lite logo\" width=\"200\" \u002F>\n\u003C\u002Fp>\n\n\u003Ch1 align=\"center\">qv-lite\u003C\u002Fh1>\n\n\u003Cp align=\"center\">\n  \u003Cstrong>Lightweight Edge KV Time-Series Database Engine.\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\".\u002FREADME_CN.md\">中文\u003C\u002Fa> | \u003Cstrong>English\u003C\u002Fstrong>\n\u003C\u002Fp>\n\n## Table of Contents\n\n- [Features](#features)\n- [Installation](#installation)\n- [Usage](#usage)\n  - [Open & Close](#open--close)\n  - [Write](#write)\n  - [Query](#query)\n  - [Table Management](#table-management)\n  - [Data Types](#data-types)\n  - [Aggregation](#aggregation)\n  - [Condition Filtering](#condition-filtering)\n- [Configuration](#configuration)\n- [Compression Algorithms](#compression-algorithms)\n- [Data Encoding](#data-encoding)\n- [Dependencies](#dependencies)\n\n## Features\n\n- **Embedded & Lightweight** — Designed for edge gateways, industrial controllers, and IoT devices with limited CPU\u002Fmemory\u002Fdisk.\n- **Minimal & Self-Contained** — Compact, auditable codebase with minimal third-party dependencies. Straightforward on-disk layout: one metadata file per table, timestamp-named segment files, and WAL logs — no heavy database engine, easy to inspect, back up, and migrate.\n- **High Write Throughput** — Single-threaded synchronous design. Achieves **8,000,000+ points\u002Fsecond** single-node write performance.\n- **Adaptive Type Encoding** — Auto-detects input data structure at runtime and selects the optimal compression codec per data type for maximum storage efficiency.\n- **High Compression Ratio** — Compact data encoding plus secondary block compression for extreme storage savings.\n- **Multi-Table Support** — Manage multiple independent tables, each with its own schema definition and column set.\n- **Downsampling Queries** — Sliding-window aggregation (avg \u002F min \u002F max) for long time-range queries.\n- **Data Expiration** — Configurable time-based automatic data eviction.\n- **Dedup & Min Interval** — Configurable deduplication window and minimum write interval to prevent duplicate data.\n- **Block-Level Index** — Binary-search block index for fast time-range filtering without scanning irrelevant data.\n\n## Installation\n\n```bash\ngo get github.com\u002FmababaNiubi\u002Fqv-lite\u002Ftsdb\n```\n\n## Usage\n\n### Open & Close\n\n```go\nimport (\n    \"context\"\n    \"github.com\u002FmababaNiubi\u002Fqv-lite\u002Ftsdb\"\n)\n\ndb, err := tsdb.Open(tsdb.Config{\n    Path: \".\u002FqvLite-data\",\n}, context.Background())\nif err != nil {\n    panic(err)\n}\ndefer db.Close()\n```\n\n`Open` creates or opens a database at the given path. The `default` table is auto-created on first use. `Close()` flushes all buffered data and releases resources.\n\n### Write\n\n```go\nimport (\n    \"time\"\n    \"github.com\u002FmababaNiubi\u002Fvariant\"\n)\n\n\u002F\u002F Write to the default table (tableName = \"\" or \"default\")\nwritten, err := db.Write(\"\", \"sensor_temp\", time.Now().UnixNano(), variant.New(25.6))\n\n\u002F\u002F Write to a named table\nwritten, err := db.Write(\"metrics\", \"cpu_usage\", time.Now().UnixNano(), variant.New(42.5))\n```\n\n- `tableName` — empty string or `\"default\"` writes to the auto-created default table.\n- `tag` — the time-series identifier (e.g. sensor name, metric key).\n- `timestamp` — Unix timestamp in **nanoseconds**.\n- `value` — use `variant.New(v)` to wrap any supported Go value.\n- Returns `(true, nil)` if written; `(false, nil)` if skipped by dedup or min-interval rules.\n\n### Query\n\n```go\n\u002F\u002F Range query with downsampling — best for long time ranges\npoints, err := db.Query(\"default\", \"sensor_temp\",\n    time.Now().Add(-1*time.Hour).UnixNano(), \u002F\u002F startTime (ns)\n    time.Now().UnixNano(),                   \u002F\u002F endTime (ns)\n    1000,                                    \u002F\u002F maxNumber of returned points\n    tsdb.AvgFusion,                          \u002F\u002F aggregation mode\n    nil,                                     \u002F\u002F condition filter (nil = no filter)\n)\n\n\u002F\u002F Fetch all raw data (no downsampling)\npoints, err := db.QueryAll(\"default\", \"sensor_temp\",\n    time.Now().Add(-30*time.Minute).UnixNano(),\n    time.Now().UnixNano(),\n    nil,\n)\n\n\u002F\u002F Get the latest value for a tag\npoint, err := db.QueryLatest(\"default\", \"sensor_temp\")\nif point != nil {\n    fmt.Printf(\"latest: time=%d, value=%v\\n\", point.Tms, point.V)\n}\n```\n\n| Method | Description |\n|--------|-------------|\n| `Query(tableName, tag, startTime, endTime, maxNumber, polymerization, cond)` | Range query. For spans > 1 hour, returns up to `maxNumber` downsampled points. For spans ≤ 1 hour, returns all raw points directly. |\n| `QueryAll(tableName, tag, startTime, endTime, cond)` | Returns all raw data points in the range without limit. |\n| `QueryLatest(tableName, tag)` | Returns the most recent point for the given tag. |\n\nAll timestamps are in **nanoseconds** (UnixNano). `maxNumber` defaults to 10000 when set to 0.\n\n### Table Management\n\n```go\n\u002F\u002F Create a simple single-column table — highest write performance\nerr := db.CreateTable(tsdb.TableInfo{\n    ColumnAttribute: tsdb.ColumnAttribute{\n        Name: \"device\",\n        Desc: \"attributes\",\n        Type: tsdb.ColumnTypeFloat,\n        FloatPrecision: 2,\n    },\n})\n\n\u002F\u002F Create a multi-column table\nerr = db.CreateTable(tsdb.TableInfo{\n    ColumnAttribute: tsdb.ColumnAttribute{\n        Name: \"metrics\",\n        Desc: \"Dev01.CPU\",\n        Type: tsdb.ColumnTypeStructure,\n        Structure: []tsdb.ColumnAttribute{\n            {Name: \"value\",   Type: tsdb.ColumnTypeFloat, FloatPrecision: 2}, \u002F\u002F auto-calculate precision\n            {Name: \"quality\", Type: tsdb.ColumnTypeInt},\n            {Name: \"status\",  Type: tsdb.ColumnTypeString},\n        },\n    },\n})\n\n\u002F\u002F Create an adaptive-schema table (auto-discovers fields at runtime, slower than fixed schema)\nerr = db.CreateTable(tsdb.TableInfo{\n    ColumnAttribute: tsdb.ColumnAttribute{\n        Name: \"events\",\n        Desc: \"dynamic event log\",\n        Type: tsdb.ColumnTypeUnknown, \u002F\u002F adaptive — discovers field format at runtime\n    },\n})\n```\n\nTable metadata is persisted in `{db_path}\u002Ftable.json` and automatically reloaded on next `Open`.\n\n### Data Types\n\n| Constant | Type | Description |\n|----------|------|-------------|\n| `ColumnTypeUnknown` (0) | Adaptive | Auto-detect nested structure at runtime |\n| `ColumnTypeInt` (1) | Integer | Signed 64-bit integer |\n| `ColumnTypeFloat` (2) | Float | 64-bit float with configurable decimal precision |\n| `ColumnTypeString` (3) | String | UTF-8 string |\n| `ColumnTypeBool` (4) | Boolean | true \u002F false |\n| `ColumnTypeJson` (5) | JSON | Arbitrary nested variant |\n| `ColumnTypeStructure` (6) | Fixed Struct | Pre-defined column schema |\n\n### Aggregation\n\n| Constant | Value | Description |\n|----------|-------|-------------|\n| `AvgFusion` | 0 | Sliding-window average |\n| `MinFusion` | 1 | Sliding-window minimum |\n| `MaxFusion` | 2 | Sliding-window maximum |\n\n### Condition Filtering\n\nThe `cond` parameter in query methods supports filtering data points by value:\n\n```go\n\u002F\u002F Equality filter — only return points where value == \"ok\"\ncond := tsdb.Condition{\n    Operator: tsdb.OpEqual,\n    Value:    variant.New(\"ok\"),\n}\npoints, err := db.QueryAll(\"default\", \"status\", startTs, endTs, cond)\n\n\u002F\u002F Logical AND — combine multiple conditions\nlogicalCond := tsdb.LogicalCondition{\n    Op:   tsdb.LogicalAnd,\n    Cond: []any{\n        tsdb.Condition{Operator: tsdb.OpGreaterThan, Value: variant.New(80)},\n        tsdb.Condition{Operator: tsdb.OpLessThan,    Value: variant.New(100)},\n    },\n}\npoints, err := db.QueryAll(\"default\", \"cpu\", startTs, endTs, logicalCond)\n```\n\n## Configuration\n\n### Config\n\n| Field | Type | Default | Description |\n|-------|------|---------|-------------|\n| `Path` | `string` | `\".\u002FqvLite-data\"` | Database data storage path |\n| `WalConfig` | `WalConfig` | — | WAL settings (see below) |\n| `MaxSegmentSize` | `int64` | `67108864` (64MB) | Maximum segment file size in bytes |\n| `MaxSegmentTimeInterval` | `int64` | `0` (unlimited) | Maximum segment time span in seconds |\n| `MaxStorageTime` | `int64` | `3600` (1 hour) | Reject writes whose timestamps are too far ahead of current time |\n| `ExpirationMinuteTime` | `int64` | `0` (disabled) | Auto-evict data older than this many minutes (checked on each write) |\n| `DedupWindowMs` | `int64` | `0` (disabled) | Dedup window in ms — skips writes with the same value for the same tag within this window |\n| `MinIntervalMs` | `int64` | `0` (disabled) | Minimum interval between consecutive writes in ms — writes arriving too quickly are skipped |\n| `SecondaryCompressionName` | `string` | `\"zstd\"` | Block compression: `\"zstd\"`, `\"lz4\"`, `\"snappy\"`, `\"gzip\"`, `\"none\"` |\n\n### WalConfig\n\n| Field | Type | Default           | Description |\n|-------|------|-------------------|-------------|\n| `MaxFileSize` | `int64` | `67108864` (64MB) | Maximum WAL in-memory cache size in bytes |\n| `MaxFileNumber` | `int` | —                 | Maximum number of WAL files |\n| `CloseBuffer` | `bool` | `false`           | Whether to disable in-memory WAL buffering |\n| `MaxBufferBatchSize` | `int` | `4096`            | Max entries to buffer before sorting and flushing |\n\n**`CloseBuffer` behavior:**\n\n| Scenario | `CloseBuffer = true` | `CloseBuffer = false` |\n|----------|---------------------|----------------------|\n| Out-of-order writes | Rejected for the same key | Allowed within `MaxBufferBatchSize` batch window |\n| Write\u002Fquery performance | Lower | Higher |\n| Crash safety | Data safe | May lose buffered data |\n| Memory usage | Typically within `MaxFileSize` | ~3–4× `MaxFileSize` |\n\n> Control database memory usage by tuning `MaxFileSize`.\n\n## Compression Algorithms\n\n| Data Type | Strategy |\n|-----------|----------|\n| Timestamp | delta-of-delta + scaling + simple8b \u002F RLE |\n| Integer | delta + zigzag + simple8b \u002F RLE |\n| Float | XOR-delta with mantissa truncation based on decimal precision |\n| Boolean | bit-packed (8 per byte) or all-true\u002Fall-false RLE |\n| String | uvarint length-prefixed concatenation + Snappy |\n| JSON | variant binary serialization + LZ4 |\n| Fixed Struct | column-by-column encoding, each with typed sub-encoder |\n| Adaptive Struct | auto-discovers columns from Maps, recursive nesting, self-describing |\n\n## Data Encoding\n\n### On-Disk Layout\n\n```\n{db_path}\u002F\n  table.json              # Table metadata\n  default\u002F                # Default table directory\n    meta.json             # Tag dictionary\n    data\u002F                 # Segment data files\n      1234567890.tsb      # Segment (BlockFile format)\n      1234567890.tsb.idx  # Segment block index\n    wal\u002F                  # WAL directory\n      1234567890.wal      # WAL log file\n```\n\n### Segment File (.tsb) — BlockFile\n\nEach `.tsb` file is a **BlockFile** containing compressed data blocks plus a block-level index at the end:\n\n```\n┌──────────────────────────────────────────────────────┐\n│                    BlockFile (.tsb)                    │\n├───────────────┬──────────────────────────────────────┤\n│ FileHeader    │ Magic, version, tx state, codec, etc. │\n├───────────────┼──────────────────────────────────────┤\n│ Physical      │ ┌ LenPrefix ───────────────────────┐ │\n│ Block 0       │ │ CompressedPayload                │ │\n│               │ │  = N logical blocks concatenated  │ │\n│               │ │  then compressed together         │ │\n├───────────────┤ └──────────────────────────────────┘ │\n│ Physical      │ ┌ LenPrefix ───────────────────────┐ │\n│ Block 1       │ │ CompressedPayload                │ │\n│               │ │  ...                              │ │\n├───────────────┤ └──────────────────────────────────┘ │\n│     ...       │                                       │\n├───────────────┼──────────────────────────────────────┤\n│ Block Index   │ Count + IndexEntry × M               │\n│               │  · RawOff  → logical offset           │\n│               │  · CompOff → physical offset          │\n│               │  · CompLen → compressed size          │\n│               │  · Crc32   → checksum                 │\n└───────────────┴──────────────────────────────────────┘\n```\n\nThe **block index** sits at the end of the file and maps each physical block's position to its decompressed logical offset, enabling direct seek to any block.\n\n### Segment Index File (.tsb.idx)\n\nSits alongside each `.tsb` segment for time-range + tag filtering without reading segment data:\n\n```\n┌──────────────────────────────────────────────────┐\n│              Segment Index (.tsb.idx)              │\n├──────────────┬───────────────────────────────────┤\n│ Header       │ Magic, BlockCount, MinTime, MaxTime│\n├──────────────┼───────────────────────────────────┤\n│ Entry [0]    │ TagCode, MinTime, MaxTime,         │\n│              │ Offset, DataSize                   │\n├──────────────┼───────────────────────────────────┤\n│ Entry [1]    │ ...                                │\n├──────────────┼───────────────────────────────────┤\n│     ...      │  (one entry per logical block)     │\n└──────────────┴───────────────────────────────────┘\n```\n\nEach entry records which tag column a logical block belongs to, its time range, and where to find it in the `.tsb` file. Queries binary-search the index to skip irrelevant blocks.\n\n### Logical Block\n\nThe basic unit written by column encoders. Multiple logical blocks are concatenated and compressed into one physical block:\n\n```\n┌──────────────────────────────────────────────────────────┐\n│                    One Logical Block                       │\n├────────────────┬─────────────────────────────────────────┤\n│ SegmentHeader  │          Payload (compressed data)       │\n│                │  ┌─────────────────┬──────────────────┐ │\n│  · TagCode     │  │ CompressedValue │ CompressedTime   │ │\n│  · MinTime     │  │ (type-specific) │ (delta-of-delta) │ │\n│  · MaxTime     │  └─────────────────┴──────────────────┘ │\n│  · DataSize    │                                          │\n│  · CRC32       │                                          │\n└────────────────┴──────────────────────────────────────────┘\n```\n\n### Payload Formats\n\n**Scalar types** (Int, Float, String, Bool, Json):\n\n```\n┌────────────────────┬─────────────────────┬──────────────────┐\n│ ValueByteLength    │ CompressedValueData │ CompressedTime   │\n│  (length prefix)   │ (type encoder)      │ (TimeEncoder)    │\n└────────────────────┴─────────────────────┴──────────────────┘\n```\n\n**Structure types** (fixed \u002F adaptive structs):\n\n```\n┌────────┬──────────┬──────────┬─────┬──────────────────┬──────────────────┐\n│ Marker │ Field0Len│ Field1Len│ ... │ FieldData Concat │ CompressedTime   │\n│ (1B)   │  (each 8B)          │     │ (per-field enc.) │ (TimeEncoder)    │\n└────────┴──────────┴──────────┴─────┴──────────────────┴──────────────────┘\n```\n\nEach field within a struct is independently encoded using its own type-specific compressor, then concatenated. The marker identifies whether the schema is fixed (pre-registered) or adaptive (auto-discovered from Maps).\n\n","qv-lite 是一个轻量级嵌入式键值型时序数据库引擎，专为资源受限的边缘设备设计。它支持高吞吐写入（单节点超800万点\u002F秒）、自适应数据类型编码与高压缩比存储、多表管理、基于时间窗口的聚合查询（avg\u002Fmin\u002Fmax）、块级索引加速时间范围过滤、数据自动过期及去重。采用纯 Go 实现，无外部依赖，磁盘布局简洁（元数据文件+时间戳分段文件+WAL），便于部署、备份与审计。适用于工业网关、IoT终端、嵌入式控制器等边缘侧实时指标采集与短期存储场景。",2,"2026-07-08 04:30:15","CREATED_QUERY"]