[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-81737":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":12,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":13,"stars30d":13,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":15,"rankGlobal":10,"rankLanguage":10,"license":16,"archived":17,"fork":17,"defaultBranch":18,"hasWiki":19,"hasPages":17,"topics":20,"createdAt":10,"pushedAt":10,"updatedAt":28,"readmeContent":29,"aiSummary":30,"trendingCount":14,"starSnapshotCount":14,"syncStatus":31,"lastSyncTime":32,"discoverSource":33},81737,"pgstream","mujib77\u002Fpgstream","mujib77","Real-time PostgreSQL WAL reader built in  GO - captures INSERT, UPDATE, DELETE using logical replication","",null,"Go",31,1,0,0.9,"MIT License",false,"main",true,[21,22,23,24,25,26,27],"cdc","database-internals","golang","logical-replication","mermaidjs","postgresql","wal","2026-06-12 02:04:19","![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-blue)\n![CI](https:\u002F\u002Fgithub.com\u002Fmujib77\u002Fpgstream\u002Factions\u002Fworkflows\u002Fci.yml\u002Fbadge.svg)\n# pgstream\n\nA real-time PostgreSQL WAL reader built in Go.\nCaptures every INSERT, UPDATE, and DELETE from\nyour Postgres database as it happens.\n\n## What is pgstream?\n\nEvery time you insert, update, or delete a row in\nPostgreSQL, Postgres writes that change to a file\ncalled the WAL (Write Ahead Log) before touching\nthe actual data. This is how Postgres makes sure\nnothing gets lost if the server crashes.\n\npgstream taps into that log and streams every change\nout in real time. You can see exactly what changed,\nin which table, and what the old and new values were.\n\nThis is the same mechanism that powers production\nCDC (Change Data Capture) tools like Debezium and\npglogical used by companies running Postgres at scale.\n\n## How it works\n\n**Replication Slot**\n\npgstream creates a replication slot in your Postgres\ndatabase. Think of it as a bookmark in the WAL stream.\nPostgres tracks your position and makes sure no changes\nget deleted until pgstream has read them.\n\n**pgoutput**\n\nWAL records are stored in raw binary. pgoutput is a\nbuilt-in Postgres plugin that decodes that binary into\nreadable messages like INSERT, UPDATE, DELETE along\nwith the actual row data.\n\n**LSN (Log Sequence Number)**\n\nEvery record in the WAL has a unique position called\nan LSN. pgstream tracks the LSN of every event so you\nalways know exactly where in the stream each change\ncame from.\n\n## Architecture\n\n```mermaid\nflowchart TD\n    A[Postgres Database] --> B[WAL - Write Ahead Log]\n    B --> C[Replication Slot + pgoutput]\n    C --> D[Connector - connects to WAL stream]\n    D --> E[Decoder - decodes raw bytes into events]\n    E --> F[Handler - outputs clean JSON events]\n```\n\n## Project Structure\n\n```\npgstream\u002F\n├── main.go\n├── config\u002F\n│   └── config.go\n├── internal\u002F\n│   ├── connector\u002F\n│   │   └── connector.go\n│   ├── decoder\u002F\n│   │   └── decoder.go\n│   ├── handler\u002F\n│   │   └── handler.go\n│   └── models\u002F\n│       └── event.go\n```\n\n## Setup\n\nPrerequisites:\n- PostgreSQL with logical replication enabled\n- Go 1.21 or higher\n\n**1. Enable logical replication**\n\nIn `postgresql.conf`:\n\n```\nwal_level = logical\n```\n\nRestart Postgres after changing this.\n\n**2. Clone the repo**\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fmujib77\u002Fpgstream\ncd pgstream\n```\n\n**3. Create a .env file**\n\n```\nDATABASE_URL=postgres:\u002F\u002Fuser:password@localhost:5432\u002Fdbname?replication=database\nSLOT_NAME=pgstream_slot\nPUBLICATION_NAME=pgstream_pub\n```\n\n**4. Create publication in your database**\n\n```sql\nCREATE PUBLICATION pgstream_pub FOR ALL TABLES;\n```\n\n**5. Install dependencies**\n\n```bash\ngo mod tidy\n```\n\n**6. Run**\n\n```bash\ngo run main.go\n```\n\n## Example Output\n\n```\nconnecting to postgres...\nconnected!\nreplication slot created: pgstream_slot\nstarting replication...\nlistening for changes...\n\n[INSERT] table=users lsn=0\u002F16C752F8\ndata={\n  \"email\": \"wal@gmail.com\",\n  \"id\": \"1\",\n  \"name\": \"Mujib\"\n}\n\n[UPDATE] table=users lsn=0\u002F16C75410\nold={\n  \"name\": \"Mujib\"\n}\nnew={\n  \"name\": \"Bum\"\n}\n\n[DELETE] table=users lsn=0\u002F16C754A8\ndata={\n  \"email\": \"wal@gmail.com\",\n  \"id\": \"1\",\n  \"name\": \"Bum\"\n}\n```\n\n## Tech Stack\n\n- Go\n- pglogrepl - Postgres logical replication protocol\n- pgx - Postgres driver for Go\n- PostgreSQL logical decoding with pgoutput plugin\n","pgstream 是一个用 Go 语言编写的实时 PostgreSQL WAL 读取器，能够捕获数据库中的 INSERT、UPDATE 和 DELETE 操作。它通过逻辑复制机制直接从 PostgreSQL 的写前日志（WAL）中读取数据变更，并将其转换为可读的 JSON 格式输出。该项目利用了 Postgres 内置的 pgoutput 插件来解码二进制 WAL 记录，并通过 LSN（日志序列号）跟踪每个事件的位置，确保数据变更的准确性和完整性。适用于需要实时监控和响应数据库变化的场景，如数据同步、审计日志生成或触发基于数据变更的业务流程。",2,"2026-06-11 04:06:12","CREATED_QUERY"]