[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-81759":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":12,"openIssues":14,"contributorsCount":14,"subscribersCount":14,"size":14,"stars1d":14,"stars7d":14,"stars30d":14,"stars90d":14,"forks30d":14,"starsTrendScore":14,"compositeScore":15,"rankGlobal":10,"rankLanguage":10,"license":16,"archived":17,"fork":17,"defaultBranch":18,"hasWiki":17,"hasPages":17,"topics":19,"createdAt":10,"pushedAt":10,"updatedAt":26,"readmeContent":27,"aiSummary":28,"trendingCount":14,"starSnapshotCount":14,"syncStatus":13,"lastSyncTime":29,"discoverSource":30},81759,"zenq","malikhan-dev\u002Fzenq","malikhan-dev","ZenQ is an internal DSL for Go that provides expressive querying and streaming pipelines over in-memory and asynchronous data sources. While inspired by the elegance of C# LINQ, it is built with its own identity, design principles, and performance‑oriented architecture—featuring a rich syntax that is native to Go.","",null,"Go",25,2,0,41.43,"MIT License",false,"main",[20,21,22,23,24,25],"collections-framework","domain-specific-language","dsl","fluent-api","integrated-query","internal-dsl","2026-06-12 04:01:35","[![Starstruck](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGitHub-Starstruck-yellow?style=for-the-badge&logo=github)](https:\u002F\u002Fgithub.com\u002Fusers\u002Fmalikhan-dev\u002Fachievements\u002Fstarstruck)\n![Go Version](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGo-1.25+-00ADD8?style=for-the-badge&logo=go)\n![Tests](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Ftests-passing-brightgreen?style=for-the-badge&logo=go)\n![Coverage](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fcoverage-75%25-brightgreen?style=for-the-badge)\n![Maintained](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fmaintained-yes-brightgreen?style=for-the-badge)\n![License](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-green?style=for-the-badge)\n![Version](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fversion-1.7.2.1-blue?style=for-the-badge)\n\n\n\n\n# Zen-Q (zenq)\n\n**Expressive, Polymorphic Queries with Streaming Capabilities and a User-Friendly API, inspired by LINQ.** \n \n```go\n\n\tctx, cancel := context.WithCancel(context.Background())\n\n\tdefer cancel()\n\n\tvar CsvStreamConfig contracts.CsvStreamConf[customer]\n\n    CsvStreamConfig.StreamHeaders = false\n\n\tCsvStreamConfig.FilePath = \"users_data2.csv\"\n\n\tif stream:= streams.FromCsv(ctx, CsvStreamConfig); stream.Initiated{\n\n\t\tCsvData := stream.FilterStream(func(c customer) bool {\n\t\t\treturn c.Index > 0\n\t\t}).TakeAll()\n\n\n\t} \n\n\t\n\n   var jsonStreamConfig contracts.JsonStreamConf\n\n   jsonStreamConfig.FilePath = \"users_data.json\"\n\n\n\tif jsonStream:= FromJsonArr[User](ctx, jsonStreamConfig.StreamConf); jsonStream.Initiated {\n\n\t\t   JsonData := jsonStream.FilterStream(func(c User) bool {\n\t\t\t\treturn c.ID > 0\n\t\t\t})\n\t\t\n\t\t\n\t\t  for v := range JsonData.Channel {\n\t\t\t\ttime.Sleep(time.Millisecond * 10)\n\t\t\t\tfmt.Println(\" value: \", v)\n\t\t  }\n\t} \n\n\n‌\n```\n\n\n\n\n# License (MIT)\n\nThis library was written and designed by Mohammadreza Malikhan. The source code is free to use with proper attribution. This project is licensed under the MIT License (see the `LICENSE` file for details).\n\n# Intro\n\nzenq is a DSL (Domain Specific Language) for Go that helps you filter, search, validate, process, and stream your data in a fluent and readable way. It is inspired by LINQ in C# and Streams in Java, while staying practical for Go developers. make sure you review the benchmarks section at the end of this document. \n\nAt its core, zenq is a modular library. Currently, it has two modules: **Collections** and **Streams**. streams used to initiate communications with async data-sources, such as a csv file. \n\nThere are two ways of processing collections:\n1. Using default APIs.\n2. Using the advanced collection query engine known as **Thor**. \n\nThor is designed and architected to provide the maximum performance possible. It uses the operation fusion pattern to provide maximum speed and run the entire query chain in a single execution unit. Streams, on the other hand, use famous Golang concepts such as channels and goroutines to allow the user to stream data while respecting the cancellation concepts of Go. at the moment zenq operations allowed on various data sources, in-memory slices, channels, csv files and json files. more and more data-sources will be supported soon.\n\nHere are some examples:\n\n```go\n\u002F\u002F A streaming example\nctx, cancel := context.WithCancel(context.Background())\ndefer cancel()\n\ncount := 0\nbuffer_size := 10\n\nfor v := range FromData[ComplexObjectToSearch](ctx, items).FilterStream(func(search ComplexObjectToSearch) bool {\n \treturn search.Id > 2\n\t}).TakeAll() {\n\n\t\tfmt.Println(v)\n\t\tcount++\n\t\tif count == 100000 {\n\t\t\tcancel()\n\t\t\tbreak\n\t\t}\n\t}\n```\n\n``` go\n\u002F\u002F Grouping Collections using the Thor engine\n\n    collections.Group[bool, ComplexObjectToSearch](\n        collections.From(items).Where(func(search ComplexObjectToSearch) bool {\n            return search.Age > 20\n        }),\n        func(item ComplexObjectToSearch) bool {\n            return item.Flag\n        },\n    ).Collect()\n\n```\n\n``` go\n\u002F\u002F The default APIs for collections\ncollections.From(items).Where(\"Name\", \"John\").Where(\"Flag\", true).First().Collect()\n```\n\n\n## Installation\n\n``` bash\ngo get github.com\u002Fmalikhan-dev\u002Fzenq@latest\n\ngo mod tidy\n```\n\n## Default Collections API\n\nimport path\n\n``` go\ncollections  \"github.com\u002Fmalikhan-dev\u002Fzenq\u002Fcollections\"\n```\n\n\n### `Queryable[T]`\n\n`Queryable[T]` is the core type passed between chained operations such as `Where`, `First`, `FirstOrDefault`, `All`, and `AllOrDefault`.\n\nIt wraps:\n- A data slice: `[]T`\n- An error slice: `[]error`\n\nCollectors unwrap this type into concrete results.\n\n```go\ntype Queryable[T any] struct {\n    Items []T\n    Err   []OpError\n}\n```\n\n---\n\n### `From([]T)`\n\n`From([]T)` creates a `Queryable[T]` from a slice and is usually the starting point of a query chain. It accepts a slice of `[]T` and returns a pointer to `Queryable[T]`.\n\n---\n\n### `Where()`\n\n`Where(fieldName, fieldValue)` filters a slice using a field name and value.\n- `fieldName` must be a `string`.\n- `fieldValue` can be any type, but it must exactly match the actual type of the target field.\n\nThis function modifies the current `Queryable[T]` and returns the same pointer for further chaining.\n\n``` go\n_, err2 := From(items).Where(\"Name\", \"John\").Where(\"Flag\", true).FirstOrDefault().Collect()\n```\n\n**Important:** The field value must be exactly the same type as the struct field.\nFor example, if the field type is `uint32`, you must pass `uint32(2)` instead of `2`.\n\n``` go\n_, err := From(Examples).Where(\"Id\", uint32(2)).AllOrDefault().Collect()\n```\n---\n\n### `First()` and `FirstOrDefault()`\n\nThese functions return the first item in the current query chain.\n- `First()` panics if no item is found.\n- `FirstOrDefault()` appends an error instead of panicking.\n\nBoth still return a pointer to `Queryable[T]`.\n\n---\n\n### `All()` and `AllOrDefault()`\n\nThese functions return all items in the current query chain.\n- `All()` panics if no item is found.\n- `AllOrDefault()` appends an error instead of panicking.\n\nBoth still return a pointer to `Queryable[T]`.\n\n---\n\n## Collectors\n\n**Available since version `v1.3.2`**\n\nAfter a chained operation such as:\n\n``` go\nzenq.From(data).Where(...).AllOrDefault()\n```\n\nYou can use collectors to unwrap the `Queryable[T]` result into concrete values.\n\n- `Collect()` returns the full result set and errors.\n- `CollectRange(cnt)` returns a limited number of items based on the `cnt` argument, along with errors.\n- `Pipe(buffersize)` (formerly `CollectChan(buffersize)`) collects data and errors using Go channels for large datasets. Available since version `v1.4.0`.\n\n``` go\nres, err := From(items).Where(\"Flag\", true).Filter(func(item ComplexObjectToSearch) bool {\n    return item.Id > 200000\n}).AllOrDefault().CollectRange(500)\n```\n``` go\n\u002F\u002F Using Pipe\nfor item := range From(items).Where(\"Flag\", true).AllOrDefault().Pipe(256) {\n    if item.Err.Code != 0 {\n        t.Error(item.Err)\n    }\n}\n```\n\n``` go\n\u002F\u002F Grouping and Piping\ngroupable := zenq.GroupBy[bool, student](zenq.From(students).AllOrDefault(), \"Present\")\n\nfor item := range groupable.Pipe(0) {\n    for k, v := range item.Value {\n        \u002F\u002F process items\n    }\n}\n\u002F\u002F Changed to Pipe() since v1.4.1\n```\n\n\n`Pipe(size)` returns a new type named `CollectStream`.\n\n``` go\ntype CollectStream[T any] struct {\n    Value T\n    Err   OpError\n}\n\n* If `Err.Code == 0`, it means there is no error.\n* `Pipe()` returns data and errors in a single type, which is `CollectStream`.\n\n```\n---\n\n## Nested Search Example\n\nImagine you have a slice of users, and each user has multiple addresses.\nNow suppose you want to find all users where a specific city exists in their addresses. zenq makes this kind of nested search much easier to express.\n\n``` go\nresults, errors := From(UserList).Filter(func(user Users) bool {\n    return Any(user.Addr, func(address Address) bool {\n        return address.City == \"Karaj\"\n    })\n}).AllOrDefault().Collect()\n\nBy reading this example, you can get a good sense of how the core functions work together in real use cases.\n```\n---\n\n## `Any()`\n\n`Any()` accepts:\n- A slice.\n- A predicate function that returns a boolean.\n\nIt returns `true` if at least one item matches the condition, otherwise `false`. This is especially useful for nested queries.\n\n```go\nresult := Any(items, func(item ComplexObjectToSearch) bool {\n    return item.Flag\n})\n```\n\n---\n\n## `GroupBy()`\n\n`GroupBy()` accepts:\n- A queryable.\n- A string for the property name.\n\nIt groups the data based on the specific key.\n\n``` go\nresult, err := GroupBy[bool, SysUser](From(users), \"Flag\").Collect()\n\nresult, err2 := GroupBy[uint32, SysUser](From(users).Filter(func(user SysUser) bool {\n    return user.Id > 0\n}), \"AuthorityId\").Collect()\n\n```\n\n\n\n# Thor Collection Api\n\nA faster, more Go-idiomatic alternative to the default collections API is to use the **Thor** engine to query your data. The Thor engine uses the operator fusion pattern to ensure maximum speed and a single execution unit.\n\n\nimport path\n``` go\ncollections \"github.com\u002Fmalikhan-dev\u002Fzenq\u002Fcollections\u002FThor\"\n```\n\n### Core Concepts:\n\n**`CollectionCompiledQueryable[T]`**: After each chain of operation, we use this type as a contract (much like `Queryable` in the default collections API).\n\n   \n**`AssertCompiledQueryable[T]`**: In our query chains, if we want to assert the result like the `Any()` operator, this is the output type.\n\n\n**`GroupCompiledQueryable[K, T]`**: After a grouping operation, the returning type is `GroupCompiledQueryable`.\n\n\nAll three types nest `CompiledQueryable[T]` inside them. `CompiledQueryable` represents the result of the operation in the `Items` property and the list of operators.\n\n\n``` go\ntype CompiledQueryable[T any] struct {\n    Operators []zenqOperator[T]\n    Items     *[]T\n}\n```\n\nThor Engine APIs are as follows:\n\n**`From[T any]`**: Accepts a slice of `[]T` and returns a `*CollectionCompiledQueryable[T]` to initiate a query chain.\n\n  \n**`Where[T any]`**: Accepts a function `func(T) bool` as an argument, filters the collection, and returns a `*CollectionCompiledQueryable[T]`.\n\n  \n**`Collect()`**: Collects the result and returns the `CollectionCompiledQueryable[T]` which holds the data.\n\n  \n\n**Example:**\n\n``` go\nresult := collections.From(items).Where(func(search ComplexObjectToSearch) bool {\n    return search.Name == \"Jane\" && search.Flag == false\n}).Collect()\n\nresult2 := collections.From(result).Any(func(search ComplexObjectToSearch) bool {\n    return (search.Name != \"Jane\") || (search.Flag != false)\n}).Assert()\n\nif result2 {\n    t.Error(\"result should be false\")\n}\n```\n\n**`Group` and `Collect`**: The `Group` function expects a `CompiledQueryable[T]` as an argument and a Key Selector function. For collecting the result of a group, we can use the `collections.Collect()` function.\n\nA grouping example: filtering users whose age is greater than 20 and grouping them by their presence:\n\n``` go\nres := \n    collections.Group[bool, ComplexObjectToSearch](\n        collections.From(items).Where(func(search ComplexObjectToSearch) bool {\n            return search.Age > 20\n        }),\n        func(item ComplexObjectToSearch) bool {\n            return item.Flag\n        },\n    ).Collect()\n\n\nfmt.Println(res.Items[false][1])\nfmt.Println(res.Items[true][1])\n\n```\n\n\n**`Assert()`**: Asserts the collection on a given criteria.\n\n``` go\nresult2 := collections.From(result).Any(func(search ComplexObjectToSearch) bool {\n    return (search.Name != \"Jane\") || (search.Flag != false)\n}).Assert()\n\n```\n\n\n\n## Nested Search Example (Thor Api)\n\nImagine you have a slice of users, and each user has multiple addresses.\nNow suppose you want to find all users where a specific city exists in their addresses. how can we make such query using Thor API?\n\n``` go\n\n\tres :=\n\t\tcollections.From(UserList).Where(func(user Users) bool {\n\n\t\t\treturn collections.From(user.Addr).Any(func(address Address) bool {\n\t\t\t\treturn address.City == \"Karaj\"\n\t\t\t}).Assert()\n\n\t\t}).Collect()\n\n\tfmt.Println(res)\n\n```\n---\n\n\n# zenq Stream API\n\nWhen dealing with large datasets, it is not always recommended to collect everything into memory using the traditional `Queryable` execution model.\n\nzenq provides a Stream API that allows data to be processed incrementally as it flows through a pipeline. Also, streams can be executed with a compiled mode mechanism which is 35% faster than regular streams.\n\nCurrently there are 4 adapters available to initiate a stream:\n\n\n\nimport path\n``` go\n\nstreams  \"github.com\u002Fmalikhan-dev\u002Fzenq\u002Fstreams\"\n\n```\n\n\n## FromData\n\nCreates a stream from in-memory data.\n\n**Args:**\n1. A context to manage cancellation.\n2. A slice of objects.\n   \n\n## FromChannel\n\nCreates a stream from an existing Go channel.\n\n**Args:**\n1. A context to manage cancellation.\n2. A read channel of `T`.\n\n\n## FromCsv\n\nCreates a stream from a specific csv file. can perform filters on the stream of data.\n\n**Args:**\n\n1. A context to manage cancellation\n2. A contracts.CsvStreamConf[T] type that configures how the stream will initiate.\n\n- contracts.CsvStreamConf[T] contains following properties:\n\n``` go\n\t\ttype StreamConf struct {\n\t\t\tFilePath string\n\t\t\tBufferSize int\n\t\t\tParseErrorCallback func([]error, int)\n\t\t\tItemCount int\n\t\t}\n\t\ttype CsvStreamConf[T any] struct {\n\t\t\tParser        func(row []string) (T, []error)\n\t\t\tStreamHeaders bool\n\t\t\tStreamConf\n\t\t}\n\n```\n  1- A parser thats responsible to map a csv row to a type. \n  \n  2- A flag Represents that headers of csv should be streamed or not. \n  \n  3- A FilePath of the csv file\n  \n  4 - A BufferSize. atleast 128 recommended.\n  \n  5 - A callback for when the parser cant parse the row and an error occures, other rows will be streamed though, unless user signals the cancelation through the 'context'.\n  \n  6 - An ItemCount for when we want to fetch a limited number of csv rows. use 0 to fetch them all.\n\n\n\n## FromJsonArr\n\nCreates a stream from a specific json file. can perform filters on the stream of data.\n\n**Args:**\n\n1. A context to manage cancellation\n2. A contracts.StreamConf type that configures how the stream will initiate.\n\n ``` go\n\t\ttype StreamConf struct {\n\t\t\tFilePath string\t\t\n\t\t\tBufferSize int\n\t\t\tParseErrorCallback func([]error, int)\n\t\t\tItemCount int **unsupported at the moment**\n\t\t}\n ```\n \n  1 - A FilePath of the csv file\n  \n  2 - A BufferSize. atleast 128 recommended.\n  \n  3 - A callback for when the parser cant parse the row and an error occures, other rows will be streamed though, unless user signals the cancelation through the 'context'.\n  \n  4 - To be supported on the next releases.\n\n  \n# Stream Pipelines\n\nOnce a stream is created, it can be processed using different pipeline stages.\n\n## FilterStream\n\nWorks similarly to `Where()` or `Filter()`, but operates on streamed data.\n\n**Args:**\n1. A function to filter the stream of data (`predicate func(T) bool`).\n\n\n## Throttle\n\nAdds a delay between streamed items.\n\n**Args:**\n1. duration time.Duration.\n\n**Important:**\n- Use e.g., `100 * time.Millisecond`.\n- Use `0` for no delay.\n\n## MapStream\n\nTransforms streamed data into another type.\n\n**Args:**\n1. A context to manage cancellation.\n2. A read channel of `T`.\n3. A mapping function that maps `T` to another type `M`.\n\nIt returns a channel of `M`.\n\n---\n\n\nStreams respect `context.Context` cancellation to:\n- Prevent goroutine leaks.\n- Support early termination.\n- Properly manage pipeline lifecycle.\n\n\n# Initiated Stream\n\nit is strongly recommended that when initiating a stream from an asynch source, check that the stream is actually possible. a go idiomatic stream initiation can be something like:\n\n``` go\n\n\n\tif stream := FromJsonArr[User](ctx, jsonStreamConfig.StreamConf); stream.Initiated {\n\n\t\tdata := stream.FilterStream(func(c User) bool {\n\t\t\treturn c.ID > 0\n\t\t})\n\n\t\tfor v := range data.Channel {\n\t\t\ttime.Sleep(time.Millisecond * 10)\n\t\t\tfmt.Println(\" value: \", v)\n\t\t}\n\t}\n\n```\n\n# Example Of Streams\n\nProcess a Stream From Data\n\n``` go\n\nctx, cancel := context.WithCancel(context.Background())\ndefer cancel()\n\n\nfor v := range FromData[ComplexObjectToSearch](ctx, items).FilterStream(func(search ComplexObjectToSearch) bool {\n\treturn search.Id > 2\n}).Throttle(0).TakeAll() {\n\n}\n\n\n```\n\nprocess stream from a channel\n\n``` go\n\n\n\tchannel := make(chan ComplexObjectToSearch, buffer_size)\n\n\tgo func() {\n\t\tfor i := 0; i \u003C 100; i++ {\n\t\t\tchannel \u003C- ComplexObjectToSearch{\n\t\t\t\tName: \"Jack\",\n\t\t\t\tFlag: true,\n\t\t\t\tId:   i,\n\t\t\t\tAge:  i,\n\t\t\t}\n\t\t}\n\t\tclose(channel)\n\t}()\n\n\tfor v := range FromChannel[ComplexObjectToSearch](ctx, channel).FilterStream(func(complex ComplexObjectToSearch) bool {\n\t\treturn complex.Id > 2\n\t}).Throttle(time.Millisecond * 500).TakeAll() {\n\n\n```\n\n\n## A Real‑World Example of Querying CSV Files\n\nimagine we have a csv file with the following structure. the first 3 rows have wrong values for Index, cause it should be an int, like other rows. \n\nour goal is to read the entire csv file, and then have a groupped object based on the index field (a map[k][T]).\n\nwe also need to filter the rows that their index field is greater than 60. we want to collect streams then use the thor engine to group the objects. if any other errors occures besides those 3 first rows, the operation must be stopped.\n\n\n\n\n\n``` csv\n\tIndex,CustomerId,FirstName,LastName,Company,City,Country,Phone1,Phone2,Email,SubscriptionDate,Website\n    C681dDd0cc422f7,C681dDd0cc422f7,Kelli,Hardy,Petty Ltd,Huangfort,Sao Tome and Principe,020.324.2191x2022,424-157-8216,kristopher62@oliver.com,2020-12-20,http:\u002F\u002Fwww.kidd.com\u002F,\n    C681dDd0cc422f7,C681dDd0cc422f7,Kelli,Hardy,Petty Ltd,Huangfort,Sao Tome and Principe,020.324.2191x2022,424-157-8216,kristopher62@oliver.com,2020-12-20,http:\u002F\u002Fwww.kidd.com\u002F,\n    C681dDd0cc422f7,a940cE42e035F28,Lynn,Pham,\"Brennan, Camacho and Tapia\",East Pennyshire,Portugal,846.468.6834x611,001-248-691-0006,mpham@rios-guzman.com,2020-08-21,https:\u002F\u002Fwww.murphy.com\u002F,\n    60,9Cf5E6AFE0aeBfd,Shelley,Harris,\"Prince, Malone and Pugh\",Port Jasminborough,Togo,423.098.0315x8373,+1-386-458-8944x15194,zachary96@mitchell-bryant.org,2020-12-10,https:\u002F\u002Fwww.ryan.com\u002F,\n    65,aEcbe5365BbC67D,Eddie,Jimenez,Caldwell Group,West Kristine,Ethiopia,+1-235-657-1073x6306,(026)401-7353x2417,kristiwhitney@bernard.com,2022-03-24,http:\u002F\u002Fcherry.com\u002F,\n    65,FCBdfCEAe20A8Dc,Chloe,Hutchinson,Simon LLC,South Julia,Netherlands,981-544-9452,+1-288-552-4666x060,leah85@sutton-terrell.com,2022-05-15,https:\u002F\u002Fmitchell.info\u002F,\n```\n\nHere's How:\n\n\nfirst we define the data type\n\n\n``` go\ntype customer struct {\n\t\tIndex            int\n\t\tCustomerId       string\n\t\tFirstName        string\n\t\tLastName         string\n\t\tCompany          string\n\t\tCity             string\n\t\tCountry          string\n\t\tPhone1           string\n\t\tPhone2           string\n\t\tEmail            string\n\t\tSubscriptionDate string\n\t\tWebsite          string\n\t}\n\n```\n\nthen its time to configure the streaming options. simply create an instance of CsvStreamConf[Customer]. its important to pass along sufficient amount for the buffer size. since we expect the three first rows are corrupted then we configure our Error Callback in a way that if any other errors occures, we signal the stream to stop.\n\n\n```\n\n    ctx, cancel := context.WithCancel(context.Background())\n\n    defer cancel()\n\n    var CsvStreamConfig contracts.CsvStreamConf[customer]  \u002F\u002Fdefine the stream config\n\n\tCsvStreamConfig.StreamHeaders = false\n\n\tCsvStreamConfig.FilePath = \"customers-100.csv\"\n\n\tCsvStreamConfig.BufferSize = 256\n\n\tCsvStreamConfig.ParseErrorCallback = func(err error, i int) {\n\n\t\tfmt.Println(err, \" at\", i)\n\n\t\tif i > 4 {  \u002F\u002F we expect that the first 3 rows have problems and if we have error on other records we want to cancel, please note that headernames is the 1 row\n\t\t\tcancel()\n\t\t}\n\t}\n\n\tCsvStreamConfig.Parser = func(row []string) (customer, error) {\n\t\tindex, err := strconv.Atoi(row[0])\n\t\treturn customer{\n\t\t\tCustomerId:       row[1],\n\t\t\tIndex:            index,\n\t\t\tFirstName:        row[2],\n\t\t\tLastName:         row[3],\n\t\t\tCompany:          row[4],\n\t\t\tCity:             row[5],\n\t\t\tCountry:          row[6],\n\t\t\tPhone1:           row[7],\n\t\t\tPhone2:           row[8],\n\t\t\tEmail:            row[9],\n\t\t\tSubscriptionDate: row[10],\n\t\t\tWebsite:          row[11],\n\t\t}, err\n\t}\n\n\n```\n\n\nthen its time to design our pipelines. we plugin the csv adapter (FromCsv) then we call the filter pipeline so that we can have all the customers that their index field is bigger than 60. then we call TakeAll(), the result is now compatible for the thor engine to start grouping by passing the data to group function. and finally we collect the grouping result. \n\n\n```\n\ndata := FromCsv(ctx, CsvStreamConfig).FilterStream(func(c customer) bool {\n\t\treturn c.Index > 60\n\t}).TakeAll()\n\n\nGroupCollection := TCollection.Group[int, customer](TCollection.From(data), func(t customer) int {\n\t\treturn t.Index\n\t}).Collect()\n\n\nfor k, v := range GroupCollection.Items {\n\t\tfmt.Println(k)\n\t\tfmt.Println(v)\n\t}\n\n```\n\n\nall together this is the final statements\n\n\n``` go\n\n\ttype customer struct {\n\t\tIndex            int\n\t\tCustomerId       string\n\t\tFirstName        string\n\t\tLastName         string\n\t\tCompany          string\n\t\tCity             string\n\t\tCountry          string\n\t\tPhone1           string\n\t\tPhone2           string\n\t\tEmail            string\n\t\tSubscriptionDate string\n\t\tWebsite          string\n\t}\n\n    ctx, cancel := context.WithCancel(context.Background())\n\n\tdefer cancel()\n\n\tvar CsvStreamConfig contracts.CsvStreamConf[customer]  \u002F\u002Fdefine the stream config\n\n\tCsvStreamConfig.StreamHeaders = false\n\n\tCsvStreamConfig.FilePath = \"customers-100.csv\"\n\n\tCsvStreamConfig.BufferSize = 256\n\n\tCsvStreamConfig.ParseErrorCallback = func(err error, i int) {\n\n\t\tfmt.Println(err, \" at\", i)\n\n\t\tif i > 3 {    \u002F\u002F we expect that the first 3 rows have problems and if we have error on other records we want to cancel\n\t\t\tcancel()\n\t\t}\n\t}\n\n\tCsvStreamConfig.Parser = func(row []string) (customer, error) {\n\t\tindex, err := strconv.Atoi(row[0])\n\t\treturn customer{\n\t\t\tCustomerId:       row[1],\n\t\t\tIndex:            index,\n\t\t\tFirstName:        row[2],\n\t\t\tLastName:         row[3],\n\t\t\tCompany:          row[4],\n\t\t\tCity:             row[5],\n\t\t\tCountry:          row[6],\n\t\t\tPhone1:           row[7],\n\t\t\tPhone2:           row[8],\n\t\t\tEmail:            row[9],\n\t\t\tSubscriptionDate: row[10],\n\t\t\tWebsite:          row[11],\n\t\t}, err\n\t}\n\n\ndata := FromCsv(ctx, CsvStreamConfig).FilterStream(func(c customer) bool {\n\t\treturn c.Index > 0\n\t}).TakeAll()\n\n\nGroupCollection := TCollection.Group[int, customer](TCollection.From(data), func(t customer) int {\n\t\treturn t.Index\n\t}).Collect()\n\n\nfor k, v := range GroupCollection.Items {\n\t\tfmt.Println(k)\n\t\tfmt.Println(v)\n\t}\n\n\n```\n\n\n## A Real‑World Example of Querying JSON Files\nimagine we want to read a json file and stream its data in real-time. we dont want to wait for all the rows of our json array to be read. and its required that we skip any errors that might happens at the first row. \n\n\n``` go\n[\n  {\n    \"id\": 1,\n    \"username\": \"user_1\",\n    \"email\": \"user_1@example.com\",\n    \"is_active\": true,\n    \"created_at\": \"2026-05-23T13:15:06.534680\"\n  },\n  {\n    \"id\": 2,\n    \"username\": \"user_2\",\n    \"email\": \"user_2@example.com\",\n    \"is_active\": false,\n    \"created_at\": \"2026-05-22T13:15:06.534750\"\n  },\n  {\n    \"id\": 3,\n    \"username\": \"user_3\",\n    \"email\": \"user_3@example.com\",\n    \"is_active\": true,\n    \"created_at\": \"2026-05-21T13:15:06.534758\"\n  },\n  {\n    \"id\": 4,\n    \"username\": \"user_4\",\n    \"email\": \"user_4@example.com\",\n    \"is_active\": true,\n    \"created_at\": \"2026-05-20T13:15:06.534763\"\n  }\n]\n```\n\n\n``` go\n\ntype User struct {\n\t\tID        int    `json:\"id\"`\n\t\tUsername  string `json:\"username\"`\n\t\tEmail     string `json:\"email\"`\n\t\tIsActive  bool   `json:\"is_active\"`\n\t\tCreatedAt string `json:\"created_at\"`\n\t}\n\n\tctx, cancel := context.WithCancel(context.Background())\n\n\tdefer cancel()\n\n\tvar jsonStreamConfig contracts.JsonStreamConf\n\n\tjsonStreamConfig.FilePath = \"users_data.json\"\n\n\tjsonStreamConfig.BufferSize = 256\n\n\tjsonStreamConfig.ParseErrorCallback = func(err []error, i int) {\n\n\t\tfmt.Println(err, \" at\", i)\n\n\t\tif i > 1 {\n\t\t\tcancel()\n\t\t}\n\t}\n\n\tdata := FromJsonArr[User](ctx, jsonStreamConfig.StreamConf).FilterStream(func(c User) bool {\n\t\treturn c.ID > 0\n\t})\n\n\tfor v := range data.Channel {\n\t\ttime.Sleep(time.Millisecond * 10)\n\t\tfmt.Println(\" value: \", v)\n\t}\n\n```\n\n\n# Compiled Streams\nthe main difference between streams and compiled streams is that the compiled streams starts the streaming from a single execution unit. while the streams pass around the data after each pipelines. in the following example we initiate a compilable stream using the method CompileFromQueryable, which accepts a slice, then we used filter pipeline to filter it, after that we called CompileStream which is our execution unit, we can remove the throttle pipeline if we dont need any delays. please note that this section is an experimental part of the project.\n\n``` go\n\n\tctx, cancel := context.WithCancel(context.Background())\n\n\tdefer cancel()\n\n\tfor i := range Throttle(ctx, CompileStream(ctx, Filter(CompileFromQueryable(items), func(student ComplexObjectToSearch) bool {\n\t\treturn !student.Flag\n\t})), time.Duration(250*time.Millisecond)) {\n\t\tfmt.Println(i)\n\t}\n\n```\n\n\n\n\n## benchmark\n\nin a slice of 50,000,000 users it took less than 2 seconds just to filter them and around 4 seconds to filter then group the items. to achieve this result we used thor collections api, not the default api collections.\n\n\n\n\n\n\n## Project Status\n\nzenq is actively evolving, and more operators, examples, and documentation are on the way.\n\nIf you find it useful, feel free to star the repository (it motivates us) and follow future updates!\n","ZenQ 是一个用于 Go 语言的内部 DSL，旨在提供对内存中及异步数据源的表达式查询和流处理管道。该项目受到 C# LINQ 的启发，但根据 Go 语言特性进行了重新设计，以实现更高效的数据处理。其核心功能包括流畅的 API 设计、支持多态查询以及内置了针对 CSV 和 JSON 文件等数据源的流处理能力。ZenQ 适合需要在 Go 项目中进行复杂数据操作或构建高性能数据处理流水线的应用场景。","2026-06-01 03:56:05","CREATED_QUERY"]