[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-73052":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":19,"compositeScore":20,"rankGlobal":10,"rankLanguage":10,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":24,"hasPages":22,"topics":25,"createdAt":10,"pushedAt":10,"updatedAt":46,"readmeContent":47,"aiSummary":48,"trendingCount":16,"starSnapshotCount":16,"syncStatus":49,"lastSyncTime":50,"discoverSource":51},73052,"maestro","Netflix\u002Fmaestro","Netflix","Maestro: Netflix’s Workflow Orchestrator","https:\u002F\u002Fmaestro-doc.github.io",null,"Java",3787,291,163,19,0,3,14,9,68.3,"Apache License 2.0",false,"main",true,[26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45],"agentic-workflow","analytics","automation","batch-processing","dag","data-engineering","data-ops","data-orchestrator","data-pipelines","data-science","elt","etl","java","machine-learning","mlops","orchestration","scheduler","workflow","workflow-engine","workflow-orchestration","2026-06-12 04:01:07","Maestro\n===================================\n[Maestro](https:\u002F\u002Fnetflixtechblog.com\u002Forchestrating-data-ml-workflows-at-scale-with-netflix-maestro-aaa2b41b800c)\nis a general-purpose workflow orchestrator that \nprovides a fully managed workflow-as-a-service (WAAS) to the data platform users at Netflix.\n\nIt serves thousands of users, including data scientists, data engineers, machine learning engineers,\nsoftware engineers, content producers, and business analysts, for various use cases.\nIt schedules hundreds of thousands of workflows, millions of jobs every day\nand operates with a strict SLO even when there are spikes in the traffic.\nMaestro is highly scalable and extensible to support existing and new use cases and offers enhanced usability to end users.\n\nYou can read more details about it in our series of blog posts\n- [Maestro: Data\u002FML Workflow Orchestrator at Netflix](https:\u002F\u002Fnetflixtechblog.com\u002Fmaestro-netflixs-workflow-orchestrator-ee13a06f9c78)\n- [Orchestrating Data\u002FML Workflows at Scale With Netflix Maestro](https:\u002F\u002Fnetflixtechblog.com\u002Forchestrating-data-ml-workflows-at-scale-with-netflix-maestro-aaa2b41b800c)\n- [100X Faster: How We Supercharged Netflix Maestro's Workflow Engine](https:\u002F\u002Fnetflixtechblog.com\u002F100x-faster-how-we-supercharged-netflix-maestros-workflow-engine-028e9637f041)\n- [Incremental Processing using Netflix Maestro and Apache Iceberg](https:\u002F\u002Fnetflixtechblog.com\u002Fincremental-processing-using-netflix-maestro-and-apache-iceberg-b8ba072ddeeb)\n\n# Get started\n## Prerequisite\n- Git\n- Java 21\n- Gradle\n- Docker\n\n\n## Build it\n- `.\u002Fgradlew build`\n\n## Run it\n- `.\u002Fgradlew bootRun`\n\n## Run it with AWS module\n- `docker compose -f maestro-aws\u002Fdocker-compose.yml up`\n- `.\u002Fgradlew bootRun --args='--spring.profiles.active=aws'`\n\n## Create a sample workflow\n- `curl --header \"user: tester\" -X POST 'http:\u002F\u002F127.0.0.1:8080\u002Fapi\u002Fv3\u002Fworkflows' -H \"Content-Type: application\u002Fjson\" -d @maestro-server\u002Fsrc\u002Ftest\u002Fresources\u002Fsamples\u002Fsample-dag-test-1.json`\n\n## Get the sample workflow definition\n- `curl -X GET 'http:\u002F\u002F127.0.0.1:8080\u002Fapi\u002Fv3\u002Fworkflows\u002Fsample-dag-test-1\u002Fversions\u002Flatest'`\n\n## Trigger to run the sample workflow\n- `curl --header \"user: tester\" -X POST 'http:\u002F\u002F127.0.0.1:8080\u002Fapi\u002Fv3\u002Fworkflows\u002Fsample-dag-test-1\u002Fversions\u002Flatest\u002Factions\u002Fstart' -H \"Content-Type: application\u002Fjson\" -d '{\"initiator\": {\"type\": \"manual\"}}'`\n\n## Get the sample workflow instance\n- `curl -X GET 'http:\u002F\u002F127.0.0.1:8080\u002Fapi\u002Fv3\u002Fworkflows\u002Fsample-dag-test-1\u002Finstances\u002F1\u002Fruns\u002F1'`\n\n## Delete the sample workflow and its data\n- `curl --header \"user: tester\" -X DELETE 'http:\u002F\u002F127.0.0.1:8080\u002Fapi\u002Fv3\u002Fworkflows\u002Fsample-dag-test-1'`\n\n## Run it with maestro-extensions (foreach flattening service)\nThe `maestro-extensions` module runs as a separate Spring Boot service that listens to maestro\nevents via SQS (subscribed to the SNS topic maestro-server publishes to) and provides additional\nfunctionality such as foreach step flattening views.\n\nTo run maestro-server and maestro-extensions together locally:\n1. Start LocalStack (provides local SQS\u002FSNS):\n   - `docker compose -f maestro-aws\u002Fdocker-compose.yml up -d`\n2. Start maestro-server (port 8080):\n   - `.\u002Fgradlew :maestro-server:bootRun --args='--spring.profiles.active=aws'`\n3. Start maestro-extensions (port 8081):\n   - `.\u002Fgradlew :maestro-extensions:bootRun`\n\nOnce both services are running, maestro-extensions will consume step instance status change events\nfrom the `maestro-event` SQS queue and process foreach flattening. Query the flattened views via\nthe extensions REST API on port 8081.\n\n## Run it with Kubernetes support\n- setup kubernetes configs so the kubectl command works\n- `.\u002Fgradlew bootRun`\n- `curl --header \"user: tester\" -X POST 'http:\u002F\u002F127.0.0.1:8080\u002Fapi\u002Fv3\u002Fworkflows' -H \"Content-Type: application\u002Fjson\" -d @maestro-server\u002Fsrc\u002Ftest\u002Fresources\u002Fsamples\u002Fsample-kubernetes-wf.json`\n- `curl --header \"user: tester\" -X POST 'http:\u002F\u002F127.0.0.1:8080\u002Fapi\u002Fv3\u002Fworkflows\u002Fsample-kubernetes-wf\u002Fversions\u002Flatest\u002Factions\u002Fstart' -H \"Content-Type: application\u002Fjson\" -d '{\"initiator\": {\"type\": \"manual\"}}'`\n\n## Python SDK client\n \n### Installation\n\n```bash\npip install maestro-sdk\n```\n\n### Creating a workflow\n\n```python\nfrom maestro import Workflow, Job\n\nwf = Workflow(id=\"test-wf\")\nwf.owner(\"tester\").tags(\"test\")\nwf.job(Job(id=\"job1\", type='NoOp'))\nwf_yaml = wf.to_yaml()\n```\n\n### Pushing a workflow to Maestro server\n\n```python\nfrom maestro import Workflow, Job, MaestroClient\n\nwf = Workflow(id=\"test-wf\")\nwf.owner(\"tester\").tags(\"test\")\nwf.job(Job(id=\"job1\", type='NoOp'))\nwf_yaml = wf.to_yaml()\n\nclient = MaestroClient(base_url=\"http:\u002F\u002F127.0.0.1:8080\", user=\"tester\")\nresponse = client.push_yaml(wf_yaml)\nprint(response)\n```\n\n### Starting a workflow\n\n```python\nfrom maestro import MaestroClient\n\nclient = MaestroClient(base_url=\"http:\u002F\u002F127.0.0.1:8080\", user=\"tester\")\nresponse = client.start(workflow_id=\"test-wf\", run_params={\"foo\": {\"value\": \"bar\", \"type\": \"STRING\"}})\nprint(response)\n```\n\nPlease check [Maestro python](https:\u002F\u002Fgithub.com\u002Fjun-he\u002Fmaestro-python) project for more details.\n\n\n## Get in touch\nJoin our community [Slack workspace](https:\u002F\u002Fjoin.slack.com\u002Ft\u002Fmaestro-oss\u002Fshared_invite\u002Fzt-3is5iwz9e-Px3UqLfzG8lEoWhTk5D4yA) for discussions!\n\n# License\nCopyright 2024 Netflix, Inc.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http:\u002F\u002Fwww.apache.org\u002Flicenses\u002FLICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","Maestro是Netflix开发的一个通用工作流编排器，为数据平台用户提供完全托管的工作流即服务（WAAS）。它支持数据科学家、数据工程师、机器学习工程师等多种角色在不同场景下的需求，每天调度数以万计的工作流和数百万个任务，并能在流量激增时保持严格的SLO。Maestro具备高度可扩展性和可扩展性，能够适应现有及新兴用例，同时提供增强的用户体验。该项目使用Java语言编写，采用Apache License 2.0授权，适用于需要大规模数据处理、ETL\u002FELT流程管理、机器学习模型训练等复杂工作流自动化的企业级应用场景。",2,"2026-06-11 03:44:05","high_star"]