crawlee
apify
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
AI 简介
Crawlee 是一个面向 Node.js 的网页抓取与浏览器自动化开源库，用于构建高可靠性网络爬虫。它支持 Puppeteer、Playwright、Cheerio 等多种底层驱动，兼容 headless 与 headful 模式，内置代理轮换、请求去重、自动限速、数据持久化及反反爬适配能力，可高效提取 HTML、PDF、图片等资源，广泛应用于 AI 数据采集、RAG 内容构建、LLM 训练数据准备等场景。
TypeScript
Apache License 2.024.6k
Stars
1.5k
Forks
130
Watchers
129
Issues
Star 增长
今日0
近 7 天0
近 30 天+527
综合评分79.56
默认分支master