[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-94733":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":15,"compositeScore":18,"rankGlobal":10,"rankLanguage":10,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":30,"readmeContent":31,"aiSummary":32,"trendingCount":15,"starSnapshotCount":15,"syncStatus":14,"lastSyncTime":33,"discoverSource":34},94733,"linksift","loveisbl1nd\u002Flinksift","loveisbl1nd","Local-first, self-hosted media downloader powered by yt-dlp, Flask, Docker, and ffmpeg.","",null,"Python",147,92,2,0,11,18,53.21,"MIT License",false,"main",true,[24,25,26,27,28,29],"docker","ffmpeg","flask","media-downloader","self-hosted","yt-dlp","2026-08-24 04:01:22","# LinkSift\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"static\u002Ffavicon.svg\" alt=\"LinkSift\" width=\"72\">\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Cstrong>Turn links into a tidy local queue.\u003C\u002Fstrong>\u003Cbr>\n  Inspect, format, and save media from a focused self-hosted web workspace.\n\u003C\u002Fp>\n\n\u003Cp align=\"center\">\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Floveisbl1nd\u002Flinksift\u002Factions\u002Fworkflows\u002Fci.yml\">\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Floveisbl1nd\u002Flinksift\u002Factions\u002Fworkflows\u002Fci.yml\u002Fbadge.svg\" alt=\"CI status\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Floveisbl1nd\u002Flinksift\u002Freleases\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fgithub\u002Fv\u002Frelease\u002Floveisbl1nd\u002Flinksift?display_name=tag&sort=semver&label=release&labelColor=10171b&color=c8f55a\" alt=\"Latest release\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Floveisbl1nd\u002Flinksift\u002Fpkgs\u002Fcontainer\u002Flinksift\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FGHCR-linux%2Famd64%20%7C%20arm64-2496ED?logo=github&logoColor=white\" alt=\"GHCR architectures\">\u003C\u002Fa>\n  \u003Ca href=\"LICENSE\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-c8f55a?labelColor=10171b\" alt=\"MIT License\">\u003C\u002Fa>\n  \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Fyt-dlp\u002Fyt-dlp\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpowered%20by-yt--dlp-10171b\" alt=\"Powered by yt-dlp\">\u003C\u002Fa>\n  \u003Ca href=\"Dockerfile\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fruntime-Docker-2496ED?logo=docker&logoColor=white\" alt=\"Docker runtime\">\u003C\u002Fa>\n\u003C\u002Fp>\n\nLinkSift is a local-first media downloader powered by [yt-dlp](https:\u002F\u002Fgithub.com\u002Fyt-dlp\u002Fyt-dlp) and ffmpeg. Paste one or more supported URLs, inspect the available metadata, choose MP4 or MP3, and follow each download from the same queue.\n\n> Built for personal, authorized use. Respect copyright law, platform terms, and creators' rights. LinkSift does not support DRM circumvention or bypassing access controls.\n\n## At a glance\n\n| | |\n| --- | --- |\n| **Deployment** | Versioned GHCR image for normal use; source build and local launcher for contributors |\n| **Interface** | Responsive browser UI with light, dark, and system themes |\n| **Formats** | MP4 video or MP3 audio |\n| **Queue** | Multiple URLs, quality selection, concurrency limit, live progress |\n| **Runtime** | Python + Flask, yt-dlp, ffmpeg, Gunicorn, non-root container |\n| **Privacy model** | Local by default; no built-in account, telemetry, or public service |\n\n## Screenshots\n\n\u003Ctable>\n  \u003Ctr>\n    \u003Ctd width=\"50%\">\u003Cimg src=\"assets\u002Fscreenshot-home.png\" alt=\"LinkSift light theme desktop workspace\">\u003C\u002Ftd>\n    \u003Ctd width=\"50%\">\u003Cimg src=\"assets\u002Fscreenshot-dark.png\" alt=\"LinkSift dark theme desktop workspace\">\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd align=\"center\">\u003Csub>Light theme - desktop workspace\u003C\u002Fsub>\u003C\u002Ftd>\n    \u003Ctd align=\"center\">\u003Csub>Dark theme - desktop workspace\u003C\u002Fsub>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\u003Ctable>\n  \u003Ctr>\n    \u003Ctd width=\"50%\">\u003Cimg src=\"assets\u002Fscreenshot-mobile-light.png\" alt=\"LinkSift light theme mobile workspace\">\u003C\u002Ftd>\n    \u003Ctd width=\"50%\">\u003Cimg src=\"assets\u002Fscreenshot-mobile-dark.png\" alt=\"LinkSift dark theme mobile workspace\">\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd align=\"center\">\u003Csub>Light theme - mobile\u003C\u002Fsub>\u003C\u002Ftd>\n    \u003Ctd align=\"center\">\u003Csub>Dark theme - mobile\u003C\u002Fsub>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n## The workflow\n\n\u003Ctable>\n  \u003Ctr>\n    \u003Ctd width=\"33%\">\u003Cstrong>01 - Inspect\u003C\u002Fstrong>\u003Cbr>\u003Cbr>Paste a URL or a batch of URLs. LinkSift asks yt-dlp for metadata without downloading the media first.\u003C\u002Ftd>\n    \u003Ctd width=\"33%\">\u003Cstrong>02 - Choose\u003C\u002Fstrong>\u003Cbr>\u003Cbr>Pick MP4 or MP3, then select an available video quality when the source provides one.\u003C\u002Ftd>\n    \u003Ctd width=\"33%\">\u003Cstrong>03 - Collect\u003C\u002Fstrong>\u003Cbr>\u003Cbr>Watch progress, speed, and ETA. Save completed files through the browser or an optional folder picker.\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n## What is included\n\n- **Local-first by design** - Compose binds to `127.0.0.1:8899` by default.\n- **Batch-friendly queue** - paste one or more supported URLs and process them in sequence.\n- **MP4 and MP3 output** - choose a preferred format before inspection.\n- **Quality selection** - choose from the available video heights returned by yt-dlp.\n- **Live progress** - phase, percentage, downloaded bytes, speed, ETA, and final status.\n- **Browser save controls** - use the default browser download flow or choose a folder in Chromium-based browsers.\n- **Predictable runtime** - Docker includes Python, yt-dlp, ffmpeg, Gunicorn, and a non-root `linksift` user.\n- **Verifiable releases** - version tags publish amd64\u002Farm64 images with OCI metadata, an SBOM, and GitHub build-provenance attestations.\n- **Offline CI** - regression tests mock external tools and never call media platforms.\n\n## Quick start\n\nDocker is the supported end-user path. Install [Docker Desktop](https:\u002F\u002Fwww.docker.com\u002Fproducts\u002Fdocker-desktop\u002F), then start the published image:\n\n```bash\ndocker run -d --name linksift --restart unless-stopped -p 127.0.0.1:8899:8899 -v linksift-downloads:\u002Fapp\u002Fdownloads ghcr.io\u002Floveisbl1nd\u002Flinksift:latest\n```\n\nOpen \u003Chttp:\u002F\u002Flocalhost:8899>. You do not need Python, yt-dlp, ffmpeg, or a virtual environment on the host.\n\nDownloads persist in the named `linksift-downloads` Docker volume. Pin a numbered image such as `0.1.0` instead of `latest` when reproducibility matters. Stop and remove the container with `docker stop linksift` followed by `docker rm linksift`; the volume remains intact.\n\nTo use Compose with the published image after cloning the repository:\n\n```bash\ndocker compose -f compose.ghcr.yml up -d\n```\n\nTo build the current source locally instead, run `docker compose up --build -d`.\n\n## Development\n\nThe local launcher is for contributors and requires Python 3.12, yt-dlp, and ffmpeg:\n\n```bash\n.\u002Flinksift.sh\n```\n\nBefore opening a pull request, run:\n\n```bash\npython -m unittest discover -s tests -v\npython -m py_compile app.py\ndocker compose config\ndocker compose -f compose.ghcr.yml config\ndocker build -t linksift:local .\n```\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) for the contributor workflow and pull request checklist.\n\n## Releases and image verification\n\nPushing a tag in the form `vMAJOR.MINOR.PATCH` runs the release pipeline. It repeats the offline validation suite, builds `linux\u002Famd64` and `linux\u002Farm64` images, publishes SemVer and `latest` tags to [GHCR](https:\u002F\u002Fgithub.com\u002Floveisbl1nd\u002Flinksift\u002Fpkgs\u002Fcontainer\u002Flinksift), attaches supply-chain metadata, and creates the matching GitHub Release.\n\nAfter installing the [GitHub CLI](https:\u002F\u002Fcli.github.com\u002F), verify that a published image was built by this repository's release workflow:\n\n```bash\ngh attestation verify oci:\u002F\u002Fghcr.io\u002Floveisbl1nd\u002Flinksift:0.2.0 -R loveisbl1nd\u002Flinksift\n```\n\nMaintainers should follow [RELEASING.md](RELEASING.md), including the one-time GHCR visibility check. An attestation establishes build origin; it does not replace source or dependency review.\n\n## Configuration\n\n| Variable | Default | Purpose |\n| --- | ---: | --- |\n| `PORT` | `8899` | HTTP port used by the development server. |\n| `HOST` | `127.0.0.1` | Bind address. Keep it local unless a protected reverse proxy is in front. |\n| `LINKSIFT_DOWNLOAD_TIMEOUT` | `3600` | Maximum seconds allowed for one yt-dlp process. |\n| `LINKSIFT_MAX_CONCURRENT_DOWNLOADS` | `3` | Number of download worker slots — jobs actually running at the same time, clamped to 1–16. Jobs beyond this limit wait in a FIFO queue instead of being rejected. Invalid, zero, or negative values fall back to the default. |\n| `LINKSIFT_MAX_QUEUED_DOWNLOADS` | `200` | Maximum jobs allowed to wait in the queue, not counting running jobs. When the queue is full, `POST \u002Fapi\u002Fdownload` returns 429. Invalid, zero, or negative values fall back to the default. |\n| `LINKSIFT_CONCURRENT_FRAGMENTS` | `4` | Fragment parallelism inside a single DASH\u002FHLS download (yt-dlp `--concurrent-fragments`), clamped to 1–16. Invalid, zero, or negative values fall back to the default. |\n| `LINKSIFT_JOB_TTL` | `86400` | Seconds a terminal job (done, error, timed_out, or cancelled) and its files are kept before automatic cleanup. Invalid, zero, or negative values fall back to the default. |\n| `LINKSIFT_MAX_PLAYLIST_ITEMS` | `200` | Maximum playlist entries expanded per inspection. Longer playlists are truncated to the first N items. Invalid values fall back to the default. |\n| `LINKSIFT_JOB_RETRIES` | `2` | Extra fresh-extraction attempts after a failed download attempt (0–5). Only transient failures (HTTP 403\u002F429\u002F5xx, network resets, timeouts) are retried. Invalid values fall back to the default. |\n| `LINKSIFT_RETRY_BASE_DELAY` | `2` | Seconds before the first retry; doubles per retry and is capped at 15 s. Backoff time counts against `LINKSIFT_DOWNLOAD_TIMEOUT`. Invalid or negative values fall back to the default. |\n| `LINKSIFT_PO_TOKEN_PROVIDER_URL` | unset | Base URL of a bgutil PO token provider (robust mode). Must be an `http`\u002F`https` URL with a hostname; invalid values are ignored with a warning. |\n| `LINKSIFT_NO_UPDATE` | unset | Set to `1` to skip the startup update of yt-dlp and yt-dlp-ejs. |\n\n`LINKSIFT_MAX_CONCURRENT_DOWNLOADS` controls how many downloads run at once; `LINKSIFT_MAX_QUEUED_DOWNLOADS` controls how many may wait behind them. Queued jobs report `status: \"queued\"` and a 1-based `queue_position` from `GET \u002Fapi\u002Fstatus\u002F\u003Cjob_id>`, and can be cancelled before they start. `LINKSIFT_CONCURRENT_FRAGMENTS` only parallelizes fragmented (DASH\u002FHLS) downloads — it does not speed up every URL, and higher concurrency values increase CPU and network load without guaranteeing faster downloads. Total resource use scales with both settings combined: up to `LINKSIFT_MAX_CONCURRENT_DOWNLOADS × LINKSIFT_CONCURRENT_FRAGMENTS` fragment connections plus one ffmpeg process per running job can be active at the same time.\n\nJob state is held in memory. The Docker command therefore uses one Gunicorn worker; restarting the service or container clears queued and active job state, and partially downloaded `.part` files are not resumed automatically after a restart (the TTL cleanup removes them instead). This is accepted behavior for the local-first design. Do not add workers until job state moves to shared storage.\n\nDownloaded files and job status are a temporary cache, not an archive: LinkSift removes finished jobs and their files after `LINKSIFT_JOB_TTL` seconds and sweeps stale leftover files it created at startup and periodically while running. Active downloads are never touched by TTL cleanup. Save completed files through the browser before the TTL expires. Playlists larger than `LINKSIFT_MAX_PLAYLIST_ITEMS` only queue the first configured number of items; truncation is detected from the playlist size reported by yt-dlp, and unavailable or malformed playlist entries are skipped without failing the request.\n\n## YouTube reliability\n\nYouTube periodically rejects freshly extracted media URLs with HTTP 403 and challenges clients with JavaScript puzzles. LinkSift ships three layers of mitigation; none of them guarantees zero 403s, but together they make transient failures recover automatically.\n\n**Base mode (default image).** The container bundles a pinned [Deno](https:\u002F\u002Fdeno.com\u002F) runtime and the [yt-dlp-ejs](https:\u002F\u002Fgithub.com\u002Fyt-dlp\u002Fejs) solver, so yt-dlp can solve YouTube's JS challenges out of the box (`yt-dlp -v` should list `deno` under JS runtimes, not \"JS runtimes: none\"). On top of that, LinkSift retries failed downloads with a **fresh extraction**: a transient failure (HTTP 403\u002F429\u002F5xx, connection reset, network timeout) re-runs the whole yt-dlp process — obtaining new signed media URLs — up to `LINKSIFT_JOB_RETRIES` extra times with exponential backoff, while keeping `.part` files so the download resumes instead of restarting. The status API reports `attempt`\u002F`max_attempts` and the UI shows \"Retrying — attempt N of M\".\n\n**Robust mode (optional PO token provider).** For setups that still hit 403s, an optional overlay adds a [bgutil PO token provider](https:\u002F\u002Fgithub.com\u002FBrainicism\u002Fbgutil-ytdlp-pot-provider) sidecar plus the matching yt-dlp plugin (GPL-licensed, so it is not part of the default image):\n\n```bash\ndocker compose -f docker-compose.yml -f docker-compose.youtube-robust.yml up -d --build\n```\n\nThe provider is only reachable inside the Docker network (no host port is published), LinkSift waits for it to become healthy, and plugin\u002Fsidecar versions are pinned together. `GET \u002Fapi\u002Fhealth` reports the active capabilities: `youtube_js_runtime` and `youtube_ejs` for the base layers, `po_token_provider_configured` (the environment URL is set and valid) and `po_token_provider` (the URL is valid **and** the plugin is actually installed — only then are provider arguments passed to yt-dlp).\n\nCookies remain strictly optional: they are a way to access login-restricted content, not the default fix for 403 errors.\n\n**Troubleshooting.**\n\n- `yt-dlp -v` inside the container should show a `deno` JS runtime and EJS solver; if it prints \"JS runtimes: none\", the image is outdated — rebuild or pull a newer tag.\n- Check `GET \u002Fapi\u002Fhealth`: `capabilities.youtube_js_runtime`\u002F`youtube_ejs` should be `true` in Docker; `po_token_provider` is `true` only in robust mode.\n- In robust mode, `docker logs linksift-bgutil-provider` shows provider activity; LinkSift logs a warning and ignores the provider when `LINKSIFT_PO_TOKEN_PROVIDER_URL` is invalid.\n- If `\u002Fapi\u002Fhealth` shows `po_token_provider_configured: true` but `po_token_provider: false`, the provider URL is set but the plugin is missing — you are most likely running the default image. Rebuild with the robust overlay (`docker compose -f docker-compose.yml -f docker-compose.youtube-robust.yml up -d --build`); LinkSift logs a warning and simply ignores the provider in the meantime.\n- Persistent, non-transient failures (private\u002Fremoved videos, \"Sign in to confirm…\") are not retried by design.\n\n## Supported sites\n\nLinkSift accepts the sites supported by [yt-dlp](https:\u002F\u002Fgithub.com\u002Fyt-dlp\u002Fyt-dlp\u002Fblob\u002Fmaster\u002Fsupportedsites.md), including YouTube, TikTok, Instagram, Reddit, Facebook, Vimeo, Twitch, SoundCloud, Loom, Streamable, Pinterest, Tumblr, Threads, LinkedIn, and many more.\n\nThe supported-site list changes with yt-dlp releases. LinkSift updates yt-dlp at container startup by default; set `LINKSIFT_NO_UPDATE=1` to opt out.\n\n## Security and network exposure\n\nLinkSift accepts URLs for yt-dlp to process and has no built-in authentication. **Do not expose it directly to the internet or an untrusted LAN.** If remote access is required, place it behind a reverse proxy with TLS, authentication, rate limiting, and egress controls that you operate.\n\nFor a vulnerability report, use [GitHub Private Vulnerability Reporting](https:\u002F\u002Fgithub.com\u002Floveisbl1nd\u002Flinksift\u002Fsecurity\u002Fadvisories\u002Fnew) instead of opening a public issue. See [SECURITY.md](SECURITY.md) for the disclosure policy.\n\n## Project layout\n\n```text\napp.py                 Flask API, queue state, and download worker\ntemplates\u002Findex.html   Responsive browser interface\nstatic\u002F                Favicon and static assets\nassets\u002F                README screenshots\nDockerfile             Production container image (base + youtube-robust targets)\ndocker-compose.yml     Local Docker deployment\ndocker-compose.youtube-robust.yml  Optional PO token provider overlay\ncompose.ghcr.yml       Deployment using the published GHCR image\nlinksift.sh            Contributor-only local launcher\ntests\u002F                 Offline regression suite\n.github\u002F               CI, issue forms, and pull request template\nPROVENANCE.md           Verified source history and metrics boundary\nTHIRD_PARTY_NOTICES.md  Preserved licenses for inherited source\nRELEASING.md            Tagged release and verification runbook\nROADMAP.md              Maintainer direction and contribution candidates\n```\n\n## Contributing\n\nBug reports, documentation improvements, tests, and focused pull requests are welcome. Please read [CONTRIBUTING.md](CONTRIBUTING.md), the contributor-oriented [ROADMAP.md](ROADMAP.md), [SECURITY.md](SECURITY.md), and the issue templates before contributing.\n\n## Project provenance\n\nLinkSift began from an MIT-licensed ReClip source baseline and is now maintained independently with its own identity, history, releases, and adoption metrics. The exact upstream repository and commit, the scope of LinkSift's changes, and the history boundary are recorded in [PROVENANCE.md](PROVENANCE.md). The inherited MIT notice is preserved in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md); no upstream endorsement is implied.\n\n## License\n\n[MIT](LICENSE) - Copyright (c) 2026 iaht. Inherited portions retain the notice in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).\n","LinkSift 是一个本地优先、可自托管的媒体下载工具，用于将网页链接批量转换为本地音视频文件。它基于 yt-dlp 和 ffmpeg 实现媒体提取与格式转换，提供 Web 界面（Flask 构建）、多链接队列管理、MP4\u002FMP3 格式选择、实时下载进度追踪及明暗主题适配；所有处理在本地 Docker 容器中完成，不依赖云端服务或用户账户。适用于个人合规场景下的离线内容存档、课程资料整理、播客下载等需求，强调隐私保护与版权尊重。","2026-08-15 02:30:05","CREATED_QUERY"]