[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80648":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":15,"stars7d":16,"stars30d":17,"stars90d":15,"forks30d":15,"starsTrendScore":14,"compositeScore":18,"rankGlobal":10,"rankLanguage":10,"license":19,"archived":20,"fork":20,"defaultBranch":21,"hasWiki":22,"hasPages":20,"topics":23,"createdAt":10,"pushedAt":10,"updatedAt":24,"readmeContent":25,"aiSummary":26,"trendingCount":15,"starSnapshotCount":15,"syncStatus":13,"lastSyncTime":27,"discoverSource":28},80648,"condenseit","wildlifechorus\u002Fcondenseit","wildlifechorus","Self-hosted AI news digest. Collect RSS feeds, YouTube channels, website diffs, Google News searches, Hacker News, Reddit, GitHub Releases, and podcasts, summarize with a local LLM (Ollama), OpenRouter, or any OpenAI-compatible endpoint, learn your preferences from star ratings and engagement signals, and read a daily digest in the browser","",null,"Python",64,2,1,0,5,14,45.33,"MIT License",false,"main",true,[],"2026-06-12 04:01:29","# CondenseIt\n\n> Self-hosted AI news digest. Collect RSS feeds, YouTube channels, website\n> diffs, Google News searches, Hacker News, Reddit, GitHub Releases, and podcasts,\n> summarize with a local LLM (Ollama), OpenRouter, or any OpenAI-compatible endpoint,\n> learn your preferences from star ratings and engagement signals, and read a daily\n> digest in the browser.\n\n![Python 3.11+](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Fpython-3.11%2B-blue)\n![License MIT](https:\u002F\u002Fimg.shields.io\u002Fbadge\u002Flicense-MIT-green)\n\n---\n\n## Preview\n\n![CondenseIt walkthrough with demo data](docs\u002Fassets\u002Fdemo\u002Fwalkthrough.gif)\n\n---\n\n## Modes\n\n| | Local | Remote | OpenAI-compatible |\n|-|-------|--------|-------------------|\n| **LLM** | Ollama on your Mac (Metal) | OpenRouter (cloud) | Any `\u002Fv1\u002Fchat\u002Fcompletions` server |\n| **Scheduling** | `condenseit run` from CLI, or `CONDENSEIT_SCHEDULER_ENABLED=1` | Built-in scheduler (`CONDENSEIT_SCHEDULER_ENABLED=1`) | Same as local or remote |\n| **Setup** | `uv sync` + `condenseit serve` | `bootstrap-server.sh` + `deploy.sh` | Set `OPENAI_API_BASE_URL` + `OPENAI_API_KEY` |\n| **Cost** | Free (local hardware) | Pay-per-token via OpenRouter | Depends on server |\n\nBoth modes use the same unified web UI: digest reader and admin panel.\n\n---\n\n## What it does\n\nCondenseIt pulls from the sources you configure, scores and summarizes each\narticle with a local or cloud LLM, ranks articles by your learned preferences,\nand produces a daily digest you can read in the browser.\n\nThe ranking engine learns from multiple signals:\n\n| Signal | Type | Effect |\n|--------|------|--------|\n| **Star rating 4-5** | Explicit positive | Boosts matched terms, category, and source |\n| **Star rating 1-2** | Explicit negative | Penalises matched terms, category, and source |\n| **Mark as read** | Implicit mild positive | Mild boost to category and content profile |\n| **Save for later** | Implicit strong positive | Strong boost to category and content profile |\n| **Star (permanent save)** | No ranking effect | Saved forever in the Starred page; survives all future digest runs |\n| **Dismiss** | Implicit mild negative | Mild penalty to category and content profile |\n\nScores are additive across multiple named signals: keywords, term overlap,\nbigram phrases, TF-IDF cosine, category, source, three implicit channels,\nsynonym boost, semantic embedding similarity, LLM-extracted topic score, and an\noptional LLM rerank pass. Every article card shows a collapsible \"Why ranked\nhere?\" breakdown so you can see exactly what drove its position.\n\n**Supported source types** (all configured from Admin > Sources, no API keys needed):\n\n| Type | What it collects |\n|------|-----------------|\n| **RSS \u002F Atom** | Any RSS or Atom feed URL; extracts `og:image` thumbnails from article pages |\n| **YouTube** | Transcripts from channel videos via the public channel RSS feed; attaches video thumbnail |\n| **Website watch** | Detects meaningful changes on any web page |\n| **Google News search** | Google News RSS search with operator support (`site:`, `when:`, `intitle:`, `source:`) |\n| **Hacker News** | Top\u002Fbest\u002Fnew\u002Fask\u002Fshow stories via the public Firebase JSON API |\n| **Reddit** | Posts from any public subreddit (hot\u002Fnew\u002Ftop\u002Frising, configurable score threshold) |\n| **GitHub Releases** | Release notes from any public repository's Atom feed |\n| **Podcasts** | New episodes from podcast RSS feeds, with iTunes search to find feed URLs |\n\n### Admin panel\n\n| Page | What it does |\n|------|-------------|\n| **Sources** | Add\u002Fremove all source types; per-source keyword filters (show only, hide, highlight); filter by category; set priority; import\u002Fexport OPML |\n| **LLM** | Provider (Ollama \u002F OpenRouter \u002F OpenAI-compatible), model, OpenRouter cheapest-model option, OpenAI-compatible base URL, Ollama pull\u002Fdelete |\n| **API keys** | OpenRouter key storage, encrypted at rest in SQLite |\n| **Schedule** | Enable\u002Fdisable automatic digest runs, set daily run times, and view the next scheduled run |\n| **Settings** | Article limits, category balance, article age cutoff, language filter, summary format, and **ranking weights** |\n| **Preferences** | Learning profile: rating distribution, liked\u002Fdisliked terms and phrases, category and source scores, implicit signal counts, and time-decay info |\n| **Security** | Change the admin password and warn when the default password is still active |\n| **Budget** | OpenRouter account usage, local pipeline cost tracking, and daily\u002Fmonthly budget limits |\n| **Logs** | Full output captured from recent digest runs |\n\n---\n\n## Local mode (Ollama on your Mac)\n\n### Prerequisites\n\n- Python 3.11+, [uv](https:\u002F\u002Fgithub.com\u002Fastral-sh\u002Fuv)\n- [Ollama](https:\u002F\u002Follama.ai) installed and running\n- Node.js 18+ (for the frontend build)\n\n### Quick start\n\n```bash\ngit clone https:\u002F\u002Fgithub.com\u002Fwildlifechorus\u002Fcondenseit\ncd condenseit\n\n# Install dependencies\nuv sync\n\n# Pull a model\nollama pull llama3.2:3b\n\n# Build the frontend\ncd frontend && npm ci && npm run build && cd ..\n\n# Copy and edit config\ncp config.example.yaml config.yaml\ncp .env.example .env\n# Edit config.yaml: add your feeds under the feeds\u002Fyoutube_channels sections.\n# The example defaults to llm.provider: \"openrouter\" (requires OPENROUTER_API_KEY in .env).\n# Change llm.provider to \"ollama\" if you installed Ollama above, or set it to\n# \"openai\" and configure llm.openai_base_url + OPENAI_API_KEY for any\n# OpenAI-compatible server (LM Studio, vLLM, llama.cpp, etc.).\n\n# Start the web UI\ncondenseit serve --port 8899\n# Open http:\u002F\u002Flocalhost:8899\n\n# Run a digest (in a separate terminal, or from the web UI)\ncondenseit run\n```\n\n### Automatic scheduling\n\nEnable the built-in scheduler in `.env`:\n\n```\nCONDENSEIT_SCHEDULER_ENABLED=1\n```\n\nThen configure run times in **Admin > Schedule** (or set a default in\n`config.schedule.times` in `config.yaml`). Changes made in the admin UI take\neffect immediately without a restart.\n\n---\n\n## Remote mode (VPS + OpenRouter)\n\n### Prerequisites\n\n- A VPS with Ubuntu\u002FDebian, SSH access, nginx, and python3\n- An [OpenRouter](https:\u002F\u002Fopenrouter.ai) API key\n- A domain pointed at your VPS\n\n### One-time setup\n\n```bash\n# Copy config for your domain\ncp scripts\u002Fnginx\u002Fdigest.example.com.conf scripts\u002Fnginx\u002Fyour.domain.conf\n# Edit the file: replace \"digest.example.com\" with your domain\n\n# Set VPS connection details in your local .env\necho 'DIGEST_PWA_SSH_HOST=your-ssh-host' >> .env\necho 'DIGEST_PWA_DOMAIN=your.domain' >> .env\necho 'CONDENSEIT_AUTH_PASSWORD=choose-a-strong-password' >> .env\n\n# Bootstrap the VPS (installs condenseit, systemd service, nginx)\n.\u002Fscripts\u002Fbootstrap-server.sh\n```\n\nThe bootstrap script will prompt for your OpenRouter API key, app password,\nand scheduler preference, then write everything to `~\u002Fcondenseit\u002F.env` on\nthe VPS. Secrets never touch the systemd unit file.\n\n### Deploy\n\n```bash\n.\u002Fscripts\u002Fdeploy.sh\n```\n\nThis builds the frontend, packages a wheel, rsyncs everything to the VPS,\nand restarts the service. Run again any time you update sources or config.\n\n### TLS\n\n```bash\nssh your-vps 'sudo certbot --nginx -d your.domain'\n```\n\n---\n\n## Configuration\n\nSee [`config.example.yaml`](config.example.yaml) and [`.env.example`](.env.example)\nfor all options with inline comments.\n\nDetailed setup guides:\n\n- [Local deployment](docs\u002Fdeploy-local.md)\n- [VPS deployment](docs\u002Fdeploy-vps.md)\n- [Firebase \u002F Cloud Run deployment](docs\u002Fdeploy-firebase.md)\n- [Add to home screen (iOS and Android)](docs\u002Fadd-to-home-screen.md)\n\nKey `config.yaml` sections:\n\n- `llm` - provider (`ollama` \u002F `openrouter` \u002F `fallback` \u002F `openai`), model, budget limits, OpenAI-compatible base URL\n- `feeds` \u002F `youtube_channels` \u002F `watch_urls` - legacy YAML-seeded sources (still supported)\n- `schedule.times` - default daily run times (overridden by Admin > Schedule)\n- `vps` - SSH target for `scripts\u002Fdeploy.sh`\n\nAll sources (including the new types) are managed from **Admin > Sources** in the web UI and stored in SQLite. The YAML keys above act as initial seeds that are imported once on first run.\n\nSettings also editable live in the admin panel (stored in SQLite, no restart needed):\n\n- **Schedule** - run times\n- **Digest** - `max_articles_per_digest`, `balance_digest_categories`,\n  `max_articles_per_category`, `max_article_age_hours`,\n  `preferred_languages`, `max_key_takeaways`, `max_summary_paragraphs`\n- **Ranking weights** - `tfidf_preference_weight`, `category_preference_weight`,\n  `source_preference_weight`, `implicit_signal_weight`,\n  `rating_decay_half_life_days`, `min_ratings_for_learning`,\n  `embedding_preference_weight`, `topic_score_weight`\n- **LLM** - provider, model, OpenRouter model, cheapest-model selection, OpenAI-compatible base URL and model\n- **Budget** - OpenRouter daily and monthly budget limits\n- **Security** - admin password\n\n---\n\n## Preference learning\n\nThe ranking engine builds a preference profile from everything you do in the\ndigest reader. No minimum setup is required; it activates automatically once\nyou have at least 5 star ratings (configurable via `min_ratings_for_learning`).\n\n**Explicit ratings (1-5 stars)**\n\n- Ratings 4-5 add to the liked term profile, bigram profile, category average,\n  and source average.\n- Ratings 1-2 penalise those same signals.\n- Rating 3 is neutral for terms but still contributes to category\u002Fsource means.\n- All rating rows decay exponentially over time (default half-life: 30 days) so\n  stale preferences fade and recent tastes dominate.\n\n**Implicit signals**\n\nThree engagement actions contribute automatically without requiring a star rating:\n\n- **Read** (mark as read): treated as a mild positive (equivalent to ~3.8 stars).\n- **Save for later**: treated as a strong positive (~4.5 stars).\n- **Dismiss**: treated as a mild negative (~1.5 stars), distinct from \"mark as\n  read\". Dismiss tells the engine you saw the article and were not interested,\n  which penalises the article's terms, category, and source in future ranking.\n\nImplicit contributions are scaled by `implicit_signal_weight` (default `0.5`)\nso they always have less influence than explicit star ratings.\n\n**Topic synonyms**\n\nOptional synonym groups let the engine propagate profile weight across related\nterms without retraining. For example, rating a \"kubernetes\" article highly will\nalso boost articles mentioning \"k8s\" or \"helm\" when they are in the same\nsynonym group:\n\n```yaml\nrelevance:\n  topic_synonyms:\n    kubernetes: [\"k8s\", \"helm\", \"kubectl\"]\n    security:   [\"infosec\", \"cybersecurity\", \"appsec\"]\n```\n\n**AI-powered ranking (optional, incremental)**\n\nWhen an LLM provider is available the engine adds three additional layers on top\nof classical ranking. Each layer is independently controlled and off by default:\n\n- **Semantic embeddings** - article text and your liked\u002Fdisliked articles are\n  encoded as vectors. The engine scores each candidate by cosine similarity to\n  the centroid of your liked embeddings minus disliked embeddings. Embeddings\n  are generated once and cached in SQLite (keyed by URL + content hash), so\n  subsequent digest runs are fast. Configure with `embedding_provider`\n  (`\"ollama\"` \u002F `\"openrouter\"` \u002F `\"off\"`), `embedding_model`, and\n  `embedding_preference_weight`.\n\n- **Topic\u002Fentity enrichment** - the LLM already summarizes each article; the\n  same call now also extracts `topics`, `entities`, and a `novelty` score\n  (1-5). These are persisted in an `article_enrichment` table and used to build\n  a topic profile from your ratings. Articles matching liked topics are boosted;\n  articles matching disliked topics are penalised. Weight controlled by\n  `topic_score_weight`. Topics and a \"novel\" badge are displayed on each card.\n\n- **LLM reranker** - after classical scoring a compact profile narrative is\n  built from your top liked\u002Fdisliked terms, categories, and sources. The LLM\n  is asked to score the top-K candidates (configurable via `llm_rerank_top_k`,\n  default 30) by relevance and return a brief reason. The LLM relevance score\n  is blended with the classical score (`llm_rerank_blend`, default `0.3`). The\n  reason appears in the \"Why ranked here?\" panel. Enable with\n  `llm_rerank_enabled: true` in `config.yaml`.\n\n- **Cold-start bootstrap** - if you have no ratings yet, visit **Admin >\n  Preferences** and describe your interests in plain text. The LLM derives\n  initial keywords, synonyms, and a profile summary that seed the engine before\n  any ratings exist. These are stored in the DB and override YAML defaults.\n\n**Score transparency**\n\nEvery article card in the digest shows a collapsible \"Why ranked here?\" panel\nlisting each contributing signal as a proportional bar (classical + AI signals).\nThe **Admin > Preferences** page shows the full learned profile: rating\ndistribution histogram, liked\u002Fdisliked terms sized by weight, category and\nsource bars, bigram phrases, top liked\u002Fdisliked LLM topics, embedding status,\nimplicit signal counts, and the current decay weight of your oldest rating.\n\nAll ranking weights are adjustable live in **Admin > Digest** without restarting\nthe server.\n\n---\n\n## Budget tracking\n\nWhen using OpenRouter, the web UI shows a **Budget** page under Admin with:\n\n- OpenRouter account usage (daily \u002F weekly \u002F monthly credits)\n- Local spending broken down by model\n- Cost per digest run\n\nBudget limits (`openrouter_daily_budget_usd`, `openrouter_monthly_budget_usd`\nin `config.yaml`) stop the pipeline before they are exceeded.\n\n---\n\n## Language filtering\n\nSet preferred languages in **Admin > Settings** (ISO 639-1 codes, e.g. `en`,\n`pt`, `de`). Articles in other languages are excluded before ranking. Leave\nempty to accept all languages. Uses the `langdetect` library; detection\nfailures always keep the article.\n\n---\n\n## Development\n\n```bash\nuv sync --extra dev\npytest -q\nruff check src tests\ncd frontend && npm ci && npm run build\n```\n\n---\n\n## License\n\nMIT\n","CondenseIt 是一个自托管的AI新闻摘要工具，它能够从RSS源、YouTube频道、网站更新、Google新闻搜索、Hacker News、Reddit、GitHub发布和播客中收集信息，并通过本地LLM（如Ollama）、OpenRouter或任何兼容OpenAI的端点进行总结。该项目支持学习用户的偏好，依据用户对内容的评分与互动反馈调整内容排序，最终以每日摘要的形式在浏览器中展示给用户。使用Python 3.11+开发，具备灵活的部署选项，既可以在本地运行也可以利用云服务。适用于需要高效获取并管理多渠道信息的个人或团队，特别适合那些希望通过自动化手段来节省时间同时保持信息流相关性的人士。","2026-06-11 04:01:30","CREATED_QUERY"]