[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-80206":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":8,"htmlUrl":8,"language":9,"languages":8,"totalLinesOfCode":8,"stars":10,"forks":11,"watchers":12,"openIssues":11,"contributorsCount":11,"subscribersCount":11,"size":11,"stars1d":11,"stars7d":11,"stars30d":11,"stars90d":11,"forks30d":11,"starsTrendScore":11,"compositeScore":11,"rankGlobal":8,"rankLanguage":8,"license":13,"archived":14,"fork":14,"defaultBranch":15,"hasWiki":16,"hasPages":14,"topics":17,"createdAt":8,"pushedAt":8,"updatedAt":18,"readmeContent":19,"aiSummary":20,"trendingCount":11,"starSnapshotCount":11,"syncStatus":21,"lastSyncTime":22,"discoverSource":23},80206,"threat-detection-","dayaa-hash\u002Fthreat-detection-","dayaa-hash",null,"JavaScript",69,0,70,"MIT License",false,"main",true,[],"2026-06-12 02:03:59","![ost_logo](https:\u002F\u002Fuser-images.githubusercontent.com\u002F44299200\u002F210261186-1f0486a7-79e8-41b6-85f1-9e69915123aa.png)\n\n# OSINT Toolkit\n\n> **⚠️ Warning**\n> OSINT Toolkit is not production ready yet. This is an early prototype that still needs some work to be done.\n\n## A Full-Stack Web Application Built for Security Analysts\n\nOSINT Toolkit is a self-hostable, on-demand analysis platform designed for security specialists. It consolidates various security tools into a single, easy-to-use environment, streamlining everyday tasks. Optimized for single-user operation, OSINT Toolkit runs locally in a Docker container and is not intended for long-term data storage or management. Instead, it focuses on accelerating daily workflows, such as news aggregation and analysis, IOC and email investigations, and the creation of threat detection rules. To further enhance efficiency, OSINT Toolkit integrates generative AI capabilities, providing additional support for analysis and decision-making.\n\n---\n\n## 📋 Table of Contents\n\n- [Features](#features)\n- [Integrated Services](#integrated-services)\n- [Tech Stack](#tech-stack)\n- [Architecture](#architecture)\n- [Prerequisites](#prerequisites)\n- [Installation](#installation)\n- [Configuration](#configuration)\n- [API Keys Setup](#api-keys-setup)\n- [Usage](#usage)\n- [Development](#development)\n- [License](#license)\n- [Contributing](#contributing)\n\n---\n\n## ✨ Features\n\n### 1. Newsfeed\nThe Newsfeed module keeps you up to date about the latest cybersecurity news by aggregating articles from trusted sources such as Wired, The Hacker News, Security Magazine, Threatpost, TechCrunch Security, and Dark Reading. Stay up-to-date with industry trends and potential threats without having to visit multiple websites or subscribe to numerous newsletters. The module extracts IOCs automatically from the news articles and lets you analyze news in no time using AI.\n\n![newsfeed](https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F0c23cc14-4a1a-4c34-9fb8-5064a0f23889)\n\n**Key Capabilities:**\n- RSS feed aggregation from multiple security news sources\n- Automatic IOC extraction from articles\n- AI-powered news analysis\n- Keyword-based filtering\n- Scheduled news updates\n\n---\n\n### 2. IOC Tools\nThe IOC Tools module helps you analyze different types of indicators of compromise (IOCs) such as IP addresses, hashes, email addresses, domains, and URLs. It leverages services like VirusTotal, AlienVault, AbuseIPDB, and social media platforms like Reddit and Twitter to gather information about the IOCs. The module automatically detects the type of IOC being analyzed and utilizes the appropriate services to provide relevant information, enabling you to identify potential threats and take necessary actions to protect your organization.\n\n![ioc_lookup](https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F40b1e656-ba6c-4f36-b8dd-beee0dca3fdd)\n\n**Sub-modules:**\n- **Single Lookup** - Analyze individual IOCs\n- **Bulk Lookup** - Analyze multiple IOCs at once\n- **Extractor** - Extract IOCs from text or files\n- **Defang\u002FFang** - Safely format IOCs for sharing\n\n---\n\n### 3. Email Analyzer\nThe Email Analyzer module allows you to analyze `.eml` files for potential threats. Simply drag and drop an `.eml` file into the module, and it will parse the file, perform basic security checks, extract indicators of compromise (IOCs), and analyze messages with the help of AI.\n\n**Key Capabilities:**\n- EML file parsing\n- Header analysis\n- Security checks (SPF, DKIM, DMARC)\n- IOC extraction (URLs, IPs, domains, hashes)\n- Email routing\u002Fhop analysis\n- Attachment analysis\n- AI-powered message analysis\n\n---\n\n### 4. Domain Finder (Domain Monitoring)\nThe Domain Finder module helps you protect your organization from phishing attacks by searching for recently registered domains that match specific patterns. By utilizing the URLScan.io API, you can view screenshots of websites associated with domains without visiting them directly. Additionally, you can check each domain and its resolved IP against multiple threat intelligence services.\n\n**Key Capabilities:**\n- Domain search by patterns\u002Fkeywords\n- URLScan.io integration for screenshots\n- Multi-service threat intelligence lookup\n- Historical domain analysis\n- IP reputation checking\n\n---\n\n### 5. AI Templates\nThe AI Templates module provides powerful AI-based solutions for log data analysis, email text analysis, and source code explanation. It lets you create templates for AI tasks and supports you in the prompt engineering process.\n\n![ai_templates](https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F42c52c8c-7d2d-4b70-b25c-666d6993832c)\n\n**Key Capabilities:**\n- Custom AI prompt templates\n- Log analysis\n- Email text analysis\n- Source code explanation\n- Multiple LLM provider support (OpenAI, Anthropic, Google Gemini)\n- Template management and sharing\n\n---\n\n### 6. CVSS Calculator\nThe CVSS Calculator module allows you to calculate the CVSS 3.1 score of a vulnerability and export the calculation as a markdown or JSON file.\n\n**Key Capabilities:**\n- CVSS 3.1 base score calculation\n- Environmental score adjustment\n- Temporal score adjustment\n- Export to Markdown\u002FJSON\n- Visual score display\n\n---\n\n### 7. Detection Rules\nThe Detection Rules module is a GUI for creating Sigma, YARA and Snort\u002FSuricata rules.\n\n**Key Capabilities:**\n- Sigma rule creation\n- YARA rule creation\n- Snort\u002FSuricata rule creation\n- Rule validation\n- Rule templates\n\n---\n\n## 🔌 Integrated Services\n\nOSINT Toolkit integrates with over 20+ threat intelligence and security services:\n\n| IPs | Domains | URLs | Emails | Hashes | CVEs |\n|-----|---------|------|--------|--------|------|\n| AbuseIPDB | AlienVault | AlienVault | Emailrep.io | AlienVault | GitHub |\n| AlienVault | CheckPhish.ai | CheckPhish.ai | GitHub | GitHub | NIST NVD |\n| CheckPhish.ai | GitHub | GitHub | Hunter.io | Maltiverse | |\n| CrowdSec | Maltiverse | Google Safe Browsing | Have I Been Pwned | Pulsedive | |\n| GitHub | Pulsedive | Maltiverse | Reddit | Reddit | |\n| IPQualityScore | Shodan | Pulsedive | Twitter | ThreatFox | |\n| Maltiverse | ThreatFox | Shodan | | Twitter | |\n| Pulsedive | Reddit | ThreatFox | | Virustotal | |\n| Shodan | Twitter | Reddit | | | |\n| Reddit | URLScan | Twitter | | | |\n| ThreatFox | Virustotal | URLScan | | | |\n| Twitter | | Virustotal | | | |\n| Virustotal | | | | | |\n\n### Service Categories\n\n- **Threat Intelligence**: AlienVault OTX, Maltiverse, Pulsedive, ThreatFox, VirusTotal, CrowdSec, URLhaus, MalwareBazaar\n- **Security Scanning**: CheckPhish.ai, Google Safe Browsing, IPQualityScore, URLScan.io\n- **Network Infrastructure**: Shodan, BGPView\n- **Social Media**: Reddit, Twitter\u002FX\n- **Email Identity**: EmailRep.io, Have I Been Pwned, Hunter.io\n- **Development\u002FResearch**: GitHub, NIST NVD\n\n---\n\n## 🛠 Tech Stack\n\n### Backend\n- **Framework**: FastAPI\n- **Database**: SQLAlchemy (SQLite)\n- **AI\u002FML**: LangChain (OpenAI, Anthropic, Google Gemini)\n- **Task Scheduling**: APScheduler\n- **HTTP Client**: aiohttp\n- **RSS Parsing**: Feedparser\n\n### Frontend\n- **Framework**: React 18\n- **UI Library**: Material UI (MUI)\n- **State Management**: Recoil\n- **Routing**: React Router\n- **Charts**: Recharts, Nivo\n- **Markdown**: React Markdown, React MD Editor\n\n### Infrastructure\n- **Containerization**: Docker & Docker Compose\n- **Web Server**: Nginx (Frontend)\n- **ASGI Server**: Uvicorn (Backend)\n\n---\n\n## 🏗 Architecture\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│                     OSINT Toolkit                           │\n├─────────────────────────────────────────────────────────────┤\n│                                                             │\n│  ┌─────────────────────┐    ┌─────────────────────────┐   │\n│  │     Frontend        │    │       Backend           │   │\n│  │   (React + MUI)     │◄──►│    (FastAPI)            │   │\n│  │                     │    │                         │   │\n│  │  - React Router     │    │  - API Routes           │   │\n│  │  - Recoil State     │    │  - SQLAlchemy ORM       │   │\n│  │  - Recharts\u002FNivo    │    │  - LangChain LLM        │   │\n│  │  - Nginx            │    │  - APScheduler          │   │\n│  └─────────────────────┘    └───────────┬─────────────┘   │\n│                                         │                  │\n│  ┌──────────────────────────────────────▼────────────────┐ │\n│  │              Data Layer (SQLite)                      │ │\n│  │  - Settings, API Keys, Templates, Newsfeed Cache      │ │\n│  └───────────────────────────────────────────────────────┘ │\n│                                                             │\n└─────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## 📦 Prerequisites\n\n- Docker Engine 20.10+\n- Docker Compose 2.0+\n- 4GB RAM minimum\n- 10GB disk space\n\n---\n\n## 🚀 Installation\n\n### Quick Start\n\n1. **Clone or download the repository**\n   ```bash\n   git clone https:\u002F\u002Fgithub.com\u002Ftrickest\u002Fosint-toolkit.git\n   cd osint-toolkit\n   ```\n\n2. **Start the application**\n   ```bash\n   docker-compose up -d\n   ```\n\n3. **Access the application**\n   - Open your browser to: `http:\u002F\u002Flocalhost:4000`\n\n### Manual Build (Optional)\n\n```bash\n# Build and start containers\ndocker-compose up -d --build\n\n# View logs\ndocker-compose logs -f\n\n# Stop containers\ndocker-compose down\n```\n\n---\n\n## ⚙️ Configuration\n\n### Environment Variables\n\nThe application uses default configurations. For production use, you may want to customize:\n\n| Variable | Description | Default |\n|----------|-------------|---------|\n| `BACKEND_PORT` | Backend port | `8000` |\n| `FRONTEND_PORT` | Frontend port | `4000` |\n| `DATA_DIR` | Data directory | `.\u002Fdata` |\n\n### Data Persistence\n\nAll data is stored in the `.\u002Fdata` directory:\n- SQLite database\n- Uploaded files\n- Cached data\n\n---\n\n## 🔑 API Keys Setup\n\nOSINT Toolkit requires API keys for full functionality. Configure keys through the web interface:\n\n1. Navigate to **Settings** → **API Keys** in the sidebar\n2. Enter your API keys for the services you want to use\n3. Save your configuration\n\n### Required Keys by Feature\n\n| Feature | Required Services |\n|---------|-------------------|\n| Newsfeed | OpenAI (optional for AI analysis) |\n| IOC Lookup | VirusTotal, AlienVault, AbuseIPDB, etc. |\n| Email Analyzer | OpenAI (optional for AI analysis) |\n| Domain Finder | URLScan.io |\n| AI Templates | OpenAI, Anthropic, or Google Gemini |\n\n### Getting API Keys\n\n- **VirusTotal**: https:\u002F\u002Fwww.virustotal.com\u002Fgui\u002Fjoin-us\n- **AlienVault OTX**: https:\u002F\u002Fotx.alienvault.com\u002Fapi\n- **AbuseIPDB**: https:\u002F\u002Fwww.abuseipdb.com\u002Faccount\u002Fapi\n- **Shodan**: https:\u002F\u002Faccount.shodan.io\u002F\n- **URLScan.io**: https:\u002F\u002Furlscan.io\u002Fdocs\u002Fapi\u002F\n- **OpenAI**: https:\u002F\u002Fplatform.openai.com\u002Fapi-keys\n- **Anthropic**: https:\u002F\u002Fconsole.anthropic.com\u002F\n- **Google Gemini**: https:\u002F\u002Faistudio.google.com\u002Fapp\u002Fapikey\n\n> **Note**: Many services offer free tiers. Start with free services and add paid keys as needed.\n\n---\n\n## 📖 Usage\n\n### Getting Started\n\n1. **Configure API Keys** - Set up at least one threat intelligence service\n2. **Explore Features** - Try each module from the sidebar\n3. **Customize Settings** - Adjust modules, keywords, and CTI profile\n\n### Feature Workflows\n\n#### IOC Analysis\n1. Go to **IOC Tools** → **Single Lookup**\n2. Enter an IOC (IP, domain, hash, URL, email)\n3. View results from multiple services\n\n#### Email Analysis\n1. Go to **Email Analyzer**\n2. Drag and drop an `.eml` file\n3. Review extracted IOCs and security checks\n\n#### Domain Monitoring\n1. Go to **Domain Finder**\n2. Enter domain patterns to search\n3. Review screenshots and threat intelligence\n\n---\n\n## 🔧 Development\n\n### Project Structure\n\n```\nosint-toolkit\u002F\n├── backend\u002F\n│   ├── app\u002F\n│   │   ├── core\u002F           # Core functionality\n│   │   │   ├── database\u002F   # Database models\n│   │   │   ├── settings\u002F   # Configuration\n│   │   │   ├── alerts\u002F     # Alerting system\n│   │   │   └── scheduler\u002F  # Task scheduling\n│   │   ├── features\u002F       # Feature modules\n│   │   │   ├── newsfeed\u002F\n│   │   │   ├── ioc_tools\u002F\n│   │   │   ├── email_analyzer\u002F\n│   │   │   ├── domain_lookup\u002F\n│   │   │   └── llm_templates\u002F\n│   │   └── utils\u002F          # Utilities\n│   ├── main.py             # Application entry\n│   └── requirements.txt    # Python dependencies\n│\n├── frontend\u002F\n│   ├── src\u002F\n│   │   ├── components\u002F     # React components\n│   │   ├── api.js          # API client\n│   │   ├── App.js          # Main app\n│   │   └── sidebarConfig.js\n│   ├── package.json        # Node dependencies\n│   └── nginx.conf          # Nginx config\n│\n├── data\u002F                   # Data persistence\n├── docker-compose.yaml     # Docker orchestration\n└── README.md              # This file\n```\n\n### Local Development\n\n```bash\n# Backend (requires Python 3.10+)\ncd backend\npip install -r requirements.txt\nuvicorn main:app --reload\n\n# Frontend (requires Node.js 18+)\ncd frontend\nnpm install\nnpm start\n```\n\n---\n\n## 📄 License\n\nSee [LICENSE.md](LICENSE.md) for details.\n\n---\n\n## 🤝 Contributing\n\nContributions are welcome! Please read our [contributing guidelines](CONTRIBUTING.md) before submitting PRs.\n\n### Ways to Contribute\n\n- Report bugs\n- Suggest new features\n- Improve documentation\n- Add new integrations\n- Fix issues\n\n---\n\n## 📞 Support\n\n- **Issues**: https:\u002F\u002Fgithub.com\u002Ftrickest\u002Fosint-toolkit\u002Fissues\n- **Discussions**: https:\u002F\u002Fgithub.com\u002Ftrickest\u002Fosint-toolkit\u002Fdiscussions\n\n---\n\n## 🔗 Links\n\n- [Documentation](https:\u002F\u002Fgithub.com\u002Ftrickest\u002Fosint-toolkit#readme)\n- [Docker Hub](https:\u002F\u002Fhub.docker.com\u002Fr\u002Ftrickest\u002Fosint-toolkit)\n- [GitHub Repository](https:\u002F\u002Fgithub.com\u002Ftrickest\u002Fosint-toolkit)\n\n---\n\n*Built with ❤️ for the security community*\n\n\n### CVSS Calculator\nThe CVSS Calculator module allows you to calculate the CVSS 3.1 score of a vulnerability and export the calculation as a markdown or JSON file.\n\n\n### Detection Rules\nThe Detection Rules module is a GUI for creating Sigma, Yara and Snort\u002FSuricate rules.\n\n\n\n## Deploy with docker\n1. Download the repository and extract the files\n2. Navigate to the directory where the `docker-compose.yaml` file is located\n3. Run the following command: `docker-compose up -d`\n4. Once the container is running, you can access the application in your browser at http:\u002F\u002Flocalhost:4000\n","OSINT Toolkit 是一个面向安全分析师的全栈Web应用程序，旨在整合多种安全工具于单一易用环境中，以简化日常任务。其核心功能包括新闻聚合与分析、IOC（入侵指标）及电子邮件调查、威胁检测规则创建等，并通过集成生成式AI技术来增强分析效率和决策支持。项目采用JavaScript开发，运行在Docker容器中，适合单用户本地操作场景使用，特别适用于需要快速获取网络安全信息并进行初步分析的安全专家。需要注意的是，该项目目前仍处于早期原型阶段，尚未准备好投入生产环境。",2,"2026-06-11 03:59:39","CREATED_QUERY"]