BiliBiliCommentsAnalysis

Scrapy, a fast high-level web crawling & scraping framework for Python.

deep learning for image processing including classification and object-detection etc.

Elegant Scraper and Crawler Framework for Golang

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

A scalable web crawler framework for Java.

Use LLMs to dig out what you care about from massive amounts of information and a variety of sources daily.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Redis-based components for Scrapy.

BibiGPT v1 · one-Click AI Summary for Audio/Video & Chat with Learning Content: Bilibili | YouTube | Tweet丨TikTok丨Dropbox丨Google Drive丨Local files | Websites丨Podcasts | Meetings | Lectures, etc. 音视频内容 AI 一键总结 & 对话：哔哩哔哩丨YouTube丨推特丨小红书丨抖音丨快手丨百度网盘丨阿里云盘丨网页丨播客丨会议丨本地文件等 (原 BiliGPT 省流神器 & AI课代表)

:sparkling_heart: High available distributed ip proxy pool, powerd by Scrapy and Redis