scrapeGPT

Public

ScrapeGPT is a RAG-based Telegram bot designed to scrape and analyze websites, then answer questions based on the scraped content. The bot utilizes Retrieval Augmented Generation and webscraping to return natural language answers to the user's queries.

crawler huggingface large-language-models llm ollama proxy rag retrieval-augmented-generation robots-txt scraper

Creat：2024-02-13T21:17:26

Update：2025-02-22T23:31:58

Stars

Stars Increase

Related projects

Gpt_academic

academic

为GPT/GLM等LLM大语言模型提供实用化交互接口，特别优化论文阅读/润色/写作体验，模块化设计，支持自定义快捷按钮&函数插件，支持Python和C++等项目剖析&自译解功能，PDF/LaTex论文翻译&总结功能，支持并行问询多种LLM模型，支持chatglm3等本地模型。接入通义千问, deepseekcoder, 讯飞星火, 文心一言, llama2, rwkv, claude2, moss等。

68091

1周前

+16today

Scrapy

crawler

Scrapy, a fast high-level web crawling & scraping framework for Python.

54767

1周前

+17today

LLaMA Factory

Hot

agent

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

46052

1周前

+96today

LLMs From Scratch

Hot

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

43585

7个月前

+79today

Flowise

artificial-intelligence

Drag & drop UI to build your customized LLM flow

36901

1周前

+42today

Colly

crawler

Elegant Scraper and Crawler Framework for Golang

23984

1周前

+7today

Chinese LLaMA Alpaca

alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

18780

1周前

+2today

Crawlee

apify

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

17336

1周前

+8today