Popular GitHub Repositories for Scraper

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Use LLMs to dig out what you care about from massive amounts of information and a variety of sources daily.

Turn any webpage into structured data using LLMs

A community-driven way to read and chat with AI bots - powered by chatGPT.

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Automate webpages at scale, scrape web data completely and accurately with high performance, distributed AI-RPA.

Lego AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.

Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.

ScraperAI is an open-source, AI-powered tool designed to simplify web scraping for users of all skill levels.

Sentiment Analysis of news on stock prices

?Free ChatGPT Line Bot with Horoscope, Music Broadcast, Google Image Search...

Implementation of "Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading." In Findings of ACL2021

Automate your LinkedIn job applications with AI! This bot utilizes GPT models such as GPT-4, GPT-3.5, and Google's Gemini Pro for Easy Apply form filling, customizable to your preferences and job search criteria.

ScrapeGPT is a RAG-based Telegram bot designed to scrape and analyze websites, then answer questions based on the scraped content. The bot utilizes Retrieval Augmented Generation and webscraping to return natural language answers to the user's queries.

Reddit_Commentator_AIHawk is a Python project showcasing the power of artificial intelligence in social media interaction. This tool demonstrates AI's capability to generate contextually relevant Reddit comments using GPT models and LangChain. It automates the process of analyzing trending posts and creating engaging responses.

RAG Web Browser is an Apify Actor to feed your LLM applications and RAG pipelines with up-to-date text content scraped from the web.

A simple library designed for rapid bot and ai scraper development using Selenium, the POM (Page Object Model) design and llm data extraction.

Powerful trading signals bot, based on ChatGPT sentiment analysis of news headlines.

AI web scraper built with Crawl4AI for extracting structured leads data from websites.

Code to scrape course reviews from Coursera and perform sentiment analysis and topic modeling