BigHugger
GH Repository · ScrapeGraphAI

Scrapegraph-ai

Python scraper based on AI

stars
31,050
30-day movement
+15251/day
Related entries
60
Connections
2
python/uvmakedockerfirecrawl-alternativewebscrapingweb-scraperweb-searchweb-data-extractionscrapingweb-scrapingragscraping-pythonpythonweb-datallmai-searchweb-crawlerai-scrapingcrawlermarkdownlarge-language-modeldata-extractionweb-crawlersPython

Scrapegraph-ai is a Python-based web scraper that uses large language models to drive scraping and crawling, tagged around RAG, markdown output, and web data extraction. The repository ships an AGENTS.md and includes toolchain files for Docker, Make, and Python (uv).

Reach for it when you want LLM-guided scraping and crawling in Python rather than hand-written extraction rules.

Use it to

  • Scrape web pages with LLM-driven extraction
  • Crawl sites and collect web data
  • Feed scraped content into RAG pipelines
  • Extract data as markdown
  • Explore a self-described firecrawl alternative

For Python developers building AI-powered scraping pipelines

Role
rag
Language
Python
Licence
MIT
Forks
3,126
Open issues
6
Last push
2026-09-07
Latest release
v0.2.8 · 2024-04-26
topicsweb-scrapingllmragcrawlerdata-extractionpython