Browse projects/Scrapling
Run Scrapling on your computer
☆ 100.7K Code updated 9 Nov 2026 Archived on GitHub🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Turn web pages into Markdown, plain text or HTML, with optional CSS selectors.
Save content from web pages
Give your agent the pages you want to read or collect. It can save each whole page or select the parts you need, such as articles, tables or product descriptions. Choose ordinary HTTP fetching or a browser for pages that render their content with JavaScript.
Scrapling’s command and MCP tools remain available for follow-up work. This runs a separate local browser; it does not use your signed-in browser or guarantee access to sites that block automated requests.
You bring
- a folder containing pages.json (an array of url or local file requests) and any local HTML files; each request can set name, format (md/txt/html), selector, browser, wait_selector and timeout
pages
You get
- one Markdown, text or HTML file per page, using its name when supplied
page-1.md - the source address, selection, response status and file checksums for every saved page
pages.json - Scrapling's full command line, including extraction options
- Scrapling's page-fetching and extraction tools in your AI app
Licences & source
Scrapling BSD-3-Clause; upstream browser and dependency notices remain in its image. UGT's driver and checks are Apache-2.0.
Setup & verification
- Fetch the web pages you choose, including JavaScript and page resources in browser mode.
- Scrapling at commit
3a4bb5b0a008
Run on clean x86-64 (Intel/AMD) and ARM64 machines on 10 Nov 2026. Its check passed on a known sample and rejected deliberately broken results.
This checks the job this page describes, not every feature of the original project.
Known issues
- Supply http/https URLs, or a file path inside inputs for static local HTML extraction. Browser mode needs a URL, runs on the CPU and may take longer. Respect the site's access rules; blocked requests and empty selections fail with an explanation.
- The batch stops at the first failed page. A partial folder is not a completed result; only a successful run writes pages.json. Original fetched HTML is retained in outputs/sources for checking.
