webclaw
A powerful web scraping engine that handles Cloudflare and antibot protections to reliably fetch and structure web data.
Install
mkdir -p .claude/skills/webclaw && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/8893" && unzip -o skill.zip -d .claude/skills/webclaw && rm skill.zipInstalls to .claude/skills/webclaw
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Web extraction engine with antibot bypass. Scrape, crawl, extract, summarize, search, map, diff, monitor, research, and analyze any URL — including Cloudflare-protected sites. Use when you need reliable web content, the built-in web_fetch fails, or you need structured data extraction from web pages.Key capabilities
- →Extract content from URLs including bot-protected sites
- →Crawl entire websites with depth and page limits
- →Perform structured data extraction using JSON schemas
- →Summarize web pages and detect content changes
- →Map site structures to discover all URLs
How it works
The engine uses a local-first approach, falling back to a cloud-based extraction service for JavaScript-rendered or bot-protected pages when an API key is provided.
Inputs & outputs
When to use webclaw
- →Extracting product pricing tables
- →Crawling entire sites for data
- →Monitoring web content for changes
- →Summarizing long web pages
About webclaw
Extracts content from any URL, including sites with Cloudflare or DataDome protection. It performs scraping, crawling, summarizing, and structured data extraction with built-in antibot bypass.
Web extraction engine with antibot bypass. Scrape, crawl, extract, summarize, search, map, diff, monitor, research, and analyze any URL — including Cloudflare-protected sites. Use when you need reliable web content, the built-in web_fetch fails, or you need structured data extraction from web pages.
When not to use it
- →When simple, fast lookups are sufficient and web_fetch suffices
- →When no WEBCLAW_API_KEY is available for protected sites
Prerequisites
Limitations
- →Requires an API key for bot-protected or JavaScript-rendered content
- →Rate limiting is managed server-side
How it compares
Unlike standard fetch tools, this engine includes a 20-step optimization pipeline and cloud-based antibot bypass for reliable data extraction.
Compared to similar skills
webclaw side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| webclaw (this skill) | 1 | 4mo | Review | Intermediate |
| scientific-brainstorming | 37 | 7mo | No flags | Intermediate |
| brightdata-web-mcp | 9 | 6mo | Review | Intermediate |
| web-scraper | 0 | 1mo | No flags | Intermediate |
Try saying
Example prompts that trigger this skill in your AI assistant.
You might also like
scientific-brainstorming
davila7
Research ideation partner. Generate hypotheses, explore interdisciplinary connections, challenge assumptions, develop methodologies, identify research gaps, for creative scientific problem-solving.
brightdata-web-mcp
patchy631
Search the web, scrape websites, extract structured data from URLs, and automate browsers using Bright Data's Web MCP. Use when fetching live web content, bypassing blocks/CAPTCHAs, getting product data from Amazon/eBay, social media posts, or when standard requests fail.
web-scraper
shobcoder
Scrape, crawl, and extract data from websites. Use when users ask to scrape web pages, extract content, crawl websites, or collect data from the internet.
graphify-knowledge-graph
Aradotso
> Skill by [ara.so](https://ara.so) — Daily 2026 Skills collection.
yams
trvon
Code indexing, semantic search, and knowledge graph for project memory
youtube-comments-api-skill
aiskillstore
This skill helps users extract structured video list data and comment data from YouTube using the BrowserAct API. The Agent should proactively apply this skill when users request searching for YouTube videos and their comments, analyzing viewer sentiment for a specific video topic, gathering audienc