Extract and crawl data from websites using structured prompts and batch processing.

Install

mkdir -p .claude/skills/web-scraper-shobcoder && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/10989" && unzip -o skill.zip -d .claude/skills/web-scraper-shobcoder && rm skill.zip

Installs to .claude/skills/web-scraper-shobcoder

Activation

This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.

Scrape, crawl, and extract data from websites. Use when users ask to scrape web pages, extract content, crawl websites, or collect data from the internet.
154 chars✓ has a “when” trigger
Intermediate

Key capabilities

  • →Crawls websites
  • →Extracts structured data
  • →Supports batch URL processing
  • →Handles dynamic content

How it works

It uses various scraping techniques, including browser rendering, to extract structured content from websites based on user prompts.

Inputs & outputs

You give it
URLs and extraction prompts
You get back
Structured JSON data

When to use web-scraper

  • →Scraping articles for analysis
  • →Gathering product price lists
  • →Extracting structured data from web pages

About web-scraper

Automates website crawling and content extraction into structured JSON formats, supporting research and data collection tasks.

Scrape, crawl, and extract data from websites. Use when users ask to scrape web pages, extract content, crawl websites, or collect data from the internet.

When not to use it

  • →When the website prohibits scraping
  • →When the user needs real-time interaction

Limitations

  • →Respect website terms of service
  • →Dynamic content may require browser mode

How it compares

It provides a structured, prompt-based approach to web scraping rather than manual parsing.

Compared to similar skills

web-scraper side by side with the closest alternatives in the catalog.

SkillInstallsUpdatedSafetyDifficulty
web-scraper (this skill)05moNo flagsIntermediate
pi-share18moReviewBeginner
apify-trend-analysis06moReviewIntermediate
brightdata-web-mcp99moReviewIntermediate

Try saying

Example prompts that trigger this skill in your AI assistant.

You might also like

pi-share

mitsuhiko

Load and parse session transcripts from shittycodingagent.ai/buildwithpi.ai/buildwithpi.com (pi-share) URLs. Fetches gists, decodes embedded session data, and extracts conversation history.

14

apify-trend-analysis

Anhvu1107

ALWAYS use this when the request matches Apify Trend Analysis: Discover and track emerging trends across Google Trends, Instagram, Facebook, YouTube, and TikTok to inform content strategy.

00

brightdata-web-mcp

patchy631

Search the web, scrape websites, extract structured data from URLs, and automate browsers using Bright Data's Web MCP. Use when fetching live web content, bypassing blocks/CAPTCHAs, getting product data from Amazon/eBay, social media posts, or when standard requests fail.

951

webclaw

0xmassi

Web extraction engine with antibot bypass. Scrape, crawl, extract, summarize, search, map, diff, monitor, research, and analyze any URL — including Cloudflare-protected sites. Use when you need reliable web content, the built-in web_fetch fails, or you need structured data extraction from web pages.

14

juicebox-core-workflow-b

jeremylongshore

Implement Juicebox candidate enrichment workflow. Use when enriching profile data, gathering additional candidate details, or building comprehensive candidate profiles. Trigger with phrases like "juicebox enrich profile", "juicebox candidate details", "enrich candidate data", "juicebox profile enrichment".

13

openevidence-core-workflow-b

jeremylongshore

Execute OpenEvidence DeepConsult workflow for comprehensive medical research. Use when implementing deep research synthesis, complex clinical questions, or when physicians need extensive literature review. Trigger with phrases like "openevidence deepconsult", "deep research", "comprehensive evidence", "literature synthesis".

00

Search skills

Search the agent skills registry