web-scraping
Enables scraping of structured information from websites using standard CSS and XPath selection patterns.
Install
mkdir -p .claude/skills/web-scraping && curl -L -o skill.zip "https://agentskills.codes/api/skills/download/11755" && unzip -o skill.zip -d .claude/skills/web-scraping && rm skill.zipInstalls to .claude/skills/web-scraping
Activation
This is the description your AI agent reads to decide when to run this skill — the better it matches your request, the more reliably it fires.
Extract structured data from web pages using CSS selectors and XPathKey capabilities
- →Fetch HTML content from URLs
- →Parse and extract specific elements
- →Handle pagination
- →Output data in JSON format
- →Output data in CSV format
How it works
This skill fetches HTML content from specified URLs and then parses it to extract data using CSS selectors, XPath expressions, or text patterns. It can also manage pagination.
Inputs & outputs
When to use web-scraping
- →Scraping product pricing from a site
- →Collecting articles from a news page
- →Extracting table data to JSON
About this skill
Web Scraping
Extract structured data from web pages.
Capabilities
- Fetch HTML content from URLs
- Parse and extract specific elements (tables, lists, text)
- Handle pagination
- Output in JSON or CSV format
Supported Selectors
- CSS selectors:
.class,#id,tag - XPath expressions
- Text patterns (regex)
Rate Limiting
Always respect robots.txt and implement delays between requests. Default delay: 1 second between requests.
Example
Scrape product names and prices from example.com/products
Output as JSON with fields: name, price, url
When not to use it
- →The website's robots.txt disallows scraping
- →The task requires interacting with dynamic content that is not rendered in the initial HTML
Limitations
- →Must respect robots.txt
- →Implements a default delay of 1 second between requests
- →Limited to extracting data from web pages
How it compares
This skill automates the extraction of structured data from web pages using defined selectors, which is more efficient than manual data collection.
Compared to similar skills
web-scraping side by side with the closest alternatives in the catalog.
| Skill | Installs | Updated | Safety | Difficulty |
|---|---|---|---|---|
| web-scraping (this skill) | 0 | 7mo | No flags | Beginner |
| firecrawl-scraper | 24 | 8mo | Caution | Beginner |
| web-scraper | 0 | 3mo | Review | Intermediate |
| google-maps-reviews-api-skill | 0 | 1mo | Review | Beginner |
Try saying
Example prompts that trigger this skill in your AI assistant.
You might also like
firecrawl-scraper
jackspace
Scrape and extract web content, convert HTML to markdown, and bypass bot protection for dynamic sites using Firecrawl API.
web-scraper
KevanPatira
Web scraping inteligente multi-estrategia. Extrai dados estruturados de paginas web (tabelas, listas, precos). Paginacao, monitoramento e export CSV/JSON.
google-maps-reviews-api-skill
aiskillstore
This skill is designed to help users automatically extract reviews from Google Maps via the Google Maps Reviews API. Agent should proactively apply this skill when users request to find reviews for local businesses (e.g., coffee shops, clinics), monitor customer feedback for a specific brand or loca
web-scanning
josephblais
Web content discovery, URL crawling, and page inventory for accessibility audits. Use when scanning web pages, crawling sites for audit scope, or building page inventories for multi-page audits.
indexing-issue-auditor
ranbot-ai
High-level technical SEO and site architecture auditor. Invoke to scan local or live environments for indexing, crawl budget, and structural errors.
crawl4ai
basher83
This skill should be used when users need to scrape websites, extract structured data, handle JavaScript-heavy pages, crawl multiple URLs, or build automated web data pipelines. Includes optimized extraction patterns with schema generation for efficient, LLM-free extraction.