freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(agent): update skill with the latest changes

Browse Source
This commit is contained in:
Karim shoair
2026-04-05 19:42:00 +02:00
parent d0a19a6fe1
commit a61113fc29
5 changed files with 14 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agent-skill/Scrapling-Skill.zip
BIN
View File
Binary file not shown.
agent-skill/Scrapling-Skill/SKILL.md
+3 -3
View File
@@ -1,7 +1,7 @@
---
name: scrapling-official
description: Scrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. Use when asked to scrape, crawl, or extract data from websites; web_fetch fails; the site has anti-bot protections; write Python code to scrape/crawl; or write spiders.
version: "0.4.4"
version: "0.4.5"
license: Complete terms in LICENSE.txt
metadata:
homepage: "https://scrapling.readthedocs.io/en/latest/index.html"
@@ -40,7 +40,7 @@ Blazing fast crawls with real-time stats and streaming. Built by Web Scrapers fo
Create a virtual Python environment through any way available, like `venv`, then inside the environment do:
`pip install "scrapling[all]>=0.4.4"`
`pip install "scrapling[all]>=0.4.5"`
Then do this to download all the browsers' dependencies:
@@ -104,7 +104,7 @@ Those options are shared between the 4 HTTP request commands:
| --proxy | TEXT | Proxy URL in format "http://username:password@host:port" |
| -s, --css-selector | TEXT | CSS selector to extract specific content from the page. It returns all matches. |
| -p, --params | TEXT | Query parameters in format "key=value" (can be used multiple times) |
| --follow-redirects / --no-follow-redirects | None | Whether to follow redirects (default: True) |
| --follow-redirects / --no-follow-redirects | None | Whether to follow redirects (default: "safe", rejects redirects to internal/private IPs) |
| --verify / --no-verify | None | Whether to verify SSL certificates (default: True) |
| --impersonate | TEXT | Browser to impersonate. Can be a single browser (e.g., Chrome) or a comma-separated list for random selection (e.g., Chrome, Firefox, Safari). |
| --stealthy-headers / --no-stealthy-headers | None | Use stealthy browser headers (default: True) |
agent-skill/Scrapling-Skill/examples/README.md
+1 -1
View File
@@ -9,7 +9,7 @@ All examples collect **all 100 quotes across 10 pages**.
Make sure Scrapling is installed:
```bash
pip install "scrapling[all]>=0.4.4"
pip install "scrapling[all]>=0.4.5"
scrapling install --force
```
agent-skill/Scrapling-Skill/references/fetching/static.md
+9 -9
View File
@@ -15,7 +15,7 @@ All methods for making requests here share some arguments, so let's discuss them
- **url**: The targeted URL
- **stealthy_headers**: If enabled (default), it creates and adds real browser headers. It also sets a Google referer header.
- **follow_redirects**: As the name implies, tell the fetcher to follow redirections. **Enabled by default**
- **follow_redirects**: Controls redirect behavior. **Defaults to `"safe"`**, which follows redirects but rejects those targeting internal/private IPs (SSRF protection). Pass `True` to follow all redirects without restriction, or `False` to disable redirects entirely.
- **timeout**: The number of seconds to wait for each request to be finished. **Defaults to 30 seconds**.
- **retries**: The number of retries that the fetcher will do for failed requests. **Defaults to three retries**.
- **retry_delay**: Number of seconds to wait between retry attempts. **Defaults to 1 second**.
@@ -50,7 +50,7 @@ Examples are the best way to explain this:
>>> from scrapling.fetchers import Fetcher
>>> # Basic GET
>>> page = Fetcher.get('https://example.com')
>>> page = Fetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True, follow_redirects=True)
>>> page = Fetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True)
>>> page = Fetcher.get('https://scrapling.requestcatcher.com/get', proxy='http://username:password@localhost:8030')
>>> # With parameters
>>> page = Fetcher.get('https://example.com/search', params={'q': 'query'})
@@ -69,7 +69,7 @@ And for asynchronous requests, it's a small adjustment
>>> from scrapling.fetchers import AsyncFetcher
>>> # Basic GET
>>> page = await AsyncFetcher.get('https://example.com')
>>> page = await AsyncFetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True, follow_redirects=True)
>>> page = await AsyncFetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True)
>>> page = await AsyncFetcher.get('https://scrapling.requestcatcher.com/get', proxy='http://username:password@localhost:8030')
>>> # With parameters
>>> page = await AsyncFetcher.get('https://example.com/search', params={'q': 'query'})
@@ -105,7 +105,7 @@ The `page` object in all cases is a [Response](choosing.md#response-object) obje
>>> from scrapling.fetchers import Fetcher
>>> # Basic POST
>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, params={'q': 'query'})
>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, stealthy_headers=True, follow_redirects=True)
>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, stealthy_headers=True)
>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030', impersonate="chrome")
>>> # Another example of form-encoded data
>>> page = Fetcher.post('https://example.com/submit', data={'username': 'user', 'password': 'pass'}, http3=True)
@@ -117,7 +117,7 @@ And for asynchronous requests, it's a small adjustment
>>> from scrapling.fetchers import AsyncFetcher
>>> # Basic POST
>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'})
>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, stealthy_headers=True, follow_redirects=True)
>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, stealthy_headers=True)
>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030', impersonate="chrome")
>>> # Another example of form-encoded data
>>> page = await AsyncFetcher.post('https://example.com/submit', data={'username': 'user', 'password': 'pass'}, http3=True)
@@ -129,7 +129,7 @@ And for asynchronous requests, it's a small adjustment
>>> from scrapling.fetchers import Fetcher
>>> # Basic PUT
>>> page = Fetcher.put('https://example.com/update', data={'status': 'updated'})
>>> page = Fetcher.put('https://example.com/update', data={'status': 'updated'}, stealthy_headers=True, follow_redirects=True, impersonate="chrome")
>>> page = Fetcher.put('https://example.com/update', data={'status': 'updated'}, stealthy_headers=True, impersonate="chrome")
>>> page = Fetcher.put('https://example.com/update', data={'status': 'updated'}, proxy='http://username:password@localhost:8030')
>>> # Another example of form-encoded data
>>> page = Fetcher.put("https://scrapling.requestcatcher.com/put", data={'key': ['value1', 'value2']})
@@ -139,7 +139,7 @@ And for asynchronous requests, it's a small adjustment
>>> from scrapling.fetchers import AsyncFetcher
>>> # Basic PUT
>>> page = await AsyncFetcher.put('https://example.com/update', data={'status': 'updated'})
>>> page = await AsyncFetcher.put('https://example.com/update', data={'status': 'updated'}, stealthy_headers=True, follow_redirects=True, impersonate="chrome")
>>> page = await AsyncFetcher.put('https://example.com/update', data={'status': 'updated'}, stealthy_headers=True, impersonate="chrome")
>>> page = await AsyncFetcher.put('https://example.com/update', data={'status': 'updated'}, proxy='http://username:password@localhost:8030')
>>> # Another example of form-encoded data
>>> page = await AsyncFetcher.put("https://scrapling.requestcatcher.com/put", data={'key': ['value1', 'value2']})
@@ -149,14 +149,14 @@ And for asynchronous requests, it's a small adjustment
```python
>>> from scrapling.fetchers import Fetcher
>>> page = Fetcher.delete('https://example.com/resource/123')
>>> page = Fetcher.delete('https://example.com/resource/123', stealthy_headers=True, follow_redirects=True, impersonate="chrome")
>>> page = Fetcher.delete('https://example.com/resource/123', stealthy_headers=True, impersonate="chrome")
>>> page = Fetcher.delete('https://example.com/resource/123', proxy='http://username:password@localhost:8030')
```
And for asynchronous requests, it's a small adjustment
```python
>>> from scrapling.fetchers import AsyncFetcher
>>> page = await AsyncFetcher.delete('https://example.com/resource/123')
>>> page = await AsyncFetcher.delete('https://example.com/resource/123', stealthy_headers=True, follow_redirects=True, impersonate="chrome")
>>> page = await AsyncFetcher.delete('https://example.com/resource/123', stealthy_headers=True, impersonate="chrome")
>>> page = await AsyncFetcher.delete('https://example.com/resource/123', proxy='http://username:password@localhost:8030')
```
agent-skill/Scrapling-Skill/references/mcp-server.md
+1 -1
View File
@@ -27,7 +27,7 @@ Fast HTTP GET with browser fingerprint impersonation (TLS, headers). Suitable fo
| `retry_delay` | int | 1 | Seconds between retries |
| `stealthy_headers` | bool | true | Generate realistic browser headers and Google referer |
| `http3` | bool | false | Use HTTP/3 (may conflict with `impersonate`) |
| `follow_redirects` | bool | true | Follow HTTP redirects |
| `follow_redirects` | bool or "safe" | "safe" | Follow redirects. "safe" rejects redirects to internal/private IPs |
| `max_redirects` | int | 30 | Max redirects (-1 for unlimited) |
| `headers` | dict or null | null | Custom request headers |
| `cookies` | dict or null | null | Request cookies |