freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: style adjustment

Browse Source
This commit is contained in:
Karim shoair
2026-03-30 03:53:56 +02:00
parent 1f1e475772
commit a403156b2e
36 changed files with 159 additions and 159 deletions
Download Patch File Download Diff File Expand all files Collapse all files
agent-skill/Scrapling-Skill/references/spiders/requests-responses.md
+8 -8
View File
@@ -1,6 +1,6 @@
# Requests & Responses
This page covers the `Request` object in detail — how to construct requests, pass data between callbacks, control priority and deduplication, and use `response.follow()` for link-following.
This page covers the `Request` object in detail: how to construct requests, pass data between callbacks, control priority and deduplication, and use `response.follow()` for link-following.
## The Request Object
@@ -25,7 +25,7 @@ Here are all the arguments you can pass to `Request`:
| Argument | Type | Default | Description |
|---------------|------------|------------|-------------------------------------------------------------------------------------------------------|
| `url` | `str` | *required* | The URL to fetch |
| `sid` | `str` | `""` | Session ID — routes the request to a specific session (see [Sessions](sessions.md)) |
| `sid` | `str` | `""` | Session ID - routes the request to a specific session (see [Sessions](sessions.md)) |
| `callback` | `callable` | `None` | Async generator method to process the response. Defaults to `parse()` |
| `priority` | `int` | `0` | Higher values are processed first |
| `dont_filter` | `bool` | `False` | If `True`, skip deduplication (allow duplicate requests) |
@@ -54,7 +54,7 @@ yield Request(
```python
async def parse(self, response: Response):
# Minimal — inherits callback, sid, priority from current request
# Minimal - inherits callback, sid, priority from current request
yield response.follow("/next-page")
# Override specific fields
@@ -95,9 +95,9 @@ yield response.follow("/page", referer_flow=False)
Callbacks are async generator methods on your spider that process responses. They must `yield` one of three types:
- **`dict`** — A scraped item, added to the results
- **`Request`** — A follow-up request, added to the queue
- **`None`** — Silently ignored
- **`dict`**: A scraped item, added to the results
- **`Request`**: A follow-up request, added to the queue
- **`None`**: Silently ignored
```python
class MySpider(Spider):
@@ -124,11 +124,11 @@ Requests with higher priority values are processed first. This is useful when so
```python
async def parse(self, response: Response):
# High priority — process product pages first
# High priority - process product pages first
for link in response.css("a.product::attr(href)").getall():
yield response.follow(link, callback=self.parse_product, priority=10)
# Low priority — pagination links processed after products
# Low priority - pagination links processed after products
next_page = response.css("a.next::attr(href)").get()
if next_page:
yield response.follow(next_page, callback=self.parse, priority=0)