docs: add new api references
This commit is contained in:
@@ -0,0 +1,18 @@
|
||||
---
|
||||
search:
|
||||
exclude: true
|
||||
---
|
||||
|
||||
# Proxy Rotation
|
||||
|
||||
The `ProxyRotator` class provides thread-safe proxy rotation for any fetcher or session.
|
||||
|
||||
You can import it directly like below:
|
||||
|
||||
```python
|
||||
from scrapling.fetchers import ProxyRotator
|
||||
```
|
||||
|
||||
## ::: scrapling.engines.toolbelt.proxy_rotation.ProxyRotator
|
||||
handler: python
|
||||
:docstring:
|
||||
@@ -0,0 +1,18 @@
|
||||
---
|
||||
search:
|
||||
exclude: true
|
||||
---
|
||||
|
||||
# Response Class
|
||||
|
||||
The `Response` class wraps HTTP responses returned by all fetchers, providing access to status, headers, body, cookies, and a `Selector` for parsing.
|
||||
|
||||
You can import the `Response` class like below:
|
||||
|
||||
```python
|
||||
from scrapling.engines.toolbelt.custom import Response
|
||||
```
|
||||
|
||||
## ::: scrapling.engines.toolbelt.custom.Response
|
||||
handler: python
|
||||
:docstring:
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
search:
|
||||
exclude: true
|
||||
---
|
||||
|
||||
# Spider Classes
|
||||
|
||||
Here's the reference information for the spider framework classes' parameters, attributes, and methods.
|
||||
|
||||
You can import them directly like below:
|
||||
|
||||
```python
|
||||
from scrapling.spiders import Spider, Request, CrawlResult, SessionManager, Response
|
||||
```
|
||||
|
||||
## ::: scrapling.spiders.Spider
|
||||
handler: python
|
||||
:docstring:
|
||||
|
||||
## ::: scrapling.spiders.Request
|
||||
handler: python
|
||||
:docstring:
|
||||
|
||||
## Result Classes
|
||||
|
||||
## ::: scrapling.spiders.result.CrawlResult
|
||||
handler: python
|
||||
:docstring:
|
||||
|
||||
## ::: scrapling.spiders.result.CrawlStats
|
||||
handler: python
|
||||
:docstring:
|
||||
|
||||
## ::: scrapling.spiders.result.ItemList
|
||||
handler: python
|
||||
:docstring:
|
||||
|
||||
## Session Management
|
||||
|
||||
## ::: scrapling.spiders.session.SessionManager
|
||||
handler: python
|
||||
:docstring:
|
||||
+4
-1
@@ -54,7 +54,10 @@ nav = [
|
||||
{Selector = "api-reference/selector.md"},
|
||||
{Fetchers = "api-reference/fetchers.md"},
|
||||
{"MCP Server" = "api-reference/mcp-server.md"},
|
||||
{"Custom Types" = "api-reference/custom-types.md"}
|
||||
{"Custom Types" = "api-reference/custom-types.md"},
|
||||
{Response = "api-reference/response.md"},
|
||||
{Spiders = "api-reference/spiders.md"},
|
||||
{"Proxy Rotation" = "api-reference/proxy-rotation.md"}
|
||||
]},
|
||||
{"Writing your retrieval system" = "development/adaptive_storage_system.md"},
|
||||
{"Using Scrapling's custom types" = "development/scrapling_custom_types.md"}
|
||||
|
||||
Reference in New Issue
Block a user