docs: style adjustment
This commit is contained in:
Karim shoair
@@ -53,7 +53,7 @@
|
||||
|
||||
Scrapling is an adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl.
|
||||
|
||||
Its parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls with pause/resume and automatic proxy rotation — all in a few lines of Python. One library, zero compromises.
|
||||
Its parser learns from website changes and automatically relocates your elements when pages update. Its fetchers bypass anti-bot systems like Cloudflare Turnstile out of the box. And its spider framework lets you scale up to concurrent, multi-session crawls with pause/resume and automatic proxy rotation - all in a few lines of Python. One library, zero compromises.
|
||||
|
||||
Blazing fast crawls with real-time stats and streaming. Built by Web Scrapers for Web Scrapers and regular users, there's something for everyone.
|
||||
|
||||
@@ -129,7 +129,7 @@ MySpider().start()
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> provides 900+ stable APIs across 16+ platforms including TikTok, X, YouTube & Instagram, with 40M+ datasets. <br /> Also offers <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">DISCOUNTED AI models</a> — Claude, GPT, GEMINI & more up to 71% off.
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> provides 900+ stable APIs across 16+ platforms including TikTok, X, YouTube & Instagram, with 40M+ datasets. <br /> Also offers <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">DISCOUNTED AI models</a> - Claude, GPT, GEMINI & more up to 71% off.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
@@ -196,12 +196,12 @@ MySpider().start()
|
||||
|
||||
## Key Features
|
||||
|
||||
### Spiders — A Full Crawling Framework
|
||||
### Spiders - A Full Crawling Framework
|
||||
- 🕷️ **Scrapy-like Spider API**: Define spiders with `start_urls`, async `parse` callbacks, and `Request`/`Response` objects.
|
||||
- ⚡ **Concurrent Crawling**: Configurable concurrency limits, per-domain throttling, and download delays.
|
||||
- 🔄 **Multi-Session Support**: Unified interface for HTTP requests, and stealthy headless browsers in a single spider — route requests to different sessions by ID.
|
||||
- 🔄 **Multi-Session Support**: Unified interface for HTTP requests, and stealthy headless browsers in a single spider - route requests to different sessions by ID.
|
||||
- 💾 **Pause & Resume**: Checkpoint-based crawl persistence. Press Ctrl+C for a graceful shutdown; restart to resume from where you left off.
|
||||
- 📡 **Streaming Mode**: Stream scraped items as they arrive via `async for item in spider.stream()` with real-time stats — ideal for UI, pipelines, and long-running crawls.
|
||||
- 📡 **Streaming Mode**: Stream scraped items as they arrive via `async for item in spider.stream()` with real-time stats - ideal for UI, pipelines, and long-running crawls.
|
||||
- 🛡️ **Blocked Request Detection**: Automatic detection and retry of blocked requests with customizable logic.
|
||||
- 📦 **Built-in Export**: Export results through hooks and your own pipeline or the built-in JSON/JSONL with `result.items.to_json()` / `result.items.to_jsonl()` respectively.
|
||||
|
||||
@@ -328,7 +328,7 @@ Pause and resume long crawls with checkpoints by running the spider like this:
|
||||
```python
|
||||
QuotesSpider(crawldir="./crawl_data").start()
|
||||
```
|
||||
Press Ctrl+C to pause gracefully — progress is saved automatically. Later, when you start the spider again, pass the same `crawldir`, and it will resume from where it stopped.
|
||||
Press Ctrl+C to pause gracefully - progress is saved automatically. Later, when you start the spider again, pass the same `crawldir`, and it will resume from where it stopped.
|
||||
|
||||
### Advanced Parsing & Navigation
|
||||
```python
|
||||
@@ -413,7 +413,7 @@ scrapling extract stealthy-fetch 'https://nopecha.com/demo/cloudflare' captchas.
|
||||
|
||||
## Performance Benchmarks
|
||||
|
||||
Scrapling isn't just powerful—it's also blazing fast. The following benchmarks compare Scrapling's parser with the latest versions of other popular libraries.
|
||||
Scrapling isn't just powerful-it's also blazing fast. The following benchmarks compare Scrapling's parser with the latest versions of other popular libraries.
|
||||
|
||||
### Text Extraction Speed Test (5000 nested elements)
|
||||
|
||||
@@ -525,7 +525,7 @@ This work is licensed under the BSD-3-Clause License.
|
||||
## Acknowledgments
|
||||
|
||||
This project includes code adapted from:
|
||||
- Parsel (BSD License)—Used for [translator](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/core/translator.py) submodule
|
||||
- Parsel (BSD License)-Used for [translator](https://github.com/D4Vinci/Scrapling/blob/main/scrapling/core/translator.py) submodule
|
||||
|
||||
---
|
||||
<div align="center"><small>Designed & crafted with ❤️ by Karim Shoair.</small></div><br>
|
||||
|
||||
Reference in New Issue
Block a user