Merge branch 'dev' into docs/fix-contributing-docs-link
This commit is contained in:
Karim shoair
@@ -73,7 +73,7 @@ jobs:
|
||||
- name: Install all browsers dependencies
|
||||
run: |
|
||||
python3 -m pip install --upgrade pip
|
||||
python3 -m pip install playwright==1.59.0 patchright==1.59.1
|
||||
python3 -m pip install playwright==1.60.0 patchright==1.60.1
|
||||
|
||||
- name: Get Playwright version
|
||||
id: playwright-version
|
||||
|
||||
@@ -141,16 +141,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> provides 900+ stable APIs across 16+ platforms including TikTok, X, YouTube & Instagram, with 40M+ datasets. <br /> Also offers <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">DISCOUNTED AI models</a> - Claude, GPT, GEMINI & more up to 71% off.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> provides fast Residential and ISP proxies for developers and scrapers. Global IP coverage, high anonymity, smart rotation, and reliable performance for automation and data extraction. Use <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> to simplify large-scale web crawling.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -172,16 +162,6 @@ MySpider().start()
|
||||
Read a full review of <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">Scrapling on The Web Scraping Club</a> (Nov 2025), the #1 newsletter dedicated to Web Scraping.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">Stable proxies</a> for scraping, automation, and multi-accounting. Clean IPs, fast response, and reliable performance under load. Built for scalable workflows.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -192,22 +172,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> provides scalable residential proxies with 80M+ IPs across 195+ countries, delivering fast, reliable connections, automatic rotation, and strong anti-block performance. Free trial available.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> provides residential proxies from just $0.018/IP or $0.68/GB. 20M+ IPs across 90+ countries. Sticky or rotating sessions, managed from desktop or mobile app.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - reliable proxy provider with the highest quality IP on the market. Use promo code SCRAPLING35 for 35% discount on proxies.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>Do you want to show your ad here? Click [here](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# Sponsors
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: scrapling-official
|
||||
description: Scrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. Use when asked to scrape, crawl, or extract data from websites; web_fetch fails; the site has anti-bot protections; write Python code to scrape/crawl; or write spiders.
|
||||
version: "0.4.8"
|
||||
version: "0.4.9"
|
||||
license: Complete terms in LICENSE.txt
|
||||
metadata:
|
||||
homepage: "https://scrapling.readthedocs.io/en/latest/index.html"
|
||||
@@ -40,7 +40,7 @@ Blazing fast crawls with real-time stats and streaming. Built by Web Scrapers fo
|
||||
|
||||
Create a virtual Python environment through any way available, like `venv`, then inside the environment do:
|
||||
|
||||
`pip install "scrapling[all]>=0.4.8"`
|
||||
`pip install "scrapling[all]>=0.4.9"`
|
||||
|
||||
Then do this to download all the browsers' dependencies:
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ All examples collect **all 100 quotes across 10 pages**.
|
||||
Make sure Scrapling is installed:
|
||||
|
||||
```bash
|
||||
pip install "scrapling[all]>=0.4.8"
|
||||
pip install "scrapling[all]>=0.4.9"
|
||||
scrapling install --force
|
||||
```
|
||||
|
||||
|
||||
@@ -208,7 +208,7 @@ Docker alternative:
|
||||
|
||||
```bash
|
||||
docker pull pyd4vinci/scrapling
|
||||
docker run -i --rm scrapling mcp
|
||||
docker run -i --rm pyd4vinci/scrapling mcp
|
||||
```
|
||||
|
||||
The MCP server name when registering with a client is `ScraplingServer`. The command is the path to the `scrapling` binary and the argument is `mcp`.
|
||||
+20
-25
@@ -137,16 +137,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> يوفر أكثر من 900 واجهة API مستقرة عبر أكثر من 16 منصة تشمل TikTok و X و YouTube و Instagram، مع أكثر من 40 مليون مجموعة بيانات. <br /> يقدم أيضاً <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">نماذج ذكاء اصطناعي بأسعار مخفضة</a> - Claude و GPT و GEMINI والمزيد بخصم يصل إلى 71%.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> يوفر بروكسيات سكنية و ISP سريعة للمطورين والسكرابرز. تغطية IP عالمية، إخفاء هوية عالي، تدوير ذكي، وأداء موثوق للأتمتة واستخراج البيانات. استخدم <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> لتبسيط زحف الويب على نطاق واسع.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -168,16 +158,6 @@ MySpider().start()
|
||||
اقرأ مراجعة كاملة عن <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">Scrapling على The Web Scraping Club</a> (نوفمبر 2025)، النشرة الإخبارية الأولى المخصصة لكشط الويب.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">بروكسيات مستقرة</a> للكشط والأتمتة وإدارة الحسابات المتعددة. عناوين IP نظيفة، استجابة سريعة، وأداء موثوق تحت الضغط. مصممة لسير العمل القابل للتوسع.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -188,23 +168,38 @@ MySpider().start()
|
||||
يوفر <a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> بروكسيات سكنية قابلة للتوسع مع أكثر من 80 مليون عنوان IP في أكثر من 195 دولة، ويقدم اتصالات سريعة وموثوقة، وتدوير تلقائي، وأداء قوي ضد الحظر. تجربة مجانية متاحة.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
يوفر <a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> بروكسيات سكنية بدءًا من 0.018 دولار فقط لكل IP أو 0.68 دولار لكل جيجابايت. أكثر من 20 مليون عنوان IP في أكثر من 90 دولة. جلسات ثابتة أو متناوبة، تتم إدارتها من تطبيق سطح المكتب أو الجوال.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - مزود بروكسيات موثوق يقدم أعلى جودة IP في السوق. استخدم كود الخصم SCRAPLING35 للحصول على خصم 35% على البروكسيات.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>هل تريد عرض إعلانك هنا؟ انقر [هنا](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# الرعاة
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-24
@@ -137,16 +137,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> 提供覆盖 16+ 平台（包括 TikTok、X、YouTube 和 Instagram）的 900+ 稳定 API，拥有 4000 万+ 数据集。<br /> 还提供<a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">优惠 AI 模型</a> - Claude、GPT、GEMINI 等，最高优惠 71%。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> 提供面向开发者和爬虫的快速住宅和 ISP 代理。全球 IP 覆盖、高匿名性、智能轮换，以及可靠的自动化和数据提取性能。使用 <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> 简化大规模网页爬取。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -168,16 +158,6 @@ MySpider().start()
|
||||
阅读 <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">The Web Scraping Club 上关于 Scrapling 的完整评测</a>（2025 年 11 月），这是排名第一的网页抓取专业通讯。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">稳定的代理</a>，适用于数据抓取、自动化和多账号管理。干净的 IP、快速响应、高负载下可靠的性能。专为可扩展的工作流程而构建。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -188,22 +168,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> 提供可扩展的住宅代理，覆盖 195+ 国家/地区的 8000 万+ IP，提供快速可靠的连接、自动轮换和强大的反屏蔽性能。提供免费试用。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> 提供住宅代理，价格低至每个 IP 仅 $0.018 或每 GB $0.68。覆盖 90+ 国家/地区的 2000 万+ IP。支持固定或轮换会话，可通过桌面或移动应用进行管理。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - 市场上 IP 质量最高的可靠代理提供商。使用优惠码 SCRAPLING35 可享代理 35% 折扣。
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>想在这里展示您的广告吗？点击 [这里](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# 赞助商
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-24
@@ -137,16 +137,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> bietet über 900 stabile APIs auf mehr als 16 Plattformen, darunter TikTok, X, YouTube und Instagram, mit über 40 Mio. Datensätzen. <br /> Bietet außerdem <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">vergünstigte KI-Modelle</a> - Claude, GPT, GEMINI und mehr mit bis zu 71% Rabatt.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> bietet schnelle Residential- und ISP-Proxies für Entwickler und Scraper. Globale IP-Abdeckung, hohe Anonymität, intelligente Rotation und zuverlässige Leistung für Automatisierung und Datenextraktion. Verwenden Sie <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a>, um großflächiges Web-Crawling zu vereinfachen.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -168,16 +158,6 @@ MySpider().start()
|
||||
Lesen Sie eine vollständige Rezension von <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">Scrapling auf The Web Scraping Club</a> (Nov. 2025), dem führenden Newsletter für Web Scraping.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">Stabile Proxys</a> für Scraping, Automatisierung und Multi-Accounting. Saubere IPs, schnelle Reaktionszeiten und zuverlässige Leistung unter Last. Entwickelt für skalierbare Workflows.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -188,22 +168,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> bietet skalierbare Residential-Proxys mit über 80 Mio. IPs in mehr als 195 Ländern und liefert schnelle, zuverlässige Verbindungen, automatische Rotation und starke Anti-Block-Leistung. Kostenlose Testversion verfügbar.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> bietet Residential-Proxys ab nur 0,018 $/IP oder 0,68 $/GB. Über 20 Mio. IPs in mehr als 90 Ländern. Sticky oder rotierende Sessions, verwaltet über die Desktop- oder Mobile-App.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - zuverlässiger Proxy-Anbieter mit der höchsten IP-Qualität auf dem Markt. Nutze den Promo-Code SCRAPLING35 für 35% Rabatt auf Proxys.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>Möchten Sie Ihre Anzeige hier zeigen? Klicken Sie [hier](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# Sponsoren
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-24
@@ -137,16 +137,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> ofrece más de 900 APIs estables en más de 16 plataformas, incluyendo TikTok, X, YouTube e Instagram, con más de 40M de conjuntos de datos. <br /> También ofrece <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">modelos de IA con descuento</a> - Claude, GPT, GEMINI y más con hasta un 71% de descuento.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> ofrece proxies residenciales e ISP rápidos para desarrolladores y scrapers. Cobertura IP global, alto anonimato, rotación inteligente y rendimiento fiable para automatización y extracción de datos. Usa <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> para simplificar el crawling web a gran escala.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -168,16 +158,6 @@ MySpider().start()
|
||||
Lee una reseña completa de <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">Scrapling en The Web Scraping Club</a> (nov. 2025), el boletín número uno dedicado al Web Scraping.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">Proxies estables</a> para scraping, automatización y multicuentas. IPs limpias, respuesta rápida y rendimiento fiable bajo carga. Diseñado para flujos de trabajo escalables.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -188,22 +168,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> ofrece proxies residenciales escalables con más de 80 millones de IPs en más de 195 países, brindando conexiones rápidas y fiables, rotación automática y un sólido rendimiento anti-bloqueo. Prueba gratuita disponible.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> ofrece proxies residenciales desde solo $0,018/IP o $0,68/GB. Más de 20 millones de IPs en más de 90 países. Sesiones fijas o rotativas, gestionadas desde la aplicación de escritorio o móvil.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - proveedor de proxies confiable con la mayor calidad de IP del mercado. Usa el código promocional SCRAPLING35 para obtener un 35% de descuento en proxies.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>¿Quieres mostrar tu anuncio aquí? Haz clic [aquí](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# Patrocinadores
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-24
@@ -137,16 +137,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> propose plus de 900 APIs stables sur plus de 16 plateformes, dont TikTok, X, YouTube et Instagram, avec plus de 40M de jeux de données. <br /> Propose également des <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">modèles IA à prix réduit</a> - Claude, GPT, GEMINI et plus, jusqu'à 71% de réduction.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> fournit des proxies résidentiels et ISP rapides pour les développeurs et les scrapeurs. Couverture IP mondiale, anonymat élevé, rotation intelligente et performances fiables pour l'automatisation et l'extraction de données. Utilisez <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> pour simplifier le crawling web à grande échelle.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -168,16 +158,6 @@ MySpider().start()
|
||||
Lisez une critique complète de <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">Scrapling sur The Web Scraping Club</a> (nov. 2025), la newsletter n°1 dédiée au Web Scraping.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">Des proxys stables</a> pour le scraping, l'automatisation et la gestion multi-comptes. Des IPs propres, une réponse rapide et des performances fiables sous charge. Conçu pour des flux de travail évolutifs.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -188,22 +168,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> propose des proxys résidentiels évolutifs avec plus de 80 millions d'IPs dans plus de 195 pays, offrant des connexions rapides et fiables, une rotation automatique et de solides performances anti-blocage. Essai gratuit disponible.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> propose des proxys résidentiels à partir de seulement 0,018 $/IP ou 0,68 $/Go. Plus de 20 millions d'IPs dans plus de 90 pays. Sessions fixes ou rotatives, gérées depuis l'application de bureau ou mobile.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - fournisseur de proxys fiable offrant la meilleure qualité d'IP du marché. Utilisez le code promo SCRAPLING35 pour obtenir 35% de réduction sur les proxys.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>Vous souhaitez afficher votre publicité ici ? Cliquez [ici](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# Sponsors
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-24
@@ -137,16 +137,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> は TikTok、X、YouTube、Instagram を含む 16 以上のプラットフォームで 900 以上の安定した API を提供し、4,000 万以上のデータセットを保有。<br /> さらに <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">割引 AI モデル</a>も提供 - Claude、GPT、GEMINI など最大 71% オフ。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> は開発者やスクレイパー向けの高速なレジデンシャルおよび ISP プロキシを提供。グローバル IP カバレッジ、高い匿名性、スマートなローテーション、自動化とデータ抽出のための信頼性の高いパフォーマンス。<a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> で大規模ウェブクローリングを簡素化。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -168,16 +158,6 @@ MySpider().start()
|
||||
<a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">The Web Scraping Club で Scrapling の詳細レビュー</a>（2025年11月）をお読みください。Web スクレイピング専門の No.1 ニュースレターです。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">安定したプロキシ</a>。スクレイピング、自動化、マルチアカウント管理に対応。クリーンな IP、高速レスポンス、高負荷時でも信頼性の高いパフォーマンス。スケーラブルなワークフロー向けに設計。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -188,22 +168,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> は195カ国以上、8,000万以上のIPを備えたスケーラブルな住宅用プロキシを提供し、高速で信頼性の高い接続、自動ローテーション、強力なブロック回避性能を実現します。無料トライアルあり。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> はIPあたり $0.018 またはGBあたり $0.68 からの住宅用プロキシを提供します。90カ国以上で2,000万以上のIPを保有。固定セッションまたはローテーションセッションをデスクトップまたはモバイルアプリで管理できます。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - 市場最高品質のIPを提供する信頼性の高いプロキシプロバイダー。プロモコード SCRAPLING35 でプロキシが35%割引になります。
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>ここに広告を表示したいですか？[こちら](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)をクリック</sub></i>
|
||||
# スポンサー
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-24
@@ -137,16 +137,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a>는 TikTok, X, YouTube, Instagram 등 16개 이상 플랫폼에서 900개 이상의 안정적인 API를 제공하며, 4,000만 이상의 데이터셋을 보유하고 있습니다. <br /> <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">할인된 AI 모델</a>도 제공 - Claude, GPT, GEMINI 등 최대 71% 할인.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a>는 개발자와 스크레이퍼를 위한 빠른 레지덴셜 및 ISP 프록시를 제공합니다. 글로벌 IP 커버리지, 높은 익명성, 스마트 로테이션, 자동화와 데이터 추출을 위한 안정적인 성능. <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a>로 대규모 웹 크롤링을 간소화하세요.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -168,16 +158,6 @@ MySpider().start()
|
||||
<a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">The Web Scraping Club에서 Scrapling의 전체 리뷰</a>(2025년 11월)를 읽어보세요. 웹 스크래핑 전문 No.1 뉴스레터입니다.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">안정적인 프록시</a>. 스크래핑, 자동화, 멀티 계정 관리에 적합합니다. 깨끗한 IP, 빠른 응답, 높은 부하에서도 신뢰할 수 있는 성능. 확장 가능한 워크플로우를 위해 설계되었습니다.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -188,22 +168,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a>는 195개국 이상에서 8천만 개 이상의 IP를 갖춘 확장 가능한 주거용 프록시를 제공하며, 빠르고 안정적인 연결, 자동 회전, 강력한 차단 방지 성능을 제공합니다. 무료 체험판 이용 가능.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a>는 IP당 $0.018 또는 GB당 $0.68의 저렴한 가격부터 시작하는 주거용 프록시를 제공합니다. 90개국 이상에서 2천만 개 이상의 IP를 보유하고 있으며, 고정 또는 회전 세션을 데스크톱이나 모바일 앱에서 관리할 수 있습니다.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - 시장에서 가장 높은 품질의 IP를 제공하는 신뢰할 수 있는 프록시 제공업체입니다. 프로모 코드 SCRAPLING35를 사용하면 프록시 35% 할인을 받을 수 있습니다.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>여기에 광고를 게재하고 싶으신가요? [여기](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)를 클릭하세요</sub></i>
|
||||
# 스폰서
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-25
@@ -139,16 +139,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> oferece mais de 900 APIs estáveis em mais de 16 plataformas, incluindo TikTok, X, YouTube e Instagram, com mais de 40M de datasets. <br /> Também oferece <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">modelos de IA com desconto</a> - Claude, GPT, GEMINI e mais com até 71% de desconto.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> fornece proxies residenciais e ISP rápidos para desenvolvedores e scrapers. Cobertura global de IPs, alto anonimato, rotação inteligente e desempenho confiável para automação e extração de dados. Use o <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> para simplificar o crawling web em larga escala.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -170,16 +160,6 @@ MySpider().start()
|
||||
Leia uma análise completa do <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">Scrapling no The Web Scraping Club</a> (nov. 2025), a newsletter número 1 dedicada a Web Scraping.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">Proxies estáveis</a> para scraping, automação e multi-accounting. IPs limpos, resposta rápida e desempenho confiável sob carga. Feito para fluxos de trabalho escaláveis.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -190,23 +170,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> fornece proxies residenciais escaláveis com mais de 80M de IPs em mais de 195 países, entregando conexões rápidas e confiáveis, rotação automática e forte desempenho anti-bloqueio. Teste grátis disponível.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> oferece proxies residenciais a partir de apenas $0,018/IP ou $0,68/GB. Mais de 20M de IPs em mais de 90 países. Sessões fixas ou rotativas, gerenciadas pelo aplicativo desktop ou móvel.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - provedor de proxies confiável com a mais alta qualidade de IP do mercado. Use o código promocional SCRAPLING35 para obter 35% de desconto em proxies.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>Quer mostrar seu anúncio aqui? Clique [aqui](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# Patrocinadores
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
+20
-24
@@ -140,16 +140,6 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank">TikHub.io</a> предоставляет более 900 стабильных API на 16+ платформах, включая TikTok, X, YouTube и Instagram, с более чем 40 млн наборов данных. <br /> Также предлагает <a href="https://ai.tikhub.io/?ref=KarimShoair" target="_blank">AI-модели со скидкой</a> - Claude, GPT, GEMINI и другие со скидкой до 71%.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank">Nsocks</a> предоставляет быстрые резидентные и ISP прокси для разработчиков и скраперов. Глобальное покрытие IP, высокая анонимность, умная ротация и надёжная производительность для автоматизации и извлечения данных. Используйте <a href="https://www.xcrawl.com/?keyword=2p67aivg" target="_blank">Xcrawl</a> для упрощения масштабного веб-краулинга.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
@@ -171,16 +161,6 @@ MySpider().start()
|
||||
Прочитайте полный обзор <a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank">Scrapling на The Web Scraping Club</a> (ноябрь 2025) - рассылка №1, посвящённая веб-скрейпингу.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank">Стабильные прокси</a> для скрапинга, автоматизации и мультиаккаунтинга. Чистые IP, быстрый отклик и надёжная работа под нагрузкой. Созданы для масштабируемых рабочих процессов.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
@@ -191,22 +171,38 @@ MySpider().start()
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank">Swiftproxy</a> предоставляет масштабируемые резидентные прокси с более чем 80 млн IP в 195+ странах, обеспечивая быстрые и надёжные соединения, автоматическую ротацию и высокую устойчивость к блокировкам. Доступна бесплатная пробная версия.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank">9Proxy</a> предоставляет резидентные прокси всего от $0,018 за IP или $0,68 за ГБ. Более 20 млн IP в 90+ странах. Закреплённые или ротационные сессии, управление через настольное или мобильное приложение.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="200">
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
</td>
|
||||
<td>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank">NodeMaven</a> - надёжный провайдер прокси с самым высоким качеством IP на рынке. Используйте промокод SCRAPLING35 для получения скидки 35% на прокси.
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<i><sub>Хотите показать здесь свою рекламу? Нажмите [здесь](https://github.com/sponsors/D4Vinci/sponsorships?tier_id=586646)</sub></i>
|
||||
# Спонсоры
|
||||
|
||||
<!-- sponsors -->
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br/>
|
||||
|
||||
<a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
|
||||
<a href="https://visit.decodo.com/Dy6W0b" target="_blank" title="Try the Most Efficient Residential Proxies for Free"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/decodo.png"></a>
|
||||
<a href="https://hasdata.com/?utm_source=github&utm_medium=banner&utm_campaign=D4Vinci" target="_blank" title="The web scraping service that actually beats anti-bot systems!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/hasdata.png"></a>
|
||||
<a href="https://proxyempire.io/?ref=scrapling&utm_source=scrapling" target="_blank" title="Collect The Data Your Project Needs with the Best Residential Proxies"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/ProxyEmpire.png"></a>
|
||||
<a href="https://www.webshare.io/?referral_code=48r2m2cd5uz1" target="_blank" title="The Most Reliable Proxy with Unparalleled Performance"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/webshare.png"></a>
|
||||
<a href="https://www.ipfoxy.com/?r=scrapling" target="_blank" title="Unlock the Full Potential of Global Business with IPFoxy's High-Quality Rotating and Dedicated Proxy Services."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPFoxy.jpg"></a>
|
||||
<a href="https://www.ipcook.com/?ref=EAENO9&utm_source=github&utm_medium=referral&utm_campaign=d4vinci_scrapling" target="_blank" title="Fast Proxies. Smart Pricing. Premium Performance."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/IPCook.png"></a>
|
||||
<a href="https://proxiware.com/?ref=scrapling" target="_blank" title="Collect Any Data. At Any Scale."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/proxiware.png"></a>
|
||||
|
||||
|
||||
|
||||
@@ -129,7 +129,7 @@ If you are using the Docker image, then it would be something like
|
||||
"ScraplingServer": {
|
||||
"command": "docker",
|
||||
"args": [
|
||||
"run", "-i", "--rm", "scrapling", "mcp"
|
||||
"run", "-i", "--rm", "pyd4vinci/scrapling", "mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -50,7 +50,7 @@ The extract command is a set of simple terminal tools that:
|
||||
scrapling extract get "https://example.com" content.txt
|
||||
|
||||
# Or use the Docker image with something like this:
|
||||
docker run -v $(pwd)/output:/output scrapling extract get "https://blog.example.com" /output/article.md
|
||||
docker run -v $(pwd)/output:/output pyd4vinci/scrapling extract get "https://blog.example.com" /output/article.md
|
||||
```
|
||||
|
||||
- **Extract Specific Content**
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# Support and Advertisement
|
||||
|
||||
I've been creating all of these projects in my spare time and have invested considerable resources & effort in providing them to the community for free. By becoming a sponsor, you'd be directly funding my coffee reserves, helping me fulfill my responsibilities, and enabling me to continuously update existing projects and potentially create new ones.
|
||||
|
||||
You can sponsor me directly through the [GitHub Sponsors program](https://github.com/sponsors/D4Vinci) or [Buy Me a Coffee](https://buymeacoffee.com/d4vinci).
|
||||
|
||||
+6
-8
@@ -71,25 +71,23 @@ MySpider().start()
|
||||
<a href="https://tikhub.io/?utm_source=github.com/D4Vinci/Scrapling&utm_medium=marketing_social&utm_campaign=retargeting&utm_content=carousel_ad" target="_blank" title="Unlock the Power of Social Media Data & AI">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/TikHub.jpg" class="ad">
|
||||
</a>
|
||||
<a href="https://www.nsocks.com/?keyword=2p67aivg" target="_blank" title="Scalable Web Data Access for AI Applications">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/nsocks.png" class="ad">
|
||||
</a>
|
||||
<a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting.">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/petrosky.png" class="ad">
|
||||
</a>
|
||||
<a href="https://substack.thewebscraping.club/p/scrapling-hands-on-guide?utm_source=github&utm_medium=repo&utm_campaign=scrapling" target="_blank" title="The #1 newsletter dedicated to Web Scraping">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/TWSC.png" class="ad">
|
||||
</a>
|
||||
<a href="http://mangoproxy.com/?utm_source=D4Vinci&utm_medium=GitHub&utm_campaign=D4Vinci" target="_blank" title="Proxies You Can Rely On: Residential, Server, and Mobile">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/MangoProxy.png" class="ad">
|
||||
</a>
|
||||
<a href="https://www.swiftproxy.net/?ref=D4Vinci" target="_blank" title="Scalable Solutions for Web Data Access">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SwiftProxy.png" class="ad">
|
||||
</a>
|
||||
<a href="https://9proxy.com/pricing?tab=traffic&utm_source=Github&utm_campaign=D4vinci" target="_blank" title="Top-Tier Residential Proxy Solution for the Highest Success Rate">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/9proxy.jpg" class="ad">
|
||||
</a>
|
||||
<a href="https://go.nodemaven.com/scrapling" target="_blank" title="Proxies with the Highest IP Scores">
|
||||
<img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NodeMaven.png">
|
||||
</a>
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://www.crawleo.dev/?utm_source=github&utm_medium=sponsor&utm_campaign=scrapling" target="_blank" title="Supercharge your AI with Real-Time Web Intelligence"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/crawleo.png"></a>
|
||||
<br />
|
||||
</div>
|
||||
<!-- /sponsors -->
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 74 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 20 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 5.6 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 15 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 27 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 22 KiB |
+4
-4
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
|
||||
[project]
|
||||
name = "scrapling"
|
||||
# Static version instead of a dynamic version so we can get better layer caching while building docker, check the docker file to understand
|
||||
version = "0.4.8"
|
||||
version = "0.4.9"
|
||||
description = "Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be!"
|
||||
readme = {file = "README.md", content-type = "text/markdown"}
|
||||
license = {file = "LICENSE"}
|
||||
@@ -61,7 +61,7 @@ classifiers = [
|
||||
"Typing :: Typed",
|
||||
]
|
||||
dependencies = [
|
||||
"lxml>=6.1.0",
|
||||
"lxml>=6.1.1",
|
||||
"cssselect>=1.4.0",
|
||||
"orjson>=3.11.8",
|
||||
"tld>=0.13.2",
|
||||
@@ -73,8 +73,8 @@ dependencies = [
|
||||
fetchers = [
|
||||
"click>=8.3.0",
|
||||
"curl_cffi>=0.15.0",
|
||||
"playwright==1.59.0",
|
||||
"patchright==1.59.1",
|
||||
"playwright==1.60.0",
|
||||
"patchright==1.60.1",
|
||||
"browserforge>=1.2.4",
|
||||
"apify-fingerprint-datapoints>=0.13.0",
|
||||
"msgspec>=0.21.1",
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
__author__ = "Karim Shoair (karim.shoair@pm.me)"
|
||||
__version__ = "0.4.8"
|
||||
__version__ = "0.4.9"
|
||||
__copyright__ = "Copyright (c) 2024 Karim Shoair"
|
||||
|
||||
from typing import Any, TYPE_CHECKING
|
||||
|
||||
+3
-1
@@ -2,6 +2,7 @@ from pathlib import Path
|
||||
from subprocess import check_output
|
||||
from sys import executable as python_executable
|
||||
|
||||
from scrapling import __version__
|
||||
from scrapling.core.utils import log
|
||||
from scrapling.engines.toolbelt.custom import Response
|
||||
from scrapling.core.utils._shell import _CookieParser, _ParseHeaders
|
||||
@@ -10,7 +11,7 @@ from scrapling.core._types import List, Optional, Dict, Tuple, Any, Callable
|
||||
from orjson import loads as json_loads, JSONDecodeError
|
||||
|
||||
try:
|
||||
from click import command, option, Choice, group, argument
|
||||
from click import command, option, Choice, group, argument, version_option
|
||||
except (ImportError, ModuleNotFoundError) as e:
|
||||
raise ModuleNotFoundError(
|
||||
"You need to install scrapling with any of the extras to enable Shell commands. See: https://scrapling.readthedocs.io/en/latest/#installation"
|
||||
@@ -650,6 +651,7 @@ def stealthy_fetch(
|
||||
|
||||
|
||||
@group()
|
||||
@version_option(version=__version__, prog_name="Scrapling")
|
||||
def main():
|
||||
pass
|
||||
|
||||
|
||||
@@ -13,8 +13,8 @@ from scrapling.core._types import Dict, Literal, Tuple
|
||||
__OS_NAME__ = platform_system()
|
||||
OSName = Literal["linux", "macos", "windows"]
|
||||
# Current versions hardcoded for now (Playwright doesn't allow to know the version of a browser without launching it)
|
||||
chromium_version = 147
|
||||
chrome_version = 147
|
||||
chromium_version = 148
|
||||
chrome_version = 148
|
||||
|
||||
|
||||
@lru_cache(1, typed=True)
|
||||
|
||||
+4
-2
@@ -671,7 +671,7 @@ class Selector(SelectorsGeneration):
|
||||
element_data = self.retrieve(identifier or selector)
|
||||
if element_data:
|
||||
elements = self.relocate(element_data, percentage)
|
||||
if elements is not None and auto_save:
|
||||
if elements and auto_save:
|
||||
self.save(elements[0], identifier or selector)
|
||||
|
||||
return self.__handle_elements(elements)
|
||||
@@ -991,7 +991,9 @@ class Selector(SelectorsGeneration):
|
||||
SequenceMatcher(None, v, candidate_attributes.get(k, "")).ratio()
|
||||
for k, v in original_attributes.items()
|
||||
)
|
||||
checks += len(candidate_attributes)
|
||||
# Using `max` so candidates with extra attributes are penalized and candidates
|
||||
# with fewer attributes don't get inflated scores from a smaller denominator
|
||||
checks += max(len(original_attributes), len(candidate_attributes))
|
||||
else:
|
||||
if not candidate_attributes:
|
||||
# Both don't have attributes, this must mean something
|
||||
|
||||
@@ -64,7 +64,7 @@ class ResponseCacheManager:
|
||||
async with await anyio.open_file(temp_path, "wb") as f:
|
||||
await f.write(serialized)
|
||||
|
||||
await temp_path.rename(self._cache_path(fingerprint))
|
||||
await temp_path.replace(self._cache_path(fingerprint))
|
||||
except Exception as e:
|
||||
if await temp_path.exists():
|
||||
await temp_path.unlink()
|
||||
|
||||
@@ -50,7 +50,7 @@ class CheckpointManager:
|
||||
async with await anyio.open_file(temp_path, "wb") as f:
|
||||
await f.write(serialized)
|
||||
|
||||
await temp_path.rename(self._checkpoint_path)
|
||||
await temp_path.replace(self._checkpoint_path)
|
||||
|
||||
log.info(f"Checkpoint saved: {len(data.requests)} requests, {len(data.seen)} seen URLs")
|
||||
except Exception as e:
|
||||
|
||||
+2
-2
@@ -14,12 +14,12 @@
|
||||
"mimeType": "image/png"
|
||||
}
|
||||
],
|
||||
"version": "0.4.8",
|
||||
"version": "0.4.9",
|
||||
"packages": [
|
||||
{
|
||||
"registryType": "pypi",
|
||||
"identifier": "scrapling",
|
||||
"version": "0.4.8",
|
||||
"version": "0.4.9",
|
||||
"runtimeHint": "uvx",
|
||||
"packageArguments": [
|
||||
{
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[metadata]
|
||||
name = scrapling
|
||||
version = 0.4.8
|
||||
version = 0.4.9
|
||||
author = Karim Shoair
|
||||
author_email = karim.shoair@pm.me
|
||||
description = Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be!
|
||||
|
||||
@@ -4,8 +4,9 @@ from unittest.mock import patch, MagicMock
|
||||
import pytest_httpbin
|
||||
|
||||
from scrapling.parser import Selector
|
||||
from scrapling import __version__
|
||||
from scrapling.cli import (
|
||||
shell, mcp, get, post, put, delete, fetch, stealthy_fetch
|
||||
main, shell, mcp, get, post, put, delete, fetch, stealthy_fetch
|
||||
)
|
||||
|
||||
|
||||
@@ -32,6 +33,12 @@ class TestCLI:
|
||||
def runner(self):
|
||||
return CliRunner()
|
||||
|
||||
def test_version_flag(self, runner):
|
||||
"""Test that the --version flag prints the Scrapling version and exits"""
|
||||
result = runner.invoke(main, ['--version'])
|
||||
assert result.exit_code == 0
|
||||
assert result.output.strip() == f'Scrapling, version {__version__}'
|
||||
|
||||
def test_shell_command(self, runner):
|
||||
"""Test shell command"""
|
||||
with patch('scrapling.core.shell.CustomShell') as mock_shell:
|
||||
|
||||
@@ -56,6 +56,32 @@ class TestParserAdaptive:
|
||||
assert relocated[0].has_class("new-class")
|
||||
assert relocated[0].css(".new-description")[0].text == "Description 1"
|
||||
|
||||
def test_relocation_auto_save_no_match_above_threshold(self):
|
||||
"""Adaptive relocation with `auto_save=True` must not crash when no element
|
||||
clears the `percentage` threshold (relocate() returns an empty list)."""
|
||||
original_html = """
|
||||
<div class="container">
|
||||
<article class="product" id="target">
|
||||
<h3>Widget</h3>
|
||||
<p class="desc">A widget</p>
|
||||
</article>
|
||||
</div>
|
||||
"""
|
||||
# Unrelated structure so nothing can match a high threshold
|
||||
changed_html = "<html><body><span>totally unrelated content</span></body></html>"
|
||||
|
||||
old_page = Selector(original_html, url="example.com", adaptive=True)
|
||||
new_page = Selector(changed_html, url="example.com", adaptive=True)
|
||||
|
||||
old_page.css("#target", identifier="target", auto_save=True)
|
||||
|
||||
# Before the fix this raised `IndexError: list index out of range` because the
|
||||
# guard checked `elements is not None` but relocate() returns [] (never None).
|
||||
result = new_page.css(
|
||||
"#target", identifier="target", adaptive=True, auto_save=True, percentage=95
|
||||
)
|
||||
assert list(result) == []
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_element_relocation_async(self):
|
||||
"""Test relocating element after structure change in async mode"""
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
Tests for Selector.find_similar() with non-default parameters.
|
||||
Target file: tests/parser/test_general.py (append to TestSimilarElements class)
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from scrapling import Selector
|
||||
|
||||
@@ -61,14 +62,10 @@ class TestFindSimilarAdvanced:
|
||||
first = product_page.css("div.product")[0]
|
||||
# Ignore both data-price and data-category → only class matters → all 3 divs match
|
||||
ignore_all_data = first.find_similar(
|
||||
similarity_threshold=0.2,
|
||||
ignore_attributes=["data-price", "data-category"]
|
||||
similarity_threshold=0.2, ignore_attributes=["data-price", "data-category"]
|
||||
)
|
||||
# Ignore nothing → data-category difference (fruit vs veggie) may reduce matches
|
||||
ignore_nothing = first.find_similar(
|
||||
similarity_threshold=0.9,
|
||||
ignore_attributes=[]
|
||||
)
|
||||
ignore_nothing = first.find_similar(similarity_threshold=0.9, ignore_attributes=[])
|
||||
assert len(ignore_all_data) >= len(ignore_nothing)
|
||||
|
||||
def test_find_similar_on_text_node_returns_empty(self, product_page):
|
||||
@@ -76,3 +73,31 @@ class TestFindSimilarAdvanced:
|
||||
text_node = product_page.css(".name::text")[0]
|
||||
result = text_node.find_similar()
|
||||
assert len(result) == 0
|
||||
|
||||
def test_find_similar_attribute_count_mismatch_scoring(self):
|
||||
"""The similarity denominator uses max() of both attribute counts, so candidates
|
||||
with fewer attributes don't get inflated scores and candidates with extra
|
||||
attributes stay penalized."""
|
||||
html = """
|
||||
<html><body>
|
||||
<div class="cards">
|
||||
<div class="card" data-kind="primary" data-color="red" data-size="large">Alpha</div>
|
||||
<div class="card">Beta</div>
|
||||
<div class="card" data-kind="primary" data-color="red" data-size="large" data-id="x">Gamma</div>
|
||||
<div class="card" data-kind="primary" data-color="red" data-size="large">Delta</div>
|
||||
</div>
|
||||
</body></html>
|
||||
"""
|
||||
page = Selector(html, adaptive=False)
|
||||
first = page.css("div.card")[0] # Alpha
|
||||
|
||||
similar = first.find_similar(similarity_threshold=0.9, ignore_attributes=[])
|
||||
texts = {el.text for el in similar}
|
||||
|
||||
# An exact attribute match must pass
|
||||
assert "Delta" in texts
|
||||
# Beta matches 1 of Alpha's 4 attributes; the old denominator counted candidate
|
||||
# attributes only, inflating it to a perfect score (1.0 / 1)
|
||||
assert "Beta" not in texts
|
||||
# Gamma's extra attribute dilutes the score (4.0 / 5) - the intentional penalty
|
||||
assert "Gamma" not in texts
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
pytest>=2.8.0,<9
|
||||
pytest-cov
|
||||
playwright==1.59.0
|
||||
playwright==1.60.0
|
||||
werkzeug<3.0.0
|
||||
pytest-httpbin==2.1.0
|
||||
pytest-asyncio
|
||||
|
||||
@@ -49,6 +49,27 @@ class TestResponseCacheManager:
|
||||
assert dict(restored.headers) == dict(original.headers)
|
||||
assert dict(restored.request_headers) == dict(original.request_headers)
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_put_overwrites_existing_entry(self):
|
||||
"""Re-caching the same fingerprint must replace the stored response.
|
||||
|
||||
Regression test for a Windows-only failure: ``Path.rename`` cannot
|
||||
overwrite an existing destination on Windows (raising ``WinError 183``),
|
||||
so the second ``put`` was caught by the error handler, the temp file was
|
||||
removed, and ``get`` kept returning the stale body. ``Path.replace``
|
||||
overwrites atomically on every platform.
|
||||
"""
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
cache = ResponseCacheManager(tmpdir)
|
||||
fp = b"\x05" * 20
|
||||
|
||||
await cache.put(fp, _make_response(body=b"<html>first</html>"), "GET")
|
||||
await cache.put(fp, _make_response(body=b"<html>second</html>"), "GET")
|
||||
|
||||
restored = await cache.get(fp)
|
||||
assert restored is not None
|
||||
assert restored.body == b"<html>second</html>"
|
||||
|
||||
@pytest.mark.anyio
|
||||
async def test_get_cache_miss(self):
|
||||
with tempfile.TemporaryDirectory() as tmpdir:
|
||||
|
||||
Reference in New Issue
Block a user