From a35cc4bdd6bb20991362d8a63824762ece06c02b Mon Sep 17 00:00:00 2001 From: Karim shoair Date: Sat, 27 Dec 2025 20:51:28 +0200 Subject: [PATCH] docs: updating overview and benchmarks pages with the new changes --- docs/benchmarks.md | 2 +- docs/overview.md | 34 ++++++++++++++++++++++------------ 2 files changed, 23 insertions(+), 13 deletions(-) diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 37fc6b1..f404010 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -1,6 +1,6 @@ # Performance Benchmarks -Scrapling isn't just powerful—it's also blazing fast, and the updates since version 0.3 deliver exceptional performance improvements across all operations! +Scrapling isn't just powerful—it's also blazing fast, and the updates since version 0.3 have delivered exceptional performance improvements across all operations. The following benchmarks compare Scrapling's parser with other popular libraries. ## Benchmark Results diff --git a/docs/overview.md b/docs/overview.md index 561e10b..ad8dac3 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -1,4 +1,4 @@ -We will start by quickly reviewing the parsing capabilities. Then, we will fetch websites with custom browsers, make requests, and parse the response. +We will start by quickly reviewing the parsing capabilities. Then we will fetch websites using custom browsers, make requests, and parse the responses. Here's an HTML document generated by ChatGPT that we will be using as an example throughout this page: ```html @@ -82,7 +82,7 @@ page.get_all_text(ignore_tags=('script', 'style')) ``` ## Finding elements -If there's an element you want to find on the page, you will! Your creativity level is the only limitation! +If there's an element you want to find on the page, you will find it! Your creativity level is the only limitation! Finding the first HTML `section` element ```python @@ -94,14 +94,14 @@ Find all `section` elements section_elements = page.find_all('section') # [,

Customer Revie...' parent='
] ``` -Find all `section` elements whose `id` attribute value is `products` +Find all `section` elements whose `id` attribute value is `products`. ```python section_elements = page.find_all('section', {'id':"products"}) # Same as section_elements = page.find_all('section', id="products") # [] ``` -Find all `section` elements whose `id` attribute value contains `product` +Find all `section` elements whose `id` attribute value contains `product`. ```python section_elements = page.find_all('section', {'id*':"product"}) ``` @@ -218,7 +218,7 @@ Using the elements we found above
] >>> section_element.siblings [

Customer Revie...' parent='
] ->>> section_element.next # gets the next element, the same logic applies to `quote.previous` +>>> section_element.next # gets the next element, the same logic applies to `quote.previous`.

Customer Revie...' parent='
>>> section_element.children.css('h2::text') ['Products'] @@ -237,7 +237,7 @@ You can search for a specific ancestor of an element that satisfies a function; ``` ## Fetching websites -Instead of passing the raw HTML to Scrapling, you can get a website's response directly through HTTP requests or by fetching it from browsers. +Instead of passing the raw HTML to Scrapling, you can retrieve a website's response directly via HTTP requests or by fetching it in a browser. A fetcher is made for every use case. @@ -267,7 +267,7 @@ For Async requests, you will replace the import like below: > Notes: > > 1. You have the `stealthy_headers` argument, which, when enabled, makes requests to generate real browser headers and use them, including a referer header, as if this request came from a Google search of this domain. It's enabled by default. -> 2. The `impersonate` argument allows you to fake the TLS fingerprint for a specific version of a browser. +> 2. The `impersonate` argument lets you fake the TLS fingerprint for a specific browser version. > 3. There's also the `http3` argument, which, when enabled, makes the fetcher use HTTP/3 for requests, which makes your requests more authentic This is just the tip of the iceberg with this fetcher; check out the rest from [here](fetching/static.md) @@ -275,7 +275,7 @@ This is just the tip of the iceberg with this fetcher; check out the rest from [ ### Dynamic loading We have you covered if you deal with dynamic websites like most today! -The `DynamicFetcher` class (previously known as `PlayWrightFetcher`) provides many options to fetch/load websites' pages through browsers. +The `DynamicFetcher` class (formerly `PlayWrightFetcher`) offers many options for fetching and loading web pages using Chromium-based browsers. ```python >>> from scrapling.fetchers import DynamicFetcher >>> page = DynamicFetcher.fetch('https://www.google.com/search?q=%22Scrapling%22', disable_resources=True) # Vanilla Playwright option @@ -286,10 +286,9 @@ The `DynamicFetcher` class (previously known as `PlayWrightFetcher`) provides ma >>> page.css_first("#search a::attr(href)") 'https://github.com/D4Vinci/Scrapling' ``` -It's built on top of [Playwright](https://playwright.dev/python/) and it's currently providing three main run options that can be mixed as you want: +It's built on top of [Playwright](https://playwright.dev/python/), and it's currently providing two main run options that can be mixed as you want: - Vanilla Playwright without any modifications other than the ones you chose. It uses the Chromium browser. -- Stealthy Playwright with custom stealth mode explicitly written for it. It's not top-tier stealth mode, but it bypasses many online tests like [Sannysoft's](https://bot.sannysoft.com/). Check out the `StealthyFetcher` class below for more advanced stealth mode. It uses the Chromium browser. - Real browsers like your Chrome browser by passing the `real_chrome` argument or the CDP URL of your browser to be controlled by the Fetcher, and most of the options can be enabled on it. @@ -298,7 +297,18 @@ Again, this is just the tip of the iceberg with this fetcher. Check out the rest ### Dynamic anti-protection loading We also have you covered if you deal with dynamic websites with annoying anti-protections! -The `StealthyFetcher` class uses a custom version of a modified Firefox browser called [Camoufox](https://github.com/daijro/camoufox), bypassing most bot detections by default. Scrapling offers a faster custom version, includes extra tools, and features easy configurations to further increase undetectability. +The `StealthyFetcher` class uses a stealthy version of the `DynamicFetcher` explained above. + +Some of the things it does: + +1. It easily bypasses all types of Cloudflare's Turnstile/Interstitial automatically. +2. It bypasses CDP runtime leaks and WebRTC leaks. +3. It isolates JS execution, removes many Playwright fingerprints, and stops detection through some of the known behaviors that bots do. +4. It generates canvas noise to prevent fingerprinting through canvas. +5. It automatically patches known methods to detect running in headless mode and provides an option to defeat timezone mismatch attacks. +6. It makes requests look as if they came from Google's search page of the requested website. +7. and other anti-protection options... + ```python >>> from scrapling.fetchers import StealthyFetcher >>> page = StealthyFetcher.fetch('https://www.browserscan.net/bot-detection') # Running headless by default @@ -318,4 +328,4 @@ Again, this is just the tip of the iceberg with this fetcher. Check out the rest --- -That's Scrapling at a glance. If you want to learn more about it, continue to the next section. \ No newline at end of file +That's Scrapling at a glance. If you want to learn more, continue to the next section. \ No newline at end of file