freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(browsers): add a pre-navigation hook to allow page setup

Browse Source 
Solves #238
This commit is contained in:
Karim shoair
2026-04-13 03:57:27 +02:00
parent d19e861e45
commit 5e13d3ece6
7 changed files with 56 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scrapling/fetchers/chrome.py
+4 -2
View File
@@ -23,7 +23,8 @@ class DynamicFetcher(BaseFetcher):
:param load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the Response object.
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
:param page_action: Added for automation. A function that takes the `page` object, runs after navigation, and does the automation you need.
:param page_setup: A function that takes the `page` object, runs before navigation. Use it to register event listeners or routes that must be set up before the page loads.
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
:param init_script: An absolute path to a JavaScript file to be executed on page creation with this request.
:param locale: Set the locale for the browser if wanted. Defaults to the system default locale.
@@ -65,7 +66,8 @@ class DynamicFetcher(BaseFetcher):
:param load_dom: Enabled by default, wait for all JavaScript on page(s) to fully load and execute.
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the Response object.
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
:param page_action: Added for automation. A function that takes the `page` object, runs after navigation, and does the automation you need.
:param page_setup: A function that takes the `page` object, runs before navigation. Use it to register event listeners or routes that must be set up before the page loads.
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
:param init_script: An absolute path to a JavaScript file to be executed on page creation with this request.
:param locale: Set the locale for the browser if wanted. Defaults to the system default locale.
scrapling/fetchers/stealth_chrome.py
+4 -2
View File
@@ -27,7 +27,8 @@ class StealthyFetcher(BaseFetcher):
:param network_idle: Wait for the page until there are no network connections for at least 500 ms.
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
:param page_action: Added for automation. A function that takes the `page` object, runs after navigation, and does the automation you need.
:param page_setup: A function that takes the `page` object, runs before navigation. Use it to register event listeners or routes that must be set up before the page loads.
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
:param init_script: An absolute path to a JavaScript file to be executed on page creation for all pages in this session.
:param locale: Specify user locale, for example, `en-GB`, `de-DE`, etc. Locale will affect navigator.language value, Accept-Language request header value as well as number and date formatting
@@ -78,7 +79,8 @@ class StealthyFetcher(BaseFetcher):
:param network_idle: Wait for the page until there are no network connections for at least 500 ms.
:param timeout: The timeout in milliseconds that is used in all operations and waits through the page. The default is 30,000
:param wait: The time (milliseconds) the fetcher will wait after everything finishes before closing the page and returning the ` Response ` object.
:param page_action: Added for automation. A function that takes the `page` object and does the automation you need.
:param page_action: Added for automation. A function that takes the `page` object, runs after navigation, and does the automation you need.
:param page_setup: A function that takes the `page` object, runs before navigation. Use it to register event listeners or routes that must be set up before the page loads.
:param wait_selector: Wait for a specific CSS selector to be in a specific state.
:param init_script: An absolute path to a JavaScript file to be executed on page creation for all pages in this session.
:param locale: Specify user locale, for example, `en-GB`, `de-DE`, etc. Locale will affect navigator.language value, Accept-Language request header value as well as number and date formatting