b0c2b1818b
Return to the default text behaviour
...
The logic to remove comment tags before returning text was causing errors sometimes so from now on leave `keep_comments` set to True better as it is.
2024-11-14 22:21:45 +02:00
eaa7da27c6
Better logic to handle json responses
2024-11-14 00:03:26 +02:00
d4b896b4e9
Parser - New logic to handle JSON responses passed from Fetchers
2024-11-13 23:14:27 +02:00
ab7b0d45ad
StaticEngine - Fix the headers bug
2024-11-13 23:13:39 +02:00
b2993d2fcf
Update and rename py.typeda to py.typed
2024-11-13 21:56:02 +13:00
067809974e
Rename py.typed to py.typeda
2024-11-13 21:55:51 +13:00
bd38fdb347
Corrections and rephrasing
2024-11-11 16:40:46 +02:00
b3418f7f50
Corrections and rephrasing
2024-11-11 16:40:35 +02:00
d6b454dc43
Corrections and rephrasing
2024-11-11 16:40:30 +02:00
eb100e45f7
Corrections and rephrasing
2024-11-11 16:40:21 +02:00
3d4b427168
Rework Selectors generation
...
- Renamed both `css_selector` and `xpath_selector` both to `generate_css_selector` and `generate_xpath_selector` for clarity and not interrupting the auto-completion while coding.
- Added `generate_full_css_selector` and `generate_full_xpath_selector` methods
2024-11-10 18:19:48 +02:00
239e2c73bf
Correctly get page content after JS Execution
...
- Thanks for @AbdullahY36 for the heads-up.
**Update reference** https://developer.mozilla.org/en-US/docs/Web/API/Document/DOMContentLoaded_event
2024-11-10 16:11:46 +02:00
43045e3eb2
Get page content after JS Execution
...
Thanks for @AbdullahY36
2024-11-10 15:54:44 +02:00
4b2814404a
Fixing the functions-based filtering logic
2024-11-09 22:32:05 +02:00
6b62579a7e
Making find_all/find methods on steroids
...
Adding the ability to find elements by regex patterns and functions.
2024-11-09 21:25:17 +02:00
b90c8adb02
Adding the argument automatch_domain to all fetchers
2024-11-09 15:38:27 +02:00
03091beb8b
Small clarification
2024-11-09 13:40:50 +02:00
fd7d787ab0
Add extra_headers argument to PW and isolate google referer feature from stealth mode as google_search
2024-11-09 00:54:11 +02:00
ee8f78feb1
Update camo.py
2024-11-09 00:48:11 +02:00
fd34cd9afa
Adding arguments google_search and extra_headers to StealthyFetcher
2024-11-09 00:43:24 +02:00
530b54d964
Clearer doc-string
2024-11-09 00:17:41 +02:00
1b015bbd07
Type correction
2024-11-08 23:59:12 +02:00
b919536527
Updating the keywords whitelisting logic in find/find_all functions
2024-11-07 12:56:32 +02:00
91f6b46e6d
Adding new filter and search functions to parser
2024-11-07 00:31:14 +02:00
b25d2ee79b
Adding new find_all and find functions to parser
2024-11-07 00:01:14 +02:00
6b40af426f
Better logic for css_first and xpath_first
2024-11-06 22:14:33 +02:00
5e848ef046
Adding new class type TextHandlers
2024-11-06 21:03:19 +02:00
7a84511b7a
Making Adaptor and Adaptor re/re_first arguments consistent with the TextHandler ones
2024-11-06 20:47:21 +02:00
7af75f78a5
Better logic for Adaptors re_first method
2024-11-06 20:24:30 +02:00
a1fd5ddc7e
Correcting headless argument type
2024-11-06 20:18:44 +02:00
a0a3e62075
Adding addons and humanize arguments to StealthyFetcher
2024-11-06 20:14:42 +02:00
5ad2126030
Small speed boost for fetching by some caching
2024-11-06 19:56:39 +02:00
5548941cf1
Correcting return type for css_first and xpath_first
2024-11-06 12:28:43 +02:00
10bcdfb596
Moving bypasses folder deeper
2024-11-06 12:19:26 +02:00
6cf506cb22
PlaywrightEngine - Validate CDP URLs in all cases
2024-11-06 11:24:56 +02:00
6497b1b559
Type correction
2024-11-06 02:20:33 +02:00
3622040440
Adding css_first and xpath_first for easier usage
2024-11-05 23:41:51 +02:00
009c2914aa
Fixing re_first logic in Adaptors type
2024-11-05 22:59:47 +02:00
a60662af4a
Update camo.py
2024-11-05 11:38:35 +02:00
bd612bb0e2
Updating the disable_resources doc-string
2024-11-04 20:16:04 +02:00
544d23b696
Toolbelt fingerprint functions - Doc strings and type annotations stuff
2024-11-03 23:31:09 +02:00
e270b00811
Toolbelt subpackage custom functions general adjustments
2024-11-03 23:17:20 +02:00
2ff0fbb0e5
Engines utils - Navigation functions
...
- Adding doc strings and type annotations stuff.
- Making error messages more detailed.
- Renaming the `construct_websocket_url` function for more clarity.
2024-11-03 22:46:40 +02:00
b8afb390ae
Renaming fetcher file to fetchers for clarity
2024-11-03 22:43:16 +02:00
282b130864
Static Engine - Doc strings and type annotations stuff
2024-11-03 22:25:34 +02:00
3af174526f
PlayWright Engine - Doc strings and type annotations stuff
2024-11-03 22:24:44 +02:00
54cacb8a4e
Camoufox Engine - Doc strings and type annotations stuff
2024-11-03 22:20:29 +02:00
61fd592a6a
Adding documentation to PlaywrightFetcher class
2024-11-03 21:04:11 +02:00
c1575b8e2e
Updating doc-strings for more clarity
2024-11-03 16:49:33 +02:00
1d2c68125a
Update fetcher.py
2024-11-03 16:40:42 +02:00
Karim shoair