Commit Graph

370 Commits

Author SHA1 Message Date
Karim shoair 2d704b2a8b fix(shell): Fixing a bug with content converting 2025-09-19 04:50:15 +03:00
Karim shoair a9d05cc0ef style: Removing dead code/docstrings and correcting type hints 2025-09-19 04:49:30 +03:00
Karim shoair bf5fe6d451 fix(shell): Fix view command edge case 2025-09-19 04:21:15 +03:00
Karim shoair 504ecdf7dc fix(shell): Fix parsing curl --data-raw 2025-09-19 04:20:16 +03:00
Karim shoair b197f8a5f9 fix(parser): Improve selectors re function 2025-09-18 13:56:37 +03:00
Abdullah 7ba333c3c9 perf(session): improve page management and speed up execution
Close pages immediately after tasks finish to reduce resource usage and
improve overall performance. Update corresponding test files to reflect
the new page lifecycle management.
2025-09-17 18:49:45 +03:00
Karim shoair 570303f938 fix(parser): Make html_content and prettify return strings not bytes (depends on the encoding) 2025-09-16 01:58:51 +03:00
Karim shoair 5683965367 fix(sessions): Fixing a bug with max pages counter 2025-09-16 01:47:31 +03:00
Karim shoair b7327466cf build: pump version up 2025-09-15 23:35:08 +03:00
Karim shoair 05c2d9aa00 feat(CLI): Make the session classes available in the shell 2025-09-15 23:34:36 +03:00
Karim shoair 5bcd4c2927 fix: A workaround for Playwright issue with page content retrieving on windows 2025-09-15 21:34:20 +03:00
Karim shoair 1547366c85 fix: fixing body issue with cli view command 2025-09-15 21:32:57 +03:00
Karim shoair a6aabda67c build: Pump version up 2025-09-15 13:52:09 +03:00
Karim shoair 7a7097b775 fix: Stop removing browser default page
Looks like it's problematic with some systems so it doesn't worth the trouble
2025-09-15 13:43:47 +03:00
Karim shoair f848e6b7f2 fix: Fixes to handle the new extra deps group 2025-09-15 00:19:19 +03:00
Karim shoair fdac239fcc build: Move click library to be part of the fetchers extra 2025-09-14 20:32:46 +03:00
Karim shoair ce4fc31f2c feat: Make .body return the passed content as it is without any processing
This makes it possible to download files and deal with non-HTML requests (ex: #81 )
2025-09-14 20:21:45 +03:00
Karim shoair 67ca139ff9 fix: Fixes for multiple encoding issues (#80 & #81 ) 2025-09-14 20:20:29 +03:00
Karim shoair 0b3509a295 fix(shell): Fix a small typo 2025-09-14 05:05:55 +03:00
Karim shoair 13d7e70cb7 refactor: Make all fetchers as an optional dependency group
+ Removing some dead code
2025-09-13 16:07:48 +03:00
Karim shoair 8c4d6158d6 feat(browser fetchers): A new option to control page JS 2025-09-13 03:59:12 +03:00
Karim shoair 330d03559c style: applying the new ruff rules to all files 2025-09-13 03:22:53 +03:00
Karim shoair 60be9dc816 feat: Validate all fetch-level parameters
So it's validated like the session-level ones. Needs improvement, but that's for later
2025-09-13 03:22:08 +03:00
Karim shoair 0c7db3c9d9 fix: validation keywords 2025-09-13 02:07:04 +03:00
Karim shoair adb82008e5 style: Using keywords for validation
Less lines of code and more stylish
2025-09-12 04:33:00 +03:00
Karim shoair e4b1e00e63 build: Pump up the version 2025-09-11 04:14:32 +03:00
Karim shoair 05a714980d docs: Update all docstring according to the new changes 2025-09-11 04:13:36 +03:00
Karim shoair 9838d741d0 refactor/feat(browser fetchers): make it possible to have a configuration per page in sessions
- Also, no need for the `page_action` argument function to return the page again
2025-09-11 03:48:55 +03:00
Karim shoair ecaec65049 refactor/fix(browser fetchers): Solving a logic bug in page rotation
- Pages that get reused in rotation are possibly contaminated from previous settings used on them, and some are stubborn to remove, so the new approach replaces finished pages with new ones.
- Removed the `max_pages` from sync `StealthySession` to match `DynamicSession` (Doesn't mean anything in sync code)
2025-09-08 16:08:57 +03:00
Karim shoair f298b76b8e perf(StealthySession): Close persistent context default page 2025-09-07 18:48:51 +03:00
Karim shoair 710f994339 perf(DynamicSession): Close persistent context default page 2025-09-07 18:48:40 +03:00
Karim shoair 4341477a66 perf(StealthyFetcher): Slight optimization 2025-09-07 04:58:55 +03:00
Karim shoair 72e26c23e8 fix(DynamicFetcher): Improve stealth mode 2025-09-07 03:13:51 +03:00
Karim shoair 934d8a1452 build: Pump up the version 2025-09-02 16:33:52 +03:00
Karim shoair 6ad1bab2e4 feat(DynamicFetcher): add the ability to add init_script to context (#56) 2025-09-02 16:25:13 +03:00
Karim shoair 9543f171aa feat(StealthyFetcher): add the ability to add init_script to context 2025-09-02 16:19:22 +03:00
Karim shoair af9a62a539 fix: fix for shell imported by mistake issue (#76) 2025-09-02 15:23:15 +03:00
Karim shoair 812cf2d21b build: Pump up the version 2025-09-01 06:43:41 +03:00
Karim shoair 967c0956a1 feat(DynamicSession): Improve stealth mode 2025-08-31 02:57:12 +03:00
Karim shoair 6eb580a6d0 feat(DynamicSession): Improve normal fetches speed 2025-08-31 02:56:34 +03:00
Karim shoair b5be2547d7 fix(dynamicSession): issue with headless 2025-08-31 01:29:47 +03:00
Karim shoair 76721f906e docs(cli): Correcting docstrings 2025-08-30 22:12:33 +03:00
Karim shoair c66c48e3ae fix(extract): Correcting docs for the css-selector option 2025-08-29 21:34:39 +03:00
Karim shoair cf57af588e docs: update doc strings with correct naming 2025-08-25 06:08:15 +03:00
Karim shoair 2f402f4835 style: add flags for tests coverage
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
Karim shoair 009bbb20c0 refactor(dynamic Fetcher): small optimization 2025-08-16 23:20:54 +03:00
Karim shoair 6c9a960a44 fix(dynamicFetcher): Fix a bug with real_chrome argument 2025-08-16 16:01:32 +03:00
Karim shoair 84337ef1ac fix(parser): count all nested children of ignored tags in get_all_text 2025-08-16 14:24:51 +03:00
Karim shoair 13700e2692 fix: improve error handling 2025-08-15 04:52:11 +03:00
Karim shoair 825896c6a8 fix: improve checking for valid proxy and valid CDP URL 2025-08-03 16:31:16 +03:00