Commit Graph

26 Commits

Author SHA1 Message Date
Karim shoair debe03256b refactor: Making all the codebase acceptable by PyRight
Also fixes #97
2025-10-05 04:03:39 +03:00
Karim shoair e149c715dd feat(CF solver): better implementation to help with all custom implementations of turnstile 2025-10-02 20:05:39 +03:00
Karim shoair 8ad7cd6343 docs: Update all pages/docstrings to reflect recent changes 2025-09-29 03:57:40 +03:00
Karim shoair 716a5c3572 feat(cf solver): Make solver able to handle Turnstile and interstitial 2025-09-28 02:53:51 +03:00
Karim shoair 82b09620b2 perf: Improving all browser-based fetches by ~15% 2025-09-20 06:20:23 +03:00
Abdullah 7ba333c3c9 perf(session): improve page management and speed up execution
Close pages immediately after tasks finish to reduce resource usage and
improve overall performance. Update corresponding test files to reflect
the new page lifecycle management.
2025-09-17 18:49:45 +03:00
Karim shoair 5bcd4c2927 fix: A workaround for Playwright issue with page content retrieving on windows 2025-09-15 21:34:20 +03:00
Karim shoair 7a7097b775 fix: Stop removing browser default page
Looks like it's problematic with some systems so it doesn't worth the trouble
2025-09-15 13:43:47 +03:00
Karim shoair 13d7e70cb7 refactor: Make all fetchers as an optional dependency group
+ Removing some dead code
2025-09-13 16:07:48 +03:00
Karim shoair 8c4d6158d6 feat(browser fetchers): A new option to control page JS 2025-09-13 03:59:12 +03:00
Karim shoair 60be9dc816 feat: Validate all fetch-level parameters
So it's validated like the session-level ones. Needs improvement, but that's for later
2025-09-13 03:22:08 +03:00
Karim shoair adb82008e5 style: Using keywords for validation
Less lines of code and more stylish
2025-09-12 04:33:00 +03:00
Karim shoair 05a714980d docs: Update all docstring according to the new changes 2025-09-11 04:13:36 +03:00
Karim shoair 9838d741d0 refactor/feat(browser fetchers): make it possible to have a configuration per page in sessions
- Also, no need for the `page_action` argument function to return the page again
2025-09-11 03:48:55 +03:00
Karim shoair ecaec65049 refactor/fix(browser fetchers): Solving a logic bug in page rotation
- Pages that get reused in rotation are possibly contaminated from previous settings used on them, and some are stubborn to remove, so the new approach replaces finished pages with new ones.
- Removed the `max_pages` from sync `StealthySession` to match `DynamicSession` (Doesn't mean anything in sync code)
2025-09-08 16:08:57 +03:00
Karim shoair f298b76b8e perf(StealthySession): Close persistent context default page 2025-09-07 18:48:51 +03:00
Karim shoair 4341477a66 perf(StealthyFetcher): Slight optimization 2025-09-07 04:58:55 +03:00
Karim shoair 9543f171aa feat(StealthyFetcher): add the ability to add init_script to context 2025-09-02 16:19:22 +03:00
Karim shoair 2f402f4835 style: add flags for tests coverage
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
Karim shoair ae9ccaec79 style: A lot of type hints correction
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
Karim shoair 264ae02aa7 refactor: huge change, many features/class got a better naming
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
Karim shoair d1aa0be6e4 refactor(StealthyFetcher): Remove virtual mode and use persistent context
Solves #64 completely too
2025-07-27 03:41:28 +03:00
Karim shoair 53318f3a2a fix(StealthyFetcher): Fix passed proxy type 2025-06-26 16:36:30 +03:00
Karim shoair 63ff12bcd6 docs: correcting a small mistake 2025-06-23 03:38:34 +03:00
Karim shoair 78b81c537f fix(fetchers): Fix the bug of referer and google_search argument conflict 2025-06-23 03:36:11 +03:00
Karim shoair 69b983b42f feat(fetchers): Improve StealthyFetcher + Adding StealthySession/AsyncStealthySession classes 2025-06-23 03:23:28 +03:00