Karim shoair
adb82008e5
style: Using keywords for validation
...
Less lines of code and more stylish
2025-09-12 04:33:00 +03:00
Karim shoair
e4b1e00e63
build: Pump up the version
2025-09-11 04:14:32 +03:00
Karim shoair
05a714980d
docs: Update all docstring according to the new changes
2025-09-11 04:13:36 +03:00
Karim shoair
9838d741d0
refactor/feat(browser fetchers): make it possible to have a configuration per page in sessions
...
- Also, no need for the `page_action` argument function to return the page again
2025-09-11 03:48:55 +03:00
Karim shoair
931993f901
tests: changing tests accordingly
2025-09-08 16:09:24 +03:00
Karim shoair
ecaec65049
refactor/fix(browser fetchers): Solving a logic bug in page rotation
...
- Pages that get reused in rotation are possibly contaminated from previous settings used on them, and some are stubborn to remove, so the new approach replaces finished pages with new ones.
- Removed the `max_pages` from sync `StealthySession` to match `DynamicSession` (Doesn't mean anything in sync code)
2025-09-08 16:08:57 +03:00
Karim shoair
f298b76b8e
perf(StealthySession): Close persistent context default page
2025-09-07 18:48:51 +03:00
Karim shoair
710f994339
perf(DynamicSession): Close persistent context default page
2025-09-07 18:48:40 +03:00
Karim shoair
05761d4070
docs: Add YT video to the mcp page
2025-09-07 05:06:35 +03:00
Karim shoair
4341477a66
perf(StealthyFetcher): Slight optimization
2025-09-07 04:58:55 +03:00
Karim shoair
9e5ff5bb1b
chore: Add issue template for other issues
2025-09-07 04:36:00 +03:00
Karim shoair
72e26c23e8
fix(DynamicFetcher): Improve stealth mode
2025-09-07 03:13:51 +03:00
Karim shoair
31c14b3997
[ImgBot] Optimize images ( #79 )
2025-09-05 13:51:45 +03:00
Karim shoair
fa76d64d61
docs: add new sponsor to the README
2025-09-05 13:46:17 +03:00
Karim shoair
6f232b3874
misc: add new sponsor logo
2025-09-05 13:44:37 +03:00
Karim shoair
23997f2755
v0.3.1 ( #78 )
2025-09-02 17:20:53 +03:00
Karim shoair
934d8a1452
build: Pump up the version
2025-09-02 16:33:52 +03:00
Karim shoair
e06f8d36db
docs: add init_script new argument
2025-09-02 16:32:19 +03:00
Karim shoair
6ad1bab2e4
feat(DynamicFetcher): add the ability to add init_script to context ( #56 )
2025-09-02 16:25:13 +03:00
Karim shoair
9543f171aa
feat(StealthyFetcher): add the ability to add init_script to context
2025-09-02 16:19:22 +03:00
Karim shoair
d49e047736
tests: change tests accordingly
2025-09-02 15:23:30 +03:00
Karim shoair
af9a62a539
fix: fix for shell imported by mistake issue ( #76 )
2025-09-02 15:23:15 +03:00
Karim shoair
e7482a267a
docs: add interactive shell video to README.md
2025-09-01 22:11:40 +03:00
Karim shoair
9021c4f9b7
docs: Changing sponsors order
2025-09-01 16:03:47 +03:00
Karim shoair
d5aaeb9816
v0.3
2025-09-01 07:26:15 +03:00
Karim shoair
812cf2d21b
build: Pump up the version
2025-09-01 06:43:41 +03:00
Karim shoair
9e9e79d9b0
docs: fixes and adjustments
2025-09-01 06:39:17 +03:00
Karim shoair
35b88fbb62
docs: add an api reference to the mcp server
2025-09-01 01:02:08 +03:00
Karim shoair
7befcd54bc
chore: update the setup file description
2025-08-31 23:50:33 +03:00
Karim shoair
e7858d7307
docs: updating the mkdocs file accordingly
2025-08-31 23:41:53 +03:00
Karim shoair
e0c005b2cc
docs: add a page to the mcp server
2025-08-31 23:40:50 +03:00
Karim shoair
734e5f73db
docs: Changing fonts and fixing some mistakes
2025-08-31 18:52:15 +03:00
Karim shoair
878be9ccbf
tests: update tests accordingly
2025-08-31 02:57:30 +03:00
Karim shoair
967c0956a1
feat(DynamicSession): Improve stealth mode
2025-08-31 02:57:12 +03:00
Karim shoair
6eb580a6d0
feat(DynamicSession): Improve normal fetches speed
2025-08-31 02:56:34 +03:00
Karim shoair
b5be2547d7
fix(dynamicSession): issue with headless
2025-08-31 01:29:47 +03:00
Karim shoair
7153fdbcce
Merge branch 'dev' of https://github.com/D4Vinci/Scrapling into dev
2025-08-30 22:42:09 +03:00
Karim shoair
beebddd6fe
docs: add cli overview page
2025-08-30 22:42:05 +03:00
Karim shoair
c97a82f2d4
docs: add the page for the extract command
2025-08-30 22:35:44 +03:00
Karim shoair
47dd2ed28f
Merge branch 'main' into dev
2025-08-30 22:35:25 +03:00
Karim shoair
76721f906e
docs(cli): Correcting docstrings
2025-08-30 22:12:33 +03:00
Karim shoair
9dfca4c6f2
misc: Add ko-fi username to FUNDING.yml
2025-08-30 22:08:25 +03:00
Karim shoair
bf55205436
docs: Update README.md
2025-08-30 20:11:43 +03:00
Karim shoair
8edcd611d3
docs: replace the test website
2025-08-30 18:51:31 +03:00
Karim shoair
c66c48e3ae
fix(extract): Correcting docs for the css-selector option
2025-08-29 21:34:39 +03:00
Karim shoair
cd388059e8
docs: Add a page about the interactive shell
2025-08-29 20:31:00 +03:00
Karim shoair
badc3b21e1
docs: update article A Free Alternative to AI for Robust Web Scraping
2025-08-28 23:18:56 +03:00
Karim shoair
6d419e0f4d
docs: update BS article
2025-08-28 05:01:38 +03:00
Karim shoair
e14522f57a
docs: update the StealthyFetcher page
2025-08-28 04:53:22 +03:00
Karim shoair
1f87210b93
docs: Add max_pages explanation to DynamicFetcher
2025-08-28 04:36:49 +03:00
Karim shoair
31d555687f
docs: update the DynamicFetcher page
2025-08-28 03:59:00 +03:00
Karim shoair
18f9e1efb0
docs: update the fetcher page
2025-08-28 02:07:37 +03:00
Karim shoair
2435d00f16
docs: update fetchers choosing page
2025-08-28 02:07:22 +03:00
Karim shoair
d65a30e6b6
docs: update fetchers choosing page
2025-08-27 16:21:46 +03:00
Karim shoair
c6088f641b
docs: update Readme's downloads badge
2025-08-26 20:43:22 +03:00
Karim shoair
9120fe14bf
docs: update fetchers choosing page
2025-08-26 04:54:00 +03:00
Karim shoair
bac3ffefcf
docs: update fetchers page in the API reference
2025-08-25 06:15:20 +03:00
Karim shoair
41aceb2c43
docs: update storage dev tutorial
2025-08-25 06:10:46 +03:00
Karim shoair
cf57af588e
docs: update doc strings with correct naming
2025-08-25 06:08:15 +03:00
Karim shoair
6ddabaa593
docs: replace automatch page with adaptive + updating
2025-08-25 05:57:29 +03:00
Karim shoair
d5d4003277
docs: update the sponsors page
2025-08-25 00:42:31 +03:00
Karim shoair
a917899e4f
docs: replace adaptor with selector in the API reference
2025-08-25 00:42:01 +03:00
Karim shoair
545e03229d
docs: Update the 'main classes' page
2025-08-25 00:40:24 +03:00
Karim shoair
8eb51f5653
docs: updating 'Querying elements' page
2025-08-24 22:36:00 +03:00
Karim shoair
040246177f
docs: Update all benchmarks
2025-08-24 16:06:07 +03:00
Karim shoair
dc0107666f
docs: small update to main page
2025-08-24 16:03:17 +03:00
Karim shoair
eaf3751f76
docs: Update overview page
2025-08-24 16:02:21 +03:00
Karim shoair
b467cd025f
docs: update website main page
2025-08-24 04:00:16 +03:00
Karim shoair
0c63f95d07
docs: update the README
2025-08-24 03:04:52 +03:00
Karim shoair
a8f5c55f06
Merge branch 'main' into dev
2025-08-23 23:33:10 +03:00
Karim shoair
bdbc877b4b
[ImgBot] Optimize images ( #72 )
2025-08-23 23:20:32 +03:00
Karim shoair
377a5ceb09
Move docs to the main branch ( #71 )
2025-08-23 23:17:00 +03:00
Karim shoair
f8fbd4b0e0
ops: combine the auto-release/publish workflows
2025-08-23 22:43:03 +03:00
Karim shoair
637580e130
tests: stop testing geoip
...
The DB is downloaded from GitHub releases using public API so we get rate limited a lot
2025-08-23 22:20:51 +03:00
Karim shoair
b50c85fa60
docs: Update the README file to reflect the new version changes
2025-08-23 19:22:35 +03:00
Karim shoair
5389b5c7ff
docs: add PetroSky as a top sponsor
2025-08-21 04:12:13 +03:00
Karim shoair
dd4fcc47c8
test: remove geoip from the async camoufox test
...
to lower errors of rate limiting caused by downloading the geo db from GitHub
2025-08-20 16:36:58 +03:00
Karim shoair
293206b466
Update index.md
2025-08-20 16:12:12 +03:00
Karim shoair
9d880f2731
Update index.md
2025-08-20 16:11:02 +03:00
Karim shoair
df3ddb1c78
docs: Update how sponsors displayed on the website
2025-08-20 15:36:01 +03:00
Karim shoair
cc84fa2f54
Merge branch 'main' into docs
2025-08-20 15:30:16 +03:00
Karim shoair
07773eb2f3
Merge branch 'main' into dev
2025-08-20 15:30:02 +03:00
Karim shoair
4fa92e1073
docs: update how sponsors displayed
2025-08-20 15:20:25 +03:00
Karim shoair
8a2bb0573b
chore: replace sponsors banners with logos
2025-08-20 15:04:19 +03:00
Karim shoair
44d2b55f99
test: fix for GH actions
2025-08-20 04:33:20 +03:00
Karim shoair
6b8d861b92
build: Move non-primarily deps to optional extras
2025-08-20 03:58:10 +03:00
Karim shoair
85885d9962
tests: add geoip to Stealth tests
2025-08-20 01:36:55 +03:00
Karim shoair
57a025a34f
build: update dependencies and fix geoip issue
2025-08-20 01:10:17 +03:00
Karim shoair
5aea62256b
test: adding new tests and updating existing ones
...
Code Coverage is now 92%
2025-08-17 01:03:19 +03:00
Karim shoair
2f402f4835
style: add flags for tests coverage
...
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
Karim shoair
009bbb20c0
refactor(dynamic Fetcher): small optimization
2025-08-16 23:20:54 +03:00
Karim shoair
3e0ca33111
test: adding new tests and updating existing ones
2025-08-16 19:07:08 +03:00
Karim shoair
4c33c47609
test: Update and improve Camo/Dynamic tests
2025-08-16 16:02:17 +03:00
Karim shoair
6c9a960a44
fix(dynamicFetcher): Fix a bug with real_chrome argument
2025-08-16 16:01:32 +03:00
Karim shoair
e6e5b3cc80
test: adding new tests and updating existing ones
...
The coverage is now 78%
2025-08-16 14:58:11 +03:00
Karim shoair
84337ef1ac
fix(parser): count all nested children of ignored tags in get_all_text
2025-08-16 14:24:51 +03:00
Karim shoair
8bfbcec2d7
test: remove the test for install command
...
GitHub is already reinstalls the library with it with every test
2025-08-15 06:01:53 +03:00
Karim shoair
0e3358007f
test: adding new tests and updating existing ones
2025-08-15 04:52:51 +03:00
Karim shoair
13700e2692
fix: improve error handling
2025-08-15 04:52:11 +03:00
Karim shoair
ba81890daf
test: adding new tests and updating existing ones
2025-08-03 16:37:47 +03:00
Karim shoair