Karim shoair
7089fadf8c
docs: adjustments
2026-03-07 17:59:04 +02:00
Karim shoair
685d1fdcc4
docs: move up a sponsor
2026-03-07 17:45:36 +02:00
Karim shoair
5215d02666
docs: update website sponsors placement
2026-03-06 16:58:17 +02:00
Karim shoair
69c720e710
docs: update ads prices
2026-03-06 16:45:51 +02:00
Karim shoair
bcdfba4d07
docs: add French language to the translated README files
2026-03-03 21:26:05 +02:00
Karim shoair
b40a1a37b9
docs: add a new special sponsor
2026-03-03 21:17:13 +02:00
Karim shoair
bf79102951
docs: sponsors field adjustment
...
Removed unnecessary line breaks in the description.
2026-03-03 14:18:44 +02:00
Karim shoair
5ddcfa1073
docs: add new platinum sponsor
2026-03-03 00:24:46 +02:00
Karim shoair
f86fa1aad5
docs: add new sponsor image
2026-03-03 00:08:30 +02:00
Karim shoair
4ba93d78ac
docs: updates to the sponsorship plans
2026-03-02 17:44:20 +02:00
Karim shoair
08ffa7a9d7
docs: updates to the sponsorship plans
2026-03-02 17:38:40 +02:00
Karim shoair
5db4c96681
docs: update a sponsor status
2026-03-02 17:32:00 +02:00
Karim shoair
0af4fa24a6
docs: add a citations section
2026-02-28 17:03:05 +02:00
Karim shoair
b9fa6b62e6
docs: make the contribution rules clearer
2026-02-28 16:57:41 +02:00
Karim shoair
50a69c3583
ops: update workflows triggers
...
and pinning playwright version here too
2026-02-28 16:45:26 +02:00
Karim shoair
69f916d694
docs: Enhance CONTRIBUTING rules with PR guidelines and updates
...
Added guidelines for making pull requests and updated documentation on building and testing.
2026-02-28 16:41:44 +02:00
Karim shoair
2206d052fd
docs: update a sponsor url
2026-02-28 02:25:15 +02:00
Karim shoair
1f53620623
docs: remove a sponsor
2026-02-27 14:59:43 +02:00
Karim shoair
bbdc4cec5d
ops: add the mcp server to the docker file
2026-02-27 06:20:05 +02:00
Karim shoair
2b50123fa6
v0.4.1 ( #154 )
2026-02-27 06:12:45 +02:00
Karim shoair
d0cc6e5a3d
ops: pump mcp server version up
2026-02-27 04:52:16 +02:00
Karim shoair
e34651d887
fix(stealth): improve stealth mode by removing unnecessary scripts
...
It doesn't add anything to the table as before (it might turn out I was wrong and add them again lol)
2026-02-27 03:50:14 +02:00
Karim shoair
b50e8f050f
fix(mcp): make mcp use less tokens by striping useless tags
2026-02-27 03:39:48 +02:00
Karim shoair
d750fec468
fix(solver): Solve CF faster and handle websites that show captcha twice
2026-02-27 02:32:21 +02:00
Karim shoair
ef29568cd0
docs: add a code snippet on how to do scrapling install from code
2026-02-27 01:17:54 +02:00
Karim shoair
5f885ae5f5
docs(mcp): update cursor url
2026-02-27 01:03:56 +02:00
Karim shoair
efb320bb8c
docs: style adjustments
2026-02-27 00:49:00 +02:00
Karim shoair
a8359a4878
build: pump version up and the deps
2026-02-27 00:32:25 +02:00
Karim shoair
921314c314
fix: complete MCP schema validation for remaining bare array types
...
- Fix params (Dict|List|Tuple → Dict), urls (Tuple[str,...] → List[str]),
- Renamed _NormalizeCredentials/_ContentTranslator to snake_case.
- Also, raise a ValueError on invalid credentials instead of returning None silently.
2026-02-27 00:28:44 +02:00
Karim shoair
86e0f0f250
docs: update deps for the website
2026-02-27 00:11:02 +02:00
Karim shoair
53dd360296
docs: add mcp integrated badge
2026-02-26 23:56:42 +02:00
Karim shoair
2904441b42
fix: make MCP get schemas validator-safe ( #137 )
2026-02-26 23:54:48 +02:00
Karim shoair
6c12c196ac
Merge branch 'dev' into fix/mcp-schema-auth-cookies
2026-02-26 23:54:18 +02:00
Karim shoair
bcc93f392d
Merge branch 'main' into dev
2026-02-26 18:57:27 +02:00
Karim shoair
b54a49b717
ops: Upgrade GitHub Actions to latest versions ( #144 )
2026-02-26 18:56:54 +02:00
Karim shoair
46fd1506dc
Merge branch 'main' into upgrade-github-actions-node24-general
2026-02-26 18:56:36 +02:00
Karim shoair
908ede1c64
ops: Upgrade GitHub Actions for Node 24 compatibility ( #143 )
2026-02-26 18:56:20 +02:00
Karim shoair
ccce0fc552
Merge branch 'main' into upgrade-github-actions-node24
2026-02-26 18:55:45 +02:00
Karim shoair
9708354c80
Fix: Improve regex precision for Cloudflare challenge detection ( #133 )
2026-02-26 18:53:13 +02:00
Karim shoair
50e0be5c1c
Merge branch 'dev' into fix/cloudflare-regex
2026-02-26 18:52:37 +02:00
Karim shoair
558e9a6fa0
Merge branch 'main' into dev
2026-02-25 23:55:53 +02:00
Karim shoair
7ebe3a01a3
docs: removing an irrelevant article
2026-02-25 23:54:24 +02:00
Karim shoair
6943ec27f8
docs: typo correction
2026-02-25 02:23:36 +02:00
Karim shoair
e7b0fe7b46
Merge branch 'main' into dev
2026-02-25 02:23:08 +02:00
Karim shoair
5b948d34df
docs: add the trending badge
2026-02-25 02:06:09 +02:00
Karim shoair
d934fe876c
Merge branch 'main' into dev
2026-02-25 01:59:14 +02:00
Karim shoair
8c9402f9f0
docs: adding new sponsor
2026-02-24 15:28:37 +02:00
Karim shoair
c0aec234eb
docs: update sponsorship plans and corrections
2026-02-24 15:19:23 +02:00
Karim shoair
b503f537ce
docs: corrections to the sponsorship page
2026-02-23 03:49:16 +02:00
Karim shoair
a0cc3857d0
docs: corrections to the sponsorship page
2026-02-23 03:27:43 +02:00
Karim shoair
aed842c42e
docs: update sponsorship plans
2026-02-23 03:25:18 +02:00
Karim shoair
f537613921
docs: shortening the demo code in the README
2026-02-23 03:04:27 +02:00
Karim shoair
a6beaaba89
docs: adjustments to sponsors placement
2026-02-23 02:58:25 +02:00
Karim shoair
4072c5e83d
docs: update readme URLs
2026-02-23 02:45:37 +02:00
Karim shoair
1f5a7d5d2c
ops: add mcp registry file
2026-02-23 02:31:28 +02:00
Karim shoair
2b56a87e36
Merge branch 'main' into dev
2026-02-18 04:47:32 +02:00
Karim shoair
cd13a695cb
docs: adjustments to website metadata
2026-02-18 04:33:39 +02:00
Karim shoair
a9b785aa8f
docs: add media preview image
2026-02-18 04:26:09 +02:00
Karim shoair
60eea3ba68
docs: add media preview image
2026-02-18 00:04:53 +02:00
Karim shoair
f21e7462e4
docs: fix cookie consent
2026-02-15 15:11:48 +02:00
Karim shoair
1bbb18ad8e
build: fix readme path for pypi
2026-02-15 07:16:28 +02:00
Karim shoair
04d796b821
v0.4
2026-02-15 07:13:28 +02:00
Karim shoair
95ad2842e9
docs: add an entry for the 0.4 version
2026-02-15 04:40:43 +02:00
Karim shoair
b19fa5e694
fix(checkpoint): correct error message
2026-02-15 04:36:43 +02:00
Karim shoair
e35ba3455c
test: add tests for spiders engine
...
Automated tests written by AI
2026-02-15 04:35:28 +02:00
Karim shoair
02b11c535e
fix(spiders): corrections and removal of dead code
2026-02-15 04:25:32 +02:00
Karim shoair
b8b9a7f864
docs: general correction across all the pages
...
Automated. I don't know how I missed all of that!
2026-02-15 03:58:51 +02:00
Karim shoair
93bcb1681b
style: docstrings corrections for accuracy
2026-02-15 03:57:14 +02:00
Karim shoair
1bf6bb8d65
docs: update the images urls to use main branch
2026-02-15 03:29:57 +02:00
Karim shoair
7b8fc911e1
docs: add a page for spiders advanced usage
2026-02-15 02:40:55 +02:00
Karim shoair
7b4523af8e
docs: update the titles for all files
2026-02-15 02:35:39 +02:00
Karim shoair
c3374bce41
docs: add a note about response body
2026-02-15 01:03:36 +02:00
Karim shoair
ba4d33337f
fix: make browser fetchers type hints always present
2026-02-15 01:03:15 +02:00
Karim shoair
77e825e7be
docs(spider): add a new page for proxy management & blocking
2026-02-15 01:01:43 +02:00
Karim shoair
dee0fd00b8
docs: use more accurate naming for proxy rotation strategy
2026-02-14 23:21:08 +02:00
Karim shoair
0ab54852c4
fix: use more accurate naming for proxy rotation strategy
...
It can be considered the simplest form of round robin since we don't have weights or anything (as was originally planned), but let's change it to avoid confusion or useless debates. Here goes nothing
2026-02-14 23:19:54 +02:00
Karim shoair
c94ebd2fa9
docs(spiders): add a page to explain sessions logic
2026-02-14 17:19:55 +02:00
Karim shoair
108e11fc30
docs: add a new issue template for documentation issues
2026-02-14 17:08:32 +02:00
Karim shoair
dddb179c50
docs: add new api references
2026-02-14 17:07:55 +02:00
Karim shoair
a8f701f8ab
docs(spiders): add a new page for requests & responses
2026-02-14 04:04:18 +02:00
Karim shoair
2997d562d5
test: remove irrelevent test
2026-02-14 03:34:41 +02:00
Karim shoair
e785cf26ce
fix(parser)!: Optimize parser for repeated operations
2026-02-14 03:34:13 +02:00
Karim shoair
d0a5a01299
test: update spider check
2026-02-14 02:51:39 +02:00
Karim shoair
6a0c323482
docs: adding a new Pick Your Path section to the overview page
2026-02-14 00:43:05 +02:00
Karim shoair
c6a662eb38
docs(dynamic pages): add an important warning
2026-02-13 18:45:44 +02:00
Karim shoair
2eea66dad1
style(spiders): Lower the default number of concurrent requests
2026-02-13 17:32:17 +02:00
Karim shoair
a62330d2f1
fix: add typed overloads to Selectors.get() for proper default type inference
...
Type checkers now correctly infer the return type based on the default value:
- .get() → TextHandler | None
- .get("") → TextHandler | str
- .get(0) → TextHandler | int
2026-02-13 17:30:55 +02:00
Karim shoair
1c2c7c55bf
docs: use a better looking architecture image instead of mermaid
2026-02-12 15:17:58 +02:00
Karim shoair
14143dd0f0
docs: add getting started page to the spiders section
2026-02-12 04:45:39 +02:00
Karim shoair
bbb8c26df0
docs: Improve spider architecture page
2026-02-12 04:30:16 +02:00
Karim shoair
d0b2cec849
docs: simplify the architecture mermaid
2026-02-12 03:30:14 +02:00
Karim shoair
5a599445e2
docs: update libraries
2026-02-12 03:29:41 +02:00
Karim shoair
10cd46ee23
docs: add the first page of the spiders section
2026-02-11 03:55:23 +02:00
Karim shoair
8902034418
docs: improve wording, style, and use more visual ways
2026-02-11 03:04:35 +02:00
Karim shoair
6ac40ac3af
build: adjustments to build file for PyPi
2026-02-11 00:29:20 +02:00
Karim shoair
a38f5af615
docs: fix image in the interactive shell page
2026-02-11 00:19:38 +02:00
Karim shoair
b52d35c6ec
docs: update beautifulsoup migration guide
2026-02-10 22:13:04 +02:00
Karim shoair
8e4b4a22c9
docs: updating extract commans page
2026-02-10 22:09:55 +02:00
Karim shoair
7fb57d667c
docs: updating Interactive shell page and some corrections
2026-02-10 22:07:55 +02:00
Karim shoair
a72886f158
docs: update camoufox instructions for StealthyFetcher
2026-02-10 21:58:08 +02:00
Karim shoair