freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,212 Commits 1 Branch 0 Tags
7cbe566acdbc3cfc78a375b454bf6c226e10a665
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Karim shoair e045d0f73f Merge branch 'dev' into dev 2026-03-08 16:31:25 +02:00
Karim shoair 49db2b643b Merge branch 'main' into dev 2026-03-08 16:27:25 +02:00
Karim shoair b9b5bb0c80 docs: add badges for the agent skill and Clawhub 2026-03-08 06:56:50 +02:00
Karim shoair 68e6c3c3e8 docs: adjustments 2026-03-08 06:39:49 +02:00
Karim shoair d1a0b815f5 docs: update translation files with new sponsor 2026-03-08 06:27:04 +02:00
Karim shoair cb7e9b9ef3 docs: add new platinum sponsor 2026-03-08 06:16:48 +02:00
Karim shoair 25de97f534 docs: update the agent skill readme 2026-03-08 03:30:04 +02:00
Karim shoair 68da2af556 docs: update the agent skill readme 2026-03-08 03:21:23 +02:00
Karim shoair 54f2c96adf Merge branch 'main' into dev 2026-03-08 03:06:17 +02:00
Karim shoair f469fb9b8b feat: Upload the library agent skill 2026-03-08 03:06:01 +02:00
Karim shoair aa3f4b4b0e docs: remove previous versions pages 
The Change log page is sufficient.
 2026-03-08 02:47:53 +02:00
Karim shoair 27db107886 ops: exclude agent skill from workflow triggers 2026-03-08 02:46:54 +02:00
Karim shoair 98be98e875 docs: adjustments 2026-03-07 18:57:53 +02:00
Karim shoair cd73d901ab docs: adjustments 2026-03-07 18:54:29 +02:00
Karim shoair df7ded440c docs: table adjustments 2026-03-07 18:18:01 +02:00
Karim shoair 7089fadf8c docs: adjustments 2026-03-07 17:59:04 +02:00
Karim shoair 685d1fdcc4 docs: move up a sponsor 2026-03-07 17:45:36 +02:00
Karim shoair 5215d02666 docs: update website sponsors placement 2026-03-06 16:58:17 +02:00
Karim shoair 69c720e710 docs: update ads prices 2026-03-06 16:45:51 +02:00
Karim shoair c8e1d8d75d fix(type hints): use correct import for Python < 3.12 (Fixes #163 ) 2026-03-05 16:45:10 +02:00
Karim shoairandChristian Rost 37f5c48a5f build: pump up the version 
Co-Authored-By: Christian Rost <chr@baltic-online.de>
 2026-03-03 23:09:36 +02:00
Karim shoairandChristian Rost 1839506622 fix(browsers): make flag concatenation type-safe 
Better implementation of #157 that's accepted by pyright/mypy.
Thanks to @rostchri .

Co-Authored-By: Christian Rost <chr@baltic-online.de>
 2026-03-03 23:08:15 +02:00
Karim shoair 1142c4bf27 Merge branch 'main' into dev 2026-03-03 23:02:23 +02:00
Karim shoair bcdfba4d07 docs: add French language to the translated README files 2026-03-03 21:26:05 +02:00
Karim shoair b40a1a37b9 docs: add a new special sponsor 2026-03-03 21:17:13 +02:00
Karim shoair efe2c89a20 Merge branch 'main' into dev 2026-03-03 14:19:10 +02:00
Karim shoair bf79102951 docs: sponsors field adjustment 
Removed unnecessary line breaks in the description.
 2026-03-03 14:18:44 +02:00
Karim shoair 45a03cf3e3 Merge branch 'main' into dev 2026-03-03 03:16:19 +02:00
Karim shoair 5ddcfa1073 docs: add new platinum sponsor 2026-03-03 00:24:46 +02:00
Karim shoair f86fa1aad5 docs: add new sponsor image 2026-03-03 00:08:30 +02:00
Karim shoair 9bb605c814 Merge branch 'main' into dev 2026-03-02 22:37:35 +02:00
Karim shoair 4ba93d78ac docs: updates to the sponsorship plans 2026-03-02 17:44:20 +02:00
Karim shoair 08ffa7a9d7 docs: updates to the sponsorship plans 2026-03-02 17:38:40 +02:00
Karim shoair 5db4c96681 docs: update a sponsor status 2026-03-02 17:32:00 +02:00
Karim shoair e04d59bd79 docs: update a sponsor status 2026-03-02 17:31:14 +02:00
Karim shoair 0af4fa24a6 docs: add a citations section 2026-02-28 17:03:05 +02:00
Karim shoair b9fa6b62e6 docs: make the contribution rules clearer 2026-02-28 16:57:41 +02:00
Karim shoair 50a69c3583 ops: update workflows triggers 
and pinning playwright version here too
 2026-02-28 16:45:26 +02:00
Karim shoair 69f916d694 docs: Enhance CONTRIBUTING rules with PR guidelines and updates 
Added guidelines for making pull requests and updated documentation on building and testing.
 2026-02-28 16:41:44 +02:00
Karim shoair 2206d052fd docs: update a sponsor url 2026-02-28 02:25:15 +02:00
Karim shoair 1f53620623 docs: remove a sponsor 2026-02-27 14:59:43 +02:00
Karim shoair bbdc4cec5d ops: add the mcp server to the docker file 2026-02-27 06:20:05 +02:00
Karim shoair 2b50123fa6 v0.4.1 (#154) 2026-02-27 06:12:45 +02:00
Karim shoair d0cc6e5a3d ops: pump mcp server version up 2026-02-27 04:52:16 +02:00
Karim shoair e34651d887 fix(stealth): improve stealth mode by removing unnecessary scripts 
It doesn't add anything to the table as before (it might turn out I was wrong and add them again lol)
 2026-02-27 03:50:14 +02:00
Karim shoair b50e8f050f fix(mcp): make mcp use less tokens by striping useless tags 2026-02-27 03:39:48 +02:00
Karim shoair d750fec468 fix(solver): Solve CF faster and handle websites that show captcha twice 2026-02-27 02:32:21 +02:00
Karim shoair ef29568cd0 docs: add a code snippet on how to do scrapling install from code 2026-02-27 01:17:54 +02:00
Karim shoair 5f885ae5f5 docs(mcp): update cursor url 2026-02-27 01:03:56 +02:00
Karim shoair efb320bb8c docs: style adjustments 2026-02-27 00:49:00 +02:00
Karim shoair a8359a4878 build: pump version up and the deps 2026-02-27 00:32:25 +02:00
Karim shoair 921314c314 fix: complete MCP schema validation for remaining bare array types 
- Fix params (Dict|List|Tuple → Dict), urls (Tuple[str,...] → List[str]),
- Renamed _NormalizeCredentials/_ContentTranslator to snake_case.
- Also, raise a ValueError on invalid credentials instead of returning None silently.
 2026-02-27 00:28:44 +02:00
Karim shoair 86e0f0f250 docs: update deps for the website 2026-02-27 00:11:02 +02:00
Karim shoair 53dd360296 docs: add mcp integrated badge 2026-02-26 23:56:42 +02:00
Karim shoair 2904441b42 fix: make MCP get schemas validator-safe (#137) 2026-02-26 23:54:48 +02:00
Karim shoair 6c12c196ac Merge branch 'dev' into fix/mcp-schema-auth-cookies 2026-02-26 23:54:18 +02:00
Karim shoair bcc93f392d Merge branch 'main' into dev 2026-02-26 18:57:27 +02:00
Karim shoair b54a49b717 ops: Upgrade GitHub Actions to latest versions (#144) 2026-02-26 18:56:54 +02:00
Karim shoair 46fd1506dc Merge branch 'main' into upgrade-github-actions-node24-general 2026-02-26 18:56:36 +02:00
Karim shoair 908ede1c64 ops: Upgrade GitHub Actions for Node 24 compatibility (#143) 2026-02-26 18:56:20 +02:00
Karim shoair ccce0fc552 Merge branch 'main' into upgrade-github-actions-node24 2026-02-26 18:55:45 +02:00
Karim shoair 9708354c80 Fix: Improve regex precision for Cloudflare challenge detection (#133) 2026-02-26 18:53:13 +02:00
Karim shoair 50e0be5c1c Merge branch 'dev' into fix/cloudflare-regex 2026-02-26 18:52:37 +02:00
Karim shoair 558e9a6fa0 Merge branch 'main' into dev 2026-02-25 23:55:53 +02:00
Karim shoair 7ebe3a01a3 docs: removing an irrelevant article 2026-02-25 23:54:24 +02:00
Karim shoair 6943ec27f8 docs: typo correction 2026-02-25 02:23:36 +02:00
Karim shoair e7b0fe7b46 Merge branch 'main' into dev 2026-02-25 02:23:08 +02:00
Karim shoair 5b948d34df docs: add the trending badge 2026-02-25 02:06:09 +02:00
Karim shoair d934fe876c Merge branch 'main' into dev 2026-02-25 01:59:14 +02:00
Karim shoair 8c9402f9f0 docs: adding new sponsor 2026-02-24 15:28:37 +02:00
Karim shoair c0aec234eb docs: update sponsorship plans and corrections 2026-02-24 15:19:23 +02:00
Karim shoair b503f537ce docs: corrections to the sponsorship page 2026-02-23 03:49:16 +02:00
Karim shoair a0cc3857d0 docs: corrections to the sponsorship page 2026-02-23 03:27:43 +02:00
Karim shoair aed842c42e docs: update sponsorship plans 2026-02-23 03:25:18 +02:00
Karim shoair f537613921 docs: shortening the demo code in the README 2026-02-23 03:04:27 +02:00
Karim shoair a6beaaba89 docs: adjustments to sponsors placement 2026-02-23 02:58:25 +02:00
Karim shoair 4072c5e83d docs: update readme URLs 2026-02-23 02:45:37 +02:00
Karim shoair 1f5a7d5d2c ops: add mcp registry file 2026-02-23 02:31:28 +02:00
Karim shoair 2b56a87e36 Merge branch 'main' into dev 2026-02-18 04:47:32 +02:00
Karim shoair cd13a695cb docs: adjustments to website metadata 2026-02-18 04:33:39 +02:00
Karim shoair a9b785aa8f docs: add media preview image 2026-02-18 04:26:09 +02:00
Karim shoair 60eea3ba68 docs: add media preview image 2026-02-18 00:04:53 +02:00
Karim shoair f21e7462e4 docs: fix cookie consent 2026-02-15 15:11:48 +02:00
Karim shoair 1bbb18ad8e build: fix readme path for pypi 2026-02-15 07:16:28 +02:00
Karim shoair 04d796b821 v0.4 2026-02-15 07:13:28 +02:00
Karim shoair 95ad2842e9 docs: add an entry for the 0.4 version 2026-02-15 04:40:43 +02:00
Karim shoair b19fa5e694 fix(checkpoint): correct error message 2026-02-15 04:36:43 +02:00
Karim shoair e35ba3455c test: add tests for spiders engine 
Automated tests written by AI
 2026-02-15 04:35:28 +02:00
Karim shoair 02b11c535e fix(spiders): corrections and removal of dead code 2026-02-15 04:25:32 +02:00
Karim shoair b8b9a7f864 docs: general correction across all the pages 
Automated. I don't know how I missed all of that!
 2026-02-15 03:58:51 +02:00
Karim shoair 93bcb1681b style: docstrings corrections for accuracy 2026-02-15 03:57:14 +02:00
Karim shoair 1bf6bb8d65 docs: update the images urls to use main branch 2026-02-15 03:29:57 +02:00
Karim shoair 7b8fc911e1 docs: add a page for spiders advanced usage 2026-02-15 02:40:55 +02:00
Karim shoair 7b4523af8e docs: update the titles for all files 2026-02-15 02:35:39 +02:00
Karim shoair c3374bce41 docs: add a note about response body 2026-02-15 01:03:36 +02:00
Karim shoair ba4d33337f fix: make browser fetchers type hints always present 2026-02-15 01:03:15 +02:00
Karim shoair 77e825e7be docs(spider): add a new page for proxy management & blocking 2026-02-15 01:01:43 +02:00
Karim shoair dee0fd00b8 docs: use more accurate naming for proxy rotation strategy 2026-02-14 23:21:08 +02:00
Karim shoair 0ab54852c4 fix: use more accurate naming for proxy rotation strategy 
It can be considered the simplest form of round robin since we don't have weights or anything (as was originally planned), but let's change it to avoid confusion or useless debates. Here goes nothing
 2026-02-14 23:19:54 +02:00
Karim shoair c94ebd2fa9 docs(spiders): add a page to explain sessions logic 2026-02-14 17:19:55 +02:00