Commit Graph

59 Commits

Author SHA1 Message Date
Karim shoair 614d136f8c build: pump up version and deps 2026-04-15 20:44:38 +02:00
Karim shoair 887eeee4c2 build: pump up version and deps 2026-04-12 18:47:13 +02:00
Karim shoair d0a19a6fe1 build: pump version up 2026-04-05 19:41:10 +02:00
Karim shoair 911d3af632 build: pumping up protego minimum version 2026-04-05 02:33:15 +02:00
Abdullah 07129ce4b1 feat(deps): move protego to fetchers optional dependency
protego is only used by the spider framework for robots.txt compliance.

Moving it from core dependencies to the optional 'fetchers' group reduces
the dependency footprint for users who don't need the spider framework.

for pyproject.toml file
2026-04-03 17:51:00 +02:00
Abdullah a2bf00c995 Merge branch 'D4Vinci:dev' into dev 2026-04-03 17:38:28 +02:00
Karim shoair 9dcfe47da7 build: pump up deps 2026-04-03 17:18:15 +02:00
Abdullah 1d15349e07 feat(deps): add protego for robots.txt parsing and fix pyright type error in static.py 2026-04-03 15:08:33 +02:00
Karim shoair 966e17a1dc build: pump version up 2026-04-01 17:38:54 +02:00
Karim shoair 422b4713ec build: pump up version 2026-03-25 00:19:58 +02:00
Karim shoair 27409927a3 build: pump up deps and browsers versions 2026-03-08 23:57:10 +02:00
Karim shoair 37f5c48a5f build: pump up the version
Co-Authored-By: Christian Rost <chr@baltic-online.de>
2026-03-03 23:09:36 +02:00
Karim shoair a8359a4878 build: pump version up and the deps 2026-02-27 00:32:25 +02:00
Karim shoair 1bbb18ad8e build: fix readme path for pypi 2026-02-15 07:16:28 +02:00
Karim shoair 6ac40ac3af build: adjustments to build file for PyPi 2026-02-11 00:29:20 +02:00
Karim shoair 0489923b06 build: pump up the version status 2026-02-10 21:53:00 +02:00
Karim shoair b1ce0b414b ops: Add mypy and pyright to the code quality workflow 2026-02-07 17:28:44 +02:00
Karim shoair e777342741 build: pump up version and deps 2026-02-07 16:30:32 +02:00
Karim shoair 90c52c45c7 feat(parser): replacing tldextract with tld library
This might break the adaptive data users have for websites BUT:
1. tld uses ~3.7x less memory during extraction operations (1.5 MB vs 5.7 MB).
2. tld uses ~56% less memory on import (5.2 MB vs 11.9 MB).
3. Zero dependencies (vs 3 for tldextract).

In return, it's 30ms slower for extracting 5000 URLs, which is negligible. Also, the type hints aren't always accurate, but it's fine; I corrected them.
2026-01-23 00:00:48 +02:00
Karim shoair 95955af3be build: Add w3lib to deps 2026-01-20 16:36:34 +02:00
Karim shoair 32f4f2adf4 build: Add AnyIO to the dependencies 2026-01-11 17:01:33 +02:00
Karim shoair 3a360acced build: pump version up temporary 2026-01-08 01:37:47 +02:00
Karim shoair 30a77de2be build: pump version up and browserforge version 2026-01-03 01:48:34 +02:00
Karim shoair ee13af9516 build: update deps
and docstrings
2025-12-29 16:08:51 +02:00
Karim shoair ccf97803d5 build: update deps 2025-12-26 03:21:40 +02:00
Karim shoair b40c16a2cd build: pump version up 2025-12-24 21:35:41 +02:00
Karim shoair e53f1cb363 build: forcing playwright version
Until the Python driver for patchright 1.57 releases
2025-12-18 01:45:21 +02:00
Karim shoair fd9fc83c6c build: pump version up and deps 2025-12-17 00:04:24 +02:00
Karim shoair 16ff07a287 build: Pump version up and mcp 2025-12-03 02:20:18 +02:00
Karim shoair abc9aaebfe build: Pump deps up
msgspec now supports py14
2025-11-24 13:03:18 +02:00
Karim shoair 2bae3261a0 build: Pumping version and deps up
This also updates the Maxmind database for camoufox
2025-11-22 21:41:47 +02:00
Karim shoair ceda6eb0c1 docs: Add AI translations to other languages 2025-11-14 02:56:00 +02:00
Karim shoair 42a8b43207 build: Pump version up 2025-11-08 20:07:26 +02:00
Karim shoair f9a514cc64 build: Pump up deps 2025-10-26 05:04:53 +03:00
Karim shoair a54ceb3335 build: Pumping up the deps and the version 2025-10-19 15:38:46 +03:00
Karim shoair f4ce433ff3 build: pump up the version and deps 2025-10-05 04:08:19 +03:00
Karim shoair 0cf3bcf3aa ops: improve the layering of the docker image 2025-09-29 02:53:06 +03:00
Karim shoair 840aef6884 build: Pump up version and deps 2025-09-28 22:04:09 +03:00
Karim shoair 5164b801ec build: pump version up and update deps 2025-09-20 00:33:07 +03:00
Karim shoair 67db865aee feat(DynamicFetcher): Replace rebrowser with patchright 2025-09-20 00:23:29 +03:00
Karim shoair fdac239fcc build: Move click library to be part of the fetchers extra 2025-09-14 20:32:46 +03:00
Karim shoair 13d7e70cb7 refactor: Make all fetchers as an optional dependency group
+ Removing some dead code
2025-09-13 16:07:48 +03:00
Karim shoair d7e3deae2a fix: fix invalid dep version 2025-09-13 05:10:56 +03:00
Karim shoair 521366e77e build: bump up deps 2025-09-13 04:29:09 +03:00
Karim shoair 6b8d861b92 build: Move non-primarily deps to optional extras 2025-08-20 03:58:10 +03:00
Karim shoair 57a025a34f build: update dependencies and fix geoip issue 2025-08-20 01:10:17 +03:00
Karim shoair 9d3b1335a9 build: update deps 2025-07-26 19:00:38 +03:00
Karim shoair 4bba8598c2 build: update deps 2025-07-09 16:20:21 +03:00
Karim shoair 403f7bfe72 build: Updating PyPI classifiers 2025-07-07 00:10:20 +03:00
Karim shoair 939b78f3cf build: adding mcp as dependency 2025-07-07 00:09:54 +03:00