freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • eceef48ef5 fix(Fetchers/page_action): Fixing logic Karim shoair 2024-12-16 12:34:23 +02:00
  • 3ecffcb81c style: Rewrite sync Fetchers tests to a cleaner version Karim shoair 2024-12-16 01:27:18 +02:00
  • 2606f7a12d test: clearer naming for async tests Karim shoair 2024-12-16 01:01:59 +02:00
  • 448587ff41 test: add tests for StealthyFetcher async fetch Karim shoair 2024-12-16 01:00:05 +02:00
  • 79a911e24e feat(StealthyFetcher): Add async fetch support Karim shoair 2024-12-16 00:59:19 +02:00
  • efb3270c67 style: Rewrite asyncFetcher tests to a cleaner version Karim shoair 2024-12-16 00:47:02 +02:00
  • 361ee440aa test: adding pytest-asyncio plugin to tests requirements file Karim shoair 2024-12-16 00:20:26 +02:00
  • de015f2b18 chore: Restructuring fetchers into cleaner structure Karim shoair 2024-12-16 00:19:48 +02:00
  • aac77e488c test: add test for PlaywrightFetcher Async support Karim shoair 2024-12-16 00:17:13 +02:00
  • af4f2c0f74 feat(PlaywrightFetcher): Add async support for PlaywrightFetcher Karim shoair 2024-12-16 00:15:47 +02:00
  • 889c111855 refactor(Playwright Engine): Separate what we can for cleaner code and the async function later Karim shoair 2024-12-15 23:25:02 +02:00
  • 6c17bd886e style: using better data structures for constants Karim shoair 2024-12-15 23:19:47 +02:00
  • a90192df03 build: adding tests for AsyncFetcher class Karim shoair 2024-12-15 16:54:52 +02:00
  • b10cfd3643 feat: Adding AsyncFetcher class version of Fetcher Karim shoair 2024-12-15 16:51:58 +02:00
  • faf728a794 style: moving repeated arguments from inside the functions to __init__ Karim shoair 2024-12-15 16:49:48 +02:00
  • 445af3c702 perf: Give repeated usage of Fetcher a slight performance increase Karim shoair 2024-12-15 16:13:08 +02:00
  • 299793af3f feat: adding the retries argument for all methods of Fetcher class Karim shoair 2024-12-15 15:59:15 +02:00
  • 838dd62ee4 build: pumping up camoufox Karim shoair 2024-12-13 20:43:53 +02:00
  • f9bee4cbbb build: Pumping up camoufox version to solve browserforge issue Karim shoair 2024-12-12 14:37:31 +02:00
  • bfe906315e feat: adding geoip parameter to the StealthyFetcher Karim shoair 2024-12-12 12:43:49 +02:00
  • 6f87420e35 build: disable the 501 error test for playwright Karim shoair 2024-12-11 21:44:03 +02:00
  • e254341974 fix: forgot to stage it with last commit Karim shoair 2024-12-11 21:43:36 +02:00
  • 193827e27b refactor(api)!: Unifying log under 1 logger and removing debug parameter Karim shoair 2024-12-11 21:41:37 +02:00
  • f30eb6ab6c build: disable the 404 error test for playwright Karim shoair 2024-12-11 20:27:07 +02:00
  • dcf51876a5 fix: Adaptor.body returns raw HTML without processing Karim shoair 2024-12-11 20:18:05 +02:00
  • 9f0001af26 feat: logging for response status Karim shoair 2024-12-11 20:14:32 +02:00
  • e332444411 Update tests.yml Karim shoair 2024-12-10 23:35:21 +02:00
  • 70b54241e4 Update tox.ini Karim shoair 2024-12-10 23:32:54 +02:00
  • ba11c433ff Preparing to release 0.2.9 soon Karim shoair 2024-12-10 22:56:51 +02:00
  • e60a57c63f Python 3.8 is not supported anymore Karim shoair 2024-12-10 22:37:37 +02:00
  • 93ed768312 Bumping up the libraries versions for better stealth and speed! Karim shoair 2024-12-10 22:22:34 +02:00
  • b4f90615c3 Adding keep_cdata argument for Adaptor and Response classes Karim shoair 2024-12-10 22:06:08 +02:00
  • 5f9c3980dc Adding urljoin method to Adaptors and Responses Karim shoair 2024-12-10 22:02:43 +02:00
  • d0f18951ac fix: Enable WebGL by default Karim shoair 2024-12-04 23:38:30 +02:00
  • 45e86f50a3 Adding empty methods (get/get_all/extract/extract_all) Karim shoair 2024-12-04 14:28:37 +02:00
  • 31e838ce9b Make standard string methods return TextHandler again instead of str Karim shoair 2024-12-04 14:27:36 +02:00
  • 012820c635 Merge pull request #21 from D4Vinci/dev Karim shoair 2024-11-30 18:16:06 +02:00
  • c481a1c8ad Pumping version up to 0.2.8 Karim shoair 2024-11-30 18:13:19 +02:00
  • a3a4f1ff19 A cleanup script to use with development Karim shoair 2024-11-30 18:01:35 +02:00
  • f7916e504e Update tox.ini Karim shoair 2024-11-30 14:32:42 +02:00
  • 8321cfc180 Add vermin hook to check for minimum python version Karim shoair 2024-11-30 14:28:10 +02:00
  • b38c38b03b Forcing Tox to always use pytest config file Karim shoair 2024-11-30 14:14:59 +02:00
  • f81e3b3cc3 Turn off threaded testing for Python 3.8 on tox Karim shoair 2024-11-30 14:08:18 +02:00
  • 1a17b2ce46 Sorting all library imports with isort Karim shoair 2024-11-30 14:02:06 +02:00
  • 26bfa88e50 Turn off threaded testing for Python 3.8 on tox Karim shoair 2024-11-30 14:01:30 +02:00
  • 4dddebbe24 Update .bandit.yml Karim shoair 2024-11-30 13:47:46 +02:00
  • 7608d86bf2 Pumping up camoufox version Karim shoair 2024-11-30 11:24:47 +02:00
  • 82f8e4a073 Fixing import typo, thanks to #20 Karim shoair 2024-11-30 11:23:07 +02:00
  • 1cf7a1d205 Make Github tests workflow work on main and dev branches only Karim shoair 2024-11-29 21:19:39 +02:00
  • 97fecc9d5f Style adjustments by flake8 Karim shoair 2024-11-29 21:14:52 +02:00
  • d6cc07deaa Add scrapless banner Karim shoair 2024-11-29 11:26:48 +02:00
  • f8ad18ab8a Rephrasing some parts to be clearer Karim shoair 2024-11-27 11:50:20 +02:00
  • 26aebbace2 Merge pull request #18 from D4Vinci/dev Karim shoair 2024-11-26 23:10:49 +02:00
  • 06a47f9644 Small fix for the ResponseEncoding class Karim shoair 2024-11-26 22:55:22 +02:00
  • 28b783e86b Improving the stealth mode a bit for Playwright Karim shoair 2024-11-26 22:41:04 +02:00
  • 14ffdbd6fc Handling problematic encodings Karim shoair 2024-11-26 22:27:05 +02:00
  • d3fbcab3df Pumping version up to 0.2.7 Karim shoair 2024-11-26 21:27:22 +02:00
  • ccf38606d8 Making the README clearer a bit Karim shoair 2024-11-26 21:25:16 +02:00
  • dbd674aa7c Adding proxy support for basic requests Karim shoair 2024-11-26 21:24:41 +02:00
  • 1cf7cee1a2 Fetchers - Wait for page to be stable again if the user wanted to wait for a selector Karim shoair 2024-11-26 20:26:54 +02:00
  • d6a59774ea StealthyFetcher - The option to disable ads with ublock origin addon Karim shoair 2024-11-26 20:23:02 +02:00
  • 60d551d839 PlaywrightFetcher - Add the option to set locale of the browser Karim shoair 2024-11-26 17:39:03 +02:00
  • 0176bc0523 PlaywrightFetcher - Make proxies available without enabling stealth Karim shoair 2024-11-26 16:48:35 +02:00
  • 468d9b8f63 Update README.md Karim shoair 2024-11-25 12:28:06 +02:00
  • 1ab0f74cce Updating SerpApi Logo Karim shoair 2024-11-25 12:26:19 +02:00
  • 109933c04a Merge pull request #17 from D4Vinci/dev Karim shoair 2024-11-25 11:47:38 +02:00
  • b3817c5b3a Update SerpApi.png Karim shoair 2024-11-25 11:46:13 +02:00
  • cdc7ed8fa8 Removing CapSolver Karim shoair 2024-11-25 11:26:42 +02:00
  • bbbc97aef9 Merge pull request #16 from D4Vinci/dev Karim shoair 2024-11-24 15:36:28 +02:00
  • 438867e0be Pumping the version up to 0.2.6 Karim shoair 2024-11-24 15:21:58 +02:00
  • 12d638ac2e Updating the doc to reflect new changes Karim shoair 2024-11-24 15:20:18 +02:00
  • c1429074a7 Setting the compatible version of playwright with cdp patches Karim shoair 2024-11-24 15:15:58 +02:00
  • a8c576ad28 Adding the real_chrome option to the fetcher Karim shoair 2024-11-24 15:15:08 +02:00
  • e96e7d4440 Adding the option to use real chrome browser directly Karim shoair 2024-11-24 15:14:42 +02:00
  • 088117441d Making the type hints more accurate before update Karim shoair 2024-11-24 13:49:31 +02:00
  • 15b7e3acee Stop using generated headers by browserforge Karim shoair 2024-11-24 13:47:21 +02:00
  • 7f330efc52 Turn-off the hide_canvas option by default Karim shoair 2024-11-24 13:44:16 +02:00
  • 58151c12e8 Update the minimum browser version for headers generated Karim shoair 2024-11-24 13:41:06 +02:00
  • e94c5034e3 Merge pull request #15 from D4Vinci/docs Karim shoair 2024-11-23 17:52:12 +02:00
  • 1929d9bebb Fix for docs Karim shoair 2024-11-23 17:48:05 +02:00
  • e51556f80c Merge branch 'main' into docs Karim shoair 2024-11-23 16:48:22 +02:00
  • 4cea8c9926 Merge pull request #14 from D4Vinci/dev Karim shoair 2024-11-23 16:39:26 +02:00
  • 0e01c1cc03 Update test_general.py Karim shoair 2024-11-23 16:24:25 +02:00
  • 34b645c9a5 Fix for playwright tests on GitHub actions Karim shoair 2024-11-23 16:19:50 +02:00
  • e7f2a83e16 Increasing the time a little bit for the performance test Karim shoair 2024-11-23 16:12:00 +02:00
  • 2b925b4a33 Pumping version up to 0.2.5 Karim shoair 2024-11-23 16:06:11 +02:00
  • 454b79e00f Make tests run in parallel on GitHub actions Karim shoair 2024-11-23 16:04:47 +02:00
  • 723a545061 Fixing the "Unknown Status Code" test Karim shoair 2024-11-23 16:01:51 +02:00
  • 7017823136 Covering more parts in Playwright tests Karim shoair 2024-11-23 15:57:59 +02:00
  • 4d9a494e35 Covering more parts in StealthyFetcher tests Karim shoair 2024-11-23 15:55:50 +02:00
  • f33ff247dd Adding tests for response encoding and response status text Karim shoair 2024-11-23 15:55:21 +02:00
  • 073ab2c7e3 Universally handling different response encoding and charsets for all responses Karim shoair 2024-11-23 15:40:41 +02:00
  • 869412761a Fix for the wait_selector argument resolving to more than 1 element Karim shoair 2024-11-22 22:15:52 +02:00
  • 18eb96adb1 Merge pull request #13 from andyfcx/main Karim shoair 2024-11-21 11:21:45 +02:00
  • 63112bfc50 fix: make sure to default utf-8 as encoding and strip unwanted single or double quotes andyfcx 2024-11-21 14:54:38 +08:00
  • 3f39bd52d4 Merge branch 'main' into docs Karim shoair 2024-11-20 14:36:30 +02:00
  • e9b0102d55 Merge pull request #11 from D4Vinci/dev Karim shoair 2024-11-20 13:35:06 +02:00
  • 17d7934484 Pump version to 0.2.4 Karim shoair 2024-11-20 13:27:39 +02:00
  • 19ad82caef Calculate status text manually if it's not returned by PlayWright API Karim shoair 2024-11-20 13:27:18 +02:00
  • 4c74d9bc97 Fixing a bug with reading response bytes in playwright/camoufox when network_idle is used Karim shoair 2024-11-20 12:33:54 +02:00