freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 14738030e0 Merge pull request #10 from D4Vinci/dev Karim shoair 2024-11-20 01:48:17 +02:00
  • 649f4cfae7 Pumping up camoufox version Karim shoair 2024-11-20 01:41:46 +02:00
  • 526c516693 Pump version up to 0.2.3 Karim shoair 2024-11-20 01:28:01 +02:00
  • cb3ab4a011 Merge branch 'main' into dev Karim shoair 2024-11-19 20:48:05 +02:00
  • b69cc7c983 Fix for the stealth mode on playwright fetcher Karim shoair 2024-11-19 14:35:24 +02:00
  • f443d774c1 Fixing the doc-string to match Sphinx Karim shoair 2024-11-19 14:33:54 +02:00
  • 50cd40cc42 Merge pull request #9 from D4Vinci/dev Karim shoair 2024-11-16 22:04:03 +02:00
  • 0c6e7708d1 Update README.md Karim shoair 2024-11-16 21:55:53 +02:00
  • d8a8af8d05 Updating the README to reflect new changes Karim shoair 2024-11-16 21:29:05 +02:00
  • b60ac7f56a Change debug mode to be off by default Karim shoair 2024-11-16 21:11:58 +02:00
  • 422c135aae Adding a file import initialized fetchers for cleaner looking code Karim shoair 2024-11-16 21:10:26 +02:00
  • cefc95cc3d Pump version up to 0.2.2 Karim shoair 2024-11-16 13:57:33 +02:00
  • 572df6b3b4 Fixing the way Response object handles sub items in some edge cases Karim shoair 2024-11-16 13:54:55 +02:00
  • 105ec5b067 Update README.md Karim shoair 2024-11-15 23:18:02 +02:00
  • e2a64a0707 Adding SerpApi as a sponsor Karim shoair 2024-11-15 23:07:59 +02:00
  • 773fcd59bf Merge pull request #8 from D4Vinci/dev Karim shoair 2024-11-15 18:25:14 +02:00
  • 90e38af552 Correcting the return string for the Response object in all functions/methods Karim shoair 2024-11-15 16:44:47 +02:00
  • 5e8275c3eb Adding the option to randomize the OS fingerprints with the StealthyFetcher Karim shoair 2024-11-15 16:36:32 +02:00
  • 2abb70218f Adding the proxy support to browser-based Fetchers Karim shoair 2024-11-15 14:43:03 +02:00
  • b993adb4a7 Rephrasing some parts Karim shoair 2024-11-15 11:26:41 +02:00
  • ae7959b4a4 Update README.md Karim shoair 2024-11-15 02:50:24 +02:00
  • 7dd9d3090d Pumping the version up to 0.2.1 Karim shoair 2024-11-15 02:43:29 +02:00
  • c334755377 Reflecting the new changes in the README and adding small notes Karim shoair 2024-11-15 02:33:47 +02:00
  • 095d50cd75 Making the Response returned from all fetchers same as Adaptor object Karim shoair 2024-11-15 02:23:14 +02:00
  • b0c2b1818b Return to the default text behaviour Karim shoair 2024-11-14 22:21:45 +02:00
  • ea81e0284d Update tests Github action Karim shoair 2024-11-14 16:33:28 +02:00
  • 81f4e18a86 Merge remote-tracking branch 'origin/main' into dev Karim shoair 2024-11-14 16:30:41 +02:00
  • 9e337251ca Update README.md Karim shoair 2024-11-14 16:09:05 +02:00
  • e7e9cf8b3d Adding Evomi as a sponsor Karim shoair 2024-11-14 16:05:37 +02:00
  • b9c64e8dbf Update and rename README.md to Adding Evomi as a sponsor Karim shoair 2024-11-14 16:04:48 +02:00
  • eaa7da27c6 Better logic to handle json responses Karim shoair 2024-11-14 00:03:26 +02:00
  • d4b896b4e9 Parser - New logic to handle JSON responses passed from Fetchers Karim shoair 2024-11-13 23:14:27 +02:00
  • ab7b0d45ad StaticEngine - Fix the headers bug Karim shoair 2024-11-13 23:13:39 +02:00
  • 4e5f7d385a Merge pull request #5 from D4Vinci/main Karim shoair 2024-11-13 22:51:29 +02:00
  • 2b3c9b8ff3 Merge pull request #4 from marksmayo/main Karim shoair 2024-11-13 11:18:53 +02:00
  • b2993d2fcf Update and rename py.typeda to py.typed Mark Mayo 2024-11-13 21:56:02 +13:00
  • 067809974e Rename py.typed to py.typeda Mark Mayo 2024-11-13 21:55:51 +13:00
  • 4b5f9e12a2 Merge pull request #3 from D4Vinci/dev Karim shoair 2024-11-11 19:29:41 +02:00
  • 64c9fe905e Corrections and rephrasing Karim shoair 2024-11-11 16:40:54 +02:00
  • bd38fdb347 Corrections and rephrasing Karim shoair 2024-11-11 16:40:46 +02:00
  • b3418f7f50 Corrections and rephrasing Karim shoair 2024-11-11 16:40:35 +02:00
  • d6b454dc43 Corrections and rephrasing Karim shoair 2024-11-11 16:40:30 +02:00
  • eb100e45f7 Corrections and rephrasing Karim shoair 2024-11-11 16:40:21 +02:00
  • 79bf4060f4 Updating find_all examples Karim shoair 2024-11-11 12:51:40 +02:00
  • e8935b1af8 Update README.md Karim shoair 2024-11-10 23:45:48 +02:00
  • 3d4b427168 Rework Selectors generation Karim shoair 2024-11-10 18:19:48 +02:00
  • de2d65856c Adding playwright/camoufox tests for cookies and improve the timeout test Karim shoair 2024-11-10 16:31:28 +02:00
  • 239e2c73bf Correctly get page content after JS Execution Karim shoair 2024-11-10 16:11:46 +02:00
  • 43045e3eb2 Get page content after JS Execution Karim shoair 2024-11-10 15:54:44 +02:00
  • 7d2f55bd4c First version of the 0.2 README Karim shoair 2024-11-10 00:09:30 +02:00
  • 4b2814404a Fixing the functions-based filtering logic Karim shoair 2024-11-09 22:32:05 +02:00
  • 6b62579a7e Making find_all/find methods on steroids Karim shoair 2024-11-09 21:25:17 +02:00
  • b90c8adb02 Adding the argument automatch_domain to all fetchers Karim shoair 2024-11-09 15:38:27 +02:00
  • 03091beb8b Small clarification Karim shoair 2024-11-09 13:40:50 +02:00
  • fd7d787ab0 Add extra_headers argument to PW and isolate google referer feature from stealth mode as google_search Karim shoair 2024-11-09 00:54:11 +02:00
  • ee8f78feb1 Update camo.py Karim shoair 2024-11-09 00:48:11 +02:00
  • fd34cd9afa Adding arguments google_search and extra_headers to StealthyFetcher Karim shoair 2024-11-09 00:43:24 +02:00
  • 530b54d964 Clearer doc-string Karim shoair 2024-11-09 00:17:41 +02:00
  • 1b015bbd07 Type correction Karim shoair 2024-11-08 23:59:12 +02:00
  • b919536527 Updating the keywords whitelisting logic in find/find_all functions Karim shoair 2024-11-07 12:56:32 +02:00
  • 51ce40a561 Update ROADMAP.md Karim shoair 2024-11-07 00:44:23 +02:00
  • 91f6b46e6d Adding new filter and search functions to parser Karim shoair 2024-11-07 00:31:14 +02:00
  • 846a185b49 Updating parser tests to use find and css_first functions Karim shoair 2024-11-07 00:02:36 +02:00
  • b25d2ee79b Adding new find_all and find functions to parser Karim shoair 2024-11-07 00:01:14 +02:00
  • 6b40af426f Better logic for css_first and xpath_first Karim shoair 2024-11-06 22:14:33 +02:00
  • 5e848ef046 Adding new class type TextHandlers Karim shoair 2024-11-06 21:03:19 +02:00
  • 7a84511b7a Making Adaptor and Adaptor re/re_first arguments consistent with the TextHandler ones Karim shoair 2024-11-06 20:47:21 +02:00
  • 7af75f78a5 Better logic for Adaptors re_first method Karim shoair 2024-11-06 20:24:30 +02:00
  • a1fd5ddc7e Correcting headless argument type Karim shoair 2024-11-06 20:18:44 +02:00
  • a0a3e62075 Adding addons and humanize arguments to StealthyFetcher Karim shoair 2024-11-06 20:14:42 +02:00
  • 5ad2126030 Small speed boost for fetching by some caching Karim shoair 2024-11-06 19:56:39 +02:00
  • a1513ba769 Update tests.yml Karim shoair 2024-11-06 12:54:52 +02:00
  • b542b5cc39 Recaching TOX environments Karim shoair 2024-11-06 12:50:14 +02:00
  • c30853ee2c Update tox.ini Karim shoair 2024-11-06 12:39:38 +02:00
  • fc30cfed4c Update tests.yml Karim shoair 2024-11-06 12:36:46 +02:00
  • cd1ff846a7 Update ROADMAP.md Karim shoair 2024-11-06 12:31:57 +02:00
  • d632bb14af Moving parser tests to a better structure Karim shoair 2024-11-06 12:30:41 +02:00
  • 5548941cf1 Correcting return type for css_first and xpath_first Karim shoair 2024-11-06 12:28:43 +02:00
  • e4eb442709 Adding tests for PlayWrightFetcher class Karim shoair 2024-11-06 12:21:55 +02:00
  • 4222626ec1 Adding tests for Fetcher class Karim shoair 2024-11-06 12:21:18 +02:00
  • 4400363146 Renaming fetchers tests folder for readability Karim shoair 2024-11-06 12:21:06 +02:00
  • 9a7f489eb7 make tests verbose for easier debugging later Karim shoair 2024-11-06 12:20:05 +02:00
  • 10bcdfb596 Moving bypasses folder deeper Karim shoair 2024-11-06 12:19:26 +02:00
  • 6cf506cb22 PlaywrightEngine - Validate CDP URLs in all cases Karim shoair 2024-11-06 11:24:56 +02:00
  • 6497b1b559 Type correction Karim shoair 2024-11-06 02:20:33 +02:00
  • 24806aecdc Update .gitignore Karim shoair 2024-11-05 23:41:57 +02:00
  • 3622040440 Adding css_first and xpath_first for easier usage Karim shoair 2024-11-05 23:41:51 +02:00
  • 009c2914aa Fixing re_first logic in Adaptors type Karim shoair 2024-11-05 22:59:47 +02:00
  • d50497dff3 Update .gitignore Karim shoair 2024-11-05 22:56:57 +02:00
  • ca4b5f205c Since tests can finally now run in Github actions without issues Karim shoair 2024-11-05 13:10:51 +02:00
  • 2a275c436c Update tests.yml Karim shoair 2024-11-05 13:02:11 +02:00
  • 69cfbe45be Update tests.yml Karim shoair 2024-11-05 12:59:39 +02:00
  • 6f91d06b46 Update tests.yml Karim shoair 2024-11-05 12:57:57 +02:00
  • 2020214784 please work Karim shoair 2024-11-05 11:45:59 +02:00
  • 92d002032d Update tests.yml Karim shoair 2024-11-05 11:44:10 +02:00
  • 1bbb5dd75a Update tests.yml Karim shoair 2024-11-05 11:41:58 +02:00
  • d4358d32a8 Update tests.yml Karim shoair 2024-11-05 11:38:38 +02:00
  • a60662af4a Update camo.py Karim shoair 2024-11-05 11:38:35 +02:00
  • 8933a99a6e Update tests.yml Karim shoair 2024-11-05 11:29:28 +02:00
  • f200e9b290 looks like playwright need to run as normal user Karim shoair 2024-11-05 11:28:12 +02:00