Commit Graph

30 Commits

Author SHA1 Message Date
Karim shoair a403156b2e docs: style adjustment 2026-03-30 03:53:56 +02:00
Karim shoair bf9e2da329 Merge branch 'dev' into test/edge-cases-filter-ancestors-find-similar 2026-03-17 22:15:59 +02:00
Karim shoair 136c389787 fix(Texthandler): Replace get_all with getall to match the Selector class 2026-03-17 21:53:17 +02:00
awanawona cec5acd68a test: add edge case tests for filter, iterancestors, and find_similar
- test_selectors_filter.py: covers chained filter(), empty result,
  all-pass predicate, and calling filter() on empty Selectors
- test_ancestor_navigation.py: covers iterancestors() order, depth,
  text-node safety, root-element edge case, and find_ancestor() with
  no match
- test_find_similar_advanced.py: covers similarity_threshold levels,
  match_text=True behavior, ignore_attributes combinations, and
  text-node safety
2026-03-16 18:01:11 +08:00
haosenwang1018 7178279586 test: add coverage for TextHandler regex, clean, and TextHandlers.re()
Several critical code paths in custom_types.py lacked test coverage:

- TextHandler.re(check_match=True): returns bool, not TextHandlers
- TextHandler.re(replace_entities=False): entity preservation path
- TextHandler.re() with capture groups: flatten behavior
- TextHandler.re_first() default value when no match
- TextHandler.clean(remove_entities=True): entity replacement path
- TextHandler.json() valid and invalid input
- TextHandlers.re(): list-level regex with result flattening
- TextHandlers.extract()/get_all(): identity return

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 17:27:55 +08:00
Matt Hillebrand 7cbe566acd Move test for get_all_text to test_parser_advanced.py 2026-03-08 08:20:30 -07:00
Matt Hillebrand 00ad6b4c5f Add two ignored elements to unit test 2026-03-07 17:11:39 -08:00
Matt Hillebrand 5457a697fa fix: Selector.get_all_text() doesn't get all text #167
- Updated text extraction using recursion
- Added new unit test
2026-03-07 17:11:39 -08:00
Karim shoair 2997d562d5 test: remove irrelevent test 2026-02-14 03:34:41 +02:00
Karim shoair 9acc8194c5 test: update the tests accordingly 2026-02-06 02:45:13 +02:00
Karim shoair d0f52c3855 test: remove the tests for the removed html_utils module 2026-01-20 16:37:17 +02:00
Karim shoair eae088a07b tests: Multiple changes to tests
- Remove all tests for the old encoding logic
- Stop using the `nopecha` test page to test Cloudflare solver
- Remove useless tests like testing for infinite timeout
- Fixes to make the code compatible with new changes
2025-09-14 20:24:46 +03:00
Karim shoair cf57af588e docs: update doc strings with correct naming 2025-08-25 06:08:15 +03:00
Karim shoair 5aea62256b test: adding new tests and updating existing ones
Code Coverage is now 92%
2025-08-17 01:03:19 +03:00
Karim shoair 3e0ca33111 test: adding new tests and updating existing ones 2025-08-16 19:07:08 +03:00
Karim shoair e6e5b3cc80 test: adding new tests and updating existing ones
The coverage is now 78%
2025-08-16 14:58:11 +03:00
Karim shoair b9c7a5af2e refactor: replace's Selector inpt (text/body) with 1 argument called content 2025-07-29 06:08:15 +03:00
Karim shoair 264ae02aa7 refactor: huge change, many features/class got a better naming
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
Karim shoair 0c649987f8 test: remove irrelevant test case 2025-07-28 15:10:23 +03:00
Karim shoair cd3321a6f0 fix(tests): fix for GitHub actions 2025-05-08 03:11:01 +03:00
Karim shoair 0c8dd63f87 chore: migrating to ruff and updating pre-commit hooks 2025-04-13 17:32:00 +02:00
Karim shoair 20ef45349a test: Rewrite automatch tests to a cleaner version and adding async test 2024-12-16 13:18:19 +02:00
Karim shoair 69e31612a5 test: Rewrite parser tests to a cleaner version and adding more tests 2024-12-16 13:17:09 +02:00
Karim shoair 193827e27b refactor(api)!: Unifying log under 1 logger and removing debug parameter
So now you control the logging and the debugging from the shell through the logger with the name 'scrapling'
2024-12-11 21:41:37 +02:00
Karim shoair 1a17b2ce46 Sorting all library imports with isort 2024-11-30 14:02:06 +02:00
Karim shoair 0e01c1cc03 Update test_general.py 2024-11-23 16:24:25 +02:00
Karim shoair e7f2a83e16 Increasing the time a little bit for the performance test
It's prolonged on GitHub Actions for obvious reasons
2024-11-23 16:12:00 +02:00
Karim shoair 3d4b427168 Rework Selectors generation
- Renamed both `css_selector` and `xpath_selector` both to `generate_css_selector` and `generate_xpath_selector` for clarity and not interrupting the auto-completion while coding.
- Added `generate_full_css_selector` and `generate_full_xpath_selector` methods
2024-11-10 18:19:48 +02:00
Karim shoair 846a185b49 Updating parser tests to use find and css_first functions 2024-11-07 00:02:36 +02:00
Karim shoair d632bb14af Moving parser tests to a better structure
and covering a bit more parts
2024-11-06 12:30:41 +02:00