6652b8ac18
fix: solve the response log issue so now you get log for every response from Fetchers
2024-12-19 12:29:34 +02:00
838b8b1196
style: fixing a typo in logging
2024-12-16 13:39:30 +02:00
193827e27b
refactor(api)!: Unifying log under 1 logger and removing debug parameter
...
So now you control the logging and the debugging from the shell through the logger with the name 'scrapling'
2024-12-11 21:41:37 +02:00
dcf51876a5
fix: Adaptor.body returns raw HTML without processing
...
If possible, otherwise returns `Adaptor.html_content`
2024-12-11 20:18:05 +02:00
b4f90615c3
Adding keep_cdata argument for Adaptor and Response classes
...
This will force lxml to keep cdata while parsing html if you want
2024-12-10 22:06:08 +02:00
5f9c3980dc
Adding urljoin method to Adaptors and Responses
2024-12-10 22:02:43 +02:00
45e86f50a3
Adding empty methods (get/get_all/extract/extract_all)
...
For easy copy-paste from Scrapy/parsel code when needed :)
2024-12-04 14:28:37 +02:00
1a17b2ce46
Sorting all library imports with isort
2024-11-30 14:02:06 +02:00
572df6b3b4
Fixing the way Response object handles sub items in some edge cases
...
+ Do it with the least impact on performance
2024-11-16 13:54:55 +02:00
b0c2b1818b
Return to the default text behaviour
...
The logic to remove comment tags before returning text was causing errors sometimes so from now on leave `keep_comments` set to True better as it is.
2024-11-14 22:21:45 +02:00
eaa7da27c6
Better logic to handle json responses
2024-11-14 00:03:26 +02:00
d4b896b4e9
Parser - New logic to handle JSON responses passed from Fetchers
2024-11-13 23:14:27 +02:00
4b2814404a
Fixing the functions-based filtering logic
2024-11-09 22:32:05 +02:00
6b62579a7e
Making find_all/find methods on steroids
...
Adding the ability to find elements by regex patterns and functions.
2024-11-09 21:25:17 +02:00
b919536527
Updating the keywords whitelisting logic in find/find_all functions
2024-11-07 12:56:32 +02:00
91f6b46e6d
Adding new filter and search functions to parser
2024-11-07 00:31:14 +02:00
b25d2ee79b
Adding new find_all and find functions to parser
2024-11-07 00:01:14 +02:00
6b40af426f
Better logic for css_first and xpath_first
2024-11-06 22:14:33 +02:00
5e848ef046
Adding new class type TextHandlers
2024-11-06 21:03:19 +02:00
7a84511b7a
Making Adaptor and Adaptor re/re_first arguments consistent with the TextHandler ones
2024-11-06 20:47:21 +02:00
7af75f78a5
Better logic for Adaptors re_first method
2024-11-06 20:24:30 +02:00
5548941cf1
Correcting return type for css_first and xpath_first
2024-11-06 12:28:43 +02:00
3622040440
Adding css_first and xpath_first for easier usage
2024-11-05 23:41:51 +02:00
009c2914aa
Fixing re_first logic in Adaptors type
2024-11-05 22:59:47 +02:00
eaae5908c9
Make auto_match enabled by default
2024-11-03 13:33:24 +02:00
942aa08c85
Handling JSON responses better
2024-11-03 13:00:23 +02:00
145c03daff
Big structure changes (check commit description)
...
- Moved most of the parser functions/files to the core package.
- Converted tools file to a package and made separate files for similar functions.
- Now all fetcher engines return a Response object
- Instead of selecting an engine to use and passing config to it, we have separate fetcher classes so the user can choose what to use while importing.
- I added a new custom fetcher so the user can create and use an engine.
- More...
2024-11-03 01:04:02 +02:00
23c4181ff2
Using the new types file
2024-10-31 01:32:18 +03:00
d546b3499b
Handling an edge case and adding more commentary
2024-10-16 19:32:01 +03:00
a01cfe66db
Version 0.1.2
...
Check the changelog in releases
2024-10-16 18:21:56 +03:00
84b0ea6121
Handle importing SupportsIndex in Python 3.6 and 3.7
2024-10-14 00:04:36 +03:00
a721073b7c
Going online (first public version)
2024-10-13 23:38:48 +03:00
Karim shoair