freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(parser): Better approach for web pages where the encoding is not always correctly declared

Browse Source 
Fixes #110 and avoids defaulting to a specific encoding like #111
This commit is contained in:
Karim shoair
2025-11-22 20:24:55 +02:00
parent 48313307d0
commit 170205599d
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scrapling/parser.py
+1 -1
View File
@@ -121,7 +121,7 @@ class Selector(SelectorsGeneration):
self.__text = None self.__text = None
if root is None: if root is None:
if isinstance(content, str): if isinstance(content, str):
body = content.strip().replace("\x00", "").encode(encoding) or b"<html/>" body = content.strip().replace("\x00", "") or "<html/>"
elif isinstance(content, bytes): elif isinstance(content, bytes):
body = content.replace(b"\x00", b"") body = content.replace(b"\x00", b"")
else: else: