diff --git a/docs/cli/interactive-shell.md b/docs/cli/interactive-shell.md index ab8d70d..e56939b 100644 --- a/docs/cli/interactive-shell.md +++ b/docs/cli/interactive-shell.md @@ -128,7 +128,7 @@ The shell provides a few functions to help you convert curl commands from the br - **Convert Curl command to Request Object** ```python - >>> curl_cmd = '''curl 'https://httpbin.org/post' \ + >>> curl_cmd = '''curl 'https://scrapling.requestcatcher.com/post' \ ... -X POST \ ... -H 'Content-Type: application/json' \ ... -d '{"name": "test", "value": 123}' ''' @@ -137,7 +137,7 @@ The shell provides a few functions to help you convert curl commands from the br >>> request.method 'post' >>> request.url - 'https://httpbin.org/post' + 'https://scrapling.requestcatcher.com/post' >>> request.headers {'Content-Type': 'application/json'} ``` diff --git a/docs/fetching/static.md b/docs/fetching/static.md index b338332..524e63e 100644 --- a/docs/fetching/static.md +++ b/docs/fetching/static.md @@ -48,8 +48,8 @@ Examples are the best way to explain this, as follows. >>> from scrapling.fetchers import Fetcher >>> # Basic GET >>> page = Fetcher.get('https://example.com') ->>> page = Fetcher.get('https://httpbin.org/get', stealthy_headers=True, follow_redirects=True) ->>> page = Fetcher.get('https://httpbin.org/get', proxy='http://username:password@localhost:8030') +>>> page = Fetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True, follow_redirects=True) +>>> page = Fetcher.get('https://scrapling.requestcatcher.com/get', proxy='http://username:password@localhost:8030') >>> # With parameters >>> page = Fetcher.get('https://example.com/search', params={'q': 'query'}) >>> @@ -67,8 +67,8 @@ And for asynchronous requests, it's a small adjustment >>> from scrapling.fetchers import AsyncFetcher >>> # Basic GET >>> page = await AsyncFetcher.get('https://example.com') ->>> page = await AsyncFetcher.get('https://httpbin.org/get', stealthy_headers=True, follow_redirects=True) ->>> page = await AsyncFetcher.get('https://httpbin.org/get', proxy='http://username:password@localhost:8030') +>>> page = await AsyncFetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True, follow_redirects=True) +>>> page = await AsyncFetcher.get('https://scrapling.requestcatcher.com/get', proxy='http://username:password@localhost:8030') >>> # With parameters >>> page = await AsyncFetcher.get('https://example.com/search', params={'q': 'query'}) >>> @@ -102,9 +102,9 @@ Needless to say, the `page` object in all cases is [Response](choosing.md#respon ```python >>> from scrapling.fetchers import Fetcher >>> # Basic POST ->>> page = Fetcher.post('https://httpbin.org/post', data={'key': 'value'}, params={'q': 'query'}) ->>> page = Fetcher.post('https://httpbin.org/post', data={'key': 'value'}, stealthy_headers=True, follow_redirects=True) ->>> page = Fetcher.post('https://httpbin.org/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030', impersonate="chrome") +>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, params={'q': 'query'}) +>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, stealthy_headers=True, follow_redirects=True) +>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030', impersonate="chrome") >>> # Another example of form-encoded data >>> page = Fetcher.post('https://example.com/submit', data={'username': 'user', 'password': 'pass'}, http3=True) >>> # JSON data @@ -114,9 +114,9 @@ And for asynchronous requests, it's a small adjustment ```python >>> from scrapling.fetchers import AsyncFetcher >>> # Basic POST ->>> page = await AsyncFetcher.post('https://httpbin.org/post', data={'key': 'value'}) ->>> page = await AsyncFetcher.post('https://httpbin.org/post', data={'key': 'value'}, stealthy_headers=True, follow_redirects=True) ->>> page = await AsyncFetcher.post('https://httpbin.org/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030', impersonate="chrome") +>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}) +>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, stealthy_headers=True, follow_redirects=True) +>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030', impersonate="chrome") >>> # Another example of form-encoded data >>> page = await AsyncFetcher.post('https://example.com/submit', data={'username': 'user', 'password': 'pass'}, http3=True) >>> # JSON data @@ -130,7 +130,7 @@ And for asynchronous requests, it's a small adjustment >>> page = Fetcher.put('https://example.com/update', data={'status': 'updated'}, stealthy_headers=True, follow_redirects=True, impersonate="chrome") >>> page = Fetcher.put('https://example.com/update', data={'status': 'updated'}, proxy='http://username:password@localhost:8030') >>> # Another example of form-encoded data ->>> page = Fetcher.put("https://httpbin.org/put", data={'key': ['value1', 'value2']}) +>>> page = Fetcher.put("https://scrapling.requestcatcher.com/put", data={'key': ['value1', 'value2']}) ``` And for asynchronous requests, it's a small adjustment ```python @@ -140,7 +140,7 @@ And for asynchronous requests, it's a small adjustment >>> page = await AsyncFetcher.put('https://example.com/update', data={'status': 'updated'}, stealthy_headers=True, follow_redirects=True, impersonate="chrome") >>> page = await AsyncFetcher.put('https://example.com/update', data={'status': 'updated'}, proxy='http://username:password@localhost:8030') >>> # Another example of form-encoded data ->>> page = await AsyncFetcher.put("https://httpbin.org/put", data={'key': ['value1', 'value2']}) +>>> page = await AsyncFetcher.put("https://scrapling.requestcatcher.com/put", data={'key': ['value1', 'value2']}) ``` #### DELETE @@ -176,8 +176,8 @@ with FetcherSession( retries=3 ) as session: # Make multiple requests with the same settings - page1 = session.get('https://httpbin.org/get') - page2 = session.post('https://httpbin.org/post', data={'key': 'value'}) + page1 = session.get('https://scrapling.requestcatcher.com/get') + page2 = session.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}) page3 = session.get('https://api.github.com/events') # All requests share the same session and connection pool @@ -189,9 +189,9 @@ And here's an async example async with FetcherSession(impersonate='firefox', http3=True) as session: # All standard HTTP methods available response = async session.get('https://example.com') - response = async session.post('https://httpbin.org/post', json={'data': 'value'}) - response = async session.put('https://httpbin.org/put', data={'update': 'info'}) - response = async session.delete('https://httpbin.org/delete') + response = async session.post('https://scrapling.requestcatcher.com/post', json={'data': 'value'}) + response = async session.put('https://scrapling.requestcatcher.com/put', data={'update': 'info'}) + response = async session.delete('https://scrapling.requestcatcher.com/delete') ``` or better ```python diff --git a/docs/overview.md b/docs/overview.md index 51e9cac..acde1b3 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -245,23 +245,23 @@ A fetcher is made for every use case. For simple HTTP requests, there's a `Fetcher` class that can be imported and used as below: ```python from scrapling.fetchers import Fetcher -page = Fetcher.get('https://httpbin.org/get', impersonate="chrome") +page = Fetcher.get('https://scrapling.requestcatcher.com/get', impersonate="chrome") ``` With that out of the way, here's how to do all HTTP methods: ```python >>> from scrapling.fetchers import Fetcher ->>> page = Fetcher.get('https://httpbin.org/get', stealthy_headers=True, follow_redirects=True) ->>> page = Fetcher.post('https://httpbin.org/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030') ->>> page = Fetcher.put('https://httpbin.org/put', data={'key': 'value'}) ->>> page = Fetcher.delete('https://httpbin.org/delete') +>>> page = Fetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True, follow_redirects=True) +>>> page = Fetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030') +>>> page = Fetcher.put('https://scrapling.requestcatcher.com/put', data={'key': 'value'}) +>>> page = Fetcher.delete('https://scrapling.requestcatcher.com/delete') ``` For Async requests, you will replace the import like below: ```python >>> from scrapling.fetchers import AsyncFetcher ->>> page = await AsyncFetcher.get('https://httpbin.org/get', stealthy_headers=True, follow_redirects=True) ->>> page = await AsyncFetcher.post('https://httpbin.org/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030') ->>> page = await AsyncFetcher.put('https://httpbin.org/put', data={'key': 'value'}) ->>> page = await AsyncFetcher.delete('https://httpbin.org/delete') +>>> page = await AsyncFetcher.get('https://scrapling.requestcatcher.com/get', stealthy_headers=True, follow_redirects=True) +>>> page = await AsyncFetcher.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'}, proxy='http://username:password@localhost:8030') +>>> page = await AsyncFetcher.put('https://scrapling.requestcatcher.com/put', data={'key': 'value'}) +>>> page = await AsyncFetcher.delete('https://scrapling.requestcatcher.com/delete') ``` > Notes: