Merge branch 'main' into upgrade-github-actions-node24-general
This commit is contained in:
@@ -37,12 +37,12 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0 # Full history for better analysis
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: '3.10'
|
||||
cache: 'pip'
|
||||
@@ -177,7 +177,7 @@ jobs:
|
||||
|
||||
- name: Upload Bandit report
|
||||
if: always() && steps.bandit.outcome != 'skipped'
|
||||
uses: actions/upload-artifact@v4
|
||||
uses: actions/upload-artifact@v6
|
||||
with:
|
||||
name: bandit-security-report
|
||||
path: bandit-report.json
|
||||
|
||||
@@ -25,7 +25,7 @@ jobs:
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@v4
|
||||
uses: actions/checkout@v6
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v3
|
||||
|
||||
@@ -18,7 +18,7 @@ jobs:
|
||||
contents: write
|
||||
id-token: write
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
@@ -27,7 +27,7 @@ jobs:
|
||||
run: echo "title=${{ github.event.pull_request.title }}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Save PR body to file
|
||||
uses: actions/github-script@v6
|
||||
uses: actions/github-script@v8
|
||||
with:
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
@@ -57,7 +57,7 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: 3.12
|
||||
|
||||
|
||||
@@ -44,10 +44,10 @@ jobs:
|
||||
TOXENV: py313
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: actions/setup-python@v5
|
||||
uses: actions/setup-python@v6
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: 'pip'
|
||||
@@ -69,7 +69,7 @@ jobs:
|
||||
|
||||
- name: Retrieve Playwright browsers from cache if any
|
||||
id: playwright-cache
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.cache/ms-playwright
|
||||
@@ -92,7 +92,7 @@ jobs:
|
||||
|
||||
# Cache tox environments
|
||||
- name: Cache tox environments
|
||||
uses: actions/cache@v4
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: .tox
|
||||
# Include python version and os in the cache key
|
||||
|
||||
@@ -1,34 +0,0 @@
|
||||
|
||||
If you have issues with the browser installation, such as resource management, we recommend you try the Cloud Browser from [Scrapeless](https://www.scrapeless.com/en/product/scraping-browser?utm_source=official&utm_term=scrapling) for free!
|
||||
|
||||
The usage is straightforward: create an account and [get your API key](https://docs.scrapeless.com/en/scraping-browser/quickstart/getting-started/?utm_source=official&utm_term=scrapling), then pass it to the `DynamicSession` like this:
|
||||
|
||||
```python
|
||||
from urllib.parse import urlencode
|
||||
|
||||
from scrapling.fetchers import DynamicSession
|
||||
|
||||
# Configure your browser session
|
||||
config = {
|
||||
"token": "YOUR_API_KEY",
|
||||
"sessionName": "scrapling-session",
|
||||
"sessionTTL": "300", # 5 minutes
|
||||
"proxyCountry": "ANY",
|
||||
"sessionRecording": "false",
|
||||
}
|
||||
|
||||
# Build WebSocket URL
|
||||
ws_endpoint = f"wss://browser.scrapeless.com/api/v2/browser?{urlencode(config)}"
|
||||
print('Connecting to Scrapeless...')
|
||||
|
||||
with DynamicSession(cdp_url=ws_endpoint, disable_resources=True) as s:
|
||||
print("Connected!")
|
||||
page = s.fetch("https://httpbin.org/headers", network_idle=True)
|
||||
print(f"Page loaded, content length: {len(page.body)}")
|
||||
print(page.json())
|
||||
```
|
||||
The `DynamicSession` class instance will work as usual, so no further explanation is needed.
|
||||
|
||||
However, the Scrapeless Cloud Browser can be configured with proxy options, like the proxy country in the config above, [custom fingerprint](https://docs.scrapeless.com/en/scraping-browser/features/advanced-privacy-anti-detection/custom-fingerprint/?utm_source=official&utm_term=scrapling) configuration, [captcha solving](https://docs.scrapeless.com/en/scraping-browser/features/advanced-privacy-anti-detection/supported-captchas/?utm_source=official&utm_term=scrapling), and more.
|
||||
|
||||
Check out the [Scrapeless's browser documentation](https://docs.scrapeless.com/en/scraping-browser/quickstart/introduction/?utm_source=official&utm_term=scrapling) for more details.
|
||||
+1
-2
@@ -50,8 +50,7 @@ nav = [
|
||||
]},
|
||||
{Tutorials = [
|
||||
{"A Free Alternative to AI for Robust Web Scraping" = "tutorials/replacing_ai.md"},
|
||||
{"Migrating from BeautifulSoup" = "tutorials/migrating_from_beautifulsoup.md"},
|
||||
{"Using Scrapeless browser" = "tutorials/external.md"}
|
||||
{"Migrating from BeautifulSoup" = "tutorials/migrating_from_beautifulsoup.md"}
|
||||
]},
|
||||
{Development = [
|
||||
{"API Reference" = [
|
||||
|
||||
Reference in New Issue
Block a user