Merge branch 'main' into upgrade-github-actions-node24-general
This commit is contained in:
@@ -37,12 +37,12 @@ jobs:
|
|||||||
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout code
|
- name: Checkout code
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v6
|
||||||
with:
|
with:
|
||||||
fetch-depth: 0 # Full history for better analysis
|
fetch-depth: 0 # Full history for better analysis
|
||||||
|
|
||||||
- name: Set up Python
|
- name: Set up Python
|
||||||
uses: actions/setup-python@v5
|
uses: actions/setup-python@v6
|
||||||
with:
|
with:
|
||||||
python-version: '3.10'
|
python-version: '3.10'
|
||||||
cache: 'pip'
|
cache: 'pip'
|
||||||
@@ -177,7 +177,7 @@ jobs:
|
|||||||
|
|
||||||
- name: Upload Bandit report
|
- name: Upload Bandit report
|
||||||
if: always() && steps.bandit.outcome != 'skipped'
|
if: always() && steps.bandit.outcome != 'skipped'
|
||||||
uses: actions/upload-artifact@v4
|
uses: actions/upload-artifact@v6
|
||||||
with:
|
with:
|
||||||
name: bandit-security-report
|
name: bandit-security-report
|
||||||
path: bandit-report.json
|
path: bandit-report.json
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ jobs:
|
|||||||
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v6
|
||||||
|
|
||||||
- name: Set up Docker Buildx
|
- name: Set up Docker Buildx
|
||||||
uses: docker/setup-buildx-action@v3
|
uses: docker/setup-buildx-action@v3
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ jobs:
|
|||||||
contents: write
|
contents: write
|
||||||
id-token: write
|
id-token: write
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v6
|
||||||
with:
|
with:
|
||||||
fetch-depth: 0
|
fetch-depth: 0
|
||||||
|
|
||||||
@@ -27,7 +27,7 @@ jobs:
|
|||||||
run: echo "title=${{ github.event.pull_request.title }}" >> $GITHUB_OUTPUT
|
run: echo "title=${{ github.event.pull_request.title }}" >> $GITHUB_OUTPUT
|
||||||
|
|
||||||
- name: Save PR body to file
|
- name: Save PR body to file
|
||||||
uses: actions/github-script@v6
|
uses: actions/github-script@v8
|
||||||
with:
|
with:
|
||||||
script: |
|
script: |
|
||||||
const fs = require('fs');
|
const fs = require('fs');
|
||||||
@@ -57,7 +57,7 @@ jobs:
|
|||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
- name: Set up Python
|
- name: Set up Python
|
||||||
uses: actions/setup-python@v5
|
uses: actions/setup-python@v6
|
||||||
with:
|
with:
|
||||||
python-version: 3.12
|
python-version: 3.12
|
||||||
|
|
||||||
|
|||||||
@@ -44,10 +44,10 @@ jobs:
|
|||||||
TOXENV: py313
|
TOXENV: py313
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v6
|
||||||
|
|
||||||
- name: Set up Python ${{ matrix.python-version }}
|
- name: Set up Python ${{ matrix.python-version }}
|
||||||
uses: actions/setup-python@v5
|
uses: actions/setup-python@v6
|
||||||
with:
|
with:
|
||||||
python-version: ${{ matrix.python-version }}
|
python-version: ${{ matrix.python-version }}
|
||||||
cache: 'pip'
|
cache: 'pip'
|
||||||
@@ -69,7 +69,7 @@ jobs:
|
|||||||
|
|
||||||
- name: Retrieve Playwright browsers from cache if any
|
- name: Retrieve Playwright browsers from cache if any
|
||||||
id: playwright-cache
|
id: playwright-cache
|
||||||
uses: actions/cache@v4
|
uses: actions/cache@v5
|
||||||
with:
|
with:
|
||||||
path: |
|
path: |
|
||||||
~/.cache/ms-playwright
|
~/.cache/ms-playwright
|
||||||
@@ -92,7 +92,7 @@ jobs:
|
|||||||
|
|
||||||
# Cache tox environments
|
# Cache tox environments
|
||||||
- name: Cache tox environments
|
- name: Cache tox environments
|
||||||
uses: actions/cache@v4
|
uses: actions/cache@v5
|
||||||
with:
|
with:
|
||||||
path: .tox
|
path: .tox
|
||||||
# Include python version and os in the cache key
|
# Include python version and os in the cache key
|
||||||
|
|||||||
@@ -1,34 +0,0 @@
|
|||||||
|
|
||||||
If you have issues with the browser installation, such as resource management, we recommend you try the Cloud Browser from [Scrapeless](https://www.scrapeless.com/en/product/scraping-browser?utm_source=official&utm_term=scrapling) for free!
|
|
||||||
|
|
||||||
The usage is straightforward: create an account and [get your API key](https://docs.scrapeless.com/en/scraping-browser/quickstart/getting-started/?utm_source=official&utm_term=scrapling), then pass it to the `DynamicSession` like this:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from urllib.parse import urlencode
|
|
||||||
|
|
||||||
from scrapling.fetchers import DynamicSession
|
|
||||||
|
|
||||||
# Configure your browser session
|
|
||||||
config = {
|
|
||||||
"token": "YOUR_API_KEY",
|
|
||||||
"sessionName": "scrapling-session",
|
|
||||||
"sessionTTL": "300", # 5 minutes
|
|
||||||
"proxyCountry": "ANY",
|
|
||||||
"sessionRecording": "false",
|
|
||||||
}
|
|
||||||
|
|
||||||
# Build WebSocket URL
|
|
||||||
ws_endpoint = f"wss://browser.scrapeless.com/api/v2/browser?{urlencode(config)}"
|
|
||||||
print('Connecting to Scrapeless...')
|
|
||||||
|
|
||||||
with DynamicSession(cdp_url=ws_endpoint, disable_resources=True) as s:
|
|
||||||
print("Connected!")
|
|
||||||
page = s.fetch("https://httpbin.org/headers", network_idle=True)
|
|
||||||
print(f"Page loaded, content length: {len(page.body)}")
|
|
||||||
print(page.json())
|
|
||||||
```
|
|
||||||
The `DynamicSession` class instance will work as usual, so no further explanation is needed.
|
|
||||||
|
|
||||||
However, the Scrapeless Cloud Browser can be configured with proxy options, like the proxy country in the config above, [custom fingerprint](https://docs.scrapeless.com/en/scraping-browser/features/advanced-privacy-anti-detection/custom-fingerprint/?utm_source=official&utm_term=scrapling) configuration, [captcha solving](https://docs.scrapeless.com/en/scraping-browser/features/advanced-privacy-anti-detection/supported-captchas/?utm_source=official&utm_term=scrapling), and more.
|
|
||||||
|
|
||||||
Check out the [Scrapeless's browser documentation](https://docs.scrapeless.com/en/scraping-browser/quickstart/introduction/?utm_source=official&utm_term=scrapling) for more details.
|
|
||||||
+1
-2
@@ -50,8 +50,7 @@ nav = [
|
|||||||
]},
|
]},
|
||||||
{Tutorials = [
|
{Tutorials = [
|
||||||
{"A Free Alternative to AI for Robust Web Scraping" = "tutorials/replacing_ai.md"},
|
{"A Free Alternative to AI for Robust Web Scraping" = "tutorials/replacing_ai.md"},
|
||||||
{"Migrating from BeautifulSoup" = "tutorials/migrating_from_beautifulsoup.md"},
|
{"Migrating from BeautifulSoup" = "tutorials/migrating_from_beautifulsoup.md"}
|
||||||
{"Using Scrapeless browser" = "tutorials/external.md"}
|
|
||||||
]},
|
]},
|
||||||
{Development = [
|
{Development = [
|
||||||
{"API Reference" = [
|
{"API Reference" = [
|
||||||
|
|||||||
Reference in New Issue
Block a user