docs: make the contribution rules clearer

This commit is contained in:
Karim shoair
2026-02-28 16:57:41 +02:00
parent 50a69c3583
commit b9fa6b62e6
+54 -14
View File
@@ -63,6 +63,37 @@ Please follow these coding conventions as we do when writing code for Scrapling:
> Please dont put your name in the code you contribute; git provides enough metadata to identify the author of the code.
## Development
### Getting started
1. Fork the repository and clone your fork:
```bash
git clone https://github.com/<your-username>/Scrapling.git
cd Scrapling
git checkout dev
```
2. Create a virtual environment and install dependencies:
```bash
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activate
pip install -e ".[all]"
pip install -r tests/requirements.txt
```
3. Install browser dependencies:
```bash
scrapling install
```
4. Set up pre-commit hooks:
```bash
pip install pre-commit
pre-commit install
```
### Tips
Setting the scrapling logging level to `debug` makes it easier to know what's happening in the background.
```python
import logging
@@ -73,15 +104,6 @@ Bonus: You can install the beta of the upcoming update from the dev branch as fo
pip3 install git+https://github.com/D4Vinci/Scrapling.git@dev
```
## Building Documentation
Documentation is built using [Zensical](https://zensical.org/). You can build it locally using the following commands:
```bash
pip install zensical
pip install -r docs/requirements.txt
zensical build --clean # Build the static site
zensical serve # Local preview
```
## Tests
Scrapling includes a comprehensive test suite that can be executed with pytest. However, first, you need to install all libraries and `pytest-plugins` listed in `tests/requirements.txt`. Then, running the tests will result in an output like this:
```bash
@@ -93,16 +115,34 @@ Scrapling includes a comprehensive test suite that can be executed with pytest.
configfile: pytest.ini
plugins: asyncio-1.2.0, anyio-4.11.0, xdist-3.8.0, httpbin-2.1.0, cov-7.0.0
asyncio: mode=Mode.STRICT, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
10 workers [515 items]
scheduling tests via LoadScheduling
10 workers [515 items]
scheduling tests via LoadScheduling
...<shortened>...
=============================== 271 passed in 52.68s ==============================
```
Hence, we used `-n auto` in the command above to run tests in threads to increase speed.
Here, `-n auto` runs tests in parallel across multiple processes to increase speed.
**Note:** You may need to run browser tests sequentially (`DynamicFetcher`/`StealthyFetcher`) to avoid conflicts. To run non-browser tests in parallel and browser tests separately:
```bash
# Non-browser tests (parallel)
pytest tests/ -k "not (DynamicFetcher or StealthyFetcher)" -n auto
# Browser tests (sequential)
pytest tests/ -k "DynamicFetcher or StealthyFetcher"
```
Bonus: You can also see the test coverage with the `pytest` plugin below
```bash
pytest --cov=scrapling tests/
```
## Building Documentation
Documentation is built using [Zensical](https://zensical.org/). You can build it locally using the following commands:
```bash
pip install zensical
pip install -r docs/requirements.txt
zensical build --clean # Build the static site
zensical serve # Local preview
```