a67eeb2e7d
feat: add CLI entry-point and launch script for PEFT fine-tuning
...
- finetune.py: argparse CLI with all config flags, CSV data loading,
chronological train/val split, and full training pipeline
Usage: python -m peft.finetune --data_path data.csv --value_col y
Multi-GPU: torchrun --nproc_per_node=4 -m peft.finetune ...
- finetune.sh: env-var driven launch script for single/multi-GPU
Usage: DATA_PATH=data.csv NUM_GPUS=4 bash peft/finetune.sh
2026-04-08 13:53:35 -04:00
eca7ca3428
feat: add multi-GPU PEFT trainer for TimesFM 2.5
...
PEFTTrainer with production-grade training loop:
- PyTorch DDP multi-GPU via torchrun
- Mixed-precision training (fp16/bf16) with GradScaler
- Gradient checkpointing for long contexts
- Cosine-with-warmup LR schedule
- MSE loss + optional pinball quantile loss (9 channels)
- Early stopping on validation loss
- Adapter-only checkpointing (safetensors)
- W&B logging (rank-0 only)
- Differentiable training forward that replicates the 2.5
patch -> RevIN -> transformer -> output-head -> un-RevIN path
2026-04-08 13:52:59 -04:00
9875d926fe
feat: add TimeSeriesDataset for PEFT fine-tuning
...
Sliding-window dataset that produces (context, mask, target) tuples:
- Accepts list of arrays, long-format, or wide-format DataFrames
- Context length auto-rounded to multiple of patch_len (32)
- Left-pads short series with proper masking
- Configurable stride for window overlap
2026-04-08 13:52:19 -04:00
7357458e45
feat: add LoRA/DoRA adapter layers for TimesFM 2.5 (PyTorch)
...
Implement production-grade PEFT adapters targeting the 2.5 architecture:
- LoRALinear: low-rank A/B decomposition with scaling (alpha/rank)
- DoRALinear: weight-decomposed LoRA (magnitude + direction)
- inject_adapters(): freezes base weights, wraps target nn.Linear modules
- Supports fused QKV (qkv_proj), attention output, and FFN layers
- num_adapter_layers controls how many top layers get adapters (0=all)
- target_modules selects 'all', 'attention', or 'ffn'
- merge_adapters(): folds adapter deltas back into base nn.Linear
- save/load_adapter_weights(): safetensors adapter-only checkpoints
- PEFTConfig dataclass with all hyperparameters
References:
LoRA — https://arxiv.org/abs/2106.09685
DoRA — https://arxiv.org/abs/2402.09353
2026-04-08 13:51:55 -04:00
7d8f3d971d
2.0.0 initial
2025-09-12 00:18:08 +00:00
55f71de98a
Create PEFT README.md
2024-08-04 12:27:56 +05:30
d72ff835fb
keep only a single bash script
2024-08-04 09:56:11 +05:30
18da73a783
remove helper and fix early stopping logic
2024-07-18 19:18:56 +00:00
5901805e7e
add bash scripts for running diff FT strategies
2024-07-17 21:39:14 +00:00
a59979d3b4
add example dora.sh file
2024-07-16 19:20:04 +00:00
845661dfa3
update usage tutorial
2024-07-16 18:24:50 +00:00
461c2cd194
update checkpoint dir name
2024-07-16 16:26:30 +00:00
71d980267d
add parameter efficient finetuning pipeline
...
Why? this commit adds a generic finetuning pipeline with LoRA and DoRA support
2024-07-16 01:26:36 +00:00
darkpowerxo