darkpowerxo
9875d926fe
feat: add TimeSeriesDataset for PEFT fine-tuning
...
Sliding-window dataset that produces (context, mask, target) tuples:
- Accepts list of arrays, long-format, or wide-format DataFrames
- Context length auto-rounded to multiple of patch_len (32)
- Left-pads short series with proper masking
- Configurable stride for window overlap
2026-04-08 13:52:19 -04:00
darkpowerxo
7357458e45
feat: add LoRA/DoRA adapter layers for TimesFM 2.5 (PyTorch)
...
Implement production-grade PEFT adapters targeting the 2.5 architecture:
- LoRALinear: low-rank A/B decomposition with scaling (alpha/rank)
- DoRALinear: weight-decomposed LoRA (magnitude + direction)
- inject_adapters(): freezes base weights, wraps target nn.Linear modules
- Supports fused QKV (qkv_proj), attention output, and FFN layers
- num_adapter_layers controls how many top layers get adapters (0=all)
- target_modules selects 'all', 'attention', or 'ffn'
- merge_adapters(): folds adapter deltas back into base nn.Linear
- save/load_adapter_weights(): safetensors adapter-only checkpoints
- PEFTConfig dataclass with all hyperparameters
References:
LoRA — https://arxiv.org/abs/2106.09685
DoRA — https://arxiv.org/abs/2402.09353
2026-04-08 13:51:55 -04:00
darkpowerxo