timesfm

freedak/timesfm

Fork 0

Commit Graph

Author	SHA1	Message	Date
darkpowerxo	18d5eb2d44	fix: improve PEFT device consistency and XReg output slicing - Initialize LoRA parameters on the same device as the base linear layer - Load adapter weights directly to the model device instead of hardcoded CPU - Slice XReg linear regression outputs to match the specified sequence lengths - Replace batch-wide covariate normalization with per-input normalization in create_covariate_matrix to prevent cross-input scale leakage. - Refactor BatchedInContextXRegLinear.fit to solve ridge regression per instance rather than as a single global matrix solve, avoiding cross-contamination between batched inputs. - Truncate JAX regression outputs to the actual train/test lengths after the padded matrix multiply, fixing shape mismatches for non-power-of-2 horizons (e.g. horizon=24 was returning 32 elements).	2026-04-08 21:43:52 -04:00
darkpowerxo	7357458e45	feat: add LoRA/DoRA adapter layers for TimesFM 2.5 (PyTorch) Implement production-grade PEFT adapters targeting the 2.5 architecture: - LoRALinear: low-rank A/B decomposition with scaling (alpha/rank) - DoRALinear: weight-decomposed LoRA (magnitude + direction) - inject_adapters(): freezes base weights, wraps target nn.Linear modules - Supports fused QKV (qkv_proj), attention output, and FFN layers - num_adapter_layers controls how many top layers get adapters (0=all) - target_modules selects 'all', 'attention', or 'ffn' - merge_adapters(): folds adapter deltas back into base nn.Linear - save/load_adapter_weights(): safetensors adapter-only checkpoints - PEFTConfig dataclass with all hyperparameters References: LoRA — https://arxiv.org/abs/2106.09685 DoRA — https://arxiv.org/abs/2402.09353	2026-04-08 13:51:55 -04:00

Author

SHA1

Message

Date

darkpowerxo

18d5eb2d44

fix: improve PEFT device consistency and XReg output slicing

- Initialize LoRA parameters on the same device as the base linear layer
- Load adapter weights directly to the model device instead of hardcoded CPU
- Slice XReg linear regression outputs to match the specified sequence lengths

- Replace batch-wide covariate normalization with per-input normalization
  in create_covariate_matrix to prevent cross-input scale leakage.
- Refactor BatchedInContextXRegLinear.fit to solve ridge regression
  per instance rather than as a single global matrix solve, avoiding
  cross-contamination between batched inputs.
- Truncate JAX regression outputs to the actual train/test lengths
  after the padded matrix multiply, fixing shape mismatches for
  non-power-of-2 horizons (e.g. horizon=24 was returning 32 elements).

2026-04-08 21:43:52 -04:00

darkpowerxo

7357458e45

feat: add LoRA/DoRA adapter layers for TimesFM 2.5 (PyTorch)

Implement production-grade PEFT adapters targeting the 2.5 architecture:

- LoRALinear: low-rank A/B decomposition with scaling (alpha/rank)
- DoRALinear: weight-decomposed LoRA (magnitude + direction)
- inject_adapters(): freezes base weights, wraps target nn.Linear modules
  - Supports fused QKV (qkv_proj), attention output, and FFN layers
  - num_adapter_layers controls how many top layers get adapters (0=all)
  - target_modules selects 'all', 'attention', or 'ffn'
- merge_adapters(): folds adapter deltas back into base nn.Linear
- save/load_adapter_weights(): safetensors adapter-only checkpoints
- PEFTConfig dataclass with all hyperparameters

References:
  LoRA — https://arxiv.org/abs/2106.09685
  DoRA — https://arxiv.org/abs/2402.09353

2026-04-08 13:51:55 -04:00

2 Commits