From 1f9cb2bf92eb138dfb1571672662cd28237c996b Mon Sep 17 00:00:00 2001 From: Funto-Adeyemi Date: Sun, 12 Nov 2023 21:29:14 +0000 Subject: [PATCH] Add troubleshooting.md file and revert yapf changes --- .style.yapf | 4 -- README.md | 144 +-------------------------------------------- TROUBLESHOOTING.md | 143 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 145 insertions(+), 146 deletions(-) delete mode 100644 .style.yapf create mode 100644 TROUBLESHOOTING.md diff --git a/.style.yapf b/.style.yapf deleted file mode 100644 index 725b3cc..0000000 --- a/.style.yapf +++ /dev/null @@ -1,4 +0,0 @@ -[style] -based_on_style = google -indent_width = 2 -spaces_before_comment = 2 \ No newline at end of file diff --git a/README.md b/README.md index 81e383a..da74ea8 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,8 @@ This is not an officially supported Google product. We recommend at least 32GB RAM to load TimesFM dependencies. +**Need help?** See [TROUBLESHOOTING.md](TROUBLESHOOTING.md) for common installation and usage issues. + ## Update - Dec. 30, 2024 - We are launching a 500m checkpoint as a part of TimesFM-2.0 release. This new checkpoint can be upto 25% better than v1.0 on leading benchmarks and also has a 4 times longer max. context length. - Launched [finetuning support](https://github.com/google-research/timesfm/blob/master/notebooks/finetuning.ipynb) that lets you finetune the weights of the pretrained TimesFM model on your own data. @@ -309,145 +311,3 @@ spaces_before_comment = 2 ``` Please run `yapf --in-place --recursive ` on all affected files. - -## Troubleshooting - -### Installation Issues - -#### ARM/Apple Silicon Compatibility -**Problem:** `lingvo` dependency fails on Apple Silicon (M1/M2/M3) machines. -``` -ERROR: Could not build wheels for lingvo -``` -**Solution:** This is a known issue. The `lingvo` dependency doesn't support ARM architectures. We recommend: -- Use x86_64 emulation via Rosetta 2: `arch -x86_64 pip install timesfm[pax]` -- Use the PyTorch version instead, which has better ARM support: `pip install timesfm[torch]` -- Use Docker with x86_64 emulation for consistent environments - -#### Memory Issues During Installation -**Problem:** Installation fails with memory errors. -``` -Killed (signal 9) -``` -**Solution:** -- Ensure at least 32GB RAM is available -- Close other applications during installation -- Use `pip install --no-cache-dir timesfm[torch]` to reduce memory usage -- Install in a clean virtual environment - -#### JAX/PyTorch Version Conflicts -**Problem:** Conflicting JAX and PyTorch installations. -``` -ImportError: cannot import name 'jax' from 'jax' -``` -**Solution:** -- For PyTorch-only usage: `pip install timesfm[torch]` -- For covariates with PyTorch: `pip install timesfm[torch] && pip install jax jaxlib` -- For PAX version: `pip install timesfm[pax]` - -### Runtime Errors - -#### Model Loading Issues -**Problem:** Checkpoint download fails or is corrupted. -``` -HfFileNotFoundError: 404 Client Error -``` -**Solution:** -- Check internet connectivity -- Verify Hugging Face Hub access: `huggingface-cli login` -- Clear cache: `rm -rf ~/.cache/huggingface/` -- Use explicit checkpoint paths if needed - -#### CUDA/GPU Issues -**Problem:** GPU not detected or CUDA errors. -``` -RuntimeError: CUDA out of memory -``` -**Solutions:** -- Reduce `per_core_batch_size` (try 16, 8, or 4) -- Reduce `context_len` to minimum needed -- Use `backend="cpu"` for testing -- Check GPU memory: `nvidia-smi` - -#### Context Length Errors -**Problem:** Input series longer than model capacity. -``` -ValueError: context_len must be <= 512 for v1.0 models -``` -**Solutions:** -- Use TimesFM-2.0 for longer contexts (up to 2048) -- Ensure `context_len` is multiple of 32 -- Truncate input series if necessary -- Set appropriate `context_len` in model initialization - -### Data Issues - -#### Frequency Mapping Problems -**Problem:** Unexpected forecasting results with wrong frequency. -``` -Warning: Frequency 'D' mapped to category 0 -``` -**Solutions:** -- Verify frequency mapping: D→0 (high), W/M→1 (medium), Q/Y→2 (low) -- Override automatic mapping by specifying frequency manually -- Check data granularity matches chosen frequency category - -#### Missing Values in Time Series -**Problem:** NaN or missing values in input data. -``` -ValueError: Input contains NaN values -``` -**Solutions:** -- Pre-process data to handle missing values (forward fill, interpolation) -- Ensure continuous time series without gaps -- Remove or impute missing values before forecasting - -#### Covariate Dimension Mismatches -**Problem:** Covariate lengths don't match forecast horizon. -``` -ValueError: Dynamic covariates must cover context + horizon -``` -**Solutions:** -- Ensure dynamic covariates have length = context + horizon -- Check static vs dynamic covariate classification -- Verify covariate data alignment with time series - -### Performance Issues - -#### Slow Inference -**Problem:** Forecasting takes unexpectedly long. -**Solutions:** -- Use GPU backend: `backend="gpu"` -- Optimize batch size: increase `per_core_batch_size` -- Use appropriate model size for your use case -- Profile with smaller data first - -#### Memory Usage -**Problem:** High memory consumption during inference. -**Solutions:** -- Reduce batch size: `per_core_batch_size=1` -- Process data in chunks -- Use smaller context length when possible -- Monitor memory with `htop` or `nvidia-smi` - -### Common Error Messages - -#### `ModuleNotFoundError: No module named 'xreg_lib'` -**Cause:** Missing JAX dependencies for covariates functionality. -**Solution:** `pip install jax jaxlib` - -#### `ValueError: horizon_len must be positive` -**Cause:** Invalid horizon length specified. -**Solution:** Set `horizon_len > 0` in model initialization. - -#### `RuntimeError: Expected input batch_size (X) to be divisible by batch_size (Y)` -**Cause:** Batch size mismatch. -**Solution:** Adjust `per_core_batch_size` or input data batching. - -### Getting Help - -If you encounter issues not covered here: -1. Check the [GitHub Issues](https://github.com/google-research/timesfm/issues) -2. Review the [notebooks/](notebooks/) for working examples -3. Verify your installation follows the exact steps in the Installation section -4. Test with the provided example data before using your own datasets diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md new file mode 100644 index 0000000..b8cb83e --- /dev/null +++ b/TROUBLESHOOTING.md @@ -0,0 +1,143 @@ +# Troubleshooting + +This document provides solutions to common issues encountered when using TimesFM. + +## Installation Issues + +### ARM/Apple Silicon Compatibility +**Problem:** `lingvo` dependency fails on Apple Silicon (M1/M2/M3) machines. +``` +ERROR: Could not build wheels for lingvo +``` +**Solution:** This is a known issue. The `lingvo` dependency doesn't support ARM architectures. We recommend: +- Use x86_64 emulation via Rosetta 2: `arch -x86_64 pip install timesfm[pax]` +- Use the PyTorch version instead, which has better ARM support: `pip install timesfm[torch]` +- Use Docker with x86_64 emulation for consistent environments + +### Memory Issues During Installation +**Problem:** Installation fails with memory errors. +``` +Killed (signal 9) +``` +**Solution:** +- Ensure at least 32GB RAM is available +- Close other applications during installation +- Use `pip install --no-cache-dir timesfm[torch]` to reduce memory usage +- Install in a clean virtual environment + +### JAX/PyTorch Version Conflicts +**Problem:** Conflicting JAX and PyTorch installations. +``` +ImportError: cannot import name 'jax' from 'jax' +``` +**Solution:** +- For PyTorch-only usage: `pip install timesfm[torch]` +- For covariates with PyTorch: `pip install timesfm[torch] && pip install jax jaxlib` +- For PAX version: `pip install timesfm[pax]` + +## Runtime Errors + +### Model Loading Issues +**Problem:** Checkpoint download fails or is corrupted. +``` +HfFileNotFoundError: 404 Client Error +``` +**Solution:** +- Check internet connectivity +- Verify Hugging Face Hub access: `huggingface-cli login` +- Clear cache: `rm -rf ~/.cache/huggingface/` +- Use explicit checkpoint paths if needed + +### CUDA/GPU Issues +**Problem:** GPU not detected or CUDA errors. +``` +RuntimeError: CUDA out of memory +``` +**Solutions:** +- Reduce `per_core_batch_size` (try 16, 8, or 4) +- Reduce `context_len` to minimum needed +- Use `backend="cpu"` for testing +- Check GPU memory: `nvidia-smi` + +### Context Length Errors +**Problem:** Input series longer than model capacity. +``` +ValueError: context_len must be <= 512 for v1.0 models +``` +**Solutions:** +- Use TimesFM-2.0 for longer contexts (up to 2048) +- Ensure `context_len` is multiple of 32 +- Truncate input series if necessary +- Set appropriate `context_len` in model initialization + +## Data Issues + +### Frequency Mapping Problems +**Problem:** Unexpected forecasting results with wrong frequency. +``` +Warning: Frequency 'D' mapped to category 0 +``` +**Solutions:** +- Verify frequency mapping: D→0 (high), W/M→1 (medium), Q/Y→2 (low) +- Override automatic mapping by specifying frequency manually +- Check data granularity matches chosen frequency category + +### Missing Values in Time Series +**Problem:** NaN or missing values in input data. +``` +ValueError: Input contains NaN values +``` +**Solutions:** +- Pre-process data to handle missing values (forward fill, interpolation) +- Ensure continuous time series without gaps +- Remove or impute missing values before forecasting + +### Covariate Dimension Mismatches +**Problem:** Covariate lengths don't match forecast horizon. +``` +ValueError: Dynamic covariates must cover context + horizon +``` +**Solutions:** +- Ensure dynamic covariates have length = context + horizon +- Check static vs dynamic covariate classification +- Verify covariate data alignment with time series + +## Performance Issues + +### Slow Inference +**Problem:** Forecasting takes unexpectedly long. +**Solutions:** +- Use GPU backend: `backend="gpu"` +- Optimize batch size: increase `per_core_batch_size` +- Use appropriate model size for your use case +- Profile with smaller data first + +### Memory Usage +**Problem:** High memory consumption during inference. +**Solutions:** +- Reduce batch size: `per_core_batch_size=1` +- Process data in chunks +- Use smaller context length when possible +- Monitor memory with `htop` or `nvidia-smi` + +## Common Error Messages + +### `ModuleNotFoundError: No module named 'xreg_lib'` +**Cause:** Missing JAX dependencies for covariates functionality. +**Solution:** `pip install jax jaxlib` + +### `ValueError: horizon_len must be positive` +**Cause:** Invalid horizon length specified. +**Solution:** Set `horizon_len > 0` in model initialization. + +### `RuntimeError: Expected input batch_size (X) to be divisible by batch_size (Y)` +**Cause:** Batch size mismatch. +**Solution:** Adjust `per_core_batch_size` or input data batching. + +## Getting Help + +If you encounter issues not covered here: +1. Check the [GitHub Issues](https://github.com/google-research/timesfm/issues) +2. Review the [notebooks/](notebooks/) for working examples +3. Verify your installation follows the exact steps in the Installation section +4. Test with the provided example data before using your own datasets \ No newline at end of file