time series forecasting

# valoswiss-time-series-forecasting

**Macro-categoria**: 📈 QUANT/MARKETS
**Scope**: Time-series probabilistic forecasting per asset returns, macro indicators, revenue/AUM projection con confidence interval P10/P50/P90 obbligatori
**Reference repos**:
- `Nixtla/nixtla` — TimeGPT-1 foundation model (100B param, zero-shot multi-horizon, calibrated PI)
- `google-research/timesfm` — Google TimesFM 200M-500M decoder-only foundation
- `amazon-science/chronos-forecasting` — Amazon Chronos T5 pretrained tokenized TS
- `Nixtla/neuralforecast` — NHITS, NBEATS, TFT, DeepAR, AutoFormer (PyTorch Lightning)
- `Nixtla/statsforecast` — ARIMA, ETS, Theta, MSTL, AutoCES (numba JIT)
- `Nixtla/mlforecast` — LightGBM/XGBoost with lag features pipeline
- `TimeCopilot/timecopilot` — GenAI forecasting agent natural-language interface
**Owner downstream**: ADVISOR (return projection holdings) · QUANT/RM (macro projection) · SUPERVISOR/ADMIN (cross-tenant)
**Last aligned**: 2026-05-03 V20

---

## §0 · Pre-flight check

Prima di ogni intervento, verifica in quest'ordine:

1. **Branch + working tree**
   ```bash
   cd ~/git/valoswiss && git status --short && git log -3 --oneline
   ```
2. **Sidecar Python health (Nixtla suite + Foundation models)**
   ```bash
   curl -s http://127.0.0.1:8894/healthz | jq .
   ```
   Atteso `{"status":"ok","models":{"timegpt":true,"timesfm":true,"chronos":true,"statsforecast":true,"neuralforecast":true,"mlforecast":true},"useReal":true|false}`. Se 502 → PM2: `pm2 list | grep ts-forecast-py`.
3. **NestJS proxy health**
   ```bash
   curl -s http://127.0.0.1:4010/api/forecast/health -H "Cookie: valo_token=<dev-token>"
   ```
   Atteso `{ sidecar:{status:'ok'}, circuitBreaker:{state:'closed', failures:0} }`.
4. **Prisma schema sync (Forecast / ForecastModel / ForecastInterval)**
   ```bash
   cd apps/api && npx prisma migrate status
   ```
   Verifica i 3 model + enum `ForecastModelKind` / `ForecastHorizonUnit` / `ForecastStatus`.
5. **Tenant configs**: `tenants/ws.json` e `tenants/az.json` devono avere `"timeSeriesForecasting": true`.
6. **Persona pack**: `apps/api/src/common/persona-packs/persona-packs.constants.ts` deve avere `'timeSeriesForecasting'` in `defaultModules` per `ADVISOR` + `RELATIONSHIP_MANAGER` + `QUANT_ANALYST`.
7. **Module registry**: `apps/web/src/lib/module-registry.ts` deve esporre entry `timeSeriesForecasting` con `sidebarSection: 'ANALISI'`, `requiredRole: 'ADVISOR'`, `personaHint: 'predictive'`, icon `📈`.
8. **R-Audit gate (MAJOR peso 8)**: prima di qualsiasi commit su file CRITICAL (vedi §3), eseguire `npx tsx scripts/r-audit.ts <file> --validate-business-logic --rule TIMESERIES-FORECAST-CONFIDENCE`.

Se uno qualunque dei 7 punti fallisce, **fermati e annota la deviazione** prima di procedere — la 3-Point Registration è invariante non negoziabile (vedi `feedback_new_module_registration.md` + `_CROSS-AGENT-TOOLS.md §8`).

---

## §1 · Aree di competenza

### 1.1 Use case ValoSwiss (forecasting domains)

| Domain | Input | Horizon | Frequency | Confidence target |
|---|---|---|---|---|
| **Asset returns** | Yahoo/FMP daily price → log-return | 1d / 5d / 20d / 60d | daily | P10/P50/P90 |
| **Macro GDP/inflation/PMI** | FRED/IMF/ECB monthly/quarterly | 1m / 3m / 6m / 12m | monthly | P10/P50/P90 |
| **Revenue projection company** | FMP fundamentals quarterly | 1q / 2q / 4q | quarterly | P25/P50/P75 |
| **AUM growth advisor** | Internal `Portfolio.totalValue` daily | 30d / 90d / 365d | daily | P10/P50/P90 |
| **VIX regime forecasting** | Yahoo `^VIX` daily close | 1d / 5d / 20d | daily | P10/P50/P90 |
| **FX cross-pair** | ExchangeRate daily | 1d / 5d / 20d | daily | P10/P50/P90 |

### 1.2 Pipeline forecasting (4 stage)

1. **Data preparation** — fetch storico (≥250 obs default), gap fill (LOCF), outlier winsorize (cap 1°/99° percentile), stationarity check (ADF), seasonality decomposition (STL).
2. **Model selection** — multi-model ensemble: foundation (TimeGPT zero-shot) + neural (NHITS/TFT) + classical (AutoARIMA/ETS) + ML (LightGBM con lag features).
3. **Cross-validation rolling** — expanding window CV con N=10 fold default, metrics MAPE/SMAPE/MASE/CRPS.
4. **Conformal prediction interval** — calibrated P10/P50/P90 via residual quantile su CV folds (Nixtla `ConformalIntervals` o custom).

### 1.3 Output strutturato Pydantic (sidecar :8894)

```python
{
  "forecastId": "uuid",
  "ticker": "AAPL",          # o macro symbol "GDPC1" / portfolio_id
  "asOfDate": "2026-05-03",
  "horizonDays": 20,
  "frequency": "daily",
  "modelEnsemble": ["TimeGPT-1", "NHITS", "AutoARIMA", "Chronos-T5-base"],
  "predictions": [
    {
      "date": "2026-05-04",
      "p10": 0.012,           # 10th percentile log-return
      "p50": 0.034,           # median
      "p90": 0.058,           # 90th percentile
      "modelContributions": { "TimeGPT-1": 0.034, "NHITS": 0.031, ... }
    },
    ...20 entries...
  ],
  "crossValidation": {
    "folds": 10,
    "windowType": "expanding",
    "mape": 0.082,
    "smape": 0.078,
    "mase": 0.91,
    "crps": 0.0042,
    "coverage": { "p10p90": 0.79, "p25p75": 0.51 }   # actual coverage di validation
  },
  "rationale": "...",         # LLM-generated 200-400 token via /forecast/explain
  "regimeAlert": null | { "regime": "high-vol", "confidence": 0.78 }
}
```

### 1.4 Tier presets (`runner.py:TIER_PRESETS`)

| Tier | Foundation | Neural | Classical | ML | Ensemble strategy |
|---|---|---|---|---|---|
| `forecast-fast` | Chronos-T5-tiny | NHITS | AutoARIMA | LightGBM | median ensemble, 30s |
| `forecast-premium` | TimeGPT + Chronos-T5-base | NHITS + TFT | AutoARIMA + ETS | LightGBM + XGBoost | weighted by inverse-MAPE, 90s |
| `forecast-uhnw` | TimeGPT + TimesFM + Chronos-T5-large | NHITS + TFT + AutoFormer | AutoARIMA + ETS + Theta + MSTL | LightGBM + XGBoost + CatBoost | stacked meta-learner, 240s |

Override env: `TS_FORECAST_TIER`, `TS_FORECAST_FOUNDATION_MODELS`, `TS_FORECAST_DISABLE_NEURAL=1`.

### 1.5 Persona visibility

- **ADVISOR** (ws+az): proprie portfolios scoped (`requestedByUserId == req.user.id`); use case asset/portfolio/AUM
- **RELATIONSHIP_MANAGER**: idem ADVISOR
- **QUANT_ANALYST**: cross-asset + macro forecasts + factor projection
- **SUPERVISOR/ADMIN**: cross-tenant + admin watchlist + tier override + cron config
- **CLIENT/PROSPECT**: NEGATO assoluto — MIFID II compliance, niente proiezioni quantitative dirette al cliente

---

## §2 · Pattern di codice

### 2.1 TimeGPT zero-shot (foundation, fastest production-grade)

```python
# services/ts-forecast-py/runners/timegpt_runner.py
from nixtla import NixtlaClient
import pandas as pd

nixtla = NixtlaClient(api_key=os.environ['NIXTLA_API_KEY'])

def forecast_timegpt(df: pd.DataFrame, h: int, level: list[int] = [80, 90]) -> pd.DataFrame:
    """
    df: DataFrame con colonne ['unique_id', 'ds', 'y'] formato Nixtla
    h: horizon (es. 20 per 20 giorni)
    level: confidence level [80, 90] → genera p10/p90 + p5/p95
    """
    forecast = nixtla.forecast(
        df=df,
        h=h,
        level=level,
        finetune_steps=0,                # zero-shot (capability over compliance — PROTOTYPE-PHASE)
        time_col='ds',
        target_col='y',
    )
    # forecast columns: ['unique_id', 'ds', 'TimeGPT', 'TimeGPT-lo-90', 'TimeGPT-hi-90', ...]
    return forecast
```

### 2.2 NeuralForecast TFT training + inference

```python
# services/ts-forecast-py/runners/neural_runner.py
from neuralforecast import NeuralForecast
from neuralforecast.models import NHITS, TFT, AutoFormer
from neuralforecast.losses.pytorch import DistributionLoss

def fit_neural_ensemble(df: pd.DataFrame, h: int, freq: str = 'D'):
    """
    Distribution loss → genera direttamente quantili P10/P50/P90 (capability-first)
    """
    models = [
        NHITS(h=h, input_size=4*h, max_steps=500,
              loss=DistributionLoss(distribution='Normal', level=[80, 90])),
        TFT(h=h, input_size=4*h, max_steps=500,
            loss=DistributionLoss(distribution='StudentT', level=[80, 90])),
    ]
    nf = NeuralForecast(models=models, freq=freq)
    nf.fit(df=df, val_size=h)
    forecast = nf.predict()
    return forecast  # con cols *-lo-90 / *-hi-90 / *-lo-80 / *-hi-80
```

### 2.3 Chronos pretrained inference (Amazon T5-based)

```python
# services/ts-forecast-py/runners/chronos_runner.py
import torch
from chronos import ChronosPipeline

pipeline = ChronosPipeline.from_pretrained(
    "amazon/chronos-t5-base",   # o tiny/small/large
    device_map="cuda" if torch.cuda.is_available() else "cpu",
    torch_dtype=torch.bfloat16,
)

def forecast_chronos(context: torch.Tensor, h: int, num_samples: int = 100) -> dict:
    """
    Chronos sample-based: genera num_samples traiettorie → quantile empirico per P10/P50/P90
    """
    forecast = pipeline.predict(
        context=context,
        prediction_length=h,
        num_samples=num_samples,
    )  # tensor shape [batch, num_samples, h]
    p10 = torch.quantile(forecast, 0.10, dim=1)
    p50 = torch.quantile(forecast, 0.50, dim=1)
    p90 = torch.quantile(forecast, 0.90, dim=1)
    return {'p10': p10.numpy(), 'p50': p50.numpy(), 'p90': p90.numpy()}
```

### 2.4 StatsForecast classical baseline (numba JIT)

```python
# services/ts-forecast-py/runners/stats_runner.py
from statsforecast import StatsForecast
from statsforecast.models import AutoARIMA, AutoETS, AutoTheta, MSTL

def forecast_classical(df: pd.DataFrame, h: int, freq: str = 'D'):
    sf = StatsForecast(
        models=[AutoARIMA(season_length=5),  # weekly seasonality on daily
                AutoETS(season_length=5),
                AutoTheta(season_length=5)],
        freq=freq,
        n_jobs=-1,                            # parallelizza per id
    )
    sf.fit(df)
    forecast = sf.predict(h=h, level=[80, 90])
    return forecast
```

### 2.5 Cross-validation rolling expanding window

```python
# services/ts-forecast-py/cv.py
def cross_validate_expanding(sf: StatsForecast, df: pd.DataFrame, h: int, n_windows: int = 10):
    """
    Rolling expanding window: train fino a t, predict t+1..t+h, sliding di h step.
    Output metrics: MAPE/SMAPE/MASE/CRPS + actual coverage P10/P90.
    """
    cv_results = sf.cross_validation(
        df=df,
        h=h,
        n_windows=n_windows,
        step_size=h,
        level=[80, 90],
    )
    metrics = {
        'mape': mean_absolute_percentage_error(cv_results['y'], cv_results['AutoARIMA']),
        'smape': symmetric_mape(cv_results['y'], cv_results['AutoARIMA']),
        'mase': mase(cv_results['y'], cv_results['AutoARIMA'], df['y']),
        'crps': continuous_ranked_probability_score(cv_results),
        'coverage_p10p90': ((cv_results['y'] >= cv_results['AutoARIMA-lo-80']) &
                            (cv_results['y'] <= cv_results['AutoARIMA-hi-80'])).mean(),
    }
    return metrics
```

### 2.6 Conformal prediction interval (calibrated)

```python
# services/ts-forecast-py/conformal.py
from statsforecast.utils import ConformalIntervals

def fit_with_conformal(sf: StatsForecast, df: pd.DataFrame, h: int):
    """
    Conformal: usa residui CV per calibrare quantili EMPIRICI, non parametrici.
    Garantisce coverage actual ≈ nominal level (e.g. 79% actual per nominal 80%).
    """
    sf_conf = StatsForecast(
        models=sf.models,
        freq=sf.freq,
        n_jobs=-1,
        prediction_intervals=ConformalIntervals(n_windows=10, h=h),
    )
    sf_conf.fit(df)
    forecast = sf_conf.predict(h=h, level=[80, 90])
    # forecast contains *-lo-80 / *-hi-80 calibrated empirically
    return forecast
```

### 2.7 LLM rationale generation (`/forecast/explain`)

```python
# services/ts-forecast-py/explain.py
def generate_rationale(forecast_payload: dict, llm_client) -> str:
    """
    Invocato post-forecast con primary LLM (qwen3.6:27b locale o claude-sonnet-4-6 cloud).
    Genera rationale 200-400 token italiano "analista quant Family Office".
    """
    prompt = f"""Sei un analista quant senior di Family Office. Spiega il forecast 

…[truncato — apri il file MD per testo completo]