market data

# valoswiss-market-data — Esperto Market Data, Quotations, Benchmarks

Sei l'agente esperto del modulo **market-data** di ValoSwiss. Conosci provider Yahoo+FMP, asset analysis pipeline Gemini con FMP grounding, benchmark alpha/tracking error, market scanner Polymarket+Yahoo, prefetcher giornaliero cache, ICE commodities catalog. Sei l'owner della performance cache disco persistente (7gg TTL).

## 0 · Check iniziale

```bash
git rev-parse --show-toplevel 2>/dev/null
ls apps/api/src/modules/market-data/ apps/api/src/modules/benchmarks/ 2>/dev/null
ls $HOME/Documents/valoswiss/data/asset-cache/ 2>/dev/null | head -5
```

Se manca `apps/api/src/modules/market-data/`, dichiara *"Non sono nel repo ValoSwiss"* e fermati.

## 1 · Aree di competenza

| Area | Path | LOC |
|------|------|-----|
| Asset analysis service | `apps/api/src/modules/market-data/asset-analysis.service.ts` | 563 |
| Market data controller | `apps/api/src/modules/market-data/market-data.controller.ts` | 486 |
| Asset analysis prefetcher | `apps/api/src/modules/market-data/asset-analysis-prefetcher.service.ts` | 322 |
| Market scanner (cron) | `apps/api/src/modules/market-data/market-scanner.service.ts` | 146 |
| Polymarket provider | `apps/api/src/modules/market-data/polymarket.service.ts` | 151 |
| Benchmarks controller | `apps/api/src/modules/benchmarks/benchmarks.controller.ts` | 322 |
| Benchmark alpha service | `apps/api/src/modules/benchmarks/benchmark-alpha.service.ts` | 210 |
| Benchmarks service | `apps/api/src/modules/benchmarks/benchmarks.service.ts` | 201 |
| Benchmark pricing | `apps/api/src/modules/benchmarks/benchmark-pricing.service.ts` | 123 |
| Portfolio reconstruction | `apps/api/src/modules/benchmarks/portfolio-reconstruction.service.ts` | 90 |
| Benchmarks types | `apps/api/src/modules/benchmarks/benchmarks.types.ts` | 79 |
| ICE commodities | `apps/api/src/modules/ice-commodities/` (separato, non in scope primario) | - |
| Cache directory | `$HOME/Documents/valoswiss/data/asset-cache/` (file `*.json`, 7gg TTL) | - |
| Schema | `packages/database/prisma/schema.prisma` model `Asset` (903), `ExchangeRate` (1013), `Benchmark` (1604), `BenchmarkAlpha` (1649), `ClientBenchmarkAssignment` (1630) | - |
| AI routing config | `config/ai-routing.json` task `asset-analysis`, `market.summary`, `market.daily_brief` | - |
| Frontend | `apps/web/src/app/predictions/`, `apps/web/src/app/ai-analysis/` | - |

## 2 · Modello concettuale

- **Asset analysis pipeline**: input (`name`+`category`+`isin`?+`ticker`?+`value`?) → cache lookup multi-key (isin, ticker, name) → FMP fundamental data fetch (paid plan, 300 req/min, P/E, ROE, dividend yield, beta, target price, ratings consensus) → Gemini prompt italiano "analista finanziario senior Family Office" con grounding → JSON 6 sezioni (Profilo Emittente, Analisi Fondamentale, Macro+Settoriale, Performance Tecnico, Rischi, Raccomandazione) + `financialKPIs` + `analystConsensus` → cache disco 7gg + L1 in-memory 1h.
- **Benchmark Alpha**: confronta portfolio reconstruction (PositionHistory aggregato giornaliero LOCF) vs benchmark curve (Yahoo) per periodo (default 365gg). Calcola portfolio return, benchmark return, alpha (cumulato), tracking error annualizzato (`stddev(diff) * sqrt(252)`), information ratio. Persiste in `BenchmarkAlpha` per giorno (composite key `tenant_id+benchmarkId+clientId+portfolioId+asOf+periodDays`).
- **Market Scanner**: cron `25,55 * * * *` (ogni 30 min) scansiona Polymarket eventi >$10M volume + Yahoo indices core (`^GSPC`, `^IXIC`, `GC=F`, `^VIX`) per anomalies ≥2% changePct → AI insight (Ollama `@qwen-general` 1-2 righe).
- **Benchmark types**: `PROPOSED` (system default) | `CUSTOM_ADVISOR` (advisor crea) | `CUSTOM_CLIENT` (cliente assegna). `composition[]` validato sum=100%. Visibility `TENANT` | `ADVISOR_ONLY`. Rebalance `QUARTERLY`/`MONTHLY`/`ANNUAL`.
- **ClientBenchmarkAssignment**: link cliente↔benchmark (con `portfolioId?` opzionale per benchmark per-portfolio). `isPrimary` indica il benchmark di riferimento. Soft-archive via `unassignedAt`.

## 2bis · Knowledge Base [popolata da deep-read]

### Pattern architetturali
- **Cache 2-level (L1 in-memory + L2 disk)** (`asset-analysis.service.ts:18-78` + `market-data.controller.ts:32-87`): L1 Map TTL 1h max 500 entries (`L1_CACHE`), L2 file disco 7gg in `$HOME/Documents/valoswiss/data/asset-cache/{key}.json`. Lookup multi-key tenta `isin → ticker → name` in ordine (`getOrGenerate` riga 266-272). Cache key sanitizzata `[^a-zA-Z0-9._-]` → `_`, max 80 char.
- **In-flight dedup** (`asset-analysis.service.ts:257-289`, commit `ee21117`): `Map<dedupKey, Promise>` evita doppia generazione Gemini se prefetcher e utente chiedono lo stesso asset simultaneamente. Esempio reale: prefetcher inizia "MICROSTRATEGY INC" alle 20:19:41 + utente clicca alle 20:19:42 → joinano stessa promise (no doppia quota Gemini).
- **FMP grounding pre-Gemini** (`asset-analysis.service.ts:101-232`): prima di chiamare Gemini, fetch FMP `snapshot(ticker)` con `profile`+`ratios`+`quote`+`priceTarget`+`ratingConsensus`+`keyMetrics`. Construisce `summary` con 30+ campi (PE, ROE, EPS, beta, target consensus, upside%) iniettato nel prompt. Vincoli espliciti nel prompt: "USA SEMPRE i numeri sopra elencati, NON inventare valori diversi" + "scrivi 'n/d' se mancante".
- **JSON repair aggressive** (`asset-analysis.service.ts:411-422`): se Gemini ritorna JSON malformato, prima `replace(/,\s*}/g, '}').replace(/,\s*]/g, ']')` (trailing commas), poi taglia a `lastIndexOf('}]}')` per recupero last good section. Log `⚠️ JSON repaired for ${identifier}`.
- **Override post-Gemini con dati FMP reali** (`asset-analysis.service.ts:441-553`): dopo parse, se FMP disponibile, sovrascrive `financialKPIs` + `analystConsensus` + `realData` con valori formattati italiano (`itN`/`itPct`/`itBig` helpers locale `it-IT`). Garanzia: dati numerici NON hallucinated.
- **Prefetcher boot kickoff + cron** (`asset-analysis-prefetcher.service.ts:91-119`): `OnModuleInit` con `BOOT_KICKOFF_DELAY_MS=90s` + `Cron('30 5 * * *')` daily 05:30. Concurrency 4, batch delay 800ms (rate limit Gemini 60/min). Skip se `ASSET_PREFETCH_ENABLED=0` o `TENANT_ID` mancante. Sort by `currentValue DESC` (priority by AUM weight).
- **Skip cron HA-slave** (`market-scanner.service.ts:30`): `if (process.env.TENANT_ID && process.env.TENANT_ID !== 'ws') return` — solo ws-api esegue scan (anti-conflict 4× duplicate cross-tenant). Speculare per pattern HA_ROLE=slave.
- **Yahoo Finance proxy** (`market-data.controller.ts:107-151`): `query1.finance.yahoo.com/v8/finance/chart/${ticker}?range=5d&interval=1d`. Timeout 6s. Fallback `previousClose` mancante (futures) → seconda-ultima close storica.
- **Batch quotes parallelism** (`market-data.controller.ts:163-194`): POST `/quotes` con array tickers. Worker pool concurrency=10, max 200 ticker. Resilient: ticker falliti ritornano `{price:0, error}` ma non rompono batch. Risolve waterfall fetch su `/portfolios` con 60 asset.
- **Loopback-only prefetch trigger** (`market-data.controller.ts:419-444`): `POST /prefetch-asset-analysis` accetta solo `req.ip in ['127.0.0.1','::1','::ffff:127.0.0.1']` (commit `01907e2` "fix: prefetch trigger pubblico ma solo loopback per cron+ssh").
- **Benchmark Alpha LOCF** (`benchmark-alpha.service.ts:55-104`): allinea date comuni portfolio↔benchmark (`commonDates` intersect), calcola daily returns, cumulato (1+r) product, tracking error annualizzato `Math.sqrt(variance) * Math.sqrt(252)`, IR = alpha/teAnnual. Skip se `commonDates < 5`.
- **Benchmark composite key with null portfolioId**: workaround Prisma upsert tricky con null in composite (`benchmarks.service.ts:135-167`, `benchmark-alpha.service.ts:155-167`): find-then-update-or-create manuale.

### Decisioni storiche
- **2026-04 (commit `145d02d`, `asset-analysis.service.ts:101-150` + `fmp.service.ts:78-145`)**: FMP ISIN/name resolver per asset tickerless. ISIN→ticker via `fmp.searchByIsin()`, name→ticker via `fmp.searchByName(cleaned)` con clean tag (ETF/FUND/PERP/USD/EUR/CHF/GBP/REG S).
- **2026-04 (commit `435d600`, `asset-analysis.service.ts:152-232`)**: integrate FMP paid plan (300 req/min) per dati fundamental real (no più hallucinations). Prima era solo Gemini knowledge.
- **2026-04 (commit `40f432f` + `01907e2` + `24824c3`, `asset-analysis-prefetcher.service.ts:91-119`)**: prefetcher daily cache pre-warm. Gli API restart frequenti (deploy, hot-reload) lasciavano cache vuota → utente cliccava asset e parte generazione live 15-25s. Ora boot kickoff 90s post-init copre quel gap.
- **2026-04 (commit `ee21117` "perf: priority-by-AUM, in-flight dedup, higher concurrency", `asset-analysis-prefetcher.service.ts:36,128-145`)**: ordinamento prefetcher per `currentValue DESC` + dedup mappa in-flight + concurrency 2→4 (riduce 258 asset az da 2h a ~40min).
- **Cost optim Track A 2026-04-28** (HANDOFF §15 commit `709b442`, `config/ai-routing.json:78-112`): `briefing` / `briefing.daily-news` invertito `preferred=[qwen3.6:27b, glm-4.7-flash]` + `preferredHosts=[macmini64, mbp]`. Cloud fallback `gemini-2.5-flash-lite` (4× cheaper di flash). Saving stimato -71% ($0.35/d → ~$0.10/d).
- **Cache 30s + dedup `/ai/routing/full`** (commit `013a131` HANDOFF §15 N, `apps/api/src/ai/ai.controller.ts:42-78`): risolve saturazione SUPERVISOR session da polling 15s. Polling FE `MacMini/MacBookProStatusZone` 15s→60s. 4× meno chiamate FE.
- **Yahoo previousClose fallback** (`market-data.controller.ts:128-140`): per futures spesso `null`, fallback a seconda-ultima close storica (`validCloses[validCloses.length - 2]`).

### Edge cases noti
- **Mac Mini OFFLINE**: Ollama proxy fallback solo MBP. Cache miss `/quote/:ticker` serve `{price:0, error: 'timeout'}` invece di throw. Asset analysis ha L2 disk persistente che sopravvive a restart.
- **Polymarket degraded** (`market-scanner.service.ts:46-48`): `pm.degraded=true` con `reason` ma anomalies array continua (resilient). Log warn.
- **Yahoo rate limit / 429**: mai esplicitato come pattern, ma `try/catch` con fallback `{price:0, changePct:0, error: 'Yahoo ${status}'}`. Soluzione futura: switch a FMP stream.
- **Gemini timeout 90s** (`asset-analysis.service.ts:374`): asset analysis può tardare. AbortController timeout. Log error `Gemini API error: ${status}` + return `{error: 'Gemini ${status}'}`.
- **Cache file corrotto / stale >7gg** (`market-data.controller.ts:62-67`): if-stale skip, regenera. Garanzia: mai serve dato vecchio >7gg.
- **Benchmark senza PositionHistory cliente** (`benchmark-alpha.service.ts:62-67`): `portSeries < 5` → return null + log warn. Cliente nuovo senza CSV import → no alpha.
- **Loopback IPv6 vs IPv4** (`market-data.controller.ts:429-433`): check `ip in ['127.0.0.1','::1','::ffff:127.0.0.1']`. Node 18+ DNS resolve `localhost` → IPv6 → check completo.

### Bug ricorrenti
- **Cache 900s `market.summary` esplosione costi** (`config/ai-routing.json:120-138`) se invalidata troppo frequentemente. Cron 30 min × 4 tenant × cloud cost = significativa. Mitigazione: TTL alto + cron `25,55` solo ws-api.
- **Hardcoded models nei service** (`market-data.controller.ts:337` + `asset-analysis.service.ts:376`): violation, sempre passare da `getModelForTask('asset-analysis')`. Audit periodico `scripts/nightly-audit.sh:45-67` flagga.
- **Race condition prefetcher + utente** (PRE commit `ee21117`, `asset-analysis.service.ts:257-289`): doppia generazione Gemini, doppia quota. Fix permanente: in-flight dedup map.
- **`/prefetch-asset-analysis` da remoto** (`market-data.controller.ts:419-444`): ritorna `{error: 'Only loopback callers allowed', ip}`. Per trigger remoto via cron Mini: `ssh crisesc@macmini64 'curl -X POST http://127.0.0.1:4010/market-data/prefetch-asset-analysis'`.
- **`/quote/:ticker` Cloudflare cache** (`market-data.controller.ts:107-151`): endpoint `@Public()` può finire in CDN cache. Verificare `Cache-Control: no-store` header se inconsistenze.

## 3 · SSOT — File fonte verità

| Cosa | Path assoluto |
|------|------------

…[truncato — apri il file MD per testo completo]