voice agent

# valoswiss-voice-agent (31°)

**Macro-categoria**: 📞 DOMINI SINGOLI
**Scope**: Voice channel multilingual: call inbound/outbound (LiveKit + SIP/Twilio), real-time STT (Whisper/Voxtral/Canary), TTS (Cartesia/ElevenLabs), conversation orchestration (Pipecat), advisor digital twin voice cloning (consent-gated). Multilingua IT/DE/FR/EN nativo via Voxtral.
**Born**: 2026-05-03 (W1 sidecar + W2 NestJS module + W3 frontend + W4 admin/cron)
**Owner downstream**: ADVISOR (vista proprie call) · SUPERVISOR/ADMIN (cross-tenant + admin config)
**Last aligned**: 2026-05-03 V20

---

## §0 · Pre-flight check (entry rituale dell'agente)

Prima di ogni intervento, verifica in quest'ordine:

1. **Branch + working tree**
   ```bash
   cd ~/git/valoswiss && git status --short && git log -3 --oneline
   ```
2. **Sidecar Python health**
   ```bash
   curl -s http://127.0.0.1:8900/healthz | jq .
   ```
   Deve ritornare `{"status":"ok","version":"...","livekit_connected":true|false}`. Se 502/connection refused → sidecar PM2 down: `pm2 list | grep voice-agent-py`.
3. **NestJS proxy health**
   ```bash
   curl -s http://127.0.0.1:4010/api/voice-agent/health -H "Cookie: valo_token=<dev-token>"
   ```
   Deve ritornare `{ sidecar:{status:'ok'}, livekit:{connected:true}, circuitBreaker:{state:'closed', failures:0} }`.
4. **Prisma schema sync**
   ```bash
   cd apps/api && npx prisma migrate status
   ```
   Verifica che i 3 model `VoiceCall` / `VoiceTranscript` / `VoiceConsent` + enum `VoiceCallStatus` / `VoiceCallDirection` / `VoiceLanguage` siano applicati.
5. **Tenant configs**: `tenants/ws.json` e `tenants/az.json` devono avere `"voiceAgent": true`.
6. **Persona pack**: `apps/api/src/common/persona-packs/persona-packs.constants.ts` deve avere `'voiceAgent'` in `defaultModules` per `ADVISOR` + `RELATIONSHIP_MANAGER`.
7. **Module registry**: `apps/web/src/lib/module-registry.ts` deve esporre entry `voiceAgent` con `sidebarSection: 'COMUNICARE'`, `requiredRole: 'ADVISOR'`, `personaHint: 'communication'`, icon `📞`.
8. **R-Audit gate**: prima di qualsiasi commit su file CRITICAL (vedi §3), eseguire `npx tsx scripts/r-audit.ts <file> --validate-business-logic`.

Se uno qualunque degli 8 punti fallisce, **fermati e annota la deviazione** prima di procedere — la 3-Point Registration è invariante non negoziabile (vedi `feedback_new_module_registration.md`).

---

## §1 · Aree di competenza

### 1.1 Stack tecnico (repos di riferimento)

| Repo | Versione | Ruolo |
|---|---|---|
| **livekit/agents** (Apache 2.0, realtime voice) | latest | Core infra WebRTC + SIP/Twilio inbound/outbound |
| **pipecat-ai/pipecat** (BSD-2, multimodal pipeline) | latest | Conversation orchestration pipeline frame-based |
| **vocodedev/vocode-core** (MIT, phone+Zoom) | latest | Phone conversations Twilio + Zoom integration |
| **TEN-framework/ten-framework** (Apache 2.0) | latest | Multi-modal real-time agent framework |
| **SYSTRAN/faster-whisper** (MIT, CTranslate2) | latest | Local STT, fallback offline, bassa latenza CPU |
| **mistralai/Voxtral** (multilingual IT/DE/FR/EN) | latest | STT nativo quadrilingua, modello primario voice |
| **NVIDIA Canary-Qwen 2.5B** (#1 ASR leaderboard) | 2.5B | STT premium accuracy, tier UHNW calls |

### 1.2 Pipeline voice call

```
Inbound SIP/Twilio → LiveKit Room (WebRTC)
         ↓
  Pipecat Pipeline
    ├─ VAD (Voice Activity Detection) — silero-vad
    ├─ STT frame → Voxtral (IT/DE/FR/EN) | Canary-Qwen (UHNW) | faster-whisper (fallback offline)
    ├─ LLM brain → advisor-copilot (NestJS cascade) context call
    ├─ TTS frame → Cartesia (< 90ms latency) | ElevenLabs (voice clone consent-gated)
    └─ Audio out → LiveKit Room → caller
         ↓
  VoiceCall persist (Postgres)
  VoiceTranscript streaming chunks
  VoiceConsent check PRIMA di voice clone
```

### 1.3 Tier STT presets

| Tier | STT Primary | STT Fallback | TTS | Use case |
|---|---|---|---|---|
| `voice-standard` | Voxtral multilingual | faster-whisper local | Cartesia | Default ws+az advisor call |
| `voice-uhnw` | NVIDIA Canary-Qwen 2.5B | Voxtral | ElevenLabs voice clone | UHNW + Family Office consent-gated |
| `voice-offline` | faster-whisper local | — | Cartesia | Dev/test senza cloud STT |

Override env (priorità: env > tier preset):
- `VOICE_STT_PRIMARY_MODEL`
- `VOICE_STT_FALLBACK_MODEL`
- `VOICE_TTS_PROVIDER`
- `VOICE_UHNW_STT_MODEL`

### 1.4 Digital twin voice cloning (consent-gated)

Funzionalità advisor digital twin: TTS ElevenLabs con voiceprint clonato dall'advisor.

**Prerequisiti OBBLIGATORI**:
1. `VoiceConsent` record con `consentType: 'VOICE_CLONE'` + `status: 'GRANTED'` + `grantedAt` + firma digitale `consentSignature`
2. `voiceprintId` referenziato in `VaultPii` (vault-pii agent) — voiceprint NON in chiaro su DB
3. Check consent PRIMA di ogni chiamata con voice clone: se consent mancante o revocato → fallback Cartesia standard, log warning

**PROTOTYPE-PHASE**: consent-gate implementato ma MIFID II / normativa GDPR biometria come signal target post-prototipo, non hard gate bloccante in dev.

### 1.5 Lingua detection automatica

Voxtral detecta automaticamente IT/DE/FR/EN dal primo utterance. Il campo `detectedLanguage` su `VoiceTranscript` viene aggiornato al primo segmento STT. L'LLM brain (advisor-copilot) riceve `language` nel system prompt per rispondere nella lingua rilevata.

---

## §2 · Pattern di codice

### 2.1 NestJS Module structure

```
apps/api/src/modules/voice/
├─ voice.module.ts
├─ voice.controller.ts          # Roles ADVISOR/SUPERVISOR/ADMIN
├─ voice.service.ts             # facade
├─ voice.cron.ts                # @Cron('0 2 * * *') cleanup old sessions
├─ services/
│   ├─ call-runner.service.ts   # Prisma persist + Wave 1.6 getter espliciti
│   ├─ sidecar.client.ts        # circuit breaker 3-fail/30s → voice-agent-py :8900
│   ├─ transcript.service.ts    # streaming chunk persist + summary
│   ├─ consent.service.ts       # VoiceConsent check + vault-pii integration
│   └─ telegram-alerts.service.ts  # notifyCallCompleted, notifyCallFailed
├─ types/
│   ├─ voice-call.zod.ts
│   └─ voice-transcript.zod.ts
└─ README.md
```

### 2.2 Prisma schema (Wave 1.6 compliant)

```prisma
model VoiceCall {
  id                String          @id @default(cuid())
  tenantSlug        String
  direction         VoiceCallDirection
  status            VoiceCallStatus
  language          VoiceLanguage?
  detectedLanguage  VoiceLanguage?
  callerNumber      String?
  advisorUserId     String
  clientId          String?
  livekitRoomId     String?
  livekitRoomName   String?
  durationSeconds   Int?
  tier              String          @default("voice-standard")
  voiceCloneUsed    Boolean         @default(false)
  consentId         String?
  startedAt         DateTime        @default(now())
  endedAt           DateTime?
  createdAt         DateTime        @default(now())
  updatedAt         DateTime        @updatedAt
  transcripts       VoiceTranscript[]
  consent           VoiceConsent?   @relation(fields: [consentId], references: [id])

  @@index([tenantSlug, status])
  @@index([advisorUserId, startedAt(sort: Desc)])
  @@index([tenantSlug, startedAt(sort: Desc)])
}

model VoiceTranscript {
  id               String        @id @default(cuid())
  callId           String
  speakerRole      String        // 'ADVISOR' | 'CLIENT' | 'BOT'
  text             String
  detectedLanguage VoiceLanguage?
  sttModel         String
  confidenceScore  Float?
  segmentStart     Float
  segmentEnd       Float
  createdAt        DateTime      @default(now())
  call             VoiceCall     @relation(fields: [callId], references: [id])

  @@index([callId, segmentStart])
}

model VoiceConsent {
  id               String        @id @default(cuid())
  tenantSlug       String
  advisorUserId    String
  consentType      String        // 'VOICE_CLONE' | 'CALL_RECORDING' | 'TRANSCRIPT_STORAGE'
  status           String        // 'PENDING' | 'GRANTED' | 'REVOKED'
  voiceprintVaultId String?      // ref a VaultPii entry — NON voiceprint diretto
  consentSignature String?
  grantedAt        DateTime?
  revokedAt        DateTime?
  expiresAt        DateTime?
  createdAt        DateTime      @default(now())
  updatedAt        DateTime      @updatedAt
  calls            VoiceCall[]

  @@index([advisorUserId, consentType])
  @@index([tenantSlug, status])
}

enum VoiceCallStatus {
  RINGING
  ACTIVE
  COMPLETED
  FAILED
  MISSED
}

enum VoiceCallDirection {
  INBOUND
  OUTBOUND
}

enum VoiceLanguage {
  IT
  DE
  FR
  EN
}
```

**Wave 1.6 — getter espliciti obbligatori** su `TenantPrismaService`:
```typescript
// NON usare legacy cast as-any su prisma — pre-commit triage blocca
get voiceCall()       { return this.client.voiceCall; }
get voiceTranscript() { return this.client.voiceTranscript; }
get voiceConsent()    { return this.client.voiceConsent; }
```

### 2.3 NestJS controller pattern

```typescript
@Controller('voice-agent')
@UseGuards(JwtAuthGuard, RolesGuard)
export class VoiceController {
  constructor(private readonly voiceService: VoiceService) {}

  // ADVISOR: lista proprie call
  @Get('calls')
  @Roles('ADVISOR', 'RELATIONSHIP_MANAGER', 'SUPERVISOR', 'ADMIN')
  async listCalls(
    @Query() query: ListCallsQueryDto,
    @Request() req: AuthenticatedRequest,
  ) {
    const advisorUserId = req.user?.id;  // NON req.user.userId — Wave 1.6 field variant
    return this.voiceService.listCalls({ ...query, advisorUserId });
  }

  // ADVISOR: inizia outbound call
  @Post('calls/outbound')
  @Roles('ADVISOR', 'RELATIONSHIP_MANAGER')
  async startOutbound(
    @Body() dto: StartOutboundCallDto,
    @Request() req: AuthenticatedRequest,
  ) {
    const advisorUserId = req.user?.id;
    return this.voiceService.startOutbound({ ...dto, advisorUserId });
  }

  // Admin: health sidecar + LiveKit
  @Get('health')
  @Roles('ADVISOR', 'RELATIONSHIP_MANAGER', 'SUPERVISOR', 'ADMIN')
  async health() {
    return this.voiceService.health();
  }

  // Admin: consent management
  @Post('consent')
  @Roles('SUPERVISOR', 'ADMIN')
  async upsertConsent(@Body() dto: UpsertConsentDto) {
    return this.voiceService.upsertConsent(dto);
  }
}
```

### 2.4 Sidecar Python :8900 FastAPI

```python
# services/voice-agent-py/app.py
from fastapi import FastAPI
from pipecat.pipeline.pipeline import Pipeline
from pipecat.transports.services.livekit import LiveKitTransport

app = FastAPI(title="valoswiss-voice-agent", version="1.0.0")

@app.get("/healthz")
async def healthz():
    return {
        "status": "ok",
        "version": "1.0.0",
        "livekit_connected": _livekit_ok(),
        "stt_provider": os.getenv("VOICE_STT_PRIMARY_MODEL", "voxtral"),
        "tts_provider": os.getenv("VOICE_TTS_PROVIDER", "cartesia"),
    }

@app.post("/calls/start")
async def start_call(req: StartCallRequest):
    """Crea LiveKit room + avvia pipeline Pipecat"""
    room = await livekit_client.create_room(req.roomName)
    job_id = str(uuid4())
    asyncio.create_task(_run_pipeline(job_id, room, req))
    return {"jobId": job_id, "roomName": room.name, "status": "STARTING"}

@app.get("/calls/{job_id}/status")
async def call_status(job_id: str):
    state = _job_store.get(job_id)
    if not state:
        raise HTTPException(404, "job not found")
    return state

@app.post("/calls/{job_id}/end")
async def end_call(job_id: str):
    await _end_pipeline(job_id)
    return {"status": "ENDED"}

@app.post("/transcript/summary")
async def transcript_summary(req: SummaryRequest):
    """LLM summary dei transcript chunks — advisor-copilot LLM"""
    return await _summarize(req.transcriptChunks, req.language)
```

### 2.5 Pipeline Pipecat pattern

```python
async def _run_pipeline(job_id: str, room, req: StartCallRequest):
    transport = LiveKitTransport(
        url=LIVEKIT_URL,
        token=await generate_token(room.name, req.advisorUserId),
        room_name=room.name,
    )

    stt_service = _resolve_stt(req.tier)  # Voxtral | Canary | faster-whisper
    tts_service = _

…[truncato — apri il file MD per testo completo]