# Migration Plan: OpenAI Assistants API v2 → Responses API

Status: **IN PROGRESS** — adapter + per-flow wiring implemented behind the `OPENAI_API_MODE` flag (default `assistants`). See *Implementation status* below.
Goal: replace the Assistants/Threads/Runs flow with the Responses API **without a rewrite**, behind a feature flag, with per-thread-type rollout and instant rollback.

---

## Implementation status (2026-06-29)

**Done (flag-gated; default `assistants`, zero behavior change until flipped):**
- `config/constant.php`: added `OPENAI_API_MODE`, `RESPONSES_MODEL`, and the two inline prompts `instruction_assistant_prompt` / `review_assistant_prompt` (see *gap* below).
- `OpenAiTrait`: `usingResponsesApi()`, `generateResponse()`, `buildInputFromThread()`, `reshapeResponsesOutput()` + extractors, `extractMessageText()`. The lifecycle methods (`createAIThread`, `createMessage`, `createAssistant`, `updateAssistant`, `deleteAssistant`, `clearThread`) short-circuit to **local synthetic ids / no-ops** in responses mode — so the controllers (scenario authoring, chat send/audio) needed **no edits**.
- Generation wired per flow: student/public chat + trainer chat in `FetchMessageTrait`, review in the `app:fetch-message` command. Each branches on the flag; the legacy path is untouched.
- `tests/Unit/ResponsesAdapterTest.php`: locks the content-shape contract (reshaper + text extraction). Passing.

**Two plan revisions forced by the actual code (decided with the user):**

1. **D3 corrected — the `Handle*ChatMessage` jobs are dead code.** Nothing dispatches them (only `DeleteMergedAudio` is dispatched). There was never a double-generation risk from "two paths"; the real single owner per flow is already the **poll handler** (chat) / **command** (review). So instead of moving generation into the jobs, generation **stays in the poll handlers + command**, swapping `fetchLatestMessages` → `generateResponse(buildInputFromThread)`, guarded by an atomic `status 1→2` **claim** (`claimAndGenerate`) to stop concurrent polls double-generating. (This also fixes a latent duplicate-message race that existed under Assistants.) The jobs are left as-is (still legacy-only) for now.

2. **Instruction-source gap.** Two prompts lived **only on OpenAI's servers**, not in our DB: the trainer-chat instruction author (`INSTRUCTION_ASSISTANT_ID`) and the default review grader (`REVIEW_ASSISTANT_ID`). Responses passes instructions inline, so they now live in `config/constant.php` as `instruction_assistant_prompt` / `review_assistant_prompt`. **⚠️ These are currently empty — the real German prompt text must be pasted in (or set via env) before trainer chat / default-review work in responses mode.** Scenario student/public chat and *custom* review are unaffected (their instructions already live in the DB).

**Remaining before cutover:**
- Paste the two prompt texts (above).
- Phase 6 cleanup (delete Assistants methods / dead jobs) — deferred until after cutover.
- Manual dogf-ood of each flow with `OPENAI_API_MODE=responses` against a real key.

---

## 0. Guiding decisions (read first)

These three decisions shape everything below.

### D1 — No Prompt objects. Pass `instructions` inline.
The official guide migrates Assistants → **Prompts**, but:
- Prompts can **only be created in the dashboard**, not via API. Our scenarios are **trainer-authored at runtime** (`ScenariosController::store`), so a dashboard prompt-per-scenario is impossible.
- The guide itself flags **reusable prompt objects as deprecated** (2026-06-03 timeline).

**Therefore:** the instruction bundle stays in our DB (`scenario.instructions` + `rating_review_instructions`, wrapped with the Settings prefix/suffix — exactly as today) and is passed **inline** as the `instructions` parameter on every `responses.create` call. The API calls `createAssistant` / `updateAssistant` / `deleteAssistant` are **deleted**; scenario authoring becomes a pure local DB write.

### D2 — Conversation state = replay from our DB (not `previous_response_id`, not Conversations API).
We already persist every `Message` with `role`. Each request rebuilds the `input[]` array from the thread's messages. Benefits: provider-independent, no 30-day server-state expiry, fully testable offline, and avoids the Conversations-API dependency. Chats are short and auto-close daily, so full-history token cost is negligible. (Conversations API remains a documented fallback in §8.)

### D3 — Keep the async shell; move the model call into the Job/Command layer (single owner).
Today generation is driven by **two** paths that both call `fetchLatestMessages` — the frontend poll (`GET /fetch-message/{ulid}` → `FetchMessageTrait` handlers) **and** the queued Jobs / 5-sec command. Responses is synchronous, so if both call the model we'd **generate twice**. We consolidate to **one owner**: the Job (chat) / Command (review) makes the synchronous `responses.create`, persists the assistant `Message`, sets `status=0`, and socket-broadcasts. The frontend `fetch-message` endpoint becomes a **pure read** (return the persisted message if `status==0`, else "pending"). Controllers still return immediately. State machine, socket, frontend polling cadence — all preserved.

---

## 1. Surface area (what changes vs. what doesn't)

**Changes (concentrated in one trait + one controller):**
- `app/Traits/OpenAiTrait.php` — rewrite the Assistants methods as a Responses adapter.
- `app/Http/Controllers/ScenariosController.php` — `store/update/destroy` drop remote assistant calls.
- `app/Traits/FetchMessageTrait.php` — handlers become read-only (no OpenAI call); model call moves to jobs/command.
- `app/Jobs/Handle{Student,Trainer,RatingReview}ChatMessage.php` — call the model instead of polling a run.
- `app/Console/Commands/FetchMessage.php` — review generation becomes a direct call.

**Unchanged:**
- Audio: `openAiSpeechToText` (`/audio/transcriptions`) + `openAiTextToSpeech` (`/audio/speech`) — **not Assistants API, zero changes.**
- DB schema (no destructive migration — see §4).
- `CustomParsedown`, the JSON `{correction, answer, links}` contract, German prompts in `config/constant.php`.
- Thread/Message state-machine magic numbers (`status`, `chat_status`, `type`).
- Socket broadcast payloads to `SOCKET_URL/broadcast`.
- Controllers' send/audio entrypoints (they keep dispatching + returning immediately).

---

## 2. The adapter contract (critical — keeps blast radius small)

The downstream handlers read assistant output as:
```php
json_decode($value['content'][0]['text']['value'])   // → {correction, answer, links}
```
and audio extraction reads `json_decode($message->content, true)[0]["text"]["value"]`.

**The adapter MUST keep persisting `Message.content` in this exact nested shape**, so `parseAiChatMessage`, `handleReviewRatingResponse`, `handleTrainerChatResponse`, audio extraction, and history rendering all keep working untouched.

Responses returns the text at `response.output[0].content[0].text` (type `output_text`) or via the `output_text` convenience field. The adapter reshapes:

```
Responses output_text  ──►  [{ "type":"text", "text":{ "value": <output_text>, "annotations":[] } }]
```

Identifier mapping for the columns handlers read:
| Handler reads | Old (Assistants) | New (Responses) |
|---|---|---|
| `$value['id']` → `ai_message_id` | message id | `response.output[0].id` (item id) |
| `$value['run_id']` → `ai_run_id` | run id | `response.id` (the response id) |
| `thread.ai_thread_id` | thread id | local conversation key (unchanged value, no remote object) |

---

## 3. New `OpenAiTrait` shape

Add a config flag and branch. Keep old methods callable during rollout.

```php
// config/constant.php
'OPENAI_API_MODE' => env('OPENAI_API_MODE', 'assistants'), // 'assistants' | 'responses'
'RESPONSES_MODEL' => env('RESPONSES_MODEL', 'gpt-4o-mini'),
```

New methods (Responses mode):

- `buildInputFromThread(Thread $thread): array`
  Replays `Message` rows → `[{role, content:[{type:'input_text'|'output_text', text}]}]` (user→`input_text`, assistant→`output_text`, per the guide's backfill example).

- `generateResponse(string $instructions, array $input, array $opts = []): array`
  ```php
  POST {base}/responses
  {
    "model": config('constant.RESPONSES_MODEL'),
    "instructions": $instructions,        // inline — replaces the assistant
    "input": $input,                       // replayed history + new turn
    "text": { "format": { "type": "json_object" } },  // preserves JSON contract
    "store": false
  }
  ```
  Returns the **reshaped** array described in §2 (so callers persist it unchanged).

- Scenario authoring helpers become local no-ops returning a synthetic id so existing `['id']` access doesn't break:
  - `createAssistant($req)` → returns `['id' => 'local_'.Str::ulid()]`, writes nothing remote.
  - `updateAssistant`, `deleteAssistant`, `clearThread`, `createAIThread` → local/no-op (return synthetic ids / `true`).

The old Assistants methods stay until cutover is complete (flag-gated), then get deleted in §7.

---

## 4. Database — no destructive migration

Reuse columns; just change what we write into them.

| Column | New meaning under Responses | Migration? |
|---|---|---|
| `scenario.assistant_id`, `rating_review_assistant_id` | legacy; stop writing OpenAI ids (leave nullable, or write `local_*`) | none |
| `scenario.instructions`, `rating_review_instructions` | **source of truth** (already is) | none |
| `thread.ai_thread_id` | local conversation key | none |
| `thread.last_run_id`, `message.ai_run_id` | store `response.id` | none |
| `thread.last_message_id`, `message.ai_message_id` | store output item id | none |
| `thread.status` / `chat_status` / `type` | unchanged semantics | none |

Only additive change: env vars / config keys in §3. **No backfill needed** for scenarios — instructions already live in the DB. In-flight `status=1` threads drain on the old path (flag still `assistants` for them).

---

## 5. Per-flow changes

### 5a. Student chat (`type=3`) — do this first as the spike
- `ChatController::sendMessage/audioMessage`: unchanged (create user `Message`, `status=1`, dispatch `HandleStudentChatMessage`, return). Audio: `openAiSpeechToText` still runs first, transcript becomes the turn text.
- `HandleStudentChatMessage::handle`: replace `fetchLatestMessages` with `generateResponse($instructions, buildInputFromThread($thread))`, persist assistant `Message` (reshaped content), TTS if the user turn was audio, `correction` back-fill, `status=0`, socket broadcast. `$instructions` = scenario instructions (+ Settings prefix/suffix).
- `FetchMessageTrait::handleStudentChatMessage`: reduce to a **read** — if `status==0` return latest assistant message, else pending. (No OpenAI call → no double generation.)

### 5b. Public chat (`type=5`)
Same as 5a via `PublicChatController` + `handlePublicChatMessage`. Anonymous session unaffected.

### 5c. Trainer chat (`type=1`)
Lowest stakes (plain markdown, no JSON/correction). Use `text.format=text`. Good smoke-test candidate.

### 5d. Review / rating (`type=4`)
- `AutoCloseChat` / `StudentScenarioController::closeStudentChat`: build the chat-history message as today; instead of creating a review thread+run, dispatch the review job.
- `FetchMessage` command / `HandleRatingReviewChatMessage`: one `generateResponse($reviewInstructions, [chatHistory])` with `text.format=json_object`; parse `{thread_id, grade, review}`; update **source** thread `grade/review/chat_status=3`. The 5-sec poll loop is no longer needed for generation — repurpose the command to "dispatch pending review jobs" (keeps the `fetch_response_try < 2` retry guard).

---

## 6. Rollout (feature-flagged, reversible)

1. **Phase 0 — prep:** add config/env flag; confirm `queue:listen` worker runs in all envs; capture current behavior in tests (golden JSON outputs for the 4 handlers).
2. **Phase 1 — adapter:** implement Responses methods in `OpenAiTrait` + `buildInputFromThread` + reshaper. Unit-test the reshaper against §2 contract.
3. **Phase 2 — scenario authoring local:** `ScenariosController` store/update/destroy stop calling OpenAI (flag-gated). Verify create/edit/delete scenario works.
4. **Phase 3 — student chat behind flag** (5a), incl. consolidation to single-owner generation. Dogfood.
5. **Phase 4 — public + trainer + review** (5b/5c/5d).
6. **Phase 5 — cutover:** flip `OPENAI_API_MODE=responses` in prod; monitor Sentry.
7. **Phase 6 — cleanup:** delete Assistants methods + dead branches; optionally delete orphaned OpenAI assistants/threads via a one-off script.

Rollback at any phase = set flag back to `assistants`.

---

## 7. Risks & gotchas

- **Double generation** (the #1 trap): both the frontend poll and the job must NOT call the model. Enforce single-owner (D3) before flipping student/public chat.
- **Content-shape drift:** if the reshaper doesn't reproduce `content[0].text.value`, parsing, audio extraction, and history rendering all break silently → JSON fallback (`answer_status=1`). Cover with a test.
- **`run_id`/`thread_id` reads:** handlers read `$value['run_id']` and `thread.ai_thread_id`; adapter must populate them (response id / local key).
- **Token cost of full-history replay:** fine for short chats; add a prune-to-last-N guard if a chat ever runs long.
- **Latency moves into the job:** keep `->retry(3,1000)` + backoff; surface failures as `answer_status=1` as today.
- **Prompts & Conversations are both on deprecation tracks** — D1/D2 deliberately avoid both; we depend only on `responses.create` + our own DB.
- **Instruction wrapping:** preserve the Settings `instruction_prefix/suffix` + `review_instructions_prefix/suffix` wrapping when building inline `instructions`.
- **Model parity:** keep `gpt-4o-mini` initially to isolate API changes from model changes.

---

## 8. Alternative kept on the shelf

**Conversations API** (`POST /conversations`, then `responses.create` with `conversation` id): less per-call token cost on long chats, server-side item storage. Rejected as primary due to the deprecation nudge and added coupling, but it's a drop-in swap for `buildInputFromThread` if replay cost ever becomes a problem — store the conversation id in `ai_thread_id`.