# V5 Orchestrator Implementation Status

**Date**: 2026-02-07
**Approach**: Pipeline-based (simpler than V4's task queue)

## Architecture Overview (V5 vs V4)

### V4 Plan (Not Implemented)
- Task queue with dependencies (`task_queue.json`)
- Complex dependency tracking
- Multiple task types (generate, await_feedback, batch operations)
- Cascade regeneration logic

### V5 Actual (Implemented)
- **pipeline.json** - simple stage → iteration mapping
- **No dependencies** - just track selected iteration per stage
- **No symlinks** - pipeline.json is single source of truth
- **Iteration directories** - immutable log with metadata.json
- **Simple commands** - 8 commands total (init, run, new, refine, chat, select, rate, status)

## Core Data Model

### ✅ DONE: pipeline.json

```json
{
  "02-understanding": "005",
  "03-script": "006",
  "04-assets": "004",
  "05-references": "001",
  "06-characteristic": "006",
  "07-grids": "057",
  "08-script-adapted": "003",
  "09-frames": null,
  "10-videos": null,
  "11-captions": null,
  "12-assembly": null
}
```

**Status**: ✅ Implemented in `lib/pipeline.py`
- `load_pipeline()` / `save_pipeline()`
- `find_next_stage()`
- `resolve_inputs()` - gets parent iterations from pipeline
- `PIPELINE_STAGES` - ordered list of all stages
- `STAGE_DEPENDENCIES` - which parents each stage needs

### ✅ DONE: metadata.json (per iteration)

```json
{
  "stage": "07-grids",
  "iteration": "057",
  "created_at": "2026-02-07T14:20:24Z",
  "based_on": {
    "03-script": "006",
    "04-assets": "004",
    "06-characteristic": "006"
  },
  "refined_from": "056",
  "rating": null,
  "result": "07-grids/057/grid.jpg",
  "generation_mode": "chat",
  "model_prompt": "direct-chat",
  "model_image": "nanobanana-gemini3pro"
}
```

**Status**: ✅ Implemented
- `save_metadata()` in `lib/__init__.py`
- Tracks `based_on` (parent iterations)
- Tracks `refined_from` (refinement chain)
- Generation mode tracking

### ✅ DONE: Feedback Storage

**Feedback files**: `{iteration}/feedback.txt`

**Status**: ✅ Implemented
- Saved automatically when `feedback` parameter provided
- Used by `get_feedback_history_for_claude()` to build context

## CLI Commands

### ✅ DONE: Core Commands

| Command | Status | Implementation |
|---------|--------|----------------|
| `init` | ✅ DONE | Creates story dir, session.txt, pipeline.json |
| `run` | ✅ DONE | Generates next empty stage (find_next_stage) |
| `new STAGE [FEEDBACK]` | ✅ DONE | Fresh iteration from current pipeline parents |
| `refine STAGE ITER FEEDBACK` | ✅ DONE | Refine specific iteration with feedback |
| `rate STAGE ITER N` | ✅ DONE | Set rating (1-5) |
| `select STAGE ITER` | ✅ DONE | Select iteration in pipeline (validates consistency) |
| `status` | ✅ DONE | Show pipeline state table |
| `migrate` | ✅ DONE | Migrate v4 take to v5 |

### ✅ DONE: Stage-Specific Commands

| Command | Status | Stages | Notes |
|---------|--------|--------|-------|
| `chat STAGE ITER FEEDBACK` | ✅ DONE | 07-grids | Direct chat with NanoBanana (preserves thought_signature) |

### ⏳ TODO: Convenience Commands

| Command | Status | Purpose |
|---------|--------|---------|
| `accept STAGE ITER` | ❌ TODO | Shortcut: select + run in one command |

## Stage Executors

### ✅ DONE: Implemented Stages

All stage executors follow V5 pattern:
```python
def execute(take_path, story_name, inputs, feedback=None, refined_from=None) -> dict:
    """
    Args:
        inputs: {"03-script": "006", "04-assets": "004"}  # Parent iterations from pipeline
        feedback: User feedback text
        refined_from: Iteration number being refined

    Returns:
        {"status": "success", "iteration": "007", "result": "path", "user_message": "..."}
    """
```

| Stage | Status | Modes | File |
|-------|--------|-------|------|
| **02-understanding** | ✅ DONE | new, refine | `stages/stage_02_understanding.py` |
| **03-script** | ✅ DONE | new, refine | `stages/stage_03_script.py` |
| **04-assets** | ✅ DONE | new, refine | `stages/stage_04_assets.py` |
| **05-references** | ✅ DONE | new | `stages/stage_05_references.py` |
| **06-characteristic** | ✅ DONE | new, refine | `stages/stage_06_characteristic.py` |
| **07-grids** | ✅ DONE | new, refine, chat | `stages/stage_07_grids.py` |
| **08-script-adapted** | ✅ DONE | new, refine | `stages/stage_08_script_adapted.py` |
| **09-frames** | ❌ TODO | - | Not yet implemented |
| **10-videos** | ❌ TODO | - | Not yet implemented |
| **11-captions** | ❌ TODO | - | Not yet implemented |
| **12-assembly** | ❌ TODO | - | Not yet implemented |

### Stage-Specific Features

#### ✅ Stage 07: Grid Generation (Multi-Mode)

**Three generation modes implemented**:

1. **new** - Fresh generation from characteristic shot + optional feedback
   - Uses only characteristic shot as reference image
   - Optional feedback as guidance

2. **refine** - Regenerate with ALL accumulated feedback
   - Collects feedback from all iterations using same characteristic shot
   - Most recent feedback has highest priority
   - Full prompt regeneration by Claude Opus 4.5

3. **chat** - Direct chat with NanoBanana (iterative refinement)
   - Claude refines user feedback first (adds position labels, clarifies)
   - Passes refined feedback directly to NanoBanana chat API
   - Preserves `thought_signature` for multi-turn conversations
   - Good for subtle changes (head turns, expressions)
   - Not good for structural changes (moving objects, swapping positions)

**Visual style handling**:
- Reads `style.visual` from script.json
- `"cinematic"` → photorealistic
- `"animated"` or `"cartoon"` → illustrated style
- `"stylized"` → artistic interpretation

**Reference images**:
- NEW/REFINE modes: Uses only characteristic shot
- CHAT mode: No reference images (model has context from history)

**Implementation files**:
- `stages/stage_07_grids.py` - Main executor
- `prompts/stage_grid.txt` - Prompt template
- `scripts/nanobanana_text2img.py` - API wrapper with thought_signature preservation

#### ✅ Stage 08: Script Adaptation (New Stage)

**Purpose**: Adapt original script to match what was ACTUALLY generated in the grid

**Why needed**: NanoBanana may interpret scenes differently than intended
- Characters may look older/younger
- Compositions may vary
- Objects positioned differently
- Shot types may differ (close-up → medium)

**Two generation modes**:

1. **new** - Fresh adaptation from grid
   - Claude views grid image with vision
   - Compares with original script
   - Writes adapted script matching actual frames

2. **refine** - Refine existing adaptation
   - Adjusts scene descriptions based on feedback
   - Preserves overall structure

**Output**:
- `script.json` - Adapted script with updated descriptions
- `user_message.txt` - Summary of adaptations
- `feedback_request.json` - Accept/Refine/Regenerate options

**Key additions to script**:
- `frame_observations` - Notes differences from original
- `upscale_prompt` - NanoBanana prompt for frame upscaling (1080x1920)
- `adaptation_notes` - High-level summary of changes
- `grid_analysis` - Consistency observations

**Implementation files**:
- `stages/stage_08_script_adapted.py` - Executor
- `prompts/stage_script_adapted.txt` - Prompt template (269 lines)

## Library Modules

### ✅ DONE: Core Libraries

| Module | Status | Purpose |
|--------|--------|---------|
| `lib/pipeline.py` | ✅ DONE | Pipeline state, stage dependencies, input resolution |
| `lib/iteration.py` | ✅ DONE | List iterations, read/save metadata |
| `lib/consistency.py` | ✅ DONE | Validate selection (check based_on vs pipeline) |
| `lib/feedback_history_v5.py` | ✅ DONE | Walk refined_from chain for Claude context |
| `lib/path_utils_v3.py` | ✅ EXISTS | Path utilities (get_next_iteration, ensure_dir) |
| `lib/__init__.py` | ✅ DONE | Common utilities (save_metadata, logging) |

### ❌ TODO: Additional Libraries

| Module | Status | Purpose |
|--------|--------|---------|
| `lib/batch.py` | ❌ TODO | Batch operations (9/16 frames, videos in parallel) |
| `lib/api_router.py` | ❌ TODO | Route to different APIs (kling-direct, pollo, veo3) |

## Workflow Comparison: V4 Plan vs V5 Actual

### V4 Plan (Task Queue Approach)

```bash
# User approves grid
python orchestrator_v4.py "fire-truck" --feedback "approve" --run

# Output:
# Task 6: adapt script to grid → completed
# Task 7: extract 9 frames → completed
# Task 8: await_feedback (frame 1) → WAITING
#
# Review: /mnt/video/.../09-frames/scene_01/001/frame.jpg
```

**Issues with V4 approach**:
- Complex task queue management
- Many task types (generate, await_feedback, batch, etc.)
- Dependency tracking is manual and error-prone
- Queue can become corrupted

### V5 Actual (Pipeline Approach)

```bash
# User selects grid
python orchestrator_v5.py . select 07-grids 057
python orchestrator_v5.py . run

# Output:
# Generating: 08-script-adapted
# Based on: {"03-script": "006", "07-grids": "057"}
# ...
# Generated: 08-script-adapted/001
# To select: orchestrator_v5.py . select 08-script-adapted 001
```

**Advantages of V5**:
- Simple: just track which iteration is selected per stage
- No complex dependency graph
- Pipeline.json is easy to understand and edit
- Each command does one thing well

## Testing & Verification

### ✅ Tested on Real Story

**Story**: `/mnt/video/microdrama-v3/20260204-water-walking-dog`

**Test Results**:

| Stage | Iterations | Selected | Notes |
|-------|-----------|----------|-------|
| 02-understanding | 5 | 005 | ✅ |
| 03-script | 6 | 006 | ✅ |
| 04-assets | 4 | 004 | ✅ |
| 05-references | 1 | 001 | ✅ |
| 06-characteristic | 8 | 006 | ✅ (including feedback refinements) |
| 07-grids | 57 | 057 | ✅ (including chat mode iterations) |
| 08-script-adapted | 3 | 003 | ✅ NEW - just implemented |
| 09-frames | 1 | ??? | ❌ Not implemented |
| 10-videos | 1 | ??? | ❌ Not implemented |

### ✅ Tested Features

**Grid Generation Modes**:
- ✅ `new` mode with feedback - Grid 057
- ✅ `refine` mode - Collected 26 feedback entries
- ✅ `chat` mode - Multiple iterations with thought_signature preservation
- ✅ Visual style reading from script.json (cinematic → photorealistic)

**Script Adaptation**:
- ✅ `new` mode - Initial adaptation (iteration 002)
- ✅ `refine` mode - Refined with scene-specific feedback (iteration 003)
- ✅ Vision analysis - Claude viewed grid and compared with original script
- ✅ feedback_request.json generation

**Commands**:
- ✅ `status` - Shows clean table with iteration counts
- ✅ `select` - Validates consistency
- ✅ `refine` - Works for all stages
- ✅ `new` - Fresh iterations

## Remaining Work

### High Priority (Needed for Complete Pipeline)

#### ❌ Stage 09: Frame Extraction

**Purpose**: Extract individual frames from grid for video generation

**Implementation needed**:
- Extract 9 frames from 3x3 grid (or 16 from 4x4)
- Upscale each frame to 1080x1920 using NanoBanana
- Use `upscale_prompt` from adapted script
- Support refine mode for individual frame fixes

**Dependencies**: 08-script-adapted

#### ❌ Stage 10: Video Generation

**Purpose**: Generate video for each frame

**Implementation needed**:
- Support multiple APIs:
  - kling-direct (JWT auth) - **working in V3**
  - kling-pollo (Pollo proxy)
  - veo3-laozhang (LaoZhang API)
  - veo3-pollo (Pollo proxy)
- Parallel generation (9 or 16 videos)
- Read scene info from adapted script
- Generate per scene, not batch

**Dependencies**: 09-frames

**V3 scripts to adapt**:
- `scripts/generate_videos_kling_direct.py` - Working Kling implementation

#### ❌ Stage 11: Caption Generation

**Purpose**: Generate caption overlays from script + video metadata

**Implementation needed**:
- Read dialogue from adapted script
- Match timing to video duration
- Generate caption files (SRT or ASS format)

**Dependencies**: 10-videos

**V3 scripts to adapt**:
- `scripts/generate_captions_v3.py`

#### ❌ Stage 12: Assembly

**Purpose**: Concatenate videos with captions into final video

**Implementation needed**:
- Concat all videos in order
- Apply caption overlays
- Generate final video with audio

**Dependencies**: 11-captions

**V3 scripts to adapt**:
- `scripts/assemble_video_v3_kling.py` - Working for Kling

### Medium Priority (Convenience)

#### ⏳ Accept Command

```bash
orchestrator_v5.py . accept 08-script-adapted 003
# → selects 003 in pipeline.json
# → runs next stage automatically
```

**Current workaround**: `select` then `run` (two commands)

#### ⏳ Batch Status Commands

```bash
orchestrator_v5.py . status --verbose        # Show all iteration details
orchestrator_v5.py . history STAGE           # Show iteration history
orchestrator_v5.py . compare STAGE ITER1 ITER2  # Compare two iterations
```

### Low Priority (Nice to Have)

#### ⏳ Web UI

Simple web interface to:
- View pipeline status
- Browse iterations
- Provide feedback via forms
- View generated files

#### ⏳ Telegram Bot Integration

Adapt V3 bot to use V5 orchestrator:
- Send generation outputs to Telegram
- Collect feedback via bot
- Run orchestrator in background

## Key Decisions & Rationale

### Why Pipeline Instead of Task Queue?

**V4 Plan (Task Queue)**:
- Complex: 30+ tasks for 9-scene video
- Dependencies between tasks
- Multiple task types (generate, await_feedback, batch)
- Hard to debug when queue corrupts

**V5 Actual (Pipeline)**:
- Simple: 12 stages, each has selected iteration
- No dependencies in data model (handled by code)
- Easy to understand: `{"07-grids": "057"}`
- Easy to debug: just edit pipeline.json

### Why No Symlinks?

**V3/V4**: Used `current` symlink to track active iteration
**V5**: pipeline.json is single source of truth

**Advantages**:
- No filesystem state (just JSON file)
- Easy to backup/restore (copy one file)
- No broken symlinks
- Works on Windows (symlinks are problematic)

### Why Immutable Iterations?

**Never delete iterations** - always create new ones

**Advantages**:
- Complete history for debugging
- Can analyze what feedback led to what changes
- Training data for future models
- Users can manually revert by changing pipeline.json

### Why feedback_request.json?

**Purpose**: Standard format for user feedback collection

**Advantages**:
- Consistent across all stages
- Easy to parse by bots/UI
- Supports multiple feedback types (accept, refine, regenerate)
- Can add stage-specific options (e.g., restart_from)

## File Structure

```
microdrama-v2/
├── scripts-v4/
│   ├── orchestrator_v5.py                 # ✅ Main CLI entry point
│   ├── stages/
│   │   ├── stage_02_understanding.py      # ✅ DONE
│   │   ├── stage_03_script.py             # ✅ DONE
│   │   ├── stage_04_assets.py             # ✅ DONE
│   │   ├── stage_05_references.py         # ✅ DONE
│   │   ├── stage_06_characteristic.py     # ✅ DONE
│   │   ├── stage_07_grids.py              # ✅ DONE (new/refine/chat modes)
│   │   ├── stage_08_script_adapted.py     # ✅ DONE (new/refine modes)
│   │   ├── stage_09_frames.py             # ❌ TODO
│   │   ├── stage_10_videos.py             # ❌ TODO
│   │   ├── stage_11_captions.py           # ❌ TODO
│   │   └── stage_12_assembly.py           # ❌ TODO
│   └── lib/
│       ├── pipeline.py                    # ✅ DONE
│       ├── iteration.py                   # ✅ DONE
│       ├── consistency.py                 # ✅ DONE
│       ├── feedback_history_v5.py         # ✅ DONE
│       ├── batch.py                       # ❌ TODO
│       └── api_router.py                  # ❌ TODO
├── prompts/
│   ├── stage_understanding.txt            # ✅ EXISTS
│   ├── stage_script.txt                   # ✅ EXISTS
│   ├── stage_assets.txt                   # ✅ EXISTS
│   ├── stage_characteristic.txt           # ✅ EXISTS
│   ├── stage_grid.txt                     # ✅ DONE (updated for modes)
│   ├── stage_script_adapted.txt           # ✅ DONE (new)
│   ├── stage_frames.txt                   # ❌ TODO
│   ├── stage_videos.txt                   # ❌ TODO
│   ├── stage_captions.txt                 # ❌ TODO
│   └── stage_assembly.txt                 # ❌ TODO
└── docs/
    └── plans/
        ├── v4-automated-workflow.md       # ✅ Original plan (task queue approach)
        ├── v5-orchestrator-status.md      # ✅ This document
        └── stage-workflow.md              # ✅ Stage documentation

/mnt/video/microdrama-v3/{story}/001/
├── pipeline.json                          # ✅ V5 format
├── session.txt                            # ✅ Claude session ID
├── pipeline.log                           # ✅ Structured log (JSONL)
├── 02-understanding/
│   ├── 001/, 002/, ..., 005/             # ✅ Multiple iterations
│   └── (no 'current' symlink)            # ✅ V5: no symlinks
├── 03-script/
├── 04-assets/
├── 05-references/
├── 06-characteristic/
├── 07-grids/
│   └── 057/                               # ✅ Selected iteration
│       ├── grid.jpg                       # ✅ Generated grid
│       ├── nanobanana_prompt.txt          # ✅ Prompt used
│       ├── chat_history.json              # ✅ NanoBanana conversation
│       ├── metadata.json                  # ✅ V5 format
│       └── feedback.txt                   # ✅ User feedback
├── 08-script-adapted/                     # ✅ NEW STAGE
│   └── 003/                               # ✅ Selected iteration
│       ├── script.json                    # ✅ Adapted script
│       ├── user_message.txt               # ✅ Summary
│       ├── feedback_request.json          # ✅ Feedback options
│       └── metadata.json                  # ✅ V5 format
├── 09-frames/                             # ❌ TODO
├── 10-videos/                             # ❌ TODO
├── 11-captions/                           # ❌ TODO
└── 12-assembly/                           # ❌ TODO
```

## Success Metrics

### ✅ Completed (Stages 02-08)

- ✅ Simple pipeline.json data model
- ✅ No symlinks - pipeline.json is authority
- ✅ All iterations preserved
- ✅ Metadata tracks based_on and refined_from
- ✅ 8 CLI commands working
- ✅ Stages 02-08 implemented
- ✅ Three-mode grid generation (new/refine/chat)
- ✅ Script adaptation stage (new to V5)
- ✅ Feedback history accumulation
- ✅ feedback_request.json generation

### ❌ Remaining (Stages 09-12)

- ❌ Frame extraction (Stage 09)
- ❌ Video generation with API routing (Stage 10)
- ❌ Caption generation (Stage 11)
- ❌ Final assembly (Stage 12)
- ❌ Accept command shortcut
- ❌ Batch operations library
- ❌ Telegram bot integration

## Next Steps

### Immediate (Continue Pipeline)

1. **Implement Stage 09: Frame Extraction**
   - Extract frames from grid
   - Upscale with NanoBanana
   - Support refine mode for individual frames

2. **Implement Stage 10: Video Generation**
   - Adapt `generate_videos_kling_direct.py` to V5
   - Add API routing (kling-direct, pollo, veo3)
   - Parallel video generation

3. **Implement Stages 11-12: Captions & Assembly**
   - Adapt existing V3 scripts to V5 pattern
   - Generate final video

### Short-term (Polish)

1. **Add `accept` command**
2. **Add batch operations library**
3. **Write integration tests**

### Long-term (Integration)

1. **Telegram bot integration**
2. **Web UI for pipeline visualization**
3. **Multi-story support**

## Conclusion

**V5 orchestrator is 67% complete** (8 of 12 stages).

The pipeline approach is **simpler and more maintainable** than V4's task queue:
- ✅ Easy to understand (just JSON mapping)
- ✅ Easy to debug (edit pipeline.json)
- ✅ Easy to extend (add new stages)
- ✅ No complex dependency tracking

**Next priority**: Implement stages 09-12 to complete the full video generation pipeline.
