# Microdrama V3 - Claude-Orchestrated Workflow

## Core Concept

**Scripts are thin orchestrators. Claude does all the intelligence work.**

Each script:
1. Creates directory structure
2. Writes prompt file for Claude
3. Invokes `claude --session-id {story}_{take}_{stage} -p "$(cat prompt.txt)"`
4. Claude reads context, generates output, saves files
5. Script validates output, updates metadata, proceeds to next stage

**Timeline**: Days, not weeks. Claude does the work, scripts just organize.

---

## Key Changes from V2

1. **Audio/Dialogue**: Kling v2.6+ generates dialogue natively
2. **Single Frame**: Each clip uses ONE reference frame
3. **Direct Kling API**: No Pollo proxy
4. **Take-Based Structure**: `001/`, `002/` at root
5. **Stage Iterations**: Each stage has `001/`, `002/` with `current` symlinks
6. **Claude Sessions**: Each stage uses persistent Claude session ID
7. **Prompt-Driven**: Scripts generate prompts, Claude executes

---

## Directory Structure

```
/mnt/video/microdrama-v3/{story-name}/
├── pipeline.log                    # Central JSONL log
├── story.md                        # Source material
├── CLAUDE.md                       # Instructions for Claude
├── 001/                            # Take 1
│   ├── 01-story/
│   │   ├── 001/
│   │   │   ├── story.md
│   │   │   ├── prompt.txt          # Prompt fed to Claude
│   │   │   ├── session.txt         # Session ID used
│   │   │   └── metadata.json
│   │   └── current → 001
│   ├── 02-understanding/
│   │   ├── 001/
│   │   │   ├── prompt.txt          # "Read story.md, analyze for visual adaptation"
│   │   │   ├── understanding.md    # Claude's output
│   │   │   ├── session.txt         # {story}_001_understanding
│   │   │   ├── feedback.md         # User feedback (if any)
│   │   │   └── metadata.json
│   │   ├── 002/                    # If user gave feedback
│   │   └── current → 002
│   ├── 03-script/
│   │   ├── 001/
│   │   │   ├── prompt.txt
│   │   │   ├── script.json         # Claude's output
│   │   │   ├── session.txt
│   │   │   └── metadata.json
│   │   └── current → 001
│   ├── 04-assets/
│   ├── 05-references/
│   ├── 06-characteristic/
│   ├── 07-grids/
│   ├── 08-script-adapted/
│   ├── 09-frames/
│   ├── 10-videos/
│   │   ├── scene_01/
│   │   │   ├── 001/
│   │   │   │   ├── frame.jpg       # Reference frame
│   │   │   │   ├── prompt.txt      # Video generation prompt
│   │   │   │   ├── video.mp4
│   │   │   │   └── metadata.json
│   │   │   └── current → 001
│   │   └── good/                   # Selected clips
│   │       └── scene_01 → ../scene_01/current
│   └── 11-assembly/
├── 002/                            # Take 2 (full redo)
└── current → 001
```

---

## CLAUDE.md File

Each story has a `CLAUDE.md` file at root with instructions:

```markdown
# Claude Instructions for {Story Name}

You are helping create a short-form video (45-60 seconds) from this story.

## Your Capabilities

- **Read any file**: Use Read tool on any path shown
- **List directories**: Use Glob tool with patterns like `*` or `**/*.json`
- **Generate images**: Call NanoBanana API via helper scripts
- **Generate videos**: Call Kling API via helper scripts
- **Edit files**: Use Edit or Write tools
- **View images**: Can see images directly

## Current State

- **Take**: {current_take}
- **Stage**: {current_stage}
- **Session ID**: {story}_{take}_{stage}

## File Locations

- Story source: `story.md`
- Current take: `{take}/`
- Previous iterations: `{stage}/001/`, `{stage}/002/`, etc.
- Pipeline log: `pipeline.log`

## Your Task

Read `{take}/{stage}/current/prompt.txt` for your specific task.

## Key Principles

1. **ONE action per scene** - no compound actions
2. **Simple, direct language** - "driving" not "ROCKETS forward"
3. **Explicit negatives** - "moving forward, NOT staying"
4. **Dialogue in Kling** - characters can speak, specify text
5. **Grid consistency** - use reference images, repeat character descriptions

## Lessons from V2

- Veo3 blocks explicit speech → Use Kling for dialogue
- Characteristic shot validates visual direction → Generate BEFORE grid
- Normalize videos before concat → Mixed formats cause freezing
- Save good clips during iteration → Prevents loss

## Feedback from Previous Takes

{feedback_summary}

## Available Scripts

- `call_nanobanana.py` - Generate images
- `call_kling_api.py` - Generate videos with audio
- `search_yandex.py` - Find references

When you're done, summarize what you created.
```

---

## Workflow Stages

### Stage 0: Initialize Take

**Script**: `init_take_v3.py`

```bash
python scripts/init_take_v3.py "fire-truck-brakes"
# → Creates /mnt/video/microdrama-v3/fire-truck-brakes/001/
# → Writes CLAUDE.md
# → Initializes pipeline.log
```

**What it does:**
1. Creates directory structure for take 001
2. Copies `story.md` to `001/01-story/001/`
3. Generates `CLAUDE.md` with project context
4. Logs initialization to `pipeline.log`

**No Claude invocation** - just setup.

---

### Stage 1: Story Understanding

**Script**: `run_stage_v3.py --stage understanding`

```bash
python scripts/run_stage_v3.py "fire-truck-brakes" --stage understanding
```

**What it does:**
1. Creates `002-understanding/001/` directory
2. Writes `prompt.txt`:
```
Read ../01-story/current/story.md.

Analyze this story for video adaptation:

1. **Retelling**: Summarize in your own words
2. **Meaning**: What's the point/punchline?
3. **What makes it work**: Comedic/dramatic mechanism
4. **Visual potential**: What can be shown vs just words
5. **Dialogue potential**: Key lines that should be SPOKEN (Kling can generate speech)
6. **What WON'T work**: Perceptual illusions, compound actions, etc.

Write your analysis to `understanding.md` in this directory.
```

3. Invokes Claude:
```bash
claude --session-id fire_truck_brakes_001_understanding \
  -p "$(cat 001/02-understanding/001/prompt.txt)"
```

4. Waits for Claude to finish, creates `understanding.md`
5. Displays to user: "Review understanding? (y/n/edit)"
6. If edit: User edits, script creates iteration 002
7. Updates metadata, logs to pipeline.log

**Claude does**: Read story, analyze, write understanding.md

---

### Stage 2: Script Generation

**Script**: `run_stage_v3.py --stage script`

```bash
python scripts/run_stage_v3.py "fire-truck-brakes" --stage script
```

**Prompt for Claude:**
```
Read ../02-understanding/current/understanding.md.

Create a script with 8-10 scenes for this story.

Output format: JSON saved to `script.json`

```json
{
  "title": "Story Title",
  "style": {
    "visual": "cinematic-realistic | cartoon | etc",
    "genre": "comedy | horror | drama",
    "tone": "action-comedy | dark-comedy | etc"
  },
  "scenes": [
    {
      "number": 1,
      "duration": 8,
      "action": "Single clear action - ONE verb",
      "what_is_shown": "Literal description",
      "shot_type": "wide | medium | close-up",
      "camera": "static | slow-push | orbit",
      "dialogue": {
        "character": "character_id",
        "text": "What they say (Kling will generate audio)",
        "delivery": "urgent | calm | comedic"
      },
      "audio_ambient": "background sounds"
    }
  ],
  "total_duration": 72
}
```

Key rules:
- ONE action per scene
- Simple language ("driving" not "ROCKETS")
- Dialogue in separate field (Kling generates audio)
- 5-10 seconds per scene
- Hook in first 3 seconds
```

**Claude does**: Read understanding, create script.json with scenes and dialogue

---

### Stage 3: Assets Identification

**Prompt for Claude:**
```
Read ../03-script/current/script.json.

Identify all assets needed:
- Characters (name, age, appearance, clothing)
- Locations (setting descriptions)
- Objects (props, especially cultural-specific)

For dialogue characters, add:
- Voice description ("deep, authoritative")
- Emotional range

Output: `assets.json`
```

**Claude does**: Extract assets from script, create assets.json

---

### Stage 4: Reference Search

**Script**: `run_stage_v3.py --stage references`

**What it does:**
1. Reads `assets.json`
2. For each asset needing references:
   - Calls `search_yandex.py {query}`
   - Downloads top 5 images
   - Displays to user: "Select reference for {asset}: (1-5)"
   - Copies selected to `selected.jpg`

**No Claude invocation** - direct API calls + user selection

---

### Stage 5: Characteristic Shot

**Prompt for Claude:**
```
You need to generate a characteristic shot to validate visual direction.

Read:
- ../03-script/current/script.json
- ../04-assets/current/assets.json
- ../05-references/*/selected.jpg (view these images)

Create a NanoBanana prompt that shows:
- Main character in main location
- Key objects
- Visual style from script

Use this reference images for consistency.

Call: `call_nanobanana.py --prompt "your prompt" --reference ../05-references/.../selected.jpg --output shot.jpg`

Save your prompt to `prompt.txt`.
```

**Claude does**:
- Reads script, assets, views references
- Generates NanoBanana prompt
- Calls helper script to generate image
- Saves prompt.txt

**Script after Claude finishes**:
- Displays image to user: "Accept? (y/n)"
- If no: creates iteration 002, user provides feedback

---

### Stage 6: Grid Generation

**Prompt for Claude:**
```
Generate a 3x3 or 4x4 grid of all scenes.

Read:
- ../03-script/current/script.json (for scene descriptions)
- ../06-characteristic/current/shot.jpg (as reference)

Grid layout:
- 9 scenes or fewer → 3x3
- 10-16 scenes → 4x4

Create NanoBanana prompt:
- 9:16 vertical format
- Each frame labeled "frame 1", "frame 2", etc.
- Use characteristic shot as reference
- SAME character descriptions across frames
- ONE action per frame

Call: `call_nanobanana.py --prompt "your prompt" --reference ../06-characteristic/current/shot.jpg --output grid.jpg --aspect-ratio 9:16 --size 4K`

Save prompt to `prompt.txt`.
```

**Claude does**: Generate grid prompt, call NanoBanana, get grid.jpg

---

### Stage 7: Script Adaptation

**Purpose**: Adapt the original script to match what was actually generated in the grid.

**Why needed**: NanoBanana may interpret scenes differently than intended. Characters may look different, compositions may vary. Before video generation, we need a script synchronized with the ACTUAL visual frames.

**Prompt for Claude:**
```
Adapt the script to match the generated grid frames.

Read:
- ../03-script/current/script.json (original script - the intent)
- ../07-grids/current/grid.jpg (generated grid - the reality)
- ../07-grids/current/nanobanana_prompt.txt (what we asked for)

Your task:
1. View the grid image with your vision capabilities
2. Analyze each frame (1-9 for 3x3, or 1-16 for 4x4):
   - What is ACTUALLY shown?
   - Character appearances (age, clothing, features, expressions)
   - Location details (objects, lighting, layout)
   - Object states (intact, destroyed, positioned where?)
   - Camera angle (actual shot type: ECU, CU, medium, wide)
   - Action captured in this frozen moment

3. Compare with original script scenes
4. Create adapted script.json with SAME structure but descriptions matching actual frames

Key adaptations:
- "what_is_shown": Describe ACTUAL frame content
- "shot_type": Match ACTUAL camera angle
- Character descriptions: Match ACTUAL appearance
- Object states: Match ACTUAL condition and position

Add "frame_observations" field to each scene noting differences from original.

Save to: script.json
```

**Claude does**: View grid, analyze frames, write adapted script.json

**Output**: `08-script-adapted/001/script.json` - script synchronized with actual grid frames

---

### Stage 8: Frame Extraction

**Script**: `run_stage_v3.py --stage frames`

**What it does:**
1. Reads `../07-grids/current/grid.jpg`
2. Reads metadata to get layout (3x3 or 4x4)
3. Uses PIL to crop grid into individual frames
4. Saves as `frame_01.jpg` ... `frame_09.jpg` (or frame_16.jpg)

**No Claude invocation** - simple image cropping

---

### Stage 9: Video Generation

**Prompt for Claude:**
```
Generate videos for all scenes using Kling API.

For each scene in ../08-script-adapted/current/script.json:

1. Read scene N details
2. Get reference frame: `../09-frames/current/frame_N.jpg`
3. Build Kling API request:
   - Reference image: frame_N.jpg
   - Prompt: [Visual style] + [what_is_shown] + [action] + [camera] + [audio]
   - If scene has dialogue: Include in prompt + audioPrompt
   - Duration: scene.duration
   - generateAudio: true

4. Call: `call_kling_api.py --image frame_N.jpg --prompt "your prompt" --duration 8 --audio "ambient + dialogue" --output video.mp4`

5. Save prompt to `scene_N/001/prompt.txt`

Do this for all scenes.

After each scene, show preview and ask me if it's acceptable.
```

**Claude does**:
- Iterates through scenes
- Generates video prompt for each
- Calls Kling API helper
- Shows each video to user
- Handles regeneration if user says "no"

**This is the longest stage** - user reviews each scene, provides feedback.

---

### Stage 10: Select Good Clips

**Script**: `run_stage_v3.py --stage select`

**What it does:**
1. Lists all scenes with their current iterations
2. For each: "Use scene N iteration 001? (y/n)"
3. Creates symlink in `good/` directory

**No Claude invocation** - simple user interaction

---

### Stage 11: Assembly

**Prompt for Claude:**
```
Assemble final video from good clips.

Read all clips from: `../10-videos/good/scene_*/video.mp4`

Steps:
1. Check all clips exist
2. Use FFmpeg to normalize (same resolution/encoding):
   `ffmpeg -i input.mp4 -vf "scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:(ow-iw)/2:(oh-ih)/2,setsar=1" -c:v libx264 -preset medium -crf 18 -c:a aac -b:a 192k output.mp4`

3. Create concat.txt:
   ```
   file 'scene_01/video.mp4'
   file 'scene_02/video.mp4'
   ...
   ```

4. Concatenate:
   `ffmpeg -f concat -safe 0 -i concat.txt -c copy final.mp4`

5. Report: duration, filesize, resolution

Save final.mp4 in this directory.
```

**Claude does**: Normalize clips, create concat file, run FFmpeg, generate final.mp4

---

## Script Design

### Core Orchestration Script

**`run_stage_v3.py`** - Runs any stage

```python
#!/usr/bin/env python3
"""
Orchestrate a stage execution with Claude.

Usage:
  run_stage_v3.py STORY --stage STAGE [--take TAKE] [--feedback "text"]
"""

import argparse
import subprocess
from pathlib import Path
import json

STAGES = {
    "understanding": {
        "dir": "02-understanding",
        "prompt_template": "prompts/stage_understanding.txt",
        "output_file": "understanding.md"
    },
    "script": {
        "dir": "03-script",
        "prompt_template": "prompts/stage_script.txt",
        "output_file": "script.json"
    },
    # ... etc
}

def run_stage(story, stage, take="current", feedback=None):
    """
    1. Create iteration directory
    2. Write prompt.txt from template
    3. Invoke Claude with session ID
    4. Validate output
    5. Update metadata
    6. Log to pipeline.log
    """
    base_path = Path(f"/mnt/video/microdrama-v3/{story}")

    if take == "current":
        take = (base_path / "current").readlink().name

    stage_config = STAGES[stage]
    stage_dir = base_path / take / stage_config["dir"]

    # Find next iteration number
    iterations = [d.name for d in stage_dir.iterdir() if d.is_dir()]
    next_iter = f"{len(iterations) + 1:03d}"

    iter_dir = stage_dir / next_iter
    iter_dir.mkdir(parents=True, exist_ok=True)

    # Write prompt
    prompt_template = Path(stage_config["prompt_template"]).read_text()
    if feedback:
        prompt_template += f"\n\nUSER FEEDBACK:\n{feedback}"

    prompt_file = iter_dir / "prompt.txt"
    prompt_file.write_text(prompt_template)

    # Write session ID
    session_id = f"{story}_{take}_{stage}"
    (iter_dir / "session.txt").write_text(session_id)

    # Update CLAUDE.md with current state
    update_claude_md(story, take, stage)

    # Invoke Claude
    print(f"Running Claude for {stage}...")
    result = subprocess.run([
        "claude",
        "--session-id", session_id,
        "--cwd", str(iter_dir),
        "-p", prompt_template
    ], capture_output=True, text=True)

    # Validate output exists
    output_file = iter_dir / stage_config["output_file"]
    if not output_file.exists():
        print(f"ERROR: Expected output {output_file} not created")
        return False

    # Update symlink
    current_link = stage_dir / "current"
    if current_link.exists():
        current_link.unlink()
    current_link.symlink_to(next_iter)

    # Save metadata
    metadata = {
        "stage": stage,
        "take": take,
        "iteration": next_iter,
        "timestamp": datetime.now().isoformat(),
        "session_id": session_id,
        "feedback": feedback,
        "output_file": str(output_file)
    }
    (iter_dir / "metadata.json").write_text(json.dumps(metadata, indent=2))

    # Log to pipeline.log
    log_entry = {
        "timestamp": datetime.now().isoformat(),
        "take": take,
        "stage": stage,
        "iteration": next_iter,
        "action": "generate",
        "status": "success",
        "metadata_path": str(iter_dir / "metadata.json")
    }
    with open(base_path / "pipeline.log", "a") as f:
        f.write(json.dumps(log_entry) + "\n")

    print(f"✓ {stage} completed: {output_file}")
    return True
```

### Helper Scripts

**`call_nanobanana.py`** - Wrapper for image generation
```python
#!/usr/bin/env python3
import sys
import requests
import base64

# Called by Claude with: call_nanobanana.py --prompt "..." --output shot.jpg
```

**`call_kling_api.py`** - Wrapper for video generation
```python
#!/usr/bin/env python3
import sys
import requests
import base64
import time

# Called by Claude with: call_kling_api.py --image frame.jpg --prompt "..." --duration 8 --audio "..." --output video.mp4
```

**`search_yandex.py`** - Wrapper for reference search
```python
#!/usr/bin/env python3
import sys
import requests

# Called by script (not Claude): search_yandex.py "1960s fire truck"
```

---

## Logging System

**`pipeline.log`** - JSONL format

```json
{"timestamp": "2026-02-02T16:00:00Z", "take": "001", "stage": "understanding", "iteration": "001", "action": "generate", "status": "success"}
{"timestamp": "2026-02-02T16:05:00Z", "take": "001", "stage": "understanding", "iteration": "002", "action": "regenerate", "feedback": "More comedic tone"}
{"timestamp": "2026-02-02T16:10:00Z", "take": "001", "stage": "script", "iteration": "001", "action": "generate", "status": "success"}
```

---

## Feedback Collection

**Within-stage feedback**:
```bash
python run_stage_v3.py "fire-truck-brakes" --stage characteristic --feedback "Make captain older"
# → Creates iteration 002 with feedback in prompt
```

**Cross-stage feedback** (go back to earlier stage):
```bash
python run_stage_v3.py "fire-truck-brakes" --stage script --feedback "More action"
# Script warns: "This will invalidate stages 4-10. Continue? (y/n)"
# Creates new iteration in stage 3
# User must re-run stages 4-10
```

**New take** (full redo):
```bash
python init_take_v3.py "fire-truck-brakes" --new-take --feedback "Make it darker and more suspenseful"
# → Creates take 002/
# → Includes feedback from take 001 in CLAUDE.md
```

---

## Implementation Approach: Iterative with Real Examples

### Phase 1: Core Infrastructure

**Create the foundation:**
1. `init_take_v3.py` - Directory creation, CLAUDE.md generation
2. `run_stage_v3.py` - Core orchestration script (basic version)
3. Metadata and logging utilities
4. Path validation helpers

**Deliverable**: Can initialize a story, create take structure, invoke Claude with session IDs

---

### Phase 2-11: Stage-by-Stage with Example Story

**User provides an example story, then for each stage:**

1. **Manual exploration**:
   - User runs Claude manually with session ID
   - We experiment with prompts
   - See what works, what doesn't
   - Learn optimal prompt structure

2. **Script creation**:
   - Create `run_stage_v3.py` logic for this stage
   - Create prompt template based on what worked
   - Test script with same story

3. **Validation**:
   - Run script on story
   - Compare output to manual approach
   - Refine if needed

4. **Move to next stage**:
   - Repeat process for next stage
   - Build on lessons learned

**Example Flow:**

```bash
# Phase 1 done: Core infrastructure exists

# User provides story
echo "A firefighter gets a reward..." > story.md

# Stage 1: Understanding (manual)
python init_take_v3.py "fire-truck-brakes"
# → Directory created, but no understanding script yet

# We manually run Claude to explore
claude --session-id fire_truck_brakes_001_understanding -p "Read story.md and analyze..."
# → Try different prompts, see what works

# Once we know what works, create the script
# → Write run_stage_v3.py logic for understanding stage
# → Create prompts/stage_understanding.txt template

# Test automated approach
python run_stage_v3.py "fire-truck-brakes" --stage understanding
# → Compare to manual results

# ✓ Understanding stage done! Move to stage 2

# Stage 2: Script generation (manual exploration)
claude --session-id fire_truck_brakes_001_script -p "Read understanding.md and create script..."
# → Experiment with script structure, dialogue format

# Create script automation
# → Update run_stage_v3.py for script stage
# → Create prompts/stage_script.txt

# Test
python run_stage_v3.py "fire-truck-brakes" --stage script

# ✓ Script stage done! Move to stage 3

# ... continue through all stages
```

**Benefits:**
- Learn optimal prompts through real examples
- Scripts are based on proven approaches, not theory
- Can adjust based on what actually works
- Each stage builds on previous learnings
- Trial and error is explicit and documented

**Timeline**:
- Phase 1 (core): 1 day
- Phases 2-11 (stages): 1-2 stages per day = 5-7 days
- **Total**: ~1 week iteratively building with real examples

---

## Critical Files

### New Files to Create

**Scripts:**
1. `microdrama-v2/scripts/init_take_v3.py`
2. `microdrama-v2/scripts/run_stage_v3.py` (CORE - orchestrates everything)
3. `microdrama-v2/scripts/call_nanobanana.py`
4. `microdrama-v2/scripts/call_kling_api.py`
5. `microdrama-v2/scripts/search_yandex.py`
6. `microdrama-v2/scripts/status_v3.py`
7. `microdrama-v2/scripts/rollback_v3.py`
8. `microdrama-v2/scripts/cleanup_v3.py`

**Prompt Templates:**
9. `microdrama-v2/prompts/stage_understanding.txt`
10. `microdrama-v2/prompts/stage_script.txt`
11. `microdrama-v2/prompts/stage_assets.txt`
12. `microdrama-v2/prompts/stage_characteristic.txt`
13. `microdrama-v2/prompts/stage_grid.txt`
14. `microdrama-v2/prompts/stage_videos.txt`
15. `microdrama-v2/prompts/stage_assembly.txt`

**Templates:**
16. `microdrama-v2/templates/CLAUDE.md.template`

**Documentation:**
17. `microdrama-v2/docs/plans/v3-workflow.md` (this file)
18. `microdrama-v2/docs/claude-orchestration.md`

---

## Environment Variables

```bash
# Direct Kling API (user provides)
KLING_API_KEY=your_key
KLING_API_URL=https://api.klingai.com/v1/video/generate

# Existing
LAOZHANG_NANOBA_KEY=your_key
YANDEX_XML_API_KEY=your_key
```

---

## Questions for User

1. **Kling API**: Documentation, endpoint, key format?
2. **Session persistence**: How long should Claude sessions last? Per-stage or per-take?
3. **Grid size preference**: Default 3x3 or 4x4?
4. **Scene duration**: 5s, 8s, or 10s default?
5. **Cost budget**: What's acceptable per story? (~$3-5 estimated)

---

**End of Plan**
