# Confession Twist - Workflow Log

## Story
Two-sentence horror story:
> "Forgive me, Father, for I have sinned," he whispered, recounting the murder in calm, exact detail - the address, the knife, how long she twitched before going still.
> A knock on the wood panel interrupted him: "Father, confession's about to start," and he said amen, slid the screen open, and waited.

**Twist:** The man confessing IS the priest. He was rehearsing his own confession before his shift.

---

## Tools & Scripts Used

### Image Generation
- **NanoBanana** (Gemini 3 Pro) via LaoZhang API
- Script: `microdrama-v2/scripts/generate_grid.py`
  - `--prompt` - path to prompt.txt
  - `--reference` - characteristic shot for consistency
  - `--layout 4x4` - grid size
  - `--aspect-ratio 9:16` - vertical format

### Video Generation
- **Kling v2.6** via Pollo API (with audio generation)
- Script: `microdrama-v2/scripts/generate_videos_v3.py`
  - `--model kling-v26` - Kling 2.6 with audio
  - `--clip N` - generate single clip for testing
  - `--no-audio` - disable audio generation

### Frame Extraction
- **PIL/Pillow** for cropping grid into individual frames
- Simple 4x4 division with label trimming

### Video Assembly
- **FFmpeg** for captioning and concatenation
- Script: `microdrama-v2/scripts/assemble_video_v3.py`
  - Reads `video_prompts.json` for clip order and captions
  - Adds timed captions with `drawtext` filter
  - Normalizes ALL clips to same format before concat (critical!)
  - `--no-captions` - skip caption overlay
  - `--no-reencode` - use stream copy (causes issues with mixed formats)

---

## Workflow Stages

### Stage 1-5: Pre-production
- Story saved, analyzed, script created (16 scenes)
- Assets identified, characteristic shot generated

### Stage 6: Grid Generation (v004)
**Iterations:**
- v001-003: Previous attempts with issues
- v004: Final grid with story context in prompt

**Key learnings:**
- Include full story context in NanoBanana prompt
- Reference image critical for character consistency
- Specify "NO WEDDING RINGS" (priests can't marry)

**Issues fixed:**
- Frame 10: Changed from "reaction to knock" to "hand knocking from outside"
- Frame 12: Changed from "rising + reaching" (two actions) to "hands adjusting collar" (one action)

### Stage 8: Frame Extraction (frames/004)
- Grid: 3072x5504 pixels
- 4x4 division = 768x1376 per frame
- Trimmed 30px from bottom for labels (kept labels for draft)

### Stage 9: Video Prompts (script/004)
**Format change:** Single frame per clip (not start+end frame method)

Each clip prompt includes:
- Visual action description
- Audio hints for Kling's generateAudio feature
- "NO speech, NO music" directive

### Stage 10: Video Generation (videos/002)

**Model comparison (clip 1 test):**
| Model | Resolution | Duration | Size | Audio |
|-------|-----------|----------|------|-------|
| Veo3 | 720×1280 | 8s | 1.3MB | No |
| Kling O1 | 1096×1888 | 5s | 8.2MB | No |
| Kling v2.6 | 1088×1904 | 5s | 5.4MB | Yes |

**Final choice:** Kling v2.6 with audio generation

**New function added:** `call_kling_v26_pollo()` in `scripts/pollo_proxy.py`

### Stage 11: Assembly (assembly/001)

**Major issues encountered:**

1. **Mixed format concatenation bug**
   - Problem: Non-captioned clips showed still frames from last captioned clip
   - Cause: Captioned clips were re-encoded (h264), originals had different encoding
   - Fix: Normalize ALL clips to same format before concat

2. **Apostrophe escaping bug**
   - Problem: Clip 11 caption "confession's" broke FFmpeg
   - Fix: Replace straight apostrophe with curly quote (Unicode \u2019)

3. **Duration mismatch**
   - Expected: 80s (16 × 5s)
   - Got: 75.7s (missing ~1 clip worth)
   - Fix: Normalizing all clips fixed this

---

## File Structure (Current - Messy)

```
/mnt/video/microdrama-v3/confession-twist/
├── story.md
├── understanding.md
├── assets.json
├── script/
│   ├── 001/script.json         # Initial
│   ├── 002/video_prompts.json  # 8-clip format (start+end)
│   ├── 003/script.json         # No wedding rings
│   ├── 004/video_prompts.json  # 16-clip format (single frame) ← CURRENT
│   └── current → 003           # Wrong symlink!
├── characteristic/001/
├── grids/
│   ├── 001-003/                # Old iterations
│   ├── 004/grid.jpg            # ← CURRENT
│   └── current → 004
├── frames/
│   ├── 001-003/                # Old extractions
│   ├── 004/frame_01-16.jpg     # ← CURRENT (with labels)
│   └── current → 004
├── videos/
│   ├── 001/                    # Veo3 test
│   ├── 001_kling/              # Kling O1 test (8 clips)
│   └── 002/                    # Kling v2.6 final (16 clips) ← CURRENT
│       └── clip_XX/
│           ├── video.mp4           # Original from Kling
│           ├── video_captioned.mp4 # With caption (if applicable)
│           └── video_normalized.mp4# Re-encoded (if no caption)
└── assembly/
    └── 001/
        ├── concat.txt
        └── final.mp4           # ← FINAL OUTPUT (80.7s, 32.7MB)
```

---

## Cleanup Needed for Next Iteration

1. Remove old grid/frame versions (001-003)
2. Remove test video directories (001, 001_kling)
3. Fix script/current symlink
4. Decide on frame labels (remove for production)
5. Consider single videos/ directory structure

---

## Captions in Final Video

| Clip | Time | Caption |
|------|------|---------|
| 1 | 0-4s | "Forgive me, Father, for I have sinned..." |
| 3 | 0-3s | "The address..." |
| 5 | 0-3s | "The knife..." |
| 7 | 0-4s | "How long she twitched before going still..." |
| 10 | 1-3s | *KNOCK* |
| 11 | 0-4s | "Father, confession's about to start." |
| 13 | 2-4s | "Amen." |
| 15 | 1-5s | And he waited. |

---

## Cost Estimate

- Characteristic shot: ~$0.05 (NanoBanana)
- 4x4 Grid: ~$0.10 (NanoBanana)
- 16 videos × Kling v2.6: ~$2.40
- **Total: ~$2.55**

---

## Lessons Learned

1. **Always normalize clips before concatenation** - mixing re-encoded and original causes freezing
2. **Escape apostrophes** - use curly quotes for FFmpeg drawtext
3. **Single frame method** works better than start+end for this style
4. **Kling v2.6 audio** adds atmosphere without dialogue
5. **Keep iterations** but clean up after confirming final works
6. **Test one clip first** before generating all 16
