# Fire Truck Brakes - Workflow Log

## Story
Volunteer firefighter joke (9-clip comedy):
> A rural volunteer fire department's brakes fail, sending them careening uncontrollably toward a massive prairie fire. They crash directly into the inferno but manage to save the day. The grateful farmer rewards them with a check. When a reporter asks what they'll do with the money, the fire captain deadpans: "First thing we're gonna do is fix them brakes."

**Genre:** Action comedy with Michael Bay cinematography

---

## Tools & Scripts Used

### Image Generation
- **NanoBanana** (via LaoZhang API)
- Script: `microdrama-v2/scripts/generate_grid.py`
  - `--prompt` - path to story/scene prompt
  - `--reference` - characteristic shot for consistency
  - `--layout 3x3` - 9 scenes
  - `--aspect-ratio 9:16` - vertical format

### Video Generation
- **Veo3** via LaoZhang API (8-second clips with audio)
- Script: `microdrama-v2/scripts/generate_videos_v3.py`
  - `--model veo3` - Google Veo 3.1
  - `--clip N` - generate specific clip
  - Outputs: 720x1280 (upscaled from model) or 1080x1920 (manual)

### Frame Extraction
- **PIL/Pillow** for grid cropping
- 3x3 division with label handling

### Video Assembly
- **FFmpeg** for trimming, captioning, normalization, concatenation
- Script: `microdrama-v2/scripts/assemble_video_v3.py`
  - Reads `video_prompts.json` for clips and captions
  - Timed caption overlay with `drawtext` filter
  - Multiline caption support with `\n`
  - Normalizes clips to consistent format (1080x1920)
  - Manual trimming: `ffmpeg -i input.mp4 -t 6 -c copy output.mp4`

### Prompt Management
- Manual editing workflow: Edit `prompt.txt` in clip directories, then sync to `video_prompts.json`
- `prompt.txt` files saved for debugging/iteration

---

## Workflow Stages

### Stage 1: Characteristic Shot (v003)
**Key requirement:** TWO firefighters (not one) for consistency

Generated reference image showing:
- Two firefighters in turnout gear
- Prairie fire background
- Soot-covered, exhausted look
- Ages ~35-45 (no children allowed per Veo3 constraints)

### Stage 2: Grid Generation (v003)
- Layout: 3x3 (9 scenes)
- Resolution: High-res grid for frame extraction
- Style: "Ultrarealistic Michael Bay cinematography, ARRI Alexa footage, lens flares, film grain"

**Scenes:**
1. Young firefighter calling for help on radio
2. Fire truck racing toward camera (70 mph)
3. Truck driving into fire wall
4. Firefighters deploying hoses
5. Water battle against flames
6. Exhausted victory, truck in scorched field
7. Farmer handing check to captain
8. TV reporter interviewing captain
9. Captain's punchline closeup

### Stage 3: Frame Extraction (frames/003)
- Extracted 9 individual frames from grid
- Each frame: Reference for Veo3 generation

### Stage 4: Video Prompts (script/004)
Created `video_prompts.json` with:
- Single frame per clip (start_frame = end_frame)
- Detailed prompts for Veo3
- Audio descriptions (Veo3 generates synchronized audio)
- Caption overlay configuration

**Key constraints added:**
- "no subtitles, no on-screen text, no logos"
- Character consistency references
- Specific camera movements
- Audio atmosphere descriptions

### Stage 5: Video Generation (videos/003)

**Major challenge: Veo3 audio content filter**
- **PUBLIC_ERROR_AUDIO_FILTERED** triggered by explicit speech in prompts
- Solution: Use ambient audio only, remove direct speech quotes from audio section
- Example: Changed "A female reporter's voice asking: What will you do with the money" → "ambient wind, birds chirping, distant traffic sounds"

**Iteration challenges:**

1. **Static fire (Clip 1)**
   - Problem: Fire not moving, appears frozen
   - Fix: Emphasized "flames flickering, smoke billowing" and removed "no morphing" constraint

2. **Truck not moving (Clip 3)**
   - Problem: Truck stayed static despite "ROCKETS" and speed descriptions
   - Fix: Simplified language ("driving at high speed"), added "truck is moving forward, not staying"

3. **Incomplete action (Clip 4)**
   - Problem: Firefighters just running, not deploying hoses
   - Fix: Extended action sequence: "landing...starting to pull out...unrolling...connecting"

4. **Camera movement too fast (Clip 6)**
   - Problem: Zoom out too distant, lost subjects
   - Fix: Changed to "gentle slow pull-back, staying relatively close"

**Manual replacements:**
- Multiple clips manually regenerated with higher resolution (1080x1920)
- Mixed Veo3 720p and manual 1080p clips (normalized during assembly)

**Good clips saved:**
- Created `good/` subdirectory to preserve successful generations during iteration
- Final good clips: 2, 3, 5, 7, 8, 9

### Stage 6: Trimming
Manually trimmed specific clips for pacing:
- Clip 7: 8s → 6s (farmer handshake scene)
- Clip 8: 8s → 5s (reporter interview)

Command: `ffmpeg -i clip.mp4 -t 6 -c copy output.mp4`

### Stage 7: Assembly with Captions (final_v3.mp4)

**Caption configuration:**
- Multiline support using `\n` in text
- Timed overlays (0-5s or 1-7s per clip)
- White text on semi-transparent black background
- Position: Bottom of frame

**Final captions:**
1. Clip 1 (0-5s): "Help! Help!\n\"Send everybody!\""
2. Clip 5 (0-5s): "Push up, move forward now!"
3. Clip 6 (0-5s): "It was a tough one..."
4. Clip 8 (0-6s): "What will you do\nwith the money?"
5. Clip 9 (1-7s): "First thing we're gonna\ndo is fix them brakes."

**Mixed resolution handling:**
- Input: Mix of 720x1280 (Veo3) and 1080x1920 (manual)
- FFmpeg filter: `scale=1080:1920:force_original_aspect_ratio=decrease,pad=1080:1920:(ow-iw)/2:(oh-ih)/2,setsar=1`
- Output: Consistent 1080x1920 for all clips

---

## File Structure

```
/mnt/video/microdrama-v3/fire-truck-brakes/
├── story/
│   └── 001/
│       ├── story.txt              # Original joke
│       ├── story-analysis.json    # Claude analysis
│       └── scenes.json            # 9 scenes breakdown
├── characteristic/
│   └── 003/reference.jpg          # TWO firefighters
├── grids/
│   └── 003/grid.jpg               # 3x3 scene grid
├── frames/
│   └── 003/frame_01-09.jpg        # Extracted frames
├── script/
│   └── 004/video_prompts.json     # Prompts + captions config
├── videos/
│   └── 003/
│       ├── clip_01-09/
│       │   ├── video.mp4          # Generated or trimmed clip
│       │   ├── prompt.txt         # Prompt used (for debugging)
│       │   ├── video_captioned.mp4  # With caption overlay
│       │   └── video_normalized.mp4 # Re-encoded (no caption)
│       ├── good/                  # Saved successful clips
│       │   ├── clip_02/
│       │   ├── clip_03/
│       │   ├── clip_05/
│       │   ├── clip_07/video.mp4  # Trimmed to 6s
│       │   ├── clip_08/video.mp4  # Trimmed to 5s
│       │   └── clip_09/
│       ├── fire_truck_brakes_final_v3.mp4  # ← FINAL OUTPUT
│       └── fire_truck_brakes_v003_assembly_trimmed.mp4  # No captions
└── CLAUDE.md                      # Project documentation
```

---

## Iteration Summary

**Generation rounds:**
- **Round 1:** Initial 9 clips generated (some failed, some static)
- **Round 2:** Regenerated clips 1, 3, 4, 6, 8 with refined prompts
- **Round 3:** Regenerated clips 1, 4, 6 (final refinements)
- **Manual:** User provided higher-resolution replacements for clips 1, 3, 4, 5, 6, 8

**Prompt refinement strategy:**
1. User edits `prompt.txt` in clip directories
2. Read edited prompts and update `video_prompts.json`
3. Regenerate specific clips with updated prompts
4. Save good clips to `good/` directory
5. Repeat until satisfactory

**Total clips generated:** ~30+ (9 clips × 3-4 iterations + tests)

---

## Key Issues & Solutions

### 1. Veo3 Audio Content Filter
**Problem:** `PUBLIC_ERROR_AUDIO_FILTERED` error when prompts contained explicit speech
**Solution:** Remove speech quotes from audio section, use ambient audio descriptions only
**Example:** "A female reporter's voice asking: What will you do with the money" → "ambient wind, birds chirping"

### 2. Static Fire/Movement
**Problem:** Background fire appeared frozen, no flickering
**Solution:**
- Remove "no morphing" constraints
- Emphasize active verbs: "flames flickering, smoke billowing"
- Repeat movement descriptions

### 3. Truck Staying Still
**Problem:** Truck described as moving but remained static in video
**Solution:**
- Use simple language: "driving" instead of "ROCKETS"
- Add explicit negation: "truck is moving forward, not staying"
- Specify camera relation: "truck gets smaller and smaller as it moves away"

### 4. Incomplete Actions
**Problem:** Actions cut short (firefighters run but don't deploy hoses)
**Solution:** Extend action sequence with multiple connected steps: "landing...starting to pull out...unrolling...connecting"

### 5. Mixed Resolution Assembly
**Problem:** Clips from different sources had different resolutions (720p vs 1080p)
**Solution:** FFmpeg scale+pad filter normalized all to 1080x1920 during assembly

### 6. Caption Text Escaping
**Problem:** Multiline captions needed proper newline handling
**Solution:** assemble_video_v3.py properly escapes `\n` for FFmpeg drawtext filter

---

## Final Output Specifications

**File:** `fire_truck_brakes_final_v3.mp4`
- **Duration:** 67.5 seconds
- **Resolution:** 1080x1920 (9:16 portrait)
- **File size:** 82 MB
- **Format:** H.264 (libx264), AAC audio
- **Captions:** 5 timed overlays with multiline support

**Clip durations:**
- Clips 1-6: 8s each
- Clip 7: 6s (trimmed)
- Clip 8: 5s (trimmed)
- Clip 9: 8s

---

## Cost Estimate (Approximate)

- Characteristic shot (NanoBanana): $0.05
- 3x3 Grid (NanoBanana): $0.08
- 30+ Veo3 generations @ ~$0.30 each: $9.00
- **Total: ~$9.13**

(Note: Multiple iterations due to content refinement)

---

## Lessons Learned

1. **Veo3 audio filter is sensitive** - Avoid explicit speech transcriptions in audio descriptions
2. **Simple language works better** - "driving" beats "ROCKETS" for movement
3. **Emphasize negatives** - "not staying" helps counter static generation tendency
4. **Save good clips immediately** - Create `good/` directory to prevent loss during iteration
5. **Manual editing workflow** - Editing `prompt.txt` files directly, then syncing to JSON works well
6. **Mixed resolutions are manageable** - FFmpeg normalization handles 720p + 1080p mixing
7. **Trim after generation** - Easier to trim full 8s clips than regenerate with exact durations
8. **Test one clip first** - But in practice, batch generation happened anyway
9. **Multiline captions** - Use `\n` in JSON, assemble_video_v3.py handles escaping
10. **Character count matters** - Two firefighters required explicit "TWO" in characteristic shot prompt

---

## Prompt Engineering Patterns That Worked

### For Movement
❌ "The truck ROCKETS forward at insane speed"
✅ "The truck is DRIVING FORWARD, moving away from camera at speed...truck is moving forward, not staying"

### For Dynamic Elements
❌ "Massive fire in background"
✅ "Massive prairie fire raging in background - flames flickering, smoke billowing"

### For Audio (Veo3)
❌ "The young man's urgent voice: Help! Send everybody!"
✅ "roaring fire, crackling flames, wind"

### For Camera
❌ "Zoom out"
✅ "Camera zooms out and we see old battered truck in center of scorched black field"

### For Action Sequences
❌ "Firefighters deploy hoses"
✅ "Two firefighters landing on the ground and starting to pull out fire hoses from the truck, unrolling them, connecting them"

---

## Next Steps / Improvements

1. **Audio refinement** - Consider adding music or sound effects in post
2. **Color grading** - Consistent color across mixed-source clips
3. **Transition effects** - Currently hard cuts, could add crossfades
4. **Caption styling** - Custom fonts, colors per scene mood
5. **Thumbnail generation** - Extract frame for social media
6. **Aspect ratio variants** - Generate 16:9 for YouTube, 1:1 for Instagram

---

## References

- Main project docs: `/home/videobot/dev/CLAUDE.md`
- Confession Twist workflow: `microdrama-v2/docs/workflow-logs/confession-twist-workflow.md`
- Veo3 proxy: `scripts/laozhang_proxy.py`
- Video generation: `microdrama-v2/scripts/generate_videos_v3.py`
- Assembly script: `microdrama-v2/scripts/assemble_video_v3.py`
