subocr service

Service Overview: SubOCR (https://subocr.drheck.dev/) is an end-to-end Japanese burned-in video subtitle (telop / テロップ) extraction, Kanji furigana annotation, LLM study-note refinement, and interactive video study platform running on host ul, accelerated by the AMD Radeon AI PRO R9700 (gfx1201, RDNA 4, 32 GB VRAM) GPU.

1. Architecture & Network Topology

SubOCR is deployed as a containerized FastAPI + ONNX Runtime (ROCm 7.2 MIGraphX) service bound to a dedicated IPv6 ULA address on eth77, reverse-proxied through Nginx with Authentik ForwardAuth single sign-on, and paired with the local ollama-rocm (qwen2.5:32b) LLM engine on the same GPU.

• Public URL: https://subocr.drheck.dev/
• Dedicated ULA IPv6 Address: fd77:7777:5010:0:1337:f1ac:d00d:0c70 (eth77, port 8000)
• Host Directory: /data/docker/subocr/
• Database: /data/docker/subocr/data/subocr.db (SQLite 3 in WAL mode)
• Export & Sidecar Storage:
  • Per-job exports: /data/docker/subocr/data/jobs/<job_id>/
  • Automatic sidecar writes next to local videos: /data/language/Japanese/Video/... (mounted read-write at /videos inside the container)
• Authentication: Authentik ForwardAuth (subocr ProxyProvider, 30-day token validity) via /etc/nginx/sites-available/subocr
• DNS Resolution:
  • Local Unbound (danknasty, @10.0.0.1): A 10.0.0.100, AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa
  • Public Authoritative DNS (deSEC): AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa

2. Two-Stage GPU Processing Pipeline

When a job is submitted from https://subocr.drheck.dev/new (either by selecting a local video under /data/language/Japanese/Video, uploading a video file, or providing a YouTube URL fetched via yt-dlp), SubOCR executes a two-stage GPU pipeline on the AMD Radeon AI PRO R9700 (gfx1201):

Stage 1: ROCm 7.2 MIGraphX Visual Subtitle OCR (PP-OCRv5)

1. Frame Sampling & Spatial Crop:
   • OpenCV samples the video at a configurable rate $f_s$ (default $f_s = 2.0\text{ fps}$, or interval $\Delta t = 0.5\text{ s}$).
   • To isolate burned-in subtitles and ignore top-of-screen variety-show topic banners by default, frames are cropped to the lower subtitle region ($y \in [0.58H, 0.98H]$).
2. GPU Inference (MIGraphXExecutionProvider FP32):
   • Detection (PP-OCRv5_server_det_infer.onnx) and recognition (japanese_PP-OCRv5_mobile_rec_infer.onnx) run via onnxruntime-rocm with MIGraphXExecutionProvider.
   • Compiled MIGraphX kernel plans (.mxr) are cached in /data/docker/subocr/cache/migraphx so subsequent container restarts bypass AOT compilation.
   • Precision Note: migraphx_fp16_enable is explicitly set to 0 (FP32) because FP16 causes bounding-box coordinate underflow in PP-OCRv5_server_det on gfx1201.
3. Bilingual Line Classification & Spatial Role Separation:
   • Detected text boxes within each frame are sorted vertically and classified as Japanese (ja_lines) or English (en_lines) based on character script ratios.
   • Within the lower subtitle band, higher-positioned smaller Japanese commentary boxes ($y_{\text{rel}} < 0.66$) are separated from the primary bottom speaker dialogue ($y_{\text{rel}} \ge 0.66$) so simultaneous on-screen editor commentary and speaker subtitles are preserved cleanly.
4. Temporal Deduplication:
   • Consecutive frames whose normalized Japanese string similarity satisfies: $$\text{sim}(S_{t}, S_{t-1}) = \frac{2 \cdot |M|}{|S_t| + |S_{t-1}|} \ge 0.78$$ are merged into a single timed subtitle cue $[t_{\text{start}}, t_{\text{end}}]$, selecting the highest-confidence OCR recognition frame as the canonical raw text (ja_raw) and storing the initial English line (en_raw).

Hardware & ROCm Note: Always keep MIGRAPHX_DISABLE_MIOPEN_FUSION=1 and migraphx_fp16_enable=0 enabled in /data/docker/subocr/docker-compose.yml. Enabling FP16 on gfx1201 degrades bounding-box detection recall in PP-OCRv5.

Stage 2: Contextual LLM Refinement & Deterministic Furigana

1. Batched Contextual Refinement (qwen2.5:32b on ollama-rocm):
   • Extracted cues are processed in batches of 15 cues against http://172.28.0.1:11434/api/chat (qwen2.5:32b).
   • The LLM receives both the raw Japanese OCR (ja_raw) and any burned-in English subtitle (en_raw), repairing single-character visual OCR confusions (e.g., katakana ニ/二, カ/力, or visually degraded glyphs) while preserving the original spoken phrasing, slang, and dialect.
   • If en_raw is present and accurate, it is cleaned; if missing or garbled, a natural English translation (en_text) is synthesized alongside 1–3 key vocabulary/grammar notes (vocab) per cue.
2. Punctuation & On-Screen Symbol Restoration (restore_raw_punctuation):
   • Post-processing compares ja_raw and ja_text to ensure the LLM never strips on-screen quotation marks ("...", 「...」, 『...』), wave dashes (~, 〜), exclamation/question marks (!, ?), ellipses (…), or counter katakana (何カ国, 6カ国).
3. Deterministic Kanji-Only Furigana (fugashi + unidic-lite):
   • Rather than relying on LLM-generated furigana (which can hallucinate or attach readings to hiragana/katakana), SubOCR uses morphological tokenization (fugashi + unidic-lite + jaconv) in generate_kanji_furigana() to strip leading, trailing, and internal okurigana and attach <ruby>漢字<rt>かんじ</rt></ruby> annotations strictly above Kanji characters, leaving hiragana and katakana unannotated.

3. Web Application Layout & Study Controls

SubOCR is organized into three focused pages:

• Main OCR Library (https://subocr.drheck.dev/):
  • Displays all OCR jobs with status badges, cue counts, video durations, and category folders.
  • Supports pinning (📌), category filtering, searching, sorting (Newest First, Oldest First, Title (A–Z), Most Cues), inline renaming/re-categorization (✏️ Organize), and job deletion (🗑).
• New Subtitle OCR (https://subocr.drheck.dev/new):
  • Dedicated job submission form for local videos under /data/language/Japanese/Video, direct file uploads, or YouTube URLs, with optional start/end timestamps (start_sec, end_sec), frame sampling FPS, and category assignment.
• Dedicated OCR Study View (https://subocr.drheck.dev/ocr/{job_id}):
  • Sticky HTML5 video player on the left paired with a full-height synchronized subtitle cue list on the right.
  • Clicking any subtitle card seeks the video to that cue's start timestamp and highlights the active card in high-contrast amber/gold.
  • Keyboard Shortcuts (active whenever focus is outside text inputs):
    • Space or K: Play / Pause video without scrolling the page
    • A or ←: Jump to previous subtitle cue
    • D or →: Jump to next subtitle cue
    • R: Replay current subtitle cue from its start timestamp
    • L: Toggle single-cue A/B loop mode (Loop: ON/OFF)
  • Inline Cue Editing:
    • Clicking Edit on any cue card opens an inline editor for Japanese text, optional custom furigana reading override, English meaning, and vocabulary notes.
    • Editing the Japanese text automatically regenerates Kanji-only furigana (reading and ruby_html) and updates the exported .srt, .bilingual.srt, .vtt, .anki.tsv, and .study.md files on disk.

4. Operations, Maintenance & Backup / Restore

Container Management

# Check container status and logs
$ docker ps --filter name=subocr
$ docker logs -f --tail 100 subocr

# Restart after modifying /data/docker/subocr/app.py
$ docker restart subocr

# Rebuild container image if Dockerfile dependencies change
$ docker compose -f /data/docker/subocr/docker-compose.yml up -d --build

Automated System Backup & Restore

Backup Integration: SubOCR is fully integrated into the standard host ul backup and restore suite under /home/luke/git/backups/ul/scripts/ and runs automatically during sudo /home/luke/git/backups/ul/scripts/run_all_backups.

• Backup Script (/home/luke/git/backups/ul/scripts/backup/subocr):
  1. Rotates any existing backup in /data/backups/services/current/subocr/ to /data/backups/services/archive/subocr/.
  2. Executes a non-blocking SQLite WAL checkpoint (PRAGMA wal_checkpoint(TRUNCATE);) on /data/docker/subocr/data/subocr.db so the database file is completely self-contained.
  3. Creates a compressed archive (subocr_data.tar.xz) of /data/docker/subocr/ including app.py, Dockerfile, docker-compose.yml, static/, data/subocr.db, and all exported job subtitle/study files (data/jobs/*/*.srt, .vtt, .anki.tsv, .study.md), while excluding transient model/kernel caches (cache/) and uploaded raw video files (data/jobs/*/*.mp4).
  4. Generates 10% Reed-Solomon parity protection archives (subocr_data.tar.xz.par2) via par2 create -r10.
• Restore Script (/home/luke/git/backups/ul/scripts/restore/subocr):
  1. Verifies archive integrity against the par2 parity data (par2 verify).
  2. Extracts subocr_data.tar.xz into /data/docker/subocr/.
  3. Restarts the subocr container so the restored database and configuration take effect immediately.

# Run a standalone SubOCR backup
$ sudo /home/luke/git/backups/ul/scripts/backup/subocr

# Restore from the latest backup in /data/backups/services/current/subocr/
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr

# Restore from a specific timestamped backup
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr 2026-10-03_0844.30