subocr service

Service Overview: SubOCR (https://subocr.drheck.dev/) is an end-to-end Japanese burned-in video subtitle (telop / テロップ) extraction, 漢字かんじ 振ふり仮名がな annotation, offline JMdict + JLPT (N5–N1) morphological dictionary lookup, LLM study-note refinement, and interactive video study platform running on host ul, accelerated by the AMD Radeon AI PRO R9700 (gfx1201, RDNA 4, 32 GB VRAM) GPU.

1. Architecture & Network Topology

SubOCR is deployed as a containerized FastAPI + ONNX Runtime (ROCm 7.2 MIGraphX) service bound to a dedicated IPv6 ULA address on eth77, reverse-proxied through Nginx with Authentik ForwardAuth single sign-on, and paired with the local ollama-rocm (qwen2.5:32b) LLM engine on the same GPU.

$$\text{Video Frame } I(t) \longrightarrow \text{PP-OCRv5 (MIGraphX FP32)} \longrightarrow \text{Qwen2.5-32B (ROCm)} \longrightarrow \text{UniDic + JMdict/JLPT Study View}$$

• Public URL: https://subocr.drheck.dev/
• Dedicated ULA IPv6 Address: fd77:7777:5010:0:1337:f1ac:d00d:0c70 (eth77, port 8000)
• Host Directory: /data/docker/subocr/
• Databases:
  • Application Database: /data/docker/subocr/data/subocr.db (SQLite 3 in WAL mode; stores jobs, timed cues, readings, translations, and custom edits)
  • Offline Dictionary & JLPT Database: /data/docker/subocr/data/jmdict_jlpt.db (SQLite 3, 63 MB; 218,856 EDRDG JMdict entries, 499,325 indexed kanji/kana forms, and 7,972 JLPT N5–N1 vocabulary entries)
• Export & Sidecar Storage:
  • Per-job exports: /data/docker/subocr/data/jobs/<job_id>/
  • Automatic sidecar writes next to local videos: /data/language/Japanese/Video/... (mounted read-write at /videos inside the container)
• Authentication: Authentik ForwardAuth (subocr ProxyProvider, 30-day token validity) via /etc/nginx/sites-available/subocr
• DNS Resolution:
  • Local Unbound (danknasty, @10.0.0.1): A 10.0.0.100, AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa
  • Public Authoritative DNS (deSEC): AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa

2. Two-Stage GPU Processing & Morphological Dictionary Pipeline

When a job is submitted from https://subocr.drheck.dev/new (either by selecting a local video under /data/language/Japanese/Video, uploading a video file, or providing a YouTube URL fetched via yt-dlp), SubOCR executes a two-stage GPU pipeline on the AMD Radeon AI PRO R9700 (gfx1201) followed by deterministic morphological and dictionary analysis:

Stage 1: ROCm 7.2 MIGraphX Visual Subtitle OCR (PP-OCRv5)

1. Frame Sampling & Spatial Crop:
   • OpenCV samples the video at a configurable rate $f_s$ (default $2.0\text{ fps}$, or frame interval $\Delta t = 0.5\text{ s}$).
   • To isolate burned-in subtitles and ignore top-of-screen variety-show topic banners by default, frames are cropped to the lower subtitle region $y \in [0.58H, 0.98H]$.
2. GPU Inference (MIGraphXExecutionProvider FP32):
   • Detection (PP-OCRv5_server_det_infer.onnx) and recognition (japanese_PP-OCRv5_mobile_rec_infer.onnx) run via onnxruntime-rocm with MIGraphXExecutionProvider.
   • Compiled MIGraphX kernel plans (.mxr) are cached in /data/docker/subocr/cache/migraphx so subsequent container restarts bypass ahead-of-time compilation.
   • Precision Note: migraphx_fp16_enable is explicitly set to 0 (FP32) because FP16 causes bounding-box coordinate underflow in PP-OCRv5 detection on gfx1201.
3. Bilingual Line Classification & Spatial Role Separation:
   • Detected text boxes within each frame are sorted vertically and classified into Japanese (ja_lines) or English (en_lines) based on character script ratios.
   • Within the lower subtitle band, higher-positioned smaller Japanese commentary boxes ($y < 0.66H$) are separated from the primary bottom speaker dialogue ($y \ge 0.66H$) so simultaneous on-screen editor commentary and speaker subtitles are preserved cleanly.
4. Temporal Deduplication:
   • Consecutive frames whose normalized Japanese string similarity satisfies: $$\text{sim}(A, B) = \frac{2 \cdot |M|}{|A| + |B|} \ge 0.78$$ are merged into a single timed subtitle cue $[t_0, t_1]$, selecting the highest-confidence OCR recognition frame as the canonical raw text (ja_raw) and storing the initial English line (en_raw).

Hardware & ROCm Note: Always keep MIGRAPHX_DISABLE_MIOPEN_FUSION=1 and migraphx_fp16_enable=0 enabled in /data/docker/subocr/docker-compose.yml. Enabling FP16 on gfx1201 degrades bounding-box detection recall in PP-OCRv5.

Stage 2: Contextual LLM Refinement, Kanji-Only Furigana & Offline Dictionary Tagging

1. Batched Contextual Refinement (qwen2.5:32b on ollama-rocm):
   • Extracted cues are processed in batches of 15 cues against http://172.28.0.1:11434/api/chat (qwen2.5:32b).
   • The LLM receives both the raw Japanese OCR (ja_raw) and any burned-in English subtitle (en_raw), repairing single-character visual OCR confusions (e.g., katakana ニ/二, カ/力, or visually degraded glyphs) while preserving the original spoken phrasing, slang, and dialect.
   • If en_raw is present and accurate, it is cleaned; if missing or garbled, a natural English translation (en_text) is synthesized alongside contextual idiom/slang notes (vocab).
2. Punctuation & On-Screen Symbol Restoration (restore_raw_punctuation):
   • Post-processing compares ja_raw and ja_text to ensure the LLM never strips on-screen quotation marks ("...", 「...」, 『...』), wave dashes (~, 〜), exclamation/question marks (!, ?), ellipses (…), or counter katakana (何カ国, 6カ国).
   • Example preserved cue: "スペシャルゲスト~!"が<ruby>無<rt>な</rt></ruby>くなり<ruby>皆<rt>みな</rt></ruby>メンバーに
3. Deterministic Kanji-Only Furigana (fugashi + unidic-lite):
   • Morphological tokenization (fugashi + unidic-lite + jaconv) in generate_kanji_furigana() strips leading, trailing, and internal okurigana and attaches <ruby> annotations strictly above Kanji characters, leaving hiragana and katakana unannotated:
   • Example Render: 世界せかいの違ちがいに6カ国こくが大混乱だいこんらん — それ考かんがえたらすごい日本にほんっぽいっていうか
4. Exhaustive Per-Word Tokenization, Grammar Categorization & Offline JMdict / JLPT Lookup (analyze_cue_tokens):
   • Every word in every subtitle cue is tokenized by fugashi (MECAB_TAGGER), lemmatized to its dictionary base form (orthBase), and matched against the local read-only /data/docker/subocr/data/jmdict_jlpt.db SQLite database and a curated GRAMMAR_GLOSS_MAP for Japanese particles and colloquial auxiliaries (てる/てん, ちゃう, たら, じゃん, Kansai ねん/やん/へん, etc.).
   • Each token is assigned:
     • JLPT Level Badge (jlpt): N5, N4, N3, N2, N1, or Other
     • GrammarPart-of-Speech CategoryFilter & Chip Tag (pos_cat / pos_label): Grouped under the 6 Noun品詞 filter buttons (Noun名詞, Pronoun動詞, Proper Noun形容詞, Counter), Verb (Ichidan Verb副詞, Godan Verb表現, Suru Verb, Kuru Verb, displayed as surface (base) when conjugated), Adj (i-Adj, na-Adj助詞), Adv (Adverb), Expr (Interjection, Conjunction, Idiom / Context), and Particlelabeled on each vocabulary chip with its specific Japanese grammatical category (Casesee Particle,the Topiccomplete Particle,reference Endtable Particle, Aux Verb, Suffix)below)
     • English Gloss (meaning): Offline JMdict definition or grammatical function

Reference Table: Japanese Part-of-Speech (品詞) Filter Groups & Vocabulary Chip Tags

3. Web Application Layout & Study Controls

SubOCR uses a sharp-cornered (border-radius: 0) layout organized into three focused pages:

• Main OCR Library (https://subocr.drheck.dev/):
  • Displays all OCR jobs with status badges, cue counts, video durations, and category folders.
  • Provides direct export and download links (JA .SRT, Bilingual .SRT, WebVTT .VTT, Anki .TSV, Study .MD, JSON, and Save to /data) on each OCR job card.
  • Supports pinning (★), category filtering, searching, sorting (Pinned & Newest, Oldest First, Title (A–Z), Most Cues), inline renaming/re-categorization (Organize), and job deletion (Delete).
• New Subtitle OCR (https://subocr.drheck.dev/new):
  • Dedicated job submission form for local videos under /data/language/Japanese/Video, direct file uploads, or YouTube URLs, with optional start/end timestamps (start_sec, end_sec), frame sampling FPS, and category assignment.
• Dedicated OCR Study View (https://subocr.drheck.dev/ocr/{job_id}):
  • Full-viewport two-column workspace without a top header bar: compact ← Library link, category badge, and small muted video title above the sticky HTML5 video player on the left, paired with a full-height synchronized subtitle cue list on the right (native video player WebVTT overlays are disabled by default so burned-in subtitles remain unobstructed).
  • Clicking any subtitle card seeks the video to that cue's start timestamp and highlights the active card in high-contrast amber/gold (#fbbf24 border/pill and #fef08a text).
  • Fixed-Width Display & Vocabulary Filter Controls (under the player controls on the left, persisted in localStorage with fixed-width ON/OFF labels that never shift button size):
    • Display Toggles: Japanese Subs, Furigana, English Sub, Vocab, and Original OCR
    • JLPT Level Filters: N5, N4, N3, N2, N1, Other
    • Grammar品詞 Type(Part-of-Speech) Filters: Noun名詞 (Noun), Verb動詞 (Verb), Adj形容詞 (Adjective), Adv副詞 (Adverb), Expr表現 (Expression), 助詞 (Particle / Auxiliary)
  • Keyboard Shortcuts (active whenever focus is outside text inputs):
    • Space or K: Play / Pause video without scrolling the page
    • A or ←: Jump to previous subtitle cue
    • D or →: Jump to next subtitle cue
    • R: Replay current subtitle cue from its start timestamp
    • L: Toggle single-cue A/B loop mode (Loop: ON/OFF)
  • Inline Cue Editing:
    • Clicking Edit on any cue card opens an inline editor for Japanese text, optional custom furigana reading override, English meaning, and vocabulary notes.
    • Editing the Japanese text automatically clears stale readings, regenerates Kanji-only furigana (reading and ruby_html), re-runs offline JMdict + JLPT morphological token analysis, updates the card in place, and writes updated .srt, .bilingual.srt, .vtt, .anki.tsv, and .study.md files on disk.

4. Operations, Maintenance & Backup / Restore

Container Management

# Check container status and logs
$ docker ps --filter name=subocr
$ docker logs -f --tail 100 subocr

# Restart after modifying /data/docker/subocr/app.py
$ docker restart subocr

# Rebuild container image if Dockerfile dependencies change
$ docker compose -f /data/docker/subocr/docker-compose.yml up -d --build

Automated System Backup & Restore

Backup Integration: SubOCR is integrated into the standard host ul backup and restore suite under /home/luke/git/backups/ul/scripts/ and runs automatically during sudo /home/luke/git/backups/ul/scripts/run_all_backups.

• Backup Script (/home/luke/git/backups/ul/scripts/backup/subocr):
  1. Rotates any existing backup in /data/backups/services/current/subocr/ to /data/backups/services/archive/subocr/.
  2. Executes a non-blocking SQLite WAL checkpoint (PRAGMA wal_checkpoint(TRUNCATE);) on /data/docker/subocr/data/subocr.db so the database file is completely self-contained.
  3. Creates a compressed archive (subocr_data.tar.xz, ~325 KB) of /data/docker/subocr/ including app.py, Dockerfile, docker-compose.yml, static/, data/subocr.db, and all exported job subtitle/study files (data/jobs/*/*.srt, .vtt, .anki.tsv, .study.md), while excluding transient model/kernel caches (cache/), the static JMdict reference database (data/jmdict_jlpt.db), and uploaded raw video files (data/jobs/*/*.mp4).
  4. Generates 10% Reed-Solomon parity protection archives (subocr_data.tar.xz.par2) via par2 create -r10.
• Restore Script (/home/luke/git/backups/ul/scripts/restore/subocr):
  1. Verifies archive integrity against the par2 parity data (par2 verify).
  2. Extracts subocr_data.tar.xz into /data/docker/subocr/.
  3. Restarts the subocr container so the restored database and configuration take effect immediately.

# Run a standalone SubOCR backup
$ sudo /home/luke/git/backups/ul/scripts/backup/subocr

# Restore from the latest backup in /data/backups/services/current/subocr/
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr

# Restore from a specific timestamped backup
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr 2026-10-03_0845.06