subocr service

Service Overview: SubOCR (https://subocr.drheck.dev/) is an end-to-end Japanese burned-in video subtitle (telop / テロップ) extraction, 漢字かんじ 振ふり仮名がな annotation, offline JMdict + JLPT (N5–N1) morphological dictionary lookup, LLM study-note refinement, and interactive video study platform running on host ul, accelerated by the AMD Radeon AI PRO R9700 (gfx1201, RDNA 4, 32 GB VRAM) GPU.

1. Architecture & Network Topology

SubOCR is deployed as a containerized FastAPI + ONNX Runtime (ROCm 7.2 MIGraphX) service bound to a dedicated IPv6 ULA address on eth77, reverse-proxied through Nginx with Authentik ForwardAuth single sign-on, and paired with the local ollama-rocm (qwen2.5:32b) LLM engine on the same GPU.

$$\text{Video Frame } I(t) \longrightarrow \text{PP-OCRv5 (MIGraphX FP32)} \longrightarrow \text{Qwen2.5-32B (ROCm)} \longrightarrow \text{UniDic + JMdict/JLPT Study View}$$

2. Two-Stage GPU Processing & Morphological Dictionary Pipeline

When a job is submitted from https://subocr.drheck.dev/new (either by selecting a local video under /data/language/Japanese/Video, uploading a video file, or providing a YouTube URL fetched via yt-dlp), SubOCR executes a two-stage GPU pipeline on the AMD Radeon AI PRO R9700 (gfx1201) followed by deterministic morphological and dictionary analysis:

Stage 1: ROCm 7.2 MIGraphX Visual Subtitle OCR (PP-OCRv5)

  1. Frame Sampling & Spatial Crop:
    • OpenCV samples the video at a configurable rate $f_s$ (default $2.0\text{ fps}$, or frame interval $\Delta t = 0.5\text{ s}$).
    • To isolate burned-in subtitles and ignore top-of-screen variety-show topic banners by default, frames are cropped to the lower subtitle region $y \in [0.58H, 0.98H]$.
  2. GPU Inference (MIGraphXExecutionProvider FP32):
    • Detection (PP-OCRv5_server_det_infer.onnx) and recognition (japanese_PP-OCRv5_mobile_rec_infer.onnx) run via onnxruntime-rocm with MIGraphXExecutionProvider.
    • Compiled MIGraphX kernel plans (.mxr) are cached in /data/docker/subocr/cache/migraphx so subsequent container restarts bypass ahead-of-time compilation.
    • Precision Note: migraphx_fp16_enable is explicitly set to 0 (FP32) because FP16 causes bounding-box coordinate underflow in PP-OCRv5 detection on gfx1201.
  3. Bilingual Line Classification & Spatial Role Separation:
    • Detected text boxes within each frame are sorted vertically and classified into Japanese (ja_lines) or English (en_lines) based on character script ratios.
    • Within the lower subtitle band, higher-positioned smaller Japanese commentary boxes ($y < 0.66H$) are separated from the primary bottom speaker dialogue ($y \ge 0.66H$) so simultaneous on-screen editor commentary and speaker subtitles are preserved cleanly.
  4. Temporal Deduplication:
    • Consecutive frames whose normalized Japanese string similarity satisfies: $$\text{sim}(A, B) = \frac{2 \cdot |M|}{|A| + |B|} \ge 0.78$$ are merged into a single timed subtitle cue $[t_0, t_1]$, selecting the highest-confidence OCR recognition frame as the canonical raw text (ja_raw) and storing the initial English line (en_raw).

Hardware & ROCm Note: Always keep MIGRAPHX_DISABLE_MIOPEN_FUSION=1 and migraphx_fp16_enable=0 enabled in /home/docker-data/.config/containers/systemd/subocr.container, and never set UserNS=keep-id on the 23.3 GB ROCm image (default rootless user namespace starts in under 2 seconds). Enabling FP16 on gfx1201 degrades bounding-box detection recall in PP-OCRv5.

Stage 2: Contextual LLM Refinement, Kanji-Only Furigana & Offline Dictionary Tagging

  1. Whole-Conversation Person Name Identification (extract_conversation_person_names):
    • Before batched refinement or vocabulary page generation, qwen2.5:32b scans the full numbered transcript across the entire video (together with the video title) to identify recurring speakers, cast members, and mentioned individuals.
    • Extracted names are stored in jobs.person_names_json (mapping each canonical surface form and OCR variant to its reading, English romanization, role description, and 固有名詞 tag), automatically repairing visual OCR name typos (e.g., たくあん / たくみん -> たくみ) and preventing person names from being misclassified as common homophones in vocabulary chips.
  2. Batched Contextual Refinement (qwen2.5:32b on ollama-rocm):
    • Extracted cues are processed in batches of 15 cues against http://169.254.1.2:11434/api/chat (qwen2.5:32b, routed via pasta --map-host-loopback to 127.0.0.1:11434).
    • The LLM receives both the raw Japanese OCR (ja_raw) and any burned-in English subtitle (en_raw), repairing single-character visual OCR confusions (e.g., katakana ニ/二, カ/力, or visually degraded glyphs) while preserving the original spoken phrasing, slang, and dialect.
    • If en_raw is present and accurate, it is cleaned; if missing or garbled, a natural English translation (en_text) is synthesized alongside contextual idiom/slang notes (vocab).
  3. Punctuation & On-Screen Symbol Restoration (restore_raw_punctuation):
    • Post-processing compares ja_raw and ja_text to ensure the LLM never strips on-screen quotation marks ("...", 「...」, 『...』), wave dashes (~, 〜), exclamation/question marks (!, ?), ellipses (…), or counter katakana (何カ国, 6カ国).
    • Example preserved cue: "スペシャルゲスト~!"が<ruby>無<rt>な</rt></ruby>くなり<ruby>皆<rt>みな</rt></ruby>メンバーに
  4. Deterministic Kanji-Only Furigana (fugashi + unidic-lite):
    • Morphological tokenization (fugashi + unidic-lite + jaconv) in generate_kanji_furigana() strips leading, trailing, and internal okurigana and attaches <ruby> annotations strictly above Kanji characters, leaving hiragana and katakana unannotated:
    • Example Render: 世界せかいの違ちがいに6カ国こくが大混乱だいこんらん — それ考かんがえたらすごい日本にほんっぽいっていうか
  5. Exhaustive Per-Word Tokenization, Grammar Categorization & Offline JMdict / JLPT Lookup (analyze_cue_tokens):
    • Every word in every subtitle cue is tokenized by fugashi (MECAB_TAGGER), lemmatized to its dictionary base form (orthBase), and matched against the local read-only /data/docker/subocr/data/jmdict_jlpt.db SQLite database and a curated GRAMMAR_GLOSS_MAP for Japanese particles and colloquial auxiliaries (てる/てん, ちゃう, たら, じゃん, Kansai ねん/やん/へん, etc.).
    • Each token is assigned:
      • JLPT Level Badge (jlpt): N5, N4, N3, N2, N1, or Other
      • Part-of-Speech Filter & Chip Tag (pos_cat / pos_label): Grouped under the 6 品詞 filter buttons (名詞, 動詞, 形容詞, 副詞, 表現, 助詞) and labeled on each vocabulary chip with its specific Japanese grammatical category (see the complete reference table below)
      • English Gloss (meaning): Offline JMdict definition or grammatical function

Reference Table: Japanese Part-of-Speech (品詞) Filter Groups & Vocabulary Chip Tags

Chip Tag
(pos_label)		 Reading
(振ふり仮名がな)		 Filter Button
(品詞)		 English Translation Grammatical Role & Subtitle Examples
名詞 名詞めいし 名詞
(Noun)		 Common Noun General nouns and suru-noun stems (e.g., 世界, 違い, 大混乱)
固有名詞 固有名詞こゆうめいし 名詞
(Noun)		 Proper Noun Places, countries, people, and brand names (e.g., 日本, 関西, フランス)
代名詞 代名詞だいめいし 名詞
(Noun)		 Pronoun Personal and demonstrative pronouns (e.g., 私, 僕, これ, それ, どこ)
助数詞 助数詞じょすうし 名詞
(Noun)		 Counter Numerical counter suffixes (e.g., カ国, 人, 回, つ, 個)
動詞 動詞どうし 動詞
(Verb)		 Verb All main verbs (Godan, Ichidan, Suru, Kuru); displayed as conjugated (dictionary form) when inflected (e.g., 言った (言う), 考えたら (考える))
形容詞 形容詞けいようし 形容詞
(Adjective)		 i-Adjective Inflecting adjectives ending in -i (e.g., すごい, 面白い, 無い)
形容動詞 形容動詞けいようどうし 形容詞
(Adjective)		 na-Adjective Adjectival nouns that take な before nouns (e.g., 好き, 大変, 複雑)
連体詞 連体詞れんたいし 形容詞
(Adjective)		 Pre-noun Adjectival Non-inflecting words that directly modify nouns (e.g., この, その, あんな, いろんな)
副詞 副詞ふくし 副詞
(Adverb)		 Adverb Words modifying verbs, adjectives, or clauses (e.g., ちょっと, めっちゃ, 全然, やっぱり)
表現 表現ひょうげん 表現
(Expression)		 Expression / Idiom Multi-word idioms, set phrases, and contextual slang compounds
感動詞 感動詞かんどうし 表現
(Expression)		 Interjection Exclamations, greetings, and conversational fillers (e.g., えっ, はい, うわ, なるほど)
接続詞 接続詞せつぞくし 表現
(Expression)		 Conjunction Sentence/clause connectors (e.g., でも, だから, そして, ただ)
助詞 助詞じょし 助詞
(Particle)		 Particle (General) General or adverbial/quotative particles (e.g., って, とか, だけ, くらい)
格助詞 格助詞かくじょし 助詞
(Particle)		 Case Particle Marks grammatical case relationships (e.g., が, を, に, で, と, から)
係助詞 係助詞かかりじょし 助詞
(Particle)		 Topic / Binding Particle Marks topic, contrast, or inclusion (e.g., は, も, こそ)
終助詞 終助詞しゅうじょし 助詞
(Particle)		 Sentence-Ending Particle Adds tone, question, or emphasis at clause end (e.g., ね, よ, な, か, じゃん)
接続助詞 接続助詞せつぞくじょし 助詞
(Particle)		 Conjunctive Particle Connects clauses (e.g., て/で, けど, から, ば, のに, し)
助動詞 助動詞じょどうし 助詞
(Particle)		 Auxiliary Verb Inflecting verbal/copular endings and colloquial contractions (e.g., だ/です, た, ない, ます, てる/てん, ちゃう, ねん, やん, へん)
接頭辞 接頭辞せっとうじ 助詞
(Particle)		 Prefix Honorific or intensifying prefixes attached to stems (e.g., お, ご, 大, 超)
接尾辞 接尾辞せつびじ 助詞
(Particle)		 Suffix Meaning-modifying suffixes attached to stems (e.g., さん, たち, っぽい, 的, 目)

3. Web Application Layout & Study Controls

SubOCR uses a sharp-cornered (border-radius: 0) layout organized into three focused pages:

4. Operations, Maintenance & Backup / Restore

Container Management

Click to expand Rootless Podman Quadlet & Service Management Commands 
# Check container status and logs
$ (cd /tmp && sudo -u docker-data HOME=/home/docker-data XDG_RUNTIME_DIR=/run/user/1100 podman ps --filter name=subocr)
$ sudo -u docker-data XDG_RUNTIME_DIR=/run/user/1100 journalctl --user -u subocr.service -n 100 -f

# Restart after modifying /data/docker/subocr/app.py or vocab_pages.py
$ sudo systemctl --user -M docker-data@ restart subocr.service

# Rebuild container image if Dockerfile dependencies change
$ (cd /data/docker/subocr && sudo -u docker-data HOME=/home/docker-data XDG_RUNTIME_DIR=/run/user/1100 TMPDIR=/dev/shm podman build -t localhost/subocr:latest /data/docker/subocr)
$ sudo systemctl --user -M docker-data@ restart subocr.service

Automated System Backup & Restore

Backup Integration: SubOCR is integrated into the standard host ul backup and restore suite under /home/luke/git/backups/ul/scripts/ and runs automatically during sudo /home/luke/git/backups/ul/scripts/run_all_backups.

# Run a standalone SubOCR backup
$ sudo /home/luke/git/backups/ul/scripts/backup/subocr

# Restore from the latest backup in /data/backups/services/current/subocr/
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr

# Restore from a specific timestamped backup
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr 2026-10-03_1133.09
Revision #16
Created 2026-10-03 12:40:38 UTC by luKe
Updated 2026-10-20 03:46:13 UTC by luKe