Skip to main content

subocr service

Service Overview: SubOCR (https://subocr.drheck.dev/) is an end-to-end Japanese burned-in video subtitle (telop / テロップ) extraction, 漢字かんじ 振ふり仮名がな annotation, offline JMdict + JLPT (N5–N1) morphological dictionary lookup, LLM study-note refinement, and interactive video study platform running on host ul, accelerated by the AMD Radeon AI PRO R9700 (gfx1201, RDNA 4, 32 GB VRAM) GPU.

1. Architecture & Network Topology

SubOCR is deployed as a containerized FastAPI + ONNX Runtime (ROCm 7.2 MIGraphX) service bound to a dedicated IPv6 ULA address on eth77, reverse-proxied through Nginx with Authentik ForwardAuth single sign-on, and paired with the local ollama-rocm (qwen2.5:32b) LLM engine on the same GPU.

$$\text{Video Frame } I(t) \longrightarrow \text{PP-OCRv5 (MIGraphX FP32)} \longrightarrow \text{Qwen2.5-32B (ROCm)} \longrightarrow \text{UniDic }\langle\text{ruby}\rangle\text{+ JMdict/JLPT Study View}$$

  • Public URL: https://subocr.drheck.dev/
  • Dedicated ULA IPv6 Address: fd77:7777:5010:0:1337:f1ac:d00d:0c70 (eth77, port 8000)
  • Host Directory: /data/docker/subocr/
  • Databases:
    • Application Database: /data/docker/subocr/data/subocr.db (SQLite 3 in WAL mode)mode; stores jobs, timed cues, readings, translations, and custom edits)
    Offline Dictionary & JLPT Database: /data/docker/subocr/data/jmdict_jlpt.db (SQLite 3, 63 MB; 218,856 EDRDG JMdict entries, 499,325 indexed kanji/kana forms, and 7,972 JLPT N5–N1 vocabulary entries) Export & Sidecar Storage:
    • Per-job exports: /data/docker/subocr/data/jobs/<job_id>/
    • Automatic sidecar writes next to local videos: /data/language/Japanese/Video/... (mounted read-write at /videos inside the container)
    Authentication: Authentik ForwardAuth (subocr ProxyProvider, 30-day token validity) via /etc/nginx/sites-available/subocr DNS Resolution:
    • Local Unbound (danknasty, @10.0.0.1): A 10.0.0.100, AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa
    • Public Authoritative DNS (deSEC): AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa

    2. Two-Stage GPU Processing & Morphological Dictionary Pipeline

    When a job is submitted from https://subocr.drheck.dev/new (either by selecting a local video under /data/language/Japanese/Video, uploading a video file, or providing a YouTube URL fetched via yt-dlp), SubOCR executes a two-stage GPU pipeline on the AMD Radeon AI PRO R9700 (gfx1201): followed by deterministic morphological and dictionary analysis:

    Stage 1: ROCm 7.2 MIGraphX Visual Subtitle OCR (PP-OCRv5)

    1. Frame Sampling & Spatial Crop:
      • OpenCV samples the video at a configurable rate $f_s$ (default $2.0\text{ fps}$, or frame interval $\Delta t = 0.5\text{ s}$).
      • To isolate burned-in subtitles and ignore top-of-screen variety-show topic banners by default, frames are cropped to the lower subtitle region $y \in [0.58H, 0.98H]$.
    2. GPU Inference (MIGraphXExecutionProvider FP32):
      • Detection (PP-OCRv5_server_det_infer.onnx) and recognition (japanese_PP-OCRv5_mobile_rec_infer.onnx) run via onnxruntime-rocm with MIGraphXExecutionProvider.
      • Compiled MIGraphX kernel plans (.mxr) are cached in /data/docker/subocr/cache/migraphx so subsequent container restarts bypass ahead-of-time compilation.
      • Precision Note: migraphx_fp16_enable is explicitly set to 0 (FP32) because FP16 causes bounding-box coordinate underflow in PP-OCRv5 detection on gfx1201.
    3. Bilingual Line Classification & Spatial Role Separation:
      • Detected text boxes within each frame are sorted vertically and classified into Japanese (ja_lines) or English (en_lines) based on character script ratios.
      • Within the lower subtitle band, higher-positioned smaller Japanese commentary boxes ($y < 0.66H$) are separated from the primary bottom speaker dialogue ($y \ge 0.66H$) so simultaneous on-screen editor commentary and speaker subtitles are preserved cleanly.
    4. Temporal Deduplication:
      • Consecutive frames whose normalized Japanese string similarity satisfies: $$\text{sim}(A, B) = \frac{2 \cdot |M|}{|A| + |B|} \ge 0.78$$ are merged into a single timed subtitle cue $[t_0, t_1]$, selecting the highest-confidence OCR recognition frame as the canonical raw text (ja_raw) and storing the initial English line (en_raw).

    Hardware & ROCm Note: Always keep MIGRAPHX_DISABLE_MIOPEN_FUSION=1 and migraphx_fp16_enable=0 enabled in /data/docker/subocr/docker-compose.yml. Enabling FP16 on gfx1201 degrades bounding-box detection recall in PP-OCRv5.

    Stage 2: Contextual LLM RefinementRefinement, Kanji-Only Furigana & DeterministicOffline FuriganaDictionary Tagging

    1. Batched Contextual Refinement (qwen2.5:32b on ollama-rocm):
      • Extracted cues are processed in batches of 15 cues against http://172.28.0.1:11434/api/chat (qwen2.5:32b).
      • The LLM receives both the raw Japanese OCR (ja_raw) and any burned-in English subtitle (en_raw), repairing single-character visual OCR confusions (e.g., katakana ニ/二, カ/力, or visually degraded glyphs) while preserving the original spoken phrasing, slang, and dialect.
      • If en_raw is present and accurate, it is cleaned; if missing or garbled, a natural English translation (en_text) is synthesized alongside 1–3contextual key vocabulary/grammaridiom/slang notes (vocab) per cue..
    2. Punctuation & On-Screen Symbol Restoration (restore_raw_punctuation):
      • Post-processing compares ja_raw and ja_text to ensure the LLM never strips on-screen quotation marks ("...", 「...」, 『...』), wave dashes (~, 〜), exclamation/question marks (!, ?), ellipses (…), or counter katakana (何カ国, 6カ国).
      • Example preserved cue: "スペシャルゲスト~!"が<ruby>無<rt>な</rt></ruby>くなり<ruby>皆<rt>みな</rt></ruby>メンバーに
    3. Deterministic Kanji-Only Furigana (fugashi + unidic-lite):
      • Rather than relying on LLM-generated furigana (which can hallucinate readings or attach redundant readings to hiragana/katakana), SubOCR uses morphologicalMorphological tokenization (fugashi + unidic-lite + jaconv) in generate_kanji_furigana() to stripstrips leading, trailing, and internal okurigana and attachattaches <ruby> annotations strictly above Kanji characters, leaving hiragana and katakana unannotated:
      • Example Render: 世界せかいの違ちがいに6カ国こくが大混乱だいこんらん — それ考かんがえたらすごい日本にほんっぽいっていうか
    4. Exhaustive Per-Word Tokenization, Grammar Categorization & Offline JMdict / JLPT Lookup (analyze_cue_tokens):
      • Every word in every subtitle cue is tokenized by fugashi (MECAB_TAGGER), lemmatized to its dictionary base form (orthBase), and matched against the local read-only /data/docker/subocr/data/jmdict_jlpt.db SQLite database and a curated GRAMMAR_GLOSS_MAP for Japanese particles and colloquial auxiliaries (てる/てん, ちゃう, たら, じゃん, Kansai ねん/やん/へん, etc.).
      Each token is assigned:
        JLPT Level Badge (jlpt): N5, N4, N3, N2, N1, or Other Grammar Category (pos_cat / pos_label): Noun (Noun, Pronoun, Proper Noun, Counter), Verb (Ichidan Verb, Godan Verb, Suru Verb, Kuru Verb, displayed as surface (base) when conjugated), Adj (i-Adj, na-Adj), Adv (Adverb), Expr (Interjection, Conjunction, Idiom / Context), and Particle (Case Particle, Topic Particle, End Particle, Aux Verb, Suffix) English Gloss (meaning): Offline JMdict definition or grammatical function

        3. Web Application Layout & Study Controls

        SubOCR isuses a sharp-cornered (border-radius: 0) layout organized into three focused pages:

        • Main OCR Library (https://subocr.drheck.dev/):
          • Displays all OCR jobs with status badges, cue counts, video durations, and category folders.
          • Provides direct export and download links (JA .SRT, Bilingual .SRT, WebVTT .VTT, Anki .TSV, Study .MD, JSON, and Save to /data) on each OCR job card.
          • Supports pinning (★), category filtering, searching, sorting (Pinned & Newest, Oldest First, Title (A–Z), Most Cues), inline renaming/re-categorization (Organize), and job deletion (Delete).
        • New Subtitle OCR (https://subocr.drheck.dev/new):
          • Dedicated job submission form for local videos under /data/language/Japanese/Video, direct file uploads, or YouTube URLs, with optional start/end timestamps (start_sec, end_sec), frame sampling FPS, and category assignment.
        • Dedicated OCR Study View (https://subocr.drheck.dev/ocr/{job_id}):
          • StickyFull-viewport two-column workspace without a top header bar: compact ← Library link, category badge, and small muted video title above the sticky HTML5 video player on the leftleft, paired with a full-height synchronized subtitle cue list on the right.right (native video player WebVTT overlays are disabled by default so burned-in subtitles remain unobstructed).
          • Clicking any subtitle card seeks the video to that cue's start timestamp and highlights the active card in high-contrast amber/gold (#fbbf24 border/pill and #fef08a text).
          • Fixed-Width Display Visibility& TogglesVocabulary Filter Controls (topunder rightthe header,player controls on the left, persisted in localStorage): Independentwith fixed-width ON/ON/OFF toggleslabels forthat never shift button size):
            • Display Toggles: Japanese Subs, Furigana, English Sub, Vocab, and Original OCR.
          • JLPT Level Filters: N5, N4, N3, N2, N1, Other
          Grammar Type Filters: Noun, Verb, Adj, Adv, Expr, Particle Keyboard Shortcuts (active whenever focus is outside text inputs):
          • Space or K: Play / Pause video without scrolling the page
          • A or ←: Jump to previous subtitle cue
          • D or →: Jump to next subtitle cue
          • R: Replay current subtitle cue from its start timestamp
          • L: Toggle single-cue A/B loop mode (Loop: ON/OFF)
          Inline Cue Editing:
          • Clicking Edit on any cue card opens an inline editor for Japanese text, optional custom furigana reading override, English meaning, and vocabulary notes.
          • Editing the Japanese text automatically clears stale readings, regenerates Kanji-only furigana (reading and ruby_html), re-runs offline JMdict + JLPT morphological token analysis, updates the card in place, and writes updated .srt, .bilingual.srt, .vtt, .anki.tsv, and .study.md files on disk.

          4. Operations, Maintenance & Backup / Restore

          Container Management

          Click to expand Docker Compose & Service Management Commands 
          # Check container status and logs
$ docker ps --filter name=subocr
$ docker logs -f --tail 100 subocr

# Restart after modifying /data/docker/subocr/app.py
$ docker restart subocr

# Rebuild container image if Dockerfile dependencies change
$ docker compose -f /data/docker/subocr/docker-compose.yml up -d --build

          Automated System Backup & Restore

          Backup Integration: SubOCR is integrated into the standard host ul backup and restore suite under /home/luke/git/backups/ul/scripts/ and runs automatically during sudo /home/luke/git/backups/ul/scripts/run_all_backups.

          • Backup Script (/home/luke/git/backups/ul/scripts/backup/subocr):
            1. Rotates any existing backup in /data/backups/services/current/subocr/ to /data/backups/services/archive/subocr/.
            2. Executes a non-blocking SQLite WAL checkpoint (PRAGMA wal_checkpoint(TRUNCATE);) on /data/docker/subocr/data/subocr.db so the database file is completely self-contained.
            3. Creates a compressed archive (subocr_data.tar.xz, ~325 KB) of /data/docker/subocr/ including app.py, Dockerfile, docker-compose.yml, static/, data/subocr.db, and all exported job subtitle/study files (data/jobs/*/*.srt, .vtt, .anki.tsv, .study.md), while excluding transient model/kernel caches (cache/), the static JMdict reference database (data/jmdict_jlpt.db), and uploaded raw video files (data/jobs/*/*.mp4).
            4. Generates 10% Reed-Solomon parity protection archives (subocr_data.tar.xz.par2) via par2 create -r10.
          • Restore Script (/home/luke/git/backups/ul/scripts/restore/subocr):
            1. Verifies archive integrity against the par2 parity data (par2 verify).
            2. Extracts subocr_data.tar.xz into /data/docker/subocr/.
            3. Restarts the subocr container so the restored database and configuration take effect immediately.
          # Run a standalone SubOCR backup
$ sudo /home/luke/git/backups/ul/scripts/backup/subocr

# Restore from the latest backup in /data/backups/services/current/subocr/
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr

# Restore from a specific timestamped backup
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr 2026-10-03_0845.06