Skip to main content

subocr service

Service Overview: SubOCR (https://subocr.drheck.dev/) is an end-to-end Japanese burned-in video subtitle (telop / テロップ) extraction, Kanji furigana annotation, LLM study-note refinement, and interactive video study platform running on host ul, accelerated by the AMD Radeon AI PRO R9700 (gfx1201, RDNA 4, 32 GB VRAM) GPU.

1. Architecture & Network Topology

SubOCR is deployed as a containerized FastAPI + ONNX Runtime (ROCm 7.2 MIGraphX) service bound to a dedicated IPv6 ULA address on eth77, reverse-proxied through Nginx with Authentik ForwardAuth single sign-on, and paired with the local ollama-rocm (qwen2.5:32b) LLM engine on the same GPU.

    Public URL: https://subocr.drheck.dev/ Dedicated ULA IPv6 Address: fd77:7777:5010:0:1337:f1ac:d00d:0c70 (eth77, port 8000) Host Directory: /data/docker/subocr/ Database: /data/docker/subocr/data/subocr.db (SQLite 3 in WAL mode) Export & Sidecar Storage:
      Per-job exports: /data/docker/subocr/data/jobs/<job_id>/ Automatic sidecar writes next to local videos: /data/language/Japanese/Video/... (mounted read-write at /videos inside the container) Authentication: Authentik ForwardAuth (subocr ProxyProvider, 30-day token validity) via /etc/nginx/sites-available/subocr DNS Resolution:
        Local Unbound (danknasty, @10.0.0.1): A 10.0.0.100, AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa Public Authoritative DNS (deSEC): AAAA fd77:7777:5010::1337:f1ac:d00d:aaaa

        2. Two-Stage GPU Processing Pipeline

        When a job is submitted from https://subocr.drheck.dev/new (either by selecting a local video under /data/language/Japanese/Video, uploading a video file, or providing a YouTube URL fetched via yt-dlp), SubOCR executes a two-stage GPU pipeline on the AMD Radeon AI PRO R9700 (gfx1201):

        Stage 1: ROCm 7.2 MIGraphX Visual Subtitle OCR (PP-OCRv5)

          Frame Sampling & Spatial Crop:
            OpenCV samples the video at a configurable rate $f_s$ (default $f_s = 2.0\text{ fps}$, or interval $\Delta t = 0.5\text{ s}$). To isolate burned-in subtitles and ignore top-of-screen variety-show topic banners by default, frames are cropped to the lower subtitle region ($y \in [0.58H, 0.98H]$). GPU Inference (MIGraphXExecutionProvider FP32):
              Detection (PP-OCRv5_server_det_infer.onnx) and recognition (japanese_PP-OCRv5_mobile_rec_infer.onnx) run via onnxruntime-rocm with MIGraphXExecutionProvider. Compiled MIGraphX kernel plans (.mxr) are cached in /data/docker/subocr/cache/migraphx so subsequent container restarts bypass AOT compilation. Precision Note: migraphx_fp16_enable is explicitly set to 0 (FP32) because FP16 causes bounding-box coordinate underflow in PP-OCRv5_server_det on gfx1201. Bilingual Line Classification & Spatial Role Separation:
                Detected text boxes within each frame are sorted vertically and classified as Japanese (ja_lines) or English (en_lines) based on character script ratios. Within the lower subtitle band, higher-positioned smaller Japanese commentary boxes ($y_{\text{rel}} < 0.66$) are separated from the primary bottom speaker dialogue ($y_{\text{rel}} \ge 0.66$) so simultaneous on-screen editor commentary and speaker subtitles are preserved cleanly. Temporal Deduplication:
                  Consecutive frames whose normalized Japanese string similarity satisfies: $$\text{sim}(S_{t}, S_{t-1}) = \frac{2 \cdot |M|}{|S_t| + |S_{t-1}|} \ge 0.78$$ are merged into a single timed subtitle cue $[t_{\text{start}}, t_{\text{end}}]$, selecting the highest-confidence OCR recognition frame as the canonical raw text (ja_raw) and storing the initial English line (en_raw).

                  Hardware & ROCm Note: Always keep MIGRAPHX_DISABLE_MIOPEN_FUSION=1 and migraphx_fp16_enable=0 enabled in /data/docker/subocr/docker-compose.yml. Enabling FP16 on gfx1201 degrades bounding-box detection recall in PP-OCRv5.

                  Stage 2: Contextual LLM Refinement & Deterministic Furigana

                    Batched Contextual Refinement (qwen2.5:32b on ollama-rocm):
                      Extracted cues are processed in batches of 15 cues against http://172.28.0.1:11434/api/chat (qwen2.5:32b). The LLM receives both the raw Japanese OCR (ja_raw) and any burned-in English subtitle (en_raw), repairing single-character visual OCR confusions (e.g., katakana ニ/二, カ/力, or visually degraded glyphs) while preserving the original spoken phrasing, slang, and dialect. If en_raw is present and accurate, it is cleaned; if missing or garbled, a natural English translation (en_text) is synthesized alongside 1–3 key vocabulary/grammar notes (vocab) per cue. Punctuation & On-Screen Symbol Restoration (restore_raw_punctuation):
                        Post-processing compares ja_raw and ja_text to ensure the LLM never strips on-screen quotation marks ("...", 「...」, 『...』), wave dashes (~, 〜), exclamation/question marks (!, ?), ellipses (…), or counter katakana (何カ国, 6カ国). Deterministic Kanji-Only Furigana (fugashi + unidic-lite):
                          Rather than relying on LLM-generated furigana (which can hallucinate or attach readings to hiragana/katakana), SubOCR uses morphological tokenization (fugashi + unidic-lite + jaconv) in generate_kanji_furigana() to strip leading, trailing, and internal okurigana and attach <ruby>漢字<rt>かんじ</rt></ruby> annotations strictly above Kanji characters, leaving hiragana and katakana unannotated.

                          3. Web Application Layout & Study Controls

                          SubOCR is organized into three focused pages:

                            Main OCR Library (https://subocr.drheck.dev/):
                              Displays all OCR jobs with status badges, cue counts, video durations, and category folders. Supports pinning (📌), category filtering, searching, sorting (Newest First, Oldest First, Title (A–Z), Most Cues), inline renaming/re-categorization (✏️ Organize), and job deletion (🗑). New Subtitle OCR (https://subocr.drheck.dev/new):
                                Dedicated job submission form for local videos under /data/language/Japanese/Video, direct file uploads, or YouTube URLs, with optional start/end timestamps (start_sec, end_sec), frame sampling FPS, and category assignment. Dedicated OCR Study View (https://subocr.drheck.dev/ocr/{job_id}):
                                  Sticky HTML5 video player on the left paired with a full-height synchronized subtitle cue list on the right. Clicking any subtitle card seeks the video to that cue's start timestamp and highlights the active card in high-contrast amber/gold. Keyboard Shortcuts (active whenever focus is outside text inputs):
                                    Space or K: Play / Pause video without scrolling the page A or ←: Jump to previous subtitle cue D or →: Jump to next subtitle cue R: Replay current subtitle cue from its start timestamp L: Toggle single-cue A/B loop mode (Loop: ON/OFF) Inline Cue Editing:
                                      Clicking Edit on any cue card opens an inline editor for Japanese text, optional custom furigana reading override, English meaning, and vocabulary notes. Editing the Japanese text automatically regenerates Kanji-only furigana (reading and ruby_html) and updates the exported .srt, .bilingual.srt, .vtt, .anki.tsv, and .study.md files on disk.

                                      4. Operations, Maintenance & Backup / Restore

                                      Container Management

                                      Click to expand Docker Compose & Service Management Commands 
                                      # Check container status and logs
$ docker ps --filter name=subocr
$ docker logs -f --tail 100 subocr

# Restart after modifying /data/docker/subocr/app.py
$ docker restart subocr

# Rebuild container image if Dockerfile dependencies change
$ docker compose -f /data/docker/subocr/docker-compose.yml up -d --build

                                      Automated System Backup & Restore

                                      Backup Integration: SubOCR is fully integrated into the standard host ul backup and restore suite under /home/luke/git/backups/ul/scripts/ and runs automatically during sudo /home/luke/git/backups/ul/scripts/run_all_backups.

                                        Backup Script (/home/luke/git/backups/ul/scripts/backup/subocr):
                                          Rotates any existing backup in /data/backups/services/current/subocr/ to /data/backups/services/archive/subocr/. Executes a non-blocking SQLite WAL checkpoint (PRAGMA wal_checkpoint(TRUNCATE);) on /data/docker/subocr/data/subocr.db so the database file is completely self-contained. Creates a compressed archive (subocr_data.tar.xz) of /data/docker/subocr/ including app.py, Dockerfile, docker-compose.yml, static/, data/subocr.db, and all exported job subtitle/study files (data/jobs/*/*.srt, .vtt, .anki.tsv, .study.md), while excluding transient model/kernel caches (cache/) and uploaded raw video files (data/jobs/*/*.mp4). Generates 10% Reed-Solomon parity protection archives (subocr_data.tar.xz.par2) via par2 create -r10. Restore Script (/home/luke/git/backups/ul/scripts/restore/subocr):
                                            Verifies archive integrity against the par2 parity data (par2 verify). Extracts subocr_data.tar.xz into /data/docker/subocr/. Restarts the subocr container so the restored database and configuration take effect immediately. 
                                            # Run a standalone SubOCR backup
$ sudo /home/luke/git/backups/ul/scripts/backup/subocr

# Restore from the latest backup in /data/backups/services/current/subocr/
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr

# Restore from a specific timestamped backup
$ sudo /home/luke/git/backups/ul/scripts/restore/subocr 2026-10-03_0844.30