Compare commits

35 Commits
Author SHA1 Message Date
kazu634 cb6fc1df4c 動画を確認画面つきで複数削除できるようにする(ステップ13) 2026-10-15 23:27:51 +09:00
kazu634 fd63a401f7 Merge pull request 'ライブラリを番組一覧・番組ページ・再生の3段にする(ステップ12)' (#16) from library-by-series into main
Reviewed-on: #16
2026-10-12 10:29:13 +09:00
kazu634 6586d7edd2 ライブラリを番組一覧・番組ページ・再生の3段にする(ステップ12) 2026-10-10 12:18:58 +09:00
kazu634 511365f0d4 Merge pull request '番組 URL から番組を登録する(ステップ11)' (#15) from subscribe-by-url into main
Reviewed-on: #15
2026-10-10 10:43:09 +09:00
kazu634 8e0b03483b 番組 URL から番組を登録する機能を追加(ステップ11) 2026-10-07 23:06:39 +09:00
kazu634 e305791423 Merge pull request 'DB のバックアップと復元確認を追加(ステップ10)' (#14) from db-backup into main
Reviewed-on: #14
2026-10-04 18:39:59 +09:00
kazu634 67f8f81fcf NAS でのバックアップの実機結果を反映 2026-10-04 18:38:54 +09:00
kazu634 fb3e128757 DB のバックアップと復元確認のスクリプトを追加(NAS の DSM タスクスケジューラ用) 2026-10-04 16:03:54 +09:00
kazu634 d4346c06fe Merge pull request 'ファイル名の上限拡大と os.Root への置き換え(ステップ9)' (#13) from path-hardening into main
Reviewed-on: #13
2026-10-04 14:56:03 +09:00
kazu634 a2dd67b8b6 Merge remote-tracking branch 'origin/main' into path-hardening
# Conflicts:
#	PROJECT.md
2026-10-04 14:53:53 +09:00
kazu634 3fb3227aa7 Merge pull request '本番配置の結果を反映(ステップ8)' (#12) from deploy-results into main
Reviewed-on: #12
2026-10-04 14:51:32 +09:00
kazu634 9b856d63fa ファイル名の上限を拡大し、動画の配信・削除を os.Root に置き換え(エピソード ID は 64 バイト上限) 2026-10-04 13:43:07 +09:00
kazu634 3ddd2063b6 本番配置の結果を反映(配置用ユーザーでの clone、動作確認結果、新着 100 件の調査) 2026-10-04 13:13:02 +09:00
kazu634 f963dea2d1 Merge pull request '本番配置の準備を追加(ステップ7)' (#11) from deploy-setup into main
Reviewed-on: #11
2026-10-04 12:20:08 +09:00
kazu634 37d42851ff 本番配置の準備を追加(NAS の DB 定義、配置手順書、保存先の決定) 2026-10-04 12:01:19 +09:00
kazu634 80d9606750 Merge pull request '登録番組の定期確認を追加(ステップ6b)' (#10) from subscription-check into main
Reviewed-on: #10
2026-10-04 10:44:02 +09:00
kazu634 944086c8ae 登録番組の定期確認を追加(24 時間ごと、予告の除外、確認エラーの記録) 2026-10-03 23:42:25 +09:00
kazu634 82536c6398 Merge pull request 'worker を追加(ステップ6a)' (#9) from worker-download into main
Reviewed-on: #9
2026-10-03 23:27:49 +09:00
kazu634 6f150495d5 worker を追加(yt-dlp によるダウンロード、段階別進捗、ffprobe 検査、実行中の取り消し) 2026-10-03 22:56:14 +09:00
kazu634 ebf1cfcbc2 Merge pull request 'ライブラリ・再生・削除を追加(ステップ5)' (#8) from library into main
Reviewed-on: #8
2026-10-03 20:57:02 +09:00
kazu634 276cd7b5b8 ライブラリ・再生・削除を追加(Range 配信、パスの安全確認、共有ファイル名規則) 2026-10-03 20:54:16 +09:00
kazu634 637ee5b81a Merge pull request '番組登録を追加(ステップ4)' (#7) from subscriptions into main
Reviewed-on: #7
2026-10-03 19:50:49 +09:00
kazu634 787eda79ae 番組登録を追加(登録時の過去回の選択、一時停止・再開・解除、既知の回の記録) 2026-10-03 19:44:57 +09:00
kazu634 2e82a690b1 Merge pull request '待ち行列への登録と状態表示を追加(ステップ3)' (#6) from queue-ui into main
Reviewed-on: #6
2026-10-03 19:12:03 +09:00
kazu634 4a02526e76 待ち行列への登録と状態表示を追加(新着からの登録、/queue、DATABASE_URL) 2026-10-03 19:07:53 +09:00
kazu634 a71b915c81 Merge pull request 'DB の土台を追加(ステップ2)' (#5) from db-foundation into main
Reviewed-on: #5
2026-10-03 18:23:11 +09:00
kazu634 edc8149202 DB の土台を追加(マイグレーション、待ち行列・番組登録の操作、embedded-postgres によるテスト) 2026-10-03 18:07:58 +09:00
kazu634 e2d3f83db4 Merge pull request '新着一覧の画面を追加(ステップ1)' (#4) from newer-list into main
Reviewed-on: #4
2026-10-03 17:27:53 +09:00
kazu634 f0d73a14e1 新着一覧の画面を追加(ジャンル切替、予告の除外、10分キャッシュ) 2026-10-03 17:20:44 +09:00
kazu634 ffe7451d04 Merge pull request 'web コンテナの土台を追加(ステップ0)' (#3) from scaffold-web into main
Reviewed-on: #3
2026-10-03 17:04:41 +09:00
kazu634 5125614b93 web コンテナの土台を追加(HTTP サーバー、TVer API クライアント、htmx の画面枠、Dockerfile・compose.yaml) 2026-10-03 17:01:50 +09:00
kazu634 54de47fbfe Merge pull request 'AGENTS.md: チーム構成の表を整形し、Developer の役割を「テスト作成」に修正' (#2) from update-agents-md into main
Reviewed-on: #2
2026-10-03 16:57:11 +09:00
kazu634andClaude Opus 5.5 0d74a5d340 AGENTS.md: チーム構成の表を整形し、Developer の役割を「テスト作成」に修正
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BcuBr9DyfcqZY8xFQsY5Di
2026-10-03 16:56:01 +09:00
kazu634 4ad0d621e1 Merge pull request 'PROJECT.md を追加: 要件定義の初版' (#1) from add-project-md into main
Reviewed-on: #1
2026-10-03 16:32:47 +09:00
kazu634andClaude Opus 5.5 d7e01878e4 PROJECT.md を追加: 要件定義の初版
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BcuBr9DyfcqZY8xFQsY5Di
2026-10-03 16:31:05 +09:00
72 changed files with 11035 additions and 5 deletions
+6
View File
@@ -0,0 +1,6 @@
.git
.gitignore
PROJECT.md
AGENTS.md
.env
.env.*
+4
View File
@@ -0,0 +1,4 @@
DATABASE_URL=
VIDEO_HOST_DIR=/mnt/nas-tver/tver
VIDEO_DIR=/videos
WORKER_CONCURRENCY=1
+1
View File
@@ -4,6 +4,7 @@
# 秘匿情報(homelab 規約: .env に分離し、コミットしない) # 秘匿情報(homelab 規約: .env に分離し、コミットしない)
.env .env
.env.* .env.*
!.env.example
# エージェントのローカル設定(端末ごとの権限・agmsg のフック) # エージェントのローカル設定(端末ごとの権限・agmsg のフック)
.claude/settings.local.json .claude/settings.local.json
+5 -5
View File
@@ -6,11 +6,11 @@
## チーム構成 ## チーム構成
| エージェント | 種別 | 役割 | | エージェント | 種別 | 役割 |
|---|---|---| | --------------- | ----------- | --------------------------------------------- |
| **`Leader`** | claude-code | 人間の窓口。計画・タスク分解・依頼・検収 | | **`Leader`** | claude-code | 人間の窓口。計画・タスク分解・依頼・検収 |
| **`Developer`** | codex | 実作業。実装・テスト実行・修正 | | **`Developer`** | codex | 実作業。実装・テスト作成・修正 |
| **`QA`** | codex | テストの品質保証。テスト内容の検証・必要に応じたテストの修正・テスト実施・テスト結果の整理 | | **`QA`** | codex | テストの品質保証。テスト内容の検証・必要に応じたテストの修正・テスト実施・テスト結果の整理 |
参加は人間の指示を受けてから行う。名前は上の表のとおりにする(`whoami.sh` が出す候補名は他チームのものを含むので使わない)。 参加は人間の指示を受けてから行う。名前は上の表のとおりにする(`whoami.sh` が出す候補名は他チームのものを含むので使わない)。
+16
View File
@@ -0,0 +1,16 @@
FROM golang:1.27.1-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY cmd ./cmd
COPY internal ./internal
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /web ./cmd/web
FROM alpine:3.22
RUN apk add --no-cache ca-certificates tzdata
WORKDIR /app
COPY --from=build /web /app/web
USER 65532:65532
EXPOSE 8080
ENTRYPOINT ["/app/web"]
+21
View File
@@ -0,0 +1,21 @@
FROM golang:1.27.1-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY cmd ./cmd
COPY internal ./internal
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /worker ./cmd/worker
FROM python:3.13-slim
ARG YT_DLP_VERSION=2026.08.19
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates ffmpeg tzdata \
&& pip install --no-cache-dir "yt-dlp[default]==${YT_DLP_VERSION}" \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=build /worker /app/worker
ENV HOME=/tmp \
XDG_CACHE_HOME=/tmp/yt-dlp-cache
USER 65532:65532
ENTRYPOINT ["/app/worker"]
+182
View File
@@ -0,0 +1,182 @@
---
title: tver の現状
tags:
- homelab
- tver
status: 本番稼働中
---
# tver の現状
> [!info] このファイルの位置づけ
> tver プロジェクトの現状(決定事項・要件・構成・課題)を積み上げていく。運用規約は [[AGENTS]] を参照。
## 今の状態
- Go の HTTP サーバー、TVer API クライアント、ドラマ・バラエティの新着表示、確認画面つきの動画複数削除、テスト、web・worker コンテナの定義がある
- **本番稼働中。** dockageのweb・worker、NASのPostgreSQL 17、NFS動画保存を構成し、実動画の取得・再生・Range配信まで確認済み
- リポジトリ: `https://gitea.kazu634.com/kazu634/tver.git`
## 目的
TVer(`https://tver.jp/`)で公開中の動画情報を取得し、選んだ動画を yt-dlp でダウンロードし、アプリ内で再生する Web アプリを作る。
## 決定事項
| 項目 | 決定 |
|---|---|
| TVer 利用規約との関係 | 鑑賞を前提とするダウンロードのため、問題無しと判断(ユーザー判断) |
| 利用者・公開範囲 | 自分のみ・**LAN 内だけ**。EC2 nginx では公開しない |
| 保存先 | **NAS(ds923p)の `/volume1/video/tver`**。既存の`anime`・`drama`・`misc`・`personal`・`variety`と分け、admin相当の書き込み範囲を限定する。`/volume1/video`をdockageの`/mnt/nas-tver`へNFSマウントし、`.env`は`VIDEO_HOST_DIR=/mnt/nas-tver/tver`とする |
| 自動ダウンロード | **作る**(番組を登録し、新しい回を自動でダウンロードする) |
| 再生 | **アプリの中で再生する** |
| ダウンロード | yt-dlp を使う |
| デプロイ | 読み取り専用のGiteaユーザー`tver-deploy`でdockage VMの`/opt/stacks/tver`へcloneし、`docker compose up -d --build`で起動する。Dockgeにも同じスタックが表示される。更新は`git pull --ff-only`後に再ビルドする |
| 配置用のGit認証 | `tver-deploy`を`tver`の読み取り共同作業者とし、repository読み取りトークンをdockageの権限`600`の専用credentialファイルに保存する。トークンの実値はGitや文書へ残さない |
| 開発言語 | **Go**(`web`・`worker` とも)。yt-dlp は外部コマンドとして呼び、進捗・結果は yt-dlp の出力から読み取る |
| DB | **NAS上のtver専用Postgres 17コンテナ**(Giteaと同じ方式)。ユーザー・DB名は`tver`、公開ポートは`15434`。dockageからの到達を確認済み |
| DB ドライバ | `github.com/jackc/pgx/v5` の `pgxpool` |
| DB テスト | `github.com/fergusstrange/embedded-postgres` v1.34.0 の Postgres 17 を起動する。Apple Silicon では Rosetta 2 が必要 |
| 定期実行 | **`worker` 内のスケジューラ** |
| NAS のマウント | **NFS**。dockageの`/etc/fstab`へ`_netdev,nofail`付きで登録し、DSMでは接続元`192.168.10.155`だけを許可する |
| NFS 権限 | 読み書き、Squash「すべてのユーザーをadminに割り当てる」、`sys`、非同期オン。DSMのadminアカウントが無効でもUID `65532`から書き込み確認済み |
| DBの通信制限 | DSMファイアウォールでTCP `15434`を`192.168.10.155`からだけ許可する |
| DBバックアップ | NASのDSMタスクスケジューラからrootで毎日実行する。`tver-postgres`内の`pg_dump`をgzip圧縮して`/volume1/backups/tver/tver_<YYYYmmdd_HHMMSS>.sql.gz`へ保存し、14日より古い世代を削除する。同名の完成ファイルは上書きせず失敗させる。初回は一時DBへの復元と主要4表の行数一致まで確認する |
| 保持期間 | **削除は手動のみ**。自動削除はしない |
| 待ち行列 | **Postgres のテーブルだけで実装する**。専用のキュー基盤は持たない |
| 定期確認の間隔 | 登録番組の新しい回を **24 時間ごと**に確認する |
| 定期確認の走査 | worker のスケジューラが起動直後と1時間ごとに、最終確認から24時間以上経過した有効な番組を探す。番組は1件ずつ確認し、間に数秒空ける |
| 定期確認の予告 | **180秒以下**の回は自動登録せず、既知としてだけ記録する。新着画面の予告判定と同じしきい値を使う |
| 定期確認の失敗 | `last_checked_at` を更新せず、失敗日時と秘匿値を伏せた短いメッセージを保存する。次の1時間走査で再試行し、成功時にエラーを消す |
| 登録時の既存回 | **登録の都度**「公開中の回もダウンロードする」かを選ぶ。選ばない場合も公開中の回を既知として記録する |
| 番組の一時停止 | 登録は残し、自動ダウンロードだけ止める。再開後は停止中に公開された未確認の回も対象にする |
| 動画のマウント先 | web と worker は `VIDEO_DIR`(既定 `/videos`)に同じ NAS 共有をマウントする。`download_jobs.file_path` は `VIDEO_DIR` からの相対パスを保存する |
| ファイル名 | `<番組名>/<番組名> <回タイトル> [<エピソードID>].mp4`。禁止文字と制御文字を `_` に置換し、空の要素は `名称未設定`とする。番組名はUTF-8で90バイト、回タイトルは140バイト、サニタイズ後のエピソードIDは64バイトまでとし、ファイル名全体を255バイト以下に収める。64バイトを超えるIDはworkerで`invalid_output`にする。実際のTVer IDは約10文字で、yt-dlpのextractorも英数字として扱うため64バイトを上限とした。既存ファイルは改名せず、新しい保存から適用する |
| 動画ファイルの境界 | webは`VIDEO_DIR`を`os.OpenRoot`で開き、DBの相対`file_path`を`Root.Open`・`Root.Remove`へ渡す。`VIDEO_DIR`外へのパストラバーサルと外向きシンボリックリンクをOSのディレクトリ境界で拒否する |
| 同名ファイル | 移動先に同名ファイルがあれば上書きする。ファイル名にエピソード ID が入り `episode_id` は一意なので、同名は同じ回の前回処理の残りに限られる。再実行で同じ結果になることを優先する |
| ライブラリ一覧 | DBの完了済みジョブから作り、NASのフォルダ走査やファイル存在確認は行わない。番組一覧、番組ごとの回一覧、再生の3段構成にする |
| 回の自然順 | 回タイトル中の半角・全角数字の並びを数値として比較し、同順位は完了日時の古い順、さらにジョブIDの小さい順にする |
| 動画の削除 | アプリからだけ行い、削除前にサーバーが描画する確認画面を必ず挟む。ライブラリでは番組単位、番組ページでは選択した回または番組全体を複数削除できる。確認画面で確定したジョブ ID だけを実行するため、確認後に増えた回は削除しない。各回はファイルを先に削除してからジョブ記録を削除し、親の番組フォルダが完全に空なら削除する。ファイルが既に無い場合も記録は削除し、1件の失敗で残りを止めない。登録番組の `known_episodes` は残す |
| 同時ダウンロード数 | 既定は **1**。`WORKER_CONCURRENCY` で **2まで**増やせる |
| ダウンロードの再試行 | 自動では再試行しない。失敗を `failed` で残し、画面から手動で再試行する |
| 実行中の取り消し | web で `running` を `canceled` にし、worker が検知して yt-dlp のプロセスグループを停止する |
| yt-dlp の作業場所 | `VIDEO_DIR/.incoming/<job id>/` に `%(id)s.%(ext)s` で保存し、検査後に Go が `media.RelativeVideoPath` の場所へ同一ファイルシステム内で移動する。終了状態にかかわらず作業場所を片付ける |
## 要件
### 機能要件
| # | 要件 | 補足 |
|---|---|---|
| F1 | ドラマ・バラエティの新着一覧を表示する | 番組名・回タイトル・放送局・長さ・配信終了日時・サムネイル |
| F2 | 一覧を絞り込む | 予告(「60秒PR」等)を除外できる。ダウンロード済み・待ち行列中を区別して表示する |
| F3 | 選んだ動画をダウンロードの待ち行列に入れる | 複数選択できる |
| F4 | ダウンロードの進捗・結果を表示する | 失敗理由(DRM・地域制限・TVer 側の変更による抽出失敗)を区別して表示する |
| F5 | 同じ回を二重にダウンロードしない | 手動・自動の両方で共通 |
| F6 | 番組(シリーズ)を登録し、新しい回を自動でダウンロードする | 登録・解除・一覧。定期的に新しい回を確認する。番組URL(`tver.jp/series/…`)から登録でき、登録前に確認画面を表示する |
| F7 | ダウンロード済みの動画をアプリ内で再生する | シーク(HTTP Range)に対応する |
| F8 | ダウンロード済みの動画を一覧・削除する | NAS の容量管理のため |
| F9 | 保存ファイルの名前と置き場所を規則で決める | 例: `<番組名>/<番組名> <回タイトル> [<エピソードID>].mp4` |
### 非機能要件
- **yt-dlp を素早く更新できること。** TVer 側の変更で抽出が壊れやすいので、worker のイメージを再ビルドするだけで更新できるようにする
- **TVer への負荷を抑える。** 一覧取得はキャッシュし、同時ダウンロード数は 1〜2 に制限する
- **ブラウザで再生できる形式で保存する。** 映像 H.264 + 音声を MP4 に結合する(ffmpeg が必要)
- 認証は持たない(LAN 内限定のため)
- homelab の規約に従う([[homelab/docs/conventions|運用規約]]): Compose v2、秘匿情報は `.env`、`TZ=Asia/Tokyo`
## 構成(案)
| コンテナ | 役割 |
| -------- | ----------------------------------------------------- |
| `web` | UI・API。新着一覧の取得、待ち行列への登録、番組URLを含む番組登録、動画の配信(再生) |
| `worker` | Go + yt-dlp + ffmpeg。待ち行列からダウンロードし、登録番組の新しい回を定期的に確認する |
- 現在の `web` は `/healthz`、新着(`/`)、待ち行列(`/queue`)、登録番組(`/subscriptions`)、番組単位のライブラリ(`/library`)、番組ごとの回一覧(`/library/series/{series_id}`)、再生ページ(`/library/{id}`)、Range 対応の MP4 配信(`/videos/{id}`)を提供する
- ライブラリはDBの完了済みジョブだけを基に、番組一覧から番組ページ、再生ページへ進む。番組ページは完了が新しい順と、回タイトルの半角・全角数字を数値比較する話の順を切り替えられる。再生ページは話の順で前後の回へ移動できる
- 動画配信はクライアントからジョブ ID だけを受け取り、DB の相対 `file_path` を `os.Root` の `Open`・`Remove` で扱う。絶対パス・`..`・`VIDEO_DIR` 外を指すシンボリックリンクをディレクトリ境界で拒否する。削除はライブラリまたは番組ページの選択、もしくは再生ページの1件削除からサーバー描画の確認画面へ進み、解決済みのジョブ ID を実行画面へ引き継ぐ。実行時は1リクエストにつき動画ルートを1回だけ開き、各回を独立して処理して削除・失敗・対象外の件数を結果画面へ表示する。動画削除後に実ディレクトリの親が空なら削除する。削除後は新着から手動で再登録できるが、登録番組の既知記録は残るため自動再登録しない
- 新着の「番組を登録」から、公開中の回をダウンロードするか選ぶ。web はサーバー側の新着キャッシュから番組 ID と名前を引き、TVer の season ごとの公開中エピソードを取得する。取得失敗時は登録しない
- `/subscriptions`では`https://tver.jp/series/<ID>`形式の番組URLを受け付け、検証した番組IDからTVer APIの接続先を組み立てる。登録前に番組名・番組ID・公開中の回数を確認し、登録時にもサーバー側で番組名と公開中の回を取り直す。クライアントから番組名は受け取らない
- 登録と公開中の回の既知記録・任意の待ち行列投入は1トランザクションで行い、登録時刻を最終確認日時に設定する。`known_episodes` は番組解除時に連動して削除する。一時停止中は投入を拒否する
- worker のスケジューラは起動直後と1時間ごとに、`enabled` かつ最終確認から24時間以上の番組を抽出する。TVer から公開中の回を順次取得し、未確認の180秒超の回だけを `source=subscription` で投入する。180秒以下も既知にし、成功時の最終確認更新・エラー消去まで1トランザクションで行う
- TVer の番組取得に失敗した場合は最終確認日時を変えず、失敗日時と伏字済みの短いメッセージを `subscriptions` に記録して画面に表示する。番組間には3秒空け、SIGTERM では取得中または待機中の確認を中断する
- 新着の登録ボタンはエピソード ID とジャンルだけを送る。web が有効な新着キャッシュから番組情報を引き、`episode_id` の一意制約で重複登録を防ぐ。新着一覧はジョブ状態を一括取得し、失敗・取り消しには再試行を表示する
- 待ち行列は実行中・待機中・失敗・取り消し・完了を状態別に直近50件ずつ表示する。段階別の進捗と失敗理由を表示し、待機中の取り消しと失敗・取り消しの再試行ができる。実行中または待機中のジョブがある間だけ5秒ごとに一覧を更新する
- 実行中のジョブにも取り消しボタンを表示する。worker は DB の `canceled` を定期確認し、yt-dlp をプロセスグループごと停止する
- 新着はジャンルごとにメモリへ 10 分間キャッシュする。キャッシュ中は TVer セッションの再発行と一覧の再取得を行わない。TVer の取得に失敗した場合、一覧の場所にエラーを表示して詳細をログへ記録する
- 「予告を隠す」は既定でオンとし、180 秒以下の件を非表示にする。長さは秒から分へ切り上げて表示し、配信終了日時は Asia/Tokyo で表示する。ジャンルと予告表示の状態は URL に残す
- 画面は `html/template` と htmx 2.0.11 を使い、htmx は Go の `embed` から `/static/` でローカル配信する
- `compose.yaml` は web と worker を定義する。web は dockage VM の LAN アドレス `192.168.10.155:8080` で公開し、待ち受けは `WEB_ADDR` で変更できる。両方へ `DATABASE_URL` と同じ動画ボリュームを渡し、worker の並行数は `WORKER_CONCURRENCY`(既定1、最大2)で指定する
- NAS用の`deploy/nas/tver-db/compose.yaml`は`postgres:17`を`15434:5432`で公開し、`/volume1/docker/tver-db/pgdata`へ保存する。パスワードは同ディレクトリのGit管理外`.env`から注入する
- NAS用の`backup-db.sh`はコンテナ内の`pg_dump`を検査後にgzip圧縮し、同一ディレクトリ内のハードリンクで完成ファイルを上書きせず公開する。`restore-test.sh`は最新ダンプを一時DBへ復元して`download_jobs`・`subscriptions`・`known_episodes`・`schema_migrations`の行数を本番DBと比較する。一時DBはEXIT trapで必ず削除し、復元SQLの通常出力はログへ記録しない
- 本番配置と更新の手順は[[docs/deploy|tver 本番配置手順]]にまとめる。dockageでは`/volume1/video`を`/mnt/nas-tver`へNFSマウントし、その配下の`tver`だけをコンテナ内の`/videos`へ渡す
- Giteaは匿名閲覧とSSHを無効にしているため、dockageのclone・pullは読み取り専用`tver-deploy`のrepository読み取りトークンを使う。Gitのcredential helperはGiteaのHTTPS URLに限定する
- DB は NAS 上に tver 専用 Postgres 17 コンテナを置く。`internal/store` の埋め込み SQL を `schema_migrations` で管理し、`pg_advisory_lock` で複数プロセスからの同時適用を直列化する。web は起動時に `pgxpool` で接続してマイグレーションを適用する
- `download_jobs` は `episode_id` を一意にして二重登録を防ぎ、ジョブの状態・段階別進捗・結果を持つ。`subscriptions` はシリーズ単位の登録と最終確認日時を持つ。worker からの操作と画面用の一覧取得を `internal/store` にまとめる
- worker は `yt-dlp[default]` と ffmpeg を外部コマンドとして使い、映像・音声・結合・移動の段階別進捗を1秒間隔に間引いて保存する。独立した heartbeat を5秒ごとに更新し、正確な全体パーセントは保存しない。`after_move:filepath` で実パスを取り、ffprobe で H.264 と AAC の MP4 を確認して成功とする。失敗は取り消し、DRM、地域制限、ネットワーク、配信不可、抽出失敗、出力不正、その他の順に扱い、自動再試行しない
- worker は起動時にマイグレーションと停止中ジョブの `RequeueStale` を行い、登録番組の確認も直ちに1回行う。SIGTERM では yt-dlp と確認処理を止め、担当中ジョブを `queued` に戻してから終了する。`Dockerfile.worker` は yt-dlp 2026.08.19 を ARG で固定し、ffmpeg・ffprobe とともに非 root で実行する
- 保存先の NAS 共有フォルダは dockage VM に NFS でマウントし、`worker`(書き込み)と `web`(読み取り・削除)に同じ `VIDEO_DIR` でバインドマウントする
## 外部とのつながり
| 相手 | 用途 | 備考 |
|---|---|---|
| TVer `platform-api.tver.jp` | 新着一覧(`/service/api/v1/callNewerDetail/{drama\|variety}`)・番組名(`/service/api/v2/callSeries/{series_id}`)・番組のエピソード一覧 | **非公開 API**。セッションは `/v2/api/platform_users/browser/create` で発行する(yt-dlp と同じ方式) |
| TVer `service-api.tver.jp` | 番組の season 一覧(`/api/v1/callSeriesSeasons/{series_id}`) | **非公開 API**。[yt-dlp の TVer extractor](https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/tver.py) と同じ取得方式 |
| yt-dlp | ダウンロード | TVer extractor を使う |
| Gitea `gitea.kazu634.com` | 配置元 | 匿名cloneとSSHは使えない。`tver-deploy`のHTTPSトークンでclone・pullする |
| NAS ds923p(NFS) | 動画の保存先 | `/volume1/video`をdockageの`/mnt/nas-tver`へNFSv3でマウントし、アプリは`tver`サブフォルダを使う。DSMのNFS権限は`192.168.10.155`のみ |
| NAS ds923p(Postgres 17) | DB | tver専用コンテナを`15434:5432`で公開し、DSMファイアウォールで`192.168.10.155`のみ許可する。dockageから接続確認済み |
## 確認済みの事実
- 新着 API は `drama` / `variety` それぞれ 100 件を返す。予告動画も含まれる。各件に `id`・`seriesID`・`seriesTitle`・`title`・`broadcasterName`・`duration`・`endAt`・`thumbnailPath` がある [実機] 2026-10-03 確認
- 番組 `sr0lhdrtdb` の `callSeriesSeasons` は `result.contents` に `type=season` の1件(`ss3i6njoq3`)を返し、`callSeasonEpisodes` は `type=episode` の5件を返した。エピソードには新着と同じ `id`・`seriesID`・`seriesTitle`・`title`・`broadcasterName`・`duration`・`endAt`・`thumbnailPath` が含まれる。他の番組での season 数や一覧件数は未確認 [実機] 2026-10-03 確認
- dockage VM から TVer API に到達でき(HTTP 200)、送信元は日本と判定される [実機] 2026-10-03 確認
- yt-dlp 2026.08.19 で TVer のエピソードを抽出できる(`--simulate`)。映像(H.264 / `avc1`)と音声が別ストリームで、結合に ffmpeg が要る。1080p・約 50 分で約 1.2GiB [実機] 2026-10-03 確認
- 5 分の `epq432nee4` を yt-dlp 2026.08.19 で取得した。`--newline` と `--progress-template` の 1 行 1 JSON は Go の外部プロセスから逐次読める。ただし `uvx yt-dlp` のみで `pycryptodomex` が無いと AES-128 HLS を ffmpeg に委譲し、途中のダウンロード進捗が出なかった。`yt-dlp[default]` に含まれる `pycryptodomex` を加えた試行では途中の進捗が出た [実機] 2026-10-03 確認
- 別ストリームの映像と音声は進捗が個別に始まり、`format_id` で区別できる。結合処理は `postprocess:` の出力で検知でき、`--print after_move:filepath` で実ファイルのパスを取得した。完成した MP4 は 24,613,787 バイトで、ffprobe では H.264 640×360 と AAC LC だった。初回取得は約 10 秒だった [実機] 2026-10-03 確認
- SIGTERM で止めると `.part` と `.ytdl` が残り、再実行で 437,066 バイトから 438,090 バイトへ続きから取得できた [実機] 2026-10-03 確認
- 到達不能のプロキシ経由の通信失敗と存在しないエピソードは、ともに yt-dlp の終了コード 1 だった。stderr の文言で分類が必要。DRM・地域制限の失敗文言と終了コードは実測していない(yt-dlp の extractor ソースのみ確認) [実機] 2026-10-03 確認
- yt-dlp の TVer extractor は日本のみ(`_GEO_COUNTRIES = ['JP']`)。DRM 付きの作品は取得できない [公式] https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/tver.py (取得日: 2026-10-03)
- DSMのNFS権限を接続元`192.168.10.155`、読み書き、全ユーザーをadminへ割り当て、`sys`、非同期オンにすると、dockageからNFSv3で`/volume1/video`をマウントできた。設定直後の1回目は`access denied`、直後の2回目は成功したため、権限の反映待ちと考えられる [実機] 2026-10-04 確認
- DSMのadminアカウントが無効でも、UID `65532`のコンテナから`tver`サブフォルダの作成・書き込み・読み込み・rename・削除に成功した。NAS上の所有者はUID `1024`・GID `100`。`/mnt/nas-tver/tver`は作成済み [実機] 2026-10-04 確認
- NFSマウントのファイル名上限は`namlen=255` [実機] 2026-10-04 確認
- Giteaは`REQUIRE_SIGNIN_VIEW = true`・`DISABLE_SSH = true`で、匿名HTTPSの`info/refs`はHTTP `401`だった。読み取り専用ユーザー`tver-deploy`とrepository読み取りトークンで`/opt/stacks/tver`へのcloneに成功した [実機] 2026-10-04 確認
- dockageで`docker compose up -d --build`を実行してweb・workerを起動でき、Dockgeにも`tver`スタックとして表示された。DSMファイアウォール設定後、dockageからNASのTCP `15434`へ到達できた [実機] 2026-10-04 確認
- 本番の`/healthz`はHTTP `200`、新着バラエティは98件だった。4分の本編`ep5hcgopp2`は登録から20秒以内に完了し、92,340,468バイトの1080p H.264/AAC MP4をNASへ保存した。所有者はUID `1024`・GID `100`、`.incoming`は空で、`/library`とHTTP Range `206`も確認した [実機] 2026-10-04 確認
- NASは番組フォルダ内に所有者UID `0`の`@eaDir`を作る。ライブラリ一覧はDBから作るため、アプリへの影響はない [実機] 2026-10-04 確認
- `callNewerDetail/drama`は毎回100件を返し、応答に続きや総件数を示す項目はなかった。`result.contents`は`type=newer`で、`content`は`id`・`version`・`title`のみ。`page=2`・`offset=100`・`limit=200`を付けても先頭・末尾IDが同じ100件となり、パラメータは無視された。100件を超えて取得する手段は見つからなかったが、非公開APIのため不可能とは断定しない。新着一覧は直近100件までとなる一方、登録番組の自動確認は番組ごとの`callSeriesSeasons`→`callSeasonEpisodes`を使うため、新着から外れた回も取りこぼさない [実機] 2026-10-04 確認
- dockage上の使い捨て`postgres:17`コンテナへマイグレーション001〜003を適用してテスト用の行を投入し、`backup-db.sh`と`restore-test.sh`を実行した。gzip圧縮・権限600のダンプ作成、同じ`STAMP`での2回目が`ln`の失敗で非0終了して既存ダンプを保持すること、主要4表の行数一致、一時DBの削除、ログに`psql`の標準出力が残らないことを確認した [実機] 2026-10-04 確認
- NASの`/volume1/docker/tver-db/`へスクリプトを配置し、DSMタスクスケジューラのrootタスクから実行して、`/volume1/backups/tver/`へ`tver_20261004_182838.sql.gz`(2,286バイト)と`tver_20261004_183256.sql.gz`(2,289バイト)を作成した。ダンプはroot所有・権限600で、バックアップディレクトリはroot所有・`drwxr-xr-x`だった。`restore-test.sh`を`sudo`で実行し、`download_jobs` 1件・`subscriptions` 1件・`known_episodes` 1件・`schema_migrations` 3件の一致、ログへの結果記録、一時DBの削除を確認した。一般ユーザーではroot所有のログへ書き込めず失敗する [実機] 2026-10-04 確認
- DSMのACLで全員書き込み可だった`/volume1/docker`配下について、`/volume1/docker/tver-db`を`chmod 755`、`backup-db.sh`と`restore-test.sh`を`chmod 700`にしてACLが外れたことを確認した。変更後も本番アプリの`/queue`と`/subscriptions`はHTTP `200`でDBを読み取れた。PostgreSQLコンテナ再起動後の動作は未確認 [実機] 2026-10-04 確認
- platform-apiの`v2/callSeries/{series_id}`は`result.content.content.title`に番組名を返し、存在しないIDにはHTTP `404`を返した。`sr8a864g0w`では番組名がエピソード側の`seriesTitle`と一致した。`v1/callEpisode/{episode_id}`は`seriesID`・`seriesTitle`を返した(今回は使わない) [実機] 2026-10-07 確認
## 課題
- 登録済みのDSM日次バックアップタスクが翌日以降に自動実行され、新しいダンプを作ることを確認する [要確認]
- DRM・地域制限の実例探しはTVerへの大量アクセスになるため行わず、運用中に観測する。workerは失敗時の`error_message`をDBへ保存し、既知の文言に一致しない失敗は`unknown`にする。実例が出た時点で分類を見直す
## 経緯
- ステップ13で、ライブラリの番組単位削除、番組ページの回選択・番組全体削除、再生ページの1件削除を、すべてサーバー描画の確認画面を経由する方式に統一した。確認時に解決した最大1,000件のジョブ ID だけを実行し、途中の失敗後も残りを処理して結果を表示する
- ステップ12でライブラリを番組一覧、番組ページ、再生ページの3段構成にした。番組単位の集計と登録状態はDBから取得し、番組ページの話の順と再生ページの前後移動には同じ自然順を使う。動画削除後は空になった番組フォルダだけを`os.Root`経由で削除する
## 確かめ方
- Codex では `go test -count=1 ./...` を環境変数の前置なしで実行する。`go test` を許可する rules に一致させ、Postgres の共有メモリ作成に必要な権限で実行するため。初回のモジュール取得にはネットワーク接続が必要
- web を起動するには `.env.example` を参考に、Git 管理外の `.env` に実際の `DATABASE_URL` と `VIDEO_HOST_DIR` を設定する。`VIDEO_DIR` の既定は `/videos`。DB 接続がある環境では起動時にマイグレーションが適用される
- DB テストは `embedded-postgres` v1.34.0 が Postgres 17 を一度起動し、テストごとに独立した DB を作る。バイナリキャッシュ、実行ファイル、データは `/private/tmp` 配下に置く。Apple Silicon での実行には Rosetta 2 の導入が必要
- TVer 応答のテストは `internal/tver/testdata/newer_drama.json`(2026-10-03 の実応答を 3 件に縮小)と `httptest` を使い、テストから本物の TVer にはアクセスしない
- 番組一覧の応答は `internal/tver/testdata/series_seasons.json` と `season_episodes.json`(2026-10-03 の実応答を各1件に縮小)に記録した。テストは `httptest` を使い、実 API にアクセスしない
- 番組情報の応答は`internal/tver/testdata/series.json`(2026-10-07の実応答を番組情報だけに縮小)に記録した。番組URLは純関数でscheme・host・path・IDを検証し、TVerクライアントのテストは`httptest`だけへ接続する
- 新着画面のテストは偽物の TVer クライアント・store と差し替え可能な時計を使い、表示項目、絞り込みの境界、10 分キャッシュ、取得失敗、JST 表示、登録・状態表示・待ち行列の操作を確認する
- ライブラリのテストは番組単位の集計、番組別・ID指定の完了済み回取得、3画面、半角・全角数字の自然順、前後移動を確認する。一時ディレクトリの小さな動画を使い、Rangeの206・範囲外の416、`os.Root`による相対パスとシンボリックリンクの拒否、404、サーバー確認画面の入力検証・重複除去・HTMLエスケープ、複数削除の部分失敗・対象外・確認後の新着保持、ファイル削除後の記録削除、空の番組フォルダだけを削除することも確認する。共有のファイル名規則は`internal/media`で90・140・255バイトの境界とUTF-8の完全性をテストする
- worker のテストは `internal/worker/testdata` の偽 yt-dlp・ffprobe を使い、本物の TVer へ接続せずに正常完了、失敗分類、取消、終了時の待機復帰、進捗の間引き、heartbeat、出力検査、移動先、作業場所の片付けを確認する
- 登録番組の定期確認は偽の TVer クライアントと差し替え可能な時計・タイマーを使い、24時間の境界、順次確認、1時間走査、失敗記録、予告の既知化、終了時の中断を確認する。store の抽出・既知化・投入・日時更新・エラー消去は embedded Postgres 17 で確認する
- NAS用Composeは`deploy/nas/tver-db/compose.yaml`をYAMLパーサーで読み、`services.postgres`、環境変数、ポート、保存先、healthcheckを確認する。Dockerが使える環境では`docker compose --env-file deploy/nas/tver-db/.env.example -f deploy/nas/tver-db/compose.yaml config`でも展開結果を確認する
- DBバックアップスクリプトは`deploy/nas/tver-db/scripts_test.go`からPATH上の偽dockerを使って実行し、正常な圧縮、同名・同時実行時の上書き防止、空ダンプと`pg_dump`失敗、14日の世代境界、復元失敗時の一時DB削除、主要表の行数不一致を確認する。NAS実機では[[docs/deploy#5. PostgreSQLをバックアップする|配置手順]]に従って手動バックアップと復元確認を行う
- 本番の確認手順は[[docs/deploy|tver 本番配置手順]]に従い、NFS上の`/mnt/nas-tver/tver`をUID `65532`で読み書き・rename・削除できること、DBがhealthyであること、webの`/healthz`、実動画の保存・再生・シークを確認する
- `env GOPATH=/private/tmp/tver-gopath GOCACHE=/private/tmp/tver-go-cache GOMODCACHE=/private/tmp/tver-go-modcache GOPROXY=off go vet ./...` で静的解析する
- `DATABASE_URL`、`VIDEO_DIR`、`WEB_ADDR=127.0.0.1:18080` を設定して `go run ./cmd/web` で web を起動できる。worker は同じ `DATABASE_URL` と `VIDEO_DIR` を設定して `go run ./cmd/worker` で起動する。worker の実行は TVer への実アクセスを伴うため手動確認は少数回に留める
+62
View File
@@ -0,0 +1,62 @@
package main
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"os"
"strings"
"time"
_ "time/tzdata"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
"gitea.kazu634.com/kazu634/tver/internal/web"
"github.com/jackc/pgx/v5/pgxpool"
)
func main() {
pool, err := openDatabase(context.Background(), os.Getenv("DATABASE_URL"), pgxpool.New, store.Migrate)
if err != nil {
log.Fatal(err)
}
defer pool.Close()
address := os.Getenv("WEB_ADDR")
if address == "" {
address = ":8080"
}
client, err := tver.NewClient("", nil)
if err != nil {
log.Fatal(err)
}
videoDir := os.Getenv("VIDEO_DIR")
if videoDir == "" {
videoDir = "/videos"
}
server := &http.Server{
Addr: address,
Handler: web.NewHandler(client, store.New(pool), videoDir, time.Now, log.Default()),
ReadHeaderTimeout: 5 * time.Second,
}
log.Printf("web listening on %s", address)
log.Fatal(server.ListenAndServe())
}
func openDatabase(ctx context.Context, databaseURL string,
connect func(context.Context, string) (*pgxpool.Pool, error),
migrate func(context.Context, *pgxpool.Pool) error) (*pgxpool.Pool, error) {
if strings.TrimSpace(databaseURL) == "" {
return nil, errors.New("DATABASE_URL が設定されていません")
}
pool, err := connect(ctx, databaseURL)
if err != nil {
return nil, errors.New("DATABASE_URL から接続プールを作成できません")
}
if err := migrate(ctx, pool); err != nil {
pool.Close()
return nil, fmt.Errorf("DB マイグレーションに失敗: %w", err)
}
return pool, nil
}
+72
View File
@@ -0,0 +1,72 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestOpenDatabaseRequiresURLAndRunsMigration(t *testing.T) {
ctx := context.Background()
connectCalls, migrateCalls := 0, 0
connect := func(_ context.Context, url string) (*pgxpool.Pool, error) {
connectCalls++
if url != "postgres://test:secret@localhost/test" {
t.Errorf("URL = %q", url)
}
return pgxpool.New(ctx, url)
}
migrate := func(_ context.Context, _ *pgxpool.Pool) error {
migrateCalls++
return nil
}
if _, err := openDatabase(ctx, " ", connect, migrate); err == nil || !strings.Contains(err.Error(), "DATABASE_URL") {
t.Fatalf("未設定 URL のエラー: %v", err)
}
if connectCalls != 0 || migrateCalls != 0 {
t.Fatalf("未設定でも接続・移行された: %d %d", connectCalls, migrateCalls)
}
pool, err := openDatabase(ctx, "postgres://test:secret@localhost/test", connect, migrate)
if err != nil || pool == nil {
t.Fatalf("DB 初期化: %v", err)
}
pool.Close()
if connectCalls != 1 || migrateCalls != 1 {
t.Fatalf("接続・移行回数: %d %d", connectCalls, migrateCalls)
}
}
func TestOpenDatabaseReportsMigrationFailure(t *testing.T) {
ctx := context.Background()
pool, err := openDatabase(ctx, "postgres://test:secret@localhost/test", pgxpool.New,
func(context.Context, *pgxpool.Pool) error { return errors.New("migration failed") })
if pool != nil || err == nil || !strings.Contains(err.Error(), "migration failed") {
t.Fatalf("移行失敗: pool=%v err=%v", pool, err)
}
}
func TestOpenDatabaseReportsConnectionFailureWithoutLeakingURL(t *testing.T) {
migrateCalled := false
pool, err := openDatabase(context.Background(), "postgres://user:secret@localhost/test",
func(context.Context, string) (*pgxpool.Pool, error) {
return nil, errors.New("postgres://user:secret@localhost/test: connection refused")
},
func(context.Context, *pgxpool.Pool) error {
migrateCalled = true
return nil
})
if pool != nil || err == nil || strings.Contains(err.Error(), "secret") || migrateCalled {
t.Fatalf("接続失敗: pool=%v err=%v migrateCalled=%v", pool, err, migrateCalled)
}
}
func TestOpenDatabaseDoesNotExposeInvalidURL(t *testing.T) {
_, err := openDatabase(context.Background(), "postgres://user:secret@%", pgxpool.New,
func(context.Context, *pgxpool.Pool) error { t.Fatal("移行してはいけない"); return nil })
if err == nil || strings.Contains(err.Error(), "secret") {
t.Fatalf("接続 URL の秘匿: %v", err)
}
}
+99
View File
@@ -0,0 +1,99 @@
package main
import (
"context"
"errors"
"fmt"
"log"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
workerpkg "gitea.kazu634.com/kazu634/tver/internal/worker"
"github.com/jackc/pgx/v5/pgxpool"
)
const staleAfter = 30 * time.Second
func main() {
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
config, err := loadConfig(os.Getenv)
if err != nil {
log.Fatal(err)
}
pool, err := initializeDatabase(ctx, os.Getenv("DATABASE_URL"), time.Now(), pgxpool.New,
store.Migrate, func(ctx context.Context, pool *pgxpool.Pool, cutoff time.Time) (int64, error) {
return store.New(pool).RequeueStale(ctx, cutoff)
})
if err != nil {
log.Fatal(err)
}
defer pool.Close()
jobStore := store.New(pool)
runner, err := workerpkg.New(jobStore, config, log.Default())
if err != nil {
log.Fatal(err)
}
tverClient, err := tver.NewClient("", nil)
if err != nil {
log.Fatal(err)
}
scheduler, err := workerpkg.NewSubscriptionScheduler(jobStore, tverClient,
workerpkg.SubscriptionSchedulerConfig{PreviewMaxSeconds: tver.PreviewMaxSeconds}, log.Default())
if err != nil {
log.Fatal(err)
}
schedulerDone := make(chan struct{})
go func() {
defer close(schedulerDone)
scheduler.Run(ctx)
}()
log.Printf("worker started (concurrency=%d)", config.Concurrency)
if err := runner.Run(ctx); err != nil {
log.Fatal(err)
}
<-schedulerDone
log.Print("worker stopped")
}
func loadConfig(getenv func(string) string) (workerpkg.Config, error) {
concurrency := 1
if value := strings.TrimSpace(getenv("WORKER_CONCURRENCY")); value != "" {
parsed, err := strconv.Atoi(value)
if err != nil || parsed < 1 || parsed > 2 {
return workerpkg.Config{}, errors.New("WORKER_CONCURRENCY は 1 または 2 を指定してください")
}
concurrency = parsed
}
return workerpkg.Config{VideoDir: getenv("VIDEO_DIR"), YTDLPPath: getenv("YT_DLP_PATH"),
FFprobePath: getenv("FFPROBE_PATH"), Concurrency: concurrency}, nil
}
func initializeDatabase(ctx context.Context, databaseURL string, now time.Time,
connect func(context.Context, string) (*pgxpool.Pool, error),
migrate func(context.Context, *pgxpool.Pool) error,
requeue func(context.Context, *pgxpool.Pool, time.Time) (int64, error),
) (*pgxpool.Pool, error) {
if strings.TrimSpace(databaseURL) == "" {
return nil, errors.New("DATABASE_URL が設定されていません")
}
pool, err := connect(ctx, databaseURL)
if err != nil {
return nil, errors.New("DATABASE_URL から接続プールを作成できません")
}
if err := migrate(ctx, pool); err != nil {
pool.Close()
return nil, fmt.Errorf("DB マイグレーションに失敗: %w", err)
}
if _, err := requeue(ctx, pool, now.Add(-staleAfter)); err != nil {
pool.Close()
return nil, fmt.Errorf("停止中ジョブの復帰に失敗: %w", err)
}
return pool, nil
}
+97
View File
@@ -0,0 +1,97 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestLoadConfigConcurrency(t *testing.T) {
values := map[string]string{"VIDEO_DIR": "/media", "YT_DLP_PATH": "/bin/downloader", "FFPROBE_PATH": "/bin/probe"}
getenv := func(key string) string { return values[key] }
config, err := loadConfig(getenv)
if err != nil || config.Concurrency != 1 || config.VideoDir != "/media" || config.YTDLPPath != "/bin/downloader" || config.FFprobePath != "/bin/probe" {
t.Fatalf("既定設定: %+v %v", config, err)
}
values["WORKER_CONCURRENCY"] = "2"
config, err = loadConfig(getenv)
if err != nil || config.Concurrency != 2 {
t.Fatalf("並行数2: %+v %v", config, err)
}
for _, invalid := range []string{"0", "3", "bad"} {
values["WORKER_CONCURRENCY"] = invalid
if _, err := loadConfig(getenv); err == nil {
t.Errorf("不正な並行数 %q を許した", invalid)
}
}
}
func TestInitializeDatabaseRequiresURLMigratesAndRequeues(t *testing.T) {
ctx := context.Background()
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
connectCalls, migrateCalls, requeueCalls := 0, 0, 0
connect := func(_ context.Context, url string) (*pgxpool.Pool, error) {
connectCalls++
if url != "postgres://test:secret@localhost/test" {
t.Errorf("URL=%q", url)
}
return pgxpool.New(ctx, url)
}
migrate := func(context.Context, *pgxpool.Pool) error { migrateCalls++; return nil }
requeue := func(_ context.Context, _ *pgxpool.Pool, cutoff time.Time) (int64, error) {
requeueCalls++
if want := now.Add(-staleAfter); !cutoff.Equal(want) {
t.Errorf("cutoff=%v want=%v", cutoff, want)
}
return 2, nil
}
if _, err := initializeDatabase(ctx, " ", now, connect, migrate, requeue); err == nil || !strings.Contains(err.Error(), "DATABASE_URL") {
t.Fatalf("未設定 URL: %v", err)
}
if connectCalls != 0 || migrateCalls != 0 || requeueCalls != 0 {
t.Fatalf("未設定時の呼び出し: %d %d %d", connectCalls, migrateCalls, requeueCalls)
}
pool, err := initializeDatabase(ctx, "postgres://test:secret@localhost/test", now, connect, migrate, requeue)
if err != nil || pool == nil {
t.Fatalf("初期化: %v", err)
}
pool.Close()
if connectCalls != 1 || migrateCalls != 1 || requeueCalls != 1 {
t.Fatalf("呼び出し回数: %d %d %d", connectCalls, migrateCalls, requeueCalls)
}
}
func TestInitializeDatabaseStopsOnMigrationOrRequeueFailure(t *testing.T) {
ctx := context.Background()
connect := func(context.Context, string) (*pgxpool.Pool, error) {
return pgxpool.New(ctx, "postgres://localhost/test")
}
requeueCalled := false
pool, err := initializeDatabase(ctx, "postgres://localhost/test", time.Now(), connect,
func(context.Context, *pgxpool.Pool) error { return errors.New("migration failed") },
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { requeueCalled = true; return 0, nil })
if pool != nil || err == nil || !strings.Contains(err.Error(), "migration failed") || requeueCalled {
t.Fatalf("migration failure: pool=%v err=%v requeue=%v", pool, err, requeueCalled)
}
pool, err = initializeDatabase(ctx, "postgres://localhost/test", time.Now(), connect,
func(context.Context, *pgxpool.Pool) error { return nil },
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { return 0, errors.New("requeue failed") })
if pool != nil || err == nil || !strings.Contains(err.Error(), "requeue failed") {
t.Fatalf("requeue failure: pool=%v err=%v", pool, err)
}
}
func TestInitializeDatabaseDoesNotLeakURL(t *testing.T) {
_, err := initializeDatabase(context.Background(), "postgres://user:secret@localhost/test", time.Now(),
func(context.Context, string) (*pgxpool.Pool, error) {
return nil, errors.New("postgres://user:secret@localhost/test refused")
}, func(context.Context, *pgxpool.Pool) error { return nil },
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { return 0, nil })
if err == nil || strings.Contains(err.Error(), "secret") {
t.Fatalf("URL 漏えい: %v", err)
}
}
+25
View File
@@ -0,0 +1,25 @@
services:
web:
build: .
env_file: .env
environment:
TZ: Asia/Tokyo
WEB_ADDR: ":8080"
VIDEO_DIR: "${VIDEO_DIR:-/videos}"
volumes:
- "${VIDEO_HOST_DIR:?VIDEO_HOST_DIR must be set}:${VIDEO_DIR:-/videos}"
ports:
- "192.168.10.155:8080:8080"
restart: unless-stopped
worker:
build:
context: .
dockerfile: Dockerfile.worker
env_file: .env
environment:
TZ: Asia/Tokyo
VIDEO_DIR: "${VIDEO_DIR:-/videos}"
WORKER_CONCURRENCY: "${WORKER_CONCURRENCY:-1}"
volumes:
- "${VIDEO_HOST_DIR:?VIDEO_HOST_DIR must be set}:${VIDEO_DIR:-/videos}"
restart: unless-stopped
+1
View File
@@ -0,0 +1 @@
POSTGRES_PASSWORD=
+66
View File
@@ -0,0 +1,66 @@
#!/bin/bash
# tver DBバックアップ(Synology ds923p / DSMタスクスケジューラで日次実行)
#
# 配置先: /volume1/docker/tver-db/backup-db.sh
# DSM設定: コントロールパネル → タスクスケジューラ → 作成
# → スケジュールされたタスク → ユーザー定義のスクリプト
# ユーザーはroot、実行周期は毎日とする。
# 実行コマンド: bash /volume1/docker/tver-db/backup-db.sh
#
# コンテナ内のpg_dumpを使い、サーバーとクライアントのバージョンを合わせる。
# PostgreSQLのローカル接続を使うため、DBパスワードはスクリプトへ書かない。
set -euo pipefail
BACKUP_DIR="${BACKUP_DIR:-/volume1/backups/tver}"
KEEP_DAYS="${KEEP_DAYS:-14}"
CONTAINER="${CONTAINER:-tver-postgres}"
DB_USER="${DB_USER:-tver}"
DB_NAME="${DB_NAME:-tver}"
DOCKER="${DOCKER:-$(command -v docker || printf '/usr/local/bin/docker')}"
STAMP="${STAMP:-$(date +%Y%m%d_%H%M%S)}"
OUT="${BACKUP_DIR}/tver_${STAMP}.sql.gz"
RAW_TMP="${BACKUP_DIR}/.tver_${STAMP}.$$.sql"
GZ_TMP="${OUT}.tmp.$$"
case "$KEEP_DAYS" in
''|*[!0-9]*)
echo "ERROR: KEEP_DAYS must be a non-negative integer: $KEEP_DAYS" >&2
exit 1
;;
esac
mkdir -p "$BACKUP_DIR"
umask 077
cleanup() {
rm -f "$RAW_TMP" "$GZ_TMP"
}
trap cleanup EXIT
# 先に非圧縮の一時ファイルへ出し、pg_dump成功と空でないことを確認する。
"$DOCKER" exec "$CONTAINER" pg_dump -U "$DB_USER" -d "$DB_NAME" > "$RAW_TMP"
if [[ ! -s "$RAW_TMP" ]]; then
echo "ERROR: dump is empty" >&2
exit 1
fi
gzip -9 -c "$RAW_TMP" > "$GZ_TMP"
if [[ ! -s "$GZ_TMP" ]]; then
echo "ERROR: compressed dump is empty" >&2
exit 1
fi
# 完成ファイルは上書きしない。同名が同時に作られても、ハードリンクの
# 作成に成功した一方だけを公開する。一時ファイルは同じディレクトリに
# あるため同一ファイルシステム内でリンクできる。
if ! ln "$GZ_TMP" "$OUT"; then
echo "ERROR: backup already exists or could not be published: $OUT" >&2
exit 1
fi
rm -f "$GZ_TMP" "$RAW_TMP"
# KEEP_DAYS日より古い完了済みバックアップだけを削除する。
find "$BACKUP_DIR" -name 'tver_*.sql.gz' -type f -mtime "+$KEEP_DAYS" -delete
SIZE_BYTES="$(wc -c < "$OUT" | tr -d '[:space:]')"
echo "OK: $OUT ($SIZE_BYTES bytes)"
+36
View File
@@ -0,0 +1,36 @@
# tver 用 PostgreSQL スタック(Synology ds923p / 192.168.10.201)
#
# 配置先: ds923p の Container Manager「プロジェクト」
# /volume1/docker/tver-db/ に本ファイルを compose.yaml として配置する。
# パスワードは同じディレクトリの .env(POSTGRES_PASSWORD)から注入し、
# このファイルには実値を書かない。
#
# 注意(初回起動チェックポイント):
# 起動前に /volume1/docker/tver-db/pgdata を作成する。
# 初回起動後は Container Manager のヘルス状態とログを確認し、
# "database system is ready to accept connections" が出てからアプリを起動する。
# bind mount の所有権・権限で初期化に失敗した場合は、原因を確認してから
# 名前付きボリュームへの切り替えを検討する。
services:
postgres:
# Debian ベースの PostgreSQL 17。initdb の UTF-8 ロケールを利用する。
image: postgres:17
container_name: tver-postgres
restart: always
environment:
POSTGRES_USER: tver
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: tver
# 初回の initdb でのみ有効。既存の pgdata には後から反映されない。
POSTGRES_INITDB_ARGS: "--encoding=UTF8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8"
ports:
# [要確認] Container Manager で 15434 が未使用であることを起動前に確認する。
# DSM ファイアウォールでは dockage(192.168.10.155)だけを許可する。
- "15434:5432"
volumes:
- /volume1/docker/tver-db/pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U tver -d tver"]
interval: 10s
retries: 5
+73
View File
@@ -0,0 +1,73 @@
#!/bin/bash
# tver DB復元確認(Synology ds923pで手動実行)
#
# 配置先: /volume1/docker/tver-db/restore-test.sh
# 実行コマンド: bash /volume1/docker/tver-db/restore-test.sh
# 最新のバックアップを同じPostgreSQLコンテナ内の一時DBへ復元し、主要4表の
# 行数を本番DBと比較する。本番DBにはSELECTだけを実行する。
set -euo pipefail
BACKUP_DIR="${BACKUP_DIR:-/volume1/backups/tver}"
LOG_FILE="${LOG_FILE:-${BACKUP_DIR}/restore-test.log}"
CONTAINER="${CONTAINER:-tver-postgres}"
DB_USER="${DB_USER:-tver}"
DB_NAME="${DB_NAME:-tver}"
RESTORE_DB="${RESTORE_DB:-tver_restore_test}"
DOCKER="${DOCKER:-$(command -v docker || printf '/usr/local/bin/docker')}"
TABLES=(download_jobs subscriptions known_episodes schema_migrations)
mkdir -p "$BACKUP_DIR"
touch "$LOG_FILE"
chmod 600 "$LOG_FILE"
exec > >(tee -a "$LOG_FILE") 2>&1
if [[ "$RESTORE_DB" == "$DB_NAME" ]]; then
echo "ERROR: RESTORE_DB must differ from DB_NAME" >&2
exit 1
fi
# 終了状態にかかわらず、一時DBの削除を最後に試みる。
cleanup() {
status=$?
trap - EXIT
set +e
"$DOCKER" exec "$CONTAINER" dropdb --force --if-exists -U "$DB_USER" "$RESTORE_DB"
cleanup_status=$?
if ((status != 0)); then
echo "復元確認に失敗"
exit "$status"
fi
exit "$cleanup_status"
}
trap cleanup EXIT
LATEST="$(find "$BACKUP_DIR" -name 'tver_*.sql.gz' -type f -print | sort | tail -n 1)"
if [[ -z "$LATEST" ]]; then
echo "ERROR: backup dump was not found in $BACKUP_DIR" >&2
exit 1
fi
echo "復元確認を開始: $LATEST"
"$DOCKER" exec "$CONTAINER" dropdb --force --if-exists -U "$DB_USER" "$RESTORE_DB"
"$DOCKER" exec "$CONTAINER" createdb -U "$DB_USER" "$RESTORE_DB"
gzip -dc "$LATEST" \
| "$DOCKER" exec -i "$CONTAINER" psql -q -o /dev/null -v ON_ERROR_STOP=1 -U "$DB_USER" -d "$RESTORE_DB"
for table in "${TABLES[@]}"; do
query="SELECT count(*) FROM public.${table};"
production_count="$("$DOCKER" exec "$CONTAINER" psql -A -t -q -U "$DB_USER" -d "$DB_NAME" -c "$query" | tr -d '[:space:]')"
restored_count="$("$DOCKER" exec "$CONTAINER" psql -A -t -q -U "$DB_USER" -d "$RESTORE_DB" -c "$query" | tr -d '[:space:]')"
if [[ ! "$production_count" =~ ^[0-9]+$ || ! "$restored_count" =~ ^[0-9]+$ ]]; then
echo "ERROR: $table の行数を取得できませんでした: production=$production_count restore=$restored_count" >&2
exit 1
fi
if [[ "$production_count" != "$restored_count" ]]; then
echo "ERROR: $table の行数が一致しません: production=$production_count restore=$restored_count" >&2
exit 1
fi
echo "OK: $table production=$production_count restore=$restored_count"
done
echo "復元確認に成功: $LATEST"
+401
View File
@@ -0,0 +1,401 @@
package tverdb_test
import (
"bytes"
"compress/gzip"
"io"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestBackupDB(t *testing.T) {
t.Run("success and retention boundary", func(t *testing.T) {
env := newScriptEnv(t, "success")
old := filepath.Join(env.backupDir, "tver_20260901_000000.sql.gz")
boundary := filepath.Join(env.backupDir, "tver_20260920_000000.sql.gz")
unrelated := []string{
filepath.Join(env.backupDir, "restore-test.log"),
filepath.Join(env.backupDir, "other_20260901_000000.sql.gz"),
filepath.Join(env.backupDir, "tver_20260901_000000.sql"),
}
writeFile(t, old, "old")
writeFile(t, boundary, "boundary")
for _, path := range unrelated {
writeFile(t, path, "keep")
}
now := time.Now()
oldTime := now.Add(-(15*24*time.Hour + time.Hour))
boundaryTime := now.Add(-(15*24*time.Hour - time.Hour))
if err := os.Chtimes(old, oldTime, oldTime); err != nil {
t.Fatal(err)
}
if err := os.Chtimes(boundary, boundaryTime, boundaryTime); err != nil {
t.Fatal(err)
}
for _, path := range unrelated {
if err := os.Chtimes(path, oldTime, oldTime); err != nil {
t.Fatal(err)
}
}
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203", "KEEP_DAYS=14", "CONTAINER=custom-postgres", "DB_USER=custom-user", "DB_NAME=custom-db")
if err != nil {
t.Fatalf("backup failed: %v\n%s", err, output)
}
created := filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")
if got := readGzip(t, created); !strings.Contains(got, "CREATE TABLE example") {
t.Fatalf("dump=%q", got)
}
if _, err := os.Stat(old); !os.IsNotExist(err) {
t.Fatalf("old backup remains: %v", err)
}
if _, err := os.Stat(boundary); err != nil {
t.Fatalf("14-day backup was deleted: %v", err)
}
for _, path := range unrelated {
if got := readFile(t, path); got != "keep" {
t.Fatalf("unrelated file was changed: %s content=%q", path, got)
}
}
if !strings.Contains(output, "OK: "+created) || !strings.Contains(output, "bytes") {
t.Fatalf("success output=%q", output)
}
if log := readFile(t, env.dockerLog); !strings.Contains(log, "exec custom-postgres pg_dump -U custom-user -d custom-db") {
t.Fatalf("docker log=%q", log)
}
})
t.Run("creates missing backup directory", func(t *testing.T) {
env := newScriptEnv(t, "success")
if err := os.RemoveAll(env.backupDir); err != nil {
t.Fatal(err)
}
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203")
if err != nil {
t.Fatalf("backup failed: %v\n%s", err, output)
}
if got := readGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")); !strings.Contains(got, "CREATE TABLE example") {
t.Fatalf("dump=%q", got)
}
})
t.Run("does not replace an existing completed backup", func(t *testing.T) {
env := newScriptEnv(t, "success")
completed := filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")
writeGzip(t, completed, "existing")
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203")
if err == nil {
t.Fatalf("backup replaced an existing file: %s", output)
}
if got := readGzip(t, completed); got != "existing" {
t.Fatalf("existing backup was changed: %q", got)
}
assertNoBackupTemps(t, env.backupDir)
})
t.Run("same stamp concurrently allows exactly one backup", func(t *testing.T) {
env := newScriptEnv(t, "concurrent")
barrier := filepath.Join(env.dir, "barrier")
extra := []string{"STAMP=20261004_010203", "FAKE_DOCKER_BARRIER=" + barrier}
commands := make([]*exec.Cmd, 2)
outputs := make([]bytes.Buffer, 2)
for i := range commands {
commands[i] = exec.Command("bash", "backup-db.sh")
commands[i].Env = append(append(append([]string{}, os.Environ()...), env.baseEnv...), extra...)
commands[i].Stdout = &outputs[i]
commands[i].Stderr = &outputs[i]
if err := commands[i].Start(); err != nil {
t.Fatal(err)
}
}
succeeded := 0
for i, command := range commands {
if err := command.Wait(); err == nil {
succeeded++
} else if _, ok := err.(*exec.ExitError); !ok {
t.Fatalf("backup %d wait: %v output=%q", i, err, outputs[i].String())
}
}
if succeeded != 1 {
t.Fatalf("concurrent successes=%d want=1 outputs=%q / %q", succeeded, outputs[0].String(), outputs[1].String())
}
if got := readGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")); !strings.Contains(got, "CREATE TABLE example") {
t.Fatalf("completed dump=%q", got)
}
assertNoBackupTemps(t, env.backupDir)
})
for _, mode := range []string{"dump-fail", "dump-partial-fail", "empty"} {
t.Run(mode, func(t *testing.T) {
env := newScriptEnv(t, mode)
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203")
if err == nil {
t.Fatalf("backup succeeded unexpectedly: %s", output)
}
matches, globErr := filepath.Glob(filepath.Join(env.backupDir, "tver_*.sql.gz"))
if globErr != nil || len(matches) != 0 {
t.Fatalf("partial backups=%v err=%v", matches, globErr)
}
assertNoBackupTemps(t, env.backupDir)
if mode == "empty" && !strings.Contains(output, "empty") {
t.Fatalf("empty dump error missing: %q", output)
}
})
}
for _, keepDays := range []string{"-1", "invalid"} {
t.Run("invalid retention "+keepDays, func(t *testing.T) {
env := newScriptEnv(t, "success")
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203", "KEEP_DAYS="+keepDays)
if err == nil || !strings.Contains(output, "KEEP_DAYS") {
t.Fatalf("invalid KEEP_DAYS: err=%v output=%q", err, output)
}
if _, err := os.Stat(env.dockerLog); !os.IsNotExist(err) {
t.Fatalf("pg_dump was invoked: %v", err)
}
})
}
}
func TestRestoreTest(t *testing.T) {
t.Run("restores latest dump and compares rows", func(t *testing.T) {
env := newScriptEnv(t, "success")
writeGzip(t, filepath.Join(env.backupDir, "tver_20261003_010203.sql.gz"), "older")
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "newest")
output, err := env.run(t, "restore-test.sh")
if err != nil {
t.Fatalf("restore test failed: %v\n%s", err, output)
}
for _, table := range []string{"download_jobs", "subscriptions", "known_episodes", "schema_migrations"} {
if !strings.Contains(output, "OK: "+table) {
t.Errorf("missing result for %s: %s", table, output)
}
}
if got := readFile(t, env.restoreInput); got != "newest" {
t.Fatalf("restored dump=%q", got)
}
log := readFile(t, env.dockerLog)
if !strings.Contains(log, "psql -q -o /dev/null -v ON_ERROR_STOP=1") {
t.Fatalf("psql output suppression is missing: %q", log)
}
cleanupCommand := "dropdb --force --if-exists -U tver tver_restore_test"
if strings.Count(log, cleanupCommand) != 2 || !strings.HasSuffix(strings.TrimSpace(log), cleanupCommand) {
t.Fatalf("temporary DB cleanup log=%q", log)
}
for _, line := range strings.Split(strings.TrimSpace(log), "\n") {
if (strings.Contains(line, " createdb ") || strings.Contains(line, " dropdb ")) && !strings.HasSuffix(line, "tver_restore_test") {
t.Fatalf("production DB mutation was attempted: %q", line)
}
if strings.Contains(line, " psql ") && strings.Contains(line, " -d tver ") && !strings.Contains(line, "SELECT count(*)") {
t.Fatalf("production DB received a non-count command: %q", line)
}
if strings.Contains(line, " ON_ERROR_STOP=1 ") && !strings.Contains(line, " -d tver_restore_test") {
t.Fatalf("dump was restored into a non-temporary DB: %q", line)
}
}
if restoreLog := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(restoreLog, "復元確認に成功") || strings.Contains(restoreLog, "SET\n") || strings.Contains(restoreLog, "CREATE TABLE\n") {
t.Fatalf("restore log=%q", restoreLog)
}
})
for _, mode := range []string{"restore-fail", "count-mismatch"} {
t.Run(mode+" cleans temporary database", func(t *testing.T) {
env := newScriptEnv(t, mode)
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "dump")
output, err := env.run(t, "restore-test.sh")
if err == nil {
t.Fatalf("restore test succeeded unexpectedly: %s", output)
}
log := strings.TrimSpace(readFile(t, env.dockerLog))
if !strings.HasSuffix(log, "dropdb --force --if-exists -U tver tver_restore_test") {
t.Fatalf("cleanup was not last command: %q", log)
}
if mode == "count-mismatch" && !strings.Contains(output, "ERROR: known_episodes") {
t.Fatalf("mismatch not reported: %q", output)
}
if mode == "restore-fail" {
if !strings.Contains(output, "restore failed") {
t.Fatalf("restore error missing from output: %q", output)
}
if restoreLog := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(restoreLog, "restore failed") {
t.Fatalf("restore error missing from log: %q", restoreLog)
}
}
})
}
t.Run("rejects production database as restore target", func(t *testing.T) {
env := newScriptEnv(t, "success")
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "dump")
output, err := env.run(t, "restore-test.sh", "RESTORE_DB=tver")
if err == nil || !strings.Contains(output, "RESTORE_DB must differ from DB_NAME") {
t.Fatalf("production restore target was accepted: err=%v output=%q", err, output)
}
if _, err := os.Stat(env.dockerLog); !os.IsNotExist(err) {
t.Fatalf("docker was invoked: %v", err)
}
if log := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(log, "RESTORE_DB must differ from DB_NAME") {
t.Fatalf("rejection was not logged: %q", log)
}
})
t.Run("missing dump fails and cleans temporary database", func(t *testing.T) {
env := newScriptEnv(t, "success")
output, err := env.run(t, "restore-test.sh")
if err == nil || !strings.Contains(output, "backup dump was not found") {
t.Fatalf("missing dump: err=%v output=%q", err, output)
}
log := strings.TrimSpace(readFile(t, env.dockerLog))
if log != "exec tver-postgres dropdb --force --if-exists -U tver tver_restore_test" {
t.Fatalf("unexpected cleanup: %q", log)
}
if restoreLog := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(restoreLog, "backup dump was not found") {
t.Fatalf("failure was not logged: %q", restoreLog)
}
})
t.Run("honors database and container overrides", func(t *testing.T) {
env := newScriptEnv(t, "success")
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "dump")
output, err := env.run(t, "restore-test.sh", "CONTAINER=custom-postgres", "DB_USER=custom-user", "DB_NAME=production", "RESTORE_DB=temporary")
if err != nil {
t.Fatalf("restore failed: %v\n%s", err, output)
}
for _, line := range strings.Split(strings.TrimSpace(readFile(t, env.dockerLog)), "\n") {
if !strings.Contains(line, " custom-postgres ") || strings.Contains(line, " tver-postgres ") || !strings.Contains(line, " -U custom-user ") {
t.Fatalf("override missing: %q", line)
}
if (strings.Contains(line, " createdb ") || strings.Contains(line, " dropdb ")) && !strings.HasSuffix(line, "temporary") {
t.Fatalf("wrong temporary DB: %q", line)
}
}
})
t.Run("invalid gzip fails and cleans temporary database", func(t *testing.T) {
env := newScriptEnv(t, "success")
writeFile(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "not gzip")
output, err := env.run(t, "restore-test.sh")
if err == nil {
t.Fatalf("invalid gzip succeeded: %s", output)
}
log := strings.TrimSpace(readFile(t, env.dockerLog))
if !strings.HasSuffix(log, "dropdb --force --if-exists -U tver tver_restore_test") {
t.Fatalf("cleanup was not last command: %q", log)
}
})
}
type scriptEnv struct {
dir, backupDir, dockerLog, restoreInput string
baseEnv []string
}
func newScriptEnv(t *testing.T, mode string) scriptEnv {
t.Helper()
dir := t.TempDir()
backupDir := filepath.Join(dir, "backups")
if err := os.MkdirAll(backupDir, 0o755); err != nil {
t.Fatal(err)
}
scriptDir, err := filepath.Abs(".")
if err != nil {
t.Fatal(err)
}
dockerDir := filepath.Join(scriptDir, "testdata")
return scriptEnv{
dir: dir,
backupDir: backupDir,
dockerLog: filepath.Join(dir, "docker.log"),
restoreInput: filepath.Join(dir, "restore.sql"),
baseEnv: []string{
"PATH=" + dockerDir + string(os.PathListSeparator) + os.Getenv("PATH"),
"BACKUP_DIR=" + backupDir,
"LOG_FILE=" + filepath.Join(backupDir, "restore-test.log"),
"FAKE_DOCKER_LOG=" + filepath.Join(dir, "docker.log"),
"FAKE_RESTORE_INPUT=" + filepath.Join(dir, "restore.sql"),
"FAKE_DOCKER_MODE=" + mode,
},
}
}
func (e scriptEnv) run(t *testing.T, script string, extraEnv ...string) (string, error) {
t.Helper()
cmd := exec.Command("bash", script)
cmd.Env = append(append(os.Environ(), e.baseEnv...), extraEnv...)
var output bytes.Buffer
cmd.Stdout = &output
cmd.Stderr = &output
err := cmd.Run()
return output.String(), err
}
func writeFile(t *testing.T, path, content string) {
t.Helper()
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
t.Fatal(err)
}
}
func readFile(t *testing.T, path string) string {
t.Helper()
data, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
return string(data)
}
func writeGzip(t *testing.T, path, content string) {
t.Helper()
file, err := os.Create(path)
if err != nil {
t.Fatal(err)
}
writer := gzip.NewWriter(file)
if _, err := io.WriteString(writer, content); err != nil {
t.Fatal(err)
}
if err := writer.Close(); err != nil {
t.Fatal(err)
}
if err := file.Close(); err != nil {
t.Fatal(err)
}
}
func readGzip(t *testing.T, path string) string {
t.Helper()
file, err := os.Open(path)
if err != nil {
t.Fatal(err)
}
defer file.Close()
reader, err := gzip.NewReader(file)
if err != nil {
t.Fatal(err)
}
defer reader.Close()
data, err := io.ReadAll(reader)
if err != nil {
t.Fatal(err)
}
return string(data)
}
func assertNoBackupTemps(t *testing.T, backupDir string) {
t.Helper()
entries, err := os.ReadDir(backupDir)
if err != nil {
t.Fatal(err)
}
for _, entry := range entries {
name := entry.Name()
if strings.HasPrefix(name, ".tver_") || strings.Contains(name, ".sql.gz.tmp.") {
t.Errorf("temporary backup remains: %s", name)
}
}
}
+97
View File
@@ -0,0 +1,97 @@
#!/bin/bash
set -euo pipefail
printf '%s\n' "$*" >> "${FAKE_DOCKER_LOG:?}"
command_line=" $* "
if [[ "$command_line" == *" pg_dump "* ]]; then
case "${FAKE_DOCKER_MODE:-success}" in
dump-fail)
printf 'pg_dump failed\n' >&2
exit 42
;;
dump-partial-fail)
printf '%s\n' '-- incomplete dump --'
printf 'pg_dump failed after output\n' >&2
exit 42
;;
empty)
exit 0
;;
concurrent)
barrier="${FAKE_DOCKER_BARRIER:?}"
mkdir -p "$barrier"
touch "$barrier/$$"
for _ in {1..500}; do
count="$(find "$barrier" -type f | wc -l | tr -d '[:space:]')"
if ((count >= 2)); then
break
fi
sleep 0.01
done
if ((count < 2)); then
printf 'concurrent backup barrier timed out\n' >&2
exit 46
fi
;;
esac
printf '%s\n' '-- fake PostgreSQL dump --' 'CREATE TABLE example(id integer);'
exit 0
fi
if [[ "$command_line" == *" createdb "* ]]; then
exit 0
fi
if [[ "$command_line" == *" dropdb "* ]]; then
exit 0
fi
if [[ "$command_line" == *" psql "* && "$command_line" == *" ON_ERROR_STOP=1 "* ]]; then
cat > "${FAKE_RESTORE_INPUT:?}"
if [[ "${FAKE_DOCKER_MODE:-success}" == "restore-fail" ]]; then
printf 'restore failed\n' >&2
exit 43
fi
if [[ "$command_line" != *" -o /dev/null "* ]]; then
printf '%s\n' 'SET' 'CREATE TABLE'
fi
exit 0
fi
if [[ "$command_line" == *" psql "* && "$command_line" == *" SELECT count(*) FROM public."* ]]; then
database=""
query=""
while (($#)); do
case "$1" in
-d)
database="$2"
shift 2
;;
-c)
query="$2"
shift 2
;;
*)
shift
;;
esac
done
table="${query#*public.}"
table="${table%%;*}"
case "$table" in
download_jobs) count=3 ;;
subscriptions) count=2 ;;
known_episodes) count=5 ;;
schema_migrations) count=3 ;;
*) printf 'unexpected table: %s\n' "$table" >&2; exit 44 ;;
esac
if [[ "${FAKE_DOCKER_MODE:-success}" == "count-mismatch" && "$database" == "${RESTORE_DB:-tver_restore_test}" && "$table" == "known_episodes" ]]; then
count=4
fi
printf '%s\n' "$count"
exit 0
fi
printf 'unexpected docker invocation: %s\n' "$*" >&2
exit 45
+339
View File
@@ -0,0 +1,339 @@
---
title: tver 本番配置手順
tags:
- homelab
- tver
- deployment
---
# tver 本番配置手順
> [!info] 対象
> Synology NAS `ds923p`(`192.168.10.201`)に PostgreSQL 17 と動画共有を用意し、dockage(`192.168.10.155`)の `/opt/stacks/tver` で web・worker を起動する。
> [!danger] 秘匿値を記録しない
> PostgreSQL のパスワードや完成した `DATABASE_URL` は、この文書や Git 管理下のファイルに書かない。NAS と dockage の `.env` にだけ保存し、権限を `600` にする。
## 事前に確認する値
| 項目 | 値 |
|---|---|
| NAS | `ds923p` / `192.168.10.201` |
| dockage | `192.168.10.155` |
| NAS のNFS共有 | `/volume1/video` |
| dockage のNFSマウント | `/mnt/nas-tver` |
| tverの動画保存先 | NAS: `/volume1/video/tver`、dockage: `/mnt/nas-tver/tver` |
| PostgreSQL 公開ポート | `15434` |
| PostgreSQL ユーザー / DB | `tver` / `tver` |
| アプリ配置先 | `/opt/stacks/tver` |
| Gitea配置用ユーザー | `tver-deploy`(リポジトリ読み取りのみ) |
## 1. NASを準備する
### 1.1 NFSサービスを有効にする
1. DSMの「コントロール パネル」→「ファイル サービス」→「NFS」でNFSサービスを有効にする。
2. 共有フォルダ `video` が `/volume1/video` にあることを確認する。
3. `video` の「編集」→「NFS 権限」で次を追加する。
| 設定 | 値 |
|---|---|
| ホスト名またはIP | `192.168.10.155` |
| 権限 | 読み書き |
| Squash | すべてのユーザーをadminに割り当てる |
| セキュリティ | `sys` |
| 非同期を有効にする | オン |
> [!success] adminへの割り当てを確認済み
> DSMの`admin`アカウントが無効な状態でも、上記設定でUID `65532`からフォルダ作成・書き込み・読み込み・rename・削除に成功した。NAS上の所有者はUID `1024`・GID `100`だった。[実機] 2026-10-04 確認
### 1.2 PostgreSQLのファイルを用意する
1. NAS上に次のディレクトリを事前作成する。
```text
/volume1/docker/tver-db/
/volume1/docker/tver-db/pgdata/
```
2. [[../deploy/nas/tver-db/compose.yaml|NAS用compose.yaml]]、`.env.example`、`backup-db.sh`、`restore-test.sh`を`/volume1/docker/tver-db/`に配置する。ディレクトリはroot以外がファイルを削除・置換できないようにし、スクリプトはrootだけが実行できるようにする。
```bash
chmod 755 /volume1/docker/tver-db
chmod 700 /volume1/docker/tver-db/backup-db.sh
chmod 700 /volume1/docker/tver-db/restore-test.sh
ls -ld /volume1/docker/tver-db
ls -l /volume1/docker/tver-db/backup-db.sh /volume1/docker/tver-db/restore-test.sh
```
DSMのACLが付いている場合、`chmod`後の`ls`でパーミッション表示の末尾から`+`が外れ、ディレクトリが`drwxr-xr-x`、スクリプトが`-rwx------`になったことを確認する。`/volume1/docker`では全員書き込み可のACLが確認されたため、スクリプトだけでなく配置ディレクトリにも`chmod 755`を適用する。
3. `.env.example`を`.env`へコピーし、`POSTGRES_PASSWORD`に十分長いランダムな値を設定する。値には、URLへの埋め込みで追加のエスケープが不要な16進文字列を使える。
```bash
cd /volume1/docker/tver-db
umask 077
cp .env.example .env
# openssl rand -hex 64 で値を生成し、.env の POSTGRES_PASSWORD に設定する。
chmod 600 .env
```
> [!important] DBとアプリで同じパスワードを使う
> ここで設定した値を、後述するdockage側の`DATABASE_URL`にも使う。値そのものは文書やGitへ残さない。
### 1.3 Container ManagerでPostgreSQLを起動する
1. Container Managerの「プロジェクト」で、`/volume1/docker/tver-db/compose.yaml`を使う`tver-db`プロジェクトを作成する。
2. 起動前に、他のプロジェクトとポート一覧を確認し、`15434`が空いていることを確認する。
3. プロジェクトをビルド・起動する。
4. `tver-postgres`が`healthy`になることを確認する。
5. コンテナログに`database system is ready to accept connections`があることを確認する。
初期化に失敗した場合は、先に`pgdata`の所有権・権限とログを確認する。空のDBを作り直す場合を除き、既存の`pgdata`を削除しない。
### 1.4 DSMファイアウォールを設定する
TCP `15434`について、送信元`192.168.10.155`だけを許可するルールを追加する。他の送信元を許可する広いルールより前に配置し、LAN内の別ホストからDBへ接続できない構成にする。
DSMファイアウォールを設定した状態で、dockageからNASのTCP `15434`へ到達できることを確認済みである。[実機] 2026-10-04 確認
## 2. dockageを準備する
### 2.1 NFSクライアントとマウント先を用意する
```bash
sudo apt update
sudo apt install -y nfs-common
sudo mkdir -p /mnt/nas-tver
```
`/etc/fstab`へ次の1行を追加する。`_netdev`でネットワーク後に扱い、`nofail`でNAS停止時もdockageの起動を妨げない。
```fstab
192.168.10.201:/volume1/video /mnt/nas-tver nfs defaults,_netdev,nofail 0 0
```
設定を反映し、想定したNFS共有がマウントされたことを確認する。
```bash
sudo mount -a
findmnt --target /mnt/nas-tver
```
実機ではNFSv3でマウントできた。NFS権限を追加した直後の1回目は`access denied`となり、直後の2回目で成功した。NFS権限の反映待ちと考えられるため、設定内容が正しい場合は少し待って`sudo mount -a`を再実行する。[実機] 2026-10-04 確認
### 2.2 workerのUIDで書き込みを確認する
既存の`anime`・`drama`・`misc`・`personal`・`variety`とアプリのファイルを分け、admin相当の書き込み範囲を限定するため、tver専用サブフォルダを使う。Dockerグループの`kazu634`で、workerコンテナと同じUID・GID `65532:65532`の確認用コンテナを実行し、フォルダ作成、書き込み、読み取り、rename、削除を確認する。`sudo`は不要である。
```bash
docker run --rm --user 65532:65532 -v /mnt/nas-tver/tver:/videos busybox:stable sh -c 'mkdir /videos/.tver-write-test && echo ok > /videos/.tver-write-test/f && cat /videos/.tver-write-test/f && mv /videos/.tver-write-test/f /videos/.tver-write-test/g && ls -ln /videos/.tver-write-test && rm -r /videos/.tver-write-test && echo cleanup-ok'
```
期待する結果は、内容の`ok`、`ls -ln`で表示される`g`の所有者`1024 100`、最後の`cleanup-ok`である。確認後は確認用イメージを削除する。
```bash
docker rmi busybox:stable
```
`/mnt/nas-tver/tver`は作成済みで、上記のDockerコンテナによる操作は実機確認済みである。[実機] 2026-10-04 確認。別環境へ作り直す場合も、書き込み確認が成功するまでworkerを起動しない。
### 2.3 Giteaに配置用ユーザーを用意する
Giteaは`REQUIRE_SIGNIN_VIEW = true`かつ`DISABLE_SSH = true`で動作している。匿名HTTPSの`info/refs`は`401`となり、SSHも使えないため、配置専用の読み取り資格情報を使う。[実機] 2026-10-04 確認
1. Giteaの管理画面で、配置専用ユーザー`tver-deploy`を作成する。
2. `kazu634/tver`リポジトリの「設定」→「共同作業者」で`tver-deploy`を追加し、権限を「読み取り」にする。
3. `tver-deploy`でGiteaへログインし、ユーザー設定の「アプリケーション」からrepository読み取り権限だけを持つトークンを作成する。
4. トークンは作成時に安全な場所へ控え、この文書、シェル履歴、Git管理下のファイルへ実値を書かない。
### 2.4 dockageにclone用トークンを保存する
Dockerホストの管理ユーザー`kazu634`で次を実行する。トークンは非表示で読み取り、権限`600`の専用credentialファイルへ保存する。credential helperはGiteaのHTTPS URLにだけ設定する。
```bash
install -d -m 700 ~/.config/git
read -rs -p 'Gitea token: ' TOKEN; echo
umask 077
printf 'https://tver-deploy:%s@gitea.kazu634.com\n' "$TOKEN" > ~/.config/git/credentials-tver
unset TOKEN
git config --global credential.https://gitea.kazu634.com.helper 'store --file ~/.config/git/credentials-tver'
git clone https://tver-deploy@gitea.kazu634.com/kazu634/tver.git /opt/stacks/tver
cd /opt/stacks/tver
```
この手順で認証付きcloneに成功した。[実機] 2026-10-04 確認。トークンを失効・更新した場合は、同じ手順で`credentials-tver`を作り直す。
### 2.5 アプリの環境変数を用意する
`.env.example`をGit管理外の`.env`へコピーし、`DATABASE_URL`へパスワード用の伏字を入れる。ファイルを`600`にしてから、パスワードを非表示で読み取り、伏字だけを置換する。
```bash
cd /opt/stacks/tver
umask 077
cp .env.example .env
sed -i 's|^DATABASE_URL=.*|DATABASE_URL=postgres://tver:__SET_PASSWORD__@192.168.10.201:15434/tver?sslmode=disable|' .env
chmod 600 .env
read -rs -p 'PostgreSQL password: ' DB_PASSWORD; echo
sed -i "s|__SET_PASSWORD__|${DB_PASSWORD}|" .env
unset DB_PASSWORD
if grep -q '__SET_PASSWORD__' .env; then echo 'DATABASE_URLのパスワードが未設定です' >&2; exit 1; fi
```
完成した`.env`は次の項目を持つ。`DATABASE_URL`の実値は表示したり、文書へ転記したりしない。
```dotenv
DATABASE_URL=<設定済み・表示しない>
VIDEO_HOST_DIR=/mnt/nas-tver/tver
VIDEO_DIR=/videos
WORKER_CONCURRENCY=1
```
> [!warning] パスワードの文字種
> この手順で使うPostgreSQLパスワードには、URLや`sed`の区切りに影響する`@`・`:`・`/`・`?`・`#`・`%`・`|`を含めない。NAS準備時に16進文字列で生成すれば、この制約を満たせる。
### 2.6 起動する
`/opt/stacks/tver`でwebとworkerをビルドして起動する。
```bash
cd /opt/stacks/tver
docker compose up -d --build
```
この方法で起動すると、Dockgeの画面にも`tver`スタックとして表示された。[実機] 2026-10-04 確認。Dockge上でweb・workerがともにrunningになり、再起動を繰り返していないことを確認する。
コマンドでも状態とログを確認できる。
```bash
cd /opt/stacks/tver
docker compose ps
docker compose logs --tail=100 web worker
```
## 3. 動作を確認する
1. ヘルスチェックが`200`と`ok`を返すことを確認する。
```bash
curl --fail --show-error http://192.168.10.155:8080/healthz
```
2. ブラウザで`http://192.168.10.155:8080/`を開き、新着一覧が表示されることを確認する。
3. 番組またはエピソードを1件登録し、`/queue`で待ち行列、進捗、完了を確認する。
4. `/library`から動画を再生し、シークできることを確認する。
5. NASの`/volume1/video/tver`に規則どおりのMP4が作成されたことを確認する。
6. `tver-postgres`のログとweb・workerのログに、接続失敗やマイグレーション失敗がないことを確認する。
> [!success] 本番の動作確認結果
> `/healthz`はHTTP `200`、新着バラエティは98件を表示した。4分の本編`ep5hcgopp2`を登録すると20秒以内に完了し、NASの`video/tver/<番組名>/`へ92,340,468バイトの1080p H.264/AAC MP4が保存された。所有者はUID `1024`・GID `100`、`.incoming`は空だった。`/library`の表示・再生とHTTP Range `206`も確認した。[実機] 2026-10-04 確認
> [!note] Synologyの`@eaDir`
> NASが番組フォルダ内に所有者UID `0`の`@eaDir`を作る。アプリのライブラリ一覧はファイルシステムの走査ではなくDBから作るため、表示や再生には影響しない。[実機] 2026-10-04 確認
## 4. 更新する
```bash
cd /opt/stacks/tver
git status --short
git pull --ff-only
```
`git status --short`に追跡対象ファイルの変更がある場合は、内容を確認してから更新する。pull後、Dockgeで`tver`スタックを再ビルドして再配置し、[[#3. 動作を確認する]]を繰り返す。
yt-dlpを更新するときは`Dockerfile.worker`の`YT_DLP_VERSION`の既定値を変更してリポジトリへ反映し、workerイメージをキャッシュなしで再ビルドする。更新後はworkerログと少数の実動画で抽出・保存を確認する。
## 5. PostgreSQLをバックアップする
バックアップはNAS上のDSMタスクスケジューラから毎日実行する。`tver-postgres`コンテナ内の`pg_dump`を使って`/volume1/backups/tver/tver_<YYYYmmdd_HHMMSS>.sql.gz`へ保存し、既定では14日より古い世代を削除する。
> [!warning] バックアップ先も同じNASにある
> このバックアップはDBの誤操作や破損からの復元用であり、NAS本体の故障には備えられない。`/volume1/backups/tver/`はSynology Hyper Backupの対象にも含める。
### 5.1 DSMの日次タスクを登録する
DSMの「コントロール パネル」→「タスク スケジューラ」で「スケジュールされたタスク」→「ユーザー定義のスクリプト」を作成する。
| 設定 | 値 |
|---|---|
| ユーザー | `root` |
| 実行周期 | 毎日 |
| ユーザー定義のスクリプト | `bash /volume1/docker/tver-db/backup-db.sh` |
`DOCKER`、`CONTAINER`、`BACKUP_DIR`、`KEEP_DAYS`、`DB_USER`、`DB_NAME`は環境変数で上書きできる。通常運用では既定値を使う。
> [!success] NASで手動実行を確認済み
> DSMタスクスケジューラへrootの日次タスクを登録し、手動実行で2つのgzipダンプを作成した。復元確認も成功している。翌日以降の日次自動実行は未確認である。[実機] 2026-10-04 確認
> [!todo] 日次自動実行を確認する
> 翌日以降にDSMタスクの実行履歴と新しいダンプを確認する。[要確認]
### 5.2 手動でバックアップを確認する
NASへSSH接続し、初回は手動で実行する。
```bash
sudo bash /volume1/docker/tver-db/backup-db.sh
LATEST=$(sudo find /volume1/backups/tver -name 'tver_*.sql.gz' -type f -print | sort | tail -n 1)
test -n "$LATEST"
sudo test -s "$LATEST"
sudo gzip -t "$LATEST"
sudo ls -lh "$LATEST"
```
スクリプトが`OK:`、保存先、バイト数を表示し、`gzip -t`が成功することを確認する。`pg_dump`の失敗や空のダンプでは非0で終了し、完成ファイルを残さない。同じ秒の完成ファイルが既にある場合も非0で終了し、既存ファイルを上書きしない。同時実行時は同一ディレクトリ内のハードリンク作成に成功した一方だけが完成ファイルを公開する。
> [!success] NASでのバックアップ結果
> DSMタスクスケジューラから実行し、`tver_20261004_182838.sql.gz`(2,286バイト)と`tver_20261004_183256.sql.gz`(2,289バイト)を`/volume1/backups/tver/`へ作成した。ダンプはroot所有・権限600で、バックアップディレクトリはroot所有・`drwxr-xr-x`だった。[実機] 2026-10-04 確認
### 5.3 別DBへの復元を確認する
最新ダンプを本番DBへ戻す前に、同じコンテナ内の一時DB`tver_restore_test`へ復元する。
```bash
sudo bash /volume1/docker/tver-db/restore-test.sh
sudo tail -n 100 /volume1/backups/tver/restore-test.log
```
`restore-test.sh`はrootで実行する。バックアップディレクトリとログがroot所有のため、一般ユーザーでは`restore-test.log`への書き込みに失敗する。`download_jobs`、`subscriptions`、`known_episodes`、`schema_migrations`の行数が本番DBと一致し、最後に「復元確認に成功」と表示されることを確認する。復元SQLの通常出力は抑制され、ログには開始、各表の結果、成功または失敗が残る。SQLエラーは標準エラーとしてログに残る。一時DBは成功・失敗のどちらでもEXIT trapで削除される。本番DBに対しては行数取得の`SELECT`だけを実行する。
> [!success] NASでの復元確認結果
> 一般ユーザー`kazu634`での実行は`restore-test.log`へ書き込めず、`touch: Permission denied`で終了した。`sudo`で実行すると、`download_jobs` 1件、`subscriptions` 1件、`known_episodes` 1件、`schema_migrations` 3件が本番DBと一時DBで一致し、「復元確認に成功」と表示された。同じ結果がログに残り、一時DBも削除された。[実機] 2026-10-04 確認
> [!note] 配置ディレクトリの権限変更後の状態
> `/volume1/docker/tver-db`を`chmod 755`にした後も、本番アプリの`/queue`と`/subscriptions`はHTTP `200`を返し、DBを読み取れた。PostgreSQLコンテナを再起動した後の動作は未確認である。[要確認]
### 5.4 障害時に本番DBへ復元する
> [!danger] 本番DBを置き換える操作
> 先に[[#5.3 別DBへの復元を確認する]]を成功させ、復元対象のダンプを明示してから実行する。`tver`以外のDB名へ読み替えない。
1. dockageの`/opt/stacks/tver`でwebとworkerを停止し、DBへの書き込みを止める。
```bash
cd /opt/stacks/tver
docker compose stop web worker
```
2. NASで復元対象を指定し、存在・サイズ・gzip形式を確認する。
```bash
DUMP=/volume1/backups/tver/tver_<YYYYmmdd_HHMMSS>.sql.gz
test -s "$DUMP"
gzip -t "$DUMP"
```
3. NASで本番DBを作り直し、ダンプを復元する。
```bash
docker exec tver-postgres dropdb --if-exists -U tver tver
docker exec tver-postgres createdb -U tver tver
gzip -dc "$DUMP" | docker exec -i tver-postgres psql -v ON_ERROR_STOP=1 -U tver -d tver
```
4. dockageでwebとworkerを再開し、[[#3. 動作を確認する]]を実施する。
```bash
cd /opt/stacks/tver
docker compose start web worker
```
+18
View File
@@ -0,0 +1,18 @@
module gitea.kazu634.com/kazu634/tver
go 1.27.1
require (
github.com/fergusstrange/embedded-postgres v1.34.0
github.com/jackc/pgx/v5 v5.11.0
)
require (
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/lib/pq v1.10.9 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/text v0.29.0 // indirect
)
+34
View File
@@ -0,0 +1,34 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fergusstrange/embedded-postgres v1.34.0 h1:c6RKhPKFsLVU+Tdxsx8q0UxCHsvZZ/iShAnljRBXs6s=
github.com/fergusstrange/embedded-postgres v1.34.0/go.mod h1:w0YvnCgf19o6tskInrOOACtnqfVlOvluz3hlNLY7tRk=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg=
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+82
View File
@@ -0,0 +1,82 @@
package media
import (
"fmt"
"path/filepath"
"strings"
"unicode"
)
const (
maxSeriesBytes = 90
maxEpisodeBytes = 140
maxEpisodeIDBytes = 64
maxFilenameBytes = 255
)
// RelativeVideoPath は worker と web が共有する保存先の相対パスを返す。
func RelativeVideoPath(seriesTitle, episodeTitle, episodeID string) (string, error) {
series := sanitizeComponent(seriesTitle, maxSeriesBytes)
episode := sanitizeComponent(episodeTitle, maxEpisodeBytes)
id := sanitizeComponent(episodeID, -1)
if len(id) > maxEpisodeIDBytes {
return "", fmt.Errorf("エピソードIDはサニタイズ後%dバイト以下である必要があります: %dバイト", maxEpisodeIDBytes, len(id))
}
// A directory entry must fit within the NFS namlen limit. Keep the episode
// ID intact and use the remaining bytes for the episode title.
const separators = " [].mp4"
seriesInFilename := series
titleBytes := maxFilenameBytes - len(seriesInFilename) - len(id) - len(separators)
if titleBytes < 0 {
titleBytes = 0
}
episode = truncateComponent(episode, titleBytes, false)
return filepath.Join(series, seriesInFilename+" "+episode+" ["+id+"].mp4"), nil
}
func sanitizeComponent(value string, maxBytes int) string {
var result []rune
lastReplacement := false
for _, r := range value {
invalid := unicode.IsControl(r) || strings.ContainsRune(`<>:"/\|?*`, r)
if invalid {
if !lastReplacement {
result = append(result, '_')
lastReplacement = true
}
continue
}
if unicode.IsSpace(r) {
r = ' '
}
result = append(result, r)
lastReplacement = false
}
clean := strings.Trim(string(result), " .")
if clean == "" || clean == "." || clean == ".." {
return "名称未設定"
}
return truncateComponent(clean, maxBytes, true)
}
func truncateComponent(value string, maxBytes int, fallback bool) string {
if maxBytes < 0 {
return value
}
var limited []rune
bytes := 0
for _, r := range value {
size := len(string(r))
if bytes+size > maxBytes {
break
}
limited = append(limited, r)
bytes += size
}
clean := strings.Trim(string(limited), " .")
if fallback && (clean == "" || clean == "." || clean == "..") {
return "名称未設定"
}
return clean
}
+267
View File
@@ -0,0 +1,267 @@
package media
import (
"fmt"
"path/filepath"
"strings"
"testing"
"unicode"
"unicode/utf8"
)
func TestRelativeVideoPath(t *testing.T) {
got, err := RelativeVideoPath("番組名", "第1話", "ep123")
if err != nil {
t.Fatal(err)
}
want := filepath.Join("番組名", "番組名 第1話 [ep123].mp4")
if got != want {
t.Fatalf("path=%q want=%q", got, want)
}
other, err := RelativeVideoPath("番組名", "第1話", "ep456")
if err != nil {
t.Fatal(err)
}
if got == other || !strings.Contains(other, "[ep456]") {
t.Fatalf("ID による衝突回避: %q %q", got, other)
}
}
func TestRelativeVideoPathSanitizesComponents(t *testing.T) {
got, err := RelativeVideoPath(" 番組:/\\*?<>|\"\x00\n\t\x7f\u0085. ", "\x00\n", "../")
if err != nil {
t.Fatal(err)
}
if filepath.IsAbs(got) || strings.ContainsAny(got, `:*?<>|"`) || strings.Contains(got, "..") {
t.Fatalf("禁止文字が残った: %q", got)
}
for _, r := range got {
if unicode.IsControl(r) {
t.Errorf("制御文字が残った: %U in %q", r, got)
}
}
parts := strings.Split(filepath.ToSlash(got), "/")
if len(parts) != 2 || parts[0] == "" || !strings.HasSuffix(got, ".mp4") {
t.Fatalf("相対パスの形: %q", got)
}
}
func TestRelativeVideoPathHandlesEmptyAndLongNames(t *testing.T) {
for _, name := range []string{"", " ", ".", "..", "..."} {
got, err := RelativeVideoPath(name, name, "")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(got, "名称未設定") {
t.Errorf("空の代替名 %q: %q", name, got)
}
}
long := strings.Repeat("界", 200)
got, err := RelativeVideoPath(long, long, "id")
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(got), "/")
if len(parts) != 2 || len([]byte(parts[0])) != maxSeriesBytes || !utf8.ValidString(parts[0]) {
t.Fatalf("番組名の%dバイト境界: %q (%d bytes)", maxSeriesBytes, parts[0], len([]byte(parts[0])))
}
if len([]byte(parts[1])) > maxFilenameBytes || !utf8.ValidString(parts[1]) {
t.Fatalf("ファイル名の上限: %q (%d bytes)", parts[1], len([]byte(parts[1])))
}
if got := sanitizeComponent(strings.Repeat("a", maxEpisodeBytes+1), maxEpisodeBytes); len([]byte(got)) != maxEpisodeBytes {
t.Fatalf("ASCII の%dバイト境界: %d %q", maxEpisodeBytes, len([]byte(got)), got)
}
}
func TestRelativeVideoPathEpisodeIDLimit(t *testing.T) {
id := strings.Repeat("i", maxEpisodeIDBytes)
got, err := RelativeVideoPath(strings.Repeat("s", maxSeriesBytes), strings.Repeat("t", maxEpisodeBytes), id)
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(got), "/")
if len(parts) != 2 || len([]byte(parts[1])) > maxFilenameBytes || !strings.Contains(parts[1], "["+id+"]") {
t.Fatalf("64バイトIDのファイル名=%q", got)
}
if _, err := RelativeVideoPath("series", "episode", strings.Repeat("i", maxEpisodeIDBytes+1)); err == nil {
t.Fatal("65バイトのエピソードIDを受理した")
}
}
func TestRelativeVideoPathUsesFilenameBudgetInRequiredOrder(t *testing.T) {
series := strings.Repeat("s", maxSeriesBytes)
title := strings.Repeat("t", maxEpisodeBytes)
for _, tt := range []struct {
name string
idBytes int
wantSeriesInFile int
wantEpisodeInFile int
wantFilenameBytes int
}{
{"ちょうど255バイト", 17, 90, 140, 255},
{"回タイトルから1バイト削る", 18, 90, 139, 255},
{"64バイトIDでも番組名を維持", 64, 90, 93, 255},
} {
t.Run(tt.name, func(t *testing.T) {
id := strings.Repeat("i", tt.idBytes)
path, err := RelativeVideoPath(series, title, id)
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(path), "/")
if len(parts) != 2 || parts[0] != series {
t.Fatalf("ディレクトリ名=%q", parts)
}
wantFilename := strings.Repeat("s", tt.wantSeriesInFile) + " " + strings.Repeat("t", tt.wantEpisodeInFile) + " [" + id + "].mp4"
if parts[1] != wantFilename || len([]byte(parts[1])) != tt.wantFilenameBytes {
t.Fatalf("filename=%q (%d bytes) want=%q (%d bytes)", parts[1], len([]byte(parts[1])), wantFilename, tt.wantFilenameBytes)
}
})
}
}
func TestRelativeVideoPathEpisodeIDNearAndBeyondFilenameLimit(t *testing.T) {
for _, idBytes := range []int{64, 65, 248, 255, 256} {
t.Run(fmt.Sprintf("id_%d_bytes", idBytes), func(t *testing.T) {
id := strings.Repeat("i", idBytes)
path, err := RelativeVideoPath("series", "episode", id)
if idBytes > 64 {
if err == nil || path != "" {
t.Fatalf("%dバイトID: path=%q err=%v, want error", idBytes, path, err)
}
if !strings.Contains(err.Error(), "64バイト以下") {
t.Fatalf("%dバイトIDのエラー理由=%q", idBytes, err)
}
return
}
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(path), "/")
if len(parts) != 2 {
t.Fatalf("parts=%q", parts)
}
if !strings.Contains(parts[1], "["+id+"]") {
t.Fatalf("episode ID was changed: %q", parts[1])
}
if got := len([]byte(parts[1])); got > maxFilenameBytes {
t.Fatalf("episode ID %d bytes produced %d-byte filename above %d: %q", idBytes, got, maxFilenameBytes, parts[1])
}
if !utf8.ValidString(parts[1]) {
t.Fatalf("filename is not valid UTF-8: %q", parts[1])
}
})
}
}
func TestRelativeVideoPathHasNoRuneLimit(t *testing.T) {
series := strings.Repeat("a", 90)
title := strings.Repeat("b", 100)
got, err := RelativeVideoPath(series, title, "id")
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(got), "/")
if parts[0] != series || !strings.Contains(parts[1], " "+title+" [id].mp4") {
t.Fatalf("80 rune limit remains: %q", got)
}
}
func TestSanitizeComponentTrimsWhitespaceBeforeLengthLimit(t *testing.T) {
for _, tt := range []struct {
name, input, want string
}{
{"前後の空白", " \t 番組名\n ", "_ 番組名_"},
{"空白のみ", "   ", "名称未設定"},
{"制御文字のみ", "\t\n", "_"},
{"多数の先頭空白", strings.Repeat(" ", 100) + "番組名", "番組名"},
{"先頭空白と長い名前", strings.Repeat(" ", 70) + strings.Repeat("a", 100), strings.Repeat("a", maxSeriesBytes)},
} {
t.Run(tt.name, func(t *testing.T) {
if got := sanitizeComponent(tt.input, maxSeriesBytes); got != tt.want {
t.Errorf("sanitizeComponent()=%q want=%q", got, tt.want)
}
})
}
}
func TestSanitizeComponentTrimsAgainAfterLengthLimit(t *testing.T) {
for _, tt := range []struct {
name, input, want string
}{
{"90バイト目が空白", strings.Repeat("a", 89) + " " + "b", strings.Repeat("a", 89)},
{"90バイト目がドット", strings.Repeat("a", 89) + "." + "b", strings.Repeat("a", 89)},
{"切り詰め部分が空白のみ", "a" + strings.Repeat(" ", 89) + "b", "a"},
{"切り詰め部分がドットのみ", "a" + strings.Repeat(".", 89) + "b", "a"},
{"複数バイト文字の直後がドット", strings.Repeat("界", 29) + ".続", strings.Repeat("界", 29)},
} {
t.Run(tt.name, func(t *testing.T) {
got := sanitizeComponent(tt.input, maxSeriesBytes)
if got != tt.want {
t.Errorf("sanitizeComponent()=%q want=%q", got, tt.want)
}
if strings.Trim(got, " .") != got || got == "" {
t.Errorf("切り詰め後の名前が不正: %q", got)
}
})
}
}
func TestSanitizeComponentSafetyProperties(t *testing.T) {
var controls strings.Builder
for r := rune(0); r <= 0x9f; r++ {
if unicode.IsControl(r) {
controls.WriteRune(r)
}
}
inputs := []string{
"", " ", ".", "..", "...", `<>:"/\|?*`, controls.String(),
" \t 番組名\n. ", "a/b\\c", strings.Repeat("a", 500),
strings.Repeat("界", 500), strings.Repeat("😀", 500),
strings.Repeat(".", 100) + "name" + strings.Repeat(" ", 100),
string([]byte{0xff, 0xfe, 'a', 0xc0, 0xaf}),
}
for i, input := range inputs {
got := sanitizeComponent(input, maxEpisodeBytes)
if got == "" || got == "." || got == ".." {
t.Errorf("case %d: 空またはdot segment: %q", i, got)
}
if strings.Trim(got, " .") != got {
t.Errorf("case %d: 前後に空白またはdot: %q", i, got)
}
if len([]byte(got)) > 140 {
t.Errorf("case %d: 上限超過: bytes=%d %q", i, len([]byte(got)), got)
}
if !utf8.ValidString(got) {
t.Errorf("case %d: 不正なUTF-8: %q", i, got)
}
if strings.ContainsAny(got, `<>:"/\|?*`) {
t.Errorf("case %d: 禁止文字が残った: %q", i, got)
}
for _, r := range got {
if unicode.IsControl(r) {
t.Errorf("case %d: 制御文字が残った: %U in %q", i, r, got)
}
}
if again := sanitizeComponent(got, maxEpisodeBytes); again != got {
t.Errorf("case %d: 再適用で変化: %q -> %q", i, got, again)
}
path, err := RelativeVideoPath(input, input, "id")
if err != nil {
t.Errorf("case %d: パス生成エラー: %v", i, err)
continue
}
parts := strings.Split(filepath.ToSlash(path), "/")
if filepath.IsAbs(path) || len(parts) != 2 || parts[0] == "" || parts[0] == "." || parts[0] == ".." || !strings.HasSuffix(parts[1], ".mp4") {
t.Errorf("case %d: 安全でない相対パス: %q parts=%q", i, path, parts)
continue
}
if len([]byte(parts[0])) > 90 || len([]byte(parts[1])) > 255 {
t.Errorf("case %d: パス要素の上限超過: dir=%d filename=%d path=%q", i, len([]byte(parts[0])), len([]byte(parts[1])), path)
}
if !utf8.ValidString(path) {
t.Errorf("case %d: パスが不正なUTF-8: %q", i, path)
}
}
}
+205
View File
@@ -0,0 +1,205 @@
package store
import (
"context"
"errors"
"testing"
"time"
"github.com/jackc/pgx/v5"
)
func completeTestJob(t *testing.T, s *Store, id string) Job {
t.Helper()
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode(id), "manual")
if err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != job.ID {
t.Fatalf("claim: %+v %v", claimed, err)
}
if err := s.Complete(ctx, job.ID, "番組/video.mp4", 1234); err != nil {
t.Fatal(err)
}
result, err := s.GetSucceededJob(ctx, job.ID)
if err != nil {
t.Fatal(err)
}
return result
}
func TestLibrarySeriesAggregationAndJobs(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
complete := func(ep Episode, size int64, finished time.Time) Job {
t.Helper()
job, _, err := s.Enqueue(ctx, ep, "manual")
if err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != job.ID {
t.Fatalf("claim: %+v %v", claimed, err)
}
if err := s.Complete(ctx, job.ID, ep.SeriesTitle+"/video.mp4", size); err != nil {
t.Fatal(err)
}
if _, err := s.pool.Exec(ctx, `UPDATE download_jobs SET finished_at=$2 WHERE id=$1`, job.ID, finished); err != nil {
t.Fatal(err)
}
result, err := s.GetSucceededJob(ctx, job.ID)
if err != nil {
t.Fatal(err)
}
return result
}
older := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
newer := older.Add(time.Hour)
newest := newer.Add(time.Hour)
first := complete(Episode{ID: "aggregate-a1", SeriesID: "seriesA", SeriesTitle: "旧番組名", Title: "第1話", BroadcasterName: "旧局", ThumbnailPath: "/old.jpg"}, 100, older)
second := complete(Episode{ID: "aggregate-a2", SeriesID: "seriesA", SeriesTitle: "新番組名", Title: "第2話", BroadcasterName: "新局", ThumbnailPath: "/new.jpg"}, 250, newer)
complete(Episode{ID: "aggregate-b1", SeriesID: "seriesB", SeriesTitle: "別番組", Title: "第1話", BroadcasterName: "別局", ThumbnailPath: "/other.jpg"}, 500, newest)
if _, err := s.pool.Exec(ctx, `UPDATE download_jobs SET file_size_bytes=NULL WHERE id=$1`, first.ID); err != nil {
t.Fatal(err)
}
if _, err := s.pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title) VALUES ('seriesA', '登録名')`); err != nil {
t.Fatal(err)
}
series, err := s.ListLibrarySeries(ctx)
if err != nil {
t.Fatal(err)
}
if len(series) != 2 || series[0].SeriesID != "seriesB" || series[1].SeriesID != "seriesA" {
t.Fatalf("番組の並び: %+v", series)
}
got := series[1]
if got.SeriesTitle != "新番組名" || got.LatestTitle != "第2話" || got.BroadcasterName != "新局" || got.ThumbnailPath != "/new.jpg" || got.EpisodeCount != 2 || got.TotalSizeBytes != 250 || got.LatestFinishedAt == nil || !got.LatestFinishedAt.Equal(newer) || !got.Subscribed {
t.Fatalf("番組集計: %+v", got)
}
if series[0].Subscribed || series[0].TotalSizeBytes != 500 {
t.Fatalf("未登録番組: %+v", series[0])
}
jobs, err := s.ListSucceededJobsBySeries(ctx, "seriesA")
if err != nil || len(jobs) != 2 || jobs[0].ID != second.ID || jobs[1].ID != first.ID || jobs[1].FileSizeBytes != nil {
t.Fatalf("番組の回: %+v %v", jobs, err)
}
if jobs, err := s.ListSucceededJobsBySeries(ctx, "missing"); err != nil || len(jobs) != 0 {
t.Fatalf("存在しない番組: %+v %v", jobs, err)
}
if _, err := s.ListSucceededJobsBySeries(ctx, ""); err == nil {
t.Fatal("空の番組IDを許した")
}
}
func TestLibraryStoreOperations(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
first := completeTestJob(t, s, "library-1")
second := completeTestJob(t, s, "library-2")
jobs, err := s.ListSucceededJobs(ctx, 1)
if err != nil || len(jobs) != 1 || jobs[0].ID != second.ID {
t.Fatalf("完了一覧: %+v %v", jobs, err)
}
if first.FilePath == nil || *first.FilePath != "番組/video.mp4" || first.FileSizeBytes == nil || *first.FileSizeBytes != 1234 || first.FinishedAt == nil {
t.Fatalf("完了情報: %+v", first)
}
queued, _, err := s.Enqueue(ctx, sampleEpisode("not-finished"), "manual")
if err != nil {
t.Fatal(err)
}
if _, err := s.GetSucceededJob(ctx, queued.ID); !errors.Is(err, pgx.ErrNoRows) {
t.Fatalf("未完了の取得: %v", err)
}
if err := s.DeleteSucceededJob(ctx, queued.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("未完了の削除: %v", err)
}
if err := s.DeleteSucceededJob(ctx, first.ID); err != nil {
t.Fatal(err)
}
if _, err := s.GetJob(ctx, first.ID); !errors.Is(err, pgx.ErrNoRows) {
t.Fatalf("削除後の取得: %v", err)
}
if err := s.DeleteSucceededJob(ctx, first.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("再削除: %v", err)
}
if _, err := s.ListSucceededJobs(ctx, 0); err == nil {
t.Fatal("0件の上限を許した")
}
}
func TestDeleteSucceededJobKeepsKnownEpisodeAndAllowsManualRequeue(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
ep := seriesEpisode("known-library", "known-series")
created, err := s.RegisterSubscription(ctx, "known-series", "番組", []Episode{ep}, false)
if err != nil || !created {
t.Fatalf("番組登録: %t %v", created, err)
}
job, created, err := s.Enqueue(ctx, ep, "manual")
if err != nil || !created {
t.Fatalf("手動投入: %+v %t %v", job, created, err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != job.ID {
t.Fatalf("取得: %+v %v", claimed, err)
}
if err := s.Complete(ctx, job.ID, "番組/video.mp4", 10); err != nil {
t.Fatal(err)
}
if err := s.DeleteSucceededJob(ctx, job.ID); err != nil {
t.Fatal(err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "known-series", []string{ep.ID})
if err != nil || len(unknown) != 0 {
t.Fatalf("削除後の既知記録: %v %v", unknown, err)
}
requeued, created, err := s.Enqueue(ctx, ep, "manual")
if err != nil || !created || requeued.Status != "queued" || requeued.Source != "manual" {
t.Fatalf("削除後の手動再投入: %+v %t %v", requeued, created, err)
}
}
func TestListSucceededJobsByIDsAndSeriesIDs(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
first := completeTestJob(t, s, "bulk-list-1")
ep := seriesEpisode("bulk-list-2", "series-2")
second, _, err := s.Enqueue(ctx, ep, "manual")
if err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != second.ID {
t.Fatalf("claim: %+v %v", claimed, err)
}
if err := s.Complete(ctx, second.ID, "別番組/video.mp4", 5678); err != nil {
t.Fatal(err)
}
second, err = s.GetSucceededJob(ctx, second.ID)
if err != nil {
t.Fatal(err)
}
queued, _, err := s.Enqueue(ctx, seriesEpisode("bulk-list-queued", "series-2"), "manual")
if err != nil {
t.Fatal(err)
}
byIDs, err := s.ListSucceededJobsByIDs(ctx, []int64{second.ID, queued.ID, first.ID})
if err != nil || len(byIDs) != 2 || byIDs[0].ID != first.ID || byIDs[1].ID != second.ID {
t.Fatalf("ID指定の完了済み取得: %+v %v", byIDs, err)
}
bySeries, err := s.ListSucceededJobsBySeriesIDs(ctx, []string{"series-2", "series-1"})
if err != nil || len(bySeries) != 2 || bySeries[0].ID != first.ID || bySeries[1].ID != second.ID {
t.Fatalf("番組指定の完了済み取得: %+v %v", bySeries, err)
}
if empty, err := s.ListSucceededJobsByIDs(ctx, nil); err != nil || len(empty) != 0 {
t.Fatalf("空のID指定: %+v %v", empty, err)
}
if empty, err := s.ListSucceededJobsBySeriesIDs(ctx, []string{}); err != nil || len(empty) != 0 {
t.Fatalf("空の番組指定: %+v %v", empty, err)
}
}
+73
View File
@@ -0,0 +1,73 @@
package store
import (
"context"
"embed"
"fmt"
"io/fs"
"sort"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
//go:embed migrations/*.sql
var migrations embed.FS
const migrationLockID int64 = 0x747665725f646231
// Migrate は同じ接続で排他ロックを保持し、未適用の SQL を順に実行する。
func Migrate(ctx context.Context, pool *pgxpool.Pool) error {
conn, err := pool.Acquire(ctx)
if err != nil {
return fmt.Errorf("migration connection: %w", err)
}
defer conn.Release()
if _, err := conn.Exec(ctx, "SELECT pg_advisory_lock($1)", migrationLockID); err != nil {
return fmt.Errorf("migration lock: %w", err)
}
defer func() {
// 接続を返す前にロックを解放する。解放に失敗した接続は閉じる。
if _, err := conn.Exec(context.Background(), "SELECT pg_advisory_unlock($1)", migrationLockID); err != nil {
conn.Conn().Close(context.Background())
}
}()
if _, err := conn.Exec(ctx, `CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)`); err != nil {
return fmt.Errorf("migration metadata: %w", err)
}
files, err := fs.Glob(migrations, "migrations/*.sql")
if err != nil {
return err
}
sort.Strings(files)
for _, name := range files {
tx, err := conn.Begin(ctx)
if err != nil {
return err
}
var applied bool
err = tx.QueryRow(ctx, "SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version=$1)", name).Scan(&applied)
if err == nil && !applied {
var sql []byte
sql, err = migrations.ReadFile(name)
if err == nil {
_, err = tx.Exec(ctx, string(sql), pgx.QueryExecModeSimpleProtocol)
}
if err == nil {
_, err = tx.Exec(ctx, "INSERT INTO schema_migrations (version) VALUES ($1)", name)
}
}
if err != nil {
_ = tx.Rollback(ctx)
return fmt.Errorf("migration %s: %w", name, err)
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("migration %s commit: %w", name, err)
}
}
return nil
}
+44
View File
@@ -0,0 +1,44 @@
CREATE TABLE download_jobs (
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
episode_id TEXT NOT NULL UNIQUE,
series_id TEXT NOT NULL,
series_title TEXT NOT NULL,
title TEXT NOT NULL,
broadcaster_name TEXT NOT NULL,
duration_seconds INTEGER NOT NULL CHECK (duration_seconds >= 0),
end_at TIMESTAMPTZ,
thumbnail_path TEXT NOT NULL,
source TEXT NOT NULL CHECK (source IN ('manual', 'subscription')),
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'succeeded', 'failed', 'canceled')),
phase TEXT CHECK (phase IN ('video', 'audio', 'merging', 'moving')),
downloaded_bytes BIGINT CHECK (downloaded_bytes >= 0),
total_bytes BIGINT CHECK (total_bytes >= 0),
total_is_estimate BOOLEAN NOT NULL DEFAULT FALSE,
speed_bps DOUBLE PRECISION CHECK (speed_bps >= 0),
eta_seconds DOUBLE PRECISION CHECK (eta_seconds >= 0),
fragment_index INTEGER CHECK (fragment_index >= 0),
fragment_count INTEGER CHECK (fragment_count >= 0),
format_id TEXT,
error_kind TEXT CHECK (error_kind IN ('drm', 'geo', 'network', 'unavailable', 'extract', 'canceled', 'invalid_output', 'unknown')),
error_message TEXT,
file_path TEXT,
file_size_bytes BIGINT CHECK (file_size_bytes >= 0),
attempts INTEGER NOT NULL DEFAULT 0 CHECK (attempts >= 0),
heartbeat_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
started_at TIMESTAMPTZ,
finished_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX download_jobs_claim_idx ON download_jobs (created_at, id) WHERE status = 'queued';
CREATE INDEX download_jobs_status_newest_idx ON download_jobs (status, created_at DESC, id DESC);
CREATE INDEX download_jobs_stale_idx ON download_jobs (heartbeat_at) WHERE status = 'running';
CREATE TABLE subscriptions (
series_id TEXT PRIMARY KEY,
series_title TEXT NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
last_checked_at TIMESTAMPTZ
);
@@ -0,0 +1,8 @@
CREATE TABLE known_episodes (
series_id TEXT NOT NULL REFERENCES subscriptions(series_id) ON DELETE CASCADE,
episode_id TEXT NOT NULL,
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (series_id, episode_id)
);
CREATE INDEX download_jobs_series_idx ON download_jobs (series_id);
@@ -0,0 +1,6 @@
ALTER TABLE subscriptions
ADD COLUMN last_check_error_at TIMESTAMPTZ,
ADD COLUMN last_check_error TEXT;
CREATE INDEX subscriptions_due_idx
ON subscriptions (last_checked_at) WHERE enabled;
+424
View File
@@ -0,0 +1,424 @@
package store
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
var ErrInvalidTransition = errors.New("invalid job status transition")
type Store struct{ pool *pgxpool.Pool }
func New(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
type Episode struct {
ID, SeriesID, SeriesTitle, Title, BroadcasterName, ThumbnailPath string
DurationSeconds int
EndAt *time.Time
}
type Job struct {
ID int64
EpisodeID, SeriesID, SeriesTitle string
Title, BroadcasterName string
DurationSeconds int
EndAt *time.Time
ThumbnailPath, Source, Status string
Phase, FormatID *string
DownloadedBytes, TotalBytes *int64
TotalIsEstimate bool
SpeedBPS, ETASeconds *float64
FragmentIndex, FragmentCount *int32
ErrorKind, ErrorMessage, FilePath *string
FileSizeBytes *int64
Attempts int
HeartbeatAt, StartedAt, FinishedAt *time.Time
CreatedAt, UpdatedAt time.Time
}
type LibrarySeries struct {
SeriesID, SeriesTitle, LatestTitle string
BroadcasterName, ThumbnailPath string
EpisodeCount, TotalSizeBytes int64
LatestFinishedAt *time.Time
Subscribed bool
}
type Progress struct {
Phase string
DownloadedBytes *int64
TotalBytes *int64
TotalIsEstimate bool
SpeedBPS *float64
ETASeconds *float64
FragmentIndex *int32
FragmentCount *int32
FormatID *string
}
type Subscription struct {
SeriesID, SeriesTitle string
Enabled bool
CreatedAt time.Time
LastCheckedAt *time.Time
LastCheckErrorAt *time.Time
LastCheckError *string
JobCount int64
}
const jobColumns = `id, episode_id, series_id, series_title, title, broadcaster_name,
duration_seconds, end_at, thumbnail_path, source, status, phase,
downloaded_bytes, total_bytes, total_is_estimate, speed_bps, eta_seconds,
fragment_index, fragment_count, format_id, error_kind, error_message,
file_path, file_size_bytes, attempts, heartbeat_at, created_at,
started_at, finished_at, updated_at`
var claimedJobColumns = func() string {
columns := strings.Split(jobColumns, ",")
for i, column := range columns {
columns[i] = "j." + strings.TrimSpace(column)
}
return strings.Join(columns, ", ")
}()
func scanJob(row pgx.Row) (Job, error) {
var j Job
err := row.Scan(&j.ID, &j.EpisodeID, &j.SeriesID, &j.SeriesTitle, &j.Title,
&j.BroadcasterName, &j.DurationSeconds, &j.EndAt, &j.ThumbnailPath,
&j.Source, &j.Status, &j.Phase, &j.DownloadedBytes, &j.TotalBytes,
&j.TotalIsEstimate, &j.SpeedBPS, &j.ETASeconds, &j.FragmentIndex,
&j.FragmentCount, &j.FormatID, &j.ErrorKind, &j.ErrorMessage,
&j.FilePath, &j.FileSizeBytes, &j.Attempts, &j.HeartbeatAt,
&j.CreatedAt, &j.StartedAt, &j.FinishedAt, &j.UpdatedAt)
return j, err
}
// Enqueue は同じエピソードの既存ジョブを再利用する。
func (s *Store) Enqueue(ctx context.Context, ep Episode, source string) (Job, bool, error) {
row := s.pool.QueryRow(ctx, `INSERT INTO download_jobs
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9) ON CONFLICT (episode_id) DO NOTHING RETURNING `+jobColumns,
ep.ID, ep.SeriesID, ep.SeriesTitle, ep.Title, ep.BroadcasterName,
ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath, source)
job, err := scanJob(row)
if err == nil {
return job, true, nil
}
if !errors.Is(err, pgx.ErrNoRows) {
return Job{}, false, err
}
job, err = scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE episode_id=$1`, ep.ID))
return job, false, err
}
func (s *Store) GetJob(ctx context.Context, id int64) (Job, error) {
return scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE id=$1`, id))
}
func (s *Store) GetSucceededJob(ctx context.Context, id int64) (Job, error) {
return scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE id=$1 AND status='succeeded'`, id))
}
func (s *Store) ListSucceededJobs(ctx context.Context, limit int) ([]Job, error) {
if limit <= 0 {
return nil, errors.New("limit must be positive")
}
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
WHERE status='succeeded' ORDER BY finished_at DESC NULLS LAST, id DESC LIMIT $1`, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var jobs []Job
for rows.Next() {
job, err := scanJob(rows)
if err != nil {
return nil, err
}
jobs = append(jobs, job)
}
return jobs, rows.Err()
}
func (s *Store) ListLibrarySeries(ctx context.Context) ([]LibrarySeries, error) {
rows, err := s.pool.Query(ctx, `WITH totals AS (
SELECT series_id, count(*) AS episode_count,
COALESCE(sum(file_size_bytes), 0) AS total_size_bytes
FROM download_jobs WHERE status='succeeded' GROUP BY series_id
), latest AS (
SELECT DISTINCT ON (series_id) series_id, series_title, title,
broadcaster_name, thumbnail_path, finished_at, id
FROM download_jobs WHERE status='succeeded'
ORDER BY series_id, finished_at DESC NULLS LAST, id DESC
)
SELECT l.series_id, l.series_title, l.title, l.broadcaster_name,
l.thumbnail_path, t.episode_count, t.total_size_bytes, l.finished_at,
EXISTS (SELECT 1 FROM subscriptions s WHERE s.series_id=l.series_id)
FROM latest l JOIN totals t USING (series_id)
ORDER BY l.finished_at DESC NULLS LAST, l.id DESC`)
if err != nil {
return nil, err
}
defer rows.Close()
var result []LibrarySeries
for rows.Next() {
var item LibrarySeries
if err := rows.Scan(&item.SeriesID, &item.SeriesTitle, &item.LatestTitle,
&item.BroadcasterName, &item.ThumbnailPath, &item.EpisodeCount,
&item.TotalSizeBytes, &item.LatestFinishedAt, &item.Subscribed); err != nil {
return nil, err
}
result = append(result, item)
}
return result, rows.Err()
}
func (s *Store) ListSucceededJobsBySeries(ctx context.Context, seriesID string) ([]Job, error) {
if seriesID == "" {
return nil, errors.New("series ID must not be empty")
}
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
WHERE status='succeeded' AND series_id=$1
ORDER BY finished_at DESC NULLS LAST, id DESC`, seriesID)
if err != nil {
return nil, err
}
defer rows.Close()
var jobs []Job
for rows.Next() {
job, err := scanJob(rows)
if err != nil {
return nil, err
}
jobs = append(jobs, job)
}
return jobs, rows.Err()
}
func (s *Store) ListSucceededJobsByIDs(ctx context.Context, ids []int64) ([]Job, error) {
if len(ids) == 0 {
return []Job{}, nil
}
return s.listSucceededJobs(ctx, `id = ANY($1)`, ids)
}
func (s *Store) ListSucceededJobsBySeriesIDs(ctx context.Context, seriesIDs []string) ([]Job, error) {
if len(seriesIDs) == 0 {
return []Job{}, nil
}
return s.listSucceededJobs(ctx, `series_id = ANY($1)`, seriesIDs)
}
func (s *Store) listSucceededJobs(ctx context.Context, condition string, value any) ([]Job, error) {
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
WHERE status='succeeded' AND `+condition+`
ORDER BY id`, value)
if err != nil {
return nil, err
}
defer rows.Close()
var jobs []Job
for rows.Next() {
job, err := scanJob(rows)
if err != nil {
return nil, err
}
jobs = append(jobs, job)
}
return jobs, rows.Err()
}
func (s *Store) DeleteSucceededJob(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `DELETE FROM download_jobs WHERE id=$1 AND status='succeeded'`, id)
return transitionResult(tag.RowsAffected(), err)
}
// Retry は失敗・中断されたジョブの実行時情報を消して再投入する。
func (s *Store) Retry(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
error_kind=NULL, error_message=NULL, heartbeat_at=NULL, started_at=NULL,
finished_at=NULL, updated_at=now()
WHERE id=$1 AND status IN ('failed','canceled')`, id)
return transitionResult(tag.RowsAffected(), err)
}
// RetryEpisode はエピソード ID で失敗・中断されたジョブを再投入する。
func (s *Store) RetryEpisode(ctx context.Context, episodeID string) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
error_kind=NULL, error_message=NULL, heartbeat_at=NULL, started_at=NULL,
finished_at=NULL, updated_at=now()
WHERE episode_id=$1 AND status IN ('failed','canceled')`, episodeID)
return transitionResult(tag.RowsAffected(), err)
}
// ClaimNext は最も古い未実行ジョブを同時 worker と重複せず取得する。
func (s *Store) ClaimNext(ctx context.Context) (Job, error) {
return scanJob(s.pool.QueryRow(ctx, `WITH next AS (
SELECT id FROM download_jobs WHERE status='queued'
ORDER BY created_at, id LIMIT 1 FOR UPDATE SKIP LOCKED
) UPDATE download_jobs AS j SET status='running', attempts=j.attempts+1,
started_at=now(), heartbeat_at=now(), updated_at=now()
FROM next WHERE j.id=next.id RETURNING `+claimedJobColumns))
}
func (s *Store) UpdateProgress(ctx context.Context, id int64, p Progress) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET phase=$2, downloaded_bytes=$3,
total_bytes=$4, total_is_estimate=$5, speed_bps=$6, eta_seconds=$7,
fragment_index=$8, fragment_count=$9, format_id=$10,
heartbeat_at=now(), updated_at=now() WHERE id=$1 AND status='running'`,
id, p.Phase, p.DownloadedBytes, p.TotalBytes, p.TotalIsEstimate,
p.SpeedBPS, p.ETASeconds, p.FragmentIndex, p.FragmentCount, p.FormatID)
return transitionResult(tag.RowsAffected(), err)
}
// Heartbeat は進捗出力がない間も実行中ジョブの生存を記録する。
func (s *Store) Heartbeat(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=now(), updated_at=now()
WHERE id=$1 AND status='running'`, id)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) Complete(ctx context.Context, id int64, filePath string, fileSizeBytes int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='succeeded', phase=NULL,
file_path=$2, file_size_bytes=$3, heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status='running'`, id, filePath, fileSizeBytes)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) Fail(ctx context.Context, id int64, kind, message string) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='failed', phase=NULL,
error_kind=$2, error_message=$3, heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status='running'`, id, kind, message)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) Cancel(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='canceled', phase=NULL,
error_kind='canceled', heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status IN ('queued','running')`, id)
return transitionResult(tag.RowsAffected(), err)
}
// CancelQueued は worker 未着手のジョブだけを取り消す。
func (s *Store) CancelQueued(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='canceled', phase=NULL,
error_kind='canceled', heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status='queued'`, id)
return transitionResult(tag.RowsAffected(), err)
}
// RequeueStale は cutoff より前に heartbeat が止まった実行中ジョブを戻す。
func (s *Store) RequeueStale(ctx context.Context, cutoff time.Time) (int64, error) {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
heartbeat_at=NULL, started_at=NULL, updated_at=now()
WHERE status='running' AND heartbeat_at < $1`, cutoff)
return tag.RowsAffected(), err
}
// RequeueRunning は worker の正常終了時に担当中ジョブを待機中へ戻す。
func (s *Store) RequeueRunning(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
heartbeat_at=NULL, started_at=NULL, updated_at=now()
WHERE id=$1 AND status='running'`, id)
return transitionResult(tag.RowsAffected(), err)
}
func transitionResult(count int64, err error) error {
if err != nil {
return err
}
if count == 0 {
return ErrInvalidTransition
}
return nil
}
// ListJobs は指定状態のジョブを新しい順に返す。空の状態は全件を意味する。
func (s *Store) ListJobs(ctx context.Context, status string, limit int) ([]Job, error) {
if status != "" && status != "queued" && status != "running" && status != "succeeded" && status != "failed" && status != "canceled" {
return nil, fmt.Errorf("invalid status: %s", status)
}
if limit <= 0 {
return nil, errors.New("limit must be positive")
}
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
WHERE ($1='' OR status=$1) ORDER BY created_at DESC, id DESC LIMIT $2`, status, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var jobs []Job
for rows.Next() {
job, err := scanJob(rows)
if err != nil {
return nil, err
}
jobs = append(jobs, job)
}
return jobs, rows.Err()
}
func (s *Store) EpisodeStatuses(ctx context.Context, ids []string) (map[string]string, error) {
result := make(map[string]string)
if len(ids) == 0 {
return result, nil
}
rows, err := s.pool.Query(ctx, `SELECT episode_id, status FROM download_jobs WHERE episode_id = ANY($1)`, ids)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var id, status string
if err := rows.Scan(&id, &status); err != nil {
return nil, err
}
result[id] = status
}
return result, rows.Err()
}
func (s *Store) DeleteSubscription(ctx context.Context, seriesID string) (bool, error) {
tag, err := s.pool.Exec(ctx, `DELETE FROM subscriptions WHERE series_id=$1`, seriesID)
return tag.RowsAffected() == 1, err
}
func (s *Store) ListSubscriptions(ctx context.Context) ([]Subscription, error) {
rows, err := s.pool.Query(ctx, `SELECT s.series_id, s.series_title, s.enabled, s.created_at, s.last_checked_at,
s.last_check_error_at, s.last_check_error,
(SELECT count(*) FROM download_jobs j WHERE j.series_id=s.series_id)
FROM subscriptions s ORDER BY s.created_at DESC, s.series_id`)
if err != nil {
return nil, err
}
defer rows.Close()
var subs []Subscription
for rows.Next() {
var sub Subscription
if err := rows.Scan(&sub.SeriesID, &sub.SeriesTitle, &sub.Enabled, &sub.CreatedAt, &sub.LastCheckedAt,
&sub.LastCheckErrorAt, &sub.LastCheckError, &sub.JobCount); err != nil {
return nil, err
}
subs = append(subs, sub)
}
return subs, rows.Err()
}
func (s *Store) SetSubscriptionChecked(ctx context.Context, seriesID string, checkedAt time.Time) error {
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, seriesID, checkedAt)
return transitionResult(tag.RowsAffected(), err)
}
+766
View File
@@ -0,0 +1,766 @@
package store
import (
"context"
"errors"
"fmt"
"io"
"net"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
embeddedpostgres "github.com/fergusstrange/embedded-postgres"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
var adminPool *pgxpool.Pool
var testPort int
var dbSequence atomic.Uint64
func TestMain(m *testing.M) {
root, err := os.MkdirTemp("/private/tmp", "tver-pg17-")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
testPort = listener.Addr().(*net.TCPAddr).Port
listener.Close()
config := embeddedpostgres.DefaultConfig().Version(embeddedpostgres.V17).
Port(uint32(testPort)).CachePath("/private/tmp/tver-embedded-pg-cache").
RuntimePath(filepath.Join(root, "runtime")).DataPath(filepath.Join(root, "data")).
Logger(io.Discard).StartTimeout(60 * time.Second)
db := embeddedpostgres.NewDatabase(config)
if err := db.Start(); err != nil {
fmt.Fprintf(os.Stderr, "embedded Postgres 17 start: %v\n", err)
_ = os.RemoveAll(root)
os.Exit(1)
}
adminPool, err = pgxpool.New(context.Background(), config.GetConnectionURL())
if err != nil {
fmt.Fprintln(os.Stderr, err)
_ = db.Stop()
_ = os.RemoveAll(root)
os.Exit(1)
}
code := m.Run()
adminPool.Close()
if err := db.Stop(); err != nil {
fmt.Fprintf(os.Stderr, "embedded Postgres 17 stop: %v\n", err)
code = 1
}
_ = os.RemoveAll(root)
os.Exit(code)
}
func testPool(t *testing.T, migrate bool) *pgxpool.Pool {
t.Helper()
name := fmt.Sprintf("tver_test_%d", dbSequence.Add(1))
ctx := context.Background()
if _, err := adminPool.Exec(ctx, `CREATE DATABASE `+name); err != nil {
t.Fatal(err)
}
url := fmt.Sprintf("postgres://postgres:postgres@localhost:%d/%s", testPort, name)
pool, err := pgxpool.New(ctx, url)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
pool.Close()
if _, err := adminPool.Exec(ctx, `DROP DATABASE `+name); err != nil {
t.Error(err)
}
})
if migrate {
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
}
return pool
}
func sampleEpisode(id string) Episode {
end := time.Date(2026, 10, 4, 12, 0, 0, 0, time.UTC)
return Episode{ID: id, SeriesID: "series-1", SeriesTitle: "番組", Title: "第1話",
BroadcasterName: "放送局", DurationSeconds: 300, EndAt: &end, ThumbnailPath: "/thumb.jpg"}
}
func TestMigrate(t *testing.T) {
pool := testPool(t, false)
ctx := context.Background()
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var first time.Time
if err := pool.QueryRow(ctx, `SELECT applied_at FROM schema_migrations WHERE version='migrations/001_initial.sql'`).Scan(&first); err != nil {
t.Fatal(err)
}
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var count int
var second time.Time
if err := pool.QueryRow(ctx, `SELECT count(*), min(applied_at) FROM schema_migrations`).Scan(&count, &second); err != nil {
t.Fatal(err)
}
if count != 3 || !first.Equal(second) {
t.Fatalf("再適用で履歴が変わった: %d %v %v", count, first, second)
}
}
func TestMigrateConcurrent(t *testing.T) {
pool := testPool(t, false)
const workers = 8
errCh := make(chan error, workers)
var wg sync.WaitGroup
for range workers {
wg.Add(1)
go func() { defer wg.Done(); errCh <- Migrate(context.Background(), pool) }()
}
wg.Wait()
close(errCh)
for err := range errCh {
if err != nil {
t.Fatal(err)
}
}
var count int
if err := pool.QueryRow(context.Background(), `SELECT count(*) FROM schema_migrations`).Scan(&count); err != nil {
t.Fatal(err)
}
if count != 3 {
t.Fatalf("適用件数: %d", count)
}
}
func TestMigrateAddsKnownEpisodesToExistingDatabase(t *testing.T) {
pool := testPool(t, false)
ctx := context.Background()
initial, err := migrations.ReadFile("migrations/001_initial.sql")
if err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, string(initial), pgx.QueryExecModeSimpleProtocol); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `CREATE TABLE schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())`); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `INSERT INTO schema_migrations (version) VALUES ('migrations/001_initial.sql')`); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title) VALUES ('existing', '既存番組')`); err != nil {
t.Fatal(err)
}
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var count int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM schema_migrations`).Scan(&count); err != nil || count != 3 {
t.Fatalf("移行履歴: %d %v", count, err)
}
if _, err := pool.Exec(ctx, `INSERT INTO known_episodes (series_id, episode_id) VALUES ('existing', 'ep1')`); err != nil {
t.Fatalf("既存番組への既知記録: %v", err)
}
if _, err := pool.Exec(ctx, `DELETE FROM subscriptions WHERE series_id='existing'`); err != nil {
t.Fatal(err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id='existing'`).Scan(&count); err != nil || count != 0 {
t.Fatalf("アップグレード後の cascade: %d %v", count, err)
}
}
func TestMigrateAddsSubscriptionCheckErrorsToVersionTwoDatabase(t *testing.T) {
pool := testPool(t, false)
ctx := context.Background()
for _, name := range []string{"migrations/001_initial.sql", "migrations/002_known_episodes.sql"} {
sql, err := migrations.ReadFile(name)
if err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, string(sql), pgx.QueryExecModeSimpleProtocol); err != nil {
t.Fatal(err)
}
}
if _, err := pool.Exec(ctx, `CREATE TABLE schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())`); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `INSERT INTO schema_migrations (version) VALUES ('migrations/001_initial.sql'), ('migrations/002_known_episodes.sql')`); err != nil {
t.Fatal(err)
}
checked := time.Date(2026, 10, 1, 2, 3, 4, 0, time.UTC)
if _, err := pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title, last_checked_at) VALUES ('existing-v2', '既存番組', $1)`, checked); err != nil {
t.Fatal(err)
}
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var title string
var last time.Time
var errorAt, errorMessage any
if err := pool.QueryRow(ctx, `SELECT series_title, last_checked_at, last_check_error_at, last_check_error FROM subscriptions WHERE series_id='existing-v2'`).Scan(&title, &last, &errorAt, &errorMessage); err != nil || title != "既存番組" || !last.Equal(checked) || errorAt != nil || errorMessage != nil {
t.Fatalf("v2データ保持と追加列: title=%q last=%v errorAt=%v error=%v err=%v", title, last, errorAt, errorMessage, err)
}
var indexDef string
if err := pool.QueryRow(ctx, `SELECT indexdef FROM pg_indexes WHERE schemaname=current_schema() AND indexname='subscriptions_due_idx'`).Scan(&indexDef); err != nil || !strings.Contains(indexDef, "last_checked_at") || !strings.Contains(indexDef, "WHERE enabled") {
t.Fatalf("期限抽出索引: %q %v", indexDef, err)
}
var versions int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM schema_migrations`).Scan(&versions); err != nil || versions != 3 {
t.Fatalf("移行履歴: %d %v", versions, err)
}
}
func TestDownloadJobIndexes(t *testing.T) {
pool := testPool(t, true)
rows, err := pool.Query(context.Background(), `SELECT indexname, indexdef FROM pg_indexes WHERE schemaname=current_schema() AND tablename='download_jobs'`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
indexes := make(map[string]string)
for rows.Next() {
var name, definition string
if err := rows.Scan(&name, &definition); err != nil {
t.Fatal(err)
}
indexes[name] = definition
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
checks := map[string][]string{
"download_jobs_episode_id_key": {"UNIQUE", "(episode_id)"},
"download_jobs_claim_idx": {"(created_at, id)", "status = 'queued'"},
"download_jobs_status_newest_idx": {"(status, created_at DESC, id DESC)"},
"download_jobs_stale_idx": {"(heartbeat_at)", "status = 'running'"},
}
for name, parts := range checks {
definition := indexes[name]
for _, part := range parts {
if !strings.Contains(definition, part) {
t.Errorf("索引 %s に %q がない: %q", name, part, definition)
}
}
}
}
func TestEnqueueAndListing(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
first, created, err := s.Enqueue(ctx, sampleEpisode("ep1"), "manual")
if err != nil || !created || first.Status != "queued" || first.Source != "manual" || first.DurationSeconds != 300 || first.EndAt == nil {
t.Fatalf("enqueue: %+v %v %v", first, created, err)
}
again, created, err := s.Enqueue(ctx, sampleEpisode("ep1"), "subscription")
if err != nil || created || again.ID != first.ID || again.Source != "manual" {
t.Fatalf("重複: %+v %v %v", again, created, err)
}
second, _, err := s.Enqueue(ctx, sampleEpisode("ep2"), "subscription")
if err != nil {
t.Fatal(err)
}
jobs, err := s.ListJobs(ctx, "queued", 10)
if err != nil || len(jobs) != 2 || jobs[0].ID != second.ID {
t.Fatalf("list: %+v %v", jobs, err)
}
statuses, err := s.EpisodeStatuses(ctx, []string{"ep1", "ep2", "absent"})
if err != nil || len(statuses) != 2 || statuses["ep1"] != "queued" {
t.Fatalf("statuses: %+v %v", statuses, err)
}
if _, err := s.ListJobs(ctx, "bad", 10); err == nil {
t.Fatal("不正な状態を許した")
}
if _, _, err := s.Enqueue(ctx, sampleEpisode("bad"), "invalid"); err == nil {
t.Fatal("source CHECK を許した")
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET phase='bad' WHERE id=$1`, first.ID); err == nil {
t.Fatal("phase CHECK を許した")
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET status='bad' WHERE id=$1`, first.ID); err == nil {
t.Fatal("status CHECK を許した")
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET error_kind='bad' WHERE id=$1`, first.ID); err == nil {
t.Fatal("error CHECK を許した")
}
}
func TestTransitions(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode("ep1"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "video"}); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued progress: %v", err)
}
if err := s.Complete(ctx, job.ID, "/file.mp4", 100); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued complete: %v", err)
}
if err := s.Fail(ctx, job.ID, "network", "failed"); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued fail: %v", err)
}
if err := s.Retry(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued retry: %v", err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != job.ID || claimed.Status != "running" || claimed.Attempts != 1 || claimed.HeartbeatAt == nil || claimed.StartedAt == nil {
t.Fatalf("claim: %+v %v", claimed, err)
}
oldHeartbeat := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, job.ID, oldHeartbeat); err != nil {
t.Fatal(err)
}
bytes := int64(123)
total := int64(1000)
speed := 40.5
eta := 12.0
fragment := int32(2)
fragments := int32(10)
format := "video-1"
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "video", DownloadedBytes: &bytes,
TotalBytes: &total, TotalIsEstimate: true, SpeedBPS: &speed, ETASeconds: &eta,
FragmentIndex: &fragment, FragmentCount: &fragments, FormatID: &format}); err != nil {
t.Fatal(err)
}
progressed, err := s.GetJob(ctx, job.ID)
if err != nil || progressed.Phase == nil || *progressed.Phase != "video" || progressed.DownloadedBytes == nil || *progressed.DownloadedBytes != bytes || progressed.TotalBytes == nil || *progressed.TotalBytes != total || !progressed.TotalIsEstimate || progressed.SpeedBPS == nil || *progressed.SpeedBPS != speed || progressed.ETASeconds == nil || *progressed.ETASeconds != eta || progressed.FragmentIndex == nil || *progressed.FragmentIndex != fragment || progressed.FragmentCount == nil || *progressed.FragmentCount != fragments || progressed.FormatID == nil || *progressed.FormatID != format || progressed.HeartbeatAt == nil || !progressed.HeartbeatAt.After(oldHeartbeat) {
t.Fatalf("progress: %+v %v", progressed, err)
}
if err := s.Fail(ctx, job.ID, "network", "connection lost"); err != nil {
t.Fatal(err)
}
failed, err := s.GetJob(ctx, job.ID)
if err != nil || failed.Status != "failed" || failed.ErrorKind == nil || *failed.ErrorKind != "network" || failed.ErrorMessage == nil || *failed.ErrorMessage != "connection lost" || failed.HeartbeatAt != nil || failed.FinishedAt == nil {
t.Fatalf("fail: %+v %v", failed, err)
}
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "audio"}); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("failed progress: %v", err)
}
if err := s.Retry(ctx, job.ID); err != nil {
t.Fatal(err)
}
retried, err := s.GetJob(ctx, job.ID)
if err != nil || retried.Status != "queued" || retried.Phase != nil || retried.ErrorKind != nil || retried.ErrorMessage != nil || retried.DownloadedBytes != nil || retried.TotalBytes != nil || retried.TotalIsEstimate || retried.SpeedBPS != nil || retried.ETASeconds != nil || retried.FragmentIndex != nil || retried.FragmentCount != nil || retried.FormatID != nil || retried.HeartbeatAt != nil || retried.StartedAt != nil || retried.FinishedAt != nil || retried.Attempts != 1 {
t.Fatalf("retry: %+v %v", retried, err)
}
claimed, err = s.ClaimNext(ctx)
if err != nil || claimed.Attempts != 2 {
t.Fatalf("reclaim: %+v %v", claimed, err)
}
if err := s.Complete(ctx, job.ID, "/file.mp4", 100); err != nil {
t.Fatal(err)
}
completed, err := s.GetJob(ctx, job.ID)
if err != nil || completed.Status != "succeeded" || completed.Phase != nil || completed.HeartbeatAt != nil || completed.FinishedAt == nil || completed.FilePath == nil || *completed.FilePath != "/file.mp4" || completed.FileSizeBytes == nil || *completed.FileSizeBytes != 100 {
t.Fatalf("complete: %+v %v", completed, err)
}
if err := s.Cancel(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("succeeded cancel: %v", err)
}
if err := s.Retry(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("succeeded retry: %v", err)
}
if _, err := s.ClaimNext(ctx); !errors.Is(err, pgx.ErrNoRows) {
t.Fatalf("empty queue: %v", err)
}
}
func TestCancelAndStaleBoundary(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
queued, _, err := s.Enqueue(ctx, sampleEpisode("queued"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.Cancel(ctx, queued.ID); err != nil {
t.Fatal(err)
}
canceled, err := s.GetJob(ctx, queued.ID)
if err != nil || canceled.Status != "canceled" || canceled.ErrorKind == nil || *canceled.ErrorKind != "canceled" {
t.Fatalf("cancel: %+v %v", canceled, err)
}
if err := s.Retry(ctx, queued.ID); err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
if err := s.Cancel(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
if err := s.Complete(ctx, claimed.ID, "/late.mp4", 1); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("canceled complete: %v", err)
}
if err := s.Retry(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
claimed, err = s.ClaimNext(ctx)
if err != nil || claimed.Attempts != 2 {
t.Fatalf("cancel retry claim: %+v %v", claimed, err)
}
cutoff := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, claimed.ID, cutoff); err != nil {
t.Fatal(err)
}
fresh, _, err := s.Enqueue(ctx, sampleEpisode("fresh"), "manual")
if err != nil {
t.Fatal(err)
}
if _, err := s.ClaimNext(ctx); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, fresh.ID, cutoff.Add(time.Hour)); err != nil {
t.Fatal(err)
}
n, err := s.RequeueStale(ctx, cutoff)
if err != nil || n != 0 {
t.Fatalf("境界上: %d %v", n, err)
}
n, err = s.RequeueStale(ctx, cutoff.Add(time.Microsecond))
if err != nil || n != 1 {
t.Fatalf("境界後: %d %v", n, err)
}
job, err := s.GetJob(ctx, claimed.ID)
if err != nil || job.Status != "queued" || job.HeartbeatAt != nil || job.Attempts != 2 {
t.Fatalf("requeue: %+v %v", job, err)
}
stillRunning, err := s.GetJob(ctx, fresh.ID)
if err != nil || stillRunning.Status != "running" || stillRunning.HeartbeatAt == nil || !stillRunning.HeartbeatAt.Equal(cutoff.Add(time.Hour)) {
t.Fatalf("新しい heartbeat を持つジョブ: %+v %v", stillRunning, err)
}
}
func TestHeartbeatAndGracefulRequeue(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode("heartbeat"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.Heartbeat(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("待機中の heartbeat: %v", err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
old := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
if _, err := s.pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, claimed.ID, old); err != nil {
t.Fatal(err)
}
if err := s.Heartbeat(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
heartbeated, err := s.GetJob(ctx, claimed.ID)
if err != nil || heartbeated.HeartbeatAt == nil || !heartbeated.HeartbeatAt.After(old) {
t.Fatalf("heartbeat: %+v %v", heartbeated, err)
}
bytes := int64(10)
if err := s.UpdateProgress(ctx, claimed.ID, Progress{Phase: "video", DownloadedBytes: &bytes}); err != nil {
t.Fatal(err)
}
if err := s.RequeueRunning(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
requeued, err := s.GetJob(ctx, claimed.ID)
if err != nil || requeued.Status != "queued" || requeued.StartedAt != nil || requeued.HeartbeatAt != nil || requeued.Phase != nil || requeued.DownloadedBytes != nil || requeued.Attempts != 1 {
t.Fatalf("graceful requeue: %+v %v", requeued, err)
}
if err := s.RequeueRunning(ctx, claimed.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("待機中の再復帰: %v", err)
}
}
func TestRejectedTransitions(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
operations := map[string]func(int64) error{
"progress": func(id int64) error { return s.UpdateProgress(ctx, id, Progress{Phase: "video"}) },
"complete": func(id int64) error { return s.Complete(ctx, id, "/file.mp4", 1) },
"fail": func(id int64) error { return s.Fail(ctx, id, "network", "error") },
"cancel": func(id int64) error { return s.Cancel(ctx, id) },
"retry": func(id int64) error { return s.Retry(ctx, id) },
}
for status, rejected := range map[string][]string{
"queued": {"progress", "complete", "fail", "retry"},
"running": {"retry"},
"succeeded": {"progress", "complete", "fail", "cancel", "retry"},
"failed": {"progress", "complete", "fail", "cancel"},
"canceled": {"progress", "complete", "fail", "cancel"},
} {
t.Run(status, func(t *testing.T) {
job, _, err := s.Enqueue(ctx, sampleEpisode("rejected-"+status), "manual")
if err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET status=$2 WHERE id=$1`, job.ID, status); err != nil {
t.Fatal(err)
}
for _, name := range rejected {
if err := operations[name](job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Errorf("%s on %s: %v", name, status, err)
}
}
got, err := s.GetJob(ctx, job.ID)
if err != nil || got.Status != status {
t.Fatalf("拒否後の状態: %+v %v", got, err)
}
})
}
}
func TestRetryEpisode(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode("retry-by-episode"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.RetryEpisode(ctx, job.EpisodeID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("待機中の再試行: %v", err)
}
if err := s.Cancel(ctx, job.ID); err != nil {
t.Fatal(err)
}
if err := s.RetryEpisode(ctx, job.EpisodeID); err != nil {
t.Fatal(err)
}
got, err := s.GetJob(ctx, job.ID)
if err != nil || got.Status != "queued" || got.ErrorKind != nil || got.FinishedAt != nil {
t.Fatalf("エピソード ID による再試行: %+v %v", got, err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
downloaded, total := int64(10), int64(20)
format := "video-1"
if err := s.UpdateProgress(ctx, claimed.ID, Progress{
Phase: "video", DownloadedBytes: &downloaded, TotalBytes: &total,
TotalIsEstimate: true, FormatID: &format,
}); err != nil {
t.Fatal(err)
}
if err := s.Fail(ctx, claimed.ID, "network", "通信失敗"); err != nil {
t.Fatal(err)
}
if err := s.RetryEpisode(ctx, job.EpisodeID); err != nil {
t.Fatalf("失敗からの再試行: %v", err)
}
got, err = s.GetJob(ctx, job.ID)
if err != nil || got.Status != "queued" || got.Phase != nil || got.DownloadedBytes != nil || got.TotalBytes != nil || got.TotalIsEstimate || got.FormatID != nil || got.ErrorKind != nil || got.ErrorMessage != nil || got.HeartbeatAt != nil || got.StartedAt != nil || got.FinishedAt != nil || got.Attempts != 1 {
t.Fatalf("失敗情報の消去: %+v %v", got, err)
}
if err := s.RetryEpisode(ctx, "unknown"); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("存在しないエピソード: %v", err)
}
}
func TestCancelQueuedDoesNotCancelRunning(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
queued, _, err := s.Enqueue(ctx, sampleEpisode("queue-cancel"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.CancelQueued(ctx, queued.ID); err != nil {
t.Fatal(err)
}
canceled, err := s.GetJob(ctx, queued.ID)
if err != nil || canceled.Status != "canceled" || canceled.ErrorKind == nil || *canceled.ErrorKind != "canceled" || canceled.FinishedAt == nil {
t.Fatalf("待機中の取り消し: %+v %v", canceled, err)
}
if err := s.CancelQueued(ctx, queued.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("再取り消し: %v", err)
}
if err := s.CancelQueued(ctx, -1); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("存在しないジョブの取り消し: %v", err)
}
if err := s.Retry(ctx, queued.ID); err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
if err := s.CancelQueued(ctx, claimed.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("実行中の取り消し: %v", err)
}
job, err := s.GetJob(ctx, claimed.ID)
if err != nil || job.Status != "running" {
t.Fatalf("実行中の状態: %+v %v", job, err)
}
}
func TestClaimNextConcurrent(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
const jobs = 12
for i := range jobs {
if _, _, err := s.Enqueue(ctx, sampleEpisode(fmt.Sprintf("ep%d", i)), "manual"); err != nil {
t.Fatal(err)
}
}
results := make(chan int64, jobs)
errCh := make(chan error, jobs)
var wg sync.WaitGroup
for range jobs {
wg.Add(1)
go func() {
defer wg.Done()
job, err := s.ClaimNext(ctx)
if err != nil {
errCh <- err
return
}
results <- job.ID
}()
}
wg.Wait()
close(results)
close(errCh)
for err := range errCh {
t.Error(err)
}
seen := make(map[int64]bool)
for id := range results {
if seen[id] {
t.Errorf("二重 claim: %d", id)
}
seen[id] = true
}
if len(seen) != jobs {
t.Fatalf("claim 数: %d", len(seen))
}
}
func TestClaimNextSkipsLockedOldest(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
oldest, _, err := s.Enqueue(ctx, sampleEpisode("oldest"), "manual")
if err != nil {
t.Fatal(err)
}
newer, _, err := s.Enqueue(ctx, sampleEpisode("newer"), "manual")
if err != nil {
t.Fatal(err)
}
base := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, oldest.ID, base); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, newer.ID, base.Add(time.Hour)); err != nil {
t.Fatal(err)
}
tx, err := pool.Begin(ctx)
if err != nil {
t.Fatal(err)
}
defer tx.Rollback(ctx)
if _, err := tx.Exec(ctx, `SELECT id FROM download_jobs WHERE id=$1 FOR UPDATE`, oldest.ID); err != nil {
t.Fatal(err)
}
claimCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
claimed, err := s.ClaimNext(claimCtx)
if err != nil || claimed.ID != newer.ID {
t.Fatalf("ロック中の最古ジョブを飛ばせない: %+v %v", claimed, err)
}
if err := tx.Rollback(ctx); err != nil {
t.Fatal(err)
}
claimed, err = s.ClaimNext(ctx)
if err != nil || claimed.ID != oldest.ID {
t.Fatalf("ロック解放後の最古ジョブ: %+v %v", claimed, err)
}
}
func TestListJobsFiltersAndLimit(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
ids := make([]int64, 3)
for i := range ids {
job, _, err := s.Enqueue(ctx, sampleEpisode(fmt.Sprintf("list-%d", i)), "manual")
if err != nil {
t.Fatal(err)
}
ids[i] = job.ID
created := time.Date(2026, 10, 3, i, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, job.ID, created); err != nil {
t.Fatal(err)
}
}
if err := s.Cancel(ctx, ids[1]); err != nil {
t.Fatal(err)
}
all, err := s.ListJobs(ctx, "", 2)
if err != nil || len(all) != 2 || all[0].ID != ids[2] || all[1].ID != ids[1] {
t.Fatalf("全件の新しい順と上限: %+v %v", all, err)
}
queued, err := s.ListJobs(ctx, "queued", 10)
if err != nil || len(queued) != 2 || queued[0].ID != ids[2] || queued[1].ID != ids[0] {
t.Fatalf("状態別一覧: %+v %v", queued, err)
}
canceled, err := s.ListJobs(ctx, "canceled", 10)
if err != nil || len(canceled) != 1 || canceled[0].ID != ids[1] {
t.Fatalf("中断済み一覧: %+v %v", canceled, err)
}
if _, err := s.ListJobs(ctx, "", 0); err == nil {
t.Fatal("0 件の上限を許した")
}
statuses, err := s.EpisodeStatuses(ctx, []string{"list-0", "list-1", "absent"})
if err != nil || len(statuses) != 2 || statuses["list-0"] != "queued" || statuses["list-1"] != "canceled" {
t.Fatalf("状態の一括取得: %+v %v", statuses, err)
}
}
func TestSubscriptions(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
created, err := s.RegisterSubscription(ctx, "series", "番組", nil, false)
if err != nil || !created {
t.Fatalf("add: %v %v", created, err)
}
created, err = s.RegisterSubscription(ctx, "series", "別タイトル", nil, false)
if err != nil || created {
t.Fatalf("duplicate: %v %v", created, err)
}
when := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
if err := s.SetSubscriptionChecked(ctx, "series", when); err != nil {
t.Fatal(err)
}
subs, err := s.ListSubscriptions(ctx)
if err != nil || len(subs) != 1 || subs[0].SeriesTitle != "番組" || !subs[0].Enabled || subs[0].LastCheckedAt == nil || !subs[0].LastCheckedAt.Equal(when) {
t.Fatalf("list: %+v %v", subs, err)
}
deleted, err := s.DeleteSubscription(ctx, "series")
if err != nil || !deleted {
t.Fatalf("delete: %v %v", deleted, err)
}
deleted, err = s.DeleteSubscription(ctx, "series")
if err != nil || deleted {
t.Fatalf("duplicate delete: %v %v", deleted, err)
}
if err := s.SetSubscriptionChecked(ctx, "series", when); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("missing: %v", err)
}
}
+244
View File
@@ -0,0 +1,244 @@
package store
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
var ErrSubscriptionPaused = errors.New("subscription is paused")
// RegisterSubscription は登録と公開中の回の記録・任意の投入を一つの取引で行う。
func (s *Store) RegisterSubscription(ctx context.Context, seriesID, title string, episodes []Episode, downloadExisting bool) (bool, error) {
if seriesID == "" || title == "" {
return false, errors.New("series id and title are required")
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return false, err
}
defer tx.Rollback(ctx)
var inserted string
err = tx.QueryRow(ctx, `INSERT INTO subscriptions (series_id, series_title, last_checked_at)
VALUES ($1,$2,now()) ON CONFLICT (series_id) DO NOTHING RETURNING series_id`, seriesID, title).Scan(&inserted)
if errors.Is(err, pgx.ErrNoRows) {
return false, nil
}
if err != nil {
return false, err
}
if _, err := addKnownAndMaybeQueue(ctx, tx, seriesID, episodes, downloadExisting); err != nil {
return false, err
}
if err := tx.Commit(ctx); err != nil {
return false, err
}
return true, nil
}
// QueueUnseenEpisodes は未確認の回だけを既知に記録し、同じ取引で待ち行列へ入れる。
func (s *Store) QueueUnseenEpisodes(ctx context.Context, seriesID string, episodes []Episode) (int, error) {
tx, err := s.pool.Begin(ctx)
if err != nil {
return 0, err
}
defer tx.Rollback(ctx)
var enabled bool
err = tx.QueryRow(ctx, `SELECT enabled FROM subscriptions WHERE series_id=$1 FOR UPDATE`, seriesID).Scan(&enabled)
if errors.Is(err, pgx.ErrNoRows) {
return 0, ErrInvalidTransition
}
if err != nil {
return 0, err
}
if !enabled {
return 0, ErrSubscriptionPaused
}
count, err := addKnownAndMaybeQueue(ctx, tx, seriesID, episodes, true)
if err != nil {
return 0, err
}
if err := tx.Commit(ctx); err != nil {
return 0, err
}
return count, nil
}
// ListDueSubscriptions は有効で、cutoff 以前に確認された(または未確認の)番組を返す。
func (s *Store) ListDueSubscriptions(ctx context.Context, cutoff time.Time) ([]Subscription, error) {
rows, err := s.pool.Query(ctx, `SELECT series_id, series_title, enabled, created_at, last_checked_at,
last_check_error_at, last_check_error, 0::bigint
FROM subscriptions
WHERE enabled AND (last_checked_at IS NULL OR last_checked_at <= $1)
ORDER BY last_checked_at ASC NULLS FIRST, created_at, series_id`, cutoff)
if err != nil {
return nil, err
}
defer rows.Close()
var result []Subscription
for rows.Next() {
var sub Subscription
if err := rows.Scan(&sub.SeriesID, &sub.SeriesTitle, &sub.Enabled, &sub.CreatedAt,
&sub.LastCheckedAt, &sub.LastCheckErrorAt, &sub.LastCheckError, &sub.JobCount); err != nil {
return nil, err
}
result = append(result, sub)
}
return result, rows.Err()
}
// CompleteSubscriptionCheck は全公開回を既知にし、しきい値を超える未確認回だけを投入する。
// 最終確認日時の更新と前回エラーの消去も同じ取引で行う。
func (s *Store) CompleteSubscriptionCheck(ctx context.Context, seriesID string, episodes []Episode, previewMaxSeconds int, checkedAt time.Time) (int, error) {
if previewMaxSeconds < 0 {
return 0, errors.New("preview threshold must not be negative")
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return 0, err
}
defer tx.Rollback(ctx)
var enabled bool
err = tx.QueryRow(ctx, `SELECT enabled FROM subscriptions WHERE series_id=$1 FOR UPDATE`, seriesID).Scan(&enabled)
if errors.Is(err, pgx.ErrNoRows) {
return 0, ErrInvalidTransition
}
if err != nil {
return 0, err
}
if !enabled {
return 0, ErrSubscriptionPaused
}
queued := 0
for _, ep := range episodes {
if ep.ID == "" || ep.SeriesID != seriesID || ep.DurationSeconds < 0 {
return 0, fmt.Errorf("invalid episode identity for series %s", seriesID)
}
var inserted string
err := tx.QueryRow(ctx, `INSERT INTO known_episodes (series_id, episode_id)
VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING episode_id`, seriesID, ep.ID).Scan(&inserted)
if errors.Is(err, pgx.ErrNoRows) {
continue
}
if err != nil {
return 0, err
}
if ep.DurationSeconds <= previewMaxSeconds {
continue
}
tag, err := tx.Exec(ctx, `INSERT INTO download_jobs
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,'subscription')
ON CONFLICT (episode_id) DO NOTHING`, ep.ID, seriesID, ep.SeriesTitle, ep.Title,
ep.BroadcasterName, ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath)
if err != nil {
return 0, err
}
queued += int(tag.RowsAffected())
}
if _, err := tx.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2,
last_check_error_at=NULL, last_check_error=NULL WHERE series_id=$1`, seriesID, checkedAt); err != nil {
return 0, err
}
if err := tx.Commit(ctx); err != nil {
return 0, err
}
return queued, nil
}
// RecordSubscriptionCheckFailure は取得失敗を記録し、最終確認日時は変更しない。
func (s *Store) RecordSubscriptionCheckFailure(ctx context.Context, seriesID string, failedAt time.Time, message string) error {
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET last_check_error_at=$2, last_check_error=$3
WHERE series_id=$1 AND enabled`, seriesID, failedAt, message)
return transitionResult(tag.RowsAffected(), err)
}
func addKnownAndMaybeQueue(ctx context.Context, tx pgx.Tx, seriesID string, episodes []Episode, queue bool) (int, error) {
count := 0
for _, ep := range episodes {
if ep.ID == "" || ep.SeriesID != seriesID {
return 0, fmt.Errorf("invalid episode identity for series %s", seriesID)
}
var inserted string
err := tx.QueryRow(ctx, `INSERT INTO known_episodes (series_id, episode_id)
VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING episode_id`, seriesID, ep.ID).Scan(&inserted)
if errors.Is(err, pgx.ErrNoRows) {
continue
}
if err != nil {
return 0, err
}
count++
if queue {
_, err := tx.Exec(ctx, `INSERT INTO download_jobs
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,'subscription')
ON CONFLICT (episode_id) DO NOTHING`, ep.ID, seriesID, ep.SeriesTitle, ep.Title,
ep.BroadcasterName, ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath)
if err != nil {
return 0, err
}
}
}
return count, nil
}
// UnknownEpisodeIDs は指定シリーズの未確認のエピソード ID を入力順で返す。
func (s *Store) UnknownEpisodeIDs(ctx context.Context, seriesID string, ids []string) ([]string, error) {
if len(ids) == 0 {
return []string{}, nil
}
rows, err := s.pool.Query(ctx, `SELECT episode_id FROM known_episodes WHERE series_id=$1 AND episode_id=ANY($2)`, seriesID, ids)
if err != nil {
return nil, err
}
defer rows.Close()
known := make(map[string]bool)
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
known[id] = true
}
if err := rows.Err(); err != nil {
return nil, err
}
var unknown []string
emitted := make(map[string]bool)
for _, id := range ids {
if !known[id] && !emitted[id] {
unknown = append(unknown, id)
emitted[id] = true
}
}
return unknown, nil
}
func (s *Store) SetSubscriptionEnabled(ctx context.Context, seriesID string, enabled bool) error {
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET enabled=$2 WHERE series_id=$1`, seriesID, enabled)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) SubscribedSeries(ctx context.Context, ids []string) (map[string]bool, error) {
result := make(map[string]bool)
if len(ids) == 0 {
return result, nil
}
rows, err := s.pool.Query(ctx, `SELECT series_id FROM subscriptions WHERE series_id=ANY($1)`, ids)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
result[id] = true
}
return result, rows.Err()
}
+326
View File
@@ -0,0 +1,326 @@
package store
import (
"context"
"errors"
"sync"
"testing"
"time"
)
func TestListDueSubscriptionsBoundaryAndPaused(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
for _, id := range []string{"null", "old", "boundary", "fresh", "paused"} {
if created, err := s.RegisterSubscription(ctx, id, "番組 "+id, nil, false); err != nil || !created {
t.Fatalf("登録 %s: %t %v", id, created, err)
}
}
updates := map[string]*time.Time{
"null": nil, "old": timePointer(now.Add(-25 * time.Hour)),
"boundary": timePointer(now.Add(-24 * time.Hour)), "fresh": timePointer(now.Add(-24*time.Hour + time.Microsecond)),
"paused": timePointer(now.Add(-48 * time.Hour)),
}
for id, checked := range updates {
if _, err := pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, id, checked); err != nil {
t.Fatal(err)
}
}
if err := s.SetSubscriptionEnabled(ctx, "paused", false); err != nil {
t.Fatal(err)
}
due, err := s.ListDueSubscriptions(ctx, now.Add(-24*time.Hour))
if err != nil {
t.Fatal(err)
}
if len(due) != 3 || due[0].SeriesID != "null" || due[1].SeriesID != "old" || due[2].SeriesID != "boundary" {
t.Fatalf("24時間境界と一時停止: %+v", due)
}
}
func TestCompleteSubscriptionCheckKnownPreviewErrorAndAtomicity(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
if created, err := s.RegisterSubscription(ctx, "series-check", "番組", []Episode{seriesEpisode("known", "series-check")}, false); err != nil || !created {
t.Fatalf("登録: %t %v", created, err)
}
before := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
failedAt := time.Date(2026, 10, 3, 1, 2, 3, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, "series-check", before); err != nil {
t.Fatal(err)
}
if err := s.RecordSubscriptionCheckFailure(ctx, "series-check", failedAt, "network token=[REDACTED]"); err != nil {
t.Fatal(err)
}
var unchanged time.Time
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id='series-check'`).Scan(&unchanged); err != nil || !unchanged.Equal(before) {
t.Fatalf("失敗時に最終確認を更新: %v %v", unchanged, err)
}
checkedAt := time.Date(2026, 10, 3, 4, 5, 6, 0, time.UTC)
preview := seriesEpisode("preview", "series-check")
preview.DurationSeconds = 180
normal := seriesEpisode("normal", "series-check")
normal.DurationSeconds = 181
count, err := s.CompleteSubscriptionCheck(ctx, "series-check", []Episode{
seriesEpisode("known", "series-check"), preview, normal,
}, 180, checkedAt)
if err != nil || count != 1 {
t.Fatalf("確認完了: count=%d err=%v", count, err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "series-check", []string{"known", "preview", "normal"})
if err != nil || len(unknown) != 0 {
t.Fatalf("予告を含む既知化: %v %v", unknown, err)
}
var jobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-check'`).Scan(&jobs); err != nil || jobs != 1 {
t.Fatalf("予告の投入除外: %d %v", jobs, err)
}
var last time.Time
var errorAt, errorMessage any
if err := pool.QueryRow(ctx, `SELECT last_checked_at, last_check_error_at, last_check_error FROM subscriptions WHERE series_id='series-check'`).Scan(&last, &errorAt, &errorMessage); err != nil || !last.Equal(checkedAt) || errorAt != nil || errorMessage != nil {
t.Fatalf("成功時の日時・エラー消去: %v %v %v %v", last, errorAt, errorMessage, err)
}
bad := seriesEpisode("bad", "series-check")
bad.DurationSeconds = -1
if _, err := s.CompleteSubscriptionCheck(ctx, "series-check", []Episode{seriesEpisode("rollback", "series-check"), bad}, 180, checkedAt.Add(time.Hour)); err == nil {
t.Fatal("不正な回を含む確認が成功した")
}
unknown, err = s.UnknownEpisodeIDs(ctx, "series-check", []string{"rollback"})
if err != nil || len(unknown) != 1 {
t.Fatalf("失敗時の既知記録を巻き戻さない: %v %v", unknown, err)
}
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id='series-check'`).Scan(&last); err != nil || !last.Equal(checkedAt) {
t.Fatalf("失敗時の最終確認を巻き戻さない: %v %v", last, err)
}
if err := s.SetSubscriptionEnabled(ctx, "series-check", false); err != nil {
t.Fatal(err)
}
if _, err := s.CompleteSubscriptionCheck(ctx, "series-check", nil, 180, checkedAt.Add(time.Hour)); !errors.Is(err, ErrSubscriptionPaused) {
t.Fatalf("一時停止中の確認完了: %v", err)
}
if err := s.RecordSubscriptionCheckFailure(ctx, "series-check", failedAt, "ignored"); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("一時停止中のエラー記録: %v", err)
}
}
func TestCompleteSubscriptionCheckKeepsExistingManualJob(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
if created, err := s.RegisterSubscription(ctx, "series-manual", "番組", nil, false); err != nil || !created {
t.Fatalf("登録: %t %v", created, err)
}
episode := seriesEpisode("manual-existing", "series-manual")
manual, _, err := s.Enqueue(ctx, episode, "manual")
if err != nil {
t.Fatal(err)
}
checkedAt := time.Date(2026, 10, 3, 4, 5, 6, 0, time.UTC)
queued, err := s.CompleteSubscriptionCheck(ctx, "series-manual", []Episode{episode, episode}, 180, checkedAt)
if err != nil || queued != 0 {
t.Fatalf("既存手動ジョブの確認: queued=%d err=%v", queued, err)
}
got, err := s.GetJob(ctx, manual.ID)
if err != nil || got.Source != "manual" {
t.Fatalf("既存手動ジョブを変更した: %+v %v", got, err)
}
var jobs, known int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE episode_id=$1`, episode.ID).Scan(&jobs); err != nil || jobs != 1 {
t.Fatalf("ジョブが二重化した: %d %v", jobs, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id=$1 AND episode_id=$2`, episode.SeriesID, episode.ID).Scan(&known); err != nil || known != 1 {
t.Fatalf("既知記録: %d %v", known, err)
}
var last time.Time
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id=$1`, episode.SeriesID).Scan(&last); err != nil || !last.Equal(checkedAt) {
t.Fatalf("最終確認日時: %v %v", last, err)
}
}
func timePointer(value time.Time) *time.Time { return &value }
func seriesEpisode(id, seriesID string) Episode {
ep := sampleEpisode(id)
ep.SeriesID = seriesID
return ep
}
func TestRegisterSubscriptionSnapshotAndRollback(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
episodes := []Episode{seriesEpisode("ep1", "series-1"), seriesEpisode("ep2", "series-1")}
created, err := s.RegisterSubscription(ctx, "series-1", "番組", episodes, false)
if err != nil || !created {
t.Fatalf("既存回をダウンロードしない登録: %t %v", created, err)
}
subs, err := s.ListSubscriptions(ctx)
if err != nil || len(subs) != 1 || subs[0].LastCheckedAt == nil || subs[0].LastCheckedAt.Before(subs[0].CreatedAt) {
t.Fatalf("登録時の最終確認日時: %+v %v", subs, err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "series-1", []string{"ep1", "ep2", "ep3"})
if err != nil || len(unknown) != 1 || unknown[0] != "ep3" {
t.Fatalf("既知の記録: %v %v", unknown, err)
}
var jobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
t.Fatalf("既存回が投入された: %d %v", jobs, err)
}
created, err = s.RegisterSubscription(ctx, "series-1", "別の名前", episodes, true)
if err != nil || created {
t.Fatalf("重複登録: %t %v", created, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
t.Fatalf("重複登録が投入した: %d %v", jobs, err)
}
bad := seriesEpisode("bad", "series-bad")
bad.DurationSeconds = -1
created, err = s.RegisterSubscription(ctx, "series-bad", "番組", []Episode{seriesEpisode("good", "series-bad"), bad}, true)
if err == nil || created {
t.Fatalf("壊れた回の登録が成功: %t %v", created, err)
}
for _, table := range []string{"subscriptions", "known_episodes", "download_jobs"} {
var count int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM `+table+` WHERE series_id='series-bad'`).Scan(&count); err != nil || count != 0 {
t.Errorf("取引の巻き戻し %s: %d %v", table, count, err)
}
}
deleted, err := s.DeleteSubscription(ctx, "series-1")
if err != nil || !deleted {
t.Fatalf("解除: %t %v", deleted, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
t.Fatalf("解除後に既知が残った: %d %v", jobs, err)
}
created, err = s.RegisterSubscription(ctx, "series-1", "番組", episodes, true)
if err != nil || !created {
t.Fatalf("解除後の再登録: %t %v", created, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 2 {
t.Fatalf("再登録で既存回を選び直せない: %d %v", jobs, err)
}
}
func TestRegisterAndQueueUnseenEpisodes(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
manual, _, err := s.Enqueue(ctx, seriesEpisode("manual", "series-2"), "manual")
if err != nil {
t.Fatal(err)
}
created, err := s.RegisterSubscription(ctx, "series-2", "番組", []Episode{seriesEpisode("manual", "series-2"), seriesEpisode("past", "series-2")}, true)
if err != nil || !created {
t.Fatalf("公開中の回を投入: %t %v", created, err)
}
manualAfter, err := s.GetJob(ctx, manual.ID)
if err != nil || manualAfter.Source != "manual" {
t.Fatalf("既存の手動ジョブを変更: %+v %v", manualAfter, err)
}
var subscriptionJobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-2' AND source='subscription' AND status='queued'`).Scan(&subscriptionJobs); err != nil || subscriptionJobs != 1 {
t.Fatalf("登録時に作ったジョブ: %d %v", subscriptionJobs, err)
}
subs, err := s.ListSubscriptions(ctx)
if err != nil || len(subs) != 1 || subs[0].JobCount != 2 || !subs[0].Enabled || subs[0].LastCheckedAt == nil {
t.Fatalf("登録一覧と関連ジョブ数: %+v %v", subs, err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "series-2", []string{"past", "new", "manual", "new"})
if err != nil || len(unknown) != 1 || unknown[0] != "new" {
t.Fatalf("未確認の抽出: %v %v", unknown, err)
}
if err := s.SetSubscriptionEnabled(ctx, "series-2", false); err != nil {
t.Fatal(err)
}
if _, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("new", "series-2")}); !errors.Is(err, ErrSubscriptionPaused) {
t.Fatalf("一時停止中の投入: %v", err)
}
unknown, err = s.UnknownEpisodeIDs(ctx, "series-2", []string{"new"})
if err != nil || len(unknown) != 1 {
t.Fatalf("一時停止中に既知となった: %v %v", unknown, err)
}
if err := s.SetSubscriptionEnabled(ctx, "series-2", true); err != nil {
t.Fatal(err)
}
invalid := seriesEpisode("invalid", "series-2")
invalid.DurationSeconds = -1
if _, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("newer", "series-2"), invalid}); err == nil {
t.Fatal("新規回の投入が途中失敗でも確定した")
}
unknown, err = s.UnknownEpisodeIDs(ctx, "series-2", []string{"newer"})
if err != nil || len(unknown) != 1 {
t.Fatalf("新規回の投入失敗が巻き戻らない: %v %v", unknown, err)
}
count, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("past", "series-2"), seriesEpisode("new", "series-2")})
if err != nil || count != 1 {
t.Fatalf("再開後の新しい回: %d %v", count, err)
}
count, err = s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("new", "series-2")})
if err != nil || count != 0 {
t.Fatalf("重複した新しい回: %d %v", count, err)
}
var source string
if err := pool.QueryRow(ctx, `SELECT source FROM download_jobs WHERE episode_id='new'`).Scan(&source); err != nil || source != "subscription" {
t.Fatalf("新しい回の登録元: %q %v", source, err)
}
statuses, err := s.SubscribedSeries(ctx, []string{"series-2", "missing"})
if err != nil || !statuses["series-2"] || statuses["missing"] {
t.Fatalf("登録済みシリーズの一括取得: %v %v", statuses, err)
}
}
func TestQueueUnseenEpisodesConcurrent(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
created, err := s.RegisterSubscription(ctx, "series-concurrent", "番組", nil, false)
if err != nil || !created {
t.Fatalf("登録: %t %v", created, err)
}
episodes := []Episode{
seriesEpisode("concurrent-1", "series-concurrent"),
seriesEpisode("concurrent-2", "series-concurrent"),
}
const workers = 8
counts := make(chan int, workers)
errs := make(chan error, workers)
var wg sync.WaitGroup
for range workers {
wg.Add(1)
go func() {
defer wg.Done()
count, err := s.QueueUnseenEpisodes(ctx, "series-concurrent", episodes)
counts <- count
errs <- err
}()
}
wg.Wait()
close(counts)
close(errs)
for err := range errs {
if err != nil {
t.Errorf("同時投入: %v", err)
}
}
total := 0
positive := 0
for count := range counts {
total += count
if count > 0 {
positive++
}
}
if total != 2 || positive != 1 {
t.Fatalf("同時投入の記録件数: total=%d positive workers=%d", total, positive)
}
for table := range map[string]struct{}{"known_episodes": {}, "download_jobs": {}} {
var count int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM `+table+` WHERE series_id='series-concurrent'`).Scan(&count); err != nil || count != 2 {
t.Errorf("%s の重複防止: %d %v", table, count, err)
}
}
}
+171
View File
@@ -0,0 +1,171 @@
package tver
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
)
const (
DefaultBaseURL = "https://platform-api.tver.jp"
PreviewMaxSeconds = 180
)
type Session struct {
PlatformUID string `json:"platform_uid"`
PlatformToken string `json:"platform_token"`
}
type Episode struct {
ID string `json:"id"`
SeriesID string `json:"seriesID"`
SeriesTitle string `json:"seriesTitle"`
Title string `json:"title"`
BroadcasterName string `json:"broadcasterName"`
Duration int64 `json:"duration"`
EndAt int64 `json:"endAt"`
ThumbnailPath string `json:"thumbnailPath"`
BroadcastDateLabel string `json:"broadcastDateLabel"`
}
type Client struct {
baseURL *url.URL
serviceURL *url.URL
httpClient *http.Client
}
type httpStatusError struct {
statusCode int
}
func (e *httpStatusError) Error() string {
return fmt.Sprintf("HTTP %d", e.statusCode)
}
// NewClient はテスト用に API の接続先と HTTP クライアントを差し替えられる。
// 接続先が空、またはクライアントが nil の場合は本番用の既定値を使う。
func NewClient(baseURL string, httpClient *http.Client) (*Client, error) {
defaultAPI := baseURL == ""
if baseURL == "" {
baseURL = DefaultBaseURL
}
parsed, err := url.Parse(baseURL)
if err != nil || (parsed.Scheme != "https" && parsed.Scheme != "http") || parsed.Host == "" || parsed.RawQuery != "" || parsed.Fragment != "" {
return nil, fmt.Errorf("invalid TVer API base URL")
}
if httpClient == nil {
httpClient = &http.Client{Timeout: 15 * time.Second}
}
service := parsed
if defaultAPI {
service = &url.URL{Scheme: "https", Host: "service-api.tver.jp"}
}
return &Client{baseURL: parsed, serviceURL: service, httpClient: httpClient}, nil
}
func (c *Client) CreateSession(ctx context.Context) (Session, error) {
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/v2/api/platform_users/browser/create"})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint.String(), strings.NewReader("device_type=pc"))
if err != nil {
return Session{}, fmt.Errorf("create session request: %w", err)
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
var response struct {
Code int `json:"code"`
Message string `json:"message"`
Result Session `json:"result"`
}
if err := c.getJSON(req, &response); err != nil {
return Session{}, fmt.Errorf("create session: %w", err)
}
if response.Code != 0 || response.Result.PlatformUID == "" || response.Result.PlatformToken == "" {
return Session{}, fmt.Errorf("create session: invalid API result (code %d)", response.Code)
}
return response.Result, nil
}
func (c *Client) ListNewer(ctx context.Context, category string, session Session) ([]Episode, error) {
if category != "drama" && category != "variety" {
return nil, fmt.Errorf("unsupported category %q", category)
}
if session.PlatformUID == "" || session.PlatformToken == "" {
return nil, errors.New("missing TVer session credentials")
}
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v1/callNewerDetail/" + category})
query := endpoint.Query()
query.Set("platform_uid", session.PlatformUID)
query.Set("platform_token", session.PlatformToken)
endpoint.RawQuery = query.Encode()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
if err != nil {
return nil, fmt.Errorf("newer list request: %w", err)
}
req.Header.Set("x-tver-platform-type", "web")
req.Header.Set("Origin", "https://tver.jp")
req.Header.Set("Referer", "https://tver.jp/")
var response struct {
Code int `json:"code"`
Result struct {
Contents *struct {
Contents []struct {
Type string `json:"type"`
Content Episode `json:"content"`
} `json:"contents"`
} `json:"contents"`
} `json:"result"`
}
if err := c.getJSON(req, &response); err != nil {
return nil, fmt.Errorf("newer list: %w", err)
}
if response.Code != 0 {
return nil, fmt.Errorf("newer list: API code %d", response.Code)
}
if response.Result.Contents == nil || response.Result.Contents.Contents == nil {
return nil, errors.New("newer list: missing contents")
}
var episodes []Episode
for _, item := range response.Result.Contents.Contents {
if item.Type != "episode" {
continue
}
if item.Content.ID == "" {
return nil, errors.New("newer list: episode without id")
}
episodes = append(episodes, item.Content)
}
return episodes, nil
}
func (c *Client) FetchNewer(ctx context.Context, category string) ([]Episode, error) {
session, err := c.CreateSession(ctx)
if err != nil {
return nil, err
}
return c.ListNewer(ctx, category, session)
}
func (c *Client) getJSON(req *http.Request, dst any) error {
response, err := c.httpClient.Do(req)
if err != nil {
// url.Error には platform_token を含むリクエスト URL が入るため、内部のエラーだけを返す。
var urlErr *url.Error
if errors.As(err, &urlErr) {
err = urlErr.Err
}
return fmt.Errorf("request failed: %w", err)
}
defer response.Body.Close()
if response.StatusCode < 200 || response.StatusCode >= 300 {
return &httpStatusError{statusCode: response.StatusCode}
}
if err := json.NewDecoder(io.LimitReader(response.Body, 8<<20)).Decode(dst); err != nil {
return fmt.Errorf("decode JSON: %w", err)
}
return nil
}
+213
View File
@@ -0,0 +1,213 @@
package tver
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"os"
"strings"
"testing"
)
type localOnlyTransport struct {
host string
base http.RoundTripper
}
func (tr localOnlyTransport) RoundTrip(req *http.Request) (*http.Response, error) {
if req.URL.Scheme != "http" || req.URL.Host != tr.host {
return nil, fmt.Errorf("test attempted external request to %s", req.URL.Host)
}
return tr.base.RoundTrip(req)
}
func testClient(t *testing.T, handler http.HandlerFunc) *Client {
t.Helper()
server := httptest.NewServer(handler)
t.Cleanup(server.Close)
serverURL, err := url.Parse(server.URL)
if err != nil {
t.Fatal(err)
}
httpClient := server.Client()
baseTransport := httpClient.Transport
if baseTransport == nil {
baseTransport = http.DefaultTransport
}
httpClient.Transport = localOnlyTransport{host: serverURL.Host, base: baseTransport}
client, err := NewClient(server.URL, httpClient)
if err != nil {
t.Fatal(err)
}
return client
}
func TestTestClientRejectsExternalRequests(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
t.Errorf("unexpected local request to %s", r.URL)
})
_, err := client.httpClient.Get(DefaultBaseURL)
if err == nil || !strings.Contains(err.Error(), "test attempted external request") {
t.Errorf("external request error = %v", err)
}
}
func TestFetchNewerFromRecordedResponse(t *testing.T) {
fixture, err := os.ReadFile("testdata/newer_drama.json")
if err != nil {
t.Fatal(err)
}
var sessionCalls, newerCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v2/api/platform_users/browser/create":
sessionCalls++
if r.Method != http.MethodPost || r.Header.Get("Content-Type") != "application/x-www-form-urlencoded" {
t.Errorf("session request method/Content-Type: %s %s", r.Method, r.Header.Get("Content-Type"))
}
if err := r.ParseForm(); err != nil || r.PostForm.Get("device_type") != "pc" {
t.Errorf("session form: %v, %v", r.PostForm, err)
}
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
case "/service/api/v1/callNewerDetail/drama":
newerCalls++
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" {
t.Errorf("newer request method/query: %s %v", r.Method, r.URL.Query())
}
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
if got := r.Header.Get(name); got != want {
t.Errorf("header %s = %q; want %q", name, got, want)
}
}
_, _ = w.Write(fixture)
default:
t.Errorf("unexpected path %s", r.URL.Path)
http.NotFound(w, r)
}
})
episodes, err := client.FetchNewer(context.Background(), "drama")
if err != nil {
t.Fatal(err)
}
if sessionCalls != 1 || newerCalls != 1 {
t.Fatalf("calls: session=%d newer=%d", sessionCalls, newerCalls)
}
if len(episodes) != 3 {
t.Fatalf("episodes = %d; want 3", len(episodes))
}
want := Episode{
ID: "eps0z5761a",
SeriesID: "sr0lhdrtdb",
SeriesTitle: "韓国ドラマ「良いが悪い、ドンジェ」",
Title: "#10(最終話)",
BroadcasterName: "BS12トゥエルビ",
Duration: 2712,
EndAt: 1792209540,
ThumbnailPath: "/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51",
BroadcastDateLabel: "10月3日(土)放送分",
}
if episodes[0] != want {
t.Errorf("first episode = %+v; want %+v", episodes[0], want)
}
if episodes[2].Duration != 60 || episodes[2].BroadcastDateLabel != "" {
t.Errorf("third episode = %+v", episodes[2])
}
}
func TestListNewerVarietySkipsOtherTypes(t *testing.T) {
var calls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
calls++
if r.URL.Path != "/service/api/v1/callNewerDetail/variety" {
t.Errorf("path = %s", r.URL.Path)
}
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "u" || r.URL.Query().Get("platform_token") != "t" {
t.Errorf("variety request method/query: %s %v", r.Method, r.URL.Query())
}
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
if got := r.Header.Get(name); got != want {
t.Errorf("variety header %s = %q; want %q", name, got, want)
}
}
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":{"contents":[{"type":"series","content":{"id":"series-1"}},{"type":"episode","content":{"id":"episode-1","duration":42}}]}}}`))
})
episodes, err := client.ListNewer(context.Background(), "variety", Session{PlatformUID: "u", PlatformToken: "t"})
if err != nil {
t.Fatal(err)
}
if len(episodes) != 1 || episodes[0].ID != "episode-1" || episodes[0].Duration != 42 {
t.Errorf("episodes = %+v", episodes)
}
if calls != 1 {
t.Errorf("variety calls = %d; want 1", calls)
}
}
func TestFetchNewerErrors(t *testing.T) {
tests := []struct {
name string
sessionStatus int
sessionBody string
newerStatus int
newerBody string
want string
}{
{"session HTTP error", 503, "", 200, "", "create session: HTTP 503"},
{"session invalid JSON", 200, "{", 200, "", "create session: decode JSON"},
{"session API error", 200, `{"code":7,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, "", "create session: invalid API result"},
{"session missing credentials", 200, `{"code":0,"result":{}}`, 200, "", "create session: invalid API result"},
{"session missing token", 200, `{"code":0,"result":{"platform_uid":"u"}}`, 200, "", "create session: invalid API result"},
{"session missing uid", 200, `{"code":0,"result":{"platform_token":"t"}}`, 200, "", "create session: invalid API result"},
{"newer HTTP error", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 502, "", "newer list: HTTP 502"},
{"newer invalid JSON", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, "{", "newer list: decode JSON"},
{"newer API error", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":7}`, "newer list: API code 7"},
{"newer missing contents", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{}}`, "newer list: missing contents"},
{"newer missing items", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{"contents":{}}}`, "newer list: missing contents"},
{"episode missing id", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{"contents":{"contents":[{"type":"episode","content":{"title":"untitled"}}]}}}`, "newer list: episode without id"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var sessionCalls, newerCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v2/api/platform_users/browser/create":
sessionCalls++
w.WriteHeader(tt.sessionStatus)
_, _ = w.Write([]byte(tt.sessionBody))
case "/service/api/v1/callNewerDetail/drama":
newerCalls++
w.WriteHeader(tt.newerStatus)
_, _ = w.Write([]byte(tt.newerBody))
default:
t.Errorf("unexpected path %s", r.URL.Path)
http.NotFound(w, r)
}
})
_, err := client.FetchNewer(context.Background(), "drama")
if err == nil || !strings.Contains(err.Error(), tt.want) {
t.Errorf("error = %v; want %q", err, tt.want)
}
wantNewerCalls := 1
if strings.HasPrefix(tt.name, "session ") {
wantNewerCalls = 0
}
if sessionCalls != 1 || newerCalls != wantNewerCalls {
t.Errorf("calls: session=%d newer=%d; want session=1 newer=%d", sessionCalls, newerCalls, wantNewerCalls)
}
})
}
}
func TestListNewerRejectsInvalidInput(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
t.Errorf("unexpected request to %s", r.URL)
})
if _, err := client.ListNewer(context.Background(), "news", Session{PlatformUID: "u", PlatformToken: "t"}); err == nil {
t.Error("unsupported category accepted")
}
if _, err := client.ListNewer(context.Background(), "drama", Session{}); err == nil {
t.Error("empty session accepted")
}
}
+143
View File
@@ -0,0 +1,143 @@
package tver
import (
"context"
"errors"
"fmt"
"net/http"
"net/url"
"regexp"
)
var tverSeriesIDPattern = regexp.MustCompile(`^[A-Za-z0-9]+$`)
// ErrSeriesNotFound は callSeries が HTTP 404 を返したことを表す。
var ErrSeriesNotFound = errors.New("series not found")
type seriesContents struct {
Code int `json:"code"`
Result struct {
Contents []struct {
Type string `json:"type"`
Content Episode `json:"content"`
} `json:"contents"`
} `json:"result"`
}
func (c *Client) FetchSeries(ctx context.Context, seriesID string) (string, error) {
if !tverSeriesIDPattern.MatchString(seriesID) {
return "", errors.New("invalid series id")
}
session, err := c.CreateSession(ctx)
if err != nil {
return "", err
}
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v2/callSeries/" + seriesID})
query := endpoint.Query()
query.Set("platform_uid", session.PlatformUID)
query.Set("platform_token", session.PlatformToken)
endpoint.RawQuery = query.Encode()
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
if err != nil {
return "", fmt.Errorf("series request: %w", err)
}
setTVerHeaders(request)
var response struct {
Code int `json:"code"`
Result struct {
Content struct {
Content struct {
Title string `json:"title"`
} `json:"content"`
} `json:"content"`
} `json:"result"`
}
if err := c.getJSON(request, &response); err != nil {
var statusErr *httpStatusError
if errors.As(err, &statusErr) && statusErr.statusCode == http.StatusNotFound {
return "", ErrSeriesNotFound
}
return "", fmt.Errorf("series: %w", err)
}
if response.Code != 0 || response.Result.Content.Content.Title == "" {
return "", fmt.Errorf("series: invalid API result (code %d)", response.Code)
}
return response.Result.Content.Content.Title, nil
}
func (c *Client) FetchSeriesEpisodes(ctx context.Context, seriesID string) ([]Episode, error) {
if !tverSeriesIDPattern.MatchString(seriesID) {
return nil, errors.New("invalid series id")
}
seasonsURL := c.serviceURL.ResolveReference(&url.URL{Path: "/api/v1/callSeriesSeasons/" + seriesID})
request, err := http.NewRequestWithContext(ctx, http.MethodGet, seasonsURL.String(), nil)
if err != nil {
return nil, fmt.Errorf("series seasons request: %w", err)
}
setTVerHeaders(request)
var seasons seriesContents
if err := c.getJSON(request, &seasons); err != nil {
return nil, fmt.Errorf("series seasons: %w", err)
}
if seasons.Code != 0 || seasons.Result.Contents == nil {
return nil, fmt.Errorf("series seasons: invalid API result (code %d)", seasons.Code)
}
var seasonIDs []string
for _, item := range seasons.Result.Contents {
if item.Type != "season" {
continue
}
if !tverSeriesIDPattern.MatchString(item.Content.ID) {
return nil, errors.New("series seasons: invalid season id")
}
seasonIDs = append(seasonIDs, item.Content.ID)
}
if len(seasonIDs) == 0 {
return []Episode{}, nil
}
session, err := c.CreateSession(ctx)
if err != nil {
return nil, err
}
seen := make(map[string]bool)
var episodes []Episode
for _, seasonID := range seasonIDs {
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v1/callSeasonEpisodes/" + seasonID})
query := endpoint.Query()
query.Set("platform_uid", session.PlatformUID)
query.Set("platform_token", session.PlatformToken)
endpoint.RawQuery = query.Encode()
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
if err != nil {
return nil, fmt.Errorf("season episodes request: %w", err)
}
setTVerHeaders(request)
var response seriesContents
if err := c.getJSON(request, &response); err != nil {
return nil, fmt.Errorf("season episodes: %w", err)
}
if response.Code != 0 || response.Result.Contents == nil {
return nil, fmt.Errorf("season episodes: invalid API result (code %d)", response.Code)
}
for _, item := range response.Result.Contents {
if item.Type != "episode" {
continue
}
ep := item.Content
if !tverSeriesIDPattern.MatchString(ep.ID) || ep.SeriesID != seriesID {
return nil, errors.New("season episodes: invalid episode identity")
}
if !seen[ep.ID] {
episodes = append(episodes, ep)
seen[ep.ID] = true
}
}
}
return episodes, nil
}
func setTVerHeaders(request *http.Request) {
request.Header.Set("x-tver-platform-type", "web")
request.Header.Set("Origin", "https://tver.jp")
request.Header.Set("Referer", "https://tver.jp/")
}
+305
View File
@@ -0,0 +1,305 @@
package tver
import (
"context"
"errors"
"net/http"
"os"
"strings"
"testing"
)
func TestFetchSeriesEpisodesFromRecordedResponses(t *testing.T) {
seasons, err := os.ReadFile("testdata/series_seasons.json")
if err != nil {
t.Fatal(err)
}
episodes, err := os.ReadFile("testdata/season_episodes.json")
if err != nil {
t.Fatal(err)
}
var seasonCalls, sessionCalls, episodeCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/v1/callSeriesSeasons/sr0lhdrtdb":
seasonCalls++
if r.Method != http.MethodGet || r.Header.Get("x-tver-platform-type") != "web" || r.Header.Get("Origin") != "https://tver.jp" || r.Header.Get("Referer") != "https://tver.jp/" {
t.Errorf("seasons request: method=%s headers=%v", r.Method, r.Header)
}
_, _ = w.Write(seasons)
case "/v2/api/platform_users/browser/create":
sessionCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
case "/service/api/v1/callSeasonEpisodes/ss3i6njoq3":
episodeCalls++
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" || r.Header.Get("x-tver-platform-type") != "web" || r.Header.Get("Origin") != "https://tver.jp" || r.Header.Get("Referer") != "https://tver.jp/" {
t.Errorf("season episode request: method=%s query=%v headers=%v", r.Method, r.URL.Query(), r.Header)
}
_, _ = w.Write(episodes)
default:
t.Errorf("unexpected path: %s", r.URL.Path)
http.NotFound(w, r)
}
})
got, err := client.FetchSeriesEpisodes(context.Background(), "sr0lhdrtdb")
if err != nil || len(got) != 1 {
t.Fatalf("series episodes: %+v %v", got, err)
}
ep := got[0]
if ep.ID != "eps0z5761a" || ep.SeriesID != "sr0lhdrtdb" || ep.Title != "#10(最終話)" || ep.SeriesTitle != "韓国ドラマ「良いが悪い、ドンジェ」" || ep.BroadcasterName != "BS12トゥエルビ" || ep.Duration != 2712 || ep.EndAt != 1792209540 || !strings.Contains(ep.ThumbnailPath, "eps0z5761a.jpg") {
t.Errorf("episode fields: %+v", ep)
}
if seasonCalls != 1 || sessionCalls != 1 || episodeCalls != 1 {
t.Errorf("calls: seasons=%d session=%d episodes=%d", seasonCalls, sessionCalls, episodeCalls)
}
}
func TestFetchSeriesEpisodesTransportAndDecodeErrors(t *testing.T) {
validSeasons := `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`
validSession := `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`
validEpisodes := `{"code":0,"result":{"contents":[]}}`
for _, tt := range []struct {
name string
seasonStatus, sessionStatus, episodeStatus int
seasonBody, sessionBody, episodeBody string
want string
wantSession, wantEpisode int
}{
{"seasons HTTP", 503, 200, 200, "", validSession, validEpisodes, "series seasons: HTTP 503", 0, 0},
{"seasons JSON", 200, 200, 200, "{", validSession, validEpisodes, "series seasons: decode JSON", 0, 0},
{"session HTTP", 200, 503, 200, validSeasons, "", validEpisodes, "create session: HTTP 503", 1, 0},
{"episodes HTTP", 200, 200, 502, validSeasons, validSession, "", "season episodes: HTTP 502", 1, 1},
{"episodes JSON", 200, 200, 200, validSeasons, validSession, "{", "season episodes: decode JSON", 1, 1},
} {
t.Run(tt.name, func(t *testing.T) {
var seasonCalls, sessionCalls, episodeCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/v1/callSeriesSeasons/sr1":
seasonCalls++
w.WriteHeader(tt.seasonStatus)
_, _ = w.Write([]byte(tt.seasonBody))
case "/v2/api/platform_users/browser/create":
sessionCalls++
w.WriteHeader(tt.sessionStatus)
_, _ = w.Write([]byte(tt.sessionBody))
case "/service/api/v1/callSeasonEpisodes/ss1":
episodeCalls++
w.WriteHeader(tt.episodeStatus)
_, _ = w.Write([]byte(tt.episodeBody))
default:
t.Errorf("unexpected request: %s", r.URL.Path)
}
})
_, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
if err == nil || !strings.Contains(err.Error(), tt.want) {
t.Fatalf("error=%v; want %q", err, tt.want)
}
if seasonCalls != 1 || sessionCalls != tt.wantSession || episodeCalls != tt.wantEpisode {
t.Errorf("calls: seasons=%d session=%d episodes=%d", seasonCalls, sessionCalls, episodeCalls)
}
})
}
}
func TestFetchSeriesEpisodesRejectsBadResponses(t *testing.T) {
for _, tt := range []struct {
name, seasons, episodes string
}{
{"season API code", `{"code":1,"result":{"contents":[]}}`, ""},
{"missing seasons", `{"code":0,"result":{}}`, ""},
{"season without id", `{"code":0,"result":{"contents":[{"type":"season","content":{}}]}}`, ""},
{"episode API code", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":1,"result":{"contents":[]}}`},
{"wrong series", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"other"}}]}}`},
{"episode without id", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":0,"result":{"contents":[{"type":"episode","content":{"seriesID":"sr1"}}]}}`},
} {
t.Run(tt.name, func(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.Contains(r.URL.Path, "callSeriesSeasons"):
_, _ = w.Write([]byte(tt.seasons))
case strings.Contains(r.URL.Path, "browser/create"):
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
default:
_, _ = w.Write([]byte(tt.episodes))
}
})
if _, err := client.FetchSeriesEpisodes(context.Background(), "sr1"); err == nil {
t.Fatal("invalid response accepted")
}
})
}
}
func TestFetchSeriesEpisodesMultipleSeasonsDeduplicates(t *testing.T) {
var sessionCalls, episodeCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/v1/callSeriesSeasons/sr1":
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}},{"type":"season","content":{"id":"ss2"}}]}}`))
case "/v2/api/platform_users/browser/create":
sessionCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
case "/service/api/v1/callSeasonEpisodes/ss1":
episodeCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"sr1"}}]}}`))
case "/service/api/v1/callSeasonEpisodes/ss2":
episodeCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"sr1"}},{"type":"episode","content":{"id":"ep2","seriesID":"sr1"}}]}}`))
default:
t.Errorf("unexpected request: %s", r.URL.Path)
}
})
episodes, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
if err != nil || len(episodes) != 2 || episodes[0].ID != "ep1" || episodes[1].ID != "ep2" || sessionCalls != 1 || episodeCalls != 2 {
t.Fatalf("複数 season: %+v %v session=%d episode=%d", episodes, err, sessionCalls, episodeCalls)
}
}
func TestFetchSeriesEpisodesNoSeasonsSkipsSession(t *testing.T) {
var calls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
calls++
if r.URL.Path != "/api/v1/callSeriesSeasons/sr1" {
t.Errorf("unexpected request: %s", r.URL.Path)
}
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[]}}`))
})
episodes, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
if err != nil || len(episodes) != 0 || calls != 1 {
t.Fatalf("season 無し: %+v %v calls=%d", episodes, err, calls)
}
}
func TestFetchSeriesEpisodesRejectsInvalidSeriesIDBeforeRequest(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
t.Errorf("unexpected request: %s", r.URL.Path)
})
for _, id := range []string{"", "../other", "sr1?x=1"} {
if _, err := client.FetchSeriesEpisodes(context.Background(), id); err == nil {
t.Errorf("invalid series id accepted: %q", id)
}
}
}
func TestFetchSeriesFromRecordedResponse(t *testing.T) {
fixture, err := os.ReadFile("testdata/series.json")
if err != nil {
t.Fatal(err)
}
var sessionCalls, seriesCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v2/api/platform_users/browser/create":
sessionCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
case "/service/api/v2/callSeries/sr0lhdrtdb":
seriesCalls++
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" {
t.Errorf("series request method/query: %s %v", r.Method, r.URL.Query())
}
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
if got := r.Header.Get(name); got != want {
t.Errorf("series header %s = %q; want %q", name, got, want)
}
}
_, _ = w.Write(fixture)
default:
t.Errorf("unexpected request: %s", r.URL.Path)
http.NotFound(w, r)
}
})
title, err := client.FetchSeries(context.Background(), "sr0lhdrtdb")
if err != nil || title != "韓国ドラマ「良いが悪い、ドンジェ」" {
t.Fatalf("series: title=%q error=%v", title, err)
}
if sessionCalls != 1 || seriesCalls != 1 {
t.Fatalf("calls: session=%d series=%d", sessionCalls, seriesCalls)
}
}
func TestFetchSeriesNotFound(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v2/api/platform_users/browser/create" {
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
return
}
http.NotFound(w, r)
})
_, err := client.FetchSeries(context.Background(), "missing")
if !errors.Is(err, ErrSeriesNotFound) {
t.Fatalf("error=%v; want ErrSeriesNotFound", err)
}
}
func TestFetchSeriesTransportAndDecodeErrors(t *testing.T) {
validSession := `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`
for _, tt := range []struct {
name string
sessionStatus, seriesStatus int
sessionBody, seriesBody string
want string
wantSeriesCalls int
}{
{"session HTTP is not series not found", http.StatusNotFound, http.StatusOK, "", "", "create session: HTTP 404", 0},
{"series HTTP", http.StatusOK, http.StatusBadGateway, validSession, "", "series: HTTP 502", 1},
{"series JSON", http.StatusOK, http.StatusOK, validSession, "{", "series: decode JSON", 1},
} {
t.Run(tt.name, func(t *testing.T) {
var seriesCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v2/api/platform_users/browser/create":
w.WriteHeader(tt.sessionStatus)
_, _ = w.Write([]byte(tt.sessionBody))
case "/service/api/v2/callSeries/sr1":
seriesCalls++
w.WriteHeader(tt.seriesStatus)
_, _ = w.Write([]byte(tt.seriesBody))
default:
t.Errorf("unexpected request: %s", r.URL.Path)
}
})
_, err := client.FetchSeries(context.Background(), "sr1")
if err == nil || !strings.Contains(err.Error(), tt.want) || errors.Is(err, ErrSeriesNotFound) {
t.Fatalf("error=%v; want %q and not ErrSeriesNotFound", err, tt.want)
}
if seriesCalls != tt.wantSeriesCalls {
t.Fatalf("series calls=%d; want %d", seriesCalls, tt.wantSeriesCalls)
}
})
}
}
func TestFetchSeriesRejectsBadResponses(t *testing.T) {
for _, tt := range []struct {
name, body string
}{
{"empty title", `{"code":0,"result":{"content":{"content":{"title":""}}}}`},
{"API code", `{"code":7,"result":{"content":{"content":{"title":"番組"}}}}`},
} {
t.Run(tt.name, func(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/v2/api/platform_users/browser/create" {
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
return
}
_, _ = w.Write([]byte(tt.body))
})
if _, err := client.FetchSeries(context.Background(), "sr1"); err == nil {
t.Fatal("invalid response accepted")
}
})
}
}
func TestFetchSeriesRejectsInvalidSeriesIDBeforeRequest(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
t.Errorf("unexpected request: %s", r.URL.Path)
})
for _, id := range []string{"", "../other", "sr1?x=1", "sr_1"} {
if _, err := client.FetchSeries(context.Background(), id); err == nil {
t.Errorf("invalid series id accepted: %q", id)
}
}
}
+91
View File
@@ -0,0 +1,91 @@
{
"api_version": "",
"code": 0,
"message": "",
"type": "hasharray",
"result": {
"contents": {
"type": "newer",
"content": {
"id": "drama",
"version": 1,
"title": "ドラマ"
},
"thumbnailType": "horizontal",
"contents": [
{
"type": "episode",
"content": {
"id": "eps0z5761a",
"version": 4,
"title": "#10(最終話)",
"seriesID": "sr0lhdrtdb",
"duration": 2712,
"endAt": 1792209540,
"broadcastDateLabel": "10月3日(土)放送分",
"isNHKContent": false,
"isSubtitle": false,
"ribbonID": 0,
"seriesTitle": "韓国ドラマ「良いが悪い、ドンジェ」",
"isAvailable": true,
"broadcasterName": "BS12トゥエルビ",
"productionProviderName": "BS12 トゥエルビ",
"isEndingSoon": false,
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51",
"isChannelDefaultSeries": false
},
"startAt": 1791007200,
"endAt": 1792209540
},
{
"type": "episode",
"content": {
"id": "epo4tfi10u",
"version": 4,
"title": "#9",
"seriesID": "sr0lhdrtdb",
"duration": 2712,
"endAt": 1792209540,
"broadcastDateLabel": "10月3日(土)放送分",
"isNHKContent": false,
"isSubtitle": false,
"ribbonID": 0,
"seriesTitle": "韓国ドラマ「良いが悪い、ドンジェ」",
"isAvailable": true,
"broadcasterName": "BS12トゥエルビ",
"productionProviderName": "BS12 トゥエルビ",
"isEndingSoon": false,
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/epo4tfi10u.jpg?v=2d8ff444",
"isChannelDefaultSeries": false
},
"startAt": 1791007200,
"endAt": 1792209540
},
{
"type": "episode",
"content": {
"id": "ep8cxuessg",
"version": 6,
"title": "【予告】10月12日(月)24時24分 ~ 放送予定 60秒PR!",
"seriesID": "srne6nwihe",
"duration": 60,
"endAt": 1800282240,
"broadcastDateLabel": "",
"isNHKContent": false,
"isSubtitle": false,
"ribbonID": 1,
"seriesTitle": "十六夜膳房 -いざよいぜんぼう-",
"isAvailable": true,
"broadcasterName": "日テレ",
"productionProviderName": "日テレ",
"isEndingSoon": false,
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/ep8cxuessg.jpg?v=58408db2",
"isChannelDefaultSeries": false
},
"startAt": 1791007200,
"endAt": 1800282240
}
]
}
}
}
+1
View File
@@ -0,0 +1 @@
{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"eps0z5761a","version":4,"title":"#10(最終話)","seriesID":"sr0lhdrtdb","duration":2712,"endAt":1792209540,"broadcastDateLabel":"10月3日(土)放送分","isNHKContent":false,"isSubtitle":false,"ribbonID":0,"seriesTitle":"韓国ドラマ「良いが悪い、ドンジェ」","isAvailable":true,"broadcasterName":"BS12トゥエルビ","productionProviderName":"BS12 トゥエルビ","isEndingSoon":false,"thumbnailPath":"/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51","isChannelDefaultSeries":false}}]}}
+1
View File
@@ -0,0 +1 @@
{"code":0,"result":{"content":{"type":"series","content":{"id":"sr0lhdrtdb","title":"韓国ドラマ「良いが悪い、ドンジェ」"}}}}
+1
View File
@@ -0,0 +1 @@
{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss3i6njoq3","version":1,"title":"本編","season_extended":null}}]}}
+90
View File
@@ -0,0 +1,90 @@
package web
import (
"errors"
"net/http"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
type episodeActionView struct {
ID, Genre, Status, StatusText string
}
func statusText(status string) string {
switch status {
case "queued":
return "待機中"
case "running":
return "ダウンロード中"
case "succeeded":
return "完了"
case "failed":
return "失敗"
case "canceled":
return "取り消し"
default:
return ""
}
}
func (s *server) enqueue(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
ep, found := s.cachedEpisode(genre, id)
if !found {
http.Error(w, "新着キャッシュにエピソードがありません。新着を再表示してください", http.StatusNotFound)
return
}
job, _, err := s.jobs.Enqueue(r.Context(), storeEpisode(ep), "manual")
if err != nil {
s.logger.Printf("待ち行列への登録に失敗: %v", err)
http.Error(w, "待ち行列に登録できませんでした", http.StatusInternalServerError)
return
}
s.episodeActionResponse(w, r, episodeActionView{ID: id, Genre: genre, Status: job.Status, StatusText: statusText(job.Status)})
}
func (s *server) retryEpisode(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
if _, found := s.cachedEpisode(genre, id); !found {
http.Error(w, "新着キャッシュにエピソードがありません。新着を再表示してください", http.StatusNotFound)
return
}
if err := s.jobs.RetryEpisode(r.Context(), id); err != nil {
if errors.Is(err, store.ErrInvalidTransition) {
http.Error(w, "再試行できる状態ではありません", http.StatusConflict)
} else {
s.logger.Printf("ジョブの再試行に失敗: %v", err)
http.Error(w, "再試行できませんでした", http.StatusInternalServerError)
}
return
}
s.episodeActionResponse(w, r, episodeActionView{ID: id, Genre: genre, Status: "queued", StatusText: statusText("queued")})
}
func episodeForm(w http.ResponseWriter, r *http.Request) (string, string, bool) {
if err := r.ParseForm(); err != nil {
http.Error(w, "フォームを読み取れませんでした", http.StatusBadRequest)
return "", "", false
}
genre, id := r.Form.Get("genre"), r.Form.Get("episode_id")
if (genre != "drama" && genre != "variety") || id == "" {
http.Error(w, "ジャンルまたはエピソード ID が不正です", http.StatusBadRequest)
return "", "", false
}
return genre, id, true
}
func (s *server) episodeActionResponse(w http.ResponseWriter, r *http.Request, view episodeActionView) {
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/?genre="+view.Genre, http.StatusSeeOther)
return
}
s.render(w, "episode_action", view)
}
+110
View File
@@ -0,0 +1,110 @@
package web
import (
"context"
"embed"
"html/template"
"io/fs"
"log"
"net/http"
"sync"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
//go:embed templates/*.html static/*
var assets embed.FS
type page struct {
Title string
Newer *newerView
Queue *queueView
Subscriptions *subscriptionsView
Library *libraryView
Series *seriesView
Video *videoView
DeleteConfirm *deleteConfirmView
DeleteResult *deleteResultView
}
type JobStore interface {
Enqueue(context.Context, store.Episode, string) (store.Job, bool, error)
EpisodeStatuses(context.Context, []string) (map[string]string, error)
ListJobs(context.Context, string, int) ([]store.Job, error)
GetJob(context.Context, int64) (store.Job, error)
Retry(context.Context, int64) error
RetryEpisode(context.Context, string) error
Cancel(context.Context, int64) error
SubscribedSeries(context.Context, []string) (map[string]bool, error)
RegisterSubscription(context.Context, string, string, []store.Episode, bool) (bool, error)
ListSubscriptions(context.Context) ([]store.Subscription, error)
SetSubscriptionEnabled(context.Context, string, bool) error
DeleteSubscription(context.Context, string) (bool, error)
ListSucceededJobs(context.Context, int) ([]store.Job, error)
ListLibrarySeries(context.Context) ([]store.LibrarySeries, error)
ListSucceededJobsBySeries(context.Context, string) ([]store.Job, error)
ListSucceededJobsByIDs(context.Context, []int64) ([]store.Job, error)
ListSucceededJobsBySeriesIDs(context.Context, []string) ([]store.Job, error)
GetSucceededJob(context.Context, int64) (store.Job, error)
DeleteSucceededJob(context.Context, int64) error
}
type server struct {
fetcher NewerFetcher
jobs JobStore
videoDir string
now func() time.Time
logger *log.Logger
zone *time.Location
tmpl *template.Template
cacheMu sync.Mutex
cache map[string]cacheEntry
}
func NewHandler(fetcher NewerFetcher, jobs JobStore, videoDir string, now func() time.Time, logger *log.Logger) http.Handler {
tmpl := template.Must(template.ParseFS(assets, "templates/*.html"))
staticFiles, err := fs.Sub(assets, "static")
if err != nil {
panic(err)
}
zone, err := time.LoadLocation("Asia/Tokyo")
if err != nil {
panic(err)
}
if videoDir == "" {
videoDir = "/videos"
}
s := &server{fetcher: fetcher, jobs: jobs, videoDir: videoDir, now: now, logger: logger, zone: zone, tmpl: tmpl, cache: make(map[string]cacheEntry)}
mux := http.NewServeMux()
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServerFS(staticFiles)))
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
_, _ = w.Write([]byte("ok\n"))
})
mux.HandleFunc("GET /{$}", s.home)
mux.HandleFunc("GET /newer", s.newerFragment)
mux.HandleFunc("POST /episodes/enqueue", s.enqueue)
mux.HandleFunc("POST /episodes/retry", s.retryEpisode)
mux.HandleFunc("GET /queue", s.queuePage)
mux.HandleFunc("GET /queue/list", s.queueFragment)
mux.HandleFunc("POST /jobs/{id}/retry", s.retryJob)
mux.HandleFunc("POST /jobs/{id}/cancel", s.cancelJob)
mux.HandleFunc("GET /subscriptions", s.subscriptionsPage)
mux.HandleFunc("GET /subscriptions/list", s.subscriptionsFragment)
mux.HandleFunc("POST /subscriptions/from-url/confirm", s.confirmSubscriptionURL)
mux.HandleFunc("POST /subscriptions/from-url/register", s.registerSubscriptionURL)
mux.HandleFunc("GET /subscriptions/from-url/cancel", s.cancelSubscriptionURL)
mux.HandleFunc("GET /subscriptions/choice", s.subscriptionChoice)
mux.HandleFunc("POST /subscriptions/register", s.registerSubscription)
mux.HandleFunc("POST /subscriptions/{id}/pause", s.pauseSubscription)
mux.HandleFunc("POST /subscriptions/{id}/resume", s.resumeSubscription)
mux.HandleFunc("POST /subscriptions/{id}/delete", s.deleteSubscription)
mux.HandleFunc("GET /library", s.libraryPage)
mux.HandleFunc("GET /library/series/{series_id}", s.librarySeriesPage)
mux.HandleFunc("GET /library/{id}", s.videoPage)
mux.HandleFunc("GET /videos/{id}", s.serveVideo)
mux.HandleFunc("POST /library/delete/confirm", s.confirmDeleteVideos)
mux.HandleFunc("POST /library/delete", s.deleteVideos)
return mux
}
+63
View File
@@ -0,0 +1,63 @@
package web
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestHealthz(t *testing.T) {
response := httptest.NewRecorder()
newTestHandler(&fakeFetcher{}, nil).ServeHTTP(response, httptest.NewRequest(http.MethodGet, "/healthz", nil))
if response.Code != http.StatusOK || response.Body.String() != "ok\n" {
t.Errorf("healthz: status=%d body=%q", response.Code, response.Body.String())
}
}
func TestPagesAndEmbeddedAssets(t *testing.T) {
handler := newTestHandler(&fakeFetcher{}, nil)
for _, tt := range []struct{ path, title string }{
{"/", "新着"}, {"/queue", "待ち行列"}, {"/subscriptions", "登録番組"}, {"/library", "ライブラリ"},
} {
t.Run(tt.path, func(t *testing.T) {
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, tt.path, nil))
body := response.Body.String()
if response.Code != http.StatusOK || !strings.Contains(response.Header().Get("Content-Type"), "text/html") {
t.Fatalf("status=%d Content-Type=%q", response.Code, response.Header().Get("Content-Type"))
}
for _, want := range []string{"<h1>" + tt.title + "</h1>", "新着", "待ち行列", "登録番組", "ライブラリ", `hx-boost="true"`, "/static/htmx-2.0.11.min.js"} {
if !strings.Contains(body, want) {
t.Errorf("page missing %q", want)
}
}
})
}
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, "/static/htmx-2.0.11.min.js", nil))
if response.Code != http.StatusOK || !strings.Contains(response.Header().Get("Content-Type"), "javascript") || response.Body.Len() < 1000 || !strings.Contains(response.Body.String(), "htmx") {
t.Errorf("embedded htmx: status=%d Content-Type=%q bytes=%d", response.Code, response.Header().Get("Content-Type"), response.Body.Len())
}
}
func TestUnknownPathsAndMethods(t *testing.T) {
handler := newTestHandler(&fakeFetcher{}, nil)
for _, tt := range []struct {
name, method, path string
wantStatus int
}{
{"unknown page", http.MethodGet, "/missing", http.StatusNotFound},
{"root prefix", http.MethodGet, "/missing/nested", http.StatusNotFound},
{"missing static asset", http.MethodGet, "/static/missing.js", http.StatusNotFound},
{"unsupported page method", http.MethodPost, "/queue", http.StatusMethodNotAllowed},
} {
t.Run(tt.name, func(t *testing.T) {
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(tt.method, tt.path, nil))
if response.Code != tt.wantStatus {
t.Errorf("%s %s: status=%d; want %d", tt.method, tt.path, response.Code, tt.wantStatus)
}
})
}
}
+492
View File
@@ -0,0 +1,492 @@
package web
import (
"context"
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"sort"
"strconv"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
type libraryItemView struct {
ID int64
ThumbnailURL, SeriesTitle, Title string
BroadcasterName, DurationText string
FileSizeText, FinishedAtText string
}
type librarySeriesView struct {
SeriesID, SeriesTitle, LatestTitle string
ThumbnailURL, BroadcasterName string
EpisodeCount int64
TotalSizeText, LatestFinishedText string
Subscribed bool
}
type libraryView struct {
Series []librarySeriesView
SeriesCount, EpisodeCount int
TotalSizeText string
}
type seriesView struct {
SeriesID, SeriesTitle, BroadcasterName string
EpisodeCount int
TotalSizeText, Sort string
Subscribed bool
Items []libraryItemView
}
type episodeLinkView struct {
ID int64
Title string
}
type videoView struct {
libraryItemView
VideoURL, SeriesURL string
Previous, Next *episodeLinkView
}
type deleteGroupView struct {
SeriesID, SeriesTitle, TotalSizeText string
Items []libraryItemView
}
type deleteConfirmView struct {
Groups []deleteGroupView
IDs []int64
Count int
TotalSizeText string
CancelURL string
}
type deleteFailureView struct {
Title, Reason string
}
type deleteSeriesLinkView struct {
SeriesID, SeriesTitle string
}
type deleteResultView struct {
DeletedCount, ExcludedCount int
Failures []deleteFailureView
SeriesLinks []deleteSeriesLinkView
}
func (s *server) libraryPage(w http.ResponseWriter, r *http.Request) {
series, err := s.jobs.ListLibrarySeries(r.Context())
if err != nil {
s.logger.Printf("ライブラリの取得に失敗: %v", err)
http.Error(w, "ライブラリを取得できませんでした", http.StatusInternalServerError)
return
}
view := libraryView{Series: make([]librarySeriesView, 0, len(series)), SeriesCount: len(series)}
var totalBytes int64
for _, item := range series {
view.EpisodeCount += int(item.EpisodeCount)
totalBytes += item.TotalSizeBytes
viewItem := librarySeriesView{
SeriesID: item.SeriesID, SeriesTitle: item.SeriesTitle, LatestTitle: item.LatestTitle,
ThumbnailURL: thumbnailURL(item.ThumbnailPath), BroadcasterName: item.BroadcasterName,
EpisodeCount: item.EpisodeCount, TotalSizeText: formatBytes(float64(item.TotalSizeBytes)),
Subscribed: item.Subscribed,
}
if item.LatestFinishedAt != nil {
viewItem.LatestFinishedText = item.LatestFinishedAt.In(s.zone).Format("2006-01-02 15:04 MST")
}
view.Series = append(view.Series, viewItem)
}
view.TotalSizeText = fmt.Sprintf("%.2f GB", float64(totalBytes)/1_000_000_000)
s.render(w, "layout", page{Title: "ライブラリ", Library: &view})
}
func (s *server) librarySeriesPage(w http.ResponseWriter, r *http.Request) {
seriesID := r.PathValue("series_id")
if !seriesIDPattern.MatchString(seriesID) {
http.NotFound(w, r)
return
}
sortOrder := r.URL.Query().Get("sort")
if sortOrder == "" {
sortOrder = "latest"
}
if sortOrder != "latest" && sortOrder != "natural" {
http.Error(w, "並び順が不正です", http.StatusBadRequest)
return
}
jobs, err := s.jobs.ListSucceededJobsBySeries(r.Context(), seriesID)
if err != nil {
s.logger.Printf("番組のライブラリ取得に失敗: %v", err)
http.Error(w, "番組を取得できませんでした", http.StatusInternalServerError)
return
}
if len(jobs) == 0 {
http.NotFound(w, r)
return
}
latest := latestFinishedJob(jobs)
if sortOrder == "natural" {
sortJobsNaturally(jobs)
} else {
sortJobsLatest(jobs)
}
subscribed, err := s.jobs.SubscribedSeries(r.Context(), []string{seriesID})
if err != nil {
s.logger.Printf("番組登録状態の取得に失敗: %v", err)
http.Error(w, "番組を取得できませんでした", http.StatusInternalServerError)
return
}
view := seriesView{SeriesID: seriesID, SeriesTitle: latest.SeriesTitle,
BroadcasterName: latest.BroadcasterName, EpisodeCount: len(jobs),
Subscribed: subscribed[seriesID], Sort: sortOrder, Items: make([]libraryItemView, 0, len(jobs))}
var totalBytes int64
for _, job := range jobs {
if job.FileSizeBytes != nil {
totalBytes += *job.FileSizeBytes
}
view.Items = append(view.Items, s.libraryItem(job))
}
view.TotalSizeText = formatBytes(float64(totalBytes))
s.render(w, "layout", page{Title: latest.SeriesTitle, Series: &view})
}
func (s *server) videoPage(w http.ResponseWriter, r *http.Request) {
job, ok := s.succeededJob(w, r)
if !ok {
return
}
jobs, err := s.jobs.ListSucceededJobsBySeries(r.Context(), job.SeriesID)
if err != nil {
s.logger.Printf("前後の回の取得に失敗: %v", err)
http.Error(w, "動画を取得できませんでした", http.StatusInternalServerError)
return
}
sortJobsNaturally(jobs)
view := videoView{libraryItemView: s.libraryItem(job), VideoURL: "/videos/" + strconv.FormatInt(job.ID, 10),
SeriesURL: "/library/series/" + job.SeriesID}
for i := range jobs {
if jobs[i].ID != job.ID {
continue
}
if i > 0 {
view.Previous = &episodeLinkView{ID: jobs[i-1].ID, Title: jobs[i-1].Title}
}
if i+1 < len(jobs) {
view.Next = &episodeLinkView{ID: jobs[i+1].ID, Title: jobs[i+1].Title}
}
break
}
s.render(w, "layout", page{Title: job.SeriesTitle + " " + job.Title, Video: &view})
}
func (s *server) libraryItem(job store.Job) libraryItemView {
item := libraryItemView{ID: job.ID, ThumbnailURL: thumbnailURL(job.ThumbnailPath),
SeriesTitle: job.SeriesTitle, Title: job.Title, BroadcasterName: job.BroadcasterName,
DurationText: fmt.Sprintf("%d分", durationMinutes(int64(job.DurationSeconds)))}
if job.FileSizeBytes != nil {
item.FileSizeText = formatBytes(float64(*job.FileSizeBytes))
}
if job.FinishedAt != nil {
item.FinishedAtText = job.FinishedAt.In(s.zone).Format("2006-01-02 15:04 MST")
}
return item
}
func thumbnailURL(path string) string {
if path == "" {
return ""
}
return "https://statics.tver.jp" + path
}
func latestFinishedJob(jobs []store.Job) store.Job {
latest := jobs[0]
for _, job := range jobs[1:] {
if jobFinishedLater(job, latest) {
latest = job
}
}
return latest
}
func sortJobsLatest(jobs []store.Job) {
sort.SliceStable(jobs, func(i, j int) bool { return jobFinishedLater(jobs[i], jobs[j]) })
}
func jobFinishedLater(left, right store.Job) bool {
if left.FinishedAt == nil || right.FinishedAt == nil {
if left.FinishedAt != right.FinishedAt {
return left.FinishedAt != nil
}
return left.ID > right.ID
}
if !left.FinishedAt.Equal(*right.FinishedAt) {
return left.FinishedAt.After(*right.FinishedAt)
}
return left.ID > right.ID
}
func (s *server) serveVideo(w http.ResponseWriter, r *http.Request) {
job, ok := s.succeededJob(w, r)
if !ok || job.FilePath == nil {
if ok {
http.NotFound(w, r)
}
return
}
root, err := os.OpenRoot(s.videoDir)
if err != nil {
http.NotFound(w, r)
return
}
defer root.Close()
file, err := root.Open(*job.FilePath)
if err != nil {
http.NotFound(w, r)
return
}
defer file.Close()
info, err := file.Stat()
if err != nil || !info.Mode().IsRegular() {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "video/mp4")
http.ServeContent(w, r, filepath.Base(*job.FilePath), info.ModTime(), file)
}
const maxDeleteSelections = 1000
func (s *server) confirmDeleteVideos(w http.ResponseWriter, r *http.Request) {
ids, seriesIDs, ok := parseDeleteSelection(w, r, true)
if !ok {
return
}
jobsByID, err := s.jobs.ListSucceededJobsByIDs(r.Context(), ids)
if err != nil {
s.logger.Printf("削除対象IDの取得に失敗: %v", err)
http.Error(w, "削除対象を取得できませんでした", http.StatusInternalServerError)
return
}
jobsBySeries, err := s.jobs.ListSucceededJobsBySeriesIDs(r.Context(), seriesIDs)
if err != nil {
s.logger.Printf("削除対象番組の取得に失敗: %v", err)
http.Error(w, "削除対象を取得できませんでした", http.StatusInternalServerError)
return
}
byID := make(map[int64]store.Job, len(jobsByID)+len(jobsBySeries))
for _, job := range append(jobsByID, jobsBySeries...) {
byID[job.ID] = job
}
if len(byID) == 0 {
http.Error(w, "削除対象がありません", http.StatusBadRequest)
return
}
if len(byID) > maxDeleteSelections {
http.Error(w, fmt.Sprintf("削除対象が多すぎます(%d本)。番組を分けて選んでください", len(byID)), http.StatusBadRequest)
return
}
jobs := make([]store.Job, 0, len(byID))
for _, job := range byID {
jobs = append(jobs, job)
}
sort.Slice(jobs, func(i, j int) bool {
if jobs[i].SeriesTitle != jobs[j].SeriesTitle {
return jobs[i].SeriesTitle < jobs[j].SeriesTitle
}
return jobs[i].ID < jobs[j].ID
})
view := deleteConfirmView{IDs: make([]int64, 0, len(jobs)), Count: len(jobs), CancelURL: "/library"}
groupIndexes := make(map[string]int)
groupBytes := make(map[string]int64)
var totalBytes int64
for _, job := range jobs {
view.IDs = append(view.IDs, job.ID)
index, exists := groupIndexes[job.SeriesID]
if !exists {
index = len(view.Groups)
groupIndexes[job.SeriesID] = index
view.Groups = append(view.Groups, deleteGroupView{SeriesID: job.SeriesID, SeriesTitle: job.SeriesTitle})
}
view.Groups[index].Items = append(view.Groups[index].Items, s.libraryItem(job))
if job.FileSizeBytes != nil {
totalBytes += *job.FileSizeBytes
groupBytes[job.SeriesID] += *job.FileSizeBytes
}
}
for i := range view.Groups {
view.Groups[i].TotalSizeText = formatBytes(float64(groupBytes[view.Groups[i].SeriesID]))
}
view.TotalSizeText = formatBytes(float64(totalBytes))
if len(view.Groups) == 1 {
view.CancelURL = "/library/series/" + view.Groups[0].SeriesID
}
s.render(w, "layout", page{Title: "動画の削除確認", DeleteConfirm: &view})
}
func parseDeleteSelection(w http.ResponseWriter, r *http.Request, includeSeries bool) ([]int64, []string, bool) {
if err := r.ParseForm(); err != nil {
http.Error(w, "入力が不正です", http.StatusBadRequest)
return nil, nil, false
}
idSet := make(map[int64]struct{})
for _, raw := range r.Form["id"] {
id, err := strconv.ParseInt(raw, 10, 64)
if err != nil || id <= 0 {
http.Error(w, "動画IDが不正です", http.StatusBadRequest)
return nil, nil, false
}
idSet[id] = struct{}{}
}
seriesSet := make(map[string]struct{})
if includeSeries {
for _, seriesID := range r.Form["series_id"] {
if !seriesIDPattern.MatchString(seriesID) {
http.Error(w, "番組IDが不正です", http.StatusBadRequest)
return nil, nil, false
}
seriesSet[seriesID] = struct{}{}
}
}
if len(idSet)+len(seriesSet) == 0 {
http.Error(w, "削除対象がありません", http.StatusBadRequest)
return nil, nil, false
}
if len(idSet)+len(seriesSet) > maxDeleteSelections {
http.Error(w, "削除対象が多すぎます", http.StatusBadRequest)
return nil, nil, false
}
ids := make([]int64, 0, len(idSet))
for id := range idSet {
ids = append(ids, id)
}
seriesIDs := make([]string, 0, len(seriesSet))
for seriesID := range seriesSet {
seriesIDs = append(seriesIDs, seriesID)
}
return ids, seriesIDs, true
}
func (s *server) deleteVideos(w http.ResponseWriter, r *http.Request) {
ids, _, ok := parseDeleteSelection(w, r, false)
if !ok {
return
}
jobs, err := s.jobs.ListSucceededJobsByIDs(r.Context(), ids)
if err != nil {
s.logger.Printf("削除対象の取得に失敗: %v", err)
http.Error(w, "削除対象を取得できませんでした", http.StatusInternalServerError)
return
}
view := deleteResultView{ExcludedCount: len(ids) - len(jobs)}
var root *os.Root
needsRoot := false
for _, job := range jobs {
needsRoot = needsRoot || job.FilePath != nil
}
if needsRoot {
root, err = os.OpenRoot(s.videoDir)
if err == nil {
defer root.Close()
}
}
affectedSeries := make(map[string]string)
for _, job := range jobs {
affectedSeries[job.SeriesID] = job.SeriesTitle
if err != nil && job.FilePath != nil {
s.logger.Printf("動画ルートを開けないため削除に失敗 id=%d: %v", job.ID, err)
view.Failures = append(view.Failures, deleteFailureView{Title: job.Title, Reason: "動画ファイルを削除できませんでした"})
continue
}
if deleteErr := s.deleteJob(r.Context(), root, job); deleteErr != nil {
s.logger.Printf("動画の削除に失敗 id=%d: %v", job.ID, deleteErr)
view.Failures = append(view.Failures, deleteFailureView{Title: job.Title, Reason: "削除できませんでした"})
continue
}
view.DeletedCount++
}
for seriesID, title := range affectedSeries {
remaining, listErr := s.jobs.ListSucceededJobsBySeries(r.Context(), seriesID)
if listErr != nil {
s.logger.Printf("削除後の番組取得に失敗 series_id=%s: %v", seriesID, listErr)
continue
}
if len(remaining) > 0 {
view.SeriesLinks = append(view.SeriesLinks, deleteSeriesLinkView{SeriesID: seriesID, SeriesTitle: title})
}
}
sort.Slice(view.SeriesLinks, func(i, j int) bool { return view.SeriesLinks[i].SeriesTitle < view.SeriesLinks[j].SeriesTitle })
s.render(w, "layout", page{Title: "動画の削除結果", DeleteResult: &view})
}
func (s *server) deleteJob(ctx context.Context, root *os.Root, job store.Job) error {
if job.FilePath != nil {
if !filepath.IsLocal(*job.FilePath) {
return errors.New("動画ファイルの場所が不正")
}
if root == nil {
return errors.New("動画ルートを開けない")
}
file, err := root.Open(*job.FilePath)
if err == nil {
_ = file.Close()
} else if !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("動画ファイルの検査: %w", err)
}
if err := root.Remove(*job.FilePath); err != nil && !errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("動画ファイルの削除: %w", err)
}
s.removeEmptyVideoParent(root, *job.FilePath)
}
if err := s.jobs.DeleteSucceededJob(ctx, job.ID); err != nil {
return fmt.Errorf("動画の記録削除: %w", err)
}
return nil
}
func (s *server) removeEmptyVideoParent(root *os.Root, filePath string) {
parent := filepath.Dir(filePath)
if parent == "." || !filepath.IsLocal(parent) {
return
}
info, err := root.Lstat(parent)
if err != nil {
if !errors.Is(err, os.ErrNotExist) {
s.logger.Printf("動画フォルダの確認に失敗: %v", err)
}
return
}
if !info.IsDir() {
return
}
if err := root.Remove(parent); err != nil && !errors.Is(err, os.ErrNotExist) {
s.logger.Printf("空の動画フォルダを削除できませんでした: %v", err)
}
}
func (s *server) succeededJob(w http.ResponseWriter, r *http.Request) (store.Job, bool) {
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.NotFound(w, r)
return store.Job{}, false
}
job, err := s.jobs.GetSucceededJob(r.Context(), id)
if errors.Is(err, pgx.ErrNoRows) {
http.NotFound(w, r)
return store.Job{}, false
}
if err != nil {
s.logger.Printf("完了ジョブの取得に失敗: %v", err)
http.Error(w, "動画を取得できませんでした", http.StatusInternalServerError)
return store.Job{}, false
}
return job, true
}
+639
View File
@@ -0,0 +1,639 @@
package web
import (
"errors"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
func libraryJob(id int64, relative string) store.Job {
size := int64(10)
finished := time.Date(2026, 10, 3, 1, 2, 3, 0, time.UTC)
return store.Job{ID: id, EpisodeID: "ep1", SeriesID: "series1", SeriesTitle: `<script>番組</script>`, Title: `<b>第1話</b>`,
BroadcasterName: "放送局", DurationSeconds: 61, ThumbnailPath: "/thumb.jpg", Status: "succeeded",
FilePath: &relative, FileSizeBytes: &size, FinishedAt: &finished}
}
func TestLibraryAndPlaybackPage(t *testing.T) {
dir := t.TempDir()
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
page := getPage(handler, "/library")
if page.Code != http.StatusOK {
t.Fatalf("library status=%d", page.Code)
}
body := page.Body.String()
for _, want := range []string{"&lt;script&gt;番組&lt;/script&gt;", "&lt;b&gt;第1話&lt;/b&gt;", "放送局", "1番組・1本・0.00 GB", "10.0 B", "2026-10-03 10:02 JST", "https://statics.tver.jp/thumb.jpg", `/library/series/series1`, "1本"} {
if !strings.Contains(body, want) {
t.Errorf("library missing %q", want)
}
}
if strings.Contains(body, "<script>番組</script>") {
t.Fatal("library fields were not escaped")
}
video := getPage(handler, "/library/1")
if video.Code != http.StatusOK || !strings.Contains(video.Body.String(), `<video controls`) || !strings.Contains(video.Body.String(), `src="/videos/1"`) || !strings.Contains(video.Body.String(), `action="/library/delete/confirm"`) || strings.Contains(video.Body.String(), `hx-confirm`) || !strings.Contains(video.Body.String(), `/library/series/series1`) || !strings.Contains(video.Body.String(), `前の回はありません`) || !strings.Contains(video.Body.String(), `次の回はありません`) {
t.Fatalf("video page: %d %s", video.Code, video.Body.String())
}
for _, path := range []string{"/library/bad", "/library/99", "/library/1/delete"} {
response := getPage(handler, path)
want := http.StatusNotFound
if response.Code != want {
t.Errorf("%s status=%d want=%d", path, response.Code, want)
}
}
}
func TestLibrarySeriesPageSortingSubscriptionAndNotFound(t *testing.T) {
old := libraryJob(2, "show/old.mp4")
old.EpisodeID = "ep2"
old.Title = "第2話"
old.SeriesTitle = "旧番組名"
old.BroadcasterName = "旧局"
oldFinished := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
old.FinishedAt = &oldFinished
latest := libraryJob(10, "show/latest.mp4")
latest.EpisodeID = "ep10"
latest.Title = "第10話"
latest.SeriesTitle = `<em>新番組名</em>`
latest.BroadcasterName = "新局"
latestFinished := oldFinished.Add(time.Hour)
latest.FinishedAt = &latestFinished
jobs := &fakeJobStore{
jobs: map[string]store.Job{"ep2": old, "ep10": latest},
subscriptions: map[string]store.Subscription{"series1": {SeriesID: "series1"}},
}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
library := getPage(handler, "/library")
if library.Code != http.StatusOK || !strings.Contains(library.Body.String(), "登録中") || !strings.Contains(library.Body.String(), "2本") {
t.Fatalf("library series card: status=%d body=%s", library.Code, library.Body.String())
}
page := getPage(handler, "/library/series/series1")
if page.Code != http.StatusOK {
t.Fatalf("series status=%d body=%s", page.Code, page.Body.String())
}
body := page.Body.String()
for _, want := range []string{"ライブラリ", "&lt;em&gt;新番組名&lt;/em&gt;", "登録中", "新局・2本・20.0 B", `/library/10`, `/library/2`, `?sort=natural`} {
if !strings.Contains(body, want) {
t.Errorf("series page missing %q", want)
}
}
if strings.Index(body, `/library/10`) > strings.Index(body, `/library/2`) {
t.Fatal("default order is not latest first")
}
natural := getPage(handler, "/library/series/series1?sort=natural")
if natural.Code != http.StatusOK || strings.Index(natural.Body.String(), `/library/2`) > strings.Index(natural.Body.String(), `/library/10`) {
t.Fatalf("natural order: status=%d body=%s", natural.Code, natural.Body.String())
}
if got := getPage(handler, "/library/series/missing").Code; got != http.StatusNotFound {
t.Fatalf("missing series status=%d", got)
}
if got := getPage(handler, "/library/series/bad-id").Code; got != http.StatusNotFound {
t.Fatalf("invalid series status=%d", got)
}
if got := getPage(handler, "/library/series/series1?sort=bad").Code; got != http.StatusBadRequest {
t.Fatalf("invalid sort status=%d", got)
}
}
func TestLibraryOmitsSubscriptionMarkerForUnregisteredSeries(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
page := getPage(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil), "/library")
if page.Code != http.StatusOK || strings.Contains(page.Body.String(), "登録中") {
t.Fatalf("unregistered series: status=%d body=%s", page.Code, page.Body.String())
}
}
func TestLibraryPagesDoNotInspectVideoDirectory(t *testing.T) {
missingVideoDir := filepath.Join(t.TempDir(), "missing-video-root")
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "missing/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, missingVideoDir, nil)
for _, path := range []string{"/library", "/library/series/series1"} {
response := getPage(handler, path)
if response.Code != http.StatusOK {
t.Errorf("DB由来の一覧 %s: status=%d body=%s", path, response.Code, response.Body.String())
}
}
if _, err := os.Stat(missingVideoDir); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("一覧表示が動画ルートを作成または走査した: %v", err)
}
}
func TestPlaybackPreviousAndNextNaturalOrder(t *testing.T) {
jobs := make(map[string]store.Job)
for _, data := range []struct {
id int64
epid string
title string
}{{2, "ep2", "第2話"}, {10, "ep10", "第10話"}, {9, "ep9", "第9話"}} {
job := libraryJob(data.id, "show/video.mp4")
job.EpisodeID = data.epid
job.Title = data.title
jobs[data.epid] = job
}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, &fakeJobStore{jobs: jobs}, t.TempDir(), nil)
middle := getPage(handler, "/library/9")
for _, want := range []string{`href="/library/2"`, `前の回: 第2話`, `href="/library/10"`, `次の回: 第10話`} {
if !strings.Contains(middle.Body.String(), want) {
t.Errorf("middle playback missing %q", want)
}
}
first := getPage(handler, "/library/2")
if !strings.Contains(first.Body.String(), "前の回はありません") || !strings.Contains(first.Body.String(), `href="/library/9"`) {
t.Fatalf("first playback navigation: %s", first.Body.String())
}
last := getPage(handler, "/library/10")
if !strings.Contains(last.Body.String(), "次の回はありません") || !strings.Contains(last.Body.String(), `href="/library/9"`) {
t.Fatalf("last playback navigation: %s", last.Body.String())
}
}
func TestLibraryRejectsInvalidJobIDs(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
for _, id := range []string{"0", "-1", "bad", "999999999999999999999999999999999999"} {
for _, path := range []string{"/library/" + id, "/videos/" + id} {
if got := getPage(handler, path).Code; got != http.StatusNotFound {
t.Errorf("GET %s: %d", path, got)
}
}
if got := postForm(handler, "/library/delete", url.Values{"id": {id}}).Code; got != http.StatusBadRequest {
t.Errorf("POST delete id=%q: %d", id, got)
}
}
}
func TestVideoRangeAndMissingFile(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
t.Fatal(err)
}
content := []byte("0123456789")
if err := os.WriteFile(filepath.Join(dir, "show", "video.mp4"), content, 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
request := httptest.NewRequest(http.MethodGet, "/videos/1?file_path=../../outside.mp4", nil)
request.Header.Set("Range", "bytes=2-5")
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusPartialContent || response.Body.String() != "2345" || response.Header().Get("Content-Range") != "bytes 2-5/10" || response.Header().Get("Content-Type") != "video/mp4" {
t.Fatalf("range: status=%d range=%q type=%q body=%q", response.Code, response.Header().Get("Content-Range"), response.Header().Get("Content-Type"), response.Body.String())
}
full := getPage(handler, "/videos/1")
if full.Code != http.StatusOK || full.Body.String() != string(content) || full.Header().Get("Content-Type") != "video/mp4" || full.Header().Get("Accept-Ranges") != "bytes" {
t.Fatalf("full video: status=%d type=%q ranges=%q body=%q", full.Code, full.Header().Get("Content-Type"), full.Header().Get("Accept-Ranges"), full.Body.String())
}
request = httptest.NewRequest(http.MethodGet, "/videos/1", nil)
request.Header.Set("Range", "bytes=20-30")
response = httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusRequestedRangeNotSatisfiable || response.Header().Get("Content-Range") != "bytes */10" {
t.Fatalf("range outside: status=%d range=%q", response.Code, response.Header().Get("Content-Range"))
}
if err := os.Remove(filepath.Join(dir, "show", "video.mp4")); err != nil {
t.Fatal(err)
}
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("missing file status=%d", got)
}
jobs.jobs["ep1"] = store.Job{ID: 1, Status: "queued"}
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("unfinished status=%d", got)
}
if got := getPage(handler, "/library/1").Code; got != http.StatusNotFound {
t.Fatalf("unfinished playback page status=%d", got)
}
missingPath := libraryJob(1, "show/video.mp4")
missingPath.FilePath = nil
jobs.jobs["ep1"] = missingPath
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("missing DB file path status=%d", got)
}
}
func TestVideoPathTraversalAndSymlinkAreRejected(t *testing.T) {
dir := t.TempDir()
outside := t.TempDir()
outsideFile := filepath.Join(outside, "outside.mp4")
if err := os.WriteFile(outsideFile, []byte("secret"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outsideFile, filepath.Join(dir, "link.mp4")); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(dir, "linked-directory")); err != nil {
t.Fatal(err)
}
relativeOutsideFile, err := filepath.Rel(dir, outsideFile)
if err != nil {
t.Fatal(err)
}
relativeOutsideDir, err := filepath.Rel(dir, outside)
if err != nil {
t.Fatal(err)
}
if err := os.Symlink(relativeOutsideFile, filepath.Join(dir, "relative-link.mp4")); err != nil {
t.Fatal(err)
}
if err := os.Symlink(relativeOutsideDir, filepath.Join(dir, "relative-linked-directory")); err != nil {
t.Fatal(err)
}
for _, relative := range []string{"../outside.mp4", "show/../../outside.mp4", outsideFile,
"link.mp4", "linked-directory/outside.mp4", "linked-directory/missing.mp4",
"relative-link.mp4", "relative-linked-directory/outside.mp4", "relative-linked-directory/missing.mp4"} {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, relative)}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Errorf("path %q status=%d", relative, got)
}
deleted := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
if deleted.Code != http.StatusOK || len(jobs.jobs) != 1 || !strings.Contains(deleted.Body.String(), "失敗: 1件") {
t.Errorf("delete path %q status=%d jobs=%d", relative, deleted.Code, len(jobs.jobs))
}
}
if got, err := os.ReadFile(outsideFile); err != nil || string(got) != "secret" {
t.Fatalf("outside target was changed: content=%q err=%v", got, err)
}
for _, link := range []string{"link.mp4", "linked-directory", "relative-link.mp4", "relative-linked-directory"} {
if _, err := os.Lstat(filepath.Join(dir, link)); err != nil {
t.Errorf("rejected link %q was changed: %v", link, err)
}
}
}
func TestVideoSymlinkWithinRootCanBeServedAndDeleted(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "target.mp4")
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink("target.mp4", filepath.Join(dir, "link.mp4")); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "link.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if response := getPage(handler, "/videos/1"); response.Code != http.StatusOK || response.Body.String() != "video" {
t.Fatalf("serve: status=%d body=%q", response.Code, response.Body.String())
}
if response := postForm(handler, "/library/delete", url.Values{"id": {"1"}}); response.Code != http.StatusOK || len(jobs.jobs) != 0 {
t.Fatalf("delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
}
if _, err := os.Stat(target); err != nil {
t.Fatalf("symlink target was removed: %v", err)
}
if _, err := os.Lstat(filepath.Join(dir, "link.mp4")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("symlink itself remains: %v", err)
}
}
func TestVideoSymlinkedDirectoryWithinRootCanBeServedAndDeleted(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "target"), 0o755); err != nil {
t.Fatal(err)
}
target := filepath.Join(dir, "target", "video.mp4")
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink("target", filepath.Join(dir, "linked-directory")); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "linked-directory/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if response := getPage(handler, "/videos/1"); response.Code != http.StatusOK || response.Body.String() != "video" {
t.Fatalf("serve: status=%d body=%q", response.Code, response.Body.String())
}
if response := postForm(handler, "/library/delete", url.Values{"id": {"1"}}); response.Code != http.StatusOK || len(jobs.jobs) != 0 {
t.Fatalf("delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
}
if _, err := os.Stat(target); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("target file remains: %v", err)
}
if _, err := os.Lstat(filepath.Join(dir, "linked-directory")); err != nil {
t.Fatalf("directory symlink was removed: %v", err)
}
}
func TestVideoAbsoluteSymlinkWithinRootIsRejectedByOSRoot(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "target.mp4")
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, filepath.Join(dir, "absolute-link.mp4")); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "absolute-link.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("absolute symlink serve status=%d", got)
}
if response := postForm(handler, "/library/delete", url.Values{"id": {"1"}}); response.Code != http.StatusOK || len(jobs.jobs) != 1 || !strings.Contains(response.Body.String(), "失敗: 1件") {
t.Fatalf("absolute symlink delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
}
if _, err := os.Stat(target); err != nil {
t.Fatalf("target was changed: %v", err)
}
}
func TestDeleteVideoFileThenRecord(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "show", "video.mp4")
if err := os.WriteFile(path, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
fileWasGoneBeforeRecordDelete := false
jobs.beforeDeleteSucceeded = func() {
_, err := os.Stat(path)
fileWasGoneBeforeRecordDelete = errors.Is(err, os.ErrNotExist)
}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
response := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
if response.Code != http.StatusOK || len(jobs.jobs) != 0 || !fileWasGoneBeforeRecordDelete || !strings.Contains(response.Body.String(), "削除: 1件") {
t.Fatalf("delete: status=%d location=%q jobs=%d", response.Code, response.Header().Get("Location"), len(jobs.jobs))
}
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Fatalf("file remains: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, "show")); !os.IsNotExist(err) {
t.Fatalf("empty series folder remains: %v", err)
}
jobs.jobs["ep1"] = libraryJob(1, "show/missing.mp4")
if got := postForm(handler, "/library/delete", url.Values{"id": {"1"}}).Code; got != http.StatusOK || len(jobs.jobs) != 0 {
t.Fatalf("missing file delete: status=%d jobs=%d", got, len(jobs.jobs))
}
}
func TestDeleteRedirectsToSeriesWhenEpisodesRemainAndKeepsNonemptyFolder(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "show", "one.mp4"), []byte("one"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "show", "two.mp4"), []byte("two"), 0o644); err != nil {
t.Fatal(err)
}
first := libraryJob(1, "show/one.mp4")
second := libraryJob(2, "show/two.mp4")
second.EpisodeID = "ep2"
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": first, "ep2": second}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
response := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "/library/series/series1") || len(jobs.jobs) != 1 {
t.Fatalf("delete: status=%d location=%q jobs=%d", response.Code, response.Header().Get("Location"), len(jobs.jobs))
}
if _, err := os.Stat(filepath.Join(dir, "show")); err != nil {
t.Fatalf("nonempty series folder removed: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, "show", "two.mp4")); err != nil {
t.Fatalf("remaining video removed: %v", err)
}
}
func TestDeleteVideoAtRootNeverRemovesVideoRoot(t *testing.T) {
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "video.mp4"), []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "video.mp4")}}
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil), "/library/delete", url.Values{"id": {"1"}})
if response.Code != http.StatusOK {
t.Fatalf("delete status=%d", response.Code)
}
if info, err := os.Stat(dir); err != nil || !info.IsDir() {
t.Fatalf("video root was removed: info=%v err=%v", info, err)
}
}
func TestDeleteFailureKeepsRecord(t *testing.T) {
dir := t.TempDir()
blocked := filepath.Join(dir, "blocked")
if err := os.Mkdir(blocked, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(blocked, "child"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "blocked")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
response := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
if response.Code != http.StatusOK || len(jobs.jobs) != 1 || !strings.Contains(response.Body.String(), "失敗: 1件") {
t.Fatalf("failure: status=%d jobs=%d body=%s", response.Code, len(jobs.jobs), response.Body.String())
}
}
func TestDeleteConfirmationSelectionValidationAndDeduplication(t *testing.T) {
first := libraryJob(1, "show/one.mp4")
second := libraryJob(2, "show/two.mp4")
second.EpisodeID = "ep2"
second.Title = `<img src=x onerror=alert(1)>`
third := libraryJob(3, "other/three.mp4")
third.EpisodeID = "ep3"
third.SeriesID = "series2"
third.SeriesTitle = "別番組"
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": first, "ep2": second, "ep3": third}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
for name, values := range map[string]url.Values{
"IDのみ": {"id": {"1"}},
"番組のみ": {"series_id": {"series2"}},
"両方と重複除去": {"id": {"1"}, "series_id": {"series1"}},
} {
t.Run(name, func(t *testing.T) {
response := postForm(handler, "/library/delete/confirm", values)
if response.Code != http.StatusOK || len(jobs.jobs) != 3 {
t.Fatalf("status=%d jobs=%d body=%s", response.Code, len(jobs.jobs), response.Body.String())
}
if strings.Contains(response.Body.String(), `<img src=x`) || strings.Contains(response.Body.String(), `<script>番組</script>`) {
t.Fatal("確認画面でHTMLがエスケープされていない")
}
if name == "両方と重複除去" && (!strings.Contains(response.Body.String(), `&lt;script&gt;番組&lt;/script&gt;`) || !strings.Contains(response.Body.String(), `&lt;img src=x onerror=alert(1)&gt;`)) {
t.Fatalf("番組名または回タイトルのエスケープ結果がない: %s", response.Body.String())
}
if name == "両方と重複除去" && strings.Count(response.Body.String(), `name="id"`) != 2 {
t.Fatalf("重複除去後のhidden ID: %s", response.Body.String())
}
})
}
for name, values := range map[string]url.Values{
"対象なし": {},
"該当なし": {"id": {"99"}},
"不正ID": {"id": {"bad"}},
"ゼロID": {"id": {"0"}},
"不正番組ID": {"series_id": {"bad-id"}},
} {
t.Run(name, func(t *testing.T) {
if got := postForm(handler, "/library/delete/confirm", values).Code; got != http.StatusBadRequest {
t.Fatalf("status=%d", got)
}
})
}
tooMany := url.Values{}
for i := 1; i <= maxDeleteSelections+1; i++ {
tooMany.Add("id", strconv.Itoa(i))
}
if got := postForm(handler, "/library/delete/confirm", tooMany).Code; got != http.StatusBadRequest {
t.Fatalf("上限超過 status=%d", got)
}
atLimit := url.Values{}
for i := 1; i <= maxDeleteSelections; i++ {
atLimit.Add("id", strconv.Itoa(i))
}
if response := postForm(handler, "/library/delete/confirm", atLimit); response.Code != http.StatusOK || len(jobs.jobs) != 3 {
t.Fatalf("入力上限ちょうど: status=%d jobs=%d", response.Code, len(jobs.jobs))
}
}
func TestDeleteConfirmationRejectsExpandedSelectionOverLimit(t *testing.T) {
makeJobs := func(count int) map[string]store.Job {
jobs := make(map[string]store.Job, count)
for i := 1; i <= count; i++ {
job := libraryJob(int64(i), "show/video.mp4")
job.EpisodeID = "ep" + strconv.Itoa(i)
jobs[job.EpisodeID] = job
}
return jobs
}
atLimit := &fakeJobStore{jobs: makeJobs(maxDeleteSelections)}
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, atLimit, t.TempDir(), nil),
"/library/delete/confirm", url.Values{"series_id": {"series1"}})
if response.Code != http.StatusOK || len(atLimit.jobs) != maxDeleteSelections {
t.Fatalf("上限ちょうど: status=%d jobs=%d", response.Code, len(atLimit.jobs))
}
overLimit := &fakeJobStore{jobs: makeJobs(maxDeleteSelections + 1)}
response = postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, overLimit, t.TempDir(), nil),
"/library/delete/confirm", url.Values{"series_id": {"series1"}})
if response.Code != http.StatusBadRequest || len(overLimit.jobs) != maxDeleteSelections+1 ||
!strings.Contains(response.Body.String(), strconv.Itoa(maxDeleteSelections+1)+"本") ||
!strings.Contains(response.Body.String(), "番組を分けて選んでください") {
t.Fatalf("上限超過: status=%d jobs=%d body=%s", response.Code, len(overLimit.jobs), response.Body.String())
}
}
func TestBulkDeleteContinuesAfterFailureAndDoesNotExpandSeries(t *testing.T) {
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "show"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "show", "one.mp4"), []byte("one"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Mkdir(filepath.Join(dir, "show", "blocked"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "show", "blocked", "child"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
first := libraryJob(1, "show/one.mp4")
blocked := libraryJob(2, "show/blocked")
blocked.EpisodeID = "ep2"
blocked.Title = "削除失敗回"
newAfterConfirmation := libraryJob(3, "show/new.mp4")
newAfterConfirmation.EpisodeID = "ep3"
newAfterConfirmation.Title = "確認後の新着"
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": first, "ep2": blocked, "ep3": newAfterConfirmation}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
response := postForm(handler, "/library/delete", url.Values{
"id": {"1", "2", "99"},
"series_id": {"series1"},
})
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "削除: 1件") || !strings.Contains(response.Body.String(), "失敗: 1件") || !strings.Contains(response.Body.String(), "対象外: 1件") {
t.Fatalf("status=%d body=%s", response.Code, response.Body.String())
}
if _, ok := jobs.jobs["ep1"]; ok {
t.Fatal("正常な回の記録が残った")
}
if _, ok := jobs.jobs["ep2"]; !ok {
t.Fatal("失敗した回の記録が消えた")
}
if _, ok := jobs.jobs["ep3"]; !ok {
t.Fatal("確認後に増えた回が消えた")
}
if strings.Contains(response.Body.String(), filepath.Join(dir, "show", "blocked")) {
t.Fatal("内部パスが結果画面に出た")
}
for _, detail := range []string{"directory not empty", "show/blocked"} {
if strings.Contains(response.Body.String(), detail) {
t.Fatalf("エラー詳細が結果画面に出た: %q", detail)
}
}
}
func TestBulkDeleteWhenVideoRootCannotOpenDeletesOnlyRecordWithoutFile(t *testing.T) {
withFile := libraryJob(1, "show/video.mp4")
withFile.Title = `<img src=x onerror=alert(1)>`
withoutFile := libraryJob(2, "")
withoutFile.EpisodeID = "ep2"
withoutFile.Title = "記録だけの回"
withoutFile.FilePath = nil
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": withFile, "ep2": withoutFile}}
missingRoot := filepath.Join(t.TempDir(), "missing-video-root")
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, missingRoot, nil),
"/library/delete", url.Values{"id": {"1", "2"}})
if response.Code != http.StatusOK ||
!strings.Contains(response.Body.String(), "削除: 1件") ||
!strings.Contains(response.Body.String(), "失敗: 1件") {
t.Fatalf("status=%d body=%s", response.Code, response.Body.String())
}
if _, ok := jobs.jobs["ep1"]; !ok {
t.Fatal("動画ルートを開けない回の記録が消えた")
}
if _, ok := jobs.jobs["ep2"]; ok {
t.Fatal("file_pathがNULLの回の記録が残った")
}
for _, secret := range []string{missingRoot, "no such file or directory", `<img src=x`} {
if strings.Contains(response.Body.String(), secret) {
t.Fatalf("内部情報または未エスケープHTMLが結果画面に出た: %q", secret)
}
}
if !strings.Contains(response.Body.String(), `&lt;img src=x onerror=alert(1)&gt;`) {
t.Fatalf("失敗した回のタイトルが安全に表示されていない: %s", response.Body.String())
}
}
func TestBulkDeleteRejectsEmptySelection(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil), "/library/delete", nil)
if response.Code != http.StatusBadRequest || len(jobs.jobs) != 1 {
t.Fatalf("status=%d jobs=%d", response.Code, len(jobs.jobs))
}
}
func TestLibraryDeleteEntryFormsAndOldRoute(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
for _, path := range []string{"/library", "/library/series/series1", "/library/1"} {
response := getPage(handler, path)
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), `action="/library/delete/confirm"`) {
t.Fatalf("%s: status=%d body=%s", path, response.Code, response.Body.String())
}
}
if body := getPage(handler, "/library/1").Body.String(); strings.Contains(body, "hx-confirm") {
t.Fatal("再生ページにhx-confirmが残っている")
}
if got := postForm(handler, "/library/1/delete", nil).Code; got != http.StatusNotFound && got != http.StatusMethodNotAllowed {
t.Fatalf("旧経路 status=%d", got)
}
}
+101
View File
@@ -0,0 +1,101 @@
package web
import (
"sort"
"strings"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
func sortJobsNaturally(jobs []store.Job) {
sort.SliceStable(jobs, func(i, j int) bool {
if comparison := compareNaturalTitle(jobs[i].Title, jobs[j].Title); comparison != 0 {
return comparison < 0
}
left, right := jobs[i].FinishedAt, jobs[j].FinishedAt
if left == nil || right == nil {
if left != right {
return left == nil
}
} else if !left.Equal(*right) {
return left.Before(*right)
}
return jobs[i].ID < jobs[j].ID
})
}
func compareNaturalTitle(left, right string) int {
a, b := []rune(left), []rune(right)
ai, bi := 0, 0
for ai < len(a) && bi < len(b) {
_, aIsDigit := decimalDigit(a[ai])
_, bIsDigit := decimalDigit(b[bi])
if aIsDigit && bIsDigit {
var an, bn strings.Builder
for ai < len(a) {
digit, ok := decimalDigit(a[ai])
if !ok {
break
}
an.WriteByte(byte('0' + digit))
ai++
}
for bi < len(b) {
digit, ok := decimalDigit(b[bi])
if !ok {
break
}
bn.WriteByte(byte('0' + digit))
bi++
}
if comparison := compareDecimal(an.String(), bn.String()); comparison != 0 {
return comparison
}
continue
}
if a[ai] < b[bi] {
return -1
}
if a[ai] > b[bi] {
return 1
}
ai++
bi++
}
if ai == len(a) && bi < len(b) {
return -1
}
if bi == len(b) && ai < len(a) {
return 1
}
return 0
}
func decimalDigit(r rune) (int, bool) {
switch {
case '0' <= r && r <= '9':
return int(r - '0'), true
case '0' <= r && r <= '9':
return int(r - '0'), true
default:
return 0, false
}
}
func compareDecimal(left, right string) int {
left = strings.TrimLeft(left, "0")
right = strings.TrimLeft(right, "0")
if left == "" {
left = "0"
}
if right == "" {
right = "0"
}
if len(left) < len(right) {
return -1
}
if len(left) > len(right) {
return 1
}
return strings.Compare(left, right)
}
+60
View File
@@ -0,0 +1,60 @@
package web
import (
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
func TestSortJobsNaturally(t *testing.T) {
base := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
jobs := []store.Job{
{ID: 8, Title: "第10話", FinishedAt: timePointer(base)},
{ID: 7, Title: "第2話", FinishedAt: timePointer(base)},
{ID: 6, Title: "#10", FinishedAt: timePointer(base)},
{ID: 5, Title: "#9", FinishedAt: timePointer(base)},
{ID: 4, Title: "第2話", FinishedAt: timePointer(base.Add(time.Hour))},
{ID: 3, Title: "第2話", FinishedAt: timePointer(base.Add(-time.Hour))},
{ID: 2, Title: "第02話", FinishedAt: timePointer(base.Add(-time.Hour))},
{ID: 1, Title: "おまけ", FinishedAt: timePointer(base)},
}
sortJobsNaturally(jobs)
got := make([]int64, len(jobs))
for i := range jobs {
got[i] = jobs[i].ID
}
want := []int64{5, 6, 1, 2, 3, 7, 4, 8}
for i := range want {
if got[i] != want[i] {
t.Fatalf("自然順=%v want=%v", got, want)
}
}
}
func TestNaturalOrderUsesIDAfterEqualTitleAndFinishedAt(t *testing.T) {
finished := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
jobs := []store.Job{
{ID: 20, Title: "#9", FinishedAt: &finished},
{ID: 10, Title: "#9", FinishedAt: &finished},
}
sortJobsNaturally(jobs)
if jobs[0].ID != 10 || jobs[1].ID != 20 {
t.Fatalf("同順位のID順: %+v", jobs)
}
}
func TestNaturalOrderTreatsLeadingZerosAsTheSameNumber(t *testing.T) {
base := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
jobs := []store.Job{
{ID: 30, Title: "数字なし", FinishedAt: timePointer(base.Add(-time.Hour))},
{ID: 20, Title: "#09", FinishedAt: timePointer(base)},
{ID: 10, Title: "#9", FinishedAt: timePointer(base.Add(time.Hour))},
}
sortJobsNaturally(jobs)
if jobs[0].ID != 20 || jobs[1].ID != 10 || jobs[2].ID != 30 {
t.Fatalf("先頭ゼロと数字なしの自然順: %+v", jobs)
}
}
func timePointer(value time.Time) *time.Time { return &value }
+240
View File
@@ -0,0 +1,240 @@
package web
import (
"bytes"
"context"
"fmt"
"net/http"
"net/url"
"regexp"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
const (
newerCacheTTL = 10 * time.Minute
)
var platformTokenPattern = regexp.MustCompile(`(?i)(platform_token=)[^&\s]+`)
type NewerFetcher interface {
FetchNewer(context.Context, string) ([]tver.Episode, error)
FetchSeries(context.Context, string) (string, error)
FetchSeriesEpisodes(context.Context, string) ([]tver.Episode, error)
}
type cacheEntry struct {
episodes []tver.Episode
expiresAt time.Time
}
type newerOptions struct {
genre string
hidePreviews bool
}
type episodeView struct {
ID string
SeriesID string
SubscriptionAction subscriptionActionView
Genre string
Status string
StatusText string
URL string
ThumbnailURL string
SeriesTitle string
Title string
BroadcasterName string
DurationMinutes int64
BroadcastDateLabel string
EndAtText string
EndAtISO string
}
type newerView struct {
Genre string
HidePreviews bool
Episodes []episodeView
Failed bool
ErrorText string
}
func parseNewerOptions(r *http.Request) (newerOptions, error) {
options := newerOptions{genre: "drama", hidePreviews: true}
query := r.URL.Query()
if values, ok := query["genre"]; ok {
options.genre = values[len(values)-1]
}
if options.genre != "drama" && options.genre != "variety" {
return newerOptions{}, fmt.Errorf("invalid genre")
}
if values, ok := query["hide_previews"]; ok {
switch values[len(values)-1] {
case "0":
options.hidePreviews = false
case "1":
options.hidePreviews = true
default:
return newerOptions{}, fmt.Errorf("invalid hide_previews")
}
}
return options, nil
}
func (options newerOptions) pageURL() string {
query := url.Values{}
query.Set("genre", options.genre)
if options.hidePreviews {
query.Set("hide_previews", "1")
} else {
query.Set("hide_previews", "0")
}
return "/?" + query.Encode()
}
func (s *server) home(w http.ResponseWriter, r *http.Request) {
options, err := parseNewerOptions(r)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
view := s.newerView(r.Context(), options)
s.render(w, "layout", page{Title: "新着", Newer: &view})
}
func (s *server) newerFragment(w http.ResponseWriter, r *http.Request) {
options, err := parseNewerOptions(r)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
view := s.newerView(r.Context(), options)
w.Header().Set("HX-Push-Url", options.pageURL())
s.render(w, "newer_results", view)
}
func (s *server) render(w http.ResponseWriter, name string, value any) {
var body bytes.Buffer
if err := s.tmpl.ExecuteTemplate(&body, name, value); err != nil {
s.logger.Printf("テンプレートの描画に失敗: %v", err)
http.Error(w, "template error", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = body.WriteTo(w)
}
func (s *server) newerView(ctx context.Context, options newerOptions) newerView {
view := newerView{Genre: options.genre, HidePreviews: options.hidePreviews}
episodes, err := s.cachedNewer(ctx, options.genre)
if err != nil {
s.logger.Printf("TVer の新着取得に失敗(genre=%s): %s", options.genre, platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
view.Failed = true
view.ErrorText = "新着を取得できませんでした"
return view
}
visible := make([]tver.Episode, 0, len(episodes))
for _, episode := range episodes {
if options.hidePreviews && episode.Duration <= tver.PreviewMaxSeconds {
continue
}
visible = append(visible, episode)
}
ids := make([]string, 0, len(visible))
for _, episode := range visible {
ids = append(ids, episode.ID)
}
statuses, err := s.jobs.EpisodeStatuses(ctx, ids)
if err != nil {
s.logger.Printf("新着のジョブ状態取得に失敗: %v", err)
view.Failed = true
view.ErrorText = "状態を取得できませんでした"
return view
}
seriesIDs := make([]string, 0, len(visible))
seenSeries := make(map[string]bool)
for _, episode := range visible {
if episode.SeriesID != "" && !seenSeries[episode.SeriesID] {
seriesIDs = append(seriesIDs, episode.SeriesID)
seenSeries[episode.SeriesID] = true
}
}
subscribed, err := s.jobs.SubscribedSeries(ctx, seriesIDs)
if err != nil {
s.logger.Printf("登録番組の状態取得に失敗: %v", err)
view.Failed = true
view.ErrorText = "登録番組の状態を取得できませんでした"
return view
}
for _, episode := range visible {
endAt := time.Unix(episode.EndAt, 0).In(s.zone)
view.Episodes = append(view.Episodes, episodeView{
ID: episode.ID,
SeriesID: episode.SeriesID,
SubscriptionAction: subscriptionActionView{ID: episode.ID, Genre: options.genre, Subscribed: subscribed[episode.SeriesID]},
Genre: options.genre,
Status: statuses[episode.ID],
StatusText: statusText(statuses[episode.ID]),
URL: "https://tver.jp/episodes/" + url.PathEscape(episode.ID),
ThumbnailURL: "https://statics.tver.jp" + episode.ThumbnailPath,
SeriesTitle: episode.SeriesTitle,
Title: episode.Title,
BroadcasterName: episode.BroadcasterName,
DurationMinutes: durationMinutes(episode.Duration),
BroadcastDateLabel: episode.BroadcastDateLabel,
EndAtText: endAt.Format("2006-01-02 15:04 MST"),
EndAtISO: endAt.Format(time.RFC3339),
})
}
return view
}
func (s *server) cachedEpisode(genre, id string) (tver.Episode, bool) {
s.cacheMu.Lock()
defer s.cacheMu.Unlock()
entry, ok := s.cache[genre]
if !ok || !s.now().Before(entry.expiresAt) {
return tver.Episode{}, false
}
for _, ep := range entry.episodes {
if ep.ID == id {
return ep, true
}
}
return tver.Episode{}, false
}
func storeEpisode(ep tver.Episode) store.Episode {
var endAt *time.Time
if ep.EndAt > 0 {
end := time.Unix(ep.EndAt, 0)
endAt = &end
}
return store.Episode{ID: ep.ID, SeriesID: ep.SeriesID, SeriesTitle: ep.SeriesTitle,
Title: ep.Title, BroadcasterName: ep.BroadcasterName, DurationSeconds: int(ep.Duration),
EndAt: endAt, ThumbnailPath: ep.ThumbnailPath}
}
func durationMinutes(seconds int64) int64 {
minutes := seconds / 60
if seconds%60 != 0 {
minutes++
}
return max(1, minutes)
}
func (s *server) cachedNewer(ctx context.Context, genre string) ([]tver.Episode, error) {
s.cacheMu.Lock()
defer s.cacheMu.Unlock()
if cached, ok := s.cache[genre]; ok && s.now().Before(cached.expiresAt) {
return cached.episodes, nil
}
episodes, err := s.fetcher.FetchNewer(ctx, genre)
if err != nil {
return nil, err
}
s.cache[genre] = cacheEntry{episodes: episodes, expiresAt: s.now().Add(newerCacheTTL)}
return episodes, nil
}
+271
View File
@@ -0,0 +1,271 @@
package web
import (
"bytes"
"context"
"errors"
"io"
"log"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
type fakeFetcher struct {
episodes []tver.Episode
episodesByGenre map[string][]tver.Episode
err error
calls []string
seriesEpisodes []tver.Episode
seriesErr error
seriesCalls []string
seriesTitle string
seriesTitleErr error
seriesTitleCalls []string
}
func (f *fakeFetcher) FetchSeries(_ context.Context, seriesID string) (string, error) {
f.seriesTitleCalls = append(f.seriesTitleCalls, seriesID)
return f.seriesTitle, f.seriesTitleErr
}
func (f *fakeFetcher) FetchNewer(_ context.Context, genre string) ([]tver.Episode, error) {
f.calls = append(f.calls, genre)
if f.episodesByGenre != nil {
return f.episodesByGenre[genre], f.err
}
return f.episodes, f.err
}
func (f *fakeFetcher) FetchSeriesEpisodes(_ context.Context, seriesID string) ([]tver.Episode, error) {
f.seriesCalls = append(f.seriesCalls, seriesID)
return f.seriesEpisodes, f.seriesErr
}
func newTestHandler(fetcher *fakeFetcher, now func() time.Time) http.Handler {
return newTestHandlerWithStore(fetcher, &fakeJobStore{}, now)
}
func newTestHandlerWithStore(fetcher *fakeFetcher, jobs *fakeJobStore, now func() time.Time) http.Handler {
return newTestHandlerWithVideoDir(fetcher, jobs, "", now)
}
func newTestHandlerWithVideoDir(fetcher *fakeFetcher, jobs *fakeJobStore, videoDir string, now func() time.Time) http.Handler {
if now == nil {
now = func() time.Time { return time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC) }
}
return NewHandler(fetcher, jobs, videoDir, now, log.New(io.Discard, "", 0))
}
func getPage(handler http.Handler, path string) *httptest.ResponseRecorder {
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, path, nil))
return response
}
func TestNewerPageShowsEpisodeFieldsAndJST(t *testing.T) {
endAt := time.Date(2026, 10, 3, 21, 34, 0, 0, time.UTC).Unix()
fetcher := &fakeFetcher{episodes: []tver.Episode{{
ID: "ep123", SeriesTitle: "番組名", Title: "第1話", BroadcasterName: "放送局",
Duration: 2712, EndAt: endAt, ThumbnailPath: "/images/ep123.jpg",
BroadcastDateLabel: "10月3日放送分",
}}}
response := getPage(newTestHandler(fetcher, nil), "/")
if response.Code != http.StatusOK {
t.Fatalf("status = %d", response.Code)
}
body := response.Body.String()
for _, want := range []string{
"番組名", "第1話", "放送局", "10月3日放送分",
"長さ: 46分",
"https://statics.tver.jp/images/ep123.jpg",
"https://tver.jp/episodes/ep123",
"2026-10-04 06:34 JST", `datetime="2026-10-04T06:34:00&#43;09:00"`,
`value="drama" selected`, `name="hide_previews" value="1" checked`,
`hx-get="/newer"`, `hx-target="#newer-results"`,
} {
if !strings.Contains(body, want) {
t.Errorf("page missing %q", want)
}
}
if len(fetcher.calls) != 1 || fetcher.calls[0] != "drama" {
t.Errorf("calls = %v", fetcher.calls)
}
}
func TestDurationMinutesRoundsUp(t *testing.T) {
for _, tt := range []struct {
seconds int64
minutes int64
}{
{0, 1}, {1, 1}, {59, 1}, {60, 1}, {61, 2}, {2712, 46},
} {
if got := durationMinutes(tt.seconds); got != tt.minutes {
t.Errorf("%d seconds = %d minutes; want %d", tt.seconds, got, tt.minutes)
}
}
}
func TestNewerFragmentUpdatesURLAndFullPageRestoresState(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "variety-1", Duration: 200}}}
handler := newTestHandler(fetcher, nil)
response := getPage(handler, "/newer?genre=variety&hide_previews=0")
if response.Code != http.StatusOK {
t.Fatalf("fragment status = %d", response.Code)
}
if got := response.Header().Get("HX-Push-Url"); got != "/?genre=variety&hide_previews=0" {
t.Errorf("HX-Push-Url = %q", got)
}
if body := response.Body.String(); !strings.Contains(body, `id="newer-results"`) || !strings.Contains(body, "variety-1") || strings.Contains(body, "<html") {
t.Errorf("not an episode fragment: %s", body)
}
reload := getPage(handler, "/?genre=variety&hide_previews=0")
if reload.Code != http.StatusOK || !strings.Contains(reload.Body.String(), `value="variety" selected`) || strings.Contains(reload.Body.String(), `value="1" checked`) || !strings.Contains(reload.Body.String(), "variety-1") {
t.Errorf("reload did not restore state: status=%d body=%s", reload.Code, reload.Body.String())
}
if len(fetcher.calls) != 1 || fetcher.calls[0] != "variety" {
t.Errorf("calls = %v", fetcher.calls)
}
}
func TestNewerInvalidGenreReturns400WithoutFetch(t *testing.T) {
fetcher := &fakeFetcher{}
handler := newTestHandler(fetcher, nil)
for _, path := range []string{"/?genre=news", "/newer?genre=news", "/?genre=", "/?hide_previews=invalid", "/newer?hide_previews="} {
response := getPage(handler, path)
if response.Code != http.StatusBadRequest {
t.Errorf("%s: status = %d", path, response.Code)
}
}
if len(fetcher.calls) != 0 {
t.Errorf("unexpected fetch calls: %v", fetcher.calls)
}
}
func TestPreviewFilterBoundaryAndToggle(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{
{ID: "preview-180", Duration: 180},
{ID: "episode-181", Duration: 181},
}}
handler := newTestHandler(fetcher, nil)
hidden := getPage(handler, "/")
if strings.Contains(hidden.Body.String(), "preview-180") || !strings.Contains(hidden.Body.String(), "episode-181") {
t.Errorf("default filter: %s", hidden.Body.String())
}
shown := getPage(handler, "/newer?genre=drama&hide_previews=0")
if !strings.Contains(shown.Body.String(), "preview-180") || !strings.Contains(shown.Body.String(), "episode-181") {
t.Errorf("show previews: %s", shown.Body.String())
}
checked := getPage(handler, "/newer?genre=drama&hide_previews=0&hide_previews=1")
if strings.Contains(checked.Body.String(), "preview-180") || checked.Header().Get("HX-Push-Url") != "/?genre=drama&hide_previews=1" {
t.Errorf("checked form state: %s", checked.Body.String())
}
if len(fetcher.calls) != 1 {
t.Errorf("filter caused %d TVer fetches; want 1", len(fetcher.calls))
}
}
func TestNewerCacheExpiresPerGenre(t *testing.T) {
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}
handler := newTestHandler(fetcher, func() time.Time { return now })
getPage(handler, "/")
now = now.Add(9*time.Minute + 59*time.Second)
getPage(handler, "/?genre=drama&hide_previews=0")
getPage(handler, "/?genre=variety")
if got := strings.Join(fetcher.calls, ","); got != "drama,variety" {
t.Fatalf("before expiry calls = %q", got)
}
now = now.Add(2 * time.Second)
getPage(handler, "/")
getPage(handler, "/?genre=variety")
if got := strings.Join(fetcher.calls, ","); got != "drama,variety,drama" {
t.Errorf("after expiry calls = %q", got)
}
}
func TestNewerCacheKeepsGenreResultsSeparateAndExpiresAtTenMinutes(t *testing.T) {
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
fetcher := &fakeFetcher{episodesByGenre: map[string][]tver.Episode{
"drama": {{ID: "drama-only", Duration: 200}},
"variety": {{ID: "variety-only", Duration: 200}},
}}
handler := newTestHandler(fetcher, func() time.Time { return now })
checkPage := func(path, wantID, absentID string) {
t.Helper()
response := getPage(handler, path)
body := response.Body.String()
if response.Code != http.StatusOK || !strings.Contains(body, "/episodes/"+wantID) || strings.Contains(body, "/episodes/"+absentID) {
t.Errorf("%s at %s: status=%d, wrong genre contents: %s", path, now.Format(time.RFC3339), response.Code, body)
}
}
checkCalls := func(want string) {
t.Helper()
if got := strings.Join(fetcher.calls, ","); got != want {
t.Errorf("fetch calls = %q; want %q", got, want)
}
}
checkPage("/", "drama-only", "variety-only")
now = now.Add(5 * time.Minute)
checkPage("/?genre=variety", "variety-only", "drama-only")
now = now.Add(4*time.Minute + 59*time.Second)
checkPage("/", "drama-only", "variety-only")
checkCalls("drama,variety")
now = now.Add(time.Second)
checkPage("/", "drama-only", "variety-only")
checkPage("/?genre=variety", "variety-only", "drama-only")
checkCalls("drama,variety,drama")
now = now.Add(5 * time.Minute)
checkPage("/?genre=variety", "variety-only", "drama-only")
checkCalls("drama,variety,drama,variety")
}
func TestNewerPageEscapesUntrustedEpisodeFields(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{
ID: "safe-id", SeriesTitle: `<script>alert(1)</script>`, Title: `<b>title</b>`,
BroadcasterName: `<img src=x onerror=alert(1)>`, BroadcastDateLabel: `<svg onload=alert(1)>`,
ThumbnailPath: `/image.jpg" onerror="alert(1)`, Duration: 200,
}}}
response := getPage(newTestHandler(fetcher, nil), "/")
if response.Code != http.StatusOK {
t.Fatalf("status = %d", response.Code)
}
body := response.Body.String()
for _, unsafe := range []string{"<script>alert(1)</script>", "<b>title</b>", "<img src=x", "<svg onload", `onerror="alert(1)"`} {
if strings.Contains(body, unsafe) {
t.Errorf("untrusted markup was not escaped: %q", unsafe)
}
}
for _, escaped := range []string{"&lt;script&gt;", "&lt;b&gt;title&lt;/b&gt;", "&lt;img src=x onerror=alert(1)&gt;", "&lt;svg onload=alert(1)&gt;"} {
if !strings.Contains(body, escaped) {
t.Errorf("escaped field missing: %q", escaped)
}
}
}
func TestNewerFailureShowsFriendlyMessageAndRedactsToken(t *testing.T) {
fetcher := &fakeFetcher{err: errors.New("GET https://platform-api.tver.jp/?platform_token=secret-123&x=1: upstream unavailable")}
var logs bytes.Buffer
handler := NewHandler(fetcher, &fakeJobStore{}, "", time.Now, log.New(&logs, "", 0))
response := getPage(handler, "/")
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "新着を取得できませんでした") {
t.Errorf("error page: status=%d body=%s", response.Code, response.Body.String())
}
if strings.Contains(response.Body.String(), "upstream") || strings.Contains(response.Body.String(), "secret-123") {
t.Errorf("error detail leaked into page: %s", response.Body.String())
}
fragment := getPage(handler, "/newer?genre=variety&hide_previews=0")
if fragment.Code != http.StatusOK || !strings.Contains(fragment.Body.String(), `id="newer-results"`) || !strings.Contains(fragment.Body.String(), "新着を取得できませんでした") || strings.Contains(fragment.Body.String(), "<html") {
t.Errorf("error fragment: status=%d body=%s", fragment.Code, fragment.Body.String())
}
if fragment.Header().Get("HX-Push-Url") != "/?genre=variety&hide_previews=0" || strings.Contains(fragment.Body.String(), "secret-123") {
t.Errorf("error fragment URL or token leak: header=%q body=%s", fragment.Header().Get("HX-Push-Url"), fragment.Body.String())
}
if !strings.Contains(logs.String(), "upstream unavailable") || strings.Contains(logs.String(), "secret-123") {
t.Errorf("log detail or token redaction failed: %s", logs.String())
}
}
+194
View File
@@ -0,0 +1,194 @@
package web
import (
"context"
"errors"
"fmt"
"net/http"
"strconv"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
const queueListLimit = 50
type queueJobView struct {
store.Job
SourceText, CreatedAtText, PhaseText string
PercentText, SpeedText, ETAText string
FragmentText, ErrorKindText string
}
type queueView struct {
Running, Queued, Failed, Canceled, Succeeded []queueJobView
Poll bool
}
func (s *server) queuePage(w http.ResponseWriter, r *http.Request) {
view, err := s.loadQueue(r.Context())
if err != nil {
s.logger.Printf("待ち行列の取得に失敗: %v", err)
http.Error(w, "待ち行列を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "layout", page{Title: "待ち行列", Queue: &view})
}
func (s *server) queueFragment(w http.ResponseWriter, r *http.Request) {
view, err := s.loadQueue(r.Context())
if err != nil {
s.logger.Printf("待ち行列の取得に失敗: %v", err)
http.Error(w, "待ち行列を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "queue_list", view)
}
func (s *server) loadQueue(ctx context.Context) (queueView, error) {
var view queueView
for _, item := range []struct {
status string
assign func([]queueJobView)
}{
{"running", func(j []queueJobView) { view.Running = j }},
{"queued", func(j []queueJobView) { view.Queued = j }},
{"failed", func(j []queueJobView) { view.Failed = j }},
{"canceled", func(j []queueJobView) { view.Canceled = j }},
{"succeeded", func(j []queueJobView) { view.Succeeded = j }},
} {
jobs, err := s.jobs.ListJobs(ctx, item.status, queueListLimit)
if err != nil {
return queueView{}, err
}
views := make([]queueJobView, 0, len(jobs))
for _, job := range jobs {
views = append(views, s.queueJob(job))
}
item.assign(views)
}
view.Poll = len(view.Running) > 0 || len(view.Queued) > 0
return view, nil
}
func (s *server) queueJob(job store.Job) queueJobView {
view := queueJobView{Job: job, CreatedAtText: job.CreatedAt.In(s.zone).Format("2006-01-02 15:04 MST")}
if job.Source == "subscription" {
view.SourceText = "自動"
} else {
view.SourceText = "手動"
}
if job.Phase != nil {
switch *job.Phase {
case "video":
view.PhaseText = "映像"
case "audio":
view.PhaseText = "音声"
case "merging":
view.PhaseText = "結合"
case "moving":
view.PhaseText = "移動"
}
}
if job.DownloadedBytes != nil && job.TotalBytes != nil && *job.TotalBytes > 0 {
percent := 100 * float64(*job.DownloadedBytes) / float64(*job.TotalBytes)
percent = max(0, min(percent, 100))
if job.TotalIsEstimate {
view.PercentText = fmt.Sprintf("約%.0f%%", percent)
} else {
view.PercentText = fmt.Sprintf("%.0f%%", percent)
}
}
if job.SpeedBPS != nil && *job.SpeedBPS > 0 {
view.SpeedText = formatBytes(*job.SpeedBPS) + "/秒"
}
if job.ETASeconds != nil && *job.ETASeconds >= 0 {
seconds := int64(*job.ETASeconds)
view.ETAText = fmt.Sprintf("残り%d分%d秒", seconds/60, seconds%60)
}
if job.FragmentIndex != nil && job.FragmentCount != nil && *job.FragmentCount > 0 {
view.FragmentText = fmt.Sprintf("断片 %d/%d", *job.FragmentIndex, *job.FragmentCount)
}
if job.ErrorKind != nil {
view.ErrorKindText = errorKindText(*job.ErrorKind)
}
return view
}
func formatBytes(n float64) string {
for _, unit := range []string{"B", "KiB", "MiB", "GiB"} {
if n < 1024 || unit == "GiB" {
return fmt.Sprintf("%.1f %s", n, unit)
}
n /= 1024
}
return ""
}
func errorKindText(kind string) string {
switch kind {
case "drm":
return "DRM のため取得不可"
case "geo":
return "地域制限"
case "network":
return "ネットワーク"
case "unavailable":
return "配信終了・存在しない"
case "extract":
return "抽出失敗"
case "invalid_output":
return "出力形式の不正"
default:
return "不明"
}
}
func (s *server) retryJob(w http.ResponseWriter, r *http.Request) {
s.queueMutation(w, r, s.jobs.Retry)
}
func (s *server) cancelJob(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.Error(w, "ジョブ ID が不正です", http.StatusBadRequest)
return
}
job, err := s.jobs.GetJob(r.Context(), id)
if errors.Is(err, pgx.ErrNoRows) {
http.Error(w, "ジョブが見つかりません", http.StatusNotFound)
return
}
if err != nil {
s.logger.Printf("ジョブ状態の取得に失敗: %v", err)
http.Error(w, "操作できませんでした", http.StatusInternalServerError)
return
}
if job.Status != "queued" && job.Status != "running" {
http.Error(w, "待機中またはダウンロード中のジョブだけ取り消せます", http.StatusConflict)
return
}
s.queueMutation(w, r, s.jobs.Cancel)
}
func (s *server) queueMutation(w http.ResponseWriter, r *http.Request, change func(context.Context, int64) error) {
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.Error(w, "ジョブ ID が不正です", http.StatusBadRequest)
return
}
if err := change(r.Context(), id); err != nil {
if errors.Is(err, store.ErrInvalidTransition) {
http.Error(w, "操作できる状態ではありません", http.StatusConflict)
} else {
s.logger.Printf("待ち行列の操作に失敗: %v", err)
http.Error(w, "操作できませんでした", http.StatusInternalServerError)
}
return
}
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/queue", http.StatusSeeOther)
return
}
s.queueFragment(w, r)
}
+350
View File
@@ -0,0 +1,350 @@
package web
import (
"errors"
"net/http"
"net/http/httptest"
"net/url"
"regexp"
"slices"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
func postForm(handler http.Handler, path string, values url.Values) *httptest.ResponseRecorder {
r := httptest.NewRequest(http.MethodPost, path, strings.NewReader(values.Encode()))
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
r.Header.Set("HX-Request", "true")
w := httptest.NewRecorder()
handler.ServeHTTP(w, r)
return w
}
func TestEnqueueUsesCachedEpisodeAndDuplicateStatus(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "series1", SeriesTitle: "正しい番組", Title: "正しい回", BroadcasterName: "正しい局", Duration: 1234, EndAt: 1791061200, ThumbnailPath: "/correct.jpg"}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
getPage(handler, "/")
if jobs.statusCalls != 1 || len(jobs.statusIDs) != 1 || jobs.statusIDs[0] != "ep1" {
t.Fatalf("状態の一括取得: calls=%d ids=%v", jobs.statusCalls, jobs.statusIDs)
}
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}, "series_title": {"偽番組"}, "title": {"偽回"}}
response := postForm(handler, "/episodes/enqueue", values)
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "待機中") || !strings.Contains(response.Body.String(), `class="episode-action"`) || strings.Contains(response.Body.String(), "<html") || strings.Contains(response.Body.String(), "episode-card") {
t.Fatalf("登録応答: %d %s", response.Code, response.Body.String())
}
if jobs.lastEpisode.SeriesTitle != "正しい番組" || jobs.lastEpisode.Title != "正しい回" || jobs.lastEpisode.BroadcasterName != "正しい局" || jobs.lastEpisode.DurationSeconds != 1234 || jobs.lastEpisode.ThumbnailPath != "/correct.jpg" || jobs.lastEpisode.EndAt == nil {
t.Fatalf("キャッシュ外の情報を使用: %+v", jobs.lastEpisode)
}
if len(fetcher.calls) != 1 {
t.Fatalf("POST が TVer に再問い合わせ: %v", fetcher.calls)
}
duplicate := postForm(handler, "/episodes/enqueue", values)
if duplicate.Code != http.StatusOK || !strings.Contains(duplicate.Body.String(), "待機中") || len(jobs.jobs) != 1 {
t.Fatalf("二重登録: %d %s jobs=%d", duplicate.Code, duplicate.Body.String(), len(jobs.jobs))
}
page := getPage(handler, "/")
if !strings.Contains(page.Body.String(), "待機中") || strings.Contains(page.Body.String(), ">ダウンロード</button>") {
t.Fatalf("登録後の一覧: %s", page.Body.String())
}
}
func TestEnqueueRejectsInvalidOrExpiredCache(t *testing.T) {
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", Duration: 200}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, func() time.Time { return now })
valid := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
if got := postForm(handler, "/episodes/enqueue", valid).Code; got != http.StatusNotFound {
t.Fatalf("未取得キャッシュ: %d", got)
}
getPage(handler, "/")
for _, values := range []url.Values{
{"genre": {"news"}, "episode_id": {"ep1"}},
{"genre": {"drama"}},
} {
if got := postForm(handler, "/episodes/enqueue", values).Code; got != http.StatusBadRequest {
t.Errorf("不正なフォーム: %d", got)
}
}
if got := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"drama"}, "episode_id": {"unknown"}}).Code; got != http.StatusNotFound {
t.Fatalf("同じジャンルの未収録 ID: %d", got)
}
if got := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"variety"}, "episode_id": {"ep1"}}).Code; got != http.StatusNotFound {
t.Fatalf("別ジャンルのキャッシュ: %d", got)
}
now = now.Add(newerCacheTTL)
if got := postForm(handler, "/episodes/enqueue", valid).Code; got != http.StatusNotFound {
t.Fatalf("期限切れキャッシュ: %d", got)
}
if jobs.enqueueCalls != 0 || len(fetcher.calls) != 1 {
t.Fatalf("無効な入力で登録・再取得: enqueue=%d fetch=%v", jobs.enqueueCalls, fetcher.calls)
}
}
func TestEnqueueUsesSelectedGenreCacheForSameEpisodeID(t *testing.T) {
fetcher := &fakeFetcher{episodesByGenre: map[string][]tver.Episode{
"drama": {{ID: "shared", SeriesTitle: "ドラマの番組", Duration: 200}},
"variety": {{ID: "shared", SeriesTitle: "バラエティの番組", Duration: 200}},
}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
getPage(handler, "/")
getPage(handler, "/?genre=variety")
response := postForm(handler, "/episodes/enqueue", url.Values{
"genre": {"variety"}, "episode_id": {"shared"}, "series_title": {"偽タイトル"},
})
if response.Code != http.StatusOK || jobs.lastEpisode.SeriesTitle != "バラエティの番組" || jobs.enqueueCalls != 1 || !slices.Equal(fetcher.calls, []string{"drama", "variety"}) {
t.Fatalf("選択ジャンルのキャッシュ: status=%d episode=%+v calls=%v", response.Code, jobs.lastEpisode, fetcher.calls)
}
}
func TestNewerStatusesAndRetry(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "failed", Duration: 200}, {ID: "canceled", Duration: 200}, {ID: "running", Duration: 200}, {ID: "done", Duration: 200}}}
jobs := &fakeJobStore{jobs: map[string]store.Job{
"failed": {ID: 1, Status: "failed"}, "canceled": {ID: 2, Status: "canceled"},
"running": {ID: 3, Status: "running"}, "done": {ID: 4, Status: "succeeded"},
}}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
body := getPage(handler, "/").Body.String()
for _, want := range []string{"失敗", "取り消し", "ダウンロード中", "完了"} {
if !strings.Contains(body, want) {
t.Errorf("状態表示がない: %s", want)
}
}
if got := strings.Count(body, ">再試行</button>"); got != 2 {
t.Fatalf("再試行ボタン数: %d", got)
}
response := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"failed"}})
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "待機中") || jobs.jobs["failed"].Status != "queued" {
t.Fatalf("新着から再試行: %d %s", response.Code, response.Body.String())
}
if got := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"running"}}).Code; got != http.StatusConflict {
t.Fatalf("実行中の再試行: %d", got)
}
if got := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"missing"}}).Code; got != http.StatusNotFound {
t.Fatalf("キャッシュにない ID の再試行: %d", got)
}
}
func TestQueueGroupsProgressAndPolling(t *testing.T) {
phase, errorKind, errorMessage := "video", "drm", `<script>alert(1)</script>`
downloaded, total := int64(25), int64(100)
speed, eta := float64(2048), float64(63)
index, count := int32(2), int32(10)
created := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
jobs := &fakeJobStore{jobs: map[string]store.Job{
"running": {ID: 1, Status: "running", Source: "subscription", SeriesTitle: "番組A", Title: "第1話", BroadcasterName: "局A", CreatedAt: created, Phase: &phase, DownloadedBytes: &downloaded, TotalBytes: &total, TotalIsEstimate: true, SpeedBPS: &speed, ETASeconds: &eta, FragmentIndex: &index, FragmentCount: &count},
"queued": {ID: 2, Status: "queued", Source: "manual", SeriesTitle: "番組B", CreatedAt: created},
"failed": {ID: 3, Status: "failed", ErrorKind: &errorKind, ErrorMessage: &errorMessage, CreatedAt: created},
"canceled": {ID: 4, Status: "canceled", CreatedAt: created},
"done": {ID: 5, Status: "succeeded", CreatedAt: created},
}}
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
response := getPage(handler, "/queue")
if response.Code != http.StatusOK {
t.Fatalf("待ち行列: %d", response.Code)
}
body := response.Body.String()
for _, want := range []string{"ダウンロード中", "待機中", "失敗", "取り消し", "完了(直近50件)", "番組A", "第1話", "局A", "自動", "手動", "2026-10-03 09:00 JST", "映像", "約25%", "2.0 KiB/秒", "残り1分3秒", "断片 2/10", "DRM のため取得不可", "<details>", "&lt;script&gt;", `hx-get="/queue/list"`, `hx-trigger="every 5s"`} {
if !strings.Contains(body, want) {
t.Errorf("待ち行列に %q がない", want)
}
}
if strings.Contains(body, errorMessage) || !strings.Contains(body, `action="/jobs/1/cancel"`) || strings.Contains(body, "全体") {
t.Errorf("不正な詳細・操作・全体進捗: %s", body)
}
fragment := getPage(handler, "/queue/list")
if fragment.Code != http.StatusOK || strings.Contains(fragment.Body.String(), "<html") || !strings.Contains(fragment.Body.String(), `id="queue-list"`) {
t.Errorf("待ち行列断片: %d %s", fragment.Code, fragment.Body.String())
}
}
func TestQueueJobsAppearInCorrectSectionsWithLimit(t *testing.T) {
statuses := []string{"running", "queued", "failed", "canceled", "succeeded"}
labels := []string{"ダウンロード中", "待機中", "失敗", "取り消し", "完了"}
jobs := &fakeJobStore{jobs: make(map[string]store.Job)}
for i, status := range statuses {
jobs.jobs[status] = store.Job{ID: int64(i + 1), Status: status, SeriesTitle: "番組-" + status}
}
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, jobs, nil), "/queue")
if response.Code != http.StatusOK {
t.Fatalf("待ち行列: %d", response.Code)
}
if !slices.Equal(jobs.listStatuses, statuses) {
t.Errorf("状態別取得: %v", jobs.listStatuses)
}
if !slices.Equal(jobs.listLimits, []int{50, 50, 50, 50, 50}) {
t.Errorf("状態ごとの取得上限: %v", jobs.listLimits)
}
for i, label := range labels {
section := regexp.MustCompile(`(?s)<section aria-label="` + label + `">(.*?)</section>`).FindStringSubmatch(response.Body.String())
if len(section) != 2 || !strings.Contains(section[1], "番組-"+statuses[i]) {
t.Errorf("%s に %s がない: %v", label, statuses[i], section)
continue
}
for j, other := range statuses {
if j != i && strings.Contains(section[1], "番組-"+other) {
t.Errorf("%s に %s が混入", label, other)
}
}
cancelAction := `action="/jobs/` + string(rune('1'+i)) + `/cancel"`
retryAction := `action="/jobs/` + string(rune('1'+i)) + `/retry"`
switch statuses[i] {
case "running", "queued":
if !strings.Contains(section[1], cancelAction) || strings.Contains(section[1], retryAction) {
t.Errorf("%s の操作が不正: %s", statuses[i], section[1])
}
case "failed", "canceled":
if !strings.Contains(section[1], retryAction) || strings.Contains(section[1], cancelAction) {
t.Errorf("%s の操作が不正: %s", statuses[i], section[1])
}
default:
if strings.Contains(section[1], "/cancel") || strings.Contains(section[1], "/retry") {
t.Errorf("%s に操作がある: %s", statuses[i], section[1])
}
}
}
}
func TestQueueEscapesEpisodeFields(t *testing.T) {
attack := `<script>alert(1)</script>`
message := `<img src=x onerror=alert(1)>`
kind := "extract"
jobs := &fakeJobStore{jobs: map[string]store.Job{
"bad": {ID: 1, Status: "failed", SeriesTitle: attack, Title: attack,
BroadcasterName: attack, ErrorKind: &kind, ErrorMessage: &message},
}}
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, jobs, nil), "/queue")
body := response.Body.String()
if response.Code != http.StatusOK || strings.Contains(body, attack) || strings.Contains(body, message) || strings.Count(body, "&lt;script&gt;") != 3 || !strings.Contains(body, "&lt;img src=x onerror=alert(1)&gt;") {
t.Fatalf("待ち行列の HTML エスケープ: %d %s", response.Code, body)
}
}
func TestQueueCancelRetryAndStopPolling(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep": {ID: 1, Status: "queued"}}}
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
if !strings.Contains(getPage(handler, "/queue").Body.String(), `hx-get="/queue/list"`) {
t.Fatal("待機中なのに更新がない")
}
canceled := postForm(handler, "/jobs/1/cancel", nil)
if canceled.Code != http.StatusOK || !strings.Contains(canceled.Body.String(), `action="/jobs/1/retry"`) || strings.Contains(canceled.Body.String(), `hx-get="/queue/list"`) {
t.Fatalf("取り消し後: %d %s", canceled.Code, canceled.Body.String())
}
retried := postForm(handler, "/jobs/1/retry", nil)
if retried.Code != http.StatusOK || !strings.Contains(retried.Body.String(), `action="/jobs/1/cancel"`) || !strings.Contains(retried.Body.String(), `hx-get="/queue/list"`) {
t.Fatalf("再試行後: %d %s", retried.Code, retried.Body.String())
}
if got := postForm(handler, "/jobs/1/retry", nil).Code; got != http.StatusConflict {
t.Fatalf("待機中の再試行: %d", got)
}
if got := postForm(handler, "/jobs/bad/cancel", nil).Code; got != http.StatusBadRequest {
t.Fatalf("不正な ID: %d", got)
}
if got := postForm(handler, "/jobs/99/cancel", nil).Code; got != http.StatusNotFound {
t.Fatalf("存在しないジョブ: %d", got)
}
jobs.jobs["ep"] = store.Job{ID: 1, Status: "running"}
runningCanceled := postForm(handler, "/jobs/1/cancel", nil)
if runningCanceled.Code != http.StatusOK || jobs.jobs["ep"].Status != "canceled" || !strings.Contains(runningCanceled.Body.String(), `action="/jobs/1/retry"`) {
t.Fatalf("実行中の取り消し: %d %+v", runningCanceled.Code, jobs.jobs["ep"])
}
}
func TestStateChangingGETIsRejected(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep": {ID: 1, Status: "queued"}}}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
getPage(handler, "/")
for _, path := range []string{
"/episodes/enqueue?episode_id=ep&genre=drama",
"/episodes/retry?episode_id=ep&genre=drama",
"/jobs/1/cancel", "/jobs/1/retry",
} {
response := getPage(handler, path)
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
t.Errorf("GET %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
}
}
if jobs.jobs["ep"].Status != "queued" || jobs.enqueueCalls != 0 {
t.Fatalf("GET が状態を変更した: jobs=%+v enqueue=%d", jobs.jobs, jobs.enqueueCalls)
}
}
func TestStateChangingPOSTWithoutHtmxRedirects(t *testing.T) {
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
getPage(handler, "/")
form := url.Values{"episode_id": {"ep"}, "genre": {"drama"}}
request := httptest.NewRequest(http.MethodPost, "/episodes/enqueue", strings.NewReader(form.Encode()))
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/?genre=drama" || jobs.jobs["ep"].Status != "queued" {
t.Fatalf("通常 POST 後の遷移: status=%d Location=%q jobs=%+v", response.Code, response.Header().Get("Location"), jobs.jobs)
}
request = httptest.NewRequest(http.MethodPost, "/jobs/1/cancel", nil)
response = httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/queue" || jobs.jobs["ep"].Status != "canceled" {
t.Fatalf("待ち行列操作後の遷移: status=%d Location=%q jobs=%+v", response.Code, response.Header().Get("Location"), jobs.jobs)
}
}
func TestNewerStoreFailureDoesNotOfferDownload(t *testing.T) {
jobs := &fakeJobStore{err: errors.New("db unavailable")}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
response := getPage(handler, "/")
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "状態を取得できませんでした") || strings.Contains(response.Body.String(), ">ダウンロード</button>") {
t.Errorf("状態取得失敗: %d %s", response.Code, response.Body.String())
}
}
func TestEnqueueStoreFailureShowsShortError(t *testing.T) {
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
getPage(handler, "/")
jobs.err = errors.New("secret database detail")
response := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"drama"}, "episode_id": {"ep"}})
if response.Code != http.StatusInternalServerError || !strings.Contains(response.Body.String(), "待ち行列に登録できませんでした") || strings.Contains(response.Body.String(), "secret") {
t.Errorf("DB エラーの表示: %d %s", response.Code, response.Body.String())
}
}
func TestQueueProgressAndFailureLabels(t *testing.T) {
s := &server{zone: time.FixedZone("JST", 9*60*60)}
for _, tt := range []struct{ kind, label string }{
{"drm", "DRM のため取得不可"}, {"geo", "地域制限"},
{"network", "ネットワーク"}, {"unavailable", "配信終了・存在しない"},
{"extract", "抽出失敗"}, {"invalid_output", "出力形式の不正"},
{"unknown", "不明"},
} {
if got := errorKindText(tt.kind); got != tt.label {
t.Errorf("%s: %q", tt.kind, got)
}
}
for _, tt := range []struct{ phase, label string }{
{"video", "映像"}, {"audio", "音声"}, {"merging", "結合"}, {"moving", "移動"},
} {
if got := s.queueJob(store.Job{Phase: &tt.phase}).PhaseText; got != tt.label {
t.Errorf("%s: %q", tt.phase, got)
}
}
downloaded, total := int64(50), int64(100)
job := store.Job{DownloadedBytes: &downloaded, TotalBytes: &total}
if got := s.queueJob(job).PercentText; got != "50%" {
t.Errorf("確定した段階の割合: %q", got)
}
job.TotalIsEstimate = true
if got := s.queueJob(job).PercentText; got != "約50%" {
t.Errorf("推定した段階の割合: %q", got)
}
total = 0
if got := s.queueJob(job).PercentText; got != "" {
t.Errorf("総量ゼロでの割合: %q", got)
}
}
+62
View File
@@ -0,0 +1,62 @@
body { font-family: system-ui, sans-serif; margin: 0; color: #17212b; }
header { display: flex; align-items: center; gap: 2rem; padding: 1rem 2rem; border-bottom: 1px solid #d6dee4; }
nav { display: flex; flex-wrap: wrap; gap: 1.5rem; }
nav a { color: #145a86; }
main { max-width: 72rem; margin: 2rem auto; padding: 0 2rem; }
#newer-controls { display: flex; align-items: center; flex-wrap: wrap; gap: 1rem; margin-bottom: 2rem; }
#newer-controls select, #newer-controls button { font: inherit; padding: .35rem .6rem; }
.episode-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(17rem, 1fr)); gap: 1.25rem; }
.episode-card { border: 1px solid #d6dee4; border-radius: .5rem; overflow: hidden; }
.library-selection { display: flex; align-items: center; gap: .4rem; padding: .6rem .8rem; }
.library-select-grid .episode-card { display: flex; flex-direction: column; }
.library-select-grid .episode-card > a { flex: 1; }
.episode-card a { color: inherit; text-decoration: none; }
.episode-card a:hover h2 { text-decoration: underline; }
.episode-card img { display: block; width: 100%; aspect-ratio: 16 / 9; object-fit: cover; background: #e5ebef; }
.library-card-image { position: relative; }
.episode-count-badge, .subscription-badge { display: inline-block; border-radius: 999px; padding: .15rem .5rem; font-size: .8rem; line-height: 1.2; }
.episode-count-badge { position: absolute; right: .5rem; bottom: .5rem; color: #fff; background: rgba(23, 33, 43, .85); }
.subscription-badge { color: #145a86; background: #e1f1fb; vertical-align: middle; }
.episode-copy { padding: .8rem 1rem; }
.episode-copy h2 { font-size: 1.1rem; margin: 0 0 .4rem; }
.episode-copy p { margin: .3rem 0; }
.episode-action { padding: 0 .8rem .8rem; }
.subscription-action { padding: 0 .8rem .8rem; }
.subscription-action button, .subscription-item button { font: inherit; padding: .35rem .7rem; }
.subscription-item { border: 1px solid #d6dee4; border-radius: .5rem; margin: .75rem 0; padding: .8rem 1rem; }
.subscription-item form { display: inline-block; margin-right: .5rem; }
.subscription-url-entry { border: 1px solid #d6dee4; border-radius: .5rem; margin-bottom: 1.5rem; padding: .8rem 1rem; }
.subscription-url-entry input[type="url"] { font: inherit; min-width: min(32rem, 70vw); padding: .35rem .5rem; }
.subscription-url-entry button { font: inherit; padding: .35rem .7rem; }
.episode-action form { display: inline; }
.episode-action button, .queue-job button { font: inherit; padding: .35rem .7rem; }
.queue-job { border: 1px solid #d6dee4; border-radius: .5rem; margin: .75rem 0; padding: .8rem 1rem; }
.queue-job h3 { margin: 0 0 .4rem; }
.queue-job p { margin: .4rem 0; }
.queue-job pre { white-space: pre-wrap; overflow-wrap: anywhere; }
video { display: block; width: 100%; max-height: 75vh; background: #000; }
.breadcrumbs { display: flex; flex-wrap: wrap; gap: .45rem; margin-bottom: 1rem; overflow-wrap: anywhere; }
.library-sort { margin: 1rem 0; }
.library-episode-list { border-top: 1px solid #d6dee4; }
.library-episode-select-row { display: grid; grid-template-columns: auto minmax(0, 1fr); align-items: center; border-bottom: 1px solid #d6dee4; }
.library-episode-row { display: grid; grid-template-columns: 8rem minmax(0, 1fr) auto auto auto; gap: .8rem; align-items: center; padding: .7rem 0; border-bottom: 1px solid #d6dee4; color: inherit; text-decoration: none; }
.library-episode-select-row .library-episode-row { border-bottom: 0; }
.library-episode-row:hover .library-episode-title { text-decoration: underline; }
.library-episode-row img { display: block; width: 8rem; aspect-ratio: 16 / 9; object-fit: cover; background: #e5ebef; }
.library-episode-title { min-width: 0; overflow-wrap: anywhere; font-weight: 600; }
.episode-navigation { display: flex; justify-content: space-between; gap: 1rem; margin: 1rem 0; }
.episode-navigation a, .episode-navigation span { min-width: 0; overflow-wrap: anywhere; }
.delete-warning { border-left: .3rem solid #b42318; background: #fff1f0; padding: .8rem 1rem; }
.delete-group { overflow-wrap: anywhere; }
@media (max-width: 36rem) {
header { align-items: flex-start; flex-direction: column; gap: .75rem; padding: 1rem; }
main { margin: 1.25rem auto; padding: 0 1rem; }
.episode-grid { grid-template-columns: minmax(0, 1fr); }
.library-episode-row { grid-template-columns: 6rem minmax(0, 1fr); gap: .35rem .7rem; }
.library-episode-select-row { grid-template-columns: 1fr; }
.library-episode-select-row > .library-selection { padding: .7rem 0 0; }
.library-episode-row img { grid-row: 1 / span 4; width: 6rem; }
.library-episode-row > span:not(.library-episode-title) { grid-column: 2; font-size: .9rem; }
.episode-navigation { flex-direction: column; }
}
File diff suppressed because one or more lines are too long
+13
View File
@@ -0,0 +1,13 @@
Zero-Clause BSD
=============
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted.
THE SOFTWARE IS PROVIDED “AS IS” AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE
FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY
DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+361
View File
@@ -0,0 +1,361 @@
package web
import (
"context"
"sort"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
type fakeJobStore struct {
jobs map[string]store.Job
subscriptions map[string]store.Subscription
nextID int64
lastEpisode store.Episode
enqueueCalls, statusCalls int
statusIDs []string
listStatuses []string
listLimits []int
registerCalls int
registerEpisodes []store.Episode
registerDownload bool
registerErr error
subscribedCalls int
subscribedIDs []string
beforeDeleteSucceeded func()
err error
}
func (f *fakeJobStore) ensure() {
if f.jobs == nil {
f.jobs = make(map[string]store.Job)
}
}
func (f *fakeJobStore) Enqueue(_ context.Context, ep store.Episode, source string) (store.Job, bool, error) {
f.enqueueCalls++
f.lastEpisode = ep
if f.err != nil {
return store.Job{}, false, f.err
}
f.ensure()
if job, ok := f.jobs[ep.ID]; ok {
return job, false, nil
}
f.nextID++
job := store.Job{ID: f.nextID, EpisodeID: ep.ID, SeriesTitle: ep.SeriesTitle,
Title: ep.Title, BroadcasterName: ep.BroadcasterName, Source: source,
Status: "queued", CreatedAt: time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)}
f.jobs[ep.ID] = job
return job, true, nil
}
func (f *fakeJobStore) EpisodeStatuses(_ context.Context, ids []string) (map[string]string, error) {
f.statusCalls++
f.statusIDs = append([]string(nil), ids...)
if f.err != nil {
return nil, f.err
}
result := make(map[string]string)
for _, id := range ids {
if job, ok := f.jobs[id]; ok {
result[id] = job.Status
}
}
return result, nil
}
func (f *fakeJobStore) ListJobs(_ context.Context, status string, limit int) ([]store.Job, error) {
f.listStatuses = append(f.listStatuses, status)
f.listLimits = append(f.listLimits, limit)
if f.err != nil {
return nil, f.err
}
var result []store.Job
for _, job := range f.jobs {
if job.Status == status {
result = append(result, job)
}
}
sort.Slice(result, func(i, j int) bool { return result[i].ID > result[j].ID })
if len(result) > limit {
result = result[:limit]
}
return result, nil
}
func (f *fakeJobStore) GetJob(_ context.Context, id int64) (store.Job, error) {
if f.err != nil {
return store.Job{}, f.err
}
for _, job := range f.jobs {
if job.ID == id {
return job, nil
}
}
return store.Job{}, pgx.ErrNoRows
}
func (f *fakeJobStore) Retry(_ context.Context, id int64) error {
for ep, job := range f.jobs {
if job.ID == id && (job.Status == "failed" || job.Status == "canceled") {
job.Status = "queued"
f.jobs[ep] = job
return nil
}
}
return store.ErrInvalidTransition
}
func (f *fakeJobStore) RetryEpisode(_ context.Context, id string) error {
job, ok := f.jobs[id]
if !ok || (job.Status != "failed" && job.Status != "canceled") {
return store.ErrInvalidTransition
}
job.Status = "queued"
f.jobs[id] = job
return nil
}
func (f *fakeJobStore) Cancel(_ context.Context, id int64) error {
if f.err != nil {
return f.err
}
for ep, job := range f.jobs {
if job.ID == id && (job.Status == "queued" || job.Status == "running") {
job.Status = "canceled"
f.jobs[ep] = job
return nil
}
}
return store.ErrInvalidTransition
}
func (f *fakeJobStore) CancelQueued(ctx context.Context, id int64) error {
return f.Cancel(ctx, id)
}
var _ JobStore = (*fakeJobStore)(nil)
func (f *fakeJobStore) SubscribedSeries(_ context.Context, ids []string) (map[string]bool, error) {
f.subscribedCalls++
f.subscribedIDs = append([]string(nil), ids...)
if f.err != nil {
return nil, f.err
}
result := make(map[string]bool)
for _, id := range ids {
_, result[id] = f.subscriptions[id]
}
return result, nil
}
func (f *fakeJobStore) RegisterSubscription(_ context.Context, seriesID, title string, episodes []store.Episode, download bool) (bool, error) {
f.registerCalls++
f.registerEpisodes = append([]store.Episode(nil), episodes...)
f.registerDownload = download
if f.registerErr != nil {
return false, f.registerErr
}
if f.err != nil {
return false, f.err
}
if f.subscriptions == nil {
f.subscriptions = make(map[string]store.Subscription)
}
if _, exists := f.subscriptions[seriesID]; exists {
return false, nil
}
var count int64
if download {
count = int64(len(episodes))
}
checked := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
f.subscriptions[seriesID] = store.Subscription{SeriesID: seriesID, SeriesTitle: title,
Enabled: true, CreatedAt: checked, LastCheckedAt: &checked, JobCount: count}
return true, nil
}
func (f *fakeJobStore) ListSubscriptions(_ context.Context) ([]store.Subscription, error) {
if f.err != nil {
return nil, f.err
}
var result []store.Subscription
for _, sub := range f.subscriptions {
result = append(result, sub)
}
sort.Slice(result, func(i, j int) bool { return result[i].SeriesID < result[j].SeriesID })
return result, nil
}
func (f *fakeJobStore) SetSubscriptionEnabled(_ context.Context, seriesID string, enabled bool) error {
if f.err != nil {
return f.err
}
sub, exists := f.subscriptions[seriesID]
if !exists {
return store.ErrInvalidTransition
}
sub.Enabled = enabled
f.subscriptions[seriesID] = sub
return nil
}
func (f *fakeJobStore) DeleteSubscription(_ context.Context, seriesID string) (bool, error) {
if f.err != nil {
return false, f.err
}
if _, exists := f.subscriptions[seriesID]; !exists {
return false, nil
}
delete(f.subscriptions, seriesID)
return true, nil
}
func (f *fakeJobStore) ListSucceededJobs(_ context.Context, limit int) ([]store.Job, error) {
if f.err != nil {
return nil, f.err
}
var result []store.Job
for _, job := range f.jobs {
if job.Status == "succeeded" {
result = append(result, job)
}
}
sort.Slice(result, func(i, j int) bool { return result[i].ID > result[j].ID })
if len(result) > limit {
result = result[:limit]
}
return result, nil
}
func (f *fakeJobStore) ListLibrarySeries(_ context.Context) ([]store.LibrarySeries, error) {
if f.err != nil {
return nil, f.err
}
bySeries := make(map[string]*store.LibrarySeries)
latestIDs := make(map[string]int64)
for _, job := range f.jobs {
if job.Status != "succeeded" {
continue
}
item := bySeries[job.SeriesID]
if item == nil {
item = &store.LibrarySeries{SeriesID: job.SeriesID}
bySeries[job.SeriesID] = item
}
item.EpisodeCount++
if job.FileSizeBytes != nil {
item.TotalSizeBytes += *job.FileSizeBytes
}
current := store.Job{ID: latestIDs[job.SeriesID], FinishedAt: item.LatestFinishedAt}
if item.SeriesTitle == "" || jobFinishedLater(job, current) {
item.SeriesTitle = job.SeriesTitle
item.LatestTitle = job.Title
item.BroadcasterName = job.BroadcasterName
item.ThumbnailPath = job.ThumbnailPath
if job.FinishedAt != nil {
finished := *job.FinishedAt
item.LatestFinishedAt = &finished
}
latestIDs[job.SeriesID] = job.ID
}
_, item.Subscribed = f.subscriptions[job.SeriesID]
}
result := make([]store.LibrarySeries, 0, len(bySeries))
for _, item := range bySeries {
result = append(result, *item)
}
sort.Slice(result, func(i, j int) bool {
left, right := result[i].LatestFinishedAt, result[j].LatestFinishedAt
if left == nil || right == nil {
if left != right {
return left != nil
}
} else if !left.Equal(*right) {
return left.After(*right)
}
return result[i].SeriesID < result[j].SeriesID
})
return result, nil
}
func (f *fakeJobStore) ListSucceededJobsBySeries(_ context.Context, seriesID string) ([]store.Job, error) {
if f.err != nil {
return nil, f.err
}
var result []store.Job
for _, job := range f.jobs {
if job.Status == "succeeded" && job.SeriesID == seriesID {
result = append(result, job)
}
}
sortJobsLatest(result)
return result, nil
}
func (f *fakeJobStore) ListSucceededJobsByIDs(_ context.Context, ids []int64) ([]store.Job, error) {
if f.err != nil {
return nil, f.err
}
wanted := make(map[int64]struct{}, len(ids))
for _, id := range ids {
wanted[id] = struct{}{}
}
var result []store.Job
for _, job := range f.jobs {
if _, ok := wanted[job.ID]; ok && job.Status == "succeeded" {
result = append(result, job)
}
}
sortJobsLatest(result)
return result, nil
}
func (f *fakeJobStore) ListSucceededJobsBySeriesIDs(_ context.Context, seriesIDs []string) ([]store.Job, error) {
if f.err != nil {
return nil, f.err
}
wanted := make(map[string]struct{}, len(seriesIDs))
for _, id := range seriesIDs {
wanted[id] = struct{}{}
}
var result []store.Job
for _, job := range f.jobs {
if _, ok := wanted[job.SeriesID]; ok && job.Status == "succeeded" {
result = append(result, job)
}
}
sortJobsLatest(result)
return result, nil
}
func (f *fakeJobStore) GetSucceededJob(_ context.Context, id int64) (store.Job, error) {
if f.err != nil {
return store.Job{}, f.err
}
for _, job := range f.jobs {
if job.ID == id && job.Status == "succeeded" {
return job, nil
}
}
return store.Job{}, pgx.ErrNoRows
}
func (f *fakeJobStore) DeleteSucceededJob(_ context.Context, id int64) error {
if f.beforeDeleteSucceeded != nil {
f.beforeDeleteSucceeded()
}
if f.err != nil {
return f.err
}
for episodeID, job := range f.jobs {
if job.ID == id && job.Status == "succeeded" {
delete(f.jobs, episodeID)
return nil
}
}
return store.ErrInvalidTransition
}
+161
View File
@@ -0,0 +1,161 @@
package web
import (
"errors"
"net/http"
"net/url"
"regexp"
"strings"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
const seriesURLInputError = "番組ページ(https://tver.jp/series/…)の URL を入力してください"
var seriesURLPathPattern = regexp.MustCompile(`^/series/([A-Za-z0-9]+)/?$`)
var seriesIDPattern = regexp.MustCompile(`^[A-Za-z0-9]+$`)
type subscriptionURLResultView struct {
SeriesID, Title, ErrorText string
EpisodeCount int
Subscribed bool
}
type subscriptionURLRegisteredView struct {
Subscriptions subscriptionsView
}
func parseSeriesURL(raw string) (string, error) {
parsed, err := url.Parse(strings.TrimSpace(raw))
if err != nil || parsed.Scheme != "https" || parsed.User != nil || (!strings.EqualFold(parsed.Host, "tver.jp") && !strings.EqualFold(parsed.Host, "www.tver.jp")) {
return "", errors.New(seriesURLInputError)
}
match := seriesURLPathPattern.FindStringSubmatch(parsed.Path)
if len(match) != 2 {
return "", errors.New(seriesURLInputError)
}
return match[1], nil
}
func (s *server) confirmSubscriptionURL(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, "入力が不正です", http.StatusBadRequest)
return
}
seriesID, err := parseSeriesURL(r.Form.Get("series_url"))
if err != nil {
s.render(w, "subscription_url_result", subscriptionURLResultView{ErrorText: seriesURLInputError})
return
}
title, episodes, ok := s.fetchSubscriptionSeries(w, r, seriesID)
if !ok {
return
}
subscribed, err := s.isSubscribed(r.Context(), seriesID)
if err != nil {
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "subscription_url_result", subscriptionURLResultView{
SeriesID: seriesID, Title: title, EpisodeCount: len(episodes), Subscribed: subscribed,
})
}
func (s *server) registerSubscriptionURL(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
http.Error(w, "入力が不正です", http.StatusBadRequest)
return
}
seriesID := r.Form.Get("series_id")
if !seriesIDPattern.MatchString(seriesID) {
http.Error(w, "番組 ID が不正です", http.StatusBadRequest)
return
}
choice := r.Form.Get("download_existing")
if choice != "" && choice != "1" {
http.Error(w, "公開中の回の選択が不正です", http.StatusBadRequest)
return
}
subscribed, err := s.isSubscribed(r.Context(), seriesID)
if err != nil {
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
return
}
if subscribed {
s.subscriptionURLRegisteredResponse(w, r)
return
}
title, episodes, ok := s.fetchSubscriptionSeries(w, r, seriesID)
if !ok {
return
}
items := make([]store.Episode, 0, len(episodes))
for _, episode := range episodes {
items = append(items, storeEpisode(episode))
}
_, err = s.jobs.RegisterSubscription(r.Context(), seriesID, title, items, choice == "1")
if err != nil {
s.logger.Printf("番組URLからの登録に失敗: %v", err)
s.subscriptionURLFailure(w, r, "番組を登録できませんでした。再試行してください")
return
}
s.subscriptionURLRegisteredResponse(w, r)
}
func (s *server) cancelSubscriptionURL(w http.ResponseWriter, _ *http.Request) {
s.render(w, "subscription_url_result", subscriptionURLResultView{})
}
func (s *server) fetchSubscriptionSeries(w http.ResponseWriter, r *http.Request, seriesID string) (string, []tver.Episode, bool) {
title, err := s.fetcher.FetchSeries(r.Context(), seriesID)
if err != nil {
if errors.Is(err, tver.ErrSeriesNotFound) {
s.subscriptionURLFailure(w, r, "番組が見つかりません")
return "", nil, false
}
s.logSubscriptionFetchError(err)
s.subscriptionURLFailure(w, r, "番組情報を取得できませんでした。再試行してください")
return "", nil, false
}
episodes, err := s.fetcher.FetchSeriesEpisodes(r.Context(), seriesID)
if err != nil {
s.logSubscriptionFetchError(err)
s.subscriptionURLFailure(w, r, "番組情報を取得できませんでした。再試行してください")
return "", nil, false
}
return title, episodes, true
}
func (s *server) logSubscriptionFetchError(err error) {
s.logger.Printf("番組URLの情報取得に失敗: %s", platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
}
func (s *server) subscriptionURLFailure(w http.ResponseWriter, r *http.Request, message string) {
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
return
}
s.render(w, "subscription_url_result", subscriptionURLResultView{ErrorText: message})
}
func (s *server) subscriptionURLRegisteredResponse(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
return
}
view, err := s.loadSubscriptions(r.Context())
if err != nil {
s.logger.Printf("登録番組の取得に失敗: %v", err)
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
return
}
view.OutOfBand = true
s.render(w, "subscription_url_registered", subscriptionURLRegisteredView{Subscriptions: view})
}
+260
View File
@@ -0,0 +1,260 @@
package web
import (
"bytes"
"errors"
"log"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
func TestParseSeriesURL(t *testing.T) {
tests := []struct {
name, raw, want string
valid bool
}{
{"without www", "https://tver.jp/series/sr123", "sr123", true},
{"with www", "https://www.tver.jp/series/ABC123", "ABC123", true},
{"uppercase host", "https://TVER.JP/series/sr123", "sr123", true},
{"trailing slash", "https://tver.jp/series/sr123/", "sr123", true},
{"query and fragment", "https://tver.jp/series/sr123/?foo=bar#section", "sr123", true},
{"surrounding whitespace", " \thttps://tver.jp/series/sr123\n", "sr123", true},
{"http", "http://tver.jp/series/sr123", "", false},
{"subdomain suffix", "https://tver.jp.example.com/series/sr123", "", false},
{"evil prefix", "https://eviltver.jp/series/sr123", "", false},
{"userinfo on allowed host", "https://evil@tver.jp/series/sr123", "", false},
{"allowed host as userinfo", "https://tver.jp@evil.example/series/sr123", "", false},
{"episode", "https://tver.jp/episodes/ep123", "", false},
{"lp", "https://tver.jp/lp/example", "", false},
{"corner", "https://tver.jp/corner/example", "", false},
{"feature", "https://tver.jp/feature/example", "", false},
{"missing id", "https://tver.jp/series/", "", false},
{"invalid id", "https://tver.jp/series/sr_123", "", false},
{"extra path", "https://tver.jp/series/sr123/episodes", "", false},
{"encoded slash in id", "https://tver.jp/series/sr123%2Fevil", "", false},
{"encoded dot segments", "https://tver.jp/series/%2E%2E/episodes", "", false},
{"duplicate slash", "https://tver.jp//series/sr123", "", false},
{"port", "https://tver.jp:443/series/sr123", "", false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := parseSeriesURL(tt.raw)
if tt.valid {
if err != nil || got != tt.want {
t.Fatalf("parseSeriesURL(%q)=%q, %v; want %q", tt.raw, got, err, tt.want)
}
return
}
if err == nil || err.Error() != seriesURLInputError {
t.Fatalf("parseSeriesURL(%q)=%q, %v; want input error", tt.raw, got, err)
}
})
}
}
func TestSubscriptionsPageShowsSeriesURLForm(t *testing.T) {
page := getPage(newTestHandlerWithStore(&fakeFetcher{}, &fakeJobStore{}, nil), "/subscriptions")
body := page.Body.String()
if page.Code != http.StatusOK || !strings.Contains(body, `name="series_url"`) || !strings.Contains(body, `hx-post="/subscriptions/from-url/confirm"`) || !strings.Contains(body, `id="subscriptions-list"`) {
t.Fatalf("URL入力フォーム: status=%d body=%s", page.Code, body)
}
}
func TestConfirmSubscriptionURL(t *testing.T) {
fetcher := &fakeFetcher{
seriesTitle: `<script>alert("title")</script>`,
seriesEpisodes: []tver.Episode{
{ID: "ep1", SeriesID: "sr1"},
{ID: "ep2", SeriesID: "sr1"},
},
}
handler := newTestHandlerWithStore(fetcher, &fakeJobStore{}, nil)
response := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1?x=1#fragment"}})
body := response.Body.String()
for _, want := range []string{"&lt;script&gt;", "番組ID: sr1", "公開中の回数: 2件", `name="series_id" value="sr1"`, "公開中の回もダウンロードする", ">登録</button>", ">キャンセル</button>"} {
if !strings.Contains(body, want) {
t.Errorf("確認画面に %q がない: %s", want, body)
}
}
if response.Code != http.StatusOK || strings.Contains(body, `<script>`) || strings.Contains(body, `name="series_title"`) || strings.Contains(body, `checked`) {
t.Fatalf("確認画面: status=%d body=%s", response.Code, body)
}
if len(fetcher.seriesTitleCalls) != 1 || fetcher.seriesTitleCalls[0] != "sr1" || len(fetcher.seriesCalls) != 1 || fetcher.seriesCalls[0] != "sr1" {
t.Fatalf("取得したID: title=%v episodes=%v", fetcher.seriesTitleCalls, fetcher.seriesCalls)
}
}
func TestConfirmSubscriptionURLAlreadyRegisteredIncludingPaused(t *testing.T) {
fetcher := &fakeFetcher{seriesTitle: "番組", seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1"}}}
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{"sr1": {SeriesID: "sr1", SeriesTitle: "番組", Enabled: false}}}
response := postForm(newTestHandlerWithStore(fetcher, jobs, nil), "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1"}})
body := response.Body.String()
if response.Code != http.StatusOK || !strings.Contains(body, "番組") || !strings.Contains(body, "公開中の回数: 1件") || !strings.Contains(body, "登録済み") || strings.Contains(body, ">登録</button>") || len(fetcher.seriesTitleCalls) != 1 || len(fetcher.seriesCalls) != 1 {
t.Fatalf("登録済み確認: status=%d body=%s title=%v episodes=%v", response.Code, body, fetcher.seriesTitleCalls, fetcher.seriesCalls)
}
}
func TestSubscriptionURLNotFoundAndInvalidURL(t *testing.T) {
fetcher := &fakeFetcher{seriesTitleErr: tver.ErrSeriesNotFound}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
notFound := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/missing"}})
if notFound.Code != http.StatusOK || !strings.Contains(notFound.Body.String(), "番組が見つかりません") || len(fetcher.seriesCalls) != 0 {
t.Fatalf("404: status=%d body=%s episodeCalls=%v", notFound.Code, notFound.Body.String(), fetcher.seriesCalls)
}
registerNotFound := postForm(handler, "/subscriptions/from-url/register", url.Values{"series_id": {"missing"}})
if registerNotFound.Code != http.StatusOK || !strings.Contains(registerNotFound.Body.String(), "番組が見つかりません") || jobs.registerCalls != 0 {
t.Fatalf("登録時404: status=%d body=%s registerCalls=%d", registerNotFound.Code, registerNotFound.Body.String(), jobs.registerCalls)
}
invalid := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://evil.example/series/sr1"}})
if invalid.Code != http.StatusOK || !strings.Contains(invalid.Body.String(), seriesURLInputError) || len(fetcher.seriesTitleCalls) != 2 {
t.Fatalf("不正URL: status=%d body=%s titleCalls=%v", invalid.Code, invalid.Body.String(), fetcher.seriesTitleCalls)
}
}
func TestRegisterSubscriptionURLWithAndWithoutExistingEpisodes(t *testing.T) {
for _, download := range []bool{false, true} {
t.Run(map[bool]string{false: "without existing", true: "with existing"}[download], func(t *testing.T) {
fetcher := &fakeFetcher{seriesTitle: "番組", seriesEpisodes: []tver.Episode{}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
confirmation := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1"}})
if confirmation.Code != http.StatusOK || !strings.Contains(confirmation.Body.String(), "公開中の回はありません") {
t.Fatalf("0件の確認: %d %s", confirmation.Code, confirmation.Body.String())
}
values := url.Values{"series_id": {"sr1"}, "series_title": {"偽番組"}}
if download {
values.Set("download_existing", "1")
}
registered := postForm(handler, "/subscriptions/from-url/register", values)
body := registered.Body.String()
if registered.Code != http.StatusOK || !strings.Contains(body, "登録済み") || !strings.Contains(body, `id="subscriptions-list"`) || !strings.Contains(body, `hx-swap-oob="outerHTML"`) || !strings.Contains(body, `id="subscription-url-input"`) {
t.Fatalf("登録応答: %d %s", registered.Code, body)
}
if jobs.registerCalls != 1 || jobs.registerDownload != download || len(jobs.registerEpisodes) != 0 || jobs.subscriptions["sr1"].SeriesTitle != "番組" {
t.Fatalf("登録内容: calls=%d download=%t episodes=%v subscriptions=%v", jobs.registerCalls, jobs.registerDownload, jobs.registerEpisodes, jobs.subscriptions)
}
if len(fetcher.seriesTitleCalls) != 2 || len(fetcher.seriesCalls) != 2 {
t.Fatalf("確認と登録で再取得していない: title=%v episodes=%v", fetcher.seriesTitleCalls, fetcher.seriesCalls)
}
})
}
}
func TestRegisterSubscriptionURLUsesServerFetchedData(t *testing.T) {
fetcher := &fakeFetcher{seriesTitle: "本当の番組", seriesEpisodes: []tver.Episode{{
ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Title: "公開中の回", Duration: 200,
}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
response := postForm(handler, "/subscriptions/from-url/register", url.Values{
"series_id": {"sr1"}, "series_title": {"偽番組"}, "download_existing": {"1"},
})
if response.Code != http.StatusOK || jobs.registerCalls != 1 || !jobs.registerDownload || len(jobs.registerEpisodes) != 1 || jobs.registerEpisodes[0].ID != "ep1" || jobs.subscriptions["sr1"].SeriesTitle != "本当の番組" {
t.Fatalf("サーバー取得値での登録: status=%d calls=%d download=%t episodes=%+v subscriptions=%+v", response.Code, jobs.registerCalls, jobs.registerDownload, jobs.registerEpisodes, jobs.subscriptions)
}
}
func TestRegisterSubscriptionURLAlreadyRegisteredDoesNothing(t *testing.T) {
fetcher := &fakeFetcher{seriesTitleErr: errors.New("must not fetch"), seriesErr: errors.New("must not fetch")}
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{"sr1": {SeriesID: "sr1", SeriesTitle: "番組", Enabled: false}}}
response := postForm(newTestHandlerWithStore(fetcher, jobs, nil), "/subscriptions/from-url/register", url.Values{"series_id": {"sr1"}})
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "登録済み") || jobs.registerCalls != 0 || len(fetcher.seriesTitleCalls) != 0 || len(fetcher.seriesCalls) != 0 {
t.Fatalf("登録済みの再登録: status=%d body=%s register=%d title=%v episodes=%v", response.Code, response.Body.String(), jobs.registerCalls, fetcher.seriesTitleCalls, fetcher.seriesCalls)
}
}
func TestRegisterSubscriptionURLFetchFailuresDoNotRegisterAndRedactToken(t *testing.T) {
for _, tt := range []struct {
name string
seriesErr error
episodesErr error
wantEpisodeCall int
}{
{"series", errors.New("platform_token=secret-series& upstream"), nil, 0},
{"episodes", nil, errors.New("platform_token=secret-episodes& upstream"), 1},
} {
t.Run(tt.name, func(t *testing.T) {
fetcher := &fakeFetcher{seriesTitle: "番組", seriesTitleErr: tt.seriesErr, seriesErr: tt.episodesErr}
jobs := &fakeJobStore{}
var logs bytes.Buffer
handler := NewHandler(fetcher, jobs, "", time.Now, log.New(&logs, "", 0))
response := postForm(handler, "/subscriptions/from-url/register", url.Values{"series_id": {"sr1"}})
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "番組情報を取得できませんでした。再試行してください") || jobs.registerCalls != 0 || len(jobs.subscriptions) != 0 {
t.Fatalf("取得失敗: status=%d body=%s calls=%d subscriptions=%v", response.Code, response.Body.String(), jobs.registerCalls, jobs.subscriptions)
}
if strings.Contains(logs.String(), "secret-") || !strings.Contains(logs.String(), "platform_token=[REDACTED]") || len(fetcher.seriesCalls) != tt.wantEpisodeCall {
t.Fatalf("ログまたは呼出回数: logs=%s episodeCalls=%v", logs.String(), fetcher.seriesCalls)
}
})
}
}
func TestRegisterSubscriptionURLRejectsInvalidChoice(t *testing.T) {
fetcher := &fakeFetcher{}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
for _, values := range []url.Values{
{},
{"series_id": {"bad_id"}},
{"series_id": {"../sr1"}},
{"series_id": {"sr1"}, "download_existing": {"yes"}},
} {
response := postForm(handler, "/subscriptions/from-url/register", values)
if response.Code != http.StatusBadRequest {
t.Errorf("不正値 %v: status=%d body=%s", values, response.Code, response.Body.String())
}
}
if len(fetcher.seriesTitleCalls) != 0 || len(fetcher.seriesCalls) != 0 || jobs.subscribedCalls != 0 || jobs.registerCalls != 0 {
t.Fatalf("不正な hidden 値で外部取得または DB 操作を実行: title=%v episodes=%v subscribed=%d register=%d", fetcher.seriesTitleCalls, fetcher.seriesCalls, jobs.subscribedCalls, jobs.registerCalls)
}
}
func TestCancelSubscriptionURLClearsConfirmation(t *testing.T) {
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, &fakeJobStore{}, nil), "/subscriptions/from-url/cancel")
body := response.Body.String()
if response.Code != http.StatusOK || !strings.Contains(body, `id="subscription-url-result"`) || strings.Contains(body, `name="series_id"`) || strings.Contains(body, `role="alert"`) || strings.Contains(body, `role="status"`) {
t.Fatalf("キャンセル応答: status=%d body=%q", response.Code, response.Body.String())
}
}
func TestSubscriptionURLPostsWithoutHtmxRedirect(t *testing.T) {
fetcher := &fakeFetcher{seriesTitle: "番組", seriesEpisodes: []tver.Episode{}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
for _, tt := range []struct {
path string
form url.Values
}{
{"/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1"}}},
{"/subscriptions/from-url/register", url.Values{"series_id": {"sr1"}}},
} {
request := httptest.NewRequest(http.MethodPost, tt.path, strings.NewReader(tt.form.Encode()))
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/subscriptions" {
t.Errorf("通常POST %s: status=%d Location=%q", tt.path, response.Code, response.Header().Get("Location"))
}
}
if len(fetcher.seriesTitleCalls) != 1 || jobs.registerCalls != 1 {
t.Fatalf("登録POSTが処理されていない: fetch=%v register=%d", fetcher.seriesTitleCalls, jobs.registerCalls)
}
}
func TestSubscriptionURLPostRoutesRejectGet(t *testing.T) {
handler := newTestHandlerWithStore(&fakeFetcher{}, &fakeJobStore{}, nil)
for _, path := range []string{"/subscriptions/from-url/confirm", "/subscriptions/from-url/register"} {
response := getPage(handler, path)
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
t.Errorf("GET %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
}
}
}
+192
View File
@@ -0,0 +1,192 @@
package web
import (
"context"
"errors"
"net/http"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
type subscriptionActionView struct {
ID, Genre, ErrorText string
Subscribed, Choosing, DownloadExisting bool
}
type subscriptionView struct {
ID, Title, CreatedAtText, LastCheckedText string
CheckErrorAtText, CheckErrorText string
Enabled bool
Jobs int64
}
type subscriptionsView struct {
Items []subscriptionView
OutOfBand bool
}
func (s *server) subscriptionChoice(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
ep, found := s.cachedEpisode(genre, id)
if !found || ep.SeriesID == "" {
http.Error(w, "新着キャッシュに番組がありません。新着を再表示してください", http.StatusNotFound)
return
}
registered, err := s.isSubscribed(r.Context(), ep.SeriesID)
if err != nil {
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
return
}
if registered {
s.subscriptionRegisteredResponse(w, r, id, genre)
return
}
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true})
}
func (s *server) registerSubscription(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
choice := r.Form.Get("download_existing")
if choice != "" && choice != "1" {
http.Error(w, "公開中の回の選択が不正です", http.StatusBadRequest)
return
}
ep, found := s.cachedEpisode(genre, id)
if !found || ep.SeriesID == "" || ep.SeriesTitle == "" {
http.Error(w, "新着キャッシュに番組がありません。新着を再表示してください", http.StatusNotFound)
return
}
registered, err := s.isSubscribed(r.Context(), ep.SeriesID)
if err != nil {
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
return
}
if registered {
s.subscriptionRegisteredResponse(w, r, id, genre)
return
}
episodes, err := s.fetcher.FetchSeriesEpisodes(r.Context(), ep.SeriesID)
if err != nil {
s.logger.Printf("番組の公開中エピソード取得に失敗: %s", platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true, DownloadExisting: choice == "1", ErrorText: "公開中の回を取得できませんでした。再試行してください"})
return
}
items := make([]store.Episode, 0, len(episodes))
for _, episode := range episodes {
items = append(items, storeEpisode(episode))
}
_, err = s.jobs.RegisterSubscription(r.Context(), ep.SeriesID, ep.SeriesTitle, items, choice == "1")
if err != nil {
s.logger.Printf("番組登録に失敗: %v", err)
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true, DownloadExisting: choice == "1", ErrorText: "番組を登録できませんでした。再試行してください"})
return
}
s.subscriptionRegisteredResponse(w, r, id, genre)
}
func (s *server) subscriptionRegisteredResponse(w http.ResponseWriter, r *http.Request, id, genre string) {
if r.Header.Get("HX-Request") != "true" && r.Method == http.MethodPost {
http.Redirect(w, r, "/?genre="+genre, http.StatusSeeOther)
return
}
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Subscribed: true})
}
func (s *server) isSubscribed(ctx context.Context, seriesID string) (bool, error) {
result, err := s.jobs.SubscribedSeries(ctx, []string{seriesID})
return result[seriesID], err
}
func (s *server) subscriptionsPage(w http.ResponseWriter, r *http.Request) {
view, err := s.loadSubscriptions(r.Context())
if err != nil {
s.logger.Printf("登録番組の取得に失敗: %v", err)
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "layout", page{Title: "登録番組", Subscriptions: &view})
}
func (s *server) subscriptionsFragment(w http.ResponseWriter, r *http.Request) {
view, err := s.loadSubscriptions(r.Context())
if err != nil {
s.logger.Printf("登録番組の取得に失敗: %v", err)
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "subscriptions_list", view)
}
func (s *server) loadSubscriptions(ctx context.Context) (subscriptionsView, error) {
subs, err := s.jobs.ListSubscriptions(ctx)
if err != nil {
return subscriptionsView{}, err
}
view := subscriptionsView{Items: make([]subscriptionView, 0, len(subs))}
for _, sub := range subs {
item := subscriptionView{ID: sub.SeriesID, Title: sub.SeriesTitle,
CreatedAtText: sub.CreatedAt.In(s.zone).Format("2006-01-02 15:04 MST"),
LastCheckedText: "未確認", Enabled: sub.Enabled, Jobs: sub.JobCount}
if sub.LastCheckedAt != nil {
item.LastCheckedText = sub.LastCheckedAt.In(s.zone).Format("2006-01-02 15:04 MST")
}
if sub.LastCheckErrorAt != nil && sub.LastCheckError != nil {
item.CheckErrorAtText = sub.LastCheckErrorAt.In(s.zone).Format("2006-01-02 15:04 MST")
item.CheckErrorText = *sub.LastCheckError
}
view.Items = append(view.Items, item)
}
return view, nil
}
func (s *server) pauseSubscription(w http.ResponseWriter, r *http.Request) {
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
return s.jobs.SetSubscriptionEnabled(ctx, id, false)
})
}
func (s *server) resumeSubscription(w http.ResponseWriter, r *http.Request) {
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
return s.jobs.SetSubscriptionEnabled(ctx, id, true)
})
}
func (s *server) deleteSubscription(w http.ResponseWriter, r *http.Request) {
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
deleted, err := s.jobs.DeleteSubscription(ctx, id)
if err == nil && !deleted {
return store.ErrInvalidTransition
}
return err
})
}
func (s *server) subscriptionMutation(w http.ResponseWriter, r *http.Request, change func(context.Context, string) error) {
id := r.PathValue("id")
if id == "" {
http.Error(w, "番組 ID が不正です", http.StatusBadRequest)
return
}
if err := change(r.Context(), id); err != nil {
if errors.Is(err, store.ErrInvalidTransition) {
http.Error(w, "番組が見つかりません", http.StatusNotFound)
} else {
s.logger.Printf("登録番組の操作に失敗: %v", err)
http.Error(w, "登録番組を変更できませんでした", http.StatusInternalServerError)
}
return
}
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
return
}
s.subscriptionsFragment(w, r)
}
+199
View File
@@ -0,0 +1,199 @@
package web
import (
"errors"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
func TestRegisterSubscriptionChoicesAndServerOwnedSeries(t *testing.T) {
for _, download := range []bool{false, true} {
t.Run(map[bool]string{false: "without existing", true: "with existing"}[download], func(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Duration: 200}},
seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Title: "公開中", Duration: 200}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
page := getPage(handler, "/")
if page.Code != http.StatusOK || !strings.Contains(page.Body.String(), "番組を登録") || jobs.subscribedCalls != 1 || len(jobs.subscribedIDs) != 1 || jobs.subscribedIDs[0] != "sr1" {
t.Fatalf("新着の登録状態: %d ids=%v body=%s", page.Code, jobs.subscribedIDs, page.Body.String())
}
choice := getPage(handler, "/subscriptions/choice?genre=drama&episode_id=ep1")
if choice.Code != http.StatusOK || !strings.Contains(choice.Body.String(), "公開中の回もダウンロードする") || !strings.Contains(choice.Body.String(), ">登録</button>") {
t.Fatalf("登録時の選択: %d %s", choice.Code, choice.Body.String())
}
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}, "series_id": {"forged"}, "series_title": {"偽番組"}}
if download {
values.Set("download_existing", "1")
}
registered := postForm(handler, "/subscriptions/register", values)
if registered.Code != http.StatusOK || !strings.Contains(registered.Body.String(), "登録済み") {
t.Fatalf("登録応答: %d %s", registered.Code, registered.Body.String())
}
if jobs.registerCalls != 1 || jobs.registerDownload != download || len(jobs.registerEpisodes) != 1 || jobs.registerEpisodes[0].ID != "ep1" || jobs.registerEpisodes[0].SeriesID != "sr1" || jobs.subscriptions["sr1"].SeriesTitle != "本当の番組" {
t.Fatalf("登録内容: calls=%d download=%t episodes=%+v subs=%+v", jobs.registerCalls, jobs.registerDownload, jobs.registerEpisodes, jobs.subscriptions)
}
if len(fetcher.seriesCalls) != 1 || fetcher.seriesCalls[0] != "sr1" {
t.Fatalf("番組一覧取得: %v", fetcher.seriesCalls)
}
if !strings.Contains(getPage(handler, "/subscriptions").Body.String(), "最終確認: 2026-10-03 09:00 JST") {
t.Fatal("登録直後の最終確認日時が表示されない")
}
fetcher.seriesErr = errors.New("API unavailable")
duplicate := postForm(handler, "/subscriptions/register", values)
if duplicate.Code != http.StatusOK || !strings.Contains(duplicate.Body.String(), "登録済み") || len(fetcher.seriesCalls) != 1 || jobs.registerCalls != 1 {
t.Fatalf("重複登録が API を再取得: %d %s calls=%v register=%d", duplicate.Code, duplicate.Body.String(), fetcher.seriesCalls, jobs.registerCalls)
}
reloaded := getPage(handler, "/")
if !strings.Contains(reloaded.Body.String(), "登録済み") || strings.Contains(reloaded.Body.String(), ">番組を登録</button>") {
t.Fatalf("登録済みの表示: %s", reloaded.Body.String())
}
})
}
}
func TestRegisterSubscriptionRejectsMissingCacheAndFetchFailure(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
if got := postForm(handler, "/subscriptions/register", values).Code; got != http.StatusNotFound {
t.Fatalf("キャッシュ無しの登録: %d", got)
}
getPage(handler, "/")
for _, path := range []string{"/subscriptions/register", "/subscriptions/sr1/pause", "/subscriptions/sr1/resume", "/subscriptions/sr1/delete"} {
response := getPage(handler, path)
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
t.Errorf("GET での変更 %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
}
}
for _, tt := range []struct {
path string
want int
}{
{"/subscriptions/choice?genre=news&episode_id=ep1", http.StatusBadRequest},
{"/subscriptions/choice?genre=drama", http.StatusBadRequest},
{"/subscriptions/choice?genre=drama&episode_id=missing", http.StatusNotFound},
} {
response := getPage(handler, tt.path)
if response.Code != tt.want {
t.Errorf("不正な選択 %s: got=%d want=%d", tt.path, response.Code, tt.want)
}
}
for _, tt := range []struct {
values url.Values
want int
}{
{url.Values{"genre": {"news"}, "episode_id": {"ep1"}}, http.StatusBadRequest},
{url.Values{"genre": {"drama"}}, http.StatusBadRequest},
{url.Values{"genre": {"drama"}, "episode_id": {"missing"}}, http.StatusNotFound},
} {
response := postForm(handler, "/subscriptions/register", tt.values)
if response.Code != tt.want {
t.Errorf("不正な登録 %v: got=%d want=%d", tt.values, response.Code, tt.want)
}
}
values.Set("download_existing", "bad")
if got := postForm(handler, "/subscriptions/register", values).Code; got != http.StatusBadRequest {
t.Fatalf("不正な選択: %d", got)
}
values.Del("download_existing")
fetcher.seriesErr = errors.New("platform_token=secret& upstream failure")
failed := postForm(handler, "/subscriptions/register", values)
if failed.Code != http.StatusOK || !strings.Contains(failed.Body.String(), "公開中の回を取得できませんでした") || strings.Contains(failed.Body.String(), "secret") || jobs.registerCalls != 0 {
t.Fatalf("取得失敗時に未登録: %d %s calls=%d", failed.Code, failed.Body.String(), jobs.registerCalls)
}
fetcher.seriesErr = nil
fetcher.seriesEpisodes = []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}}
jobs.registerErr = errors.New("secret database detail")
failed = postForm(handler, "/subscriptions/register", values)
if failed.Code != http.StatusOK || !strings.Contains(failed.Body.String(), "番組を登録できませんでした") || strings.Contains(failed.Body.String(), "secret") || len(jobs.subscriptions) != 0 {
t.Fatalf("DB 失敗時に未登録: %d %s subscriptions=%v", failed.Code, failed.Body.String(), jobs.subscriptions)
}
}
func TestNewerShowsInitiallySubscribedSeriesOncePerSeriesLookup(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{
{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200},
{ID: "ep2", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200},
}}
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{
"sr1": {SeriesID: "sr1", SeriesTitle: "番組", Enabled: true},
}}
response := getPage(newTestHandlerWithStore(fetcher, jobs, nil), "/")
body := response.Body.String()
if response.Code != http.StatusOK || strings.Count(body, "登録済み") != 2 || strings.Contains(body, ">番組を登録</button>") || jobs.subscribedCalls != 1 || len(jobs.subscribedIDs) != 1 || jobs.subscribedIDs[0] != "sr1" {
t.Fatalf("登録済み番組の初期表示: status=%d calls=%d ids=%v body=%s", response.Code, jobs.subscribedCalls, jobs.subscribedIDs, body)
}
}
func TestSubscriptionMutationsWithoutHtmxRedirect(t *testing.T) {
fetcher := &fakeFetcher{
episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}},
seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}},
}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
getPage(handler, "/")
form := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
request := httptest.NewRequest(http.MethodPost, "/subscriptions/register", strings.NewReader(form.Encode()))
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/?genre=drama" {
t.Fatalf("通常登録 POST: status=%d Location=%q", response.Code, response.Header().Get("Location"))
}
request = httptest.NewRequest(http.MethodPost, "/subscriptions/sr1/pause", nil)
response = httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/subscriptions" || jobs.subscriptions["sr1"].Enabled {
t.Fatalf("通常操作 POST: status=%d Location=%q subscription=%+v", response.Code, response.Header().Get("Location"), jobs.subscriptions["sr1"])
}
}
func TestSubscriptionsPagePauseResumeDeleteAndEscaping(t *testing.T) {
checked := time.Date(2026, 10, 4, 2, 30, 0, 0, time.UTC)
failedAt := time.Date(2026, 10, 4, 3, 45, 0, 0, time.UTC)
checkError := `<script>alert("error")</script> token=[REDACTED]`
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{
"sr1": {SeriesID: "sr1", SeriesTitle: `<script>alert(1)</script>`, Enabled: true,
CreatedAt: time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC), LastCheckedAt: &checked,
LastCheckErrorAt: &failedAt, LastCheckError: &checkError, JobCount: 3},
"sr2": {SeriesID: "sr2", SeriesTitle: "別番組", Enabled: false, JobCount: 0},
}}
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
page := getPage(handler, "/subscriptions")
if page.Code != http.StatusOK {
t.Fatalf("登録番組ページ: %d", page.Code)
}
body := page.Body.String()
for _, want := range []string{"&lt;script&gt;", "登録日時: 2026-10-03 09:00 JST", "最終確認: 2026-10-04 11:30 JST", "最終確認: 未確認", "最後の確認エラー(2026-10-04 12:45 JST)", `&lt;script&gt;alert(&#34;error&#34;)&lt;/script&gt; token=[REDACTED]`, "関連ジョブ: 3件", "有効", "一時停止", `hx-confirm="番組の登録を解除しますか?"`} {
if !strings.Contains(body, want) {
t.Errorf("登録番組の情報 %q がない", want)
}
}
if strings.Contains(body, `<script>`) {
t.Fatal("番組名または確認エラーがエスケープされていない")
}
paused := postForm(handler, "/subscriptions/sr1/pause", nil)
if paused.Code != http.StatusOK || jobs.subscriptions["sr1"].Enabled || !strings.Contains(paused.Body.String(), `action="/subscriptions/sr1/resume"`) || strings.Contains(paused.Body.String(), "<html") {
t.Fatalf("一時停止: %d %s", paused.Code, paused.Body.String())
}
resumed := postForm(handler, "/subscriptions/sr1/resume", nil)
if resumed.Code != http.StatusOK || !jobs.subscriptions["sr1"].Enabled || !strings.Contains(resumed.Body.String(), `action="/subscriptions/sr1/pause"`) {
t.Fatalf("再開: %d %s", resumed.Code, resumed.Body.String())
}
deleted := postForm(handler, "/subscriptions/sr1/delete", nil)
if deleted.Code != http.StatusOK || len(jobs.subscriptions) != 1 || strings.Contains(deleted.Body.String(), "&lt;script&gt;") {
t.Fatalf("解除: %d %s", deleted.Code, deleted.Body.String())
}
if got := postForm(handler, "/subscriptions/sr1/delete", nil).Code; got != http.StatusNotFound {
t.Fatalf("存在しない番組の解除: %d", got)
}
}
+42
View File
@@ -0,0 +1,42 @@
{{define "layout"}}<!doctype html>
<html lang="ja">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{{.Title}} | TVer</title>
<link rel="stylesheet" href="/static/app.css">
<script src="/static/htmx-2.0.11.min.js" defer></script>
</head>
<body>
<header>
<strong>TVer</strong>
<nav aria-label="メイン" hx-boost="true">
<a href="/">新着</a>
<a href="/queue">待ち行列</a>
<a href="/subscriptions">登録番組</a>
<a href="/library">ライブラリ</a>
</nav>
</header>
<main>
{{if .Newer}}
{{template "newer_page" .Newer}}
{{else if .Queue}}
{{template "queue_page" .Queue}}
{{else if .Subscriptions}}
{{template "subscriptions_page" .Subscriptions}}
{{else if .Library}}
{{template "library_page" .Library}}
{{else if .Series}}
{{template "series_page" .Series}}
{{else if .Video}}
{{template "video_page" .Video}}
{{else if .DeleteConfirm}}
{{template "delete_confirm_page" .DeleteConfirm}}
{{else if .DeleteResult}}
{{template "delete_result_page" .DeleteResult}}
{{else}}
<h1>{{.Title}}</h1>
{{end}}
</main>
</body>
</html>{{end}}
+115
View File
@@ -0,0 +1,115 @@
{{define "library_page"}}
<h1>ライブラリ</h1>
{{if .Series}}
<p>{{.SeriesCount}}番組・{{.EpisodeCount}}本・{{.TotalSizeText}}</p>
<form action="/library/delete/confirm" method="post">
<div class="episode-grid library-select-grid">
{{range .Series}}
<article class="episode-card">
<label class="library-selection"><input type="checkbox" name="series_id" value="{{.SeriesID}}"> この番組を選択</label>
<a href="/library/series/{{.SeriesID}}">
<div class="library-card-image">
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
<span class="episode-count-badge">{{.EpisodeCount}}本</span>
</div>
<div class="episode-copy">
<h2>{{.SeriesTitle}} {{if .Subscribed}}<span class="subscription-badge">登録中</span>{{end}}</h2>
<p>放送局: {{.BroadcasterName}}</p>
<p>最新: {{.LatestTitle}}</p>
<p>最終完了: {{.LatestFinishedText}}</p>
<p>合計サイズ: {{.TotalSizeText}}</p>
</div>
</a>
</article>
{{end}}
</div>
<p><button type="submit">選択した番組を削除</button></p>
</form>
{{else}}
<p>ダウンロード済みの動画はありません</p>
{{end}}
{{end}}
{{define "series_page"}}
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><span>{{.SeriesTitle}}</span></nav>
<h1>{{.SeriesTitle}} {{if .Subscribed}}<span class="subscription-badge">登録中</span>{{end}}</h1>
<p>{{.BroadcasterName}}・{{.EpisodeCount}}本・{{.TotalSizeText}}</p>
<p class="library-sort">並び順:
{{if eq .Sort "latest"}}<strong>完了が新しい順</strong>{{else}}<a href="/library/series/{{.SeriesID}}">完了が新しい順</a>{{end}}
/
{{if eq .Sort "natural"}}<strong>話の順</strong>{{else}}<a href="/library/series/{{.SeriesID}}?sort=natural">話の順</a>{{end}}
</p>
<form action="/library/delete/confirm" method="post">
<div class="library-episode-list">
{{range .Items}}
<div class="library-episode-select-row">
<label class="library-selection"><input type="checkbox" name="id" value="{{.ID}}"> 選択</label>
<a class="library-episode-row" href="/library/{{.ID}}">
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
<span class="library-episode-title">{{.Title}}</span>
<span>{{.FinishedAtText}}</span>
<span>{{.DurationText}}</span>
<span>{{.FileSizeText}}</span>
</a>
</div>
{{end}}
</div>
<p><button type="submit">選択した回を削除</button></p>
</form>
<form action="/library/delete/confirm" method="post">
<input type="hidden" name="series_id" value="{{.SeriesID}}">
<button type="submit">この番組をすべて削除</button>
</form>
{{end}}
{{define "video_page"}}
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><a href="{{.SeriesURL}}">{{.SeriesTitle}}</a><span aria-hidden="true">›</span><span>{{.Title}}</span></nav>
<h1>{{.SeriesTitle}}</h1>
<h2>{{.Title}}</h2>
<video controls preload="metadata" src="{{.VideoURL}}"></video>
<p>放送局: {{.BroadcasterName}}/長さ: {{.DurationText}}/サイズ: {{.FileSizeText}}/完了日時: {{.FinishedAtText}}</p>
<nav class="episode-navigation" aria-label="前後の回">
{{if .Previous}}<a href="/library/{{.Previous.ID}}">← 前の回: {{.Previous.Title}}</a>{{else}}<span aria-disabled="true">前の回はありません</span>{{end}}
{{if .Next}}<a href="/library/{{.Next.ID}}">次の回: {{.Next.Title}} →</a>{{else}}<span aria-disabled="true">次の回はありません</span>{{end}}
</nav>
<form action="/library/delete/confirm" method="post">
<input type="hidden" name="id" value="{{.ID}}">
<button type="submit">削除</button>
</form>
{{end}}
{{define "delete_confirm_page"}}
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><span>削除確認</span></nav>
<h1>動画の削除確認</h1>
<p class="delete-warning">ファイルと記録を削除します。この操作は取り消せません。</p>
<p>{{.Count}}本・合計{{.TotalSizeText}}</p>
{{range .Groups}}
<section class="delete-group">
<h2>{{.SeriesTitle}}</h2>
<p>{{len .Items}}本・{{.TotalSizeText}}</p>
<ul>
{{range .Items}}<li>{{.Title}}({{.FileSizeText}})</li>{{end}}
</ul>
</section>
{{end}}
<form action="/library/delete" method="post">
{{range .IDs}}<input type="hidden" name="id" value="{{.}}">{{end}}
<button type="submit">削除を実行</button>
<a href="{{.CancelURL}}">取り消す</a>
</form>
{{end}}
{{define "delete_result_page"}}
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><span>削除結果</span></nav>
<h1>動画の削除結果</h1>
<p>削除: {{.DeletedCount}}件/失敗: {{len .Failures}}件/対象外: {{.ExcludedCount}}件</p>
{{if .Failures}}
<h2>失敗した回</h2>
<ul>{{range .Failures}}<li>{{.Title}}: {{.Reason}}</li>{{end}}</ul>
{{end}}
<p><a href="/library">ライブラリへ戻る</a></p>
{{if .SeriesLinks}}
<h2>回が残っている番組</h2>
<ul>{{range .SeriesLinks}}<li><a href="/library/series/{{.SeriesID}}">{{.SeriesTitle}}</a></li>{{end}}</ul>
{{end}}
{{end}}
+70
View File
@@ -0,0 +1,70 @@
{{define "newer_page"}}
<h1>新着</h1>
<form id="newer-controls" action="/" method="get"
hx-get="/newer" hx-trigger="change, submit"
hx-target="#newer-results" hx-swap="outerHTML">
<label for="genre">ジャンル</label>
<select id="genre" name="genre">
<option value="drama" {{if eq .Genre "drama"}}selected{{end}}>ドラマ</option>
<option value="variety" {{if eq .Genre "variety"}}selected{{end}}>バラエティ</option>
</select>
<input type="hidden" name="hide_previews" value="0">
<label for="hide-previews">
<input id="hide-previews" type="checkbox" name="hide_previews" value="1" {{if .HidePreviews}}checked{{end}}>
予告を隠す
</label>
<button type="submit">表示</button>
</form>
{{template "newer_results" .}}
{{end}}
{{define "newer_results"}}
<section id="newer-results" aria-label="新着一覧" aria-live="polite">
{{if .Failed}}
<p role="status">{{.ErrorText}}</p>
{{else if not .Episodes}}
<p>表示する新着はありません</p>
{{else}}
<div class="episode-grid">
{{range .Episodes}}
<article class="episode-card">
<a href="{{.URL}}">
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
<div class="episode-copy">
<h2>{{.SeriesTitle}}</h2>
<p>{{.Title}}</p>
<p>放送局: {{.BroadcasterName}}</p>
<p>長さ: {{.DurationMinutes}}分</p>
<p>放送日: {{.BroadcastDateLabel}}</p>
<p>配信終了: <time datetime="{{.EndAtISO}}">{{.EndAtText}}</time></p>
</div>
</a>
{{template "episode_action" .}}
{{template "subscription_action" .SubscriptionAction}}
</article>
{{end}}
</div>
{{end}}
</section>
{{end}}
{{define "episode_action"}}
<div class="episode-action">
{{if not .Status}}
<form action="/episodes/enqueue" method="post" hx-post="/episodes/enqueue" hx-target="closest .episode-action" hx-swap="outerHTML">
<input type="hidden" name="episode_id" value="{{.ID}}">
<input type="hidden" name="genre" value="{{.Genre}}">
<button type="submit">ダウンロード</button>
</form>
{{else}}
<span role="status">{{.StatusText}}</span>
{{if or (eq .Status "failed") (eq .Status "canceled")}}
<form action="/episodes/retry" method="post" hx-post="/episodes/retry" hx-target="closest .episode-action" hx-swap="outerHTML">
<input type="hidden" name="episode_id" value="{{.ID}}">
<input type="hidden" name="genre" value="{{.Genre}}">
<button type="submit">再試行</button>
</form>
{{end}}
{{end}}
</div>
{{end}}
+59
View File
@@ -0,0 +1,59 @@
{{define "queue_page"}}
<h1>待ち行列</h1>
{{template "queue_list" .}}
{{end}}
{{define "queue_list"}}
<div id="queue-list" {{if .Poll}}hx-get="/queue/list" hx-trigger="every 5s" hx-target="this" hx-swap="outerHTML"{{end}}>
<section aria-label="ダウンロード中">
<h2>ダウンロード中</h2>
{{range .Running}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="待機中">
<h2>待機中</h2>
{{range .Queued}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="失敗">
<h2>失敗</h2>
{{range .Failed}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="取り消し">
<h2>取り消し</h2>
{{range .Canceled}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="完了">
<h2>完了(直近50件)</h2>
{{range .Succeeded}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
</div>
{{end}}
{{define "queue_job"}}
<article class="queue-job">
<h3>{{.SeriesTitle}} — {{.Title}}</h3>
<p>放送局: {{.BroadcasterName}}/登録元: {{.SourceText}}/登録日時: {{.CreatedAtText}}</p>
{{if eq .Status "running"}}
<p>段階: {{if .PhaseText}}{{.PhaseText}}{{else}}準備中{{end}}
{{if .PercentText}}/{{.PercentText}}{{end}}
{{if .SpeedText}}/{{.SpeedText}}{{end}}
{{if .ETAText}}/{{.ETAText}}{{end}}
{{if .FragmentText}}/{{.FragmentText}}{{end}}</p>
<form action="/jobs/{{.ID}}/cancel" method="post" hx-post="/jobs/{{.ID}}/cancel" hx-target="#queue-list" hx-swap="outerHTML">
<button type="submit">取り消し</button>
</form>
{{end}}
{{if eq .Status "failed"}}
<p>失敗理由: {{.ErrorKindText}}</p>
{{if .ErrorMessage}}<details><summary>詳細</summary><pre>{{.ErrorMessage}}</pre></details>{{end}}
{{end}}
{{if eq .Status "queued"}}
<form action="/jobs/{{.ID}}/cancel" method="post" hx-post="/jobs/{{.ID}}/cancel" hx-target="#queue-list" hx-swap="outerHTML">
<button type="submit">取り消し</button>
</form>
{{else if or (eq .Status "failed") (eq .Status "canceled")}}
<form action="/jobs/{{.ID}}/retry" method="post" hx-post="/jobs/{{.ID}}/retry" hx-target="#queue-list" hx-swap="outerHTML">
<button type="submit">再試行</button>
</form>
{{end}}
</article>
{{end}}
+81
View File
@@ -0,0 +1,81 @@
{{define "subscription_action"}}
<div class="subscription-action">
{{if .Subscribed}}
<span role="status">登録済み</span>
{{else if .Choosing}}
{{if .ErrorText}}<p role="alert">{{.ErrorText}}</p>{{end}}
<form action="/subscriptions/register" method="post" hx-post="/subscriptions/register" hx-target="closest .subscription-action" hx-swap="outerHTML">
<input type="hidden" name="episode_id" value="{{.ID}}">
<input type="hidden" name="genre" value="{{.Genre}}">
<label><input type="checkbox" name="download_existing" value="1" {{if .DownloadExisting}}checked{{end}}>公開中の回もダウンロードする</label>
<button type="submit">登録</button>
</form>
{{else}}
<button type="button" hx-get="/subscriptions/choice?genre={{.Genre}}&amp;episode_id={{.ID}}" hx-target="closest .subscription-action" hx-swap="outerHTML">番組を登録</button>
{{end}}
</div>
{{end}}
{{define "subscriptions_page"}}
<h1>登録番組</h1>
<section class="subscription-url-entry">
<h2>番組URLから登録</h2>
<form action="/subscriptions/from-url/confirm" method="post" hx-post="/subscriptions/from-url/confirm" hx-target="#subscription-url-result" hx-swap="outerHTML">
<label>番組URL <input id="subscription-url-input" type="url" name="series_url" placeholder="https://tver.jp/series/…" required></label>
<button type="submit">確認</button>
</form>
<div id="subscription-url-result"></div>
</section>
{{template "subscriptions_list" .}}
{{end}}
{{define "subscription_url_result"}}
<div id="subscription-url-result">
{{if .ErrorText}}
<p role="alert">{{.ErrorText}}</p>
{{else if .SeriesID}}
<h3>{{.Title}}</h3>
<p>番組ID: {{.SeriesID}}</p>
{{if eq .EpisodeCount 0}}<p>公開中の回はありません</p>{{else}}<p>公開中の回数: {{.EpisodeCount}}件</p>{{end}}
{{if .Subscribed}}
<p role="status">登録済み</p>
<button type="button" hx-get="/subscriptions/from-url/cancel" hx-target="#subscription-url-result" hx-swap="outerHTML">キャンセル</button>
{{else}}
<form action="/subscriptions/from-url/register" method="post" hx-post="/subscriptions/from-url/register" hx-target="#subscription-url-result" hx-swap="outerHTML">
<input type="hidden" name="series_id" value="{{.SeriesID}}">
<label><input type="checkbox" name="download_existing" value="1">公開中の回もダウンロードする</label>
<button type="submit">登録</button>
<button type="button" hx-get="/subscriptions/from-url/cancel" hx-target="#subscription-url-result" hx-swap="outerHTML">キャンセル</button>
</form>
{{end}}
{{end}}
</div>
{{end}}
{{define "subscription_url_registered"}}
<div id="subscription-url-result"><p role="status">登録済み</p></div>
<input id="subscription-url-input" type="url" name="series_url" placeholder="https://tver.jp/series/…" required hx-swap-oob="outerHTML">
{{template "subscriptions_list" .Subscriptions}}
{{end}}
{{define "subscriptions_list"}}
<div id="subscriptions-list" {{if .OutOfBand}}hx-swap-oob="outerHTML"{{end}}>
{{range .Items}}
<article class="subscription-item">
<h2>{{.Title}}</h2>
<p>登録日時: {{.CreatedAtText}}/最終確認: {{.LastCheckedText}}/状態: {{if .Enabled}}有効{{else}}一時停止{{end}}/関連ジョブ: {{.Jobs}}件</p>
{{if .CheckErrorText}}
<details class="error"><summary>最後の確認エラー({{.CheckErrorAtText}})</summary><p>{{.CheckErrorText}}</p></details>
{{end}}
{{if .Enabled}}
<form action="/subscriptions/{{.ID}}/pause" method="post" hx-post="/subscriptions/{{.ID}}/pause" hx-target="#subscriptions-list" hx-swap="outerHTML"><button type="submit">一時停止</button></form>
{{else}}
<form action="/subscriptions/{{.ID}}/resume" method="post" hx-post="/subscriptions/{{.ID}}/resume" hx-target="#subscriptions-list" hx-swap="outerHTML"><button type="submit">再開</button></form>
{{end}}
<form action="/subscriptions/{{.ID}}/delete" method="post" hx-post="/subscriptions/{{.ID}}/delete" hx-target="#subscriptions-list" hx-swap="outerHTML" hx-confirm="番組の登録を解除しますか?"><button type="submit">解除</button></form>
</article>
{{else}}
<p>登録番組はありません</p>
{{end}}
</div>
{{end}}
+81
View File
@@ -0,0 +1,81 @@
package worker
import (
"encoding/json"
"strings"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
type outputEvent struct {
Event string `json:"event"`
FormatID string `json:"format_id"`
Filepath string `json:"filepath"`
Progress struct {
DownloadedBytes *float64 `json:"downloaded_bytes"`
TotalBytes *float64 `json:"total_bytes"`
TotalBytesEstimate *float64 `json:"total_bytes_estimate"`
Speed *float64 `json:"speed"`
ETA *float64 `json:"eta"`
FragmentIndex *float64 `json:"fragment_index"`
FragmentCount *float64 `json:"fragment_count"`
} `json:"progress"`
}
func parseProgress(line string) (store.Progress, bool) {
var event outputEvent
if err := json.Unmarshal([]byte(line), &event); err != nil {
return store.Progress{}, false
}
phase := ""
switch event.Event {
case "download":
if strings.Contains(strings.ToLower(event.FormatID), "audio") {
phase = "audio"
} else {
phase = "video"
}
case "postprocess":
phase = "merging"
default:
return store.Progress{}, false
}
total := int64Pointer(event.Progress.TotalBytes)
estimate := false
if total == nil && event.Progress.TotalBytesEstimate != nil {
total = int64Pointer(event.Progress.TotalBytesEstimate)
estimate = true
}
var formatID *string
if event.FormatID != "" {
formatID = &event.FormatID
}
return store.Progress{Phase: phase, DownloadedBytes: int64Pointer(event.Progress.DownloadedBytes),
TotalBytes: total, TotalIsEstimate: estimate, SpeedBPS: event.Progress.Speed,
ETASeconds: event.Progress.ETA, FragmentIndex: int32Pointer(event.Progress.FragmentIndex),
FragmentCount: int32Pointer(event.Progress.FragmentCount), FormatID: formatID}, true
}
func int64Pointer(value *float64) *int64 {
if value == nil {
return nil
}
result := int64(*value)
return &result
}
func int32Pointer(value *float64) *int32 {
if value == nil {
return nil
}
result := int32(*value)
return &result
}
func parseAfterMove(line string) (string, bool) {
var event outputEvent
if err := json.Unmarshal([]byte(line), &event); err != nil || event.Event != "after_move" || event.Filepath == "" {
return "", false
}
return event.Filepath, true
}
+27
View File
@@ -0,0 +1,27 @@
package worker
import "testing"
func TestParseProgress(t *testing.T) {
video, ok := parseProgress(`{"event":"download","format_id":"hls-698-1","progress":{"downloaded_bytes":25,"total_bytes_estimate":100.0,"speed":10.5,"eta":3,"fragment_index":1,"fragment_count":4}}`)
if !ok || video.Phase != "video" || video.DownloadedBytes == nil || *video.DownloadedBytes != 25 || video.TotalBytes == nil || *video.TotalBytes != 100 || !video.TotalIsEstimate || video.SpeedBPS == nil || *video.SpeedBPS != 10.5 || video.ETASeconds == nil || *video.ETASeconds != 3 || video.FragmentIndex == nil || *video.FragmentIndex != 1 || video.FragmentCount == nil || *video.FragmentCount != 4 || video.FormatID == nil || *video.FormatID != "hls-698-1" {
t.Fatalf("映像進捗: %+v ok=%v", video, ok)
}
audio, ok := parseProgress(`{"event":"download","format_id":"hls-ts_AUDIO-0_2","progress":{"downloaded_bytes":50,"total_bytes":50}}`)
if !ok || audio.Phase != "audio" || audio.TotalIsEstimate || audio.TotalBytes == nil || *audio.TotalBytes != 50 {
t.Fatalf("音声進捗: %+v ok=%v", audio, ok)
}
merging, ok := parseProgress(`{"event":"postprocess","progress":{"status":"started","postprocessor":"Merger"}}`)
if !ok || merging.Phase != "merging" {
t.Fatalf("結合進捗: %+v ok=%v", merging, ok)
}
path, ok := parseAfterMove(`{"event":"after_move","filepath":"/videos/incoming/ep.mp4"}`)
if !ok || path != "/videos/incoming/ep.mp4" {
t.Fatalf("移動後パス: %q ok=%v", path, ok)
}
for _, invalid := range []string{"noise", `{"event":"other"}`} {
if _, ok := parseProgress(invalid); ok {
t.Errorf("不正な進捗を許した: %q", invalid)
}
}
}
+504
View File
@@ -0,0 +1,504 @@
package worker
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"syscall"
"time"
"gitea.kazu634.com/kazu634/tver/internal/media"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
var (
errCanceled = errors.New("job canceled")
errShutdown = errors.New("worker shutting down")
errInvalidOutput = errors.New("invalid output")
headerSecretRE = regexp.MustCompile(`(?im)(authorization|cookie)([=:][ \t]*)([^\r\n]+)`)
paramSecretRE = regexp.MustCompile(`(?i)(token|platform_uid|platform_token)([=:][[:space:]]*)([^&[:space:]]+)`)
)
type JobStore interface {
ClaimNext(context.Context) (store.Job, error)
UpdateProgress(context.Context, int64, store.Progress) error
Heartbeat(context.Context, int64) error
GetJob(context.Context, int64) (store.Job, error)
Complete(context.Context, int64, string, int64) error
Fail(context.Context, int64, string, string) error
RequeueRunning(context.Context, int64) error
}
type Config struct {
VideoDir string
YTDLPPath string
FFprobePath string
Concurrency int
PollInterval time.Duration
ProgressInterval time.Duration
HeartbeatInterval time.Duration
CancelInterval time.Duration
}
type Runner struct {
store JobStore
config Config
logger *log.Logger
}
type cappedBuffer struct {
buffer bytes.Buffer
limit int
}
func (b *cappedBuffer) Write(p []byte) (int, error) {
written := len(p)
remaining := b.limit - b.buffer.Len()
if remaining > 0 {
if remaining < len(p) {
p = p[:remaining]
}
_, _ = b.buffer.Write(p)
}
return written, nil
}
func (b *cappedBuffer) String() string { return b.buffer.String() }
func scanLines(reader io.Reader, capture *cappedBuffer) <-chan string {
lines := make(chan string)
go func() {
defer close(lines)
scanner := bufio.NewScanner(reader)
scanner.Buffer(make([]byte, 4096), 1<<20)
for scanner.Scan() {
line := scanner.Text()
if capture != nil {
_, _ = capture.Write([]byte(line + "\n"))
}
lines <- line
}
}()
return lines
}
func New(jobs JobStore, config Config, logger *log.Logger) (*Runner, error) {
if config.VideoDir == "" {
config.VideoDir = "/videos"
}
if config.YTDLPPath == "" {
config.YTDLPPath = "yt-dlp"
}
if config.FFprobePath == "" {
config.FFprobePath = "ffprobe"
}
if config.Concurrency == 0 {
config.Concurrency = 1
}
if config.Concurrency < 1 || config.Concurrency > 2 {
return nil, errors.New("worker concurrency must be 1 or 2")
}
if config.PollInterval <= 0 {
config.PollInterval = time.Second
}
if config.ProgressInterval <= 0 {
config.ProgressInterval = time.Second
}
if config.HeartbeatInterval <= 0 {
config.HeartbeatInterval = 5 * time.Second
}
if config.CancelInterval <= 0 {
config.CancelInterval = 500 * time.Millisecond
}
if logger == nil {
logger = log.Default()
}
return &Runner{store: jobs, config: config, logger: logger}, nil
}
func (r *Runner) Run(ctx context.Context) error {
sem := make(chan struct{}, r.config.Concurrency)
var wg sync.WaitGroup
ticker := time.NewTicker(r.config.PollInterval)
defer ticker.Stop()
for {
for len(sem) < cap(sem) && ctx.Err() == nil {
job, err := r.store.ClaimNext(ctx)
if errors.Is(err, pgx.ErrNoRows) {
break
}
if err != nil {
r.logger.Printf("ジョブ取得に失敗: %v", err)
break
}
sem <- struct{}{}
wg.Add(1)
go func() {
defer wg.Done()
defer func() { <-sem }()
r.ProcessJob(ctx, job)
}()
}
select {
case <-ctx.Done():
wg.Wait()
return nil
case <-ticker.C:
}
}
}
func (r *Runner) ProcessJob(ctx context.Context, job store.Job) {
workDir := filepath.Join(r.config.VideoDir, ".incoming", strconv.FormatInt(job.ID, 10))
if err := os.RemoveAll(workDir); err != nil {
r.fail(job.ID, "unknown", fmt.Sprintf("作業ディレクトリを初期化できません: %v", err))
return
}
if err := os.MkdirAll(workDir, 0o755); err != nil {
r.fail(job.ID, "unknown", fmt.Sprintf("作業ディレクトリを作成できません: %v", err))
return
}
defer os.RemoveAll(workDir)
relative, err := media.RelativeVideoPath(job.SeriesTitle, job.Title, job.EpisodeID)
if err != nil {
r.failIfRunning(job.ID, "invalid_output", redact(err.Error()))
return
}
downloaded, stderr, err := r.download(ctx, job, workDir)
if errors.Is(err, errShutdown) {
r.requeue(job.ID)
return
}
if errors.Is(err, errCanceled) {
return
}
if err != nil {
message := redact(stderr)
if message == "" {
message = redact(err.Error())
}
kind := classifyFailure(message)
if errors.Is(err, errInvalidOutput) {
kind = "invalid_output"
}
r.failIfRunning(job.ID, kind, message)
return
}
if r.canceled(job.ID) {
return
}
if err := r.probe(ctx, downloaded); err != nil {
if ctx.Err() != nil {
r.requeue(job.ID)
return
}
r.failIfRunning(job.ID, "invalid_output", redact(err.Error()))
return
}
if r.canceled(job.ID) {
return
}
if err := r.store.UpdateProgress(ctx, job.ID, store.Progress{Phase: "moving"}); err != nil {
if r.canceled(job.ID) {
return
}
r.failIfRunning(job.ID, "unknown", "移動状態を記録できません: "+err.Error())
return
}
destination := filepath.Join(r.config.VideoDir, relative)
if err := os.MkdirAll(filepath.Dir(destination), 0o755); err != nil {
r.failIfRunning(job.ID, "unknown", "保存先を作成できません: "+err.Error())
return
}
if err := os.Rename(downloaded, destination); err != nil {
r.failIfRunning(job.ID, "unknown", "動画を保存先へ移動できません: "+err.Error())
return
}
info, err := os.Stat(destination)
if err != nil {
r.failIfRunning(job.ID, "unknown", "保存後の動画を確認できません: "+err.Error())
return
}
if err := r.store.Complete(ctx, job.ID, filepath.ToSlash(relative), info.Size()); err != nil {
if r.canceled(job.ID) {
if removeErr := os.Remove(destination); removeErr != nil && !errors.Is(removeErr, os.ErrNotExist) {
r.logger.Printf("取り消されたジョブ %d の保存済みファイル削除に失敗: %v", job.ID, removeErr)
}
return
}
r.logger.Printf("ジョブ %d の完了記録に失敗: %v", job.ID, err)
}
}
func (r *Runner) download(ctx context.Context, job store.Job, workDir string) (string, string, error) {
output := filepath.Join(workDir, "%(id)s.%(ext)s")
args := []string{"--no-playlist", "--no-simulate", "--quiet", "--progress", "--newline",
"--continue", "--retries", "0", "--fragment-retries", "0", "--extractor-retries", "0",
"-f", "bv*[vcodec^=avc1]+ba/b", "--merge-output-format", "mp4",
"--progress-template", `download:{"event":"download","format_id":%(info.format_id)j,"progress":%(progress)j}`,
"--progress-template", `postprocess:{"event":"postprocess","progress":%(progress)j}`,
"--print", `after_move:{"event":"after_move","filepath":%(filepath)j}`, "-o", output,
"https://tver.jp/episodes/" + job.EpisodeID}
cmd := exec.Command(r.config.YTDLPPath, args...)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
stdout, err := cmd.StdoutPipe()
if err != nil {
return "", "", err
}
stderrPipe, err := cmd.StderrPipe()
if err != nil {
return "", "", err
}
if err := cmd.Start(); err != nil {
return "", "", err
}
stderr := cappedBuffer{limit: 64 << 10}
stdoutLines := scanLines(stdout, nil)
stderrLines := scanLines(stderrPipe, &stderr)
waited := make(chan error, 1)
go func() { waited <- cmd.Wait() }()
heartbeat := time.NewTicker(r.config.HeartbeatInterval)
cancelCheck := time.NewTicker(r.config.CancelInterval)
defer heartbeat.Stop()
defer cancelCheck.Stop()
_ = r.store.Heartbeat(ctx, job.ID)
var afterMove string
var pending *store.Progress
nextProgress := time.Time{}
lastPhase := ""
recordProgress := func(progress store.Progress) {
pending = &progress
now := time.Now()
if progress.Phase != lastPhase || now.After(nextProgress) {
_ = r.store.UpdateProgress(ctx, job.ID, progress)
nextProgress = now.Add(r.config.ProgressInterval)
lastPhase = progress.Phase
pending = nil
}
}
stopping := ""
var forceKill <-chan time.Time
ctxDone := ctx.Done()
waitComplete := false
var waitErr error
for {
if waitComplete && stdoutLines == nil && stderrLines == nil {
if pending != nil && stopping == "" {
updateCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
_ = r.store.UpdateProgress(updateCtx, job.ID, *pending)
cancel()
}
if stopping == "canceled" {
return "", stderr.String(), errCanceled
}
if stopping == "shutdown" {
return "", stderr.String(), errShutdown
}
if waitErr != nil {
r.logger.Printf("yt-dlp ジョブ %d 失敗: %s", job.ID, redact(stderr.String()))
return "", stderr.String(), waitErr
}
path, err := pathInside(workDir, afterMove)
if err != nil {
return "", stderr.String(), err
}
return path, stderr.String(), nil
}
select {
case line, ok := <-stdoutLines:
if !ok {
stdoutLines = nil
continue
}
if path, ok := parseAfterMove(line); ok {
afterMove = path
continue
}
if progress, ok := parseProgress(line); ok {
recordProgress(progress)
}
case line, ok := <-stderrLines:
if !ok {
stderrLines = nil
continue
}
if progress, ok := parseProgress(line); ok {
recordProgress(progress)
}
case <-heartbeat.C:
_ = r.store.Heartbeat(ctx, job.ID)
case <-cancelCheck.C:
if r.canceled(job.ID) && stopping == "" {
stopping = "canceled"
if !waitComplete {
stopProcessGroup(cmd)
forceKill = time.After(2 * time.Second)
}
}
case <-ctxDone:
if stopping == "" {
stopping = "shutdown"
if !waitComplete {
stopProcessGroup(cmd)
forceKill = time.After(2 * time.Second)
}
}
ctxDone = nil
case <-forceKill:
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
forceKill = nil
case waitErr = <-waited:
waitComplete = true
waited = nil
forceKill = nil
}
}
}
func stopProcessGroup(cmd *exec.Cmd) {
if cmd.Process == nil {
return
}
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM)
}
func pathInside(root, candidate string) (string, error) {
if candidate == "" {
return "", errors.New("yt-dlp did not report after_move filepath")
}
rootAbs, err := filepath.Abs(root)
if err != nil {
return "", err
}
candidateAbs, err := filepath.Abs(candidate)
if err != nil {
return "", err
}
rel, err := filepath.Rel(rootAbs, candidateAbs)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("%w: yt-dlp filepath leaves working directory", errInvalidOutput)
}
info, err := os.Lstat(candidateAbs)
if err != nil {
return "", fmt.Errorf("%w: yt-dlp output file is missing: %v", errInvalidOutput, err)
}
if !info.Mode().IsRegular() {
return "", fmt.Errorf("%w: yt-dlp output is not a regular file", errInvalidOutput)
}
rootReal, err := filepath.EvalSymlinks(rootAbs)
if err != nil {
return "", fmt.Errorf("%w: resolve working directory: %v", errInvalidOutput, err)
}
candidateReal, err := filepath.EvalSymlinks(candidateAbs)
if err != nil {
return "", fmt.Errorf("%w: resolve yt-dlp output: %v", errInvalidOutput, err)
}
realRelative, err := filepath.Rel(rootReal, candidateReal)
if err != nil || realRelative == ".." || strings.HasPrefix(realRelative, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("%w: yt-dlp output resolves outside working directory", errInvalidOutput)
}
return candidateAbs, nil
}
func (r *Runner) probe(ctx context.Context, path string) error {
cmd := exec.CommandContext(ctx, r.config.FFprobePath, "-v", "error", "-show_entries",
"format=format_name", "-show_entries", "stream=codec_type,codec_name", "-of", "json", path)
output, err := cmd.Output()
if err != nil {
return fmt.Errorf("ffprobe failed: %w", err)
}
var result struct {
Format struct {
FormatName string `json:"format_name"`
} `json:"format"`
Streams []struct {
CodecType string `json:"codec_type"`
CodecName string `json:"codec_name"`
} `json:"streams"`
}
if err := json.Unmarshal(output, &result); err != nil {
return fmt.Errorf("ffprobe output is not JSON: %w", err)
}
video, audio := false, false
for _, stream := range result.Streams {
video = video || stream.CodecType == "video" && stream.CodecName == "h264"
audio = audio || stream.CodecType == "audio" && stream.CodecName == "aac"
}
if !strings.Contains(result.Format.FormatName, "mp4") || !video || !audio {
return fmt.Errorf("expected MP4 with H.264 and AAC, got format=%s video=%t audio=%t", result.Format.FormatName, video, audio)
}
return nil
}
func (r *Runner) canceled(id int64) bool {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
job, err := r.store.GetJob(ctx, id)
return err == nil && job.Status == "canceled"
}
func (r *Runner) failIfRunning(id int64, kind, message string) {
if r.canceled(id) {
return
}
r.fail(id, kind, message)
}
func (r *Runner) fail(id int64, kind, message string) {
if len(message) > 8192 {
message = message[:8192]
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := r.store.Fail(ctx, id, kind, message); err != nil {
r.logger.Printf("ジョブ %d の失敗記録に失敗: %v", id, err)
}
}
func (r *Runner) requeue(id int64) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := r.store.RequeueRunning(ctx, id); err != nil && !errors.Is(err, store.ErrInvalidTransition) {
r.logger.Printf("ジョブ %d の待機中への復帰に失敗: %v", id, err)
}
}
func classifyFailure(message string) string {
lower := strings.ToLower(message)
for _, test := range []struct {
kind string
patterns []string
}{
{"drm", []string{"drm", "digital rights management", "protected by"}},
{"geo", []string{"geo-restricted", "not available in your country", "not available from your location", "geo restriction", "region restriction", "geographic restriction"}},
{"network", []string{"network", "connection", "timed out", "temporary failure", "unable to download", "proxy error", "http error 5"}},
{"unavailable", []string{"media_not_found", "video unavailable", "not available", "no longer available", "http error 404", "episode does not exist"}},
{"extract", []string{"extractor error", "unable to extract", "unsupported url", "no video formats", "after_move", "output file is missing"}},
} {
for _, pattern := range test.patterns {
if strings.Contains(lower, pattern) {
return test.kind
}
}
}
return "unknown"
}
func redact(message string) string {
message = headerSecretRE.ReplaceAllString(message, "$1$2[REDACTED]")
return strings.TrimSpace(paramSecretRE.ReplaceAllString(message, "$1$2[REDACTED]"))
}
+678
View File
@@ -0,0 +1,678 @@
package worker
import (
"context"
"errors"
"io"
"log"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/media"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
type fakeStore struct {
mu sync.Mutex
job store.Job
progress []store.Progress
heartbeats int
completed bool
completePath string
completeSize int64
failedKind, failedMessage string
requeued bool
claimed bool
}
type queueStore struct {
mu sync.Mutex
jobs []store.Job
claimCounts map[int64]int
active int
maxActive int
}
func (f *queueStore) ClaimNext(context.Context) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
for i := range f.jobs {
if f.jobs[i].Status != "queued" {
continue
}
f.jobs[i].Status = "running"
f.claimCounts[f.jobs[i].ID]++
f.active++
if f.active > f.maxActive {
f.maxActive = f.active
}
return f.jobs[i], nil
}
return store.Job{}, pgx.ErrNoRows
}
func (f *queueStore) UpdateProgress(_ context.Context, id int64, _ store.Progress) error {
return f.requireRunning(id)
}
func (f *queueStore) Heartbeat(_ context.Context, id int64) error {
return f.requireRunning(id)
}
func (f *queueStore) requireRunning(id int64) error {
f.mu.Lock()
defer f.mu.Unlock()
for i := range f.jobs {
if f.jobs[i].ID == id && f.jobs[i].Status == "running" {
return nil
}
}
return store.ErrInvalidTransition
}
func (f *queueStore) GetJob(_ context.Context, id int64) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
for _, job := range f.jobs {
if job.ID == id {
return job, nil
}
}
return store.Job{}, pgx.ErrNoRows
}
func (f *queueStore) Complete(_ context.Context, id int64, _ string, _ int64) error {
return f.finish(id, "succeeded")
}
func (f *queueStore) Fail(_ context.Context, id int64, _, _ string) error {
return f.finish(id, "failed")
}
func (f *queueStore) RequeueRunning(_ context.Context, id int64) error {
return f.finish(id, "queued")
}
func (f *queueStore) finish(id int64, status string) error {
f.mu.Lock()
defer f.mu.Unlock()
for i := range f.jobs {
if f.jobs[i].ID == id && f.jobs[i].Status == "running" {
f.jobs[i].Status = status
f.active--
return nil
}
}
return store.ErrInvalidTransition
}
func (f *queueStore) state() ([]store.Job, map[int64]int, int) {
f.mu.Lock()
defer f.mu.Unlock()
jobs := append([]store.Job(nil), f.jobs...)
counts := make(map[int64]int, len(f.claimCounts))
for id, count := range f.claimCounts {
counts[id] = count
}
return jobs, counts, f.maxActive
}
func (f *fakeStore) ClaimNext(context.Context) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
if f.claimed || f.job.Status != "queued" {
return store.Job{}, pgx.ErrNoRows
}
f.claimed = true
f.job.Status = "running"
return f.job, nil
}
func (f *fakeStore) UpdateProgress(_ context.Context, _ int64, p store.Progress) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.progress = append(f.progress, p)
return nil
}
func (f *fakeStore) Heartbeat(context.Context, int64) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.heartbeats++
return nil
}
func (f *fakeStore) GetJob(context.Context, int64) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
return f.job, nil
}
func (f *fakeStore) Complete(_ context.Context, _ int64, path string, size int64) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.job.Status = "succeeded"
f.completed, f.completePath, f.completeSize = true, path, size
return nil
}
func (f *fakeStore) Fail(_ context.Context, _ int64, kind, message string) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.job.Status = "failed"
f.failedKind, f.failedMessage = kind, message
return nil
}
func (f *fakeStore) RequeueRunning(context.Context, int64) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.job.Status = "queued"
f.requeued = true
return nil
}
func (f *fakeStore) setStatus(status string) {
f.mu.Lock()
defer f.mu.Unlock()
f.job.Status = status
}
func (f *fakeStore) heartbeatCount() int {
f.mu.Lock()
defer f.mu.Unlock()
return f.heartbeats
}
func testRunner(t *testing.T, jobs *fakeStore, videoDir string) *Runner {
t.Helper()
ytdlp := filepath.Join("testdata", "fake-yt-dlp.sh")
ffprobe := filepath.Join("testdata", "fake-ffprobe.sh")
for _, path := range []string{ytdlp, ffprobe} {
info, err := os.Stat(path)
if err != nil || info.Mode().Perm()&0o111 == 0 {
t.Fatalf("偽コマンドが実行可能ではありません: %s: %v", path, err)
}
}
runner, err := New(jobs, Config{VideoDir: videoDir, YTDLPPath: ytdlp, FFprobePath: ffprobe,
Concurrency: 1, PollInterval: 5 * time.Millisecond, ProgressInterval: time.Millisecond,
HeartbeatInterval: 10 * time.Millisecond, CancelInterval: 5 * time.Millisecond}, log.New(io.Discard, "", 0))
if err != nil {
t.Fatal(err)
}
return runner
}
func runningJob() store.Job {
return store.Job{ID: 7, EpisodeID: "ep123", SeriesTitle: "番組名", Title: "第1話", Status: "running"}
}
func relativeVideoPath(t *testing.T, seriesTitle, episodeTitle, episodeID string) string {
t.Helper()
path, err := media.RelativeVideoPath(seriesTitle, episodeTitle, episodeID)
if err != nil {
t.Fatal(err)
}
return path
}
func TestProcessJobDownloadsProbesMovesAndCleans(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
argsFile := filepath.Join(dir, "args")
t.Setenv("FAKE_ARGS_FILE", argsFile)
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
want := filepath.ToSlash(relativeVideoPath(t, "番組名", "第1話", "ep123"))
if !jobs.completed || jobs.completePath != want || jobs.completeSize == 0 {
t.Fatalf("complete: done=%v path=%q size=%d", jobs.completed, jobs.completePath, jobs.completeSize)
}
if _, err := os.Stat(filepath.Join(dir, filepath.FromSlash(want))); err != nil {
t.Fatalf("保存先: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
args, err := os.ReadFile(argsFile)
if err != nil {
t.Fatal(err)
}
argumentText := string(args)
for _, want := range []string{"--newline", "--no-playlist", "--no-simulate", "--quiet", "--progress", "--continue", "--merge-output-format", "bv*[vcodec^=avc1]+ba/b", `download:{"event":"download","format_id":%(info.format_id)j,"progress":%(progress)j}`, `postprocess:{"event":"postprocess","progress":%(progress)j}`, `after_move:{"event":"after_move","filepath":%(filepath)j}`, filepath.Join(dir, ".incoming", "7", "%(id)s.%(ext)s")} {
if !strings.Contains(argumentText, want) {
t.Errorf("yt-dlp 引数に %q がない: %s", want, argumentText)
}
}
if strings.Contains(argumentText, "番組名") || strings.Contains(argumentText, "第1話") {
t.Errorf("出力テンプレートに番組情報が入った: %s", argumentText)
}
phases := make(map[string]bool)
for _, progress := range jobs.progress {
phases[progress.Phase] = true
}
for _, phase := range []string{"video", "audio", "merging", "moving"} {
if !phases[phase] {
t.Errorf("進捗段階 %s がない: %+v", phase, jobs.progress)
}
}
if jobs.heartbeats < 2 {
t.Errorf("heartbeat が少ない: %d", jobs.heartbeats)
}
}
func TestProcessJobRejectsLongEpisodeIDAndCleans(t *testing.T) {
dir := t.TempDir()
argsFile := filepath.Join(dir, "yt-dlp-args")
t.Setenv("FAKE_ARGS_FILE", argsFile)
workDir := filepath.Join(dir, ".incoming", "7")
if err := os.MkdirAll(workDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(workDir, "stale"), []byte("stale"), 0o644); err != nil {
t.Fatal(err)
}
job := runningJob()
job.EpisodeID = strings.Repeat("i", 65)
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
if jobs.completed || jobs.failedKind != "invalid_output" || !strings.Contains(jobs.failedMessage, "64バイト以下") {
t.Fatalf("長いIDの失敗: kind=%q message=%q completed=%v", jobs.failedKind, jobs.failedMessage, jobs.completed)
}
if _, err := os.Stat(argsFile); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("yt-dlp が起動された: %v", err)
}
if _, err := os.Stat(workDir); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestProcessJobKeepsPercentNamesOutOfYTDLPTemplate(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
argsFile := filepath.Join(dir, "args")
t.Setenv("FAKE_ARGS_FILE", argsFile)
job := runningJob()
job.SeriesTitle = "100%番組"
job.Title = "第%1話"
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
if !jobs.completed {
t.Fatalf("%% を含む番組を完了できなかった: %+v", jobs)
}
args, err := os.ReadFile(argsFile)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(args), job.SeriesTitle) || strings.Contains(string(args), job.Title) {
t.Fatalf("番組情報が yt-dlp 引数に入った: %s", args)
}
want := filepath.ToSlash(relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID))
if jobs.completePath != want {
t.Fatalf("保存先=%q want=%q", jobs.completePath, want)
}
}
func TestProcessJobReplacesExistingDestination(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
job := runningJob()
relative := relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID)
destination := filepath.Join(dir, relative)
if err := os.MkdirAll(filepath.Dir(destination), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(destination, []byte("stale file from previous attempt"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
got, err := os.ReadFile(destination)
if err != nil {
t.Fatal(err)
}
if !jobs.completed || string(got) != "fake mp4" || jobs.completeSize != int64(len(got)) {
t.Fatalf("既存ファイルの置換: completed=%v content=%q size=%d", jobs.completed, got, jobs.completeSize)
}
}
func TestProcessJobRejectsAfterMoveSymlinkOutsideWorkDirectory(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "symlink")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
outside := filepath.Join(t.TempDir(), "outside.mp4")
if err := os.WriteFile(outside, []byte("outside video"), 0o644); err != nil {
t.Fatal(err)
}
t.Setenv("FAKE_OUTSIDE_FILE", outside)
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
if jobs.completed || jobs.failedKind != "invalid_output" {
t.Fatalf("作業ディレクトリ外を指すシンボリックリンクを受理した: %+v", jobs)
}
if got, err := os.ReadFile(outside); err != nil || string(got) != "outside video" {
t.Fatalf("作業ディレクトリ外のファイルを変更した: %q %v", got, err)
}
}
func TestPathInsideSymlinkDerivedCases(t *testing.T) {
root := t.TempDir()
insideDir := filepath.Join(root, "real")
if err := os.Mkdir(insideDir, 0o755); err != nil {
t.Fatal(err)
}
insideFile := filepath.Join(insideDir, "video.mp4")
if err := os.WriteFile(insideFile, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
outsideDir := t.TempDir()
outsideFile := filepath.Join(outsideDir, "outside.mp4")
if err := os.WriteFile(outsideFile, []byte("outside"), 0o644); err != nil {
t.Fatal(err)
}
internalFileLink := filepath.Join(root, "internal-file-link.mp4")
internalDirLink := filepath.Join(root, "internal-dir-link")
externalDirLink := filepath.Join(root, "external-dir-link")
rootLink := filepath.Join(t.TempDir(), "root-link")
for link, target := range map[string]string{
internalFileLink: insideFile,
internalDirLink: insideDir,
externalDirLink: outsideDir,
rootLink: root,
} {
if err := os.Symlink(target, link); err != nil {
t.Fatal(err)
}
}
for _, tt := range []struct {
name, testRoot, candidate string
wantOK bool
}{
{"通常ファイル", root, insideFile, true},
{"作業ディレクトリ内を指すファイルリンク", root, internalFileLink, false},
{"after_moveがディレクトリ", root, insideDir, false},
{"作業ディレクトリ内を指すディレクトリリンク配下", root, filepath.Join(internalDirLink, "video.mp4"), true},
{"作業ディレクトリ外を指すディレクトリリンク配下", root, filepath.Join(externalDirLink, "outside.mp4"), false},
{"作業ディレクトリ自体がリンク", rootLink, filepath.Join(rootLink, "real", "video.mp4"), true},
} {
t.Run(tt.name, func(t *testing.T) {
got, err := pathInside(tt.testRoot, tt.candidate)
if tt.wantOK {
want, absErr := filepath.Abs(tt.candidate)
if absErr != nil {
t.Fatal(absErr)
}
if err != nil || got != want {
t.Fatalf("pathInside()=(%q, %v) want=(%q, nil)", got, err, want)
}
return
}
if !errors.Is(err, errInvalidOutput) || got != "" {
t.Fatalf("pathInside()=(%q, %v), invalid_output を期待", got, err)
}
})
}
}
func TestProcessJobDestinationDirectoryFailureCleansIncoming(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
job := runningJob()
seriesDir := filepath.Join(dir, filepath.Dir(relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID)))
if err := os.WriteFile(seriesDir, []byte("blocks directory"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
if jobs.completed || jobs.failedKind != "unknown" {
t.Fatalf("保存先作成失敗: %+v", jobs)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestFailureClassificationAndRedaction(t *testing.T) {
for _, tt := range []struct{ mode, kind string }{
{"drm", "drm"}, {"geo", "geo"}, {"network", "network"},
{"unavailable", "unavailable"}, {"extract", "extract"}, {"unknown", "unknown"},
} {
t.Run(tt.mode, func(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", tt.mode)
dir := t.TempDir()
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
if jobs.failedKind != tt.kind || jobs.completed || strings.Contains(jobs.failedMessage, "secret-value") {
t.Fatalf("failure: kind=%q message=%q completed=%v", jobs.failedKind, jobs.failedMessage, jobs.completed)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
})
}
}
func TestInvalidProbeFailsAndCleans(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
for _, mode := range []string{"bad-container", "bad-video", "bad-audio", "non-json", "failure"} {
t.Run(mode, func(t *testing.T) {
t.Setenv("FAKE_FFPROBE_MODE", mode)
dir := t.TempDir()
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
if jobs.failedKind != "invalid_output" || jobs.completed {
t.Fatalf("ffprobe 不合格: kind=%q complete=%v", jobs.failedKind, jobs.completed)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
})
}
}
func TestProgressIsThrottledButHeartbeatContinues(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "burst")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
jobs := &fakeStore{job: runningJob()}
runner := testRunner(t, jobs, dir)
runner.config.ProgressInterval = 50 * time.Millisecond
runner.ProcessJob(context.Background(), jobs.job)
downloadUpdates := 0
for _, progress := range jobs.progress {
if progress.Phase == "video" {
downloadUpdates++
}
}
if downloadUpdates > 2 || downloadUpdates == 0 {
t.Fatalf("進捗の間引き: %d updates (%+v)", downloadUpdates, jobs.progress)
}
if jobs.heartbeats < 5 {
t.Fatalf("進捗が止まった間の heartbeat: %d", jobs.heartbeats)
}
}
func TestRunConcurrencyTwoDoesNotDuplicateAndIsolatesFailure(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "mixed")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
jobs := &queueStore{
jobs: []store.Job{
{ID: 7, EpisodeID: "ep-ok", SeriesTitle: "成功番組", Title: "成功回", Status: "queued"},
{ID: 8, EpisodeID: "ep-fail", SeriesTitle: "失敗番組", Title: "失敗回", Status: "queued"},
},
claimCounts: make(map[int64]int),
}
runner, err := New(jobs, Config{VideoDir: dir,
YTDLPPath: filepath.Join("testdata", "fake-yt-dlp.sh"), FFprobePath: filepath.Join("testdata", "fake-ffprobe.sh"),
Concurrency: 2, PollInterval: 5 * time.Millisecond, ProgressInterval: time.Millisecond,
HeartbeatInterval: 10 * time.Millisecond, CancelInterval: 5 * time.Millisecond}, log.New(io.Discard, "", 0))
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- runner.Run(ctx) }()
deadline := time.Now().Add(3 * time.Second)
for time.Now().Before(deadline) {
state, _, _ := jobs.state()
if state[0].Status == "succeeded" && state[1].Status == "failed" {
break
}
time.Sleep(5 * time.Millisecond)
}
cancel()
if err := <-done; err != nil {
t.Fatal(err)
}
state, counts, maxActive := jobs.state()
if state[0].Status != "succeeded" || state[1].Status != "failed" {
t.Fatalf("片方の失敗がもう片方へ波及した: %+v", state)
}
if counts[7] != 1 || counts[8] != 1 {
t.Fatalf("ジョブを二重取得した: %+v", counts)
}
if maxActive != 2 {
t.Fatalf("並行実行数の最大値=%d want=2", maxActive)
}
for _, id := range []int64{7, 8} {
if _, err := os.Stat(filepath.Join(dir, ".incoming", strconv.FormatInt(id, 10))); !errors.Is(err, os.ErrNotExist) {
t.Errorf("ジョブ %d の作業ディレクトリが残った: %v", id, err)
}
}
}
func TestCancelStopsProcessGroupAndLeavesCanceled(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "hang-drm")
dir := t.TempDir()
ready := filepath.Join(dir, "ready")
terminated := filepath.Join(dir, "terminated")
t.Setenv("FAKE_READY_FILE", ready)
t.Setenv("FAKE_TERM_FILE", terminated)
jobs := &fakeStore{job: runningJob()}
runner := testRunner(t, jobs, dir)
job := jobs.job
done := make(chan struct{})
go func() { runner.ProcessJob(context.Background(), job); close(done) }()
waitForFile(t, ready)
waitForHeartbeatCount(t, jobs, 2)
jobs.setStatus("canceled")
waitDone(t, done)
if jobs.failedKind != "" || jobs.requeued || jobs.completed || jobs.job.Status != "canceled" {
t.Fatalf("取り消し後: %+v", jobs)
}
waitForFile(t, terminated)
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestShutdownStopsProcessAndRequeues(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "hang")
dir := t.TempDir()
ready := filepath.Join(dir, "ready")
t.Setenv("FAKE_READY_FILE", ready)
jobs := &fakeStore{job: runningJob()}
ctx, cancel := context.WithCancel(context.Background())
runner := testRunner(t, jobs, dir)
job := jobs.job
done := make(chan struct{})
go func() { runner.ProcessJob(ctx, job); close(done) }()
waitForFile(t, ready)
cancel()
waitDone(t, done)
if !jobs.requeued || jobs.job.Status != "queued" || jobs.failedKind != "" {
t.Fatalf("終了時の復帰: %+v", jobs)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestRunnerRejectsConcurrencyOutsideRange(t *testing.T) {
for _, concurrency := range []int{-1, 3} {
if _, err := New(&fakeStore{}, Config{Concurrency: concurrency}, nil); err == nil {
t.Errorf("concurrency=%d を許した", concurrency)
}
}
}
func TestFailureClassificationOrderAndRedaction(t *testing.T) {
for _, tt := range []struct{ message, want string }{
{"DRM geo-restricted connection video unavailable unable to extract", "drm"},
{"geo-restricted connection video unavailable unable to extract", "geo"},
{"connection video unavailable unable to extract", "network"},
{"video unavailable unable to extract", "unavailable"},
{"unable to extract", "extract"},
{"different error", "unknown"},
} {
if got := classifyFailure(tt.message); got != tt.want {
t.Errorf("classifyFailure(%q)=%q want=%q", tt.message, got, tt.want)
}
}
redacted := redact("Authorization: Bearer bearer-secret\nCookie: session=session-secret\ntoken=token-secret platform_uid=uid-secret")
for _, secret := range []string{"bearer-secret", "session-secret", "token-secret", "uid-secret"} {
if strings.Contains(redacted, secret) {
t.Errorf("秘密が残った: %q", redacted)
}
}
}
func waitForFile(t *testing.T, path string) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
if _, err := os.Stat(path); err == nil {
return
}
time.Sleep(5 * time.Millisecond)
}
t.Fatalf("ファイル待ちがタイムアウト: %s", path)
}
func waitDone(t *testing.T, done <-chan struct{}) {
t.Helper()
select {
case <-done:
case <-time.After(3 * time.Second):
t.Fatal("worker の終了待ちがタイムアウト")
}
}
func waitForHeartbeatCount(t *testing.T, jobs *fakeStore, want int) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
if jobs.heartbeatCount() >= want {
return
}
time.Sleep(5 * time.Millisecond)
}
t.Fatalf("heartbeat 待ちがタイムアウト: got=%d want=%d", jobs.heartbeatCount(), want)
}
+156
View File
@@ -0,0 +1,156 @@
package worker
import (
"context"
"errors"
"log"
"strings"
"time"
"unicode/utf8"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
type SubscriptionStore interface {
ListDueSubscriptions(context.Context, time.Time) ([]store.Subscription, error)
CompleteSubscriptionCheck(context.Context, string, []store.Episode, int, time.Time) (int, error)
RecordSubscriptionCheckFailure(context.Context, string, time.Time, string) error
}
type SeriesFetcher interface {
FetchSeriesEpisodes(context.Context, string) ([]tver.Episode, error)
}
type SubscriptionSchedulerConfig struct {
ScanInterval time.Duration
DueAfter time.Duration
SeriesInterval time.Duration
PreviewMaxSeconds int
Now func() time.Time
After func(time.Duration) <-chan time.Time
}
type SubscriptionScheduler struct {
store SubscriptionStore
fetcher SeriesFetcher
config SubscriptionSchedulerConfig
logger *log.Logger
}
func NewSubscriptionScheduler(subscriptions SubscriptionStore, fetcher SeriesFetcher, config SubscriptionSchedulerConfig, logger *log.Logger) (*SubscriptionScheduler, error) {
if subscriptions == nil || fetcher == nil {
return nil, errors.New("subscription store and fetcher are required")
}
if config.ScanInterval <= 0 {
config.ScanInterval = time.Hour
}
if config.DueAfter <= 0 {
config.DueAfter = 24 * time.Hour
}
if config.SeriesInterval <= 0 {
config.SeriesInterval = 3 * time.Second
}
if config.PreviewMaxSeconds == 0 {
config.PreviewMaxSeconds = tver.PreviewMaxSeconds
}
if config.PreviewMaxSeconds < 0 {
return nil, errors.New("preview threshold must not be negative")
}
if config.Now == nil {
config.Now = time.Now
}
if config.After == nil {
config.After = time.After
}
if logger == nil {
logger = log.Default()
}
return &SubscriptionScheduler{store: subscriptions, fetcher: fetcher, config: config, logger: logger}, nil
}
// Run は起動直後に確認し、その後は ScanInterval ごとに繰り返す。
func (s *SubscriptionScheduler) Run(ctx context.Context) {
for {
s.checkDue(ctx)
if !s.wait(ctx, s.config.ScanInterval) {
return
}
}
}
func (s *SubscriptionScheduler) checkDue(ctx context.Context) {
now := s.config.Now()
subscriptions, err := s.store.ListDueSubscriptions(ctx, now.Add(-s.config.DueAfter))
if err != nil {
if ctx.Err() == nil {
s.logger.Printf("確認対象の登録番組を取得できません: %v", err)
}
return
}
for index, subscription := range subscriptions {
if index > 0 && !s.wait(ctx, s.config.SeriesInterval) {
return
}
episodes, err := s.fetcher.FetchSeriesEpisodes(ctx, subscription.SeriesID)
if err != nil {
if ctx.Err() != nil {
return
}
message := shortError(err)
if recordErr := s.store.RecordSubscriptionCheckFailure(ctx, subscription.SeriesID, s.config.Now(), message); recordErr != nil &&
!errors.Is(recordErr, store.ErrInvalidTransition) {
s.logger.Printf("登録番組 %s の確認エラーを記録できません: %v", subscription.SeriesID, recordErr)
}
s.logger.Printf("登録番組 %s の公開中エピソードを取得できません: %s", subscription.SeriesID, message)
continue
}
items := make([]store.Episode, 0, len(episodes))
for _, episode := range episodes {
items = append(items, subscriptionEpisode(episode))
}
queued, err := s.store.CompleteSubscriptionCheck(ctx, subscription.SeriesID, items,
s.config.PreviewMaxSeconds, s.config.Now())
if err != nil {
if ctx.Err() != nil {
return
}
if !errors.Is(err, store.ErrSubscriptionPaused) && !errors.Is(err, store.ErrInvalidTransition) {
s.logger.Printf("登録番組 %s の確認結果を保存できません: %v", subscription.SeriesID, err)
}
continue
}
s.logger.Printf("登録番組 %s を確認しました(新規ジョブ=%d)", subscription.SeriesID, queued)
}
}
func (s *SubscriptionScheduler) wait(ctx context.Context, duration time.Duration) bool {
select {
case <-ctx.Done():
return false
case <-s.config.After(duration):
return true
}
}
func subscriptionEpisode(episode tver.Episode) store.Episode {
item := store.Episode{ID: episode.ID, SeriesID: episode.SeriesID, SeriesTitle: episode.SeriesTitle,
Title: episode.Title, BroadcasterName: episode.BroadcasterName,
DurationSeconds: int(episode.Duration), ThumbnailPath: episode.ThumbnailPath}
if episode.EndAt > 0 {
end := time.Unix(episode.EndAt, 0)
item.EndAt = &end
}
return item
}
func shortError(err error) string {
message := redact(err.Error())
message = strings.Join(strings.Fields(message), " ")
const maxRunes = 512
if utf8.RuneCountInString(message) <= maxRunes {
return message
}
runes := []rune(message)
return string(runes[:maxRunes])
}
+253
View File
@@ -0,0 +1,253 @@
package worker
import (
"bytes"
"context"
"errors"
"log"
"strings"
"sync"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
type schedulerCompletion struct {
seriesID string
episodes []store.Episode
limit int
at time.Time
}
type schedulerFailure struct {
seriesID, message string
at time.Time
}
type fakeSubscriptionStore struct {
mu sync.Mutex
due []store.Subscription
cutoffs []time.Time
completions []schedulerCompletion
failures []schedulerFailure
listErr error
completeErr error
}
func (f *fakeSubscriptionStore) ListDueSubscriptions(_ context.Context, cutoff time.Time) ([]store.Subscription, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.cutoffs = append(f.cutoffs, cutoff)
return append([]store.Subscription(nil), f.due...), f.listErr
}
func (f *fakeSubscriptionStore) CompleteSubscriptionCheck(_ context.Context, seriesID string, episodes []store.Episode, limit int, at time.Time) (int, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.completions = append(f.completions, schedulerCompletion{seriesID, append([]store.Episode(nil), episodes...), limit, at})
return len(episodes), f.completeErr
}
func (f *fakeSubscriptionStore) RecordSubscriptionCheckFailure(_ context.Context, seriesID string, at time.Time, message string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.failures = append(f.failures, schedulerFailure{seriesID, message, at})
return nil
}
type fakeSeriesFetcher struct {
mu sync.Mutex
calls []string
episodes map[string][]tver.Episode
errors map[string]error
block bool
started chan string
}
func (f *fakeSeriesFetcher) FetchSeriesEpisodes(ctx context.Context, seriesID string) ([]tver.Episode, error) {
f.mu.Lock()
f.calls = append(f.calls, seriesID)
block := f.block
started := f.started
episodes := append([]tver.Episode(nil), f.episodes[seriesID]...)
err := f.errors[seriesID]
f.mu.Unlock()
if started != nil {
started <- seriesID
}
if block {
<-ctx.Done()
return nil, ctx.Err()
}
return episodes, err
}
func TestSubscriptionSchedulerChecksImmediatelyAndMapsEpisodes(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}, {SeriesID: "two"}}}
fetcher := &fakeSeriesFetcher{episodes: map[string][]tver.Episode{
"one": {{ID: "ep1", SeriesID: "one", SeriesTitle: "番組", Title: "回", BroadcasterName: "局", Duration: 180, EndAt: 1_800_000_000, ThumbnailPath: "/image"}},
"two": {{ID: "ep2", SeriesID: "two", Duration: 181}},
}, errors: map[string]error{}}
var waits []time.Duration
after := func(duration time.Duration) <-chan time.Time {
waits = append(waits, duration)
ch := make(chan time.Time, 1)
ch <- now
return ch
}
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
DueAfter: 24 * time.Hour, SeriesInterval: 4 * time.Second,
Now: func() time.Time { return now }, After: after,
}, log.New(&bytes.Buffer{}, "", 0))
if err != nil {
t.Fatal(err)
}
scheduler.checkDue(context.Background())
if len(subscriptions.cutoffs) != 1 || !subscriptions.cutoffs[0].Equal(now.Add(-24*time.Hour)) {
t.Fatalf("確認対象日時: %v", subscriptions.cutoffs)
}
if len(fetcher.calls) != 2 || fetcher.calls[0] != "one" || fetcher.calls[1] != "two" || len(waits) != 1 || waits[0] != 4*time.Second {
t.Fatalf("順次確認と間隔: calls=%v waits=%v", fetcher.calls, waits)
}
if len(subscriptions.completions) != 2 || subscriptions.completions[0].limit != tver.PreviewMaxSeconds || !subscriptions.completions[0].at.Equal(now) {
t.Fatalf("確認完了: %+v", subscriptions.completions)
}
got := subscriptions.completions[0].episodes[0]
if got.ID != "ep1" || got.DurationSeconds != 180 || got.EndAt == nil || got.EndAt.Unix() != 1_800_000_000 {
t.Fatalf("エピソード変換: %+v", got)
}
}
func TestShortErrorLimitsUnicodeByCharacters(t *testing.T) {
message := shortError(errors.New(strings.Repeat("界", 260) + "\n\t" + strings.Repeat("語", 300)))
if got := len([]rune(message)); got != 512 {
t.Fatalf("エラー文字数=%d want=512", got)
}
if strings.ContainsAny(message, "\r\n\t") || strings.Contains(message, " ") {
t.Fatalf("空白を1行へ正規化していない: %q", message)
}
if !strings.HasSuffix(message, strings.Repeat("語", 251)) {
t.Fatalf("UTF-8文字の途中で切断または境界が不正: %q", message)
}
}
func TestSubscriptionSchedulerRecordsRedactedFailureWithoutCompleting(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "failed"}}}
fetcher := &fakeSeriesFetcher{errors: map[string]error{
"failed": errors.New("request failed\nplatform_token=secret-token Cookie: session-secret"),
}}
var logs bytes.Buffer
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{Now: func() time.Time { return now }}, log.New(&logs, "", 0))
if err != nil {
t.Fatal(err)
}
scheduler.checkDue(context.Background())
if len(subscriptions.completions) != 0 || len(subscriptions.failures) != 1 || !subscriptions.failures[0].at.Equal(now) {
t.Fatalf("失敗の記録: completions=%v failures=%v", subscriptions.completions, subscriptions.failures)
}
message := subscriptions.failures[0].message
if strings.Contains(message, "secret-token") || strings.Contains(message, "session-secret") || strings.Contains(logs.String(), "secret") || strings.Contains(message, "\n") {
t.Fatalf("秘密値または改行が残った: message=%q logs=%q", message, logs.String())
}
}
func TestSubscriptionSchedulerRunStartsImmediatelyThenUsesScanInterval(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
subscriptions := &fakeSubscriptionStore{}
fetcher := &fakeSeriesFetcher{}
requested := make(chan time.Duration, 2)
tick := make(chan time.Time, 1)
after := func(duration time.Duration) <-chan time.Time {
requested <- duration
return tick
}
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
ScanInterval: time.Hour, Now: func() time.Time { return now }, After: after,
}, nil)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { scheduler.Run(ctx); close(done) }()
select {
case duration := <-requested:
if duration != time.Hour {
t.Fatalf("走査間隔: %v", duration)
}
case <-time.After(time.Second):
t.Fatal("起動直後の確認が行われなかった")
}
subscriptions.mu.Lock()
initialCalls := len(subscriptions.cutoffs)
subscriptions.mu.Unlock()
if initialCalls != 1 {
t.Fatalf("起動直後の確認回数: %d", initialCalls)
}
tick <- now
select {
case <-requested:
case <-time.After(time.Second):
t.Fatal("1時間後の確認が行われなかった")
}
cancel()
select {
case <-done:
case <-time.After(time.Second):
t.Fatal("終了できなかった")
}
subscriptions.mu.Lock()
defer subscriptions.mu.Unlock()
if len(subscriptions.cutoffs) != 2 {
t.Fatalf("確認回数: %d", len(subscriptions.cutoffs))
}
}
func TestSubscriptionSchedulerCancellationInterruptsFetchAndSeriesWait(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
t.Run("fetch", func(t *testing.T) {
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}}}
fetcher := &fakeSeriesFetcher{block: true, started: make(chan string, 1)}
scheduler, _ := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{Now: func() time.Time { return now }}, nil)
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { scheduler.checkDue(ctx); close(done) }()
select {
case <-fetcher.started:
case <-time.After(time.Second):
t.Fatal("取得が開始されなかった")
}
cancel()
select {
case <-done:
case <-time.After(time.Second):
t.Fatal("取得中断で終了しなかった")
}
if len(subscriptions.failures) != 0 || len(subscriptions.completions) != 0 {
t.Fatalf("中断した番組を記録した: failures=%v completions=%v", subscriptions.failures, subscriptions.completions)
}
})
t.Run("between series", func(t *testing.T) {
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}, {SeriesID: "two"}}}
fetcher := &fakeSeriesFetcher{episodes: map[string][]tver.Episode{}, errors: map[string]error{}}
waiting := make(chan struct{})
never := make(chan time.Time)
scheduler, _ := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
Now: func() time.Time { return now }, After: func(time.Duration) <-chan time.Time { close(waiting); return never },
}, nil)
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { scheduler.checkDue(ctx); close(done) }()
<-waiting
cancel()
<-done
if len(fetcher.calls) != 1 || fetcher.calls[0] != "one" {
t.Fatalf("番組間の中断: %v", fetcher.calls)
}
})
}
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -eu
case "${FAKE_FFPROBE_MODE:-valid}" in
invalid|bad-container)
echo '{"format":{"format_name":"matroska"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
bad-video)
echo '{"format":{"format_name":"mp4"},"streams":[{"codec_type":"video","codec_name":"vp9"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
bad-audio)
echo '{"format":{"format_name":"mp4"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"opus"}]}' ;;
non-json)
echo 'not JSON' ;;
failure)
echo 'probe failed' >&2
exit 1 ;;
*)
echo '{"format":{"format_name":"mov,mp4,m4a,3gp,3g2,mj2"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
esac
+67
View File
@@ -0,0 +1,67 @@
#!/bin/sh
set -eu
[ -z "${FAKE_ARGS_FILE:-}" ] || printf '%s\n' "$@" > "$FAKE_ARGS_FILE"
output_template=
episode_url=
while [ "$#" -gt 0 ]; do
if [ "$1" = "-o" ]; then
output_template=$2
shift 2
continue
fi
case "$1" in
https://tver.jp/episodes/*) episode_url=$1 ;;
esac
shift
done
mode=${FAKE_YTDLP_MODE:-success}
if [ "$mode" = "mixed" ]; then
case "$episode_url" in
*/ep-fail) mode=unknown ;;
*) mode=success ;;
esac
fi
case "$mode" in
drm) echo 'ERROR: This video is DRM protected token=secret-value' >&2; exit 1 ;;
geo) echo 'ERROR: This video is not available from your location due to geo restriction' >&2; exit 1 ;;
network) echo 'ERROR: Unable to download JSON metadata: [Errno 61] Connection refused' >&2; exit 1 ;;
unavailable) echo 'ERROR: MEDIA_NOT_FOUND: episode does not exist' >&2; exit 1 ;;
extract) echo 'ERROR: Unable to extract player data' >&2; exit 1 ;;
unknown) echo 'ERROR: surprising failure' >&2; exit 1 ;;
hang|hang-drm)
[ -z "${FAKE_READY_FILE:-}" ] || : > "$FAKE_READY_FILE"
if [ "$mode" = "hang-drm" ]; then
trap 'echo "ERROR: DRM protected" >&2; [ -z "${FAKE_TERM_FILE:-}" ] || : > "$FAKE_TERM_FILE"; exit 143' TERM INT
else
trap '[ -z "${FAKE_TERM_FILE:-}" ] || : > "$FAKE_TERM_FILE"; exit 143' TERM INT
fi
while :; do sleep 1; done
;;
esac
output=$(printf '%s' "$output_template" | sed 's/%(id)s/fake/g; s/%(ext)s/mp4/g')
mkdir -p "$(dirname "$output")"
printf 'fake mp4' > "$output"
if [ "$mode" = "symlink" ]; then
rm "$output"
ln -s "$FAKE_OUTSIDE_FILE" "$output"
fi
if [ "$mode" = "burst" ]; then
i=1
while [ "$i" -le 20 ]; do
printf '{"event":"download","format_id":"hls-698-1","progress":{"status":"downloading","downloaded_bytes":%s,"total_bytes":100,"speed":10,"eta":2,"fragment_index":%s,"fragment_count":20}}\n' "$i" "$i"
i=$((i + 1))
done
sleep 0.12
else
echo '{"event":"download","format_id":"hls-698-1","progress":{"status":"downloading","downloaded_bytes":1024,"fragment_index":0,"fragment_count":50,"filename":"/private/tmp/tver-spike/epq432nee4.fhls-698-1.mp4","tmpfilename":"/private/tmp/tver-spike/epq432nee4.fhls-698-1.mp4.part","max_progress":null,"progress_idx":null,"elapsed":0,"total_bytes_estimate":29798400.0,"speed":4834.271158821972,"eta":null,"_eta_str":"Unknown","_speed_str":" 4.72KiB/s","_percent":0.003436426116838488,"_percent_str":" 0.0%","_total_bytes_str":" N/A","_total_bytes_estimate_str":" 28.42MiB","_downloaded_bytes_str":" 1.00KiB","_elapsed_str":"00:00:00","_default_template":" 0.0% of ~ 28.42MiB at 4.72KiB/s ETA Unknown (frag 0/50)"}}'
sleep 0.02
echo '{"event":"download","format_id":"hls-ts_AUDIO-0_2","progress":{"status":"finished","downloaded_bytes":50,"total_bytes":50,"speed":20,"eta":0}}'
sleep 0.02
echo '{"event":"postprocess","progress":{"status":"started","postprocessor":"Merger"}}' >&2
fi
printf '{"event":"after_move","filepath":"%s"}\n' "$output"