kazu634/tver
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: kazu634/tver:4ad0d621e1d6de4efe4094f2aa940b3bdd596c15
Branches Tags
kazu634/tver:main kazu634/tver:bulk-delete
kazu634/tver:v1.0.0
...
compare: kazu634/tver:v1.0.0
Branches Tags
kazu634/tver:main kazu634/tver:bulk-delete
kazu634/tver:v1.0.0
25 Commits
4ad0d621e1 ... v1.0.0
Author SHA1 Message Date
kazu634 d4346c06fe Merge pull request 'ファイル名の上限拡大と os.Root への置き換え（ステップ9）' (#13) from path-hardening into main 
Reviewed-on: #13
 2026-10-04 14:56:03 +09:00
kazu634 a2dd67b8b6 Merge remote-tracking branch 'origin/main' into path-hardening 
# Conflicts:
#	PROJECT.md
 2026-10-04 14:53:53 +09:00
kazu634 3fb3227aa7 Merge pull request '本番配置の結果を反映（ステップ8）' (#12) from deploy-results into main 
Reviewed-on: #12
 2026-10-04 14:51:32 +09:00
kazu634 9b856d63fa ファイル名の上限を拡大し、動画の配信・削除を os.Root に置き換え（エピソード ID は 64 バイト上限） 2026-10-04 13:43:07 +09:00
kazu634 3ddd2063b6 本番配置の結果を反映（配置用ユーザーでの clone、動作確認結果、新着 100 件の調査） 2026-10-04 13:13:02 +09:00
kazu634 f963dea2d1 Merge pull request '本番配置の準備を追加（ステップ7）' (#11) from deploy-setup into main 
Reviewed-on: #11
 2026-10-04 12:20:08 +09:00
kazu634 37d42851ff 本番配置の準備を追加（NAS の DB 定義、配置手順書、保存先の決定） 2026-10-04 12:01:19 +09:00
kazu634 80d9606750 Merge pull request '登録番組の定期確認を追加（ステップ6b）' (#10) from subscription-check into main 
Reviewed-on: #10
 2026-10-04 10:44:02 +09:00
kazu634 944086c8ae 登録番組の定期確認を追加（24 時間ごと、予告の除外、確認エラーの記録） 2026-10-03 23:42:25 +09:00
kazu634 82536c6398 Merge pull request 'worker を追加（ステップ6a）' (#9) from worker-download into main 
Reviewed-on: #9
 2026-10-03 23:27:49 +09:00
kazu634 6f150495d5 worker を追加（yt-dlp によるダウンロード、段階別進捗、ffprobe 検査、実行中の取り消し） 2026-10-03 22:56:14 +09:00
kazu634 ebf1cfcbc2 Merge pull request 'ライブラリ・再生・削除を追加（ステップ5）' (#8) from library into main 
Reviewed-on: #8
 2026-10-03 20:57:02 +09:00
kazu634 276cd7b5b8 ライブラリ・再生・削除を追加（Range 配信、パスの安全確認、共有ファイル名規則） 2026-10-03 20:54:16 +09:00
kazu634 637ee5b81a Merge pull request '番組登録を追加（ステップ4）' (#7) from subscriptions into main 
Reviewed-on: #7
 2026-10-03 19:50:49 +09:00
kazu634 787eda79ae 番組登録を追加（登録時の過去回の選択、一時停止・再開・解除、既知の回の記録） 2026-10-03 19:44:57 +09:00
kazu634 2e82a690b1 Merge pull request '待ち行列への登録と状態表示を追加（ステップ3）' (#6) from queue-ui into main 
Reviewed-on: #6
 2026-10-03 19:12:03 +09:00
kazu634 4a02526e76 待ち行列への登録と状態表示を追加（新着からの登録、/queue、DATABASE_URL） 2026-10-03 19:07:53 +09:00
kazu634 a71b915c81 Merge pull request 'DB の土台を追加（ステップ2）' (#5) from db-foundation into main 
Reviewed-on: #5
 2026-10-03 18:23:11 +09:00
kazu634 edc8149202 DB の土台を追加（マイグレーション、待ち行列・番組登録の操作、embedded-postgres によるテスト） 2026-10-03 18:07:58 +09:00
kazu634 e2d3f83db4 Merge pull request '新着一覧の画面を追加（ステップ1）' (#4) from newer-list into main 
Reviewed-on: #4
 2026-10-03 17:27:53 +09:00
kazu634 f0d73a14e1 新着一覧の画面を追加（ジャンル切替、予告の除外、10分キャッシュ） 2026-10-03 17:20:44 +09:00
kazu634 ffe7451d04 Merge pull request 'web コンテナの土台を追加（ステップ0）' (#3) from scaffold-web into main 
Reviewed-on: #3
 2026-10-03 17:04:41 +09:00
kazu634 5125614b93 web コンテナの土台を追加（HTTP サーバー、TVer API クライアント、htmx の画面枠、Dockerfile・compose.yaml） 2026-10-03 17:01:50 +09:00
kazu634 54de47fbfe Merge pull request 'AGENTS.md: チーム構成の表を整形し、Developer の役割を「テスト作成」に修正' (#2) from update-agents-md into main 
Reviewed-on: #2
 2026-10-03 16:57:11 +09:00
kazu634andClaude Opus 5.5 0d74a5d340 AGENTS.md: チーム構成の表を整形し、Developer の役割を「テスト作成」に修正 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BcuBr9DyfcqZY8xFQsY5Di
 2026-10-03 16:56:01 +09:00
63 changed files with 8274 additions and 25 deletions
Expand all files Collapse all files
.dockerignore
+6
View File
@@ -0,0 +1,6 @@
.git
.gitignore
PROJECT.md
AGENTS.md
.env
.env.*
.env.example
+4
View File
@@ -0,0 +1,4 @@
DATABASE_URL=
VIDEO_HOST_DIR=/mnt/nas-tver/tver
VIDEO_DIR=/videos
WORKER_CONCURRENCY=1
.gitignore
+1
View File
@@ -4,6 +4,7 @@
# 秘匿情報（homelab 規約: .env に分離し、コミットしない）
.env
.env.*
!.env.example
# エージェントのローカル設定（端末ごとの権限・agmsg のフック）
.claude/settings.local.json
AGENTS.md
+2 -2
View File
@@ -7,9 +7,9 @@
## チーム構成
| エージェント | 種別 | 役割 |
|---|---|---|
| --------------- | ----------- | --------------------------------------------- |
| **`Leader`** | claude-code | 人間の窓口。計画・タスク分解・依頼・検収 |
| **`Developer`** | codex | 実作業。実装・テスト実行・修正 |
| **`Developer`** | codex | 実作業。実装・テスト作成・修正 |
| **`QA`** | codex | テストの品質保証。テスト内容の検証・必要に応じたテストの修正・テスト実施・テスト結果の整理 |
参加は人間の指示を受けてから行う。名前は上の表のとおりにする（`whoami.sh` が出す候補名は他チームのものを含むので使わない）。
Dockerfile
+16
View File
@@ -0,0 +1,16 @@
FROM golang:1.27.1-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY cmd ./cmd
COPY internal ./internal
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /web ./cmd/web
FROM alpine:3.22
RUN apk add --no-cache ca-certificates tzdata
WORKDIR /app
COPY --from=build /web /app/web
USER 65532:65532
EXPOSE 8080
ENTRYPOINT ["/app/web"]
Dockerfile.worker
+21
View File
@@ -0,0 +1,21 @@
FROM golang:1.27.1-alpine AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY cmd ./cmd
COPY internal ./internal
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /worker ./cmd/worker
FROM python:3.13-slim
ARG YT_DLP_VERSION=2026.08.19
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates ffmpeg tzdata \
&& pip install --no-cache-dir "yt-dlp[default]==${YT_DLP_VERSION}" \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=build /worker /app/worker
ENV HOME=/tmp \
XDG_CACHE_HOME=/tmp/yt-dlp-cache
USER 65532:65532
ENTRYPOINT ["/app/worker"]
PROJECT.md
+84 -18
View File
@@ -3,7 +3,7 @@ title: tver の現状
tags:
- homelab
- tver
status: 要件定義中
status: 本番稼働中
---
# tver の現状
@@ -13,7 +13,8 @@ status: 要件定義中
## 今の状態
- **要件定義中。** 実装はまだ無い
- Go の HTTP サーバー、TVer API クライアント、ドラマ・バラエティの新着表示、テスト、web・worker コンテナの定義がある
- **本番稼働中。** dockageのweb・worker、NASのPostgreSQL 17、NFS動画保存を構成し、実動画の取得・再生・Range配信まで確認済み
- リポジトリ: `https://gitea.kazu634.com/kazu634/tver.git`
## 目的
@@ -26,18 +27,38 @@ TVer（`https://tver.jp/`）で公開中の動画情報を取得し、選んだ
|---|---|
| TVer 利用規約との関係 | 鑑賞を前提とするダウンロードのため、問題無しと判断（ユーザー判断） |
| 利用者・公開範囲 | 自分のみ・**LAN 内だけ**。EC2 nginx では公開しない |
| 保存先 | **NAS（ds923p）** |
| 保存先 | **NAS（ds923p）の `/volume1/video/tver`**。既存の`anime`・`drama`・`misc`・`personal`・`variety`と分け、admin相当の書き込み範囲を限定する。`/volume1/video`をdockageの`/mnt/nas-tver`へNFSマウントし、`.env`は`VIDEO_HOST_DIR=/mnt/nas-tver/tver`とする |
| 自動ダウンロード | **作る**（番組を登録し、新しい回を自動でダウンロードする） |
| 再生 | **アプリの中で再生する** |
| ダウンロード | yt-dlp を使う |
| デプロイ | Dockge で管理する複数コンテナ構成。dockage VM の `/opt/stacks/tver` |
| デプロイ | 読み取り専用のGiteaユーザー`tver-deploy`でdockage VMの`/opt/stacks/tver`へcloneし、`docker compose up -d --build`で起動する。Dockgeにも同じスタックが表示される。更新は`git pull --ff-only`後に再ビルドする |
| 配置用のGit認証 | `tver-deploy`を`tver`の読み取り共同作業者とし、repository読み取りトークンをdockageの権限`600`の専用credentialファイルに保存する。トークンの実値はGitや文書へ残さない |
| 開発言語 | **Go**（`web`・`worker` とも）。yt-dlp は外部コマンドとして呼び、進捗・結果は yt-dlp の出力から読み取る |
| DB | **NAS の共有 Postgres**。バックアップは規約どおり `pg_dump` → NAS |
| DB | **NAS上のtver専用Postgres 17コンテナ**（Giteaと同じ方式）。ユーザー・DB名は`tver`、公開ポートは`15434`。dockageからの到達を確認済み |
| DB ドライバ | `github.com/jackc/pgx/v5` の `pgxpool` |
| DB テスト | `github.com/fergusstrange/embedded-postgres` v1.34.0 の Postgres 17 を起動する。Apple Silicon では Rosetta 2 が必要 |
| 定期実行 | **`worker` 内のスケジューラ** |
| NAS のマウント | **NFS**。DSM で `192.168.10.155` のみ許可する |
| NAS のマウント | **NFS**。dockageの`/etc/fstab`へ`_netdev,nofail`付きで登録し、DSMでは接続元`192.168.10.155`だけを許可する |
| NFS 権限 | 読み書き、Squash「すべてのユーザーをadminに割り当てる」、`sys`、非同期オン。DSMのadminアカウントが無効でもUID `65532`から書き込み確認済み |
| DBの通信制限 | DSMファイアウォールでTCP `15434`を`192.168.10.155`からだけ許可する |
| DBバックアップ | 稼働後に、コンテナ内の`pg_dump`からNASへ保存する日次処理を用意する |
| 保持期間 | **削除は手動のみ**。自動削除はしない |
| 待ち行列 | **Postgres のテーブルだけで実装する**。専用のキュー基盤は持たない |
| 定期確認の間隔 | 登録番組の新しい回を **24 時間ごと**に確認する |
| 定期確認の走査 | worker のスケジューラが起動直後と1時間ごとに、最終確認から24時間以上経過した有効な番組を探す。番組は1件ずつ確認し、間に数秒空ける |
| 定期確認の予告 | **180秒以下**の回は自動登録せず、既知としてだけ記録する。新着画面の予告判定と同じしきい値を使う |
| 定期確認の失敗 | `last_checked_at` を更新せず、失敗日時と秘匿値を伏せた短いメッセージを保存する。次の1時間走査で再試行し、成功時にエラーを消す |
| 登録時の既存回 | **登録の都度**「公開中の回もダウンロードする」かを選ぶ。選ばない場合も公開中の回を既知として記録する |
| 番組の一時停止 | 登録は残し、自動ダウンロードだけ止める。再開後は停止中に公開された未確認の回も対象にする |
| 動画のマウント先 | web と worker は `VIDEO_DIR`（既定 `/videos`）に同じ NAS 共有をマウントする。`download_jobs.file_path` は `VIDEO_DIR` からの相対パスを保存する |
| ファイル名 | `<番組名>/<番組名> <回タイトル> [<エピソードID>].mp4`。禁止文字と制御文字を `_` に置換し、空の要素は `名称未設定`とする。番組名はUTF-8で90バイト、回タイトルは140バイト、サニタイズ後のエピソードIDは64バイトまでとし、ファイル名全体を255バイト以下に収める。64バイトを超えるIDはworkerで`invalid_output`にする。実際のTVer IDは約10文字で、yt-dlpのextractorも英数字として扱うため64バイトを上限とした。既存ファイルは改名せず、新しい保存から適用する |
| 動画ファイルの境界 | webは`VIDEO_DIR`を`os.OpenRoot`で開き、DBの相対`file_path`を`Root.Open`・`Root.Remove`へ渡す。`VIDEO_DIR`外へのパストラバーサルと外向きシンボリックリンクをOSのディレクトリ境界で拒否する |
| 同名ファイル | 移動先に同名ファイルがあれば上書きする。ファイル名にエピソード ID が入り `episode_id` は一意なので、同名は同じ回の前回処理の残りに限られる。再実行で同じ結果になることを優先する |
| 動画の削除 | ファイルを先に削除してからジョブ記録を削除する。ファイルが既に無い場合も記録は削除する。登録番組の `known_episodes` は残す |
| 同時ダウンロード数 | 既定は **1**。`WORKER_CONCURRENCY` で **2まで**増やせる |
| ダウンロードの再試行 | 自動では再試行しない。失敗を `failed` で残し、画面から手動で再試行する |
| 実行中の取り消し | web で `running` を `canceled` にし、worker が検知して yt-dlp のプロセスグループを停止する |
| yt-dlp の作業場所 | `VIDEO_DIR/.incoming/<job id>/` に `%(id)s.%(ext)s` で保存し、検査後に Go が `media.RelativeVideoPath` の場所へ同一ファイルシステム内で移動する。終了状態にかかわらず作業場所を片付ける |
## 要件
@@ -66,35 +87,80 @@ TVer（`https://tver.jp/`）で公開中の動画情報を取得し、選んだ
## 構成（案）
| コンテナ | 役割 |
|---|---|
| -------- | ----------------------------------------------------- |
| `web` | UI・API。新着一覧の取得、待ち行列への登録、番組登録、動画の配信（再生） |
| `worker` | Go + yt-dlp + ffmpeg。待ち行列からダウンロードし、登録番組の新しい回を定期的に確認する |
- DB はコンテナを持たず、NAS の共有 Postgres に接続する（番組登録・ダウンロード履歴・待ち行列）
- 保存先の NAS 共有フォルダは dockage VM に NFS でマウントし、`worker`（書き込み）と `web`（読み取り・削除）にバインドマウントする
- 現在の `web` は `/healthz`、新着（`/`）、待ち行列（`/queue`）、登録番組（`/subscriptions`）、完了済みライブラリ（`/library`）、再生ページ（`/library/{id}`）、Range 対応の MP4 配信（`/videos/{id}`）を提供する
- 動画配信と削除はクライアントからジョブ ID だけを受け取り、DB の相対 `file_path` を `os.Root` の `Open`・`Remove` で扱う。絶対パス・`..`・`VIDEO_DIR` 外を指すシンボリックリンクをディレクトリ境界で拒否する。削除後は新着から手動で再登録できるが、登録番組の既知記録は残るため自動再登録しない
- 新着の「番組を登録」から、公開中の回をダウンロードするか選ぶ。web はサーバー側の新着キャッシュから番組 ID と名前を引き、TVer の season ごとの公開中エピソードを取得する。取得失敗時は登録しない
- 登録と公開中の回の既知記録・任意の待ち行列投入は1トランザクションで行い、登録時刻を最終確認日時に設定する。`known_episodes` は番組解除時に連動して削除する。一時停止中は投入を拒否する
- worker のスケジューラは起動直後と1時間ごとに、`enabled` かつ最終確認から24時間以上の番組を抽出する。TVer から公開中の回を順次取得し、未確認の180秒超の回だけを `source=subscription` で投入する。180秒以下も既知にし、成功時の最終確認更新・エラー消去まで1トランザクションで行う
- TVer の番組取得に失敗した場合は最終確認日時を変えず、失敗日時と伏字済みの短いメッセージを `subscriptions` に記録して画面に表示する。番組間には3秒空け、SIGTERM では取得中または待機中の確認を中断する
- 新着の登録ボタンはエピソード ID とジャンルだけを送る。web が有効な新着キャッシュから番組情報を引き、`episode_id` の一意制約で重複登録を防ぐ。新着一覧はジョブ状態を一括取得し、失敗・取り消しには再試行を表示する
- 待ち行列は実行中・待機中・失敗・取り消し・完了を状態別に直近50件ずつ表示する。段階別の進捗と失敗理由を表示し、待機中の取り消しと失敗・取り消しの再試行ができる。実行中または待機中のジョブがある間だけ5秒ごとに一覧を更新する
- 実行中のジョブにも取り消しボタンを表示する。worker は DB の `canceled` を定期確認し、yt-dlp をプロセスグループごと停止する
- 新着はジャンルごとにメモリへ 10 分間キャッシュする。キャッシュ中は TVer セッションの再発行と一覧の再取得を行わない。TVer の取得に失敗した場合、一覧の場所にエラーを表示して詳細をログへ記録する
- 「予告を隠す」は既定でオンとし、180 秒以下の件を非表示にする。長さは秒から分へ切り上げて表示し、配信終了日時は Asia/Tokyo で表示する。ジャンルと予告表示の状態は URL に残す
- 画面は `html/template` と htmx 2.0.11 を使い、htmx は Go の `embed` から `/static/` でローカル配信する
- `compose.yaml` は web と worker を定義する。web は dockage VM の LAN アドレス `192.168.10.155:8080` で公開し、待ち受けは `WEB_ADDR` で変更できる。両方へ `DATABASE_URL` と同じ動画ボリュームを渡し、worker の並行数は `WORKER_CONCURRENCY`（既定1、最大2）で指定する
- NAS用の`deploy/nas/tver-db/compose.yaml`は`postgres:17`を`15434:5432`で公開し、`/volume1/docker/tver-db/pgdata`へ保存する。パスワードは同ディレクトリのGit管理外`.env`から注入する
- 本番配置と更新の手順は[[docs/deploy|tver 本番配置手順]]にまとめる。dockageでは`/volume1/video`を`/mnt/nas-tver`へNFSマウントし、その配下の`tver`だけをコンテナ内の`/videos`へ渡す
- Giteaは匿名閲覧とSSHを無効にしているため、dockageのclone・pullは読み取り専用`tver-deploy`のrepository読み取りトークンを使う。Gitのcredential helperはGiteaのHTTPS URLに限定する
- DB は NAS 上に tver 専用 Postgres 17 コンテナを置く。`internal/store` の埋め込み SQL を `schema_migrations` で管理し、`pg_advisory_lock` で複数プロセスからの同時適用を直列化する。web は起動時に `pgxpool` で接続してマイグレーションを適用する
- `download_jobs` は `episode_id` を一意にして二重登録を防ぎ、ジョブの状態・段階別進捗・結果を持つ。`subscriptions` はシリーズ単位の登録と最終確認日時を持つ。worker からの操作と画面用の一覧取得を `internal/store` にまとめる
- worker は `yt-dlp[default]` と ffmpeg を外部コマンドとして使い、映像・音声・結合・移動の段階別進捗を1秒間隔に間引いて保存する。独立した heartbeat を5秒ごとに更新し、正確な全体パーセントは保存しない。`after_move:filepath` で実パスを取り、ffprobe で H.264 と AAC の MP4 を確認して成功とする。失敗は取り消し、DRM、地域制限、ネットワーク、配信不可、抽出失敗、出力不正、その他の順に扱い、自動再試行しない
- worker は起動時にマイグレーションと停止中ジョブの `RequeueStale` を行い、登録番組の確認も直ちに1回行う。SIGTERM では yt-dlp と確認処理を止め、担当中ジョブを `queued` に戻してから終了する。`Dockerfile.worker` は yt-dlp 2026.08.19 を ARG で固定し、ffmpeg・ffprobe とともに非 root で実行する
- 保存先の NAS 共有フォルダは dockage VM に NFS でマウントし、`worker`（書き込み）と `web`（読み取り・削除）に同じ `VIDEO_DIR` でバインドマウントする
## 外部とのつながり
| 相手 | 用途 | 備考 |
|---|---|---|
| TVer `platform-api.tver.jp` | 新着一覧（`/service/api/v1/callNewerDetail/{drama\|variety}`）・番組のエピソード一覧 | **非公開 API**。セッションは `/v2/api/platform_users/browser/create` で発行する（yt-dlp と同じ方式） |
| TVer `service-api.tver.jp` | 番組の season 一覧（`/api/v1/callSeriesSeasons/{series_id}`） | **非公開 API**。[yt-dlp の TVer extractor](https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/tver.py) と同じ取得方式 |
| yt-dlp | ダウンロード | TVer extractor を使う |
| NAS ds923p（NFS） | 動画の保存先 | DSM で NFS を有効化し、共有フォルダを `192.168.10.155` のみに許可する |
| NAS ds923p（Postgres） | DB | DSM 内蔵の Postgres が `5432` を使うため、共有 Postgres コンテナの公開ポートを使う。DSM ファイアウォールで `192.168.10.155` のみ許可する |
| Gitea `gitea.kazu634.com` | 配置元 | 匿名cloneとSSHは使えない。`tver-deploy`のHTTPSトークンでclone・pullする |
| NAS ds923p（NFS） | 動画の保存先 | `/volume1/video`をdockageの`/mnt/nas-tver`へNFSv3でマウントし、アプリは`tver`サブフォルダを使う。DSMのNFS権限は`192.168.10.155`のみ |
| NAS ds923p（Postgres 17） | DB | tver専用コンテナを`15434:5432`で公開し、DSMファイアウォールで`192.168.10.155`のみ許可する。dockageから接続確認済み |
## 確認済みの事実
- 新着 API は `drama` / `variety` それぞれ 100 件を返す。予告動画も含まれる。各件に `id`・`seriesID`・`seriesTitle`・`title`・`broadcasterName`・`duration`・`endAt`・`thumbnailPath` がある [実機] 2026-10-03 確認
- 番組 `sr0lhdrtdb` の `callSeriesSeasons` は `result.contents` に `type=season` の1件（`ss3i6njoq3`）を返し、`callSeasonEpisodes` は `type=episode` の5件を返した。エピソードには新着と同じ `id`・`seriesID`・`seriesTitle`・`title`・`broadcasterName`・`duration`・`endAt`・`thumbnailPath` が含まれる。他の番組での season 数や一覧件数は未確認 [実機] 2026-10-03 確認
- dockage VM から TVer API に到達でき（HTTP 200）、送信元は日本と判定される [実機] 2026-10-03 確認
- yt-dlp 2026.08.19 で TVer のエピソードを抽出できる（`--simulate`）。映像（H.264 / `avc1`）と音声が別ストリームで、結合に ffmpeg が要る。1080p・約 50 分で約 1.2GiB [実機] 2026-10-03 確認
- 5 分の `epq432nee4` を yt-dlp 2026.08.19 で取得した。`--newline` と `--progress-template` の 1 行 1 JSON は Go の外部プロセスから逐次読める。ただし `uvx yt-dlp` のみで `pycryptodomex` が無いと AES-128 HLS を ffmpeg に委譲し、途中のダウンロード進捗が出なかった。`yt-dlp[default]` に含まれる `pycryptodomex` を加えた試行では途中の進捗が出た [実機] 2026-10-03 確認
- 別ストリームの映像と音声は進捗が個別に始まり、`format_id` で区別できる。結合処理は `postprocess:` の出力で検知でき、`--print after_move:filepath` で実ファイルのパスを取得した。完成した MP4 は 24,613,787 バイトで、ffprobe では H.264 640×360 と AAC LC だった。初回取得は約 10 秒だった [実機] 2026-10-03 確認
- SIGTERM で止めると `.part` と `.ytdl` が残り、再実行で 437,066 バイトから 438,090 バイトへ続きから取得できた [実機] 2026-10-03 確認
- 到達不能のプロキシ経由の通信失敗と存在しないエピソードは、ともに yt-dlp の終了コード 1 だった。stderr の文言で分類が必要。DRM・地域制限の失敗文言と終了コードは実測していない（yt-dlp の extractor ソースのみ確認） [実機] 2026-10-03 確認
- yt-dlp の TVer extractor は日本のみ（`_GEO_COUNTRIES = ['JP']`）。DRM 付きの作品は取得できない [公式] https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/tver.py (取得日: 2026-10-03)
- dockage VM に NAS のマウントは無い。ルートの空きは約 20GB [実機] 2026-10-03 確認
- DSMのNFS権限を接続元`192.168.10.155`、読み書き、全ユーザーをadminへ割り当て、`sys`、非同期オンにすると、dockageからNFSv3で`/volume1/video`をマウントできた。設定直後の1回目は`access denied`、直後の2回目は成功したため、権限の反映待ちと考えられる [実機] 2026-10-04 確認
- DSMのadminアカウントが無効でも、UID `65532`のコンテナから`tver`サブフォルダの作成・書き込み・読み込み・rename・削除に成功した。NAS上の所有者はUID `1024`・GID `100`。`/mnt/nas-tver/tver`は作成済み [実機] 2026-10-04 確認
- NFSマウントのファイル名上限は`namlen=255` [実機] 2026-10-04 確認
- Giteaは`REQUIRE_SIGNIN_VIEW = true`・`DISABLE_SSH = true`で、匿名HTTPSの`info/refs`はHTTP `401`だった。読み取り専用ユーザー`tver-deploy`とrepository読み取りトークンで`/opt/stacks/tver`へのcloneに成功した [実機] 2026-10-04 確認
- dockageで`docker compose up -d --build`を実行してweb・workerを起動でき、Dockgeにも`tver`スタックとして表示された。DSMファイアウォール設定後、dockageからNASのTCP `15434`へ到達できた [実機] 2026-10-04 確認
- 本番の`/healthz`はHTTP `200`、新着バラエティは98件だった。4分の本編`ep5hcgopp2`は登録から20秒以内に完了し、92,340,468バイトの1080p H.264/AAC MP4をNASへ保存した。所有者はUID `1024`・GID `100`、`.incoming`は空で、`/library`とHTTP Range `206`も確認した [実機] 2026-10-04 確認
- NASは番組フォルダ内に所有者UID `0`の`@eaDir`を作る。ライブラリ一覧はDBから作るため、アプリへの影響はない [実機] 2026-10-04 確認
- `callNewerDetail/drama`は毎回100件を返し、応答に続きや総件数を示す項目はなかった。`result.contents`は`type=newer`で、`content`は`id`・`version`・`title`のみ。`page=2`・`offset=100`・`limit=200`を付けても先頭・末尾IDが同じ100件となり、パラメータは無視された。100件を超えて取得する手段は見つからなかったが、非公開APIのため不可能とは断定しない。新着一覧は直近100件までとなる一方、登録番組の自動確認は番組ごとの`callSeriesSeasons`→`callSeasonEpisodes`を使うため、新着から外れた回も取りこぼさない [実機] 2026-10-04 確認
## 課題
- 新着 API が 100 件で打ち切りか、続きを取得できるか [要確認]
- 新着のうち DRM 付きの作品の割合 [要確認]
- 実ダウンロードの所要時間とファイルサイズ（`--simulate` の推定値のみ） [要確認]
- 非公開 API と yt-dlp の TVer 対応は、TVer 側の変更で予告なく壊れうる。壊れたことを検知する手段が要る
- NAS の準備（NFS 共有フォルダ、共有 Postgres へのデータベース・ユーザー作成、DSM ファイアウォール）は DSM での手作業になる。共有 Postgres の現行の公開ポートは [[homelab/docs/inventory|インフラ詳細]] に記載が無い [要確認]
- Go から yt-dlp の進捗を読み取る方式（`--progress-template` と `--print` で 1 行 1 JSON を出す等）を実機で確かめる [要確認]
- **DBバックアップは未着手。** コンテナ内の`pg_dump`→gzip→NAS `/volume1/backups/tver/`、既定14日保持の日次処理を用意する
- DRM・地域制限の実例探しはTVerへの大量アクセスになるため行わず、運用中に観測する。workerは失敗時の`error_message`をDBへ保存し、既知の文言に一致しない失敗は`unknown`にする。実例が出た時点で分類を見直す
## 確かめ方
- Codex では `go test -count=1 ./...` を環境変数の前置なしで実行する。`go test` を許可する rules に一致させ、Postgres の共有メモリ作成に必要な権限で実行するため。初回のモジュール取得にはネットワーク接続が必要
- web を起動するには `.env.example` を参考に、Git 管理外の `.env` に実際の `DATABASE_URL` と `VIDEO_HOST_DIR` を設定する。`VIDEO_DIR` の既定は `/videos`。DB 接続がある環境では起動時にマイグレーションが適用される
- DB テストは `embedded-postgres` v1.34.0 が Postgres 17 を一度起動し、テストごとに独立した DB を作る。バイナリキャッシュ、実行ファイル、データは `/private/tmp` 配下に置く。Apple Silicon での実行には Rosetta 2 の導入が必要
- TVer 応答のテストは `internal/tver/testdata/newer_drama.json`（2026-10-03 の実応答を 3 件に縮小）と `httptest` を使い、テストから本物の TVer にはアクセスしない
- 番組一覧の応答は `internal/tver/testdata/series_seasons.json` と `season_episodes.json`（2026-10-03 の実応答を各1件に縮小）に記録した。テストは `httptest` を使い、実 API にアクセスしない
- 新着画面のテストは偽物の TVer クライアント・store と差し替え可能な時計を使い、表示項目、絞り込みの境界、10 分キャッシュ、取得失敗、JST 表示、登録・状態表示・待ち行列の操作を確認する
- ライブラリのテストは一時ディレクトリの小さな動画を使い、Range の 206・範囲外の 416、`os.Root`による相対パスとシンボリックリンクの拒否、404、ファイル削除後の記録削除を確認する。共有のファイル名規則は`internal/media`で90・140・255バイトの境界とUTF-8の完全性をテストする
- worker のテストは `internal/worker/testdata` の偽 yt-dlp・ffprobe を使い、本物の TVer へ接続せずに正常完了、失敗分類、取消、終了時の待機復帰、進捗の間引き、heartbeat、出力検査、移動先、作業場所の片付けを確認する
- 登録番組の定期確認は偽の TVer クライアントと差し替え可能な時計・タイマーを使い、24時間の境界、順次確認、1時間走査、失敗記録、予告の既知化、終了時の中断を確認する。store の抽出・既知化・投入・日時更新・エラー消去は embedded Postgres 17 で確認する
- NAS用Composeは`deploy/nas/tver-db/compose.yaml`をYAMLパーサーで読み、`services.postgres`、環境変数、ポート、保存先、healthcheckを確認する。Dockerが使える環境では`docker compose --env-file deploy/nas/tver-db/.env.example -f deploy/nas/tver-db/compose.yaml config`でも展開結果を確認する
- 本番の確認手順は[[docs/deploy|tver 本番配置手順]]に従い、NFS上の`/mnt/nas-tver/tver`をUID `65532`で読み書き・rename・削除できること、DBがhealthyであること、webの`/healthz`、実動画の保存・再生・シークを確認する
- `env GOPATH=/private/tmp/tver-gopath GOCACHE=/private/tmp/tver-go-cache GOMODCACHE=/private/tmp/tver-go-modcache GOPROXY=off go vet ./...` で静的解析する
- `DATABASE_URL`、`VIDEO_DIR`、`WEB_ADDR=127.0.0.1:18080` を設定して `go run ./cmd/web` で web を起動できる。worker は同じ `DATABASE_URL` と `VIDEO_DIR` を設定して `go run ./cmd/worker` で起動する。worker の実行は TVer への実アクセスを伴うため手動確認は少数回に留める
cmd/web/main.go
+62
View File
@@ -0,0 +1,62 @@
package main
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"os"
"strings"
"time"
_ "time/tzdata"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
"gitea.kazu634.com/kazu634/tver/internal/web"
"github.com/jackc/pgx/v5/pgxpool"
)
func main() {
pool, err := openDatabase(context.Background(), os.Getenv("DATABASE_URL"), pgxpool.New, store.Migrate)
if err != nil {
log.Fatal(err)
}
defer pool.Close()
address := os.Getenv("WEB_ADDR")
if address == "" {
address = ":8080"
}
client, err := tver.NewClient("", nil)
if err != nil {
log.Fatal(err)
}
videoDir := os.Getenv("VIDEO_DIR")
if videoDir == "" {
videoDir = "/videos"
}
server := &http.Server{
Addr: address,
Handler: web.NewHandler(client, store.New(pool), videoDir, time.Now, log.Default()),
ReadHeaderTimeout: 5 * time.Second,
}
log.Printf("web listening on %s", address)
log.Fatal(server.ListenAndServe())
}
func openDatabase(ctx context.Context, databaseURL string,
connect func(context.Context, string) (*pgxpool.Pool, error),
migrate func(context.Context, *pgxpool.Pool) error) (*pgxpool.Pool, error) {
if strings.TrimSpace(databaseURL) == "" {
return nil, errors.New("DATABASE_URL が設定されていません")
}
pool, err := connect(ctx, databaseURL)
if err != nil {
return nil, errors.New("DATABASE_URL から接続プールを作成できません")
}
if err := migrate(ctx, pool); err != nil {
pool.Close()
return nil, fmt.Errorf("DB マイグレーションに失敗: %w", err)
}
return pool, nil
}
cmd/web/main_test.go
+72
View File
@@ -0,0 +1,72 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestOpenDatabaseRequiresURLAndRunsMigration(t *testing.T) {
ctx := context.Background()
connectCalls, migrateCalls := 0, 0
connect := func(_ context.Context, url string) (*pgxpool.Pool, error) {
connectCalls++
if url != "postgres://test:secret@localhost/test" {
t.Errorf("URL = %q", url)
}
return pgxpool.New(ctx, url)
}
migrate := func(_ context.Context, _ *pgxpool.Pool) error {
migrateCalls++
return nil
}
if _, err := openDatabase(ctx, " ", connect, migrate); err == nil || !strings.Contains(err.Error(), "DATABASE_URL") {
t.Fatalf("未設定 URL のエラー: %v", err)
}
if connectCalls != 0 || migrateCalls != 0 {
t.Fatalf("未設定でも接続・移行された: %d %d", connectCalls, migrateCalls)
}
pool, err := openDatabase(ctx, "postgres://test:secret@localhost/test", connect, migrate)
if err != nil || pool == nil {
t.Fatalf("DB 初期化: %v", err)
}
pool.Close()
if connectCalls != 1 || migrateCalls != 1 {
t.Fatalf("接続・移行回数: %d %d", connectCalls, migrateCalls)
}
}
func TestOpenDatabaseReportsMigrationFailure(t *testing.T) {
ctx := context.Background()
pool, err := openDatabase(ctx, "postgres://test:secret@localhost/test", pgxpool.New,
func(context.Context, *pgxpool.Pool) error { return errors.New("migration failed") })
if pool != nil || err == nil || !strings.Contains(err.Error(), "migration failed") {
t.Fatalf("移行失敗: pool=%v err=%v", pool, err)
}
}
func TestOpenDatabaseReportsConnectionFailureWithoutLeakingURL(t *testing.T) {
migrateCalled := false
pool, err := openDatabase(context.Background(), "postgres://user:secret@localhost/test",
func(context.Context, string) (*pgxpool.Pool, error) {
return nil, errors.New("postgres://user:secret@localhost/test: connection refused")
},
func(context.Context, *pgxpool.Pool) error {
migrateCalled = true
return nil
})
if pool != nil || err == nil || strings.Contains(err.Error(), "secret") || migrateCalled {
t.Fatalf("接続失敗: pool=%v err=%v migrateCalled=%v", pool, err, migrateCalled)
}
}
func TestOpenDatabaseDoesNotExposeInvalidURL(t *testing.T) {
_, err := openDatabase(context.Background(), "postgres://user:secret@%", pgxpool.New,
func(context.Context, *pgxpool.Pool) error { t.Fatal("移行してはいけない"); return nil })
if err == nil || strings.Contains(err.Error(), "secret") {
t.Fatalf("接続 URL の秘匿: %v", err)
}
}
cmd/worker/main.go
+99
View File
@@ -0,0 +1,99 @@
package main
import (
"context"
"errors"
"fmt"
"log"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
workerpkg "gitea.kazu634.com/kazu634/tver/internal/worker"
"github.com/jackc/pgx/v5/pgxpool"
)
const staleAfter = 30 * time.Second
func main() {
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
config, err := loadConfig(os.Getenv)
if err != nil {
log.Fatal(err)
}
pool, err := initializeDatabase(ctx, os.Getenv("DATABASE_URL"), time.Now(), pgxpool.New,
store.Migrate, func(ctx context.Context, pool *pgxpool.Pool, cutoff time.Time) (int64, error) {
return store.New(pool).RequeueStale(ctx, cutoff)
})
if err != nil {
log.Fatal(err)
}
defer pool.Close()
jobStore := store.New(pool)
runner, err := workerpkg.New(jobStore, config, log.Default())
if err != nil {
log.Fatal(err)
}
tverClient, err := tver.NewClient("", nil)
if err != nil {
log.Fatal(err)
}
scheduler, err := workerpkg.NewSubscriptionScheduler(jobStore, tverClient,
workerpkg.SubscriptionSchedulerConfig{PreviewMaxSeconds: tver.PreviewMaxSeconds}, log.Default())
if err != nil {
log.Fatal(err)
}
schedulerDone := make(chan struct{})
go func() {
defer close(schedulerDone)
scheduler.Run(ctx)
}()
log.Printf("worker started (concurrency=%d)", config.Concurrency)
if err := runner.Run(ctx); err != nil {
log.Fatal(err)
}
<-schedulerDone
log.Print("worker stopped")
}
func loadConfig(getenv func(string) string) (workerpkg.Config, error) {
concurrency := 1
if value := strings.TrimSpace(getenv("WORKER_CONCURRENCY")); value != "" {
parsed, err := strconv.Atoi(value)
if err != nil || parsed < 1 || parsed > 2 {
return workerpkg.Config{}, errors.New("WORKER_CONCURRENCY は 1 または 2 を指定してください")
}
concurrency = parsed
}
return workerpkg.Config{VideoDir: getenv("VIDEO_DIR"), YTDLPPath: getenv("YT_DLP_PATH"),
FFprobePath: getenv("FFPROBE_PATH"), Concurrency: concurrency}, nil
}
func initializeDatabase(ctx context.Context, databaseURL string, now time.Time,
connect func(context.Context, string) (*pgxpool.Pool, error),
migrate func(context.Context, *pgxpool.Pool) error,
requeue func(context.Context, *pgxpool.Pool, time.Time) (int64, error),
) (*pgxpool.Pool, error) {
if strings.TrimSpace(databaseURL) == "" {
return nil, errors.New("DATABASE_URL が設定されていません")
}
pool, err := connect(ctx, databaseURL)
if err != nil {
return nil, errors.New("DATABASE_URL から接続プールを作成できません")
}
if err := migrate(ctx, pool); err != nil {
pool.Close()
return nil, fmt.Errorf("DB マイグレーションに失敗: %w", err)
}
if _, err := requeue(ctx, pool, now.Add(-staleAfter)); err != nil {
pool.Close()
return nil, fmt.Errorf("停止中ジョブの復帰に失敗: %w", err)
}
return pool, nil
}
cmd/worker/main_test.go
+97
View File
@@ -0,0 +1,97 @@
package main
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
func TestLoadConfigConcurrency(t *testing.T) {
values := map[string]string{"VIDEO_DIR": "/media", "YT_DLP_PATH": "/bin/downloader", "FFPROBE_PATH": "/bin/probe"}
getenv := func(key string) string { return values[key] }
config, err := loadConfig(getenv)
if err != nil || config.Concurrency != 1 || config.VideoDir != "/media" || config.YTDLPPath != "/bin/downloader" || config.FFprobePath != "/bin/probe" {
t.Fatalf("既定設定: %+v %v", config, err)
}
values["WORKER_CONCURRENCY"] = "2"
config, err = loadConfig(getenv)
if err != nil || config.Concurrency != 2 {
t.Fatalf("並行数2: %+v %v", config, err)
}
for _, invalid := range []string{"0", "3", "bad"} {
values["WORKER_CONCURRENCY"] = invalid
if _, err := loadConfig(getenv); err == nil {
t.Errorf("不正な並行数 %q を許した", invalid)
}
}
}
func TestInitializeDatabaseRequiresURLMigratesAndRequeues(t *testing.T) {
ctx := context.Background()
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
connectCalls, migrateCalls, requeueCalls := 0, 0, 0
connect := func(_ context.Context, url string) (*pgxpool.Pool, error) {
connectCalls++
if url != "postgres://test:secret@localhost/test" {
t.Errorf("URL=%q", url)
}
return pgxpool.New(ctx, url)
}
migrate := func(context.Context, *pgxpool.Pool) error { migrateCalls++; return nil }
requeue := func(_ context.Context, _ *pgxpool.Pool, cutoff time.Time) (int64, error) {
requeueCalls++
if want := now.Add(-staleAfter); !cutoff.Equal(want) {
t.Errorf("cutoff=%v want=%v", cutoff, want)
}
return 2, nil
}
if _, err := initializeDatabase(ctx, " ", now, connect, migrate, requeue); err == nil || !strings.Contains(err.Error(), "DATABASE_URL") {
t.Fatalf("未設定 URL: %v", err)
}
if connectCalls != 0 || migrateCalls != 0 || requeueCalls != 0 {
t.Fatalf("未設定時の呼び出し: %d %d %d", connectCalls, migrateCalls, requeueCalls)
}
pool, err := initializeDatabase(ctx, "postgres://test:secret@localhost/test", now, connect, migrate, requeue)
if err != nil || pool == nil {
t.Fatalf("初期化: %v", err)
}
pool.Close()
if connectCalls != 1 || migrateCalls != 1 || requeueCalls != 1 {
t.Fatalf("呼び出し回数: %d %d %d", connectCalls, migrateCalls, requeueCalls)
}
}
func TestInitializeDatabaseStopsOnMigrationOrRequeueFailure(t *testing.T) {
ctx := context.Background()
connect := func(context.Context, string) (*pgxpool.Pool, error) {
return pgxpool.New(ctx, "postgres://localhost/test")
}
requeueCalled := false
pool, err := initializeDatabase(ctx, "postgres://localhost/test", time.Now(), connect,
func(context.Context, *pgxpool.Pool) error { return errors.New("migration failed") },
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { requeueCalled = true; return 0, nil })
if pool != nil || err == nil || !strings.Contains(err.Error(), "migration failed") || requeueCalled {
t.Fatalf("migration failure: pool=%v err=%v requeue=%v", pool, err, requeueCalled)
}
pool, err = initializeDatabase(ctx, "postgres://localhost/test", time.Now(), connect,
func(context.Context, *pgxpool.Pool) error { return nil },
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { return 0, errors.New("requeue failed") })
if pool != nil || err == nil || !strings.Contains(err.Error(), "requeue failed") {
t.Fatalf("requeue failure: pool=%v err=%v", pool, err)
}
}
func TestInitializeDatabaseDoesNotLeakURL(t *testing.T) {
_, err := initializeDatabase(context.Background(), "postgres://user:secret@localhost/test", time.Now(),
func(context.Context, string) (*pgxpool.Pool, error) {
return nil, errors.New("postgres://user:secret@localhost/test refused")
}, func(context.Context, *pgxpool.Pool) error { return nil },
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { return 0, nil })
if err == nil || strings.Contains(err.Error(), "secret") {
t.Fatalf("URL 漏えい: %v", err)
}
}
compose.yaml
+25
View File
@@ -0,0 +1,25 @@
services:
web:
build: .
env_file: .env
environment:
TZ: Asia/Tokyo
WEB_ADDR: ":8080"
VIDEO_DIR: "${VIDEO_DIR:-/videos}"
volumes:
- "${VIDEO_HOST_DIR:?VIDEO_HOST_DIR must be set}:${VIDEO_DIR:-/videos}"
ports:
- "192.168.10.155:8080:8080"
restart: unless-stopped
worker:
build:
context: .
dockerfile: Dockerfile.worker
env_file: .env
environment:
TZ: Asia/Tokyo
VIDEO_DIR: "${VIDEO_DIR:-/videos}"
WORKER_CONCURRENCY: "${WORKER_CONCURRENCY:-1}"
volumes:
- "${VIDEO_HOST_DIR:?VIDEO_HOST_DIR must be set}:${VIDEO_DIR:-/videos}"
restart: unless-stopped
deploy/nas/tver-db/.env.example
+1
View File
@@ -0,0 +1 @@
POSTGRES_PASSWORD=
deploy/nas/tver-db/compose.yaml
+36
View File
@@ -0,0 +1,36 @@
# tver 用 PostgreSQL スタック（Synology ds923p / 192.168.10.201）
#
# 配置先: ds923p の Container Manager「プロジェクト」
# /volume1/docker/tver-db/ に本ファイルを compose.yaml として配置する。
# パスワードは同じディレクトリの .env（POSTGRES_PASSWORD）から注入し、
# このファイルには実値を書かない。
#
# 注意（初回起動チェックポイント）:
# 起動前に /volume1/docker/tver-db/pgdata を作成する。
# 初回起動後は Container Manager のヘルス状態とログを確認し、
# "database system is ready to accept connections" が出てからアプリを起動する。
# bind mount の所有権・権限で初期化に失敗した場合は、原因を確認してから
# 名前付きボリュームへの切り替えを検討する。
services:
postgres:
# Debian ベースの PostgreSQL 17。initdb の UTF-8 ロケールを利用する。
image: postgres:17
container_name: tver-postgres
restart: always
environment:
POSTGRES_USER: tver
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: tver
# 初回の initdb でのみ有効。既存の pgdata には後から反映されない。
POSTGRES_INITDB_ARGS: "--encoding=UTF8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8"
ports:
# [要確認] Container Manager で 15434 が未使用であることを起動前に確認する。
# DSM ファイアウォールでは dockage（192.168.10.155）だけを許可する。
- "15434:5432"
volumes:
- /volume1/docker/tver-db/pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U tver -d tver"]
interval: 10s
retries: 5
docs/deploy.md
+238
View File
@@ -0,0 +1,238 @@
---
title: tver 本番配置手順
tags:
- homelab
- tver
- deployment
---
# tver 本番配置手順
> [!info] 対象
> Synology NAS `ds923p`（`192.168.10.201`）に PostgreSQL 17 と動画共有を用意し、dockage（`192.168.10.155`）の `/opt/stacks/tver` で web・worker を起動する。
> [!danger] 秘匿値を記録しない
> PostgreSQL のパスワードや完成した `DATABASE_URL` は、この文書や Git 管理下のファイルに書かない。NAS と dockage の `.env` にだけ保存し、権限を `600` にする。
## 事前に確認する値
| 項目 | 値 |
|---|---|
| NAS | `ds923p` / `192.168.10.201` |
| dockage | `192.168.10.155` |
| NAS のNFS共有 | `/volume1/video` |
| dockage のNFSマウント | `/mnt/nas-tver` |
| tverの動画保存先 | NAS: `/volume1/video/tver`、dockage: `/mnt/nas-tver/tver` |
| PostgreSQL 公開ポート | `15434` |
| PostgreSQL ユーザー / DB | `tver` / `tver` |
| アプリ配置先 | `/opt/stacks/tver` |
| Gitea配置用ユーザー | `tver-deploy`（リポジトリ読み取りのみ） |
## 1. NASを準備する
### 1.1 NFSサービスを有効にする
1. DSMの「コントロール パネル」→「ファイル サービス」→「NFS」でNFSサービスを有効にする。
2. 共有フォルダ `video` が `/volume1/video` にあることを確認する。
3. `video` の「編集」→「NFS 権限」で次を追加する。
| 設定 | 値 |
|---|---|
| ホスト名またはIP | `192.168.10.155` |
| 権限 | 読み書き |
| Squash | すべてのユーザーをadminに割り当てる |
| セキュリティ | `sys` |
| 非同期を有効にする | オン |
> [!success] adminへの割り当てを確認済み
> DSMの`admin`アカウントが無効な状態でも、上記設定でUID `65532`からフォルダ作成・書き込み・読み込み・rename・削除に成功した。NAS上の所有者はUID `1024`・GID `100`だった。[実機] 2026-10-04 確認
### 1.2 PostgreSQLのファイルを用意する
1. NAS上に次のディレクトリを事前作成する。
```text
/volume1/docker/tver-db/
/volume1/docker/tver-db/pgdata/
```
2. [[../deploy/nas/tver-db/compose.yaml|NAS用compose.yaml]] と `.env.example` を `/volume1/docker/tver-db/` に配置する。
3. `.env.example`を`.env`へコピーし、`POSTGRES_PASSWORD`に十分長いランダムな値を設定する。値には、URLへの埋め込みで追加のエスケープが不要な16進文字列を使える。
```bash
cd /volume1/docker/tver-db
umask 077
cp .env.example .env
# openssl rand -hex 64 で値を生成し、.env の POSTGRES_PASSWORD に設定する。
chmod 600 .env
```
> [!important] DBとアプリで同じパスワードを使う
> ここで設定した値を、後述するdockage側の`DATABASE_URL`にも使う。値そのものは文書やGitへ残さない。
### 1.3 Container ManagerでPostgreSQLを起動する
1. Container Managerの「プロジェクト」で、`/volume1/docker/tver-db/compose.yaml`を使う`tver-db`プロジェクトを作成する。
2. 起動前に、他のプロジェクトとポート一覧を確認し、`15434`が空いていることを確認する。
3. プロジェクトをビルド・起動する。
4. `tver-postgres`が`healthy`になることを確認する。
5. コンテナログに`database system is ready to accept connections`があることを確認する。
初期化に失敗した場合は、先に`pgdata`の所有権・権限とログを確認する。空のDBを作り直す場合を除き、既存の`pgdata`を削除しない。
### 1.4 DSMファイアウォールを設定する
TCP `15434`について、送信元`192.168.10.155`だけを許可するルールを追加する。他の送信元を許可する広いルールより前に配置し、LAN内の別ホストからDBへ接続できない構成にする。
DSMファイアウォールを設定した状態で、dockageからNASのTCP `15434`へ到達できることを確認済みである。[実機] 2026-10-04 確認
## 2. dockageを準備する
### 2.1 NFSクライアントとマウント先を用意する
```bash
sudo apt update
sudo apt install -y nfs-common
sudo mkdir -p /mnt/nas-tver
```
`/etc/fstab`へ次の1行を追加する。`_netdev`でネットワーク後に扱い、`nofail`でNAS停止時もdockageの起動を妨げない。
```fstab
192.168.10.201:/volume1/video /mnt/nas-tver nfs defaults,_netdev,nofail 0 0
```
設定を反映し、想定したNFS共有がマウントされたことを確認する。
```bash
sudo mount -a
findmnt --target /mnt/nas-tver
```
実機ではNFSv3でマウントできた。NFS権限を追加した直後の1回目は`access denied`となり、直後の2回目で成功した。NFS権限の反映待ちと考えられるため、設定内容が正しい場合は少し待って`sudo mount -a`を再実行する。[実機] 2026-10-04 確認
### 2.2 workerのUIDで書き込みを確認する
既存の`anime`・`drama`・`misc`・`personal`・`variety`とアプリのファイルを分け、admin相当の書き込み範囲を限定するため、tver専用サブフォルダを使う。Dockerグループの`kazu634`で、workerコンテナと同じUID・GID `65532:65532`の確認用コンテナを実行し、フォルダ作成、書き込み、読み取り、rename、削除を確認する。`sudo`は不要である。
```bash
docker run --rm --user 65532:65532 -v /mnt/nas-tver/tver:/videos busybox:stable sh -c 'mkdir /videos/.tver-write-test && echo ok > /videos/.tver-write-test/f && cat /videos/.tver-write-test/f && mv /videos/.tver-write-test/f /videos/.tver-write-test/g && ls -ln /videos/.tver-write-test && rm -r /videos/.tver-write-test && echo cleanup-ok'
```
期待する結果は、内容の`ok`、`ls -ln`で表示される`g`の所有者`1024 100`、最後の`cleanup-ok`である。確認後は確認用イメージを削除する。
```bash
docker rmi busybox:stable
```
`/mnt/nas-tver/tver`は作成済みで、上記のDockerコンテナによる操作は実機確認済みである。[実機] 2026-10-04 確認。別環境へ作り直す場合も、書き込み確認が成功するまでworkerを起動しない。
### 2.3 Giteaに配置用ユーザーを用意する
Giteaは`REQUIRE_SIGNIN_VIEW = true`かつ`DISABLE_SSH = true`で動作している。匿名HTTPSの`info/refs`は`401`となり、SSHも使えないため、配置専用の読み取り資格情報を使う。[実機] 2026-10-04 確認
1. Giteaの管理画面で、配置専用ユーザー`tver-deploy`を作成する。
2. `kazu634/tver`リポジトリの「設定」→「共同作業者」で`tver-deploy`を追加し、権限を「読み取り」にする。
3. `tver-deploy`でGiteaへログインし、ユーザー設定の「アプリケーション」からrepository読み取り権限だけを持つトークンを作成する。
4. トークンは作成時に安全な場所へ控え、この文書、シェル履歴、Git管理下のファイルへ実値を書かない。
### 2.4 dockageにclone用トークンを保存する
Dockerホストの管理ユーザー`kazu634`で次を実行する。トークンは非表示で読み取り、権限`600`の専用credentialファイルへ保存する。credential helperはGiteaのHTTPS URLにだけ設定する。
```bash
install -d -m 700 ~/.config/git
read -rs -p 'Gitea token: ' TOKEN; echo
umask 077
printf 'https://tver-deploy:%s@gitea.kazu634.com\n' "$TOKEN" > ~/.config/git/credentials-tver
unset TOKEN
git config --global credential.https://gitea.kazu634.com.helper 'store --file ~/.config/git/credentials-tver'
git clone https://tver-deploy@gitea.kazu634.com/kazu634/tver.git /opt/stacks/tver
cd /opt/stacks/tver
```
この手順で認証付きcloneに成功した。[実機] 2026-10-04 確認。トークンを失効・更新した場合は、同じ手順で`credentials-tver`を作り直す。
### 2.5 アプリの環境変数を用意する
`.env.example`をGit管理外の`.env`へコピーし、`DATABASE_URL`へパスワード用の伏字を入れる。ファイルを`600`にしてから、パスワードを非表示で読み取り、伏字だけを置換する。
```bash
cd /opt/stacks/tver
umask 077
cp .env.example .env
sed -i 's|^DATABASE_URL=.*|DATABASE_URL=postgres://tver:__SET_PASSWORD__@192.168.10.201:15434/tver?sslmode=disable|' .env
chmod 600 .env
read -rs -p 'PostgreSQL password: ' DB_PASSWORD; echo
sed -i "s|__SET_PASSWORD__|${DB_PASSWORD}|" .env
unset DB_PASSWORD
if grep -q '__SET_PASSWORD__' .env; then echo 'DATABASE_URLのパスワードが未設定です' >&2; exit 1; fi
```
完成した`.env`は次の項目を持つ。`DATABASE_URL`の実値は表示したり、文書へ転記したりしない。
```dotenv
DATABASE_URL=<設定済み・表示しない>
VIDEO_HOST_DIR=/mnt/nas-tver/tver
VIDEO_DIR=/videos
WORKER_CONCURRENCY=1
```
> [!warning] パスワードの文字種
> この手順で使うPostgreSQLパスワードには、URLや`sed`の区切りに影響する`@`・`:`・`/`・`?`・`#`・`%`・`|`を含めない。NAS準備時に16進文字列で生成すれば、この制約を満たせる。
### 2.6 起動する
`/opt/stacks/tver`でwebとworkerをビルドして起動する。
```bash
cd /opt/stacks/tver
docker compose up -d --build
```
この方法で起動すると、Dockgeの画面にも`tver`スタックとして表示された。[実機] 2026-10-04 確認。Dockge上でweb・workerがともにrunningになり、再起動を繰り返していないことを確認する。
コマンドでも状態とログを確認できる。
```bash
cd /opt/stacks/tver
docker compose ps
docker compose logs --tail=100 web worker
```
## 3. 動作を確認する
1. ヘルスチェックが`200`と`ok`を返すことを確認する。
```bash
curl --fail --show-error http://192.168.10.155:8080/healthz
```
2. ブラウザで`http://192.168.10.155:8080/`を開き、新着一覧が表示されることを確認する。
3. 番組またはエピソードを1件登録し、`/queue`で待ち行列、進捗、完了を確認する。
4. `/library`から動画を再生し、シークできることを確認する。
5. NASの`/volume1/video/tver`に規則どおりのMP4が作成されたことを確認する。
6. `tver-postgres`のログとweb・workerのログに、接続失敗やマイグレーション失敗がないことを確認する。
> [!success] 本番の動作確認結果
> `/healthz`はHTTP `200`、新着バラエティは98件を表示した。4分の本編`ep5hcgopp2`を登録すると20秒以内に完了し、NASの`video/tver/<番組名>/`へ92,340,468バイトの1080p H.264/AAC MP4が保存された。所有者はUID `1024`・GID `100`、`.incoming`は空だった。`/library`の表示・再生とHTTP Range `206`も確認した。[実機] 2026-10-04 確認
> [!note] Synologyの`@eaDir`
> NASが番組フォルダ内に所有者UID `0`の`@eaDir`を作る。アプリのライブラリ一覧はファイルシステムの走査ではなくDBから作るため、表示や再生には影響しない。[実機] 2026-10-04 確認
## 4. 更新する
```bash
cd /opt/stacks/tver
git status --short
git pull --ff-only
```
`git status --short`に追跡対象ファイルの変更がある場合は、内容を確認してから更新する。pull後、Dockgeで`tver`スタックを再ビルドして再配置し、[[#3. 動作を確認する]]を繰り返す。
yt-dlpを更新するときは`Dockerfile.worker`の`YT_DLP_VERSION`の既定値を変更してリポジトリへ反映し、workerイメージをキャッシュなしで再ビルドする。更新後はworkerログと少数の実動画で抽出・保存を確認する。
## 5. 稼働後に残る作業
> [!todo] PostgreSQLバックアップ
> tver専用PostgreSQL 17コンテナ内の`pg_dump`で取得し、gzip圧縮してNASの`/volume1/backups/tver/`へ保存する日次処理を用意する。保持日数は既定14日とし、稼働確認後に別作業として実装する。
go.mod
+18
View File
@@ -0,0 +1,18 @@
module gitea.kazu634.com/kazu634/tver
go 1.27.1
require (
github.com/fergusstrange/embedded-postgres v1.34.0
github.com/jackc/pgx/v5 v5.11.0
)
require (
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/lib/pq v1.10.9 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/text v0.29.0 // indirect
)
go.sum
+34
View File
@@ -0,0 +1,34 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/fergusstrange/embedded-postgres v1.34.0 h1:c6RKhPKFsLVU+Tdxsx8q0UxCHsvZZ/iShAnljRBXs6s=
github.com/fergusstrange/embedded-postgres v1.34.0/go.mod h1:w0YvnCgf19o6tskInrOOACtnqfVlOvluz3hlNLY7tRk=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg=
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
internal/media/path.go
+82
View File
@@ -0,0 +1,82 @@
package media
import (
"fmt"
"path/filepath"
"strings"
"unicode"
)
const (
maxSeriesBytes = 90
maxEpisodeBytes = 140
maxEpisodeIDBytes = 64
maxFilenameBytes = 255
)
// RelativeVideoPath は worker と web が共有する保存先の相対パスを返す。
func RelativeVideoPath(seriesTitle, episodeTitle, episodeID string) (string, error) {
series := sanitizeComponent(seriesTitle, maxSeriesBytes)
episode := sanitizeComponent(episodeTitle, maxEpisodeBytes)
id := sanitizeComponent(episodeID, -1)
if len(id) > maxEpisodeIDBytes {
return "", fmt.Errorf("エピソードIDはサニタイズ後%dバイト以下である必要があります: %dバイト", maxEpisodeIDBytes, len(id))
}
// A directory entry must fit within the NFS namlen limit. Keep the episode
// ID intact and use the remaining bytes for the episode title.
const separators = " [].mp4"
seriesInFilename := series
titleBytes := maxFilenameBytes - len(seriesInFilename) - len(id) - len(separators)
if titleBytes < 0 {
titleBytes = 0
}
episode = truncateComponent(episode, titleBytes, false)
return filepath.Join(series, seriesInFilename+" "+episode+" ["+id+"].mp4"), nil
}
func sanitizeComponent(value string, maxBytes int) string {
var result []rune
lastReplacement := false
for _, r := range value {
invalid := unicode.IsControl(r) || strings.ContainsRune(`<>:"/\|?*`, r)
if invalid {
if !lastReplacement {
result = append(result, '_')
lastReplacement = true
}
continue
}
if unicode.IsSpace(r) {
r = ' '
}
result = append(result, r)
lastReplacement = false
}
clean := strings.Trim(string(result), " .")
if clean == "" || clean == "." || clean == ".." {
return "名称未設定"
}
return truncateComponent(clean, maxBytes, true)
}
func truncateComponent(value string, maxBytes int, fallback bool) string {
if maxBytes < 0 {
return value
}
var limited []rune
bytes := 0
for _, r := range value {
size := len(string(r))
if bytes+size > maxBytes {
break
}
limited = append(limited, r)
bytes += size
}
clean := strings.Trim(string(limited), " .")
if fallback && (clean == "" || clean == "." || clean == "..") {
return "名称未設定"
}
return clean
}
internal/media/path_test.go
+267
View File
@@ -0,0 +1,267 @@
package media
import (
"fmt"
"path/filepath"
"strings"
"testing"
"unicode"
"unicode/utf8"
)
func TestRelativeVideoPath(t *testing.T) {
got, err := RelativeVideoPath("番組名", "第1話", "ep123")
if err != nil {
t.Fatal(err)
}
want := filepath.Join("番組名", "番組名 第1話 [ep123].mp4")
if got != want {
t.Fatalf("path=%q want=%q", got, want)
}
other, err := RelativeVideoPath("番組名", "第1話", "ep456")
if err != nil {
t.Fatal(err)
}
if got == other || !strings.Contains(other, "[ep456]") {
t.Fatalf("ID による衝突回避: %q %q", got, other)
}
}
func TestRelativeVideoPathSanitizesComponents(t *testing.T) {
got, err := RelativeVideoPath(" 番組:/\\*?<>|\"\x00\n\t\x7f\u0085. ", "\x00\n", "../")
if err != nil {
t.Fatal(err)
}
if filepath.IsAbs(got) || strings.ContainsAny(got, `:*?<>|"`) || strings.Contains(got, "..") {
t.Fatalf("禁止文字が残った: %q", got)
}
for _, r := range got {
if unicode.IsControl(r) {
t.Errorf("制御文字が残った: %U in %q", r, got)
}
}
parts := strings.Split(filepath.ToSlash(got), "/")
if len(parts) != 2 || parts[0] == "" || !strings.HasSuffix(got, ".mp4") {
t.Fatalf("相対パスの形: %q", got)
}
}
func TestRelativeVideoPathHandlesEmptyAndLongNames(t *testing.T) {
for _, name := range []string{"", " ", ".", "..", "..."} {
got, err := RelativeVideoPath(name, name, "")
if err != nil {
t.Fatal(err)
}
if !strings.Contains(got, "名称未設定") {
t.Errorf("空の代替名 %q: %q", name, got)
}
}
long := strings.Repeat("界", 200)
got, err := RelativeVideoPath(long, long, "id")
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(got), "/")
if len(parts) != 2 || len([]byte(parts[0])) != maxSeriesBytes || !utf8.ValidString(parts[0]) {
t.Fatalf("番組名の%dバイト境界: %q (%d bytes)", maxSeriesBytes, parts[0], len([]byte(parts[0])))
}
if len([]byte(parts[1])) > maxFilenameBytes || !utf8.ValidString(parts[1]) {
t.Fatalf("ファイル名の上限: %q (%d bytes)", parts[1], len([]byte(parts[1])))
}
if got := sanitizeComponent(strings.Repeat("a", maxEpisodeBytes+1), maxEpisodeBytes); len([]byte(got)) != maxEpisodeBytes {
t.Fatalf("ASCII の%dバイト境界: %d %q", maxEpisodeBytes, len([]byte(got)), got)
}
}
func TestRelativeVideoPathEpisodeIDLimit(t *testing.T) {
id := strings.Repeat("i", maxEpisodeIDBytes)
got, err := RelativeVideoPath(strings.Repeat("s", maxSeriesBytes), strings.Repeat("t", maxEpisodeBytes), id)
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(got), "/")
if len(parts) != 2 || len([]byte(parts[1])) > maxFilenameBytes || !strings.Contains(parts[1], "["+id+"]") {
t.Fatalf("64バイトIDのファイル名=%q", got)
}
if _, err := RelativeVideoPath("series", "episode", strings.Repeat("i", maxEpisodeIDBytes+1)); err == nil {
t.Fatal("65バイトのエピソードIDを受理した")
}
}
func TestRelativeVideoPathUsesFilenameBudgetInRequiredOrder(t *testing.T) {
series := strings.Repeat("s", maxSeriesBytes)
title := strings.Repeat("t", maxEpisodeBytes)
for _, tt := range []struct {
name string
idBytes int
wantSeriesInFile int
wantEpisodeInFile int
wantFilenameBytes int
}{
{"ちょうど255バイト", 17, 90, 140, 255},
{"回タイトルから1バイト削る", 18, 90, 139, 255},
{"64バイトIDでも番組名を維持", 64, 90, 93, 255},
} {
t.Run(tt.name, func(t *testing.T) {
id := strings.Repeat("i", tt.idBytes)
path, err := RelativeVideoPath(series, title, id)
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(path), "/")
if len(parts) != 2 || parts[0] != series {
t.Fatalf("ディレクトリ名=%q", parts)
}
wantFilename := strings.Repeat("s", tt.wantSeriesInFile) + " " + strings.Repeat("t", tt.wantEpisodeInFile) + " [" + id + "].mp4"
if parts[1] != wantFilename || len([]byte(parts[1])) != tt.wantFilenameBytes {
t.Fatalf("filename=%q (%d bytes) want=%q (%d bytes)", parts[1], len([]byte(parts[1])), wantFilename, tt.wantFilenameBytes)
}
})
}
}
func TestRelativeVideoPathEpisodeIDNearAndBeyondFilenameLimit(t *testing.T) {
for _, idBytes := range []int{64, 65, 248, 255, 256} {
t.Run(fmt.Sprintf("id_%d_bytes", idBytes), func(t *testing.T) {
id := strings.Repeat("i", idBytes)
path, err := RelativeVideoPath("series", "episode", id)
if idBytes > 64 {
if err == nil || path != "" {
t.Fatalf("%dバイトID: path=%q err=%v, want error", idBytes, path, err)
}
if !strings.Contains(err.Error(), "64バイト以下") {
t.Fatalf("%dバイトIDのエラー理由=%q", idBytes, err)
}
return
}
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(path), "/")
if len(parts) != 2 {
t.Fatalf("parts=%q", parts)
}
if !strings.Contains(parts[1], "["+id+"]") {
t.Fatalf("episode ID was changed: %q", parts[1])
}
if got := len([]byte(parts[1])); got > maxFilenameBytes {
t.Fatalf("episode ID %d bytes produced %d-byte filename above %d: %q", idBytes, got, maxFilenameBytes, parts[1])
}
if !utf8.ValidString(parts[1]) {
t.Fatalf("filename is not valid UTF-8: %q", parts[1])
}
})
}
}
func TestRelativeVideoPathHasNoRuneLimit(t *testing.T) {
series := strings.Repeat("a", 90)
title := strings.Repeat("b", 100)
got, err := RelativeVideoPath(series, title, "id")
if err != nil {
t.Fatal(err)
}
parts := strings.Split(filepath.ToSlash(got), "/")
if parts[0] != series || !strings.Contains(parts[1], " "+title+" [id].mp4") {
t.Fatalf("80 rune limit remains: %q", got)
}
}
func TestSanitizeComponentTrimsWhitespaceBeforeLengthLimit(t *testing.T) {
for _, tt := range []struct {
name, input, want string
}{
{"前後の空白", " \t　番組名\n ", "_ 番組名_"},
{"空白のみ", " 　 ", "名称未設定"},
{"制御文字のみ", "\t\n", "_"},
{"多数の先頭空白", strings.Repeat(" ", 100) + "番組名", "番組名"},
{"先頭空白と長い名前", strings.Repeat(" ", 70) + strings.Repeat("a", 100), strings.Repeat("a", maxSeriesBytes)},
} {
t.Run(tt.name, func(t *testing.T) {
if got := sanitizeComponent(tt.input, maxSeriesBytes); got != tt.want {
t.Errorf("sanitizeComponent()=%q want=%q", got, tt.want)
}
})
}
}
func TestSanitizeComponentTrimsAgainAfterLengthLimit(t *testing.T) {
for _, tt := range []struct {
name, input, want string
}{
{"90バイト目が空白", strings.Repeat("a", 89) + " " + "b", strings.Repeat("a", 89)},
{"90バイト目がドット", strings.Repeat("a", 89) + "." + "b", strings.Repeat("a", 89)},
{"切り詰め部分が空白のみ", "a" + strings.Repeat(" ", 89) + "b", "a"},
{"切り詰め部分がドットのみ", "a" + strings.Repeat(".", 89) + "b", "a"},
{"複数バイト文字の直後がドット", strings.Repeat("界", 29) + ".続", strings.Repeat("界", 29)},
} {
t.Run(tt.name, func(t *testing.T) {
got := sanitizeComponent(tt.input, maxSeriesBytes)
if got != tt.want {
t.Errorf("sanitizeComponent()=%q want=%q", got, tt.want)
}
if strings.Trim(got, " .") != got || got == "" {
t.Errorf("切り詰め後の名前が不正: %q", got)
}
})
}
}
func TestSanitizeComponentSafetyProperties(t *testing.T) {
var controls strings.Builder
for r := rune(0); r <= 0x9f; r++ {
if unicode.IsControl(r) {
controls.WriteRune(r)
}
}
inputs := []string{
"", " ", ".", "..", "...", `<>:"/\|?*`, controls.String(),
" \t　番組名\n. ", "a/b\\c", strings.Repeat("a", 500),
strings.Repeat("界", 500), strings.Repeat("😀", 500),
strings.Repeat(".", 100) + "name" + strings.Repeat(" ", 100),
string([]byte{0xff, 0xfe, 'a', 0xc0, 0xaf}),
}
for i, input := range inputs {
got := sanitizeComponent(input, maxEpisodeBytes)
if got == "" || got == "." || got == ".." {
t.Errorf("case %d: 空またはdot segment: %q", i, got)
}
if strings.Trim(got, " .") != got {
t.Errorf("case %d: 前後に空白またはdot: %q", i, got)
}
if len([]byte(got)) > 140 {
t.Errorf("case %d: 上限超過: bytes=%d %q", i, len([]byte(got)), got)
}
if !utf8.ValidString(got) {
t.Errorf("case %d: 不正なUTF-8: %q", i, got)
}
if strings.ContainsAny(got, `<>:"/\|?*`) {
t.Errorf("case %d: 禁止文字が残った: %q", i, got)
}
for _, r := range got {
if unicode.IsControl(r) {
t.Errorf("case %d: 制御文字が残った: %U in %q", i, r, got)
}
}
if again := sanitizeComponent(got, maxEpisodeBytes); again != got {
t.Errorf("case %d: 再適用で変化: %q -> %q", i, got, again)
}
path, err := RelativeVideoPath(input, input, "id")
if err != nil {
t.Errorf("case %d: パス生成エラー: %v", i, err)
continue
}
parts := strings.Split(filepath.ToSlash(path), "/")
if filepath.IsAbs(path) || len(parts) != 2 || parts[0] == "" || parts[0] == "." || parts[0] == ".." || !strings.HasSuffix(parts[1], ".mp4") {
t.Errorf("case %d: 安全でない相対パス: %q parts=%q", i, path, parts)
continue
}
if len([]byte(parts[0])) > 90 || len([]byte(parts[1])) > 255 {
t.Errorf("case %d: パス要素の上限超過: dir=%d filename=%d path=%q", i, len([]byte(parts[0])), len([]byte(parts[1])), path)
}
if !utf8.ValidString(path) {
t.Errorf("case %d: パスが不正なUTF-8: %q", i, path)
}
}
}
internal/store/library_test.go
+98
View File
@@ -0,0 +1,98 @@
package store
import (
"context"
"errors"
"testing"
"github.com/jackc/pgx/v5"
)
func completeTestJob(t *testing.T, s *Store, id string) Job {
t.Helper()
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode(id), "manual")
if err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != job.ID {
t.Fatalf("claim: %+v %v", claimed, err)
}
if err := s.Complete(ctx, job.ID, "番組/video.mp4", 1234); err != nil {
t.Fatal(err)
}
result, err := s.GetSucceededJob(ctx, job.ID)
if err != nil {
t.Fatal(err)
}
return result
}
func TestLibraryStoreOperations(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
first := completeTestJob(t, s, "library-1")
second := completeTestJob(t, s, "library-2")
jobs, err := s.ListSucceededJobs(ctx, 1)
if err != nil || len(jobs) != 1 || jobs[0].ID != second.ID {
t.Fatalf("完了一覧: %+v %v", jobs, err)
}
if first.FilePath == nil || *first.FilePath != "番組/video.mp4" || first.FileSizeBytes == nil || *first.FileSizeBytes != 1234 || first.FinishedAt == nil {
t.Fatalf("完了情報: %+v", first)
}
queued, _, err := s.Enqueue(ctx, sampleEpisode("not-finished"), "manual")
if err != nil {
t.Fatal(err)
}
if _, err := s.GetSucceededJob(ctx, queued.ID); !errors.Is(err, pgx.ErrNoRows) {
t.Fatalf("未完了の取得: %v", err)
}
if err := s.DeleteSucceededJob(ctx, queued.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("未完了の削除: %v", err)
}
if err := s.DeleteSucceededJob(ctx, first.ID); err != nil {
t.Fatal(err)
}
if _, err := s.GetJob(ctx, first.ID); !errors.Is(err, pgx.ErrNoRows) {
t.Fatalf("削除後の取得: %v", err)
}
if err := s.DeleteSucceededJob(ctx, first.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("再削除: %v", err)
}
if _, err := s.ListSucceededJobs(ctx, 0); err == nil {
t.Fatal("0件の上限を許した")
}
}
func TestDeleteSucceededJobKeepsKnownEpisodeAndAllowsManualRequeue(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
ep := seriesEpisode("known-library", "known-series")
created, err := s.RegisterSubscription(ctx, "known-series", "番組", []Episode{ep}, false)
if err != nil || !created {
t.Fatalf("番組登録: %t %v", created, err)
}
job, created, err := s.Enqueue(ctx, ep, "manual")
if err != nil || !created {
t.Fatalf("手動投入: %+v %t %v", job, created, err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != job.ID {
t.Fatalf("取得: %+v %v", claimed, err)
}
if err := s.Complete(ctx, job.ID, "番組/video.mp4", 10); err != nil {
t.Fatal(err)
}
if err := s.DeleteSucceededJob(ctx, job.ID); err != nil {
t.Fatal(err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "known-series", []string{ep.ID})
if err != nil || len(unknown) != 0 {
t.Fatalf("削除後の既知記録: %v %v", unknown, err)
}
requeued, created, err := s.Enqueue(ctx, ep, "manual")
if err != nil || !created || requeued.Status != "queued" || requeued.Source != "manual" {
t.Fatalf("削除後の手動再投入: %+v %t %v", requeued, created, err)
}
}
internal/store/migrate.go
+73
View File
@@ -0,0 +1,73 @@
package store
import (
"context"
"embed"
"fmt"
"io/fs"
"sort"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
//go:embed migrations/*.sql
var migrations embed.FS
const migrationLockID int64 = 0x747665725f646231
// Migrate は同じ接続で排他ロックを保持し、未適用の SQL を順に実行する。
func Migrate(ctx context.Context, pool *pgxpool.Pool) error {
conn, err := pool.Acquire(ctx)
if err != nil {
return fmt.Errorf("migration connection: %w", err)
}
defer conn.Release()
if _, err := conn.Exec(ctx, "SELECT pg_advisory_lock($1)", migrationLockID); err != nil {
return fmt.Errorf("migration lock: %w", err)
}
defer func() {
// 接続を返す前にロックを解放する。解放に失敗した接続は閉じる。
if _, err := conn.Exec(context.Background(), "SELECT pg_advisory_unlock($1)", migrationLockID); err != nil {
conn.Conn().Close(context.Background())
}
}()
if _, err := conn.Exec(ctx, `CREATE TABLE IF NOT EXISTS schema_migrations (
version TEXT PRIMARY KEY,
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
)`); err != nil {
return fmt.Errorf("migration metadata: %w", err)
}
files, err := fs.Glob(migrations, "migrations/*.sql")
if err != nil {
return err
}
sort.Strings(files)
for _, name := range files {
tx, err := conn.Begin(ctx)
if err != nil {
return err
}
var applied bool
err = tx.QueryRow(ctx, "SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version=$1)", name).Scan(&applied)
if err == nil && !applied {
var sql []byte
sql, err = migrations.ReadFile(name)
if err == nil {
_, err = tx.Exec(ctx, string(sql), pgx.QueryExecModeSimpleProtocol)
}
if err == nil {
_, err = tx.Exec(ctx, "INSERT INTO schema_migrations (version) VALUES ($1)", name)
}
}
if err != nil {
_ = tx.Rollback(ctx)
return fmt.Errorf("migration %s: %w", name, err)
}
if err := tx.Commit(ctx); err != nil {
return fmt.Errorf("migration %s commit: %w", name, err)
}
}
return nil
}
internal/store/migrations/001_initial.sql
+44
View File
@@ -0,0 +1,44 @@
CREATE TABLE download_jobs (
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
episode_id TEXT NOT NULL UNIQUE,
series_id TEXT NOT NULL,
series_title TEXT NOT NULL,
title TEXT NOT NULL,
broadcaster_name TEXT NOT NULL,
duration_seconds INTEGER NOT NULL CHECK (duration_seconds >= 0),
end_at TIMESTAMPTZ,
thumbnail_path TEXT NOT NULL,
source TEXT NOT NULL CHECK (source IN ('manual', 'subscription')),
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'succeeded', 'failed', 'canceled')),
phase TEXT CHECK (phase IN ('video', 'audio', 'merging', 'moving')),
downloaded_bytes BIGINT CHECK (downloaded_bytes >= 0),
total_bytes BIGINT CHECK (total_bytes >= 0),
total_is_estimate BOOLEAN NOT NULL DEFAULT FALSE,
speed_bps DOUBLE PRECISION CHECK (speed_bps >= 0),
eta_seconds DOUBLE PRECISION CHECK (eta_seconds >= 0),
fragment_index INTEGER CHECK (fragment_index >= 0),
fragment_count INTEGER CHECK (fragment_count >= 0),
format_id TEXT,
error_kind TEXT CHECK (error_kind IN ('drm', 'geo', 'network', 'unavailable', 'extract', 'canceled', 'invalid_output', 'unknown')),
error_message TEXT,
file_path TEXT,
file_size_bytes BIGINT CHECK (file_size_bytes >= 0),
attempts INTEGER NOT NULL DEFAULT 0 CHECK (attempts >= 0),
heartbeat_at TIMESTAMPTZ,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
started_at TIMESTAMPTZ,
finished_at TIMESTAMPTZ,
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX download_jobs_claim_idx ON download_jobs (created_at, id) WHERE status = 'queued';
CREATE INDEX download_jobs_status_newest_idx ON download_jobs (status, created_at DESC, id DESC);
CREATE INDEX download_jobs_stale_idx ON download_jobs (heartbeat_at) WHERE status = 'running';
CREATE TABLE subscriptions (
series_id TEXT PRIMARY KEY,
series_title TEXT NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT TRUE,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
last_checked_at TIMESTAMPTZ
);
internal/store/migrations/002_known_episodes.sql
+8
View File
@@ -0,0 +1,8 @@
CREATE TABLE known_episodes (
series_id TEXT NOT NULL REFERENCES subscriptions(series_id) ON DELETE CASCADE,
episode_id TEXT NOT NULL,
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
PRIMARY KEY (series_id, episode_id)
);
CREATE INDEX download_jobs_series_idx ON download_jobs (series_id);
internal/store/migrations/003_subscription_check_errors.sql
+6
View File
@@ -0,0 +1,6 @@
ALTER TABLE subscriptions
ADD COLUMN last_check_error_at TIMESTAMPTZ,
ADD COLUMN last_check_error TEXT;
CREATE INDEX subscriptions_due_idx
ON subscriptions (last_checked_at) WHERE enabled;
internal/store/store.go
+328
View File
@@ -0,0 +1,328 @@
package store
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
var ErrInvalidTransition = errors.New("invalid job status transition")
type Store struct{ pool *pgxpool.Pool }
func New(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
type Episode struct {
ID, SeriesID, SeriesTitle, Title, BroadcasterName, ThumbnailPath string
DurationSeconds int
EndAt *time.Time
}
type Job struct {
ID int64
EpisodeID, SeriesID, SeriesTitle string
Title, BroadcasterName string
DurationSeconds int
EndAt *time.Time
ThumbnailPath, Source, Status string
Phase, FormatID *string
DownloadedBytes, TotalBytes *int64
TotalIsEstimate bool
SpeedBPS, ETASeconds *float64
FragmentIndex, FragmentCount *int32
ErrorKind, ErrorMessage, FilePath *string
FileSizeBytes *int64
Attempts int
HeartbeatAt, StartedAt, FinishedAt *time.Time
CreatedAt, UpdatedAt time.Time
}
type Progress struct {
Phase string
DownloadedBytes *int64
TotalBytes *int64
TotalIsEstimate bool
SpeedBPS *float64
ETASeconds *float64
FragmentIndex *int32
FragmentCount *int32
FormatID *string
}
type Subscription struct {
SeriesID, SeriesTitle string
Enabled bool
CreatedAt time.Time
LastCheckedAt *time.Time
LastCheckErrorAt *time.Time
LastCheckError *string
JobCount int64
}
const jobColumns = `id, episode_id, series_id, series_title, title, broadcaster_name,
duration_seconds, end_at, thumbnail_path, source, status, phase,
downloaded_bytes, total_bytes, total_is_estimate, speed_bps, eta_seconds,
fragment_index, fragment_count, format_id, error_kind, error_message,
file_path, file_size_bytes, attempts, heartbeat_at, created_at,
started_at, finished_at, updated_at`
var claimedJobColumns = func() string {
columns := strings.Split(jobColumns, ",")
for i, column := range columns {
columns[i] = "j." + strings.TrimSpace(column)
}
return strings.Join(columns, ", ")
}()
func scanJob(row pgx.Row) (Job, error) {
var j Job
err := row.Scan(&j.ID, &j.EpisodeID, &j.SeriesID, &j.SeriesTitle, &j.Title,
&j.BroadcasterName, &j.DurationSeconds, &j.EndAt, &j.ThumbnailPath,
&j.Source, &j.Status, &j.Phase, &j.DownloadedBytes, &j.TotalBytes,
&j.TotalIsEstimate, &j.SpeedBPS, &j.ETASeconds, &j.FragmentIndex,
&j.FragmentCount, &j.FormatID, &j.ErrorKind, &j.ErrorMessage,
&j.FilePath, &j.FileSizeBytes, &j.Attempts, &j.HeartbeatAt,
&j.CreatedAt, &j.StartedAt, &j.FinishedAt, &j.UpdatedAt)
return j, err
}
// Enqueue は同じエピソードの既存ジョブを再利用する。
func (s *Store) Enqueue(ctx context.Context, ep Episode, source string) (Job, bool, error) {
row := s.pool.QueryRow(ctx, `INSERT INTO download_jobs
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9) ON CONFLICT (episode_id) DO NOTHING RETURNING `+jobColumns,
ep.ID, ep.SeriesID, ep.SeriesTitle, ep.Title, ep.BroadcasterName,
ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath, source)
job, err := scanJob(row)
if err == nil {
return job, true, nil
}
if !errors.Is(err, pgx.ErrNoRows) {
return Job{}, false, err
}
job, err = scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE episode_id=$1`, ep.ID))
return job, false, err
}
func (s *Store) GetJob(ctx context.Context, id int64) (Job, error) {
return scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE id=$1`, id))
}
func (s *Store) GetSucceededJob(ctx context.Context, id int64) (Job, error) {
return scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE id=$1 AND status='succeeded'`, id))
}
func (s *Store) ListSucceededJobs(ctx context.Context, limit int) ([]Job, error) {
if limit <= 0 {
return nil, errors.New("limit must be positive")
}
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
WHERE status='succeeded' ORDER BY finished_at DESC NULLS LAST, id DESC LIMIT $1`, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var jobs []Job
for rows.Next() {
job, err := scanJob(rows)
if err != nil {
return nil, err
}
jobs = append(jobs, job)
}
return jobs, rows.Err()
}
func (s *Store) DeleteSucceededJob(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `DELETE FROM download_jobs WHERE id=$1 AND status='succeeded'`, id)
return transitionResult(tag.RowsAffected(), err)
}
// Retry は失敗・中断されたジョブの実行時情報を消して再投入する。
func (s *Store) Retry(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
error_kind=NULL, error_message=NULL, heartbeat_at=NULL, started_at=NULL,
finished_at=NULL, updated_at=now()
WHERE id=$1 AND status IN ('failed','canceled')`, id)
return transitionResult(tag.RowsAffected(), err)
}
// RetryEpisode はエピソード ID で失敗・中断されたジョブを再投入する。
func (s *Store) RetryEpisode(ctx context.Context, episodeID string) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
error_kind=NULL, error_message=NULL, heartbeat_at=NULL, started_at=NULL,
finished_at=NULL, updated_at=now()
WHERE episode_id=$1 AND status IN ('failed','canceled')`, episodeID)
return transitionResult(tag.RowsAffected(), err)
}
// ClaimNext は最も古い未実行ジョブを同時 worker と重複せず取得する。
func (s *Store) ClaimNext(ctx context.Context) (Job, error) {
return scanJob(s.pool.QueryRow(ctx, `WITH next AS (
SELECT id FROM download_jobs WHERE status='queued'
ORDER BY created_at, id LIMIT 1 FOR UPDATE SKIP LOCKED
) UPDATE download_jobs AS j SET status='running', attempts=j.attempts+1,
started_at=now(), heartbeat_at=now(), updated_at=now()
FROM next WHERE j.id=next.id RETURNING `+claimedJobColumns))
}
func (s *Store) UpdateProgress(ctx context.Context, id int64, p Progress) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET phase=$2, downloaded_bytes=$3,
total_bytes=$4, total_is_estimate=$5, speed_bps=$6, eta_seconds=$7,
fragment_index=$8, fragment_count=$9, format_id=$10,
heartbeat_at=now(), updated_at=now() WHERE id=$1 AND status='running'`,
id, p.Phase, p.DownloadedBytes, p.TotalBytes, p.TotalIsEstimate,
p.SpeedBPS, p.ETASeconds, p.FragmentIndex, p.FragmentCount, p.FormatID)
return transitionResult(tag.RowsAffected(), err)
}
// Heartbeat は進捗出力がない間も実行中ジョブの生存を記録する。
func (s *Store) Heartbeat(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=now(), updated_at=now()
WHERE id=$1 AND status='running'`, id)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) Complete(ctx context.Context, id int64, filePath string, fileSizeBytes int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='succeeded', phase=NULL,
file_path=$2, file_size_bytes=$3, heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status='running'`, id, filePath, fileSizeBytes)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) Fail(ctx context.Context, id int64, kind, message string) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='failed', phase=NULL,
error_kind=$2, error_message=$3, heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status='running'`, id, kind, message)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) Cancel(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='canceled', phase=NULL,
error_kind='canceled', heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status IN ('queued','running')`, id)
return transitionResult(tag.RowsAffected(), err)
}
// CancelQueued は worker 未着手のジョブだけを取り消す。
func (s *Store) CancelQueued(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='canceled', phase=NULL,
error_kind='canceled', heartbeat_at=NULL, finished_at=now(), updated_at=now()
WHERE id=$1 AND status='queued'`, id)
return transitionResult(tag.RowsAffected(), err)
}
// RequeueStale は cutoff より前に heartbeat が止まった実行中ジョブを戻す。
func (s *Store) RequeueStale(ctx context.Context, cutoff time.Time) (int64, error) {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
heartbeat_at=NULL, started_at=NULL, updated_at=now()
WHERE status='running' AND heartbeat_at < $1`, cutoff)
return tag.RowsAffected(), err
}
// RequeueRunning は worker の正常終了時に担当中ジョブを待機中へ戻す。
func (s *Store) RequeueRunning(ctx context.Context, id int64) error {
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
heartbeat_at=NULL, started_at=NULL, updated_at=now()
WHERE id=$1 AND status='running'`, id)
return transitionResult(tag.RowsAffected(), err)
}
func transitionResult(count int64, err error) error {
if err != nil {
return err
}
if count == 0 {
return ErrInvalidTransition
}
return nil
}
// ListJobs は指定状態のジョブを新しい順に返す。空の状態は全件を意味する。
func (s *Store) ListJobs(ctx context.Context, status string, limit int) ([]Job, error) {
if status != "" && status != "queued" && status != "running" && status != "succeeded" && status != "failed" && status != "canceled" {
return nil, fmt.Errorf("invalid status: %s", status)
}
if limit <= 0 {
return nil, errors.New("limit must be positive")
}
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
WHERE ($1='' OR status=$1) ORDER BY created_at DESC, id DESC LIMIT $2`, status, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var jobs []Job
for rows.Next() {
job, err := scanJob(rows)
if err != nil {
return nil, err
}
jobs = append(jobs, job)
}
return jobs, rows.Err()
}
func (s *Store) EpisodeStatuses(ctx context.Context, ids []string) (map[string]string, error) {
result := make(map[string]string)
if len(ids) == 0 {
return result, nil
}
rows, err := s.pool.Query(ctx, `SELECT episode_id, status FROM download_jobs WHERE episode_id = ANY($1)`, ids)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var id, status string
if err := rows.Scan(&id, &status); err != nil {
return nil, err
}
result[id] = status
}
return result, rows.Err()
}
func (s *Store) DeleteSubscription(ctx context.Context, seriesID string) (bool, error) {
tag, err := s.pool.Exec(ctx, `DELETE FROM subscriptions WHERE series_id=$1`, seriesID)
return tag.RowsAffected() == 1, err
}
func (s *Store) ListSubscriptions(ctx context.Context) ([]Subscription, error) {
rows, err := s.pool.Query(ctx, `SELECT s.series_id, s.series_title, s.enabled, s.created_at, s.last_checked_at,
s.last_check_error_at, s.last_check_error,
(SELECT count(*) FROM download_jobs j WHERE j.series_id=s.series_id)
FROM subscriptions s ORDER BY s.created_at DESC, s.series_id`)
if err != nil {
return nil, err
}
defer rows.Close()
var subs []Subscription
for rows.Next() {
var sub Subscription
if err := rows.Scan(&sub.SeriesID, &sub.SeriesTitle, &sub.Enabled, &sub.CreatedAt, &sub.LastCheckedAt,
&sub.LastCheckErrorAt, &sub.LastCheckError, &sub.JobCount); err != nil {
return nil, err
}
subs = append(subs, sub)
}
return subs, rows.Err()
}
func (s *Store) SetSubscriptionChecked(ctx context.Context, seriesID string, checkedAt time.Time) error {
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, seriesID, checkedAt)
return transitionResult(tag.RowsAffected(), err)
}
internal/store/store_test.go
+766
View File
@@ -0,0 +1,766 @@
package store
import (
"context"
"errors"
"fmt"
"io"
"net"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
embeddedpostgres "github.com/fergusstrange/embedded-postgres"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
var adminPool *pgxpool.Pool
var testPort int
var dbSequence atomic.Uint64
func TestMain(m *testing.M) {
root, err := os.MkdirTemp("/private/tmp", "tver-pg17-")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
testPort = listener.Addr().(*net.TCPAddr).Port
listener.Close()
config := embeddedpostgres.DefaultConfig().Version(embeddedpostgres.V17).
Port(uint32(testPort)).CachePath("/private/tmp/tver-embedded-pg-cache").
RuntimePath(filepath.Join(root, "runtime")).DataPath(filepath.Join(root, "data")).
Logger(io.Discard).StartTimeout(60 * time.Second)
db := embeddedpostgres.NewDatabase(config)
if err := db.Start(); err != nil {
fmt.Fprintf(os.Stderr, "embedded Postgres 17 start: %v\n", err)
_ = os.RemoveAll(root)
os.Exit(1)
}
adminPool, err = pgxpool.New(context.Background(), config.GetConnectionURL())
if err != nil {
fmt.Fprintln(os.Stderr, err)
_ = db.Stop()
_ = os.RemoveAll(root)
os.Exit(1)
}
code := m.Run()
adminPool.Close()
if err := db.Stop(); err != nil {
fmt.Fprintf(os.Stderr, "embedded Postgres 17 stop: %v\n", err)
code = 1
}
_ = os.RemoveAll(root)
os.Exit(code)
}
func testPool(t *testing.T, migrate bool) *pgxpool.Pool {
t.Helper()
name := fmt.Sprintf("tver_test_%d", dbSequence.Add(1))
ctx := context.Background()
if _, err := adminPool.Exec(ctx, `CREATE DATABASE `+name); err != nil {
t.Fatal(err)
}
url := fmt.Sprintf("postgres://postgres:postgres@localhost:%d/%s", testPort, name)
pool, err := pgxpool.New(ctx, url)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() {
pool.Close()
if _, err := adminPool.Exec(ctx, `DROP DATABASE `+name); err != nil {
t.Error(err)
}
})
if migrate {
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
}
return pool
}
func sampleEpisode(id string) Episode {
end := time.Date(2026, 10, 4, 12, 0, 0, 0, time.UTC)
return Episode{ID: id, SeriesID: "series-1", SeriesTitle: "番組", Title: "第1話",
BroadcasterName: "放送局", DurationSeconds: 300, EndAt: &end, ThumbnailPath: "/thumb.jpg"}
}
func TestMigrate(t *testing.T) {
pool := testPool(t, false)
ctx := context.Background()
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var first time.Time
if err := pool.QueryRow(ctx, `SELECT applied_at FROM schema_migrations WHERE version='migrations/001_initial.sql'`).Scan(&first); err != nil {
t.Fatal(err)
}
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var count int
var second time.Time
if err := pool.QueryRow(ctx, `SELECT count(*), min(applied_at) FROM schema_migrations`).Scan(&count, &second); err != nil {
t.Fatal(err)
}
if count != 3 || !first.Equal(second) {
t.Fatalf("再適用で履歴が変わった: %d %v %v", count, first, second)
}
}
func TestMigrateConcurrent(t *testing.T) {
pool := testPool(t, false)
const workers = 8
errCh := make(chan error, workers)
var wg sync.WaitGroup
for range workers {
wg.Add(1)
go func() { defer wg.Done(); errCh <- Migrate(context.Background(), pool) }()
}
wg.Wait()
close(errCh)
for err := range errCh {
if err != nil {
t.Fatal(err)
}
}
var count int
if err := pool.QueryRow(context.Background(), `SELECT count(*) FROM schema_migrations`).Scan(&count); err != nil {
t.Fatal(err)
}
if count != 3 {
t.Fatalf("適用件数: %d", count)
}
}
func TestMigrateAddsKnownEpisodesToExistingDatabase(t *testing.T) {
pool := testPool(t, false)
ctx := context.Background()
initial, err := migrations.ReadFile("migrations/001_initial.sql")
if err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, string(initial), pgx.QueryExecModeSimpleProtocol); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `CREATE TABLE schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())`); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `INSERT INTO schema_migrations (version) VALUES ('migrations/001_initial.sql')`); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title) VALUES ('existing', '既存番組')`); err != nil {
t.Fatal(err)
}
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var count int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM schema_migrations`).Scan(&count); err != nil || count != 3 {
t.Fatalf("移行履歴: %d %v", count, err)
}
if _, err := pool.Exec(ctx, `INSERT INTO known_episodes (series_id, episode_id) VALUES ('existing', 'ep1')`); err != nil {
t.Fatalf("既存番組への既知記録: %v", err)
}
if _, err := pool.Exec(ctx, `DELETE FROM subscriptions WHERE series_id='existing'`); err != nil {
t.Fatal(err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id='existing'`).Scan(&count); err != nil || count != 0 {
t.Fatalf("アップグレード後の cascade: %d %v", count, err)
}
}
func TestMigrateAddsSubscriptionCheckErrorsToVersionTwoDatabase(t *testing.T) {
pool := testPool(t, false)
ctx := context.Background()
for _, name := range []string{"migrations/001_initial.sql", "migrations/002_known_episodes.sql"} {
sql, err := migrations.ReadFile(name)
if err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, string(sql), pgx.QueryExecModeSimpleProtocol); err != nil {
t.Fatal(err)
}
}
if _, err := pool.Exec(ctx, `CREATE TABLE schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())`); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `INSERT INTO schema_migrations (version) VALUES ('migrations/001_initial.sql'), ('migrations/002_known_episodes.sql')`); err != nil {
t.Fatal(err)
}
checked := time.Date(2026, 10, 1, 2, 3, 4, 0, time.UTC)
if _, err := pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title, last_checked_at) VALUES ('existing-v2', '既存番組', $1)`, checked); err != nil {
t.Fatal(err)
}
if err := Migrate(ctx, pool); err != nil {
t.Fatal(err)
}
var title string
var last time.Time
var errorAt, errorMessage any
if err := pool.QueryRow(ctx, `SELECT series_title, last_checked_at, last_check_error_at, last_check_error FROM subscriptions WHERE series_id='existing-v2'`).Scan(&title, &last, &errorAt, &errorMessage); err != nil || title != "既存番組" || !last.Equal(checked) || errorAt != nil || errorMessage != nil {
t.Fatalf("v2データ保持と追加列: title=%q last=%v errorAt=%v error=%v err=%v", title, last, errorAt, errorMessage, err)
}
var indexDef string
if err := pool.QueryRow(ctx, `SELECT indexdef FROM pg_indexes WHERE schemaname=current_schema() AND indexname='subscriptions_due_idx'`).Scan(&indexDef); err != nil || !strings.Contains(indexDef, "last_checked_at") || !strings.Contains(indexDef, "WHERE enabled") {
t.Fatalf("期限抽出索引: %q %v", indexDef, err)
}
var versions int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM schema_migrations`).Scan(&versions); err != nil || versions != 3 {
t.Fatalf("移行履歴: %d %v", versions, err)
}
}
func TestDownloadJobIndexes(t *testing.T) {
pool := testPool(t, true)
rows, err := pool.Query(context.Background(), `SELECT indexname, indexdef FROM pg_indexes WHERE schemaname=current_schema() AND tablename='download_jobs'`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
indexes := make(map[string]string)
for rows.Next() {
var name, definition string
if err := rows.Scan(&name, &definition); err != nil {
t.Fatal(err)
}
indexes[name] = definition
}
if err := rows.Err(); err != nil {
t.Fatal(err)
}
checks := map[string][]string{
"download_jobs_episode_id_key": {"UNIQUE", "(episode_id)"},
"download_jobs_claim_idx": {"(created_at, id)", "status = 'queued'"},
"download_jobs_status_newest_idx": {"(status, created_at DESC, id DESC)"},
"download_jobs_stale_idx": {"(heartbeat_at)", "status = 'running'"},
}
for name, parts := range checks {
definition := indexes[name]
for _, part := range parts {
if !strings.Contains(definition, part) {
t.Errorf("索引 %s に %q がない: %q", name, part, definition)
}
}
}
}
func TestEnqueueAndListing(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
first, created, err := s.Enqueue(ctx, sampleEpisode("ep1"), "manual")
if err != nil || !created || first.Status != "queued" || first.Source != "manual" || first.DurationSeconds != 300 || first.EndAt == nil {
t.Fatalf("enqueue: %+v %v %v", first, created, err)
}
again, created, err := s.Enqueue(ctx, sampleEpisode("ep1"), "subscription")
if err != nil || created || again.ID != first.ID || again.Source != "manual" {
t.Fatalf("重複: %+v %v %v", again, created, err)
}
second, _, err := s.Enqueue(ctx, sampleEpisode("ep2"), "subscription")
if err != nil {
t.Fatal(err)
}
jobs, err := s.ListJobs(ctx, "queued", 10)
if err != nil || len(jobs) != 2 || jobs[0].ID != second.ID {
t.Fatalf("list: %+v %v", jobs, err)
}
statuses, err := s.EpisodeStatuses(ctx, []string{"ep1", "ep2", "absent"})
if err != nil || len(statuses) != 2 || statuses["ep1"] != "queued" {
t.Fatalf("statuses: %+v %v", statuses, err)
}
if _, err := s.ListJobs(ctx, "bad", 10); err == nil {
t.Fatal("不正な状態を許した")
}
if _, _, err := s.Enqueue(ctx, sampleEpisode("bad"), "invalid"); err == nil {
t.Fatal("source CHECK を許した")
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET phase='bad' WHERE id=$1`, first.ID); err == nil {
t.Fatal("phase CHECK を許した")
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET status='bad' WHERE id=$1`, first.ID); err == nil {
t.Fatal("status CHECK を許した")
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET error_kind='bad' WHERE id=$1`, first.ID); err == nil {
t.Fatal("error CHECK を許した")
}
}
func TestTransitions(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode("ep1"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "video"}); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued progress: %v", err)
}
if err := s.Complete(ctx, job.ID, "/file.mp4", 100); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued complete: %v", err)
}
if err := s.Fail(ctx, job.ID, "network", "failed"); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued fail: %v", err)
}
if err := s.Retry(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("queued retry: %v", err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil || claimed.ID != job.ID || claimed.Status != "running" || claimed.Attempts != 1 || claimed.HeartbeatAt == nil || claimed.StartedAt == nil {
t.Fatalf("claim: %+v %v", claimed, err)
}
oldHeartbeat := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, job.ID, oldHeartbeat); err != nil {
t.Fatal(err)
}
bytes := int64(123)
total := int64(1000)
speed := 40.5
eta := 12.0
fragment := int32(2)
fragments := int32(10)
format := "video-1"
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "video", DownloadedBytes: &bytes,
TotalBytes: &total, TotalIsEstimate: true, SpeedBPS: &speed, ETASeconds: &eta,
FragmentIndex: &fragment, FragmentCount: &fragments, FormatID: &format}); err != nil {
t.Fatal(err)
}
progressed, err := s.GetJob(ctx, job.ID)
if err != nil || progressed.Phase == nil || *progressed.Phase != "video" || progressed.DownloadedBytes == nil || *progressed.DownloadedBytes != bytes || progressed.TotalBytes == nil || *progressed.TotalBytes != total || !progressed.TotalIsEstimate || progressed.SpeedBPS == nil || *progressed.SpeedBPS != speed || progressed.ETASeconds == nil || *progressed.ETASeconds != eta || progressed.FragmentIndex == nil || *progressed.FragmentIndex != fragment || progressed.FragmentCount == nil || *progressed.FragmentCount != fragments || progressed.FormatID == nil || *progressed.FormatID != format || progressed.HeartbeatAt == nil || !progressed.HeartbeatAt.After(oldHeartbeat) {
t.Fatalf("progress: %+v %v", progressed, err)
}
if err := s.Fail(ctx, job.ID, "network", "connection lost"); err != nil {
t.Fatal(err)
}
failed, err := s.GetJob(ctx, job.ID)
if err != nil || failed.Status != "failed" || failed.ErrorKind == nil || *failed.ErrorKind != "network" || failed.ErrorMessage == nil || *failed.ErrorMessage != "connection lost" || failed.HeartbeatAt != nil || failed.FinishedAt == nil {
t.Fatalf("fail: %+v %v", failed, err)
}
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "audio"}); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("failed progress: %v", err)
}
if err := s.Retry(ctx, job.ID); err != nil {
t.Fatal(err)
}
retried, err := s.GetJob(ctx, job.ID)
if err != nil || retried.Status != "queued" || retried.Phase != nil || retried.ErrorKind != nil || retried.ErrorMessage != nil || retried.DownloadedBytes != nil || retried.TotalBytes != nil || retried.TotalIsEstimate || retried.SpeedBPS != nil || retried.ETASeconds != nil || retried.FragmentIndex != nil || retried.FragmentCount != nil || retried.FormatID != nil || retried.HeartbeatAt != nil || retried.StartedAt != nil || retried.FinishedAt != nil || retried.Attempts != 1 {
t.Fatalf("retry: %+v %v", retried, err)
}
claimed, err = s.ClaimNext(ctx)
if err != nil || claimed.Attempts != 2 {
t.Fatalf("reclaim: %+v %v", claimed, err)
}
if err := s.Complete(ctx, job.ID, "/file.mp4", 100); err != nil {
t.Fatal(err)
}
completed, err := s.GetJob(ctx, job.ID)
if err != nil || completed.Status != "succeeded" || completed.Phase != nil || completed.HeartbeatAt != nil || completed.FinishedAt == nil || completed.FilePath == nil || *completed.FilePath != "/file.mp4" || completed.FileSizeBytes == nil || *completed.FileSizeBytes != 100 {
t.Fatalf("complete: %+v %v", completed, err)
}
if err := s.Cancel(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("succeeded cancel: %v", err)
}
if err := s.Retry(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("succeeded retry: %v", err)
}
if _, err := s.ClaimNext(ctx); !errors.Is(err, pgx.ErrNoRows) {
t.Fatalf("empty queue: %v", err)
}
}
func TestCancelAndStaleBoundary(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
queued, _, err := s.Enqueue(ctx, sampleEpisode("queued"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.Cancel(ctx, queued.ID); err != nil {
t.Fatal(err)
}
canceled, err := s.GetJob(ctx, queued.ID)
if err != nil || canceled.Status != "canceled" || canceled.ErrorKind == nil || *canceled.ErrorKind != "canceled" {
t.Fatalf("cancel: %+v %v", canceled, err)
}
if err := s.Retry(ctx, queued.ID); err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
if err := s.Cancel(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
if err := s.Complete(ctx, claimed.ID, "/late.mp4", 1); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("canceled complete: %v", err)
}
if err := s.Retry(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
claimed, err = s.ClaimNext(ctx)
if err != nil || claimed.Attempts != 2 {
t.Fatalf("cancel retry claim: %+v %v", claimed, err)
}
cutoff := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, claimed.ID, cutoff); err != nil {
t.Fatal(err)
}
fresh, _, err := s.Enqueue(ctx, sampleEpisode("fresh"), "manual")
if err != nil {
t.Fatal(err)
}
if _, err := s.ClaimNext(ctx); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, fresh.ID, cutoff.Add(time.Hour)); err != nil {
t.Fatal(err)
}
n, err := s.RequeueStale(ctx, cutoff)
if err != nil || n != 0 {
t.Fatalf("境界上: %d %v", n, err)
}
n, err = s.RequeueStale(ctx, cutoff.Add(time.Microsecond))
if err != nil || n != 1 {
t.Fatalf("境界後: %d %v", n, err)
}
job, err := s.GetJob(ctx, claimed.ID)
if err != nil || job.Status != "queued" || job.HeartbeatAt != nil || job.Attempts != 2 {
t.Fatalf("requeue: %+v %v", job, err)
}
stillRunning, err := s.GetJob(ctx, fresh.ID)
if err != nil || stillRunning.Status != "running" || stillRunning.HeartbeatAt == nil || !stillRunning.HeartbeatAt.Equal(cutoff.Add(time.Hour)) {
t.Fatalf("新しい heartbeat を持つジョブ: %+v %v", stillRunning, err)
}
}
func TestHeartbeatAndGracefulRequeue(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode("heartbeat"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.Heartbeat(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("待機中の heartbeat: %v", err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
old := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
if _, err := s.pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, claimed.ID, old); err != nil {
t.Fatal(err)
}
if err := s.Heartbeat(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
heartbeated, err := s.GetJob(ctx, claimed.ID)
if err != nil || heartbeated.HeartbeatAt == nil || !heartbeated.HeartbeatAt.After(old) {
t.Fatalf("heartbeat: %+v %v", heartbeated, err)
}
bytes := int64(10)
if err := s.UpdateProgress(ctx, claimed.ID, Progress{Phase: "video", DownloadedBytes: &bytes}); err != nil {
t.Fatal(err)
}
if err := s.RequeueRunning(ctx, claimed.ID); err != nil {
t.Fatal(err)
}
requeued, err := s.GetJob(ctx, claimed.ID)
if err != nil || requeued.Status != "queued" || requeued.StartedAt != nil || requeued.HeartbeatAt != nil || requeued.Phase != nil || requeued.DownloadedBytes != nil || requeued.Attempts != 1 {
t.Fatalf("graceful requeue: %+v %v", requeued, err)
}
if err := s.RequeueRunning(ctx, claimed.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("待機中の再復帰: %v", err)
}
}
func TestRejectedTransitions(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
operations := map[string]func(int64) error{
"progress": func(id int64) error { return s.UpdateProgress(ctx, id, Progress{Phase: "video"}) },
"complete": func(id int64) error { return s.Complete(ctx, id, "/file.mp4", 1) },
"fail": func(id int64) error { return s.Fail(ctx, id, "network", "error") },
"cancel": func(id int64) error { return s.Cancel(ctx, id) },
"retry": func(id int64) error { return s.Retry(ctx, id) },
}
for status, rejected := range map[string][]string{
"queued": {"progress", "complete", "fail", "retry"},
"running": {"retry"},
"succeeded": {"progress", "complete", "fail", "cancel", "retry"},
"failed": {"progress", "complete", "fail", "cancel"},
"canceled": {"progress", "complete", "fail", "cancel"},
} {
t.Run(status, func(t *testing.T) {
job, _, err := s.Enqueue(ctx, sampleEpisode("rejected-"+status), "manual")
if err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET status=$2 WHERE id=$1`, job.ID, status); err != nil {
t.Fatal(err)
}
for _, name := range rejected {
if err := operations[name](job.ID); !errors.Is(err, ErrInvalidTransition) {
t.Errorf("%s on %s: %v", name, status, err)
}
}
got, err := s.GetJob(ctx, job.ID)
if err != nil || got.Status != status {
t.Fatalf("拒否後の状態: %+v %v", got, err)
}
})
}
}
func TestRetryEpisode(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
job, _, err := s.Enqueue(ctx, sampleEpisode("retry-by-episode"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.RetryEpisode(ctx, job.EpisodeID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("待機中の再試行: %v", err)
}
if err := s.Cancel(ctx, job.ID); err != nil {
t.Fatal(err)
}
if err := s.RetryEpisode(ctx, job.EpisodeID); err != nil {
t.Fatal(err)
}
got, err := s.GetJob(ctx, job.ID)
if err != nil || got.Status != "queued" || got.ErrorKind != nil || got.FinishedAt != nil {
t.Fatalf("エピソード ID による再試行: %+v %v", got, err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
downloaded, total := int64(10), int64(20)
format := "video-1"
if err := s.UpdateProgress(ctx, claimed.ID, Progress{
Phase: "video", DownloadedBytes: &downloaded, TotalBytes: &total,
TotalIsEstimate: true, FormatID: &format,
}); err != nil {
t.Fatal(err)
}
if err := s.Fail(ctx, claimed.ID, "network", "通信失敗"); err != nil {
t.Fatal(err)
}
if err := s.RetryEpisode(ctx, job.EpisodeID); err != nil {
t.Fatalf("失敗からの再試行: %v", err)
}
got, err = s.GetJob(ctx, job.ID)
if err != nil || got.Status != "queued" || got.Phase != nil || got.DownloadedBytes != nil || got.TotalBytes != nil || got.TotalIsEstimate || got.FormatID != nil || got.ErrorKind != nil || got.ErrorMessage != nil || got.HeartbeatAt != nil || got.StartedAt != nil || got.FinishedAt != nil || got.Attempts != 1 {
t.Fatalf("失敗情報の消去: %+v %v", got, err)
}
if err := s.RetryEpisode(ctx, "unknown"); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("存在しないエピソード: %v", err)
}
}
func TestCancelQueuedDoesNotCancelRunning(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
queued, _, err := s.Enqueue(ctx, sampleEpisode("queue-cancel"), "manual")
if err != nil {
t.Fatal(err)
}
if err := s.CancelQueued(ctx, queued.ID); err != nil {
t.Fatal(err)
}
canceled, err := s.GetJob(ctx, queued.ID)
if err != nil || canceled.Status != "canceled" || canceled.ErrorKind == nil || *canceled.ErrorKind != "canceled" || canceled.FinishedAt == nil {
t.Fatalf("待機中の取り消し: %+v %v", canceled, err)
}
if err := s.CancelQueued(ctx, queued.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("再取り消し: %v", err)
}
if err := s.CancelQueued(ctx, -1); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("存在しないジョブの取り消し: %v", err)
}
if err := s.Retry(ctx, queued.ID); err != nil {
t.Fatal(err)
}
claimed, err := s.ClaimNext(ctx)
if err != nil {
t.Fatal(err)
}
if err := s.CancelQueued(ctx, claimed.ID); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("実行中の取り消し: %v", err)
}
job, err := s.GetJob(ctx, claimed.ID)
if err != nil || job.Status != "running" {
t.Fatalf("実行中の状態: %+v %v", job, err)
}
}
func TestClaimNextConcurrent(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
const jobs = 12
for i := range jobs {
if _, _, err := s.Enqueue(ctx, sampleEpisode(fmt.Sprintf("ep%d", i)), "manual"); err != nil {
t.Fatal(err)
}
}
results := make(chan int64, jobs)
errCh := make(chan error, jobs)
var wg sync.WaitGroup
for range jobs {
wg.Add(1)
go func() {
defer wg.Done()
job, err := s.ClaimNext(ctx)
if err != nil {
errCh <- err
return
}
results <- job.ID
}()
}
wg.Wait()
close(results)
close(errCh)
for err := range errCh {
t.Error(err)
}
seen := make(map[int64]bool)
for id := range results {
if seen[id] {
t.Errorf("二重 claim: %d", id)
}
seen[id] = true
}
if len(seen) != jobs {
t.Fatalf("claim 数: %d", len(seen))
}
}
func TestClaimNextSkipsLockedOldest(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
oldest, _, err := s.Enqueue(ctx, sampleEpisode("oldest"), "manual")
if err != nil {
t.Fatal(err)
}
newer, _, err := s.Enqueue(ctx, sampleEpisode("newer"), "manual")
if err != nil {
t.Fatal(err)
}
base := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, oldest.ID, base); err != nil {
t.Fatal(err)
}
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, newer.ID, base.Add(time.Hour)); err != nil {
t.Fatal(err)
}
tx, err := pool.Begin(ctx)
if err != nil {
t.Fatal(err)
}
defer tx.Rollback(ctx)
if _, err := tx.Exec(ctx, `SELECT id FROM download_jobs WHERE id=$1 FOR UPDATE`, oldest.ID); err != nil {
t.Fatal(err)
}
claimCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
claimed, err := s.ClaimNext(claimCtx)
if err != nil || claimed.ID != newer.ID {
t.Fatalf("ロック中の最古ジョブを飛ばせない: %+v %v", claimed, err)
}
if err := tx.Rollback(ctx); err != nil {
t.Fatal(err)
}
claimed, err = s.ClaimNext(ctx)
if err != nil || claimed.ID != oldest.ID {
t.Fatalf("ロック解放後の最古ジョブ: %+v %v", claimed, err)
}
}
func TestListJobsFiltersAndLimit(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
ids := make([]int64, 3)
for i := range ids {
job, _, err := s.Enqueue(ctx, sampleEpisode(fmt.Sprintf("list-%d", i)), "manual")
if err != nil {
t.Fatal(err)
}
ids[i] = job.ID
created := time.Date(2026, 10, 3, i, 0, 0, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, job.ID, created); err != nil {
t.Fatal(err)
}
}
if err := s.Cancel(ctx, ids[1]); err != nil {
t.Fatal(err)
}
all, err := s.ListJobs(ctx, "", 2)
if err != nil || len(all) != 2 || all[0].ID != ids[2] || all[1].ID != ids[1] {
t.Fatalf("全件の新しい順と上限: %+v %v", all, err)
}
queued, err := s.ListJobs(ctx, "queued", 10)
if err != nil || len(queued) != 2 || queued[0].ID != ids[2] || queued[1].ID != ids[0] {
t.Fatalf("状態別一覧: %+v %v", queued, err)
}
canceled, err := s.ListJobs(ctx, "canceled", 10)
if err != nil || len(canceled) != 1 || canceled[0].ID != ids[1] {
t.Fatalf("中断済み一覧: %+v %v", canceled, err)
}
if _, err := s.ListJobs(ctx, "", 0); err == nil {
t.Fatal("0 件の上限を許した")
}
statuses, err := s.EpisodeStatuses(ctx, []string{"list-0", "list-1", "absent"})
if err != nil || len(statuses) != 2 || statuses["list-0"] != "queued" || statuses["list-1"] != "canceled" {
t.Fatalf("状態の一括取得: %+v %v", statuses, err)
}
}
func TestSubscriptions(t *testing.T) {
s := New(testPool(t, true))
ctx := context.Background()
created, err := s.RegisterSubscription(ctx, "series", "番組", nil, false)
if err != nil || !created {
t.Fatalf("add: %v %v", created, err)
}
created, err = s.RegisterSubscription(ctx, "series", "別タイトル", nil, false)
if err != nil || created {
t.Fatalf("duplicate: %v %v", created, err)
}
when := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
if err := s.SetSubscriptionChecked(ctx, "series", when); err != nil {
t.Fatal(err)
}
subs, err := s.ListSubscriptions(ctx)
if err != nil || len(subs) != 1 || subs[0].SeriesTitle != "番組" || !subs[0].Enabled || subs[0].LastCheckedAt == nil || !subs[0].LastCheckedAt.Equal(when) {
t.Fatalf("list: %+v %v", subs, err)
}
deleted, err := s.DeleteSubscription(ctx, "series")
if err != nil || !deleted {
t.Fatalf("delete: %v %v", deleted, err)
}
deleted, err = s.DeleteSubscription(ctx, "series")
if err != nil || deleted {
t.Fatalf("duplicate delete: %v %v", deleted, err)
}
if err := s.SetSubscriptionChecked(ctx, "series", when); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("missing: %v", err)
}
}
internal/store/subscriptions.go
+244
View File
@@ -0,0 +1,244 @@
package store
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5"
)
var ErrSubscriptionPaused = errors.New("subscription is paused")
// RegisterSubscription は登録と公開中の回の記録・任意の投入を一つの取引で行う。
func (s *Store) RegisterSubscription(ctx context.Context, seriesID, title string, episodes []Episode, downloadExisting bool) (bool, error) {
if seriesID == "" || title == "" {
return false, errors.New("series id and title are required")
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return false, err
}
defer tx.Rollback(ctx)
var inserted string
err = tx.QueryRow(ctx, `INSERT INTO subscriptions (series_id, series_title, last_checked_at)
VALUES ($1,$2,now()) ON CONFLICT (series_id) DO NOTHING RETURNING series_id`, seriesID, title).Scan(&inserted)
if errors.Is(err, pgx.ErrNoRows) {
return false, nil
}
if err != nil {
return false, err
}
if _, err := addKnownAndMaybeQueue(ctx, tx, seriesID, episodes, downloadExisting); err != nil {
return false, err
}
if err := tx.Commit(ctx); err != nil {
return false, err
}
return true, nil
}
// QueueUnseenEpisodes は未確認の回だけを既知に記録し、同じ取引で待ち行列へ入れる。
func (s *Store) QueueUnseenEpisodes(ctx context.Context, seriesID string, episodes []Episode) (int, error) {
tx, err := s.pool.Begin(ctx)
if err != nil {
return 0, err
}
defer tx.Rollback(ctx)
var enabled bool
err = tx.QueryRow(ctx, `SELECT enabled FROM subscriptions WHERE series_id=$1 FOR UPDATE`, seriesID).Scan(&enabled)
if errors.Is(err, pgx.ErrNoRows) {
return 0, ErrInvalidTransition
}
if err != nil {
return 0, err
}
if !enabled {
return 0, ErrSubscriptionPaused
}
count, err := addKnownAndMaybeQueue(ctx, tx, seriesID, episodes, true)
if err != nil {
return 0, err
}
if err := tx.Commit(ctx); err != nil {
return 0, err
}
return count, nil
}
// ListDueSubscriptions は有効で、cutoff 以前に確認された（または未確認の）番組を返す。
func (s *Store) ListDueSubscriptions(ctx context.Context, cutoff time.Time) ([]Subscription, error) {
rows, err := s.pool.Query(ctx, `SELECT series_id, series_title, enabled, created_at, last_checked_at,
last_check_error_at, last_check_error, 0::bigint
FROM subscriptions
WHERE enabled AND (last_checked_at IS NULL OR last_checked_at <= $1)
ORDER BY last_checked_at ASC NULLS FIRST, created_at, series_id`, cutoff)
if err != nil {
return nil, err
}
defer rows.Close()
var result []Subscription
for rows.Next() {
var sub Subscription
if err := rows.Scan(&sub.SeriesID, &sub.SeriesTitle, &sub.Enabled, &sub.CreatedAt,
&sub.LastCheckedAt, &sub.LastCheckErrorAt, &sub.LastCheckError, &sub.JobCount); err != nil {
return nil, err
}
result = append(result, sub)
}
return result, rows.Err()
}
// CompleteSubscriptionCheck は全公開回を既知にし、しきい値を超える未確認回だけを投入する。
// 最終確認日時の更新と前回エラーの消去も同じ取引で行う。
func (s *Store) CompleteSubscriptionCheck(ctx context.Context, seriesID string, episodes []Episode, previewMaxSeconds int, checkedAt time.Time) (int, error) {
if previewMaxSeconds < 0 {
return 0, errors.New("preview threshold must not be negative")
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return 0, err
}
defer tx.Rollback(ctx)
var enabled bool
err = tx.QueryRow(ctx, `SELECT enabled FROM subscriptions WHERE series_id=$1 FOR UPDATE`, seriesID).Scan(&enabled)
if errors.Is(err, pgx.ErrNoRows) {
return 0, ErrInvalidTransition
}
if err != nil {
return 0, err
}
if !enabled {
return 0, ErrSubscriptionPaused
}
queued := 0
for _, ep := range episodes {
if ep.ID == "" || ep.SeriesID != seriesID || ep.DurationSeconds < 0 {
return 0, fmt.Errorf("invalid episode identity for series %s", seriesID)
}
var inserted string
err := tx.QueryRow(ctx, `INSERT INTO known_episodes (series_id, episode_id)
VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING episode_id`, seriesID, ep.ID).Scan(&inserted)
if errors.Is(err, pgx.ErrNoRows) {
continue
}
if err != nil {
return 0, err
}
if ep.DurationSeconds <= previewMaxSeconds {
continue
}
tag, err := tx.Exec(ctx, `INSERT INTO download_jobs
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,'subscription')
ON CONFLICT (episode_id) DO NOTHING`, ep.ID, seriesID, ep.SeriesTitle, ep.Title,
ep.BroadcasterName, ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath)
if err != nil {
return 0, err
}
queued += int(tag.RowsAffected())
}
if _, err := tx.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2,
last_check_error_at=NULL, last_check_error=NULL WHERE series_id=$1`, seriesID, checkedAt); err != nil {
return 0, err
}
if err := tx.Commit(ctx); err != nil {
return 0, err
}
return queued, nil
}
// RecordSubscriptionCheckFailure は取得失敗を記録し、最終確認日時は変更しない。
func (s *Store) RecordSubscriptionCheckFailure(ctx context.Context, seriesID string, failedAt time.Time, message string) error {
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET last_check_error_at=$2, last_check_error=$3
WHERE series_id=$1 AND enabled`, seriesID, failedAt, message)
return transitionResult(tag.RowsAffected(), err)
}
func addKnownAndMaybeQueue(ctx context.Context, tx pgx.Tx, seriesID string, episodes []Episode, queue bool) (int, error) {
count := 0
for _, ep := range episodes {
if ep.ID == "" || ep.SeriesID != seriesID {
return 0, fmt.Errorf("invalid episode identity for series %s", seriesID)
}
var inserted string
err := tx.QueryRow(ctx, `INSERT INTO known_episodes (series_id, episode_id)
VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING episode_id`, seriesID, ep.ID).Scan(&inserted)
if errors.Is(err, pgx.ErrNoRows) {
continue
}
if err != nil {
return 0, err
}
count++
if queue {
_, err := tx.Exec(ctx, `INSERT INTO download_jobs
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,'subscription')
ON CONFLICT (episode_id) DO NOTHING`, ep.ID, seriesID, ep.SeriesTitle, ep.Title,
ep.BroadcasterName, ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath)
if err != nil {
return 0, err
}
}
}
return count, nil
}
// UnknownEpisodeIDs は指定シリーズの未確認のエピソード ID を入力順で返す。
func (s *Store) UnknownEpisodeIDs(ctx context.Context, seriesID string, ids []string) ([]string, error) {
if len(ids) == 0 {
return []string{}, nil
}
rows, err := s.pool.Query(ctx, `SELECT episode_id FROM known_episodes WHERE series_id=$1 AND episode_id=ANY($2)`, seriesID, ids)
if err != nil {
return nil, err
}
defer rows.Close()
known := make(map[string]bool)
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
known[id] = true
}
if err := rows.Err(); err != nil {
return nil, err
}
var unknown []string
emitted := make(map[string]bool)
for _, id := range ids {
if !known[id] && !emitted[id] {
unknown = append(unknown, id)
emitted[id] = true
}
}
return unknown, nil
}
func (s *Store) SetSubscriptionEnabled(ctx context.Context, seriesID string, enabled bool) error {
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET enabled=$2 WHERE series_id=$1`, seriesID, enabled)
return transitionResult(tag.RowsAffected(), err)
}
func (s *Store) SubscribedSeries(ctx context.Context, ids []string) (map[string]bool, error) {
result := make(map[string]bool)
if len(ids) == 0 {
return result, nil
}
rows, err := s.pool.Query(ctx, `SELECT series_id FROM subscriptions WHERE series_id=ANY($1)`, ids)
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
result[id] = true
}
return result, rows.Err()
}
internal/store/subscriptions_test.go
+326
View File
@@ -0,0 +1,326 @@
package store
import (
"context"
"errors"
"sync"
"testing"
"time"
)
func TestListDueSubscriptionsBoundaryAndPaused(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
for _, id := range []string{"null", "old", "boundary", "fresh", "paused"} {
if created, err := s.RegisterSubscription(ctx, id, "番組 "+id, nil, false); err != nil || !created {
t.Fatalf("登録 %s: %t %v", id, created, err)
}
}
updates := map[string]*time.Time{
"null": nil, "old": timePointer(now.Add(-25 * time.Hour)),
"boundary": timePointer(now.Add(-24 * time.Hour)), "fresh": timePointer(now.Add(-24*time.Hour + time.Microsecond)),
"paused": timePointer(now.Add(-48 * time.Hour)),
}
for id, checked := range updates {
if _, err := pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, id, checked); err != nil {
t.Fatal(err)
}
}
if err := s.SetSubscriptionEnabled(ctx, "paused", false); err != nil {
t.Fatal(err)
}
due, err := s.ListDueSubscriptions(ctx, now.Add(-24*time.Hour))
if err != nil {
t.Fatal(err)
}
if len(due) != 3 || due[0].SeriesID != "null" || due[1].SeriesID != "old" || due[2].SeriesID != "boundary" {
t.Fatalf("24時間境界と一時停止: %+v", due)
}
}
func TestCompleteSubscriptionCheckKnownPreviewErrorAndAtomicity(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
if created, err := s.RegisterSubscription(ctx, "series-check", "番組", []Episode{seriesEpisode("known", "series-check")}, false); err != nil || !created {
t.Fatalf("登録: %t %v", created, err)
}
before := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
failedAt := time.Date(2026, 10, 3, 1, 2, 3, 0, time.UTC)
if _, err := pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, "series-check", before); err != nil {
t.Fatal(err)
}
if err := s.RecordSubscriptionCheckFailure(ctx, "series-check", failedAt, "network token=[REDACTED]"); err != nil {
t.Fatal(err)
}
var unchanged time.Time
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id='series-check'`).Scan(&unchanged); err != nil || !unchanged.Equal(before) {
t.Fatalf("失敗時に最終確認を更新: %v %v", unchanged, err)
}
checkedAt := time.Date(2026, 10, 3, 4, 5, 6, 0, time.UTC)
preview := seriesEpisode("preview", "series-check")
preview.DurationSeconds = 180
normal := seriesEpisode("normal", "series-check")
normal.DurationSeconds = 181
count, err := s.CompleteSubscriptionCheck(ctx, "series-check", []Episode{
seriesEpisode("known", "series-check"), preview, normal,
}, 180, checkedAt)
if err != nil || count != 1 {
t.Fatalf("確認完了: count=%d err=%v", count, err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "series-check", []string{"known", "preview", "normal"})
if err != nil || len(unknown) != 0 {
t.Fatalf("予告を含む既知化: %v %v", unknown, err)
}
var jobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-check'`).Scan(&jobs); err != nil || jobs != 1 {
t.Fatalf("予告の投入除外: %d %v", jobs, err)
}
var last time.Time
var errorAt, errorMessage any
if err := pool.QueryRow(ctx, `SELECT last_checked_at, last_check_error_at, last_check_error FROM subscriptions WHERE series_id='series-check'`).Scan(&last, &errorAt, &errorMessage); err != nil || !last.Equal(checkedAt) || errorAt != nil || errorMessage != nil {
t.Fatalf("成功時の日時・エラー消去: %v %v %v %v", last, errorAt, errorMessage, err)
}
bad := seriesEpisode("bad", "series-check")
bad.DurationSeconds = -1
if _, err := s.CompleteSubscriptionCheck(ctx, "series-check", []Episode{seriesEpisode("rollback", "series-check"), bad}, 180, checkedAt.Add(time.Hour)); err == nil {
t.Fatal("不正な回を含む確認が成功した")
}
unknown, err = s.UnknownEpisodeIDs(ctx, "series-check", []string{"rollback"})
if err != nil || len(unknown) != 1 {
t.Fatalf("失敗時の既知記録を巻き戻さない: %v %v", unknown, err)
}
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id='series-check'`).Scan(&last); err != nil || !last.Equal(checkedAt) {
t.Fatalf("失敗時の最終確認を巻き戻さない: %v %v", last, err)
}
if err := s.SetSubscriptionEnabled(ctx, "series-check", false); err != nil {
t.Fatal(err)
}
if _, err := s.CompleteSubscriptionCheck(ctx, "series-check", nil, 180, checkedAt.Add(time.Hour)); !errors.Is(err, ErrSubscriptionPaused) {
t.Fatalf("一時停止中の確認完了: %v", err)
}
if err := s.RecordSubscriptionCheckFailure(ctx, "series-check", failedAt, "ignored"); !errors.Is(err, ErrInvalidTransition) {
t.Fatalf("一時停止中のエラー記録: %v", err)
}
}
func TestCompleteSubscriptionCheckKeepsExistingManualJob(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
if created, err := s.RegisterSubscription(ctx, "series-manual", "番組", nil, false); err != nil || !created {
t.Fatalf("登録: %t %v", created, err)
}
episode := seriesEpisode("manual-existing", "series-manual")
manual, _, err := s.Enqueue(ctx, episode, "manual")
if err != nil {
t.Fatal(err)
}
checkedAt := time.Date(2026, 10, 3, 4, 5, 6, 0, time.UTC)
queued, err := s.CompleteSubscriptionCheck(ctx, "series-manual", []Episode{episode, episode}, 180, checkedAt)
if err != nil || queued != 0 {
t.Fatalf("既存手動ジョブの確認: queued=%d err=%v", queued, err)
}
got, err := s.GetJob(ctx, manual.ID)
if err != nil || got.Source != "manual" {
t.Fatalf("既存手動ジョブを変更した: %+v %v", got, err)
}
var jobs, known int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE episode_id=$1`, episode.ID).Scan(&jobs); err != nil || jobs != 1 {
t.Fatalf("ジョブが二重化した: %d %v", jobs, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id=$1 AND episode_id=$2`, episode.SeriesID, episode.ID).Scan(&known); err != nil || known != 1 {
t.Fatalf("既知記録: %d %v", known, err)
}
var last time.Time
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id=$1`, episode.SeriesID).Scan(&last); err != nil || !last.Equal(checkedAt) {
t.Fatalf("最終確認日時: %v %v", last, err)
}
}
func timePointer(value time.Time) *time.Time { return &value }
func seriesEpisode(id, seriesID string) Episode {
ep := sampleEpisode(id)
ep.SeriesID = seriesID
return ep
}
func TestRegisterSubscriptionSnapshotAndRollback(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
episodes := []Episode{seriesEpisode("ep1", "series-1"), seriesEpisode("ep2", "series-1")}
created, err := s.RegisterSubscription(ctx, "series-1", "番組", episodes, false)
if err != nil || !created {
t.Fatalf("既存回をダウンロードしない登録: %t %v", created, err)
}
subs, err := s.ListSubscriptions(ctx)
if err != nil || len(subs) != 1 || subs[0].LastCheckedAt == nil || subs[0].LastCheckedAt.Before(subs[0].CreatedAt) {
t.Fatalf("登録時の最終確認日時: %+v %v", subs, err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "series-1", []string{"ep1", "ep2", "ep3"})
if err != nil || len(unknown) != 1 || unknown[0] != "ep3" {
t.Fatalf("既知の記録: %v %v", unknown, err)
}
var jobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
t.Fatalf("既存回が投入された: %d %v", jobs, err)
}
created, err = s.RegisterSubscription(ctx, "series-1", "別の名前", episodes, true)
if err != nil || created {
t.Fatalf("重複登録: %t %v", created, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
t.Fatalf("重複登録が投入した: %d %v", jobs, err)
}
bad := seriesEpisode("bad", "series-bad")
bad.DurationSeconds = -1
created, err = s.RegisterSubscription(ctx, "series-bad", "番組", []Episode{seriesEpisode("good", "series-bad"), bad}, true)
if err == nil || created {
t.Fatalf("壊れた回の登録が成功: %t %v", created, err)
}
for _, table := range []string{"subscriptions", "known_episodes", "download_jobs"} {
var count int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM `+table+` WHERE series_id='series-bad'`).Scan(&count); err != nil || count != 0 {
t.Errorf("取引の巻き戻し %s: %d %v", table, count, err)
}
}
deleted, err := s.DeleteSubscription(ctx, "series-1")
if err != nil || !deleted {
t.Fatalf("解除: %t %v", deleted, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
t.Fatalf("解除後に既知が残った: %d %v", jobs, err)
}
created, err = s.RegisterSubscription(ctx, "series-1", "番組", episodes, true)
if err != nil || !created {
t.Fatalf("解除後の再登録: %t %v", created, err)
}
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 2 {
t.Fatalf("再登録で既存回を選び直せない: %d %v", jobs, err)
}
}
func TestRegisterAndQueueUnseenEpisodes(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
manual, _, err := s.Enqueue(ctx, seriesEpisode("manual", "series-2"), "manual")
if err != nil {
t.Fatal(err)
}
created, err := s.RegisterSubscription(ctx, "series-2", "番組", []Episode{seriesEpisode("manual", "series-2"), seriesEpisode("past", "series-2")}, true)
if err != nil || !created {
t.Fatalf("公開中の回を投入: %t %v", created, err)
}
manualAfter, err := s.GetJob(ctx, manual.ID)
if err != nil || manualAfter.Source != "manual" {
t.Fatalf("既存の手動ジョブを変更: %+v %v", manualAfter, err)
}
var subscriptionJobs int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-2' AND source='subscription' AND status='queued'`).Scan(&subscriptionJobs); err != nil || subscriptionJobs != 1 {
t.Fatalf("登録時に作ったジョブ: %d %v", subscriptionJobs, err)
}
subs, err := s.ListSubscriptions(ctx)
if err != nil || len(subs) != 1 || subs[0].JobCount != 2 || !subs[0].Enabled || subs[0].LastCheckedAt == nil {
t.Fatalf("登録一覧と関連ジョブ数: %+v %v", subs, err)
}
unknown, err := s.UnknownEpisodeIDs(ctx, "series-2", []string{"past", "new", "manual", "new"})
if err != nil || len(unknown) != 1 || unknown[0] != "new" {
t.Fatalf("未確認の抽出: %v %v", unknown, err)
}
if err := s.SetSubscriptionEnabled(ctx, "series-2", false); err != nil {
t.Fatal(err)
}
if _, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("new", "series-2")}); !errors.Is(err, ErrSubscriptionPaused) {
t.Fatalf("一時停止中の投入: %v", err)
}
unknown, err = s.UnknownEpisodeIDs(ctx, "series-2", []string{"new"})
if err != nil || len(unknown) != 1 {
t.Fatalf("一時停止中に既知となった: %v %v", unknown, err)
}
if err := s.SetSubscriptionEnabled(ctx, "series-2", true); err != nil {
t.Fatal(err)
}
invalid := seriesEpisode("invalid", "series-2")
invalid.DurationSeconds = -1
if _, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("newer", "series-2"), invalid}); err == nil {
t.Fatal("新規回の投入が途中失敗でも確定した")
}
unknown, err = s.UnknownEpisodeIDs(ctx, "series-2", []string{"newer"})
if err != nil || len(unknown) != 1 {
t.Fatalf("新規回の投入失敗が巻き戻らない: %v %v", unknown, err)
}
count, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("past", "series-2"), seriesEpisode("new", "series-2")})
if err != nil || count != 1 {
t.Fatalf("再開後の新しい回: %d %v", count, err)
}
count, err = s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("new", "series-2")})
if err != nil || count != 0 {
t.Fatalf("重複した新しい回: %d %v", count, err)
}
var source string
if err := pool.QueryRow(ctx, `SELECT source FROM download_jobs WHERE episode_id='new'`).Scan(&source); err != nil || source != "subscription" {
t.Fatalf("新しい回の登録元: %q %v", source, err)
}
statuses, err := s.SubscribedSeries(ctx, []string{"series-2", "missing"})
if err != nil || !statuses["series-2"] || statuses["missing"] {
t.Fatalf("登録済みシリーズの一括取得: %v %v", statuses, err)
}
}
func TestQueueUnseenEpisodesConcurrent(t *testing.T) {
pool := testPool(t, true)
s := New(pool)
ctx := context.Background()
created, err := s.RegisterSubscription(ctx, "series-concurrent", "番組", nil, false)
if err != nil || !created {
t.Fatalf("登録: %t %v", created, err)
}
episodes := []Episode{
seriesEpisode("concurrent-1", "series-concurrent"),
seriesEpisode("concurrent-2", "series-concurrent"),
}
const workers = 8
counts := make(chan int, workers)
errs := make(chan error, workers)
var wg sync.WaitGroup
for range workers {
wg.Add(1)
go func() {
defer wg.Done()
count, err := s.QueueUnseenEpisodes(ctx, "series-concurrent", episodes)
counts <- count
errs <- err
}()
}
wg.Wait()
close(counts)
close(errs)
for err := range errs {
if err != nil {
t.Errorf("同時投入: %v", err)
}
}
total := 0
positive := 0
for count := range counts {
total += count
if count > 0 {
positive++
}
}
if total != 2 || positive != 1 {
t.Fatalf("同時投入の記録件数: total=%d positive workers=%d", total, positive)
}
for table := range map[string]struct{}{"known_episodes": {}, "download_jobs": {}} {
var count int
if err := pool.QueryRow(ctx, `SELECT count(*) FROM `+table+` WHERE series_id='series-concurrent'`).Scan(&count); err != nil || count != 2 {
t.Errorf("%s の重複防止: %d %v", table, count, err)
}
}
}
internal/tver/client.go
+163
View File
@@ -0,0 +1,163 @@
package tver
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
)
const (
DefaultBaseURL = "https://platform-api.tver.jp"
PreviewMaxSeconds = 180
)
type Session struct {
PlatformUID string `json:"platform_uid"`
PlatformToken string `json:"platform_token"`
}
type Episode struct {
ID string `json:"id"`
SeriesID string `json:"seriesID"`
SeriesTitle string `json:"seriesTitle"`
Title string `json:"title"`
BroadcasterName string `json:"broadcasterName"`
Duration int64 `json:"duration"`
EndAt int64 `json:"endAt"`
ThumbnailPath string `json:"thumbnailPath"`
BroadcastDateLabel string `json:"broadcastDateLabel"`
}
type Client struct {
baseURL *url.URL
serviceURL *url.URL
httpClient *http.Client
}
// NewClient はテスト用に API の接続先と HTTP クライアントを差し替えられる。
// 接続先が空、またはクライアントが nil の場合は本番用の既定値を使う。
func NewClient(baseURL string, httpClient *http.Client) (*Client, error) {
defaultAPI := baseURL == ""
if baseURL == "" {
baseURL = DefaultBaseURL
}
parsed, err := url.Parse(baseURL)
if err != nil || (parsed.Scheme != "https" && parsed.Scheme != "http") || parsed.Host == "" || parsed.RawQuery != "" || parsed.Fragment != "" {
return nil, fmt.Errorf("invalid TVer API base URL")
}
if httpClient == nil {
httpClient = &http.Client{Timeout: 15 * time.Second}
}
service := parsed
if defaultAPI {
service = &url.URL{Scheme: "https", Host: "service-api.tver.jp"}
}
return &Client{baseURL: parsed, serviceURL: service, httpClient: httpClient}, nil
}
func (c *Client) CreateSession(ctx context.Context) (Session, error) {
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/v2/api/platform_users/browser/create"})
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint.String(), strings.NewReader("device_type=pc"))
if err != nil {
return Session{}, fmt.Errorf("create session request: %w", err)
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
var response struct {
Code int `json:"code"`
Message string `json:"message"`
Result Session `json:"result"`
}
if err := c.getJSON(req, &response); err != nil {
return Session{}, fmt.Errorf("create session: %w", err)
}
if response.Code != 0 || response.Result.PlatformUID == "" || response.Result.PlatformToken == "" {
return Session{}, fmt.Errorf("create session: invalid API result (code %d)", response.Code)
}
return response.Result, nil
}
func (c *Client) ListNewer(ctx context.Context, category string, session Session) ([]Episode, error) {
if category != "drama" && category != "variety" {
return nil, fmt.Errorf("unsupported category %q", category)
}
if session.PlatformUID == "" || session.PlatformToken == "" {
return nil, errors.New("missing TVer session credentials")
}
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v1/callNewerDetail/" + category})
query := endpoint.Query()
query.Set("platform_uid", session.PlatformUID)
query.Set("platform_token", session.PlatformToken)
endpoint.RawQuery = query.Encode()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
if err != nil {
return nil, fmt.Errorf("newer list request: %w", err)
}
req.Header.Set("x-tver-platform-type", "web")
req.Header.Set("Origin", "https://tver.jp")
req.Header.Set("Referer", "https://tver.jp/")
var response struct {
Code int `json:"code"`
Result struct {
Contents *struct {
Contents []struct {
Type string `json:"type"`
Content Episode `json:"content"`
} `json:"contents"`
} `json:"contents"`
} `json:"result"`
}
if err := c.getJSON(req, &response); err != nil {
return nil, fmt.Errorf("newer list: %w", err)
}
if response.Code != 0 {
return nil, fmt.Errorf("newer list: API code %d", response.Code)
}
if response.Result.Contents == nil || response.Result.Contents.Contents == nil {
return nil, errors.New("newer list: missing contents")
}
var episodes []Episode
for _, item := range response.Result.Contents.Contents {
if item.Type != "episode" {
continue
}
if item.Content.ID == "" {
return nil, errors.New("newer list: episode without id")
}
episodes = append(episodes, item.Content)
}
return episodes, nil
}
func (c *Client) FetchNewer(ctx context.Context, category string) ([]Episode, error) {
session, err := c.CreateSession(ctx)
if err != nil {
return nil, err
}
return c.ListNewer(ctx, category, session)
}
func (c *Client) getJSON(req *http.Request, dst any) error {
response, err := c.httpClient.Do(req)
if err != nil {
// url.Error には platform_token を含むリクエスト URL が入るため、内部のエラーだけを返す。
var urlErr *url.Error
if errors.As(err, &urlErr) {
err = urlErr.Err
}
return fmt.Errorf("request failed: %w", err)
}
defer response.Body.Close()
if response.StatusCode < 200 || response.StatusCode >= 300 {
return fmt.Errorf("HTTP %d", response.StatusCode)
}
if err := json.NewDecoder(io.LimitReader(response.Body, 8<<20)).Decode(dst); err != nil {
return fmt.Errorf("decode JSON: %w", err)
}
return nil
}
internal/tver/client_test.go
+213
View File
@@ -0,0 +1,213 @@
package tver
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"net/url"
"os"
"strings"
"testing"
)
type localOnlyTransport struct {
host string
base http.RoundTripper
}
func (tr localOnlyTransport) RoundTrip(req *http.Request) (*http.Response, error) {
if req.URL.Scheme != "http" || req.URL.Host != tr.host {
return nil, fmt.Errorf("test attempted external request to %s", req.URL.Host)
}
return tr.base.RoundTrip(req)
}
func testClient(t *testing.T, handler http.HandlerFunc) *Client {
t.Helper()
server := httptest.NewServer(handler)
t.Cleanup(server.Close)
serverURL, err := url.Parse(server.URL)
if err != nil {
t.Fatal(err)
}
httpClient := server.Client()
baseTransport := httpClient.Transport
if baseTransport == nil {
baseTransport = http.DefaultTransport
}
httpClient.Transport = localOnlyTransport{host: serverURL.Host, base: baseTransport}
client, err := NewClient(server.URL, httpClient)
if err != nil {
t.Fatal(err)
}
return client
}
func TestTestClientRejectsExternalRequests(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
t.Errorf("unexpected local request to %s", r.URL)
})
_, err := client.httpClient.Get(DefaultBaseURL)
if err == nil || !strings.Contains(err.Error(), "test attempted external request") {
t.Errorf("external request error = %v", err)
}
}
func TestFetchNewerFromRecordedResponse(t *testing.T) {
fixture, err := os.ReadFile("testdata/newer_drama.json")
if err != nil {
t.Fatal(err)
}
var sessionCalls, newerCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v2/api/platform_users/browser/create":
sessionCalls++
if r.Method != http.MethodPost || r.Header.Get("Content-Type") != "application/x-www-form-urlencoded" {
t.Errorf("session request method/Content-Type: %s %s", r.Method, r.Header.Get("Content-Type"))
}
if err := r.ParseForm(); err != nil || r.PostForm.Get("device_type") != "pc" {
t.Errorf("session form: %v, %v", r.PostForm, err)
}
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
case "/service/api/v1/callNewerDetail/drama":
newerCalls++
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" {
t.Errorf("newer request method/query: %s %v", r.Method, r.URL.Query())
}
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
if got := r.Header.Get(name); got != want {
t.Errorf("header %s = %q; want %q", name, got, want)
}
}
_, _ = w.Write(fixture)
default:
t.Errorf("unexpected path %s", r.URL.Path)
http.NotFound(w, r)
}
})
episodes, err := client.FetchNewer(context.Background(), "drama")
if err != nil {
t.Fatal(err)
}
if sessionCalls != 1 || newerCalls != 1 {
t.Fatalf("calls: session=%d newer=%d", sessionCalls, newerCalls)
}
if len(episodes) != 3 {
t.Fatalf("episodes = %d; want 3", len(episodes))
}
want := Episode{
ID: "eps0z5761a",
SeriesID: "sr0lhdrtdb",
SeriesTitle: "韓国ドラマ「良いが悪い、ドンジェ」",
Title: "#10（最終話）",
BroadcasterName: "BS12トゥエルビ",
Duration: 2712,
EndAt: 1792209540,
ThumbnailPath: "/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51",
BroadcastDateLabel: "10月3日(土)放送分",
}
if episodes[0] != want {
t.Errorf("first episode = %+v; want %+v", episodes[0], want)
}
if episodes[2].Duration != 60 || episodes[2].BroadcastDateLabel != "" {
t.Errorf("third episode = %+v", episodes[2])
}
}
func TestListNewerVarietySkipsOtherTypes(t *testing.T) {
var calls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
calls++
if r.URL.Path != "/service/api/v1/callNewerDetail/variety" {
t.Errorf("path = %s", r.URL.Path)
}
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "u" || r.URL.Query().Get("platform_token") != "t" {
t.Errorf("variety request method/query: %s %v", r.Method, r.URL.Query())
}
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
if got := r.Header.Get(name); got != want {
t.Errorf("variety header %s = %q; want %q", name, got, want)
}
}
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":{"contents":[{"type":"series","content":{"id":"series-1"}},{"type":"episode","content":{"id":"episode-1","duration":42}}]}}}`))
})
episodes, err := client.ListNewer(context.Background(), "variety", Session{PlatformUID: "u", PlatformToken: "t"})
if err != nil {
t.Fatal(err)
}
if len(episodes) != 1 || episodes[0].ID != "episode-1" || episodes[0].Duration != 42 {
t.Errorf("episodes = %+v", episodes)
}
if calls != 1 {
t.Errorf("variety calls = %d; want 1", calls)
}
}
func TestFetchNewerErrors(t *testing.T) {
tests := []struct {
name string
sessionStatus int
sessionBody string
newerStatus int
newerBody string
want string
}{
{"session HTTP error", 503, "", 200, "", "create session: HTTP 503"},
{"session invalid JSON", 200, "{", 200, "", "create session: decode JSON"},
{"session API error", 200, `{"code":7,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, "", "create session: invalid API result"},
{"session missing credentials", 200, `{"code":0,"result":{}}`, 200, "", "create session: invalid API result"},
{"session missing token", 200, `{"code":0,"result":{"platform_uid":"u"}}`, 200, "", "create session: invalid API result"},
{"session missing uid", 200, `{"code":0,"result":{"platform_token":"t"}}`, 200, "", "create session: invalid API result"},
{"newer HTTP error", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 502, "", "newer list: HTTP 502"},
{"newer invalid JSON", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, "{", "newer list: decode JSON"},
{"newer API error", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":7}`, "newer list: API code 7"},
{"newer missing contents", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{}}`, "newer list: missing contents"},
{"newer missing items", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{"contents":{}}}`, "newer list: missing contents"},
{"episode missing id", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{"contents":{"contents":[{"type":"episode","content":{"title":"untitled"}}]}}}`, "newer list: episode without id"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var sessionCalls, newerCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v2/api/platform_users/browser/create":
sessionCalls++
w.WriteHeader(tt.sessionStatus)
_, _ = w.Write([]byte(tt.sessionBody))
case "/service/api/v1/callNewerDetail/drama":
newerCalls++
w.WriteHeader(tt.newerStatus)
_, _ = w.Write([]byte(tt.newerBody))
default:
t.Errorf("unexpected path %s", r.URL.Path)
http.NotFound(w, r)
}
})
_, err := client.FetchNewer(context.Background(), "drama")
if err == nil || !strings.Contains(err.Error(), tt.want) {
t.Errorf("error = %v; want %q", err, tt.want)
}
wantNewerCalls := 1
if strings.HasPrefix(tt.name, "session ") {
wantNewerCalls = 0
}
if sessionCalls != 1 || newerCalls != wantNewerCalls {
t.Errorf("calls: session=%d newer=%d; want session=1 newer=%d", sessionCalls, newerCalls, wantNewerCalls)
}
})
}
}
func TestListNewerRejectsInvalidInput(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
t.Errorf("unexpected request to %s", r.URL)
})
if _, err := client.ListNewer(context.Background(), "news", Session{PlatformUID: "u", PlatformToken: "t"}); err == nil {
t.Error("unsupported category accepted")
}
if _, err := client.ListNewer(context.Background(), "drama", Session{}); err == nil {
t.Error("empty session accepted")
}
}
internal/tver/series.go
+99
View File
@@ -0,0 +1,99 @@
package tver
import (
"context"
"errors"
"fmt"
"net/http"
"net/url"
"regexp"
)
var tverSeriesIDPattern = regexp.MustCompile(`^[A-Za-z0-9]+$`)
type seriesContents struct {
Code int `json:"code"`
Result struct {
Contents []struct {
Type string `json:"type"`
Content Episode `json:"content"`
} `json:"contents"`
} `json:"result"`
}
func (c *Client) FetchSeriesEpisodes(ctx context.Context, seriesID string) ([]Episode, error) {
if !tverSeriesIDPattern.MatchString(seriesID) {
return nil, errors.New("invalid series id")
}
seasonsURL := c.serviceURL.ResolveReference(&url.URL{Path: "/api/v1/callSeriesSeasons/" + seriesID})
request, err := http.NewRequestWithContext(ctx, http.MethodGet, seasonsURL.String(), nil)
if err != nil {
return nil, fmt.Errorf("series seasons request: %w", err)
}
setTVerHeaders(request)
var seasons seriesContents
if err := c.getJSON(request, &seasons); err != nil {
return nil, fmt.Errorf("series seasons: %w", err)
}
if seasons.Code != 0 || seasons.Result.Contents == nil {
return nil, fmt.Errorf("series seasons: invalid API result (code %d)", seasons.Code)
}
var seasonIDs []string
for _, item := range seasons.Result.Contents {
if item.Type != "season" {
continue
}
if !tverSeriesIDPattern.MatchString(item.Content.ID) {
return nil, errors.New("series seasons: invalid season id")
}
seasonIDs = append(seasonIDs, item.Content.ID)
}
if len(seasonIDs) == 0 {
return []Episode{}, nil
}
session, err := c.CreateSession(ctx)
if err != nil {
return nil, err
}
seen := make(map[string]bool)
var episodes []Episode
for _, seasonID := range seasonIDs {
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v1/callSeasonEpisodes/" + seasonID})
query := endpoint.Query()
query.Set("platform_uid", session.PlatformUID)
query.Set("platform_token", session.PlatformToken)
endpoint.RawQuery = query.Encode()
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
if err != nil {
return nil, fmt.Errorf("season episodes request: %w", err)
}
setTVerHeaders(request)
var response seriesContents
if err := c.getJSON(request, &response); err != nil {
return nil, fmt.Errorf("season episodes: %w", err)
}
if response.Code != 0 || response.Result.Contents == nil {
return nil, fmt.Errorf("season episodes: invalid API result (code %d)", response.Code)
}
for _, item := range response.Result.Contents {
if item.Type != "episode" {
continue
}
ep := item.Content
if !tverSeriesIDPattern.MatchString(ep.ID) || ep.SeriesID != seriesID {
return nil, errors.New("season episodes: invalid episode identity")
}
if !seen[ep.ID] {
episodes = append(episodes, ep)
seen[ep.ID] = true
}
}
}
return episodes, nil
}
func setTVerHeaders(request *http.Request) {
request.Header.Set("x-tver-platform-type", "web")
request.Header.Set("Origin", "https://tver.jp")
request.Header.Set("Referer", "https://tver.jp/")
}
internal/tver/series_test.go
+182
View File
@@ -0,0 +1,182 @@
package tver
import (
"context"
"net/http"
"os"
"strings"
"testing"
)
func TestFetchSeriesEpisodesFromRecordedResponses(t *testing.T) {
seasons, err := os.ReadFile("testdata/series_seasons.json")
if err != nil {
t.Fatal(err)
}
episodes, err := os.ReadFile("testdata/season_episodes.json")
if err != nil {
t.Fatal(err)
}
var seasonCalls, sessionCalls, episodeCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/v1/callSeriesSeasons/sr0lhdrtdb":
seasonCalls++
if r.Method != http.MethodGet || r.Header.Get("x-tver-platform-type") != "web" || r.Header.Get("Origin") != "https://tver.jp" || r.Header.Get("Referer") != "https://tver.jp/" {
t.Errorf("seasons request: method=%s headers=%v", r.Method, r.Header)
}
_, _ = w.Write(seasons)
case "/v2/api/platform_users/browser/create":
sessionCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
case "/service/api/v1/callSeasonEpisodes/ss3i6njoq3":
episodeCalls++
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" || r.Header.Get("x-tver-platform-type") != "web" || r.Header.Get("Origin") != "https://tver.jp" || r.Header.Get("Referer") != "https://tver.jp/" {
t.Errorf("season episode request: method=%s query=%v headers=%v", r.Method, r.URL.Query(), r.Header)
}
_, _ = w.Write(episodes)
default:
t.Errorf("unexpected path: %s", r.URL.Path)
http.NotFound(w, r)
}
})
got, err := client.FetchSeriesEpisodes(context.Background(), "sr0lhdrtdb")
if err != nil || len(got) != 1 {
t.Fatalf("series episodes: %+v %v", got, err)
}
ep := got[0]
if ep.ID != "eps0z5761a" || ep.SeriesID != "sr0lhdrtdb" || ep.Title != "#10（最終話）" || ep.SeriesTitle != "韓国ドラマ「良いが悪い、ドンジェ」" || ep.BroadcasterName != "BS12トゥエルビ" || ep.Duration != 2712 || ep.EndAt != 1792209540 || !strings.Contains(ep.ThumbnailPath, "eps0z5761a.jpg") {
t.Errorf("episode fields: %+v", ep)
}
if seasonCalls != 1 || sessionCalls != 1 || episodeCalls != 1 {
t.Errorf("calls: seasons=%d session=%d episodes=%d", seasonCalls, sessionCalls, episodeCalls)
}
}
func TestFetchSeriesEpisodesTransportAndDecodeErrors(t *testing.T) {
validSeasons := `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`
validSession := `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`
validEpisodes := `{"code":0,"result":{"contents":[]}}`
for _, tt := range []struct {
name string
seasonStatus, sessionStatus, episodeStatus int
seasonBody, sessionBody, episodeBody string
want string
wantSession, wantEpisode int
}{
{"seasons HTTP", 503, 200, 200, "", validSession, validEpisodes, "series seasons: HTTP 503", 0, 0},
{"seasons JSON", 200, 200, 200, "{", validSession, validEpisodes, "series seasons: decode JSON", 0, 0},
{"session HTTP", 200, 503, 200, validSeasons, "", validEpisodes, "create session: HTTP 503", 1, 0},
{"episodes HTTP", 200, 200, 502, validSeasons, validSession, "", "season episodes: HTTP 502", 1, 1},
{"episodes JSON", 200, 200, 200, validSeasons, validSession, "{", "season episodes: decode JSON", 1, 1},
} {
t.Run(tt.name, func(t *testing.T) {
var seasonCalls, sessionCalls, episodeCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/v1/callSeriesSeasons/sr1":
seasonCalls++
w.WriteHeader(tt.seasonStatus)
_, _ = w.Write([]byte(tt.seasonBody))
case "/v2/api/platform_users/browser/create":
sessionCalls++
w.WriteHeader(tt.sessionStatus)
_, _ = w.Write([]byte(tt.sessionBody))
case "/service/api/v1/callSeasonEpisodes/ss1":
episodeCalls++
w.WriteHeader(tt.episodeStatus)
_, _ = w.Write([]byte(tt.episodeBody))
default:
t.Errorf("unexpected request: %s", r.URL.Path)
}
})
_, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
if err == nil || !strings.Contains(err.Error(), tt.want) {
t.Fatalf("error=%v; want %q", err, tt.want)
}
if seasonCalls != 1 || sessionCalls != tt.wantSession || episodeCalls != tt.wantEpisode {
t.Errorf("calls: seasons=%d session=%d episodes=%d", seasonCalls, sessionCalls, episodeCalls)
}
})
}
}
func TestFetchSeriesEpisodesRejectsBadResponses(t *testing.T) {
for _, tt := range []struct {
name, seasons, episodes string
}{
{"season API code", `{"code":1,"result":{"contents":[]}}`, ""},
{"missing seasons", `{"code":0,"result":{}}`, ""},
{"season without id", `{"code":0,"result":{"contents":[{"type":"season","content":{}}]}}`, ""},
{"episode API code", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":1,"result":{"contents":[]}}`},
{"wrong series", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"other"}}]}}`},
{"episode without id", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":0,"result":{"contents":[{"type":"episode","content":{"seriesID":"sr1"}}]}}`},
} {
t.Run(tt.name, func(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.Contains(r.URL.Path, "callSeriesSeasons"):
_, _ = w.Write([]byte(tt.seasons))
case strings.Contains(r.URL.Path, "browser/create"):
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
default:
_, _ = w.Write([]byte(tt.episodes))
}
})
if _, err := client.FetchSeriesEpisodes(context.Background(), "sr1"); err == nil {
t.Fatal("invalid response accepted")
}
})
}
}
func TestFetchSeriesEpisodesMultipleSeasonsDeduplicates(t *testing.T) {
var sessionCalls, episodeCalls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/api/v1/callSeriesSeasons/sr1":
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}},{"type":"season","content":{"id":"ss2"}}]}}`))
case "/v2/api/platform_users/browser/create":
sessionCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
case "/service/api/v1/callSeasonEpisodes/ss1":
episodeCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"sr1"}}]}}`))
case "/service/api/v1/callSeasonEpisodes/ss2":
episodeCalls++
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"sr1"}},{"type":"episode","content":{"id":"ep2","seriesID":"sr1"}}]}}`))
default:
t.Errorf("unexpected request: %s", r.URL.Path)
}
})
episodes, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
if err != nil || len(episodes) != 2 || episodes[0].ID != "ep1" || episodes[1].ID != "ep2" || sessionCalls != 1 || episodeCalls != 2 {
t.Fatalf("複数 season: %+v %v session=%d episode=%d", episodes, err, sessionCalls, episodeCalls)
}
}
func TestFetchSeriesEpisodesNoSeasonsSkipsSession(t *testing.T) {
var calls int
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
calls++
if r.URL.Path != "/api/v1/callSeriesSeasons/sr1" {
t.Errorf("unexpected request: %s", r.URL.Path)
}
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[]}}`))
})
episodes, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
if err != nil || len(episodes) != 0 || calls != 1 {
t.Fatalf("season 無し: %+v %v calls=%d", episodes, err, calls)
}
}
func TestFetchSeriesEpisodesRejectsInvalidSeriesIDBeforeRequest(t *testing.T) {
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
t.Errorf("unexpected request: %s", r.URL.Path)
})
for _, id := range []string{"", "../other", "sr1?x=1"} {
if _, err := client.FetchSeriesEpisodes(context.Background(), id); err == nil {
t.Errorf("invalid series id accepted: %q", id)
}
}
}
internal/tver/testdata/newer_drama.json
Vendored
+91
View File
@@ -0,0 +1,91 @@
{
"api_version": "",
"code": 0,
"message": "",
"type": "hasharray",
"result": {
"contents": {
"type": "newer",
"content": {
"id": "drama",
"version": 1,
"title": "ドラマ"
},
"thumbnailType": "horizontal",
"contents": [
{
"type": "episode",
"content": {
"id": "eps0z5761a",
"version": 4,
"title": "#10（最終話）",
"seriesID": "sr0lhdrtdb",
"duration": 2712,
"endAt": 1792209540,
"broadcastDateLabel": "10月3日(土)放送分",
"isNHKContent": false,
"isSubtitle": false,
"ribbonID": 0,
"seriesTitle": "韓国ドラマ「良いが悪い、ドンジェ」",
"isAvailable": true,
"broadcasterName": "BS12トゥエルビ",
"productionProviderName": "BS12 トゥエルビ",
"isEndingSoon": false,
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51",
"isChannelDefaultSeries": false
},
"startAt": 1791007200,
"endAt": 1792209540
},
{
"type": "episode",
"content": {
"id": "epo4tfi10u",
"version": 4,
"title": "#9",
"seriesID": "sr0lhdrtdb",
"duration": 2712,
"endAt": 1792209540,
"broadcastDateLabel": "10月3日(土)放送分",
"isNHKContent": false,
"isSubtitle": false,
"ribbonID": 0,
"seriesTitle": "韓国ドラマ「良いが悪い、ドンジェ」",
"isAvailable": true,
"broadcasterName": "BS12トゥエルビ",
"productionProviderName": "BS12 トゥエルビ",
"isEndingSoon": false,
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/epo4tfi10u.jpg?v=2d8ff444",
"isChannelDefaultSeries": false
},
"startAt": 1791007200,
"endAt": 1792209540
},
{
"type": "episode",
"content": {
"id": "ep8cxuessg",
"version": 6,
"title": "【予告】10月12日(月)24時24分 ~ 放送予定 60秒PR！",
"seriesID": "srne6nwihe",
"duration": 60,
"endAt": 1800282240,
"broadcastDateLabel": "",
"isNHKContent": false,
"isSubtitle": false,
"ribbonID": 1,
"seriesTitle": "十六夜膳房 -いざよいぜんぼう-",
"isAvailable": true,
"broadcasterName": "日テレ",
"productionProviderName": "日テレ",
"isEndingSoon": false,
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/ep8cxuessg.jpg?v=58408db2",
"isChannelDefaultSeries": false
},
"startAt": 1791007200,
"endAt": 1800282240
}
]
}
}
}
internal/tver/testdata/season_episodes.json
Vendored
+1
View File
@@ -0,0 +1 @@
{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"eps0z5761a","version":4,"title":"#10（最終話）","seriesID":"sr0lhdrtdb","duration":2712,"endAt":1792209540,"broadcastDateLabel":"10月3日(土)放送分","isNHKContent":false,"isSubtitle":false,"ribbonID":0,"seriesTitle":"韓国ドラマ「良いが悪い、ドンジェ」","isAvailable":true,"broadcasterName":"BS12トゥエルビ","productionProviderName":"BS12 トゥエルビ","isEndingSoon":false,"thumbnailPath":"/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51","isChannelDefaultSeries":false}}]}}
internal/tver/testdata/series_seasons.json
Vendored
+1
View File
@@ -0,0 +1 @@
{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss3i6njoq3","version":1,"title":"本編","season_extended":null}}]}}
internal/web/actions.go
+90
View File
@@ -0,0 +1,90 @@
package web
import (
"errors"
"net/http"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
type episodeActionView struct {
ID, Genre, Status, StatusText string
}
func statusText(status string) string {
switch status {
case "queued":
return "待機中"
case "running":
return "ダウンロード中"
case "succeeded":
return "完了"
case "failed":
return "失敗"
case "canceled":
return "取り消し"
default:
return ""
}
}
func (s *server) enqueue(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
ep, found := s.cachedEpisode(genre, id)
if !found {
http.Error(w, "新着キャッシュにエピソードがありません。新着を再表示してください", http.StatusNotFound)
return
}
job, _, err := s.jobs.Enqueue(r.Context(), storeEpisode(ep), "manual")
if err != nil {
s.logger.Printf("待ち行列への登録に失敗: %v", err)
http.Error(w, "待ち行列に登録できませんでした", http.StatusInternalServerError)
return
}
s.episodeActionResponse(w, r, episodeActionView{ID: id, Genre: genre, Status: job.Status, StatusText: statusText(job.Status)})
}
func (s *server) retryEpisode(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
if _, found := s.cachedEpisode(genre, id); !found {
http.Error(w, "新着キャッシュにエピソードがありません。新着を再表示してください", http.StatusNotFound)
return
}
if err := s.jobs.RetryEpisode(r.Context(), id); err != nil {
if errors.Is(err, store.ErrInvalidTransition) {
http.Error(w, "再試行できる状態ではありません", http.StatusConflict)
} else {
s.logger.Printf("ジョブの再試行に失敗: %v", err)
http.Error(w, "再試行できませんでした", http.StatusInternalServerError)
}
return
}
s.episodeActionResponse(w, r, episodeActionView{ID: id, Genre: genre, Status: "queued", StatusText: statusText("queued")})
}
func episodeForm(w http.ResponseWriter, r *http.Request) (string, string, bool) {
if err := r.ParseForm(); err != nil {
http.Error(w, "フォームを読み取れませんでした", http.StatusBadRequest)
return "", "", false
}
genre, id := r.Form.Get("genre"), r.Form.Get("episode_id")
if (genre != "drama" && genre != "variety") || id == "" {
http.Error(w, "ジャンルまたはエピソード ID が不正です", http.StatusBadRequest)
return "", "", false
}
return genre, id, true
}
func (s *server) episodeActionResponse(w http.ResponseWriter, r *http.Request, view episodeActionView) {
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/?genre="+view.Genre, http.StatusSeeOther)
return
}
s.render(w, "episode_action", view)
}
internal/web/handler.go
+98
View File
@@ -0,0 +1,98 @@
package web
import (
"context"
"embed"
"html/template"
"io/fs"
"log"
"net/http"
"sync"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
//go:embed templates/*.html static/*
var assets embed.FS
type page struct {
Title string
Newer *newerView
Queue *queueView
Subscriptions *subscriptionsView
Library *libraryView
Video *videoView
}
type JobStore interface {
Enqueue(context.Context, store.Episode, string) (store.Job, bool, error)
EpisodeStatuses(context.Context, []string) (map[string]string, error)
ListJobs(context.Context, string, int) ([]store.Job, error)
GetJob(context.Context, int64) (store.Job, error)
Retry(context.Context, int64) error
RetryEpisode(context.Context, string) error
Cancel(context.Context, int64) error
SubscribedSeries(context.Context, []string) (map[string]bool, error)
RegisterSubscription(context.Context, string, string, []store.Episode, bool) (bool, error)
ListSubscriptions(context.Context) ([]store.Subscription, error)
SetSubscriptionEnabled(context.Context, string, bool) error
DeleteSubscription(context.Context, string) (bool, error)
ListSucceededJobs(context.Context, int) ([]store.Job, error)
GetSucceededJob(context.Context, int64) (store.Job, error)
DeleteSucceededJob(context.Context, int64) error
}
type server struct {
fetcher NewerFetcher
jobs JobStore
videoDir string
now func() time.Time
logger *log.Logger
zone *time.Location
tmpl *template.Template
cacheMu sync.Mutex
cache map[string]cacheEntry
}
func NewHandler(fetcher NewerFetcher, jobs JobStore, videoDir string, now func() time.Time, logger *log.Logger) http.Handler {
tmpl := template.Must(template.ParseFS(assets, "templates/*.html"))
staticFiles, err := fs.Sub(assets, "static")
if err != nil {
panic(err)
}
zone, err := time.LoadLocation("Asia/Tokyo")
if err != nil {
panic(err)
}
if videoDir == "" {
videoDir = "/videos"
}
s := &server{fetcher: fetcher, jobs: jobs, videoDir: videoDir, now: now, logger: logger, zone: zone, tmpl: tmpl, cache: make(map[string]cacheEntry)}
mux := http.NewServeMux()
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServerFS(staticFiles)))
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
_, _ = w.Write([]byte("ok\n"))
})
mux.HandleFunc("GET /{$}", s.home)
mux.HandleFunc("GET /newer", s.newerFragment)
mux.HandleFunc("POST /episodes/enqueue", s.enqueue)
mux.HandleFunc("POST /episodes/retry", s.retryEpisode)
mux.HandleFunc("GET /queue", s.queuePage)
mux.HandleFunc("GET /queue/list", s.queueFragment)
mux.HandleFunc("POST /jobs/{id}/retry", s.retryJob)
mux.HandleFunc("POST /jobs/{id}/cancel", s.cancelJob)
mux.HandleFunc("GET /subscriptions", s.subscriptionsPage)
mux.HandleFunc("GET /subscriptions/list", s.subscriptionsFragment)
mux.HandleFunc("GET /subscriptions/choice", s.subscriptionChoice)
mux.HandleFunc("POST /subscriptions/register", s.registerSubscription)
mux.HandleFunc("POST /subscriptions/{id}/pause", s.pauseSubscription)
mux.HandleFunc("POST /subscriptions/{id}/resume", s.resumeSubscription)
mux.HandleFunc("POST /subscriptions/{id}/delete", s.deleteSubscription)
mux.HandleFunc("GET /library", s.libraryPage)
mux.HandleFunc("GET /library/{id}", s.videoPage)
mux.HandleFunc("GET /videos/{id}", s.serveVideo)
mux.HandleFunc("POST /library/{id}/delete", s.deleteVideo)
return mux
}
internal/web/handler_test.go
+63
View File
@@ -0,0 +1,63 @@
package web
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestHealthz(t *testing.T) {
response := httptest.NewRecorder()
newTestHandler(&fakeFetcher{}, nil).ServeHTTP(response, httptest.NewRequest(http.MethodGet, "/healthz", nil))
if response.Code != http.StatusOK || response.Body.String() != "ok\n" {
t.Errorf("healthz: status=%d body=%q", response.Code, response.Body.String())
}
}
func TestPagesAndEmbeddedAssets(t *testing.T) {
handler := newTestHandler(&fakeFetcher{}, nil)
for _, tt := range []struct{ path, title string }{
{"/", "新着"}, {"/queue", "待ち行列"}, {"/subscriptions", "登録番組"}, {"/library", "ライブラリ"},
} {
t.Run(tt.path, func(t *testing.T) {
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, tt.path, nil))
body := response.Body.String()
if response.Code != http.StatusOK || !strings.Contains(response.Header().Get("Content-Type"), "text/html") {
t.Fatalf("status=%d Content-Type=%q", response.Code, response.Header().Get("Content-Type"))
}
for _, want := range []string{"<h1>" + tt.title + "</h1>", "新着", "待ち行列", "登録番組", "ライブラリ", `hx-boost="true"`, "/static/htmx-2.0.11.min.js"} {
if !strings.Contains(body, want) {
t.Errorf("page missing %q", want)
}
}
})
}
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, "/static/htmx-2.0.11.min.js", nil))
if response.Code != http.StatusOK || !strings.Contains(response.Header().Get("Content-Type"), "javascript") || response.Body.Len() < 1000 || !strings.Contains(response.Body.String(), "htmx") {
t.Errorf("embedded htmx: status=%d Content-Type=%q bytes=%d", response.Code, response.Header().Get("Content-Type"), response.Body.Len())
}
}
func TestUnknownPathsAndMethods(t *testing.T) {
handler := newTestHandler(&fakeFetcher{}, nil)
for _, tt := range []struct {
name, method, path string
wantStatus int
}{
{"unknown page", http.MethodGet, "/missing", http.StatusNotFound},
{"root prefix", http.MethodGet, "/missing/nested", http.StatusNotFound},
{"missing static asset", http.MethodGet, "/static/missing.js", http.StatusNotFound},
{"unsupported page method", http.MethodPost, "/queue", http.StatusMethodNotAllowed},
} {
t.Run(tt.name, func(t *testing.T) {
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(tt.method, tt.path, nil))
if response.Code != tt.wantStatus {
t.Errorf("%s %s: status=%d; want %d", tt.method, tt.path, response.Code, tt.wantStatus)
}
})
}
}
internal/web/library.go
+153
View File
@@ -0,0 +1,153 @@
package web
import (
"errors"
"fmt"
"net/http"
"os"
"path/filepath"
"strconv"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
const libraryLimit = 100
type libraryItemView struct {
ID int64
ThumbnailURL, SeriesTitle, Title string
BroadcasterName, DurationText string
FileSizeText, FinishedAtText string
}
type libraryView struct {
Items []libraryItemView
}
type videoView struct {
libraryItemView
VideoURL string
}
func (s *server) libraryPage(w http.ResponseWriter, r *http.Request) {
jobs, err := s.jobs.ListSucceededJobs(r.Context(), libraryLimit)
if err != nil {
s.logger.Printf("ライブラリの取得に失敗: %v", err)
http.Error(w, "ライブラリを取得できませんでした", http.StatusInternalServerError)
return
}
view := libraryView{Items: make([]libraryItemView, 0, len(jobs))}
for _, job := range jobs {
view.Items = append(view.Items, s.libraryItem(job))
}
s.render(w, "layout", page{Title: "ライブラリ", Library: &view})
}
func (s *server) videoPage(w http.ResponseWriter, r *http.Request) {
job, ok := s.succeededJob(w, r)
if !ok {
return
}
item := s.libraryItem(job)
s.render(w, "layout", page{Title: job.SeriesTitle + " " + job.Title,
Video: &videoView{libraryItemView: item, VideoURL: "/videos/" + strconv.FormatInt(job.ID, 10)}})
}
func (s *server) libraryItem(job store.Job) libraryItemView {
item := libraryItemView{ID: job.ID, ThumbnailURL: "https://statics.tver.jp" + job.ThumbnailPath,
SeriesTitle: job.SeriesTitle, Title: job.Title, BroadcasterName: job.BroadcasterName,
DurationText: fmt.Sprintf("%d分", durationMinutes(int64(job.DurationSeconds)))}
if job.FileSizeBytes != nil {
item.FileSizeText = formatBytes(float64(*job.FileSizeBytes))
}
if job.FinishedAt != nil {
item.FinishedAtText = job.FinishedAt.In(s.zone).Format("2006-01-02 15:04 MST")
}
return item
}
func (s *server) serveVideo(w http.ResponseWriter, r *http.Request) {
job, ok := s.succeededJob(w, r)
if !ok || job.FilePath == nil {
if ok {
http.NotFound(w, r)
}
return
}
root, err := os.OpenRoot(s.videoDir)
if err != nil {
http.NotFound(w, r)
return
}
defer root.Close()
file, err := root.Open(*job.FilePath)
if err != nil {
http.NotFound(w, r)
return
}
defer file.Close()
info, err := file.Stat()
if err != nil || !info.Mode().IsRegular() {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "video/mp4")
http.ServeContent(w, r, filepath.Base(*job.FilePath), info.ModTime(), file)
}
func (s *server) deleteVideo(w http.ResponseWriter, r *http.Request) {
job, ok := s.succeededJob(w, r)
if !ok {
return
}
if job.FilePath != nil {
if !filepath.IsLocal(*job.FilePath) {
http.Error(w, "動画ファイルの場所が不正です", http.StatusConflict)
return
}
root, err := os.OpenRoot(s.videoDir)
if err != nil {
http.Error(w, "動画ファイルの場所が不正です", http.StatusConflict)
return
}
defer root.Close()
file, err := root.Open(*job.FilePath)
if err == nil {
_ = file.Close()
} else if !errors.Is(err, os.ErrNotExist) {
http.Error(w, "動画ファイルの場所が不正です", http.StatusConflict)
return
}
if err := root.Remove(*job.FilePath); err != nil && !errors.Is(err, os.ErrNotExist) {
s.logger.Printf("動画ファイルの削除に失敗: %v", err)
http.Error(w, "動画ファイルを削除できませんでした", http.StatusInternalServerError)
return
}
}
if err := s.jobs.DeleteSucceededJob(r.Context(), job.ID); err != nil {
s.logger.Printf("動画の記録削除に失敗: %v", err)
http.Error(w, "動画の記録を削除できませんでした", http.StatusInternalServerError)
return
}
http.Redirect(w, r, "/library", http.StatusSeeOther)
}
func (s *server) succeededJob(w http.ResponseWriter, r *http.Request) (store.Job, bool) {
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.NotFound(w, r)
return store.Job{}, false
}
job, err := s.jobs.GetSucceededJob(r.Context(), id)
if errors.Is(err, pgx.ErrNoRows) {
http.NotFound(w, r)
return store.Job{}, false
}
if err != nil {
s.logger.Printf("完了ジョブの取得に失敗: %v", err)
http.Error(w, "動画を取得できませんでした", http.StatusInternalServerError)
return store.Job{}, false
}
return job, true
}
internal/web/library_test.go
+294
View File
@@ -0,0 +1,294 @@
package web
import (
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
func libraryJob(id int64, relative string) store.Job {
size := int64(10)
finished := time.Date(2026, 10, 3, 1, 2, 3, 0, time.UTC)
return store.Job{ID: id, EpisodeID: "ep1", SeriesTitle: `<script>番組</script>`, Title: `<b>第1話</b>`,
BroadcasterName: "放送局", DurationSeconds: 61, ThumbnailPath: "/thumb.jpg", Status: "succeeded",
FilePath: &relative, FileSizeBytes: &size, FinishedAt: &finished}
}
func TestLibraryAndPlaybackPage(t *testing.T) {
dir := t.TempDir()
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
page := getPage(handler, "/library")
if page.Code != http.StatusOK {
t.Fatalf("library status=%d", page.Code)
}
body := page.Body.String()
for _, want := range []string{"&lt;script&gt;番組&lt;/script&gt;", "&lt;b&gt;第1話&lt;/b&gt;", "放送局", "長さ: 2分", "10.0 B", "2026-10-03 10:02 JST", "https://statics.tver.jp/thumb.jpg", `/library/1`} {
if !strings.Contains(body, want) {
t.Errorf("library missing %q", want)
}
}
if strings.Contains(body, "<script>番組</script>") {
t.Fatal("library fields were not escaped")
}
video := getPage(handler, "/library/1")
if video.Code != http.StatusOK || !strings.Contains(video.Body.String(), `<video controls`) || !strings.Contains(video.Body.String(), `src="/videos/1"`) || !strings.Contains(video.Body.String(), `hx-confirm="動画ファイルと記録を削除しますか？"`) {
t.Fatalf("video page: %d %s", video.Code, video.Body.String())
}
for _, path := range []string{"/library/bad", "/library/99", "/library/1/delete"} {
response := getPage(handler, path)
want := http.StatusNotFound
if strings.HasSuffix(path, "/delete") {
want = http.StatusMethodNotAllowed
}
if response.Code != want {
t.Errorf("%s status=%d want=%d", path, response.Code, want)
}
if want == http.StatusMethodNotAllowed && !strings.Contains(response.Header().Get("Allow"), "POST") {
t.Errorf("%s Allow=%q", path, response.Header().Get("Allow"))
}
}
}
func TestLibraryRejectsInvalidJobIDs(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
for _, id := range []string{"0", "-1", "bad", "999999999999999999999999999999999999"} {
for _, path := range []string{"/library/" + id, "/videos/" + id} {
if got := getPage(handler, path).Code; got != http.StatusNotFound {
t.Errorf("GET %s: %d", path, got)
}
}
if got := postForm(handler, "/library/"+id+"/delete", nil).Code; got != http.StatusNotFound {
t.Errorf("POST delete id=%q: %d", id, got)
}
}
}
func TestVideoRangeAndMissingFile(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
t.Fatal(err)
}
content := []byte("0123456789")
if err := os.WriteFile(filepath.Join(dir, "show", "video.mp4"), content, 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
request := httptest.NewRequest(http.MethodGet, "/videos/1?file_path=../../outside.mp4", nil)
request.Header.Set("Range", "bytes=2-5")
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusPartialContent || response.Body.String() != "2345" || response.Header().Get("Content-Range") != "bytes 2-5/10" || response.Header().Get("Content-Type") != "video/mp4" {
t.Fatalf("range: status=%d range=%q type=%q body=%q", response.Code, response.Header().Get("Content-Range"), response.Header().Get("Content-Type"), response.Body.String())
}
full := getPage(handler, "/videos/1")
if full.Code != http.StatusOK || full.Body.String() != string(content) || full.Header().Get("Content-Type") != "video/mp4" || full.Header().Get("Accept-Ranges") != "bytes" {
t.Fatalf("full video: status=%d type=%q ranges=%q body=%q", full.Code, full.Header().Get("Content-Type"), full.Header().Get("Accept-Ranges"), full.Body.String())
}
request = httptest.NewRequest(http.MethodGet, "/videos/1", nil)
request.Header.Set("Range", "bytes=20-30")
response = httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusRequestedRangeNotSatisfiable || response.Header().Get("Content-Range") != "bytes */10" {
t.Fatalf("range outside: status=%d range=%q", response.Code, response.Header().Get("Content-Range"))
}
if err := os.Remove(filepath.Join(dir, "show", "video.mp4")); err != nil {
t.Fatal(err)
}
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("missing file status=%d", got)
}
jobs.jobs["ep1"] = store.Job{ID: 1, Status: "queued"}
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("unfinished status=%d", got)
}
if got := getPage(handler, "/library/1").Code; got != http.StatusNotFound {
t.Fatalf("unfinished playback page status=%d", got)
}
missingPath := libraryJob(1, "show/video.mp4")
missingPath.FilePath = nil
jobs.jobs["ep1"] = missingPath
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("missing DB file path status=%d", got)
}
}
func TestVideoPathTraversalAndSymlinkAreRejected(t *testing.T) {
dir := t.TempDir()
outside := t.TempDir()
outsideFile := filepath.Join(outside, "outside.mp4")
if err := os.WriteFile(outsideFile, []byte("secret"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outsideFile, filepath.Join(dir, "link.mp4")); err != nil {
t.Fatal(err)
}
if err := os.Symlink(outside, filepath.Join(dir, "linked-directory")); err != nil {
t.Fatal(err)
}
relativeOutsideFile, err := filepath.Rel(dir, outsideFile)
if err != nil {
t.Fatal(err)
}
relativeOutsideDir, err := filepath.Rel(dir, outside)
if err != nil {
t.Fatal(err)
}
if err := os.Symlink(relativeOutsideFile, filepath.Join(dir, "relative-link.mp4")); err != nil {
t.Fatal(err)
}
if err := os.Symlink(relativeOutsideDir, filepath.Join(dir, "relative-linked-directory")); err != nil {
t.Fatal(err)
}
for _, relative := range []string{"../outside.mp4", "show/../../outside.mp4", outsideFile,
"link.mp4", "linked-directory/outside.mp4", "linked-directory/missing.mp4",
"relative-link.mp4", "relative-linked-directory/outside.mp4", "relative-linked-directory/missing.mp4"} {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, relative)}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Errorf("path %q status=%d", relative, got)
}
deleted := postForm(handler, "/library/1/delete", nil)
if deleted.Code != http.StatusConflict || len(jobs.jobs) != 1 {
t.Errorf("delete path %q status=%d jobs=%d", relative, deleted.Code, len(jobs.jobs))
}
}
if got, err := os.ReadFile(outsideFile); err != nil || string(got) != "secret" {
t.Fatalf("outside target was changed: content=%q err=%v", got, err)
}
for _, link := range []string{"link.mp4", "linked-directory", "relative-link.mp4", "relative-linked-directory"} {
if _, err := os.Lstat(filepath.Join(dir, link)); err != nil {
t.Errorf("rejected link %q was changed: %v", link, err)
}
}
}
func TestVideoSymlinkWithinRootCanBeServedAndDeleted(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "target.mp4")
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink("target.mp4", filepath.Join(dir, "link.mp4")); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "link.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if response := getPage(handler, "/videos/1"); response.Code != http.StatusOK || response.Body.String() != "video" {
t.Fatalf("serve: status=%d body=%q", response.Code, response.Body.String())
}
if response := postForm(handler, "/library/1/delete", nil); response.Code != http.StatusSeeOther || len(jobs.jobs) != 0 {
t.Fatalf("delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
}
if _, err := os.Stat(target); err != nil {
t.Fatalf("symlink target was removed: %v", err)
}
if _, err := os.Lstat(filepath.Join(dir, "link.mp4")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("symlink itself remains: %v", err)
}
}
func TestVideoSymlinkedDirectoryWithinRootCanBeServedAndDeleted(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "target"), 0o755); err != nil {
t.Fatal(err)
}
target := filepath.Join(dir, "target", "video.mp4")
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink("target", filepath.Join(dir, "linked-directory")); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "linked-directory/video.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if response := getPage(handler, "/videos/1"); response.Code != http.StatusOK || response.Body.String() != "video" {
t.Fatalf("serve: status=%d body=%q", response.Code, response.Body.String())
}
if response := postForm(handler, "/library/1/delete", nil); response.Code != http.StatusSeeOther || len(jobs.jobs) != 0 {
t.Fatalf("delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
}
if _, err := os.Stat(target); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("target file remains: %v", err)
}
if _, err := os.Lstat(filepath.Join(dir, "linked-directory")); err != nil {
t.Fatalf("directory symlink was removed: %v", err)
}
}
func TestVideoAbsoluteSymlinkWithinRootIsRejectedByOSRoot(t *testing.T) {
dir := t.TempDir()
target := filepath.Join(dir, "target.mp4")
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, filepath.Join(dir, "absolute-link.mp4")); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "absolute-link.mp4")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
t.Fatalf("absolute symlink serve status=%d", got)
}
if response := postForm(handler, "/library/1/delete", nil); response.Code != http.StatusConflict || len(jobs.jobs) != 1 {
t.Fatalf("absolute symlink delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
}
if _, err := os.Stat(target); err != nil {
t.Fatalf("target was changed: %v", err)
}
}
func TestDeleteVideoFileThenRecord(t *testing.T) {
dir := t.TempDir()
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
t.Fatal(err)
}
path := filepath.Join(dir, "show", "video.mp4")
if err := os.WriteFile(path, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
fileWasGoneBeforeRecordDelete := false
jobs.beforeDeleteSucceeded = func() {
_, err := os.Stat(path)
fileWasGoneBeforeRecordDelete = errors.Is(err, os.ErrNotExist)
}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
response := postForm(handler, "/library/1/delete", nil)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/library" || len(jobs.jobs) != 0 || !fileWasGoneBeforeRecordDelete {
t.Fatalf("delete: status=%d location=%q jobs=%d", response.Code, response.Header().Get("Location"), len(jobs.jobs))
}
if _, err := os.Stat(path); !os.IsNotExist(err) {
t.Fatalf("file remains: %v", err)
}
jobs.jobs["ep1"] = libraryJob(1, "show/missing.mp4")
if got := postForm(handler, "/library/1/delete", nil).Code; got != http.StatusSeeOther || len(jobs.jobs) != 0 {
t.Fatalf("missing file delete: status=%d jobs=%d", got, len(jobs.jobs))
}
}
func TestDeleteFailureKeepsRecord(t *testing.T) {
dir := t.TempDir()
blocked := filepath.Join(dir, "blocked")
if err := os.Mkdir(blocked, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(blocked, "child"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "blocked")}}
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
response := postForm(handler, "/library/1/delete", nil)
if response.Code != http.StatusInternalServerError || len(jobs.jobs) != 1 || !strings.Contains(response.Body.String(), "動画ファイルを削除できませんでした") {
t.Fatalf("failure: status=%d jobs=%d body=%s", response.Code, len(jobs.jobs), response.Body.String())
}
}
internal/web/newer.go
+239
View File
@@ -0,0 +1,239 @@
package web
import (
"bytes"
"context"
"fmt"
"net/http"
"net/url"
"regexp"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
const (
newerCacheTTL = 10 * time.Minute
)
var platformTokenPattern = regexp.MustCompile(`(?i)(platform_token=)[^&\s]+`)
type NewerFetcher interface {
FetchNewer(context.Context, string) ([]tver.Episode, error)
FetchSeriesEpisodes(context.Context, string) ([]tver.Episode, error)
}
type cacheEntry struct {
episodes []tver.Episode
expiresAt time.Time
}
type newerOptions struct {
genre string
hidePreviews bool
}
type episodeView struct {
ID string
SeriesID string
SubscriptionAction subscriptionActionView
Genre string
Status string
StatusText string
URL string
ThumbnailURL string
SeriesTitle string
Title string
BroadcasterName string
DurationMinutes int64
BroadcastDateLabel string
EndAtText string
EndAtISO string
}
type newerView struct {
Genre string
HidePreviews bool
Episodes []episodeView
Failed bool
ErrorText string
}
func parseNewerOptions(r *http.Request) (newerOptions, error) {
options := newerOptions{genre: "drama", hidePreviews: true}
query := r.URL.Query()
if values, ok := query["genre"]; ok {
options.genre = values[len(values)-1]
}
if options.genre != "drama" && options.genre != "variety" {
return newerOptions{}, fmt.Errorf("invalid genre")
}
if values, ok := query["hide_previews"]; ok {
switch values[len(values)-1] {
case "0":
options.hidePreviews = false
case "1":
options.hidePreviews = true
default:
return newerOptions{}, fmt.Errorf("invalid hide_previews")
}
}
return options, nil
}
func (options newerOptions) pageURL() string {
query := url.Values{}
query.Set("genre", options.genre)
if options.hidePreviews {
query.Set("hide_previews", "1")
} else {
query.Set("hide_previews", "0")
}
return "/?" + query.Encode()
}
func (s *server) home(w http.ResponseWriter, r *http.Request) {
options, err := parseNewerOptions(r)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
view := s.newerView(r.Context(), options)
s.render(w, "layout", page{Title: "新着", Newer: &view})
}
func (s *server) newerFragment(w http.ResponseWriter, r *http.Request) {
options, err := parseNewerOptions(r)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
view := s.newerView(r.Context(), options)
w.Header().Set("HX-Push-Url", options.pageURL())
s.render(w, "newer_results", view)
}
func (s *server) render(w http.ResponseWriter, name string, value any) {
var body bytes.Buffer
if err := s.tmpl.ExecuteTemplate(&body, name, value); err != nil {
s.logger.Printf("テンプレートの描画に失敗: %v", err)
http.Error(w, "template error", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = body.WriteTo(w)
}
func (s *server) newerView(ctx context.Context, options newerOptions) newerView {
view := newerView{Genre: options.genre, HidePreviews: options.hidePreviews}
episodes, err := s.cachedNewer(ctx, options.genre)
if err != nil {
s.logger.Printf("TVer の新着取得に失敗（genre=%s）: %s", options.genre, platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
view.Failed = true
view.ErrorText = "新着を取得できませんでした"
return view
}
visible := make([]tver.Episode, 0, len(episodes))
for _, episode := range episodes {
if options.hidePreviews && episode.Duration <= tver.PreviewMaxSeconds {
continue
}
visible = append(visible, episode)
}
ids := make([]string, 0, len(visible))
for _, episode := range visible {
ids = append(ids, episode.ID)
}
statuses, err := s.jobs.EpisodeStatuses(ctx, ids)
if err != nil {
s.logger.Printf("新着のジョブ状態取得に失敗: %v", err)
view.Failed = true
view.ErrorText = "状態を取得できませんでした"
return view
}
seriesIDs := make([]string, 0, len(visible))
seenSeries := make(map[string]bool)
for _, episode := range visible {
if episode.SeriesID != "" && !seenSeries[episode.SeriesID] {
seriesIDs = append(seriesIDs, episode.SeriesID)
seenSeries[episode.SeriesID] = true
}
}
subscribed, err := s.jobs.SubscribedSeries(ctx, seriesIDs)
if err != nil {
s.logger.Printf("登録番組の状態取得に失敗: %v", err)
view.Failed = true
view.ErrorText = "登録番組の状態を取得できませんでした"
return view
}
for _, episode := range visible {
endAt := time.Unix(episode.EndAt, 0).In(s.zone)
view.Episodes = append(view.Episodes, episodeView{
ID: episode.ID,
SeriesID: episode.SeriesID,
SubscriptionAction: subscriptionActionView{ID: episode.ID, Genre: options.genre, Subscribed: subscribed[episode.SeriesID]},
Genre: options.genre,
Status: statuses[episode.ID],
StatusText: statusText(statuses[episode.ID]),
URL: "https://tver.jp/episodes/" + url.PathEscape(episode.ID),
ThumbnailURL: "https://statics.tver.jp" + episode.ThumbnailPath,
SeriesTitle: episode.SeriesTitle,
Title: episode.Title,
BroadcasterName: episode.BroadcasterName,
DurationMinutes: durationMinutes(episode.Duration),
BroadcastDateLabel: episode.BroadcastDateLabel,
EndAtText: endAt.Format("2006-01-02 15:04 MST"),
EndAtISO: endAt.Format(time.RFC3339),
})
}
return view
}
func (s *server) cachedEpisode(genre, id string) (tver.Episode, bool) {
s.cacheMu.Lock()
defer s.cacheMu.Unlock()
entry, ok := s.cache[genre]
if !ok || !s.now().Before(entry.expiresAt) {
return tver.Episode{}, false
}
for _, ep := range entry.episodes {
if ep.ID == id {
return ep, true
}
}
return tver.Episode{}, false
}
func storeEpisode(ep tver.Episode) store.Episode {
var endAt *time.Time
if ep.EndAt > 0 {
end := time.Unix(ep.EndAt, 0)
endAt = &end
}
return store.Episode{ID: ep.ID, SeriesID: ep.SeriesID, SeriesTitle: ep.SeriesTitle,
Title: ep.Title, BroadcasterName: ep.BroadcasterName, DurationSeconds: int(ep.Duration),
EndAt: endAt, ThumbnailPath: ep.ThumbnailPath}
}
func durationMinutes(seconds int64) int64 {
minutes := seconds / 60
if seconds%60 != 0 {
minutes++
}
return max(1, minutes)
}
func (s *server) cachedNewer(ctx context.Context, genre string) ([]tver.Episode, error) {
s.cacheMu.Lock()
defer s.cacheMu.Unlock()
if cached, ok := s.cache[genre]; ok && s.now().Before(cached.expiresAt) {
return cached.episodes, nil
}
episodes, err := s.fetcher.FetchNewer(ctx, genre)
if err != nil {
return nil, err
}
s.cache[genre] = cacheEntry{episodes: episodes, expiresAt: s.now().Add(newerCacheTTL)}
return episodes, nil
}
internal/web/newer_test.go
+263
View File
@@ -0,0 +1,263 @@
package web
import (
"bytes"
"context"
"errors"
"io"
"log"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
type fakeFetcher struct {
episodes []tver.Episode
episodesByGenre map[string][]tver.Episode
err error
calls []string
seriesEpisodes []tver.Episode
seriesErr error
seriesCalls []string
}
func (f *fakeFetcher) FetchNewer(_ context.Context, genre string) ([]tver.Episode, error) {
f.calls = append(f.calls, genre)
if f.episodesByGenre != nil {
return f.episodesByGenre[genre], f.err
}
return f.episodes, f.err
}
func (f *fakeFetcher) FetchSeriesEpisodes(_ context.Context, seriesID string) ([]tver.Episode, error) {
f.seriesCalls = append(f.seriesCalls, seriesID)
return f.seriesEpisodes, f.seriesErr
}
func newTestHandler(fetcher *fakeFetcher, now func() time.Time) http.Handler {
return newTestHandlerWithStore(fetcher, &fakeJobStore{}, now)
}
func newTestHandlerWithStore(fetcher *fakeFetcher, jobs *fakeJobStore, now func() time.Time) http.Handler {
return newTestHandlerWithVideoDir(fetcher, jobs, "", now)
}
func newTestHandlerWithVideoDir(fetcher *fakeFetcher, jobs *fakeJobStore, videoDir string, now func() time.Time) http.Handler {
if now == nil {
now = func() time.Time { return time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC) }
}
return NewHandler(fetcher, jobs, videoDir, now, log.New(io.Discard, "", 0))
}
func getPage(handler http.Handler, path string) *httptest.ResponseRecorder {
response := httptest.NewRecorder()
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, path, nil))
return response
}
func TestNewerPageShowsEpisodeFieldsAndJST(t *testing.T) {
endAt := time.Date(2026, 10, 3, 21, 34, 0, 0, time.UTC).Unix()
fetcher := &fakeFetcher{episodes: []tver.Episode{{
ID: "ep123", SeriesTitle: "番組名", Title: "第1話", BroadcasterName: "放送局",
Duration: 2712, EndAt: endAt, ThumbnailPath: "/images/ep123.jpg",
BroadcastDateLabel: "10月3日放送分",
}}}
response := getPage(newTestHandler(fetcher, nil), "/")
if response.Code != http.StatusOK {
t.Fatalf("status = %d", response.Code)
}
body := response.Body.String()
for _, want := range []string{
"番組名", "第1話", "放送局", "10月3日放送分",
"長さ: 46分",
"https://statics.tver.jp/images/ep123.jpg",
"https://tver.jp/episodes/ep123",
"2026-10-04 06:34 JST", `datetime="2026-10-04T06:34:00&#43;09:00"`,
`value="drama" selected`, `name="hide_previews" value="1" checked`,
`hx-get="/newer"`, `hx-target="#newer-results"`,
} {
if !strings.Contains(body, want) {
t.Errorf("page missing %q", want)
}
}
if len(fetcher.calls) != 1 || fetcher.calls[0] != "drama" {
t.Errorf("calls = %v", fetcher.calls)
}
}
func TestDurationMinutesRoundsUp(t *testing.T) {
for _, tt := range []struct {
seconds int64
minutes int64
}{
{0, 1}, {1, 1}, {59, 1}, {60, 1}, {61, 2}, {2712, 46},
} {
if got := durationMinutes(tt.seconds); got != tt.minutes {
t.Errorf("%d seconds = %d minutes; want %d", tt.seconds, got, tt.minutes)
}
}
}
func TestNewerFragmentUpdatesURLAndFullPageRestoresState(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "variety-1", Duration: 200}}}
handler := newTestHandler(fetcher, nil)
response := getPage(handler, "/newer?genre=variety&hide_previews=0")
if response.Code != http.StatusOK {
t.Fatalf("fragment status = %d", response.Code)
}
if got := response.Header().Get("HX-Push-Url"); got != "/?genre=variety&hide_previews=0" {
t.Errorf("HX-Push-Url = %q", got)
}
if body := response.Body.String(); !strings.Contains(body, `id="newer-results"`) || !strings.Contains(body, "variety-1") || strings.Contains(body, "<html") {
t.Errorf("not an episode fragment: %s", body)
}
reload := getPage(handler, "/?genre=variety&hide_previews=0")
if reload.Code != http.StatusOK || !strings.Contains(reload.Body.String(), `value="variety" selected`) || strings.Contains(reload.Body.String(), `value="1" checked`) || !strings.Contains(reload.Body.String(), "variety-1") {
t.Errorf("reload did not restore state: status=%d body=%s", reload.Code, reload.Body.String())
}
if len(fetcher.calls) != 1 || fetcher.calls[0] != "variety" {
t.Errorf("calls = %v", fetcher.calls)
}
}
func TestNewerInvalidGenreReturns400WithoutFetch(t *testing.T) {
fetcher := &fakeFetcher{}
handler := newTestHandler(fetcher, nil)
for _, path := range []string{"/?genre=news", "/newer?genre=news", "/?genre=", "/?hide_previews=invalid", "/newer?hide_previews="} {
response := getPage(handler, path)
if response.Code != http.StatusBadRequest {
t.Errorf("%s: status = %d", path, response.Code)
}
}
if len(fetcher.calls) != 0 {
t.Errorf("unexpected fetch calls: %v", fetcher.calls)
}
}
func TestPreviewFilterBoundaryAndToggle(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{
{ID: "preview-180", Duration: 180},
{ID: "episode-181", Duration: 181},
}}
handler := newTestHandler(fetcher, nil)
hidden := getPage(handler, "/")
if strings.Contains(hidden.Body.String(), "preview-180") || !strings.Contains(hidden.Body.String(), "episode-181") {
t.Errorf("default filter: %s", hidden.Body.String())
}
shown := getPage(handler, "/newer?genre=drama&hide_previews=0")
if !strings.Contains(shown.Body.String(), "preview-180") || !strings.Contains(shown.Body.String(), "episode-181") {
t.Errorf("show previews: %s", shown.Body.String())
}
checked := getPage(handler, "/newer?genre=drama&hide_previews=0&hide_previews=1")
if strings.Contains(checked.Body.String(), "preview-180") || checked.Header().Get("HX-Push-Url") != "/?genre=drama&hide_previews=1" {
t.Errorf("checked form state: %s", checked.Body.String())
}
if len(fetcher.calls) != 1 {
t.Errorf("filter caused %d TVer fetches; want 1", len(fetcher.calls))
}
}
func TestNewerCacheExpiresPerGenre(t *testing.T) {
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}
handler := newTestHandler(fetcher, func() time.Time { return now })
getPage(handler, "/")
now = now.Add(9*time.Minute + 59*time.Second)
getPage(handler, "/?genre=drama&hide_previews=0")
getPage(handler, "/?genre=variety")
if got := strings.Join(fetcher.calls, ","); got != "drama,variety" {
t.Fatalf("before expiry calls = %q", got)
}
now = now.Add(2 * time.Second)
getPage(handler, "/")
getPage(handler, "/?genre=variety")
if got := strings.Join(fetcher.calls, ","); got != "drama,variety,drama" {
t.Errorf("after expiry calls = %q", got)
}
}
func TestNewerCacheKeepsGenreResultsSeparateAndExpiresAtTenMinutes(t *testing.T) {
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
fetcher := &fakeFetcher{episodesByGenre: map[string][]tver.Episode{
"drama": {{ID: "drama-only", Duration: 200}},
"variety": {{ID: "variety-only", Duration: 200}},
}}
handler := newTestHandler(fetcher, func() time.Time { return now })
checkPage := func(path, wantID, absentID string) {
t.Helper()
response := getPage(handler, path)
body := response.Body.String()
if response.Code != http.StatusOK || !strings.Contains(body, "/episodes/"+wantID) || strings.Contains(body, "/episodes/"+absentID) {
t.Errorf("%s at %s: status=%d, wrong genre contents: %s", path, now.Format(time.RFC3339), response.Code, body)
}
}
checkCalls := func(want string) {
t.Helper()
if got := strings.Join(fetcher.calls, ","); got != want {
t.Errorf("fetch calls = %q; want %q", got, want)
}
}
checkPage("/", "drama-only", "variety-only")
now = now.Add(5 * time.Minute)
checkPage("/?genre=variety", "variety-only", "drama-only")
now = now.Add(4*time.Minute + 59*time.Second)
checkPage("/", "drama-only", "variety-only")
checkCalls("drama,variety")
now = now.Add(time.Second)
checkPage("/", "drama-only", "variety-only")
checkPage("/?genre=variety", "variety-only", "drama-only")
checkCalls("drama,variety,drama")
now = now.Add(5 * time.Minute)
checkPage("/?genre=variety", "variety-only", "drama-only")
checkCalls("drama,variety,drama,variety")
}
func TestNewerPageEscapesUntrustedEpisodeFields(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{
ID: "safe-id", SeriesTitle: `<script>alert(1)</script>`, Title: `<b>title</b>`,
BroadcasterName: `<img src=x onerror=alert(1)>`, BroadcastDateLabel: `<svg onload=alert(1)>`,
ThumbnailPath: `/image.jpg" onerror="alert(1)`, Duration: 200,
}}}
response := getPage(newTestHandler(fetcher, nil), "/")
if response.Code != http.StatusOK {
t.Fatalf("status = %d", response.Code)
}
body := response.Body.String()
for _, unsafe := range []string{"<script>alert(1)</script>", "<b>title</b>", "<img src=x", "<svg onload", `onerror="alert(1)"`} {
if strings.Contains(body, unsafe) {
t.Errorf("untrusted markup was not escaped: %q", unsafe)
}
}
for _, escaped := range []string{"&lt;script&gt;", "&lt;b&gt;title&lt;/b&gt;", "&lt;img src=x onerror=alert(1)&gt;", "&lt;svg onload=alert(1)&gt;"} {
if !strings.Contains(body, escaped) {
t.Errorf("escaped field missing: %q", escaped)
}
}
}
func TestNewerFailureShowsFriendlyMessageAndRedactsToken(t *testing.T) {
fetcher := &fakeFetcher{err: errors.New("GET https://platform-api.tver.jp/?platform_token=secret-123&x=1: upstream unavailable")}
var logs bytes.Buffer
handler := NewHandler(fetcher, &fakeJobStore{}, "", time.Now, log.New(&logs, "", 0))
response := getPage(handler, "/")
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "新着を取得できませんでした") {
t.Errorf("error page: status=%d body=%s", response.Code, response.Body.String())
}
if strings.Contains(response.Body.String(), "upstream") || strings.Contains(response.Body.String(), "secret-123") {
t.Errorf("error detail leaked into page: %s", response.Body.String())
}
fragment := getPage(handler, "/newer?genre=variety&hide_previews=0")
if fragment.Code != http.StatusOK || !strings.Contains(fragment.Body.String(), `id="newer-results"`) || !strings.Contains(fragment.Body.String(), "新着を取得できませんでした") || strings.Contains(fragment.Body.String(), "<html") {
t.Errorf("error fragment: status=%d body=%s", fragment.Code, fragment.Body.String())
}
if fragment.Header().Get("HX-Push-Url") != "/?genre=variety&hide_previews=0" || strings.Contains(fragment.Body.String(), "secret-123") {
t.Errorf("error fragment URL or token leak: header=%q body=%s", fragment.Header().Get("HX-Push-Url"), fragment.Body.String())
}
if !strings.Contains(logs.String(), "upstream unavailable") || strings.Contains(logs.String(), "secret-123") {
t.Errorf("log detail or token redaction failed: %s", logs.String())
}
}
internal/web/queue.go
+194
View File
@@ -0,0 +1,194 @@
package web
import (
"context"
"errors"
"fmt"
"net/http"
"strconv"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
const queueListLimit = 50
type queueJobView struct {
store.Job
SourceText, CreatedAtText, PhaseText string
PercentText, SpeedText, ETAText string
FragmentText, ErrorKindText string
}
type queueView struct {
Running, Queued, Failed, Canceled, Succeeded []queueJobView
Poll bool
}
func (s *server) queuePage(w http.ResponseWriter, r *http.Request) {
view, err := s.loadQueue(r.Context())
if err != nil {
s.logger.Printf("待ち行列の取得に失敗: %v", err)
http.Error(w, "待ち行列を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "layout", page{Title: "待ち行列", Queue: &view})
}
func (s *server) queueFragment(w http.ResponseWriter, r *http.Request) {
view, err := s.loadQueue(r.Context())
if err != nil {
s.logger.Printf("待ち行列の取得に失敗: %v", err)
http.Error(w, "待ち行列を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "queue_list", view)
}
func (s *server) loadQueue(ctx context.Context) (queueView, error) {
var view queueView
for _, item := range []struct {
status string
assign func([]queueJobView)
}{
{"running", func(j []queueJobView) { view.Running = j }},
{"queued", func(j []queueJobView) { view.Queued = j }},
{"failed", func(j []queueJobView) { view.Failed = j }},
{"canceled", func(j []queueJobView) { view.Canceled = j }},
{"succeeded", func(j []queueJobView) { view.Succeeded = j }},
} {
jobs, err := s.jobs.ListJobs(ctx, item.status, queueListLimit)
if err != nil {
return queueView{}, err
}
views := make([]queueJobView, 0, len(jobs))
for _, job := range jobs {
views = append(views, s.queueJob(job))
}
item.assign(views)
}
view.Poll = len(view.Running) > 0 || len(view.Queued) > 0
return view, nil
}
func (s *server) queueJob(job store.Job) queueJobView {
view := queueJobView{Job: job, CreatedAtText: job.CreatedAt.In(s.zone).Format("2006-01-02 15:04 MST")}
if job.Source == "subscription" {
view.SourceText = "自動"
} else {
view.SourceText = "手動"
}
if job.Phase != nil {
switch *job.Phase {
case "video":
view.PhaseText = "映像"
case "audio":
view.PhaseText = "音声"
case "merging":
view.PhaseText = "結合"
case "moving":
view.PhaseText = "移動"
}
}
if job.DownloadedBytes != nil && job.TotalBytes != nil && *job.TotalBytes > 0 {
percent := 100 * float64(*job.DownloadedBytes) / float64(*job.TotalBytes)
percent = max(0, min(percent, 100))
if job.TotalIsEstimate {
view.PercentText = fmt.Sprintf("約%.0f%%", percent)
} else {
view.PercentText = fmt.Sprintf("%.0f%%", percent)
}
}
if job.SpeedBPS != nil && *job.SpeedBPS > 0 {
view.SpeedText = formatBytes(*job.SpeedBPS) + "/秒"
}
if job.ETASeconds != nil && *job.ETASeconds >= 0 {
seconds := int64(*job.ETASeconds)
view.ETAText = fmt.Sprintf("残り%d分%d秒", seconds/60, seconds%60)
}
if job.FragmentIndex != nil && job.FragmentCount != nil && *job.FragmentCount > 0 {
view.FragmentText = fmt.Sprintf("断片 %d/%d", *job.FragmentIndex, *job.FragmentCount)
}
if job.ErrorKind != nil {
view.ErrorKindText = errorKindText(*job.ErrorKind)
}
return view
}
func formatBytes(n float64) string {
for _, unit := range []string{"B", "KiB", "MiB", "GiB"} {
if n < 1024 || unit == "GiB" {
return fmt.Sprintf("%.1f %s", n, unit)
}
n /= 1024
}
return ""
}
func errorKindText(kind string) string {
switch kind {
case "drm":
return "DRM のため取得不可"
case "geo":
return "地域制限"
case "network":
return "ネットワーク"
case "unavailable":
return "配信終了・存在しない"
case "extract":
return "抽出失敗"
case "invalid_output":
return "出力形式の不正"
default:
return "不明"
}
}
func (s *server) retryJob(w http.ResponseWriter, r *http.Request) {
s.queueMutation(w, r, s.jobs.Retry)
}
func (s *server) cancelJob(w http.ResponseWriter, r *http.Request) {
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.Error(w, "ジョブ ID が不正です", http.StatusBadRequest)
return
}
job, err := s.jobs.GetJob(r.Context(), id)
if errors.Is(err, pgx.ErrNoRows) {
http.Error(w, "ジョブが見つかりません", http.StatusNotFound)
return
}
if err != nil {
s.logger.Printf("ジョブ状態の取得に失敗: %v", err)
http.Error(w, "操作できませんでした", http.StatusInternalServerError)
return
}
if job.Status != "queued" && job.Status != "running" {
http.Error(w, "待機中またはダウンロード中のジョブだけ取り消せます", http.StatusConflict)
return
}
s.queueMutation(w, r, s.jobs.Cancel)
}
func (s *server) queueMutation(w http.ResponseWriter, r *http.Request, change func(context.Context, int64) error) {
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
if err != nil || id <= 0 {
http.Error(w, "ジョブ ID が不正です", http.StatusBadRequest)
return
}
if err := change(r.Context(), id); err != nil {
if errors.Is(err, store.ErrInvalidTransition) {
http.Error(w, "操作できる状態ではありません", http.StatusConflict)
} else {
s.logger.Printf("待ち行列の操作に失敗: %v", err)
http.Error(w, "操作できませんでした", http.StatusInternalServerError)
}
return
}
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/queue", http.StatusSeeOther)
return
}
s.queueFragment(w, r)
}
internal/web/queue_test.go
+350
View File
@@ -0,0 +1,350 @@
package web
import (
"errors"
"net/http"
"net/http/httptest"
"net/url"
"regexp"
"slices"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
func postForm(handler http.Handler, path string, values url.Values) *httptest.ResponseRecorder {
r := httptest.NewRequest(http.MethodPost, path, strings.NewReader(values.Encode()))
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
r.Header.Set("HX-Request", "true")
w := httptest.NewRecorder()
handler.ServeHTTP(w, r)
return w
}
func TestEnqueueUsesCachedEpisodeAndDuplicateStatus(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "series1", SeriesTitle: "正しい番組", Title: "正しい回", BroadcasterName: "正しい局", Duration: 1234, EndAt: 1791061200, ThumbnailPath: "/correct.jpg"}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
getPage(handler, "/")
if jobs.statusCalls != 1 || len(jobs.statusIDs) != 1 || jobs.statusIDs[0] != "ep1" {
t.Fatalf("状態の一括取得: calls=%d ids=%v", jobs.statusCalls, jobs.statusIDs)
}
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}, "series_title": {"偽番組"}, "title": {"偽回"}}
response := postForm(handler, "/episodes/enqueue", values)
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "待機中") || !strings.Contains(response.Body.String(), `class="episode-action"`) || strings.Contains(response.Body.String(), "<html") || strings.Contains(response.Body.String(), "episode-card") {
t.Fatalf("登録応答: %d %s", response.Code, response.Body.String())
}
if jobs.lastEpisode.SeriesTitle != "正しい番組" || jobs.lastEpisode.Title != "正しい回" || jobs.lastEpisode.BroadcasterName != "正しい局" || jobs.lastEpisode.DurationSeconds != 1234 || jobs.lastEpisode.ThumbnailPath != "/correct.jpg" || jobs.lastEpisode.EndAt == nil {
t.Fatalf("キャッシュ外の情報を使用: %+v", jobs.lastEpisode)
}
if len(fetcher.calls) != 1 {
t.Fatalf("POST が TVer に再問い合わせ: %v", fetcher.calls)
}
duplicate := postForm(handler, "/episodes/enqueue", values)
if duplicate.Code != http.StatusOK || !strings.Contains(duplicate.Body.String(), "待機中") || len(jobs.jobs) != 1 {
t.Fatalf("二重登録: %d %s jobs=%d", duplicate.Code, duplicate.Body.String(), len(jobs.jobs))
}
page := getPage(handler, "/")
if !strings.Contains(page.Body.String(), "待機中") || strings.Contains(page.Body.String(), ">ダウンロード</button>") {
t.Fatalf("登録後の一覧: %s", page.Body.String())
}
}
func TestEnqueueRejectsInvalidOrExpiredCache(t *testing.T) {
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", Duration: 200}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, func() time.Time { return now })
valid := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
if got := postForm(handler, "/episodes/enqueue", valid).Code; got != http.StatusNotFound {
t.Fatalf("未取得キャッシュ: %d", got)
}
getPage(handler, "/")
for _, values := range []url.Values{
{"genre": {"news"}, "episode_id": {"ep1"}},
{"genre": {"drama"}},
} {
if got := postForm(handler, "/episodes/enqueue", values).Code; got != http.StatusBadRequest {
t.Errorf("不正なフォーム: %d", got)
}
}
if got := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"drama"}, "episode_id": {"unknown"}}).Code; got != http.StatusNotFound {
t.Fatalf("同じジャンルの未収録 ID: %d", got)
}
if got := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"variety"}, "episode_id": {"ep1"}}).Code; got != http.StatusNotFound {
t.Fatalf("別ジャンルのキャッシュ: %d", got)
}
now = now.Add(newerCacheTTL)
if got := postForm(handler, "/episodes/enqueue", valid).Code; got != http.StatusNotFound {
t.Fatalf("期限切れキャッシュ: %d", got)
}
if jobs.enqueueCalls != 0 || len(fetcher.calls) != 1 {
t.Fatalf("無効な入力で登録・再取得: enqueue=%d fetch=%v", jobs.enqueueCalls, fetcher.calls)
}
}
func TestEnqueueUsesSelectedGenreCacheForSameEpisodeID(t *testing.T) {
fetcher := &fakeFetcher{episodesByGenre: map[string][]tver.Episode{
"drama": {{ID: "shared", SeriesTitle: "ドラマの番組", Duration: 200}},
"variety": {{ID: "shared", SeriesTitle: "バラエティの番組", Duration: 200}},
}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
getPage(handler, "/")
getPage(handler, "/?genre=variety")
response := postForm(handler, "/episodes/enqueue", url.Values{
"genre": {"variety"}, "episode_id": {"shared"}, "series_title": {"偽タイトル"},
})
if response.Code != http.StatusOK || jobs.lastEpisode.SeriesTitle != "バラエティの番組" || jobs.enqueueCalls != 1 || !slices.Equal(fetcher.calls, []string{"drama", "variety"}) {
t.Fatalf("選択ジャンルのキャッシュ: status=%d episode=%+v calls=%v", response.Code, jobs.lastEpisode, fetcher.calls)
}
}
func TestNewerStatusesAndRetry(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "failed", Duration: 200}, {ID: "canceled", Duration: 200}, {ID: "running", Duration: 200}, {ID: "done", Duration: 200}}}
jobs := &fakeJobStore{jobs: map[string]store.Job{
"failed": {ID: 1, Status: "failed"}, "canceled": {ID: 2, Status: "canceled"},
"running": {ID: 3, Status: "running"}, "done": {ID: 4, Status: "succeeded"},
}}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
body := getPage(handler, "/").Body.String()
for _, want := range []string{"失敗", "取り消し", "ダウンロード中", "完了"} {
if !strings.Contains(body, want) {
t.Errorf("状態表示がない: %s", want)
}
}
if got := strings.Count(body, ">再試行</button>"); got != 2 {
t.Fatalf("再試行ボタン数: %d", got)
}
response := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"failed"}})
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "待機中") || jobs.jobs["failed"].Status != "queued" {
t.Fatalf("新着から再試行: %d %s", response.Code, response.Body.String())
}
if got := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"running"}}).Code; got != http.StatusConflict {
t.Fatalf("実行中の再試行: %d", got)
}
if got := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"missing"}}).Code; got != http.StatusNotFound {
t.Fatalf("キャッシュにない ID の再試行: %d", got)
}
}
func TestQueueGroupsProgressAndPolling(t *testing.T) {
phase, errorKind, errorMessage := "video", "drm", `<script>alert(1)</script>`
downloaded, total := int64(25), int64(100)
speed, eta := float64(2048), float64(63)
index, count := int32(2), int32(10)
created := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
jobs := &fakeJobStore{jobs: map[string]store.Job{
"running": {ID: 1, Status: "running", Source: "subscription", SeriesTitle: "番組A", Title: "第1話", BroadcasterName: "局A", CreatedAt: created, Phase: &phase, DownloadedBytes: &downloaded, TotalBytes: &total, TotalIsEstimate: true, SpeedBPS: &speed, ETASeconds: &eta, FragmentIndex: &index, FragmentCount: &count},
"queued": {ID: 2, Status: "queued", Source: "manual", SeriesTitle: "番組B", CreatedAt: created},
"failed": {ID: 3, Status: "failed", ErrorKind: &errorKind, ErrorMessage: &errorMessage, CreatedAt: created},
"canceled": {ID: 4, Status: "canceled", CreatedAt: created},
"done": {ID: 5, Status: "succeeded", CreatedAt: created},
}}
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
response := getPage(handler, "/queue")
if response.Code != http.StatusOK {
t.Fatalf("待ち行列: %d", response.Code)
}
body := response.Body.String()
for _, want := range []string{"ダウンロード中", "待機中", "失敗", "取り消し", "完了（直近50件）", "番組A", "第1話", "局A", "自動", "手動", "2026-10-03 09:00 JST", "映像", "約25%", "2.0 KiB/秒", "残り1分3秒", "断片 2/10", "DRM のため取得不可", "<details>", "&lt;script&gt;", `hx-get="/queue/list"`, `hx-trigger="every 5s"`} {
if !strings.Contains(body, want) {
t.Errorf("待ち行列に %q がない", want)
}
}
if strings.Contains(body, errorMessage) || !strings.Contains(body, `action="/jobs/1/cancel"`) || strings.Contains(body, "全体") {
t.Errorf("不正な詳細・操作・全体進捗: %s", body)
}
fragment := getPage(handler, "/queue/list")
if fragment.Code != http.StatusOK || strings.Contains(fragment.Body.String(), "<html") || !strings.Contains(fragment.Body.String(), `id="queue-list"`) {
t.Errorf("待ち行列断片: %d %s", fragment.Code, fragment.Body.String())
}
}
func TestQueueJobsAppearInCorrectSectionsWithLimit(t *testing.T) {
statuses := []string{"running", "queued", "failed", "canceled", "succeeded"}
labels := []string{"ダウンロード中", "待機中", "失敗", "取り消し", "完了"}
jobs := &fakeJobStore{jobs: make(map[string]store.Job)}
for i, status := range statuses {
jobs.jobs[status] = store.Job{ID: int64(i + 1), Status: status, SeriesTitle: "番組-" + status}
}
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, jobs, nil), "/queue")
if response.Code != http.StatusOK {
t.Fatalf("待ち行列: %d", response.Code)
}
if !slices.Equal(jobs.listStatuses, statuses) {
t.Errorf("状態別取得: %v", jobs.listStatuses)
}
if !slices.Equal(jobs.listLimits, []int{50, 50, 50, 50, 50}) {
t.Errorf("状態ごとの取得上限: %v", jobs.listLimits)
}
for i, label := range labels {
section := regexp.MustCompile(`(?s)<section aria-label="` + label + `">(.*?)</section>`).FindStringSubmatch(response.Body.String())
if len(section) != 2 || !strings.Contains(section[1], "番組-"+statuses[i]) {
t.Errorf("%s に %s がない: %v", label, statuses[i], section)
continue
}
for j, other := range statuses {
if j != i && strings.Contains(section[1], "番組-"+other) {
t.Errorf("%s に %s が混入", label, other)
}
}
cancelAction := `action="/jobs/` + string(rune('1'+i)) + `/cancel"`
retryAction := `action="/jobs/` + string(rune('1'+i)) + `/retry"`
switch statuses[i] {
case "running", "queued":
if !strings.Contains(section[1], cancelAction) || strings.Contains(section[1], retryAction) {
t.Errorf("%s の操作が不正: %s", statuses[i], section[1])
}
case "failed", "canceled":
if !strings.Contains(section[1], retryAction) || strings.Contains(section[1], cancelAction) {
t.Errorf("%s の操作が不正: %s", statuses[i], section[1])
}
default:
if strings.Contains(section[1], "/cancel") || strings.Contains(section[1], "/retry") {
t.Errorf("%s に操作がある: %s", statuses[i], section[1])
}
}
}
}
func TestQueueEscapesEpisodeFields(t *testing.T) {
attack := `<script>alert(1)</script>`
message := `<img src=x onerror=alert(1)>`
kind := "extract"
jobs := &fakeJobStore{jobs: map[string]store.Job{
"bad": {ID: 1, Status: "failed", SeriesTitle: attack, Title: attack,
BroadcasterName: attack, ErrorKind: &kind, ErrorMessage: &message},
}}
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, jobs, nil), "/queue")
body := response.Body.String()
if response.Code != http.StatusOK || strings.Contains(body, attack) || strings.Contains(body, message) || strings.Count(body, "&lt;script&gt;") != 3 || !strings.Contains(body, "&lt;img src=x onerror=alert(1)&gt;") {
t.Fatalf("待ち行列の HTML エスケープ: %d %s", response.Code, body)
}
}
func TestQueueCancelRetryAndStopPolling(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep": {ID: 1, Status: "queued"}}}
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
if !strings.Contains(getPage(handler, "/queue").Body.String(), `hx-get="/queue/list"`) {
t.Fatal("待機中なのに更新がない")
}
canceled := postForm(handler, "/jobs/1/cancel", nil)
if canceled.Code != http.StatusOK || !strings.Contains(canceled.Body.String(), `action="/jobs/1/retry"`) || strings.Contains(canceled.Body.String(), `hx-get="/queue/list"`) {
t.Fatalf("取り消し後: %d %s", canceled.Code, canceled.Body.String())
}
retried := postForm(handler, "/jobs/1/retry", nil)
if retried.Code != http.StatusOK || !strings.Contains(retried.Body.String(), `action="/jobs/1/cancel"`) || !strings.Contains(retried.Body.String(), `hx-get="/queue/list"`) {
t.Fatalf("再試行後: %d %s", retried.Code, retried.Body.String())
}
if got := postForm(handler, "/jobs/1/retry", nil).Code; got != http.StatusConflict {
t.Fatalf("待機中の再試行: %d", got)
}
if got := postForm(handler, "/jobs/bad/cancel", nil).Code; got != http.StatusBadRequest {
t.Fatalf("不正な ID: %d", got)
}
if got := postForm(handler, "/jobs/99/cancel", nil).Code; got != http.StatusNotFound {
t.Fatalf("存在しないジョブ: %d", got)
}
jobs.jobs["ep"] = store.Job{ID: 1, Status: "running"}
runningCanceled := postForm(handler, "/jobs/1/cancel", nil)
if runningCanceled.Code != http.StatusOK || jobs.jobs["ep"].Status != "canceled" || !strings.Contains(runningCanceled.Body.String(), `action="/jobs/1/retry"`) {
t.Fatalf("実行中の取り消し: %d %+v", runningCanceled.Code, jobs.jobs["ep"])
}
}
func TestStateChangingGETIsRejected(t *testing.T) {
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep": {ID: 1, Status: "queued"}}}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
getPage(handler, "/")
for _, path := range []string{
"/episodes/enqueue?episode_id=ep&genre=drama",
"/episodes/retry?episode_id=ep&genre=drama",
"/jobs/1/cancel", "/jobs/1/retry",
} {
response := getPage(handler, path)
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
t.Errorf("GET %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
}
}
if jobs.jobs["ep"].Status != "queued" || jobs.enqueueCalls != 0 {
t.Fatalf("GET が状態を変更した: jobs=%+v enqueue=%d", jobs.jobs, jobs.enqueueCalls)
}
}
func TestStateChangingPOSTWithoutHtmxRedirects(t *testing.T) {
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
getPage(handler, "/")
form := url.Values{"episode_id": {"ep"}, "genre": {"drama"}}
request := httptest.NewRequest(http.MethodPost, "/episodes/enqueue", strings.NewReader(form.Encode()))
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/?genre=drama" || jobs.jobs["ep"].Status != "queued" {
t.Fatalf("通常 POST 後の遷移: status=%d Location=%q jobs=%+v", response.Code, response.Header().Get("Location"), jobs.jobs)
}
request = httptest.NewRequest(http.MethodPost, "/jobs/1/cancel", nil)
response = httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/queue" || jobs.jobs["ep"].Status != "canceled" {
t.Fatalf("待ち行列操作後の遷移: status=%d Location=%q jobs=%+v", response.Code, response.Header().Get("Location"), jobs.jobs)
}
}
func TestNewerStoreFailureDoesNotOfferDownload(t *testing.T) {
jobs := &fakeJobStore{err: errors.New("db unavailable")}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
response := getPage(handler, "/")
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "状態を取得できませんでした") || strings.Contains(response.Body.String(), ">ダウンロード</button>") {
t.Errorf("状態取得失敗: %d %s", response.Code, response.Body.String())
}
}
func TestEnqueueStoreFailureShowsShortError(t *testing.T) {
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
getPage(handler, "/")
jobs.err = errors.New("secret database detail")
response := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"drama"}, "episode_id": {"ep"}})
if response.Code != http.StatusInternalServerError || !strings.Contains(response.Body.String(), "待ち行列に登録できませんでした") || strings.Contains(response.Body.String(), "secret") {
t.Errorf("DB エラーの表示: %d %s", response.Code, response.Body.String())
}
}
func TestQueueProgressAndFailureLabels(t *testing.T) {
s := &server{zone: time.FixedZone("JST", 9*60*60)}
for _, tt := range []struct{ kind, label string }{
{"drm", "DRM のため取得不可"}, {"geo", "地域制限"},
{"network", "ネットワーク"}, {"unavailable", "配信終了・存在しない"},
{"extract", "抽出失敗"}, {"invalid_output", "出力形式の不正"},
{"unknown", "不明"},
} {
if got := errorKindText(tt.kind); got != tt.label {
t.Errorf("%s: %q", tt.kind, got)
}
}
for _, tt := range []struct{ phase, label string }{
{"video", "映像"}, {"audio", "音声"}, {"merging", "結合"}, {"moving", "移動"},
} {
if got := s.queueJob(store.Job{Phase: &tt.phase}).PhaseText; got != tt.label {
t.Errorf("%s: %q", tt.phase, got)
}
}
downloaded, total := int64(50), int64(100)
job := store.Job{DownloadedBytes: &downloaded, TotalBytes: &total}
if got := s.queueJob(job).PercentText; got != "50%" {
t.Errorf("確定した段階の割合: %q", got)
}
job.TotalIsEstimate = true
if got := s.queueJob(job).PercentText; got != "約50%" {
t.Errorf("推定した段階の割合: %q", got)
}
total = 0
if got := s.queueJob(job).PercentText; got != "" {
t.Errorf("総量ゼロでの割合: %q", got)
}
}
internal/web/static/app.css
+27
View File
@@ -0,0 +1,27 @@
body { font-family: system-ui, sans-serif; margin: 0; color: #17212b; }
header { display: flex; align-items: center; gap: 2rem; padding: 1rem 2rem; border-bottom: 1px solid #d6dee4; }
nav { display: flex; flex-wrap: wrap; gap: 1.5rem; }
nav a { color: #145a86; }
main { max-width: 72rem; margin: 2rem auto; padding: 0 2rem; }
#newer-controls { display: flex; align-items: center; flex-wrap: wrap; gap: 1rem; margin-bottom: 2rem; }
#newer-controls select, #newer-controls button { font: inherit; padding: .35rem .6rem; }
.episode-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(17rem, 1fr)); gap: 1.25rem; }
.episode-card { border: 1px solid #d6dee4; border-radius: .5rem; overflow: hidden; }
.episode-card a { color: inherit; text-decoration: none; }
.episode-card a:hover h2 { text-decoration: underline; }
.episode-card img { display: block; width: 100%; aspect-ratio: 16 / 9; object-fit: cover; background: #e5ebef; }
.episode-copy { padding: .8rem 1rem; }
.episode-copy h2 { font-size: 1.1rem; margin: 0 0 .4rem; }
.episode-copy p { margin: .3rem 0; }
.episode-action { padding: 0 .8rem .8rem; }
.subscription-action { padding: 0 .8rem .8rem; }
.subscription-action button, .subscription-item button { font: inherit; padding: .35rem .7rem; }
.subscription-item { border: 1px solid #d6dee4; border-radius: .5rem; margin: .75rem 0; padding: .8rem 1rem; }
.subscription-item form { display: inline-block; margin-right: .5rem; }
.episode-action form { display: inline; }
.episode-action button, .queue-job button { font: inherit; padding: .35rem .7rem; }
.queue-job { border: 1px solid #d6dee4; border-radius: .5rem; margin: .75rem 0; padding: .8rem 1rem; }
.queue-job h3 { margin: 0 0 .4rem; }
.queue-job p { margin: .4rem 0; }
.queue-job pre { white-space: pre-wrap; overflow-wrap: anywhere; }
video { display: block; width: 100%; max-height: 75vh; background: #000; }
internal/web/static/htmx-2.0.11.min.js
Vendored
+1
View File
File diff suppressed because one or more lines are too long
internal/web/static/htmx-LICENSE.txt
+13
View File
@@ -0,0 +1,13 @@
Zero-Clause BSD
=============
Permission to use, copy, modify, and/or distribute this software for
any purpose with or without fee is hereby granted.
THE SOFTWARE IS PROVIDED “AS IS” AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE
FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY
DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
internal/web/store_test.go
+260
View File
@@ -0,0 +1,260 @@
package web
import (
"context"
"sort"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
type fakeJobStore struct {
jobs map[string]store.Job
subscriptions map[string]store.Subscription
nextID int64
lastEpisode store.Episode
enqueueCalls, statusCalls int
statusIDs []string
listStatuses []string
listLimits []int
registerCalls int
registerEpisodes []store.Episode
registerDownload bool
registerErr error
subscribedCalls int
subscribedIDs []string
beforeDeleteSucceeded func()
err error
}
func (f *fakeJobStore) ensure() {
if f.jobs == nil {
f.jobs = make(map[string]store.Job)
}
}
func (f *fakeJobStore) Enqueue(_ context.Context, ep store.Episode, source string) (store.Job, bool, error) {
f.enqueueCalls++
f.lastEpisode = ep
if f.err != nil {
return store.Job{}, false, f.err
}
f.ensure()
if job, ok := f.jobs[ep.ID]; ok {
return job, false, nil
}
f.nextID++
job := store.Job{ID: f.nextID, EpisodeID: ep.ID, SeriesTitle: ep.SeriesTitle,
Title: ep.Title, BroadcasterName: ep.BroadcasterName, Source: source,
Status: "queued", CreatedAt: time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)}
f.jobs[ep.ID] = job
return job, true, nil
}
func (f *fakeJobStore) EpisodeStatuses(_ context.Context, ids []string) (map[string]string, error) {
f.statusCalls++
f.statusIDs = append([]string(nil), ids...)
if f.err != nil {
return nil, f.err
}
result := make(map[string]string)
for _, id := range ids {
if job, ok := f.jobs[id]; ok {
result[id] = job.Status
}
}
return result, nil
}
func (f *fakeJobStore) ListJobs(_ context.Context, status string, limit int) ([]store.Job, error) {
f.listStatuses = append(f.listStatuses, status)
f.listLimits = append(f.listLimits, limit)
if f.err != nil {
return nil, f.err
}
var result []store.Job
for _, job := range f.jobs {
if job.Status == status {
result = append(result, job)
}
}
sort.Slice(result, func(i, j int) bool { return result[i].ID > result[j].ID })
if len(result) > limit {
result = result[:limit]
}
return result, nil
}
func (f *fakeJobStore) GetJob(_ context.Context, id int64) (store.Job, error) {
if f.err != nil {
return store.Job{}, f.err
}
for _, job := range f.jobs {
if job.ID == id {
return job, nil
}
}
return store.Job{}, pgx.ErrNoRows
}
func (f *fakeJobStore) Retry(_ context.Context, id int64) error {
for ep, job := range f.jobs {
if job.ID == id && (job.Status == "failed" || job.Status == "canceled") {
job.Status = "queued"
f.jobs[ep] = job
return nil
}
}
return store.ErrInvalidTransition
}
func (f *fakeJobStore) RetryEpisode(_ context.Context, id string) error {
job, ok := f.jobs[id]
if !ok || (job.Status != "failed" && job.Status != "canceled") {
return store.ErrInvalidTransition
}
job.Status = "queued"
f.jobs[id] = job
return nil
}
func (f *fakeJobStore) Cancel(_ context.Context, id int64) error {
if f.err != nil {
return f.err
}
for ep, job := range f.jobs {
if job.ID == id && (job.Status == "queued" || job.Status == "running") {
job.Status = "canceled"
f.jobs[ep] = job
return nil
}
}
return store.ErrInvalidTransition
}
func (f *fakeJobStore) CancelQueued(ctx context.Context, id int64) error {
return f.Cancel(ctx, id)
}
var _ JobStore = (*fakeJobStore)(nil)
func (f *fakeJobStore) SubscribedSeries(_ context.Context, ids []string) (map[string]bool, error) {
f.subscribedCalls++
f.subscribedIDs = append([]string(nil), ids...)
if f.err != nil {
return nil, f.err
}
result := make(map[string]bool)
for _, id := range ids {
_, result[id] = f.subscriptions[id]
}
return result, nil
}
func (f *fakeJobStore) RegisterSubscription(_ context.Context, seriesID, title string, episodes []store.Episode, download bool) (bool, error) {
f.registerCalls++
f.registerEpisodes = append([]store.Episode(nil), episodes...)
f.registerDownload = download
if f.registerErr != nil {
return false, f.registerErr
}
if f.err != nil {
return false, f.err
}
if f.subscriptions == nil {
f.subscriptions = make(map[string]store.Subscription)
}
if _, exists := f.subscriptions[seriesID]; exists {
return false, nil
}
var count int64
if download {
count = int64(len(episodes))
}
checked := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
f.subscriptions[seriesID] = store.Subscription{SeriesID: seriesID, SeriesTitle: title,
Enabled: true, CreatedAt: checked, LastCheckedAt: &checked, JobCount: count}
return true, nil
}
func (f *fakeJobStore) ListSubscriptions(_ context.Context) ([]store.Subscription, error) {
if f.err != nil {
return nil, f.err
}
var result []store.Subscription
for _, sub := range f.subscriptions {
result = append(result, sub)
}
sort.Slice(result, func(i, j int) bool { return result[i].SeriesID < result[j].SeriesID })
return result, nil
}
func (f *fakeJobStore) SetSubscriptionEnabled(_ context.Context, seriesID string, enabled bool) error {
if f.err != nil {
return f.err
}
sub, exists := f.subscriptions[seriesID]
if !exists {
return store.ErrInvalidTransition
}
sub.Enabled = enabled
f.subscriptions[seriesID] = sub
return nil
}
func (f *fakeJobStore) DeleteSubscription(_ context.Context, seriesID string) (bool, error) {
if f.err != nil {
return false, f.err
}
if _, exists := f.subscriptions[seriesID]; !exists {
return false, nil
}
delete(f.subscriptions, seriesID)
return true, nil
}
func (f *fakeJobStore) ListSucceededJobs(_ context.Context, limit int) ([]store.Job, error) {
if f.err != nil {
return nil, f.err
}
var result []store.Job
for _, job := range f.jobs {
if job.Status == "succeeded" {
result = append(result, job)
}
}
sort.Slice(result, func(i, j int) bool { return result[i].ID > result[j].ID })
if len(result) > limit {
result = result[:limit]
}
return result, nil
}
func (f *fakeJobStore) GetSucceededJob(_ context.Context, id int64) (store.Job, error) {
if f.err != nil {
return store.Job{}, f.err
}
for _, job := range f.jobs {
if job.ID == id && job.Status == "succeeded" {
return job, nil
}
}
return store.Job{}, pgx.ErrNoRows
}
func (f *fakeJobStore) DeleteSucceededJob(_ context.Context, id int64) error {
if f.beforeDeleteSucceeded != nil {
f.beforeDeleteSucceeded()
}
if f.err != nil {
return f.err
}
for episodeID, job := range f.jobs {
if job.ID == id && job.Status == "succeeded" {
delete(f.jobs, episodeID)
return nil
}
}
return store.ErrInvalidTransition
}
internal/web/subscriptions.go
+191
View File
@@ -0,0 +1,191 @@
package web
import (
"context"
"errors"
"net/http"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
type subscriptionActionView struct {
ID, Genre, ErrorText string
Subscribed, Choosing, DownloadExisting bool
}
type subscriptionView struct {
ID, Title, CreatedAtText, LastCheckedText string
CheckErrorAtText, CheckErrorText string
Enabled bool
Jobs int64
}
type subscriptionsView struct {
Items []subscriptionView
}
func (s *server) subscriptionChoice(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
ep, found := s.cachedEpisode(genre, id)
if !found || ep.SeriesID == "" {
http.Error(w, "新着キャッシュに番組がありません。新着を再表示してください", http.StatusNotFound)
return
}
registered, err := s.isSubscribed(r.Context(), ep.SeriesID)
if err != nil {
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
return
}
if registered {
s.subscriptionRegisteredResponse(w, r, id, genre)
return
}
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true})
}
func (s *server) registerSubscription(w http.ResponseWriter, r *http.Request) {
genre, id, ok := episodeForm(w, r)
if !ok {
return
}
choice := r.Form.Get("download_existing")
if choice != "" && choice != "1" {
http.Error(w, "公開中の回の選択が不正です", http.StatusBadRequest)
return
}
ep, found := s.cachedEpisode(genre, id)
if !found || ep.SeriesID == "" || ep.SeriesTitle == "" {
http.Error(w, "新着キャッシュに番組がありません。新着を再表示してください", http.StatusNotFound)
return
}
registered, err := s.isSubscribed(r.Context(), ep.SeriesID)
if err != nil {
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
return
}
if registered {
s.subscriptionRegisteredResponse(w, r, id, genre)
return
}
episodes, err := s.fetcher.FetchSeriesEpisodes(r.Context(), ep.SeriesID)
if err != nil {
s.logger.Printf("番組の公開中エピソード取得に失敗: %s", platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true, DownloadExisting: choice == "1", ErrorText: "公開中の回を取得できませんでした。再試行してください"})
return
}
items := make([]store.Episode, 0, len(episodes))
for _, episode := range episodes {
items = append(items, storeEpisode(episode))
}
_, err = s.jobs.RegisterSubscription(r.Context(), ep.SeriesID, ep.SeriesTitle, items, choice == "1")
if err != nil {
s.logger.Printf("番組登録に失敗: %v", err)
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true, DownloadExisting: choice == "1", ErrorText: "番組を登録できませんでした。再試行してください"})
return
}
s.subscriptionRegisteredResponse(w, r, id, genre)
}
func (s *server) subscriptionRegisteredResponse(w http.ResponseWriter, r *http.Request, id, genre string) {
if r.Header.Get("HX-Request") != "true" && r.Method == http.MethodPost {
http.Redirect(w, r, "/?genre="+genre, http.StatusSeeOther)
return
}
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Subscribed: true})
}
func (s *server) isSubscribed(ctx context.Context, seriesID string) (bool, error) {
result, err := s.jobs.SubscribedSeries(ctx, []string{seriesID})
return result[seriesID], err
}
func (s *server) subscriptionsPage(w http.ResponseWriter, r *http.Request) {
view, err := s.loadSubscriptions(r.Context())
if err != nil {
s.logger.Printf("登録番組の取得に失敗: %v", err)
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "layout", page{Title: "登録番組", Subscriptions: &view})
}
func (s *server) subscriptionsFragment(w http.ResponseWriter, r *http.Request) {
view, err := s.loadSubscriptions(r.Context())
if err != nil {
s.logger.Printf("登録番組の取得に失敗: %v", err)
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
return
}
s.render(w, "subscriptions_list", view)
}
func (s *server) loadSubscriptions(ctx context.Context) (subscriptionsView, error) {
subs, err := s.jobs.ListSubscriptions(ctx)
if err != nil {
return subscriptionsView{}, err
}
view := subscriptionsView{Items: make([]subscriptionView, 0, len(subs))}
for _, sub := range subs {
item := subscriptionView{ID: sub.SeriesID, Title: sub.SeriesTitle,
CreatedAtText: sub.CreatedAt.In(s.zone).Format("2006-01-02 15:04 MST"),
LastCheckedText: "未確認", Enabled: sub.Enabled, Jobs: sub.JobCount}
if sub.LastCheckedAt != nil {
item.LastCheckedText = sub.LastCheckedAt.In(s.zone).Format("2006-01-02 15:04 MST")
}
if sub.LastCheckErrorAt != nil && sub.LastCheckError != nil {
item.CheckErrorAtText = sub.LastCheckErrorAt.In(s.zone).Format("2006-01-02 15:04 MST")
item.CheckErrorText = *sub.LastCheckError
}
view.Items = append(view.Items, item)
}
return view, nil
}
func (s *server) pauseSubscription(w http.ResponseWriter, r *http.Request) {
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
return s.jobs.SetSubscriptionEnabled(ctx, id, false)
})
}
func (s *server) resumeSubscription(w http.ResponseWriter, r *http.Request) {
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
return s.jobs.SetSubscriptionEnabled(ctx, id, true)
})
}
func (s *server) deleteSubscription(w http.ResponseWriter, r *http.Request) {
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
deleted, err := s.jobs.DeleteSubscription(ctx, id)
if err == nil && !deleted {
return store.ErrInvalidTransition
}
return err
})
}
func (s *server) subscriptionMutation(w http.ResponseWriter, r *http.Request, change func(context.Context, string) error) {
id := r.PathValue("id")
if id == "" {
http.Error(w, "番組 ID が不正です", http.StatusBadRequest)
return
}
if err := change(r.Context(), id); err != nil {
if errors.Is(err, store.ErrInvalidTransition) {
http.Error(w, "番組が見つかりません", http.StatusNotFound)
} else {
s.logger.Printf("登録番組の操作に失敗: %v", err)
http.Error(w, "登録番組を変更できませんでした", http.StatusInternalServerError)
}
return
}
if r.Header.Get("HX-Request") != "true" {
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
return
}
s.subscriptionsFragment(w, r)
}
internal/web/subscriptions_test.go
+199
View File
@@ -0,0 +1,199 @@
package web
import (
"errors"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
func TestRegisterSubscriptionChoicesAndServerOwnedSeries(t *testing.T) {
for _, download := range []bool{false, true} {
t.Run(map[bool]string{false: "without existing", true: "with existing"}[download], func(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Duration: 200}},
seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Title: "公開中", Duration: 200}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
page := getPage(handler, "/")
if page.Code != http.StatusOK || !strings.Contains(page.Body.String(), "番組を登録") || jobs.subscribedCalls != 1 || len(jobs.subscribedIDs) != 1 || jobs.subscribedIDs[0] != "sr1" {
t.Fatalf("新着の登録状態: %d ids=%v body=%s", page.Code, jobs.subscribedIDs, page.Body.String())
}
choice := getPage(handler, "/subscriptions/choice?genre=drama&episode_id=ep1")
if choice.Code != http.StatusOK || !strings.Contains(choice.Body.String(), "公開中の回もダウンロードする") || !strings.Contains(choice.Body.String(), ">登録</button>") {
t.Fatalf("登録時の選択: %d %s", choice.Code, choice.Body.String())
}
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}, "series_id": {"forged"}, "series_title": {"偽番組"}}
if download {
values.Set("download_existing", "1")
}
registered := postForm(handler, "/subscriptions/register", values)
if registered.Code != http.StatusOK || !strings.Contains(registered.Body.String(), "登録済み") {
t.Fatalf("登録応答: %d %s", registered.Code, registered.Body.String())
}
if jobs.registerCalls != 1 || jobs.registerDownload != download || len(jobs.registerEpisodes) != 1 || jobs.registerEpisodes[0].ID != "ep1" || jobs.registerEpisodes[0].SeriesID != "sr1" || jobs.subscriptions["sr1"].SeriesTitle != "本当の番組" {
t.Fatalf("登録内容: calls=%d download=%t episodes=%+v subs=%+v", jobs.registerCalls, jobs.registerDownload, jobs.registerEpisodes, jobs.subscriptions)
}
if len(fetcher.seriesCalls) != 1 || fetcher.seriesCalls[0] != "sr1" {
t.Fatalf("番組一覧取得: %v", fetcher.seriesCalls)
}
if !strings.Contains(getPage(handler, "/subscriptions").Body.String(), "最終確認: 2026-10-03 09:00 JST") {
t.Fatal("登録直後の最終確認日時が表示されない")
}
fetcher.seriesErr = errors.New("API unavailable")
duplicate := postForm(handler, "/subscriptions/register", values)
if duplicate.Code != http.StatusOK || !strings.Contains(duplicate.Body.String(), "登録済み") || len(fetcher.seriesCalls) != 1 || jobs.registerCalls != 1 {
t.Fatalf("重複登録が API を再取得: %d %s calls=%v register=%d", duplicate.Code, duplicate.Body.String(), fetcher.seriesCalls, jobs.registerCalls)
}
reloaded := getPage(handler, "/")
if !strings.Contains(reloaded.Body.String(), "登録済み") || strings.Contains(reloaded.Body.String(), ">番組を登録</button>") {
t.Fatalf("登録済みの表示: %s", reloaded.Body.String())
}
})
}
}
func TestRegisterSubscriptionRejectsMissingCacheAndFetchFailure(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}}}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
if got := postForm(handler, "/subscriptions/register", values).Code; got != http.StatusNotFound {
t.Fatalf("キャッシュ無しの登録: %d", got)
}
getPage(handler, "/")
for _, path := range []string{"/subscriptions/register", "/subscriptions/sr1/pause", "/subscriptions/sr1/resume", "/subscriptions/sr1/delete"} {
response := getPage(handler, path)
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
t.Errorf("GET での変更 %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
}
}
for _, tt := range []struct {
path string
want int
}{
{"/subscriptions/choice?genre=news&episode_id=ep1", http.StatusBadRequest},
{"/subscriptions/choice?genre=drama", http.StatusBadRequest},
{"/subscriptions/choice?genre=drama&episode_id=missing", http.StatusNotFound},
} {
response := getPage(handler, tt.path)
if response.Code != tt.want {
t.Errorf("不正な選択 %s: got=%d want=%d", tt.path, response.Code, tt.want)
}
}
for _, tt := range []struct {
values url.Values
want int
}{
{url.Values{"genre": {"news"}, "episode_id": {"ep1"}}, http.StatusBadRequest},
{url.Values{"genre": {"drama"}}, http.StatusBadRequest},
{url.Values{"genre": {"drama"}, "episode_id": {"missing"}}, http.StatusNotFound},
} {
response := postForm(handler, "/subscriptions/register", tt.values)
if response.Code != tt.want {
t.Errorf("不正な登録 %v: got=%d want=%d", tt.values, response.Code, tt.want)
}
}
values.Set("download_existing", "bad")
if got := postForm(handler, "/subscriptions/register", values).Code; got != http.StatusBadRequest {
t.Fatalf("不正な選択: %d", got)
}
values.Del("download_existing")
fetcher.seriesErr = errors.New("platform_token=secret& upstream failure")
failed := postForm(handler, "/subscriptions/register", values)
if failed.Code != http.StatusOK || !strings.Contains(failed.Body.String(), "公開中の回を取得できませんでした") || strings.Contains(failed.Body.String(), "secret") || jobs.registerCalls != 0 {
t.Fatalf("取得失敗時に未登録: %d %s calls=%d", failed.Code, failed.Body.String(), jobs.registerCalls)
}
fetcher.seriesErr = nil
fetcher.seriesEpisodes = []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}}
jobs.registerErr = errors.New("secret database detail")
failed = postForm(handler, "/subscriptions/register", values)
if failed.Code != http.StatusOK || !strings.Contains(failed.Body.String(), "番組を登録できませんでした") || strings.Contains(failed.Body.String(), "secret") || len(jobs.subscriptions) != 0 {
t.Fatalf("DB 失敗時に未登録: %d %s subscriptions=%v", failed.Code, failed.Body.String(), jobs.subscriptions)
}
}
func TestNewerShowsInitiallySubscribedSeriesOncePerSeriesLookup(t *testing.T) {
fetcher := &fakeFetcher{episodes: []tver.Episode{
{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200},
{ID: "ep2", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200},
}}
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{
"sr1": {SeriesID: "sr1", SeriesTitle: "番組", Enabled: true},
}}
response := getPage(newTestHandlerWithStore(fetcher, jobs, nil), "/")
body := response.Body.String()
if response.Code != http.StatusOK || strings.Count(body, "登録済み") != 2 || strings.Contains(body, ">番組を登録</button>") || jobs.subscribedCalls != 1 || len(jobs.subscribedIDs) != 1 || jobs.subscribedIDs[0] != "sr1" {
t.Fatalf("登録済み番組の初期表示: status=%d calls=%d ids=%v body=%s", response.Code, jobs.subscribedCalls, jobs.subscribedIDs, body)
}
}
func TestSubscriptionMutationsWithoutHtmxRedirect(t *testing.T) {
fetcher := &fakeFetcher{
episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}},
seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}},
}
jobs := &fakeJobStore{}
handler := newTestHandlerWithStore(fetcher, jobs, nil)
getPage(handler, "/")
form := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
request := httptest.NewRequest(http.MethodPost, "/subscriptions/register", strings.NewReader(form.Encode()))
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/?genre=drama" {
t.Fatalf("通常登録 POST: status=%d Location=%q", response.Code, response.Header().Get("Location"))
}
request = httptest.NewRequest(http.MethodPost, "/subscriptions/sr1/pause", nil)
response = httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/subscriptions" || jobs.subscriptions["sr1"].Enabled {
t.Fatalf("通常操作 POST: status=%d Location=%q subscription=%+v", response.Code, response.Header().Get("Location"), jobs.subscriptions["sr1"])
}
}
func TestSubscriptionsPagePauseResumeDeleteAndEscaping(t *testing.T) {
checked := time.Date(2026, 10, 4, 2, 30, 0, 0, time.UTC)
failedAt := time.Date(2026, 10, 4, 3, 45, 0, 0, time.UTC)
checkError := `<script>alert("error")</script> token=[REDACTED]`
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{
"sr1": {SeriesID: "sr1", SeriesTitle: `<script>alert(1)</script>`, Enabled: true,
CreatedAt: time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC), LastCheckedAt: &checked,
LastCheckErrorAt: &failedAt, LastCheckError: &checkError, JobCount: 3},
"sr2": {SeriesID: "sr2", SeriesTitle: "別番組", Enabled: false, JobCount: 0},
}}
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
page := getPage(handler, "/subscriptions")
if page.Code != http.StatusOK {
t.Fatalf("登録番組ページ: %d", page.Code)
}
body := page.Body.String()
for _, want := range []string{"&lt;script&gt;", "登録日時: 2026-10-03 09:00 JST", "最終確認: 2026-10-04 11:30 JST", "最終確認: 未確認", "最後の確認エラー（2026-10-04 12:45 JST）", `&lt;script&gt;alert(&#34;error&#34;)&lt;/script&gt; token=[REDACTED]`, "関連ジョブ: 3件", "有効", "一時停止", `hx-confirm="番組の登録を解除しますか？"`} {
if !strings.Contains(body, want) {
t.Errorf("登録番組の情報 %q がない", want)
}
}
if strings.Contains(body, `<script>`) {
t.Fatal("番組名または確認エラーがエスケープされていない")
}
paused := postForm(handler, "/subscriptions/sr1/pause", nil)
if paused.Code != http.StatusOK || jobs.subscriptions["sr1"].Enabled || !strings.Contains(paused.Body.String(), `action="/subscriptions/sr1/resume"`) || strings.Contains(paused.Body.String(), "<html") {
t.Fatalf("一時停止: %d %s", paused.Code, paused.Body.String())
}
resumed := postForm(handler, "/subscriptions/sr1/resume", nil)
if resumed.Code != http.StatusOK || !jobs.subscriptions["sr1"].Enabled || !strings.Contains(resumed.Body.String(), `action="/subscriptions/sr1/pause"`) {
t.Fatalf("再開: %d %s", resumed.Code, resumed.Body.String())
}
deleted := postForm(handler, "/subscriptions/sr1/delete", nil)
if deleted.Code != http.StatusOK || len(jobs.subscriptions) != 1 || strings.Contains(deleted.Body.String(), "&lt;script&gt;") {
t.Fatalf("解除: %d %s", deleted.Code, deleted.Body.String())
}
if got := postForm(handler, "/subscriptions/sr1/delete", nil).Code; got != http.StatusNotFound {
t.Fatalf("存在しない番組の解除: %d", got)
}
}
internal/web/templates/layout.html
+36
View File
@@ -0,0 +1,36 @@
{{define "layout"}}<!doctype html>
<html lang="ja">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{{.Title}} | TVer</title>
<link rel="stylesheet" href="/static/app.css">
<script src="/static/htmx-2.0.11.min.js" defer></script>
</head>
<body>
<header>
<strong>TVer</strong>
<nav aria-label="メイン" hx-boost="true">
<a href="/">新着</a>
<a href="/queue">待ち行列</a>
<a href="/subscriptions">登録番組</a>
<a href="/library">ライブラリ</a>
</nav>
</header>
<main>
{{if .Newer}}
{{template "newer_page" .Newer}}
{{else if .Queue}}
{{template "queue_page" .Queue}}
{{else if .Subscriptions}}
{{template "subscriptions_page" .Subscriptions}}
{{else if .Library}}
{{template "library_page" .Library}}
{{else if .Video}}
{{template "video_page" .Video}}
{{else}}
<h1>{{.Title}}</h1>
{{end}}
</main>
</body>
</html>{{end}}
internal/web/templates/library.html
+34
View File
@@ -0,0 +1,34 @@
{{define "library_page"}}
<h1>ライブラリ</h1>
{{if .Items}}
<div class="episode-grid">
{{range .Items}}
<article class="episode-card">
<a href="/library/{{.ID}}">
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
<div class="episode-copy">
<h2>{{.SeriesTitle}}</h2>
<p>{{.Title}}</p>
<p>放送局: {{.BroadcasterName}}</p>
<p>長さ: {{.DurationText}}</p>
<p>サイズ: {{.FileSizeText}}</p>
<p>完了日時: {{.FinishedAtText}}</p>
</div>
</a>
</article>
{{end}}
</div>
{{else}}
<p>ダウンロード済みの動画はありません</p>
{{end}}
{{end}}
{{define "video_page"}}
<h1>{{.SeriesTitle}}</h1>
<h2>{{.Title}}</h2>
<video controls preload="metadata" src="{{.VideoURL}}"></video>
<p>放送局: {{.BroadcasterName}}／長さ: {{.DurationText}}／サイズ: {{.FileSizeText}}／完了日時: {{.FinishedAtText}}</p>
<form action="/library/{{.ID}}/delete" method="post" hx-confirm="動画ファイルと記録を削除しますか？">
<button type="submit">削除</button>
</form>
{{end}}
internal/web/templates/newer.html
+70
View File
@@ -0,0 +1,70 @@
{{define "newer_page"}}
<h1>新着</h1>
<form id="newer-controls" action="/" method="get"
hx-get="/newer" hx-trigger="change, submit"
hx-target="#newer-results" hx-swap="outerHTML">
<label for="genre">ジャンル</label>
<select id="genre" name="genre">
<option value="drama" {{if eq .Genre "drama"}}selected{{end}}>ドラマ</option>
<option value="variety" {{if eq .Genre "variety"}}selected{{end}}>バラエティ</option>
</select>
<input type="hidden" name="hide_previews" value="0">
<label for="hide-previews">
<input id="hide-previews" type="checkbox" name="hide_previews" value="1" {{if .HidePreviews}}checked{{end}}>
予告を隠す
</label>
<button type="submit">表示</button>
</form>
{{template "newer_results" .}}
{{end}}
{{define "newer_results"}}
<section id="newer-results" aria-label="新着一覧" aria-live="polite">
{{if .Failed}}
<p role="status">{{.ErrorText}}</p>
{{else if not .Episodes}}
<p>表示する新着はありません</p>
{{else}}
<div class="episode-grid">
{{range .Episodes}}
<article class="episode-card">
<a href="{{.URL}}">
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
<div class="episode-copy">
<h2>{{.SeriesTitle}}</h2>
<p>{{.Title}}</p>
<p>放送局: {{.BroadcasterName}}</p>
<p>長さ: {{.DurationMinutes}}分</p>
<p>放送日: {{.BroadcastDateLabel}}</p>
<p>配信終了: <time datetime="{{.EndAtISO}}">{{.EndAtText}}</time></p>
</div>
</a>
{{template "episode_action" .}}
{{template "subscription_action" .SubscriptionAction}}
</article>
{{end}}
</div>
{{end}}
</section>
{{end}}
{{define "episode_action"}}
<div class="episode-action">
{{if not .Status}}
<form action="/episodes/enqueue" method="post" hx-post="/episodes/enqueue" hx-target="closest .episode-action" hx-swap="outerHTML">
<input type="hidden" name="episode_id" value="{{.ID}}">
<input type="hidden" name="genre" value="{{.Genre}}">
<button type="submit">ダウンロード</button>
</form>
{{else}}
<span role="status">{{.StatusText}}</span>
{{if or (eq .Status "failed") (eq .Status "canceled")}}
<form action="/episodes/retry" method="post" hx-post="/episodes/retry" hx-target="closest .episode-action" hx-swap="outerHTML">
<input type="hidden" name="episode_id" value="{{.ID}}">
<input type="hidden" name="genre" value="{{.Genre}}">
<button type="submit">再試行</button>
</form>
{{end}}
{{end}}
</div>
{{end}}
internal/web/templates/queue.html
+59
View File
@@ -0,0 +1,59 @@
{{define "queue_page"}}
<h1>待ち行列</h1>
{{template "queue_list" .}}
{{end}}
{{define "queue_list"}}
<div id="queue-list" {{if .Poll}}hx-get="/queue/list" hx-trigger="every 5s" hx-target="this" hx-swap="outerHTML"{{end}}>
<section aria-label="ダウンロード中">
<h2>ダウンロード中</h2>
{{range .Running}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="待機中">
<h2>待機中</h2>
{{range .Queued}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="失敗">
<h2>失敗</h2>
{{range .Failed}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="取り消し">
<h2>取り消し</h2>
{{range .Canceled}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
<section aria-label="完了">
<h2>完了（直近50件）</h2>
{{range .Succeeded}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
</section>
</div>
{{end}}
{{define "queue_job"}}
<article class="queue-job">
<h3>{{.SeriesTitle}} — {{.Title}}</h3>
<p>放送局: {{.BroadcasterName}}／登録元: {{.SourceText}}／登録日時: {{.CreatedAtText}}</p>
{{if eq .Status "running"}}
<p>段階: {{if .PhaseText}}{{.PhaseText}}{{else}}準備中{{end}}
{{if .PercentText}}／{{.PercentText}}{{end}}
{{if .SpeedText}}／{{.SpeedText}}{{end}}
{{if .ETAText}}／{{.ETAText}}{{end}}
{{if .FragmentText}}／{{.FragmentText}}{{end}}</p>
<form action="/jobs/{{.ID}}/cancel" method="post" hx-post="/jobs/{{.ID}}/cancel" hx-target="#queue-list" hx-swap="outerHTML">
<button type="submit">取り消し</button>
</form>
{{end}}
{{if eq .Status "failed"}}
<p>失敗理由: {{.ErrorKindText}}</p>
{{if .ErrorMessage}}<details><summary>詳細</summary><pre>{{.ErrorMessage}}</pre></details>{{end}}
{{end}}
{{if eq .Status "queued"}}
<form action="/jobs/{{.ID}}/cancel" method="post" hx-post="/jobs/{{.ID}}/cancel" hx-target="#queue-list" hx-swap="outerHTML">
<button type="submit">取り消し</button>
</form>
{{else if or (eq .Status "failed") (eq .Status "canceled")}}
<form action="/jobs/{{.ID}}/retry" method="post" hx-post="/jobs/{{.ID}}/retry" hx-target="#queue-list" hx-swap="outerHTML">
<button type="submit">再試行</button>
</form>
{{end}}
</article>
{{end}}
internal/web/templates/subscriptions.html
+44
View File
@@ -0,0 +1,44 @@
{{define "subscription_action"}}
<div class="subscription-action">
{{if .Subscribed}}
<span role="status">登録済み</span>
{{else if .Choosing}}
{{if .ErrorText}}<p role="alert">{{.ErrorText}}</p>{{end}}
<form action="/subscriptions/register" method="post" hx-post="/subscriptions/register" hx-target="closest .subscription-action" hx-swap="outerHTML">
<input type="hidden" name="episode_id" value="{{.ID}}">
<input type="hidden" name="genre" value="{{.Genre}}">
<label><input type="checkbox" name="download_existing" value="1" {{if .DownloadExisting}}checked{{end}}>公開中の回もダウンロードする</label>
<button type="submit">登録</button>
</form>
{{else}}
<button type="button" hx-get="/subscriptions/choice?genre={{.Genre}}&amp;episode_id={{.ID}}" hx-target="closest .subscription-action" hx-swap="outerHTML">番組を登録</button>
{{end}}
</div>
{{end}}
{{define "subscriptions_page"}}
<h1>登録番組</h1>
{{template "subscriptions_list" .}}
{{end}}
{{define "subscriptions_list"}}
<div id="subscriptions-list">
{{range .Items}}
<article class="subscription-item">
<h2>{{.Title}}</h2>
<p>登録日時: {{.CreatedAtText}}／最終確認: {{.LastCheckedText}}／状態: {{if .Enabled}}有効{{else}}一時停止{{end}}／関連ジョブ: {{.Jobs}}件</p>
{{if .CheckErrorText}}
<details class="error"><summary>最後の確認エラー（{{.CheckErrorAtText}}）</summary><p>{{.CheckErrorText}}</p></details>
{{end}}
{{if .Enabled}}
<form action="/subscriptions/{{.ID}}/pause" method="post" hx-post="/subscriptions/{{.ID}}/pause" hx-target="#subscriptions-list" hx-swap="outerHTML"><button type="submit">一時停止</button></form>
{{else}}
<form action="/subscriptions/{{.ID}}/resume" method="post" hx-post="/subscriptions/{{.ID}}/resume" hx-target="#subscriptions-list" hx-swap="outerHTML"><button type="submit">再開</button></form>
{{end}}
<form action="/subscriptions/{{.ID}}/delete" method="post" hx-post="/subscriptions/{{.ID}}/delete" hx-target="#subscriptions-list" hx-swap="outerHTML" hx-confirm="番組の登録を解除しますか？"><button type="submit">解除</button></form>
</article>
{{else}}
<p>登録番組はありません</p>
{{end}}
</div>
{{end}}
internal/worker/progress.go
+81
View File
@@ -0,0 +1,81 @@
package worker
import (
"encoding/json"
"strings"
"gitea.kazu634.com/kazu634/tver/internal/store"
)
type outputEvent struct {
Event string `json:"event"`
FormatID string `json:"format_id"`
Filepath string `json:"filepath"`
Progress struct {
DownloadedBytes *float64 `json:"downloaded_bytes"`
TotalBytes *float64 `json:"total_bytes"`
TotalBytesEstimate *float64 `json:"total_bytes_estimate"`
Speed *float64 `json:"speed"`
ETA *float64 `json:"eta"`
FragmentIndex *float64 `json:"fragment_index"`
FragmentCount *float64 `json:"fragment_count"`
} `json:"progress"`
}
func parseProgress(line string) (store.Progress, bool) {
var event outputEvent
if err := json.Unmarshal([]byte(line), &event); err != nil {
return store.Progress{}, false
}
phase := ""
switch event.Event {
case "download":
if strings.Contains(strings.ToLower(event.FormatID), "audio") {
phase = "audio"
} else {
phase = "video"
}
case "postprocess":
phase = "merging"
default:
return store.Progress{}, false
}
total := int64Pointer(event.Progress.TotalBytes)
estimate := false
if total == nil && event.Progress.TotalBytesEstimate != nil {
total = int64Pointer(event.Progress.TotalBytesEstimate)
estimate = true
}
var formatID *string
if event.FormatID != "" {
formatID = &event.FormatID
}
return store.Progress{Phase: phase, DownloadedBytes: int64Pointer(event.Progress.DownloadedBytes),
TotalBytes: total, TotalIsEstimate: estimate, SpeedBPS: event.Progress.Speed,
ETASeconds: event.Progress.ETA, FragmentIndex: int32Pointer(event.Progress.FragmentIndex),
FragmentCount: int32Pointer(event.Progress.FragmentCount), FormatID: formatID}, true
}
func int64Pointer(value *float64) *int64 {
if value == nil {
return nil
}
result := int64(*value)
return &result
}
func int32Pointer(value *float64) *int32 {
if value == nil {
return nil
}
result := int32(*value)
return &result
}
func parseAfterMove(line string) (string, bool) {
var event outputEvent
if err := json.Unmarshal([]byte(line), &event); err != nil || event.Event != "after_move" || event.Filepath == "" {
return "", false
}
return event.Filepath, true
}
internal/worker/progress_test.go
+27
View File
@@ -0,0 +1,27 @@
package worker
import "testing"
func TestParseProgress(t *testing.T) {
video, ok := parseProgress(`{"event":"download","format_id":"hls-698-1","progress":{"downloaded_bytes":25,"total_bytes_estimate":100.0,"speed":10.5,"eta":3,"fragment_index":1,"fragment_count":4}}`)
if !ok || video.Phase != "video" || video.DownloadedBytes == nil || *video.DownloadedBytes != 25 || video.TotalBytes == nil || *video.TotalBytes != 100 || !video.TotalIsEstimate || video.SpeedBPS == nil || *video.SpeedBPS != 10.5 || video.ETASeconds == nil || *video.ETASeconds != 3 || video.FragmentIndex == nil || *video.FragmentIndex != 1 || video.FragmentCount == nil || *video.FragmentCount != 4 || video.FormatID == nil || *video.FormatID != "hls-698-1" {
t.Fatalf("映像進捗: %+v ok=%v", video, ok)
}
audio, ok := parseProgress(`{"event":"download","format_id":"hls-ts_AUDIO-0_2","progress":{"downloaded_bytes":50,"total_bytes":50}}`)
if !ok || audio.Phase != "audio" || audio.TotalIsEstimate || audio.TotalBytes == nil || *audio.TotalBytes != 50 {
t.Fatalf("音声進捗: %+v ok=%v", audio, ok)
}
merging, ok := parseProgress(`{"event":"postprocess","progress":{"status":"started","postprocessor":"Merger"}}`)
if !ok || merging.Phase != "merging" {
t.Fatalf("結合進捗: %+v ok=%v", merging, ok)
}
path, ok := parseAfterMove(`{"event":"after_move","filepath":"/videos/incoming/ep.mp4"}`)
if !ok || path != "/videos/incoming/ep.mp4" {
t.Fatalf("移動後パス: %q ok=%v", path, ok)
}
for _, invalid := range []string{"noise", `{"event":"other"}`} {
if _, ok := parseProgress(invalid); ok {
t.Errorf("不正な進捗を許した: %q", invalid)
}
}
}
internal/worker/runner.go
+504
View File
@@ -0,0 +1,504 @@
package worker
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"syscall"
"time"
"gitea.kazu634.com/kazu634/tver/internal/media"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
var (
errCanceled = errors.New("job canceled")
errShutdown = errors.New("worker shutting down")
errInvalidOutput = errors.New("invalid output")
headerSecretRE = regexp.MustCompile(`(?im)(authorization|cookie)([=:][ \t]*)([^\r\n]+)`)
paramSecretRE = regexp.MustCompile(`(?i)(token|platform_uid|platform_token)([=:][[:space:]]*)([^&[:space:]]+)`)
)
type JobStore interface {
ClaimNext(context.Context) (store.Job, error)
UpdateProgress(context.Context, int64, store.Progress) error
Heartbeat(context.Context, int64) error
GetJob(context.Context, int64) (store.Job, error)
Complete(context.Context, int64, string, int64) error
Fail(context.Context, int64, string, string) error
RequeueRunning(context.Context, int64) error
}
type Config struct {
VideoDir string
YTDLPPath string
FFprobePath string
Concurrency int
PollInterval time.Duration
ProgressInterval time.Duration
HeartbeatInterval time.Duration
CancelInterval time.Duration
}
type Runner struct {
store JobStore
config Config
logger *log.Logger
}
type cappedBuffer struct {
buffer bytes.Buffer
limit int
}
func (b *cappedBuffer) Write(p []byte) (int, error) {
written := len(p)
remaining := b.limit - b.buffer.Len()
if remaining > 0 {
if remaining < len(p) {
p = p[:remaining]
}
_, _ = b.buffer.Write(p)
}
return written, nil
}
func (b *cappedBuffer) String() string { return b.buffer.String() }
func scanLines(reader io.Reader, capture *cappedBuffer) <-chan string {
lines := make(chan string)
go func() {
defer close(lines)
scanner := bufio.NewScanner(reader)
scanner.Buffer(make([]byte, 4096), 1<<20)
for scanner.Scan() {
line := scanner.Text()
if capture != nil {
_, _ = capture.Write([]byte(line + "\n"))
}
lines <- line
}
}()
return lines
}
func New(jobs JobStore, config Config, logger *log.Logger) (*Runner, error) {
if config.VideoDir == "" {
config.VideoDir = "/videos"
}
if config.YTDLPPath == "" {
config.YTDLPPath = "yt-dlp"
}
if config.FFprobePath == "" {
config.FFprobePath = "ffprobe"
}
if config.Concurrency == 0 {
config.Concurrency = 1
}
if config.Concurrency < 1 || config.Concurrency > 2 {
return nil, errors.New("worker concurrency must be 1 or 2")
}
if config.PollInterval <= 0 {
config.PollInterval = time.Second
}
if config.ProgressInterval <= 0 {
config.ProgressInterval = time.Second
}
if config.HeartbeatInterval <= 0 {
config.HeartbeatInterval = 5 * time.Second
}
if config.CancelInterval <= 0 {
config.CancelInterval = 500 * time.Millisecond
}
if logger == nil {
logger = log.Default()
}
return &Runner{store: jobs, config: config, logger: logger}, nil
}
func (r *Runner) Run(ctx context.Context) error {
sem := make(chan struct{}, r.config.Concurrency)
var wg sync.WaitGroup
ticker := time.NewTicker(r.config.PollInterval)
defer ticker.Stop()
for {
for len(sem) < cap(sem) && ctx.Err() == nil {
job, err := r.store.ClaimNext(ctx)
if errors.Is(err, pgx.ErrNoRows) {
break
}
if err != nil {
r.logger.Printf("ジョブ取得に失敗: %v", err)
break
}
sem <- struct{}{}
wg.Add(1)
go func() {
defer wg.Done()
defer func() { <-sem }()
r.ProcessJob(ctx, job)
}()
}
select {
case <-ctx.Done():
wg.Wait()
return nil
case <-ticker.C:
}
}
}
func (r *Runner) ProcessJob(ctx context.Context, job store.Job) {
workDir := filepath.Join(r.config.VideoDir, ".incoming", strconv.FormatInt(job.ID, 10))
if err := os.RemoveAll(workDir); err != nil {
r.fail(job.ID, "unknown", fmt.Sprintf("作業ディレクトリを初期化できません: %v", err))
return
}
if err := os.MkdirAll(workDir, 0o755); err != nil {
r.fail(job.ID, "unknown", fmt.Sprintf("作業ディレクトリを作成できません: %v", err))
return
}
defer os.RemoveAll(workDir)
relative, err := media.RelativeVideoPath(job.SeriesTitle, job.Title, job.EpisodeID)
if err != nil {
r.failIfRunning(job.ID, "invalid_output", redact(err.Error()))
return
}
downloaded, stderr, err := r.download(ctx, job, workDir)
if errors.Is(err, errShutdown) {
r.requeue(job.ID)
return
}
if errors.Is(err, errCanceled) {
return
}
if err != nil {
message := redact(stderr)
if message == "" {
message = redact(err.Error())
}
kind := classifyFailure(message)
if errors.Is(err, errInvalidOutput) {
kind = "invalid_output"
}
r.failIfRunning(job.ID, kind, message)
return
}
if r.canceled(job.ID) {
return
}
if err := r.probe(ctx, downloaded); err != nil {
if ctx.Err() != nil {
r.requeue(job.ID)
return
}
r.failIfRunning(job.ID, "invalid_output", redact(err.Error()))
return
}
if r.canceled(job.ID) {
return
}
if err := r.store.UpdateProgress(ctx, job.ID, store.Progress{Phase: "moving"}); err != nil {
if r.canceled(job.ID) {
return
}
r.failIfRunning(job.ID, "unknown", "移動状態を記録できません: "+err.Error())
return
}
destination := filepath.Join(r.config.VideoDir, relative)
if err := os.MkdirAll(filepath.Dir(destination), 0o755); err != nil {
r.failIfRunning(job.ID, "unknown", "保存先を作成できません: "+err.Error())
return
}
if err := os.Rename(downloaded, destination); err != nil {
r.failIfRunning(job.ID, "unknown", "動画を保存先へ移動できません: "+err.Error())
return
}
info, err := os.Stat(destination)
if err != nil {
r.failIfRunning(job.ID, "unknown", "保存後の動画を確認できません: "+err.Error())
return
}
if err := r.store.Complete(ctx, job.ID, filepath.ToSlash(relative), info.Size()); err != nil {
if r.canceled(job.ID) {
if removeErr := os.Remove(destination); removeErr != nil && !errors.Is(removeErr, os.ErrNotExist) {
r.logger.Printf("取り消されたジョブ %d の保存済みファイル削除に失敗: %v", job.ID, removeErr)
}
return
}
r.logger.Printf("ジョブ %d の完了記録に失敗: %v", job.ID, err)
}
}
func (r *Runner) download(ctx context.Context, job store.Job, workDir string) (string, string, error) {
output := filepath.Join(workDir, "%(id)s.%(ext)s")
args := []string{"--no-playlist", "--no-simulate", "--quiet", "--progress", "--newline",
"--continue", "--retries", "0", "--fragment-retries", "0", "--extractor-retries", "0",
"-f", "bv*[vcodec^=avc1]+ba/b", "--merge-output-format", "mp4",
"--progress-template", `download:{"event":"download","format_id":%(info.format_id)j,"progress":%(progress)j}`,
"--progress-template", `postprocess:{"event":"postprocess","progress":%(progress)j}`,
"--print", `after_move:{"event":"after_move","filepath":%(filepath)j}`, "-o", output,
"https://tver.jp/episodes/" + job.EpisodeID}
cmd := exec.Command(r.config.YTDLPPath, args...)
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
stdout, err := cmd.StdoutPipe()
if err != nil {
return "", "", err
}
stderrPipe, err := cmd.StderrPipe()
if err != nil {
return "", "", err
}
if err := cmd.Start(); err != nil {
return "", "", err
}
stderr := cappedBuffer{limit: 64 << 10}
stdoutLines := scanLines(stdout, nil)
stderrLines := scanLines(stderrPipe, &stderr)
waited := make(chan error, 1)
go func() { waited <- cmd.Wait() }()
heartbeat := time.NewTicker(r.config.HeartbeatInterval)
cancelCheck := time.NewTicker(r.config.CancelInterval)
defer heartbeat.Stop()
defer cancelCheck.Stop()
_ = r.store.Heartbeat(ctx, job.ID)
var afterMove string
var pending *store.Progress
nextProgress := time.Time{}
lastPhase := ""
recordProgress := func(progress store.Progress) {
pending = &progress
now := time.Now()
if progress.Phase != lastPhase || now.After(nextProgress) {
_ = r.store.UpdateProgress(ctx, job.ID, progress)
nextProgress = now.Add(r.config.ProgressInterval)
lastPhase = progress.Phase
pending = nil
}
}
stopping := ""
var forceKill <-chan time.Time
ctxDone := ctx.Done()
waitComplete := false
var waitErr error
for {
if waitComplete && stdoutLines == nil && stderrLines == nil {
if pending != nil && stopping == "" {
updateCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
_ = r.store.UpdateProgress(updateCtx, job.ID, *pending)
cancel()
}
if stopping == "canceled" {
return "", stderr.String(), errCanceled
}
if stopping == "shutdown" {
return "", stderr.String(), errShutdown
}
if waitErr != nil {
r.logger.Printf("yt-dlp ジョブ %d 失敗: %s", job.ID, redact(stderr.String()))
return "", stderr.String(), waitErr
}
path, err := pathInside(workDir, afterMove)
if err != nil {
return "", stderr.String(), err
}
return path, stderr.String(), nil
}
select {
case line, ok := <-stdoutLines:
if !ok {
stdoutLines = nil
continue
}
if path, ok := parseAfterMove(line); ok {
afterMove = path
continue
}
if progress, ok := parseProgress(line); ok {
recordProgress(progress)
}
case line, ok := <-stderrLines:
if !ok {
stderrLines = nil
continue
}
if progress, ok := parseProgress(line); ok {
recordProgress(progress)
}
case <-heartbeat.C:
_ = r.store.Heartbeat(ctx, job.ID)
case <-cancelCheck.C:
if r.canceled(job.ID) && stopping == "" {
stopping = "canceled"
if !waitComplete {
stopProcessGroup(cmd)
forceKill = time.After(2 * time.Second)
}
}
case <-ctxDone:
if stopping == "" {
stopping = "shutdown"
if !waitComplete {
stopProcessGroup(cmd)
forceKill = time.After(2 * time.Second)
}
}
ctxDone = nil
case <-forceKill:
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
forceKill = nil
case waitErr = <-waited:
waitComplete = true
waited = nil
forceKill = nil
}
}
}
func stopProcessGroup(cmd *exec.Cmd) {
if cmd.Process == nil {
return
}
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM)
}
func pathInside(root, candidate string) (string, error) {
if candidate == "" {
return "", errors.New("yt-dlp did not report after_move filepath")
}
rootAbs, err := filepath.Abs(root)
if err != nil {
return "", err
}
candidateAbs, err := filepath.Abs(candidate)
if err != nil {
return "", err
}
rel, err := filepath.Rel(rootAbs, candidateAbs)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("%w: yt-dlp filepath leaves working directory", errInvalidOutput)
}
info, err := os.Lstat(candidateAbs)
if err != nil {
return "", fmt.Errorf("%w: yt-dlp output file is missing: %v", errInvalidOutput, err)
}
if !info.Mode().IsRegular() {
return "", fmt.Errorf("%w: yt-dlp output is not a regular file", errInvalidOutput)
}
rootReal, err := filepath.EvalSymlinks(rootAbs)
if err != nil {
return "", fmt.Errorf("%w: resolve working directory: %v", errInvalidOutput, err)
}
candidateReal, err := filepath.EvalSymlinks(candidateAbs)
if err != nil {
return "", fmt.Errorf("%w: resolve yt-dlp output: %v", errInvalidOutput, err)
}
realRelative, err := filepath.Rel(rootReal, candidateReal)
if err != nil || realRelative == ".." || strings.HasPrefix(realRelative, ".."+string(filepath.Separator)) {
return "", fmt.Errorf("%w: yt-dlp output resolves outside working directory", errInvalidOutput)
}
return candidateAbs, nil
}
func (r *Runner) probe(ctx context.Context, path string) error {
cmd := exec.CommandContext(ctx, r.config.FFprobePath, "-v", "error", "-show_entries",
"format=format_name", "-show_entries", "stream=codec_type,codec_name", "-of", "json", path)
output, err := cmd.Output()
if err != nil {
return fmt.Errorf("ffprobe failed: %w", err)
}
var result struct {
Format struct {
FormatName string `json:"format_name"`
} `json:"format"`
Streams []struct {
CodecType string `json:"codec_type"`
CodecName string `json:"codec_name"`
} `json:"streams"`
}
if err := json.Unmarshal(output, &result); err != nil {
return fmt.Errorf("ffprobe output is not JSON: %w", err)
}
video, audio := false, false
for _, stream := range result.Streams {
video = video || stream.CodecType == "video" && stream.CodecName == "h264"
audio = audio || stream.CodecType == "audio" && stream.CodecName == "aac"
}
if !strings.Contains(result.Format.FormatName, "mp4") || !video || !audio {
return fmt.Errorf("expected MP4 with H.264 and AAC, got format=%s video=%t audio=%t", result.Format.FormatName, video, audio)
}
return nil
}
func (r *Runner) canceled(id int64) bool {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
job, err := r.store.GetJob(ctx, id)
return err == nil && job.Status == "canceled"
}
func (r *Runner) failIfRunning(id int64, kind, message string) {
if r.canceled(id) {
return
}
r.fail(id, kind, message)
}
func (r *Runner) fail(id int64, kind, message string) {
if len(message) > 8192 {
message = message[:8192]
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := r.store.Fail(ctx, id, kind, message); err != nil {
r.logger.Printf("ジョブ %d の失敗記録に失敗: %v", id, err)
}
}
func (r *Runner) requeue(id int64) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := r.store.RequeueRunning(ctx, id); err != nil && !errors.Is(err, store.ErrInvalidTransition) {
r.logger.Printf("ジョブ %d の待機中への復帰に失敗: %v", id, err)
}
}
func classifyFailure(message string) string {
lower := strings.ToLower(message)
for _, test := range []struct {
kind string
patterns []string
}{
{"drm", []string{"drm", "digital rights management", "protected by"}},
{"geo", []string{"geo-restricted", "not available in your country", "not available from your location", "geo restriction", "region restriction", "geographic restriction"}},
{"network", []string{"network", "connection", "timed out", "temporary failure", "unable to download", "proxy error", "http error 5"}},
{"unavailable", []string{"media_not_found", "video unavailable", "not available", "no longer available", "http error 404", "episode does not exist"}},
{"extract", []string{"extractor error", "unable to extract", "unsupported url", "no video formats", "after_move", "output file is missing"}},
} {
for _, pattern := range test.patterns {
if strings.Contains(lower, pattern) {
return test.kind
}
}
}
return "unknown"
}
func redact(message string) string {
message = headerSecretRE.ReplaceAllString(message, "$1$2[REDACTED]")
return strings.TrimSpace(paramSecretRE.ReplaceAllString(message, "$1$2[REDACTED]"))
}
internal/worker/runner_test.go
+678
View File
@@ -0,0 +1,678 @@
package worker
import (
"context"
"errors"
"io"
"log"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/media"
"gitea.kazu634.com/kazu634/tver/internal/store"
"github.com/jackc/pgx/v5"
)
type fakeStore struct {
mu sync.Mutex
job store.Job
progress []store.Progress
heartbeats int
completed bool
completePath string
completeSize int64
failedKind, failedMessage string
requeued bool
claimed bool
}
type queueStore struct {
mu sync.Mutex
jobs []store.Job
claimCounts map[int64]int
active int
maxActive int
}
func (f *queueStore) ClaimNext(context.Context) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
for i := range f.jobs {
if f.jobs[i].Status != "queued" {
continue
}
f.jobs[i].Status = "running"
f.claimCounts[f.jobs[i].ID]++
f.active++
if f.active > f.maxActive {
f.maxActive = f.active
}
return f.jobs[i], nil
}
return store.Job{}, pgx.ErrNoRows
}
func (f *queueStore) UpdateProgress(_ context.Context, id int64, _ store.Progress) error {
return f.requireRunning(id)
}
func (f *queueStore) Heartbeat(_ context.Context, id int64) error {
return f.requireRunning(id)
}
func (f *queueStore) requireRunning(id int64) error {
f.mu.Lock()
defer f.mu.Unlock()
for i := range f.jobs {
if f.jobs[i].ID == id && f.jobs[i].Status == "running" {
return nil
}
}
return store.ErrInvalidTransition
}
func (f *queueStore) GetJob(_ context.Context, id int64) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
for _, job := range f.jobs {
if job.ID == id {
return job, nil
}
}
return store.Job{}, pgx.ErrNoRows
}
func (f *queueStore) Complete(_ context.Context, id int64, _ string, _ int64) error {
return f.finish(id, "succeeded")
}
func (f *queueStore) Fail(_ context.Context, id int64, _, _ string) error {
return f.finish(id, "failed")
}
func (f *queueStore) RequeueRunning(_ context.Context, id int64) error {
return f.finish(id, "queued")
}
func (f *queueStore) finish(id int64, status string) error {
f.mu.Lock()
defer f.mu.Unlock()
for i := range f.jobs {
if f.jobs[i].ID == id && f.jobs[i].Status == "running" {
f.jobs[i].Status = status
f.active--
return nil
}
}
return store.ErrInvalidTransition
}
func (f *queueStore) state() ([]store.Job, map[int64]int, int) {
f.mu.Lock()
defer f.mu.Unlock()
jobs := append([]store.Job(nil), f.jobs...)
counts := make(map[int64]int, len(f.claimCounts))
for id, count := range f.claimCounts {
counts[id] = count
}
return jobs, counts, f.maxActive
}
func (f *fakeStore) ClaimNext(context.Context) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
if f.claimed || f.job.Status != "queued" {
return store.Job{}, pgx.ErrNoRows
}
f.claimed = true
f.job.Status = "running"
return f.job, nil
}
func (f *fakeStore) UpdateProgress(_ context.Context, _ int64, p store.Progress) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.progress = append(f.progress, p)
return nil
}
func (f *fakeStore) Heartbeat(context.Context, int64) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.heartbeats++
return nil
}
func (f *fakeStore) GetJob(context.Context, int64) (store.Job, error) {
f.mu.Lock()
defer f.mu.Unlock()
return f.job, nil
}
func (f *fakeStore) Complete(_ context.Context, _ int64, path string, size int64) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.job.Status = "succeeded"
f.completed, f.completePath, f.completeSize = true, path, size
return nil
}
func (f *fakeStore) Fail(_ context.Context, _ int64, kind, message string) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.job.Status = "failed"
f.failedKind, f.failedMessage = kind, message
return nil
}
func (f *fakeStore) RequeueRunning(context.Context, int64) error {
f.mu.Lock()
defer f.mu.Unlock()
if f.job.Status != "running" {
return store.ErrInvalidTransition
}
f.job.Status = "queued"
f.requeued = true
return nil
}
func (f *fakeStore) setStatus(status string) {
f.mu.Lock()
defer f.mu.Unlock()
f.job.Status = status
}
func (f *fakeStore) heartbeatCount() int {
f.mu.Lock()
defer f.mu.Unlock()
return f.heartbeats
}
func testRunner(t *testing.T, jobs *fakeStore, videoDir string) *Runner {
t.Helper()
ytdlp := filepath.Join("testdata", "fake-yt-dlp.sh")
ffprobe := filepath.Join("testdata", "fake-ffprobe.sh")
for _, path := range []string{ytdlp, ffprobe} {
info, err := os.Stat(path)
if err != nil || info.Mode().Perm()&0o111 == 0 {
t.Fatalf("偽コマンドが実行可能ではありません: %s: %v", path, err)
}
}
runner, err := New(jobs, Config{VideoDir: videoDir, YTDLPPath: ytdlp, FFprobePath: ffprobe,
Concurrency: 1, PollInterval: 5 * time.Millisecond, ProgressInterval: time.Millisecond,
HeartbeatInterval: 10 * time.Millisecond, CancelInterval: 5 * time.Millisecond}, log.New(io.Discard, "", 0))
if err != nil {
t.Fatal(err)
}
return runner
}
func runningJob() store.Job {
return store.Job{ID: 7, EpisodeID: "ep123", SeriesTitle: "番組名", Title: "第1話", Status: "running"}
}
func relativeVideoPath(t *testing.T, seriesTitle, episodeTitle, episodeID string) string {
t.Helper()
path, err := media.RelativeVideoPath(seriesTitle, episodeTitle, episodeID)
if err != nil {
t.Fatal(err)
}
return path
}
func TestProcessJobDownloadsProbesMovesAndCleans(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
argsFile := filepath.Join(dir, "args")
t.Setenv("FAKE_ARGS_FILE", argsFile)
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
want := filepath.ToSlash(relativeVideoPath(t, "番組名", "第1話", "ep123"))
if !jobs.completed || jobs.completePath != want || jobs.completeSize == 0 {
t.Fatalf("complete: done=%v path=%q size=%d", jobs.completed, jobs.completePath, jobs.completeSize)
}
if _, err := os.Stat(filepath.Join(dir, filepath.FromSlash(want))); err != nil {
t.Fatalf("保存先: %v", err)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
args, err := os.ReadFile(argsFile)
if err != nil {
t.Fatal(err)
}
argumentText := string(args)
for _, want := range []string{"--newline", "--no-playlist", "--no-simulate", "--quiet", "--progress", "--continue", "--merge-output-format", "bv*[vcodec^=avc1]+ba/b", `download:{"event":"download","format_id":%(info.format_id)j,"progress":%(progress)j}`, `postprocess:{"event":"postprocess","progress":%(progress)j}`, `after_move:{"event":"after_move","filepath":%(filepath)j}`, filepath.Join(dir, ".incoming", "7", "%(id)s.%(ext)s")} {
if !strings.Contains(argumentText, want) {
t.Errorf("yt-dlp 引数に %q がない: %s", want, argumentText)
}
}
if strings.Contains(argumentText, "番組名") || strings.Contains(argumentText, "第1話") {
t.Errorf("出力テンプレートに番組情報が入った: %s", argumentText)
}
phases := make(map[string]bool)
for _, progress := range jobs.progress {
phases[progress.Phase] = true
}
for _, phase := range []string{"video", "audio", "merging", "moving"} {
if !phases[phase] {
t.Errorf("進捗段階 %s がない: %+v", phase, jobs.progress)
}
}
if jobs.heartbeats < 2 {
t.Errorf("heartbeat が少ない: %d", jobs.heartbeats)
}
}
func TestProcessJobRejectsLongEpisodeIDAndCleans(t *testing.T) {
dir := t.TempDir()
argsFile := filepath.Join(dir, "yt-dlp-args")
t.Setenv("FAKE_ARGS_FILE", argsFile)
workDir := filepath.Join(dir, ".incoming", "7")
if err := os.MkdirAll(workDir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(workDir, "stale"), []byte("stale"), 0o644); err != nil {
t.Fatal(err)
}
job := runningJob()
job.EpisodeID = strings.Repeat("i", 65)
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
if jobs.completed || jobs.failedKind != "invalid_output" || !strings.Contains(jobs.failedMessage, "64バイト以下") {
t.Fatalf("長いIDの失敗: kind=%q message=%q completed=%v", jobs.failedKind, jobs.failedMessage, jobs.completed)
}
if _, err := os.Stat(argsFile); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("yt-dlp が起動された: %v", err)
}
if _, err := os.Stat(workDir); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestProcessJobKeepsPercentNamesOutOfYTDLPTemplate(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
argsFile := filepath.Join(dir, "args")
t.Setenv("FAKE_ARGS_FILE", argsFile)
job := runningJob()
job.SeriesTitle = "100%番組"
job.Title = "第%1話"
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
if !jobs.completed {
t.Fatalf("%% を含む番組を完了できなかった: %+v", jobs)
}
args, err := os.ReadFile(argsFile)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(args), job.SeriesTitle) || strings.Contains(string(args), job.Title) {
t.Fatalf("番組情報が yt-dlp 引数に入った: %s", args)
}
want := filepath.ToSlash(relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID))
if jobs.completePath != want {
t.Fatalf("保存先=%q want=%q", jobs.completePath, want)
}
}
func TestProcessJobReplacesExistingDestination(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
job := runningJob()
relative := relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID)
destination := filepath.Join(dir, relative)
if err := os.MkdirAll(filepath.Dir(destination), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(destination, []byte("stale file from previous attempt"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
got, err := os.ReadFile(destination)
if err != nil {
t.Fatal(err)
}
if !jobs.completed || string(got) != "fake mp4" || jobs.completeSize != int64(len(got)) {
t.Fatalf("既存ファイルの置換: completed=%v content=%q size=%d", jobs.completed, got, jobs.completeSize)
}
}
func TestProcessJobRejectsAfterMoveSymlinkOutsideWorkDirectory(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "symlink")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
outside := filepath.Join(t.TempDir(), "outside.mp4")
if err := os.WriteFile(outside, []byte("outside video"), 0o644); err != nil {
t.Fatal(err)
}
t.Setenv("FAKE_OUTSIDE_FILE", outside)
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
if jobs.completed || jobs.failedKind != "invalid_output" {
t.Fatalf("作業ディレクトリ外を指すシンボリックリンクを受理した: %+v", jobs)
}
if got, err := os.ReadFile(outside); err != nil || string(got) != "outside video" {
t.Fatalf("作業ディレクトリ外のファイルを変更した: %q %v", got, err)
}
}
func TestPathInsideSymlinkDerivedCases(t *testing.T) {
root := t.TempDir()
insideDir := filepath.Join(root, "real")
if err := os.Mkdir(insideDir, 0o755); err != nil {
t.Fatal(err)
}
insideFile := filepath.Join(insideDir, "video.mp4")
if err := os.WriteFile(insideFile, []byte("video"), 0o644); err != nil {
t.Fatal(err)
}
outsideDir := t.TempDir()
outsideFile := filepath.Join(outsideDir, "outside.mp4")
if err := os.WriteFile(outsideFile, []byte("outside"), 0o644); err != nil {
t.Fatal(err)
}
internalFileLink := filepath.Join(root, "internal-file-link.mp4")
internalDirLink := filepath.Join(root, "internal-dir-link")
externalDirLink := filepath.Join(root, "external-dir-link")
rootLink := filepath.Join(t.TempDir(), "root-link")
for link, target := range map[string]string{
internalFileLink: insideFile,
internalDirLink: insideDir,
externalDirLink: outsideDir,
rootLink: root,
} {
if err := os.Symlink(target, link); err != nil {
t.Fatal(err)
}
}
for _, tt := range []struct {
name, testRoot, candidate string
wantOK bool
}{
{"通常ファイル", root, insideFile, true},
{"作業ディレクトリ内を指すファイルリンク", root, internalFileLink, false},
{"after_moveがディレクトリ", root, insideDir, false},
{"作業ディレクトリ内を指すディレクトリリンク配下", root, filepath.Join(internalDirLink, "video.mp4"), true},
{"作業ディレクトリ外を指すディレクトリリンク配下", root, filepath.Join(externalDirLink, "outside.mp4"), false},
{"作業ディレクトリ自体がリンク", rootLink, filepath.Join(rootLink, "real", "video.mp4"), true},
} {
t.Run(tt.name, func(t *testing.T) {
got, err := pathInside(tt.testRoot, tt.candidate)
if tt.wantOK {
want, absErr := filepath.Abs(tt.candidate)
if absErr != nil {
t.Fatal(absErr)
}
if err != nil || got != want {
t.Fatalf("pathInside()=(%q, %v) want=(%q, nil)", got, err, want)
}
return
}
if !errors.Is(err, errInvalidOutput) || got != "" {
t.Fatalf("pathInside()=(%q, %v), invalid_output を期待", got, err)
}
})
}
}
func TestProcessJobDestinationDirectoryFailureCleansIncoming(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
job := runningJob()
seriesDir := filepath.Join(dir, filepath.Dir(relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID)))
if err := os.WriteFile(seriesDir, []byte("blocks directory"), 0o644); err != nil {
t.Fatal(err)
}
jobs := &fakeStore{job: job}
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
if jobs.completed || jobs.failedKind != "unknown" {
t.Fatalf("保存先作成失敗: %+v", jobs)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestFailureClassificationAndRedaction(t *testing.T) {
for _, tt := range []struct{ mode, kind string }{
{"drm", "drm"}, {"geo", "geo"}, {"network", "network"},
{"unavailable", "unavailable"}, {"extract", "extract"}, {"unknown", "unknown"},
} {
t.Run(tt.mode, func(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", tt.mode)
dir := t.TempDir()
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
if jobs.failedKind != tt.kind || jobs.completed || strings.Contains(jobs.failedMessage, "secret-value") {
t.Fatalf("failure: kind=%q message=%q completed=%v", jobs.failedKind, jobs.failedMessage, jobs.completed)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
})
}
}
func TestInvalidProbeFailsAndCleans(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "success")
for _, mode := range []string{"bad-container", "bad-video", "bad-audio", "non-json", "failure"} {
t.Run(mode, func(t *testing.T) {
t.Setenv("FAKE_FFPROBE_MODE", mode)
dir := t.TempDir()
jobs := &fakeStore{job: runningJob()}
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
if jobs.failedKind != "invalid_output" || jobs.completed {
t.Fatalf("ffprobe 不合格: kind=%q complete=%v", jobs.failedKind, jobs.completed)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
})
}
}
func TestProgressIsThrottledButHeartbeatContinues(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "burst")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
jobs := &fakeStore{job: runningJob()}
runner := testRunner(t, jobs, dir)
runner.config.ProgressInterval = 50 * time.Millisecond
runner.ProcessJob(context.Background(), jobs.job)
downloadUpdates := 0
for _, progress := range jobs.progress {
if progress.Phase == "video" {
downloadUpdates++
}
}
if downloadUpdates > 2 || downloadUpdates == 0 {
t.Fatalf("進捗の間引き: %d updates (%+v)", downloadUpdates, jobs.progress)
}
if jobs.heartbeats < 5 {
t.Fatalf("進捗が止まった間の heartbeat: %d", jobs.heartbeats)
}
}
func TestRunConcurrencyTwoDoesNotDuplicateAndIsolatesFailure(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "mixed")
t.Setenv("FAKE_FFPROBE_MODE", "valid")
dir := t.TempDir()
jobs := &queueStore{
jobs: []store.Job{
{ID: 7, EpisodeID: "ep-ok", SeriesTitle: "成功番組", Title: "成功回", Status: "queued"},
{ID: 8, EpisodeID: "ep-fail", SeriesTitle: "失敗番組", Title: "失敗回", Status: "queued"},
},
claimCounts: make(map[int64]int),
}
runner, err := New(jobs, Config{VideoDir: dir,
YTDLPPath: filepath.Join("testdata", "fake-yt-dlp.sh"), FFprobePath: filepath.Join("testdata", "fake-ffprobe.sh"),
Concurrency: 2, PollInterval: 5 * time.Millisecond, ProgressInterval: time.Millisecond,
HeartbeatInterval: 10 * time.Millisecond, CancelInterval: 5 * time.Millisecond}, log.New(io.Discard, "", 0))
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- runner.Run(ctx) }()
deadline := time.Now().Add(3 * time.Second)
for time.Now().Before(deadline) {
state, _, _ := jobs.state()
if state[0].Status == "succeeded" && state[1].Status == "failed" {
break
}
time.Sleep(5 * time.Millisecond)
}
cancel()
if err := <-done; err != nil {
t.Fatal(err)
}
state, counts, maxActive := jobs.state()
if state[0].Status != "succeeded" || state[1].Status != "failed" {
t.Fatalf("片方の失敗がもう片方へ波及した: %+v", state)
}
if counts[7] != 1 || counts[8] != 1 {
t.Fatalf("ジョブを二重取得した: %+v", counts)
}
if maxActive != 2 {
t.Fatalf("並行実行数の最大値=%d want=2", maxActive)
}
for _, id := range []int64{7, 8} {
if _, err := os.Stat(filepath.Join(dir, ".incoming", strconv.FormatInt(id, 10))); !errors.Is(err, os.ErrNotExist) {
t.Errorf("ジョブ %d の作業ディレクトリが残った: %v", id, err)
}
}
}
func TestCancelStopsProcessGroupAndLeavesCanceled(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "hang-drm")
dir := t.TempDir()
ready := filepath.Join(dir, "ready")
terminated := filepath.Join(dir, "terminated")
t.Setenv("FAKE_READY_FILE", ready)
t.Setenv("FAKE_TERM_FILE", terminated)
jobs := &fakeStore{job: runningJob()}
runner := testRunner(t, jobs, dir)
job := jobs.job
done := make(chan struct{})
go func() { runner.ProcessJob(context.Background(), job); close(done) }()
waitForFile(t, ready)
waitForHeartbeatCount(t, jobs, 2)
jobs.setStatus("canceled")
waitDone(t, done)
if jobs.failedKind != "" || jobs.requeued || jobs.completed || jobs.job.Status != "canceled" {
t.Fatalf("取り消し後: %+v", jobs)
}
waitForFile(t, terminated)
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestShutdownStopsProcessAndRequeues(t *testing.T) {
t.Setenv("FAKE_YTDLP_MODE", "hang")
dir := t.TempDir()
ready := filepath.Join(dir, "ready")
t.Setenv("FAKE_READY_FILE", ready)
jobs := &fakeStore{job: runningJob()}
ctx, cancel := context.WithCancel(context.Background())
runner := testRunner(t, jobs, dir)
job := jobs.job
done := make(chan struct{})
go func() { runner.ProcessJob(ctx, job); close(done) }()
waitForFile(t, ready)
cancel()
waitDone(t, done)
if !jobs.requeued || jobs.job.Status != "queued" || jobs.failedKind != "" {
t.Fatalf("終了時の復帰: %+v", jobs)
}
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
t.Fatalf("作業ディレクトリが残った: %v", err)
}
}
func TestRunnerRejectsConcurrencyOutsideRange(t *testing.T) {
for _, concurrency := range []int{-1, 3} {
if _, err := New(&fakeStore{}, Config{Concurrency: concurrency}, nil); err == nil {
t.Errorf("concurrency=%d を許した", concurrency)
}
}
}
func TestFailureClassificationOrderAndRedaction(t *testing.T) {
for _, tt := range []struct{ message, want string }{
{"DRM geo-restricted connection video unavailable unable to extract", "drm"},
{"geo-restricted connection video unavailable unable to extract", "geo"},
{"connection video unavailable unable to extract", "network"},
{"video unavailable unable to extract", "unavailable"},
{"unable to extract", "extract"},
{"different error", "unknown"},
} {
if got := classifyFailure(tt.message); got != tt.want {
t.Errorf("classifyFailure(%q)=%q want=%q", tt.message, got, tt.want)
}
}
redacted := redact("Authorization: Bearer bearer-secret\nCookie: session=session-secret\ntoken=token-secret platform_uid=uid-secret")
for _, secret := range []string{"bearer-secret", "session-secret", "token-secret", "uid-secret"} {
if strings.Contains(redacted, secret) {
t.Errorf("秘密が残った: %q", redacted)
}
}
}
func waitForFile(t *testing.T, path string) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
if _, err := os.Stat(path); err == nil {
return
}
time.Sleep(5 * time.Millisecond)
}
t.Fatalf("ファイル待ちがタイムアウト: %s", path)
}
func waitDone(t *testing.T, done <-chan struct{}) {
t.Helper()
select {
case <-done:
case <-time.After(3 * time.Second):
t.Fatal("worker の終了待ちがタイムアウト")
}
}
func waitForHeartbeatCount(t *testing.T, jobs *fakeStore, want int) {
t.Helper()
deadline := time.Now().Add(2 * time.Second)
for time.Now().Before(deadline) {
if jobs.heartbeatCount() >= want {
return
}
time.Sleep(5 * time.Millisecond)
}
t.Fatalf("heartbeat 待ちがタイムアウト: got=%d want=%d", jobs.heartbeatCount(), want)
}
internal/worker/subscriptions.go
+156
View File
@@ -0,0 +1,156 @@
package worker
import (
"context"
"errors"
"log"
"strings"
"time"
"unicode/utf8"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
type SubscriptionStore interface {
ListDueSubscriptions(context.Context, time.Time) ([]store.Subscription, error)
CompleteSubscriptionCheck(context.Context, string, []store.Episode, int, time.Time) (int, error)
RecordSubscriptionCheckFailure(context.Context, string, time.Time, string) error
}
type SeriesFetcher interface {
FetchSeriesEpisodes(context.Context, string) ([]tver.Episode, error)
}
type SubscriptionSchedulerConfig struct {
ScanInterval time.Duration
DueAfter time.Duration
SeriesInterval time.Duration
PreviewMaxSeconds int
Now func() time.Time
After func(time.Duration) <-chan time.Time
}
type SubscriptionScheduler struct {
store SubscriptionStore
fetcher SeriesFetcher
config SubscriptionSchedulerConfig
logger *log.Logger
}
func NewSubscriptionScheduler(subscriptions SubscriptionStore, fetcher SeriesFetcher, config SubscriptionSchedulerConfig, logger *log.Logger) (*SubscriptionScheduler, error) {
if subscriptions == nil || fetcher == nil {
return nil, errors.New("subscription store and fetcher are required")
}
if config.ScanInterval <= 0 {
config.ScanInterval = time.Hour
}
if config.DueAfter <= 0 {
config.DueAfter = 24 * time.Hour
}
if config.SeriesInterval <= 0 {
config.SeriesInterval = 3 * time.Second
}
if config.PreviewMaxSeconds == 0 {
config.PreviewMaxSeconds = tver.PreviewMaxSeconds
}
if config.PreviewMaxSeconds < 0 {
return nil, errors.New("preview threshold must not be negative")
}
if config.Now == nil {
config.Now = time.Now
}
if config.After == nil {
config.After = time.After
}
if logger == nil {
logger = log.Default()
}
return &SubscriptionScheduler{store: subscriptions, fetcher: fetcher, config: config, logger: logger}, nil
}
// Run は起動直後に確認し、その後は ScanInterval ごとに繰り返す。
func (s *SubscriptionScheduler) Run(ctx context.Context) {
for {
s.checkDue(ctx)
if !s.wait(ctx, s.config.ScanInterval) {
return
}
}
}
func (s *SubscriptionScheduler) checkDue(ctx context.Context) {
now := s.config.Now()
subscriptions, err := s.store.ListDueSubscriptions(ctx, now.Add(-s.config.DueAfter))
if err != nil {
if ctx.Err() == nil {
s.logger.Printf("確認対象の登録番組を取得できません: %v", err)
}
return
}
for index, subscription := range subscriptions {
if index > 0 && !s.wait(ctx, s.config.SeriesInterval) {
return
}
episodes, err := s.fetcher.FetchSeriesEpisodes(ctx, subscription.SeriesID)
if err != nil {
if ctx.Err() != nil {
return
}
message := shortError(err)
if recordErr := s.store.RecordSubscriptionCheckFailure(ctx, subscription.SeriesID, s.config.Now(), message); recordErr != nil &&
!errors.Is(recordErr, store.ErrInvalidTransition) {
s.logger.Printf("登録番組 %s の確認エラーを記録できません: %v", subscription.SeriesID, recordErr)
}
s.logger.Printf("登録番組 %s の公開中エピソードを取得できません: %s", subscription.SeriesID, message)
continue
}
items := make([]store.Episode, 0, len(episodes))
for _, episode := range episodes {
items = append(items, subscriptionEpisode(episode))
}
queued, err := s.store.CompleteSubscriptionCheck(ctx, subscription.SeriesID, items,
s.config.PreviewMaxSeconds, s.config.Now())
if err != nil {
if ctx.Err() != nil {
return
}
if !errors.Is(err, store.ErrSubscriptionPaused) && !errors.Is(err, store.ErrInvalidTransition) {
s.logger.Printf("登録番組 %s の確認結果を保存できません: %v", subscription.SeriesID, err)
}
continue
}
s.logger.Printf("登録番組 %s を確認しました（新規ジョブ=%d）", subscription.SeriesID, queued)
}
}
func (s *SubscriptionScheduler) wait(ctx context.Context, duration time.Duration) bool {
select {
case <-ctx.Done():
return false
case <-s.config.After(duration):
return true
}
}
func subscriptionEpisode(episode tver.Episode) store.Episode {
item := store.Episode{ID: episode.ID, SeriesID: episode.SeriesID, SeriesTitle: episode.SeriesTitle,
Title: episode.Title, BroadcasterName: episode.BroadcasterName,
DurationSeconds: int(episode.Duration), ThumbnailPath: episode.ThumbnailPath}
if episode.EndAt > 0 {
end := time.Unix(episode.EndAt, 0)
item.EndAt = &end
}
return item
}
func shortError(err error) string {
message := redact(err.Error())
message = strings.Join(strings.Fields(message), " ")
const maxRunes = 512
if utf8.RuneCountInString(message) <= maxRunes {
return message
}
runes := []rune(message)
return string(runes[:maxRunes])
}
internal/worker/subscriptions_test.go
+253
View File
@@ -0,0 +1,253 @@
package worker
import (
"bytes"
"context"
"errors"
"log"
"strings"
"sync"
"testing"
"time"
"gitea.kazu634.com/kazu634/tver/internal/store"
"gitea.kazu634.com/kazu634/tver/internal/tver"
)
type schedulerCompletion struct {
seriesID string
episodes []store.Episode
limit int
at time.Time
}
type schedulerFailure struct {
seriesID, message string
at time.Time
}
type fakeSubscriptionStore struct {
mu sync.Mutex
due []store.Subscription
cutoffs []time.Time
completions []schedulerCompletion
failures []schedulerFailure
listErr error
completeErr error
}
func (f *fakeSubscriptionStore) ListDueSubscriptions(_ context.Context, cutoff time.Time) ([]store.Subscription, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.cutoffs = append(f.cutoffs, cutoff)
return append([]store.Subscription(nil), f.due...), f.listErr
}
func (f *fakeSubscriptionStore) CompleteSubscriptionCheck(_ context.Context, seriesID string, episodes []store.Episode, limit int, at time.Time) (int, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.completions = append(f.completions, schedulerCompletion{seriesID, append([]store.Episode(nil), episodes...), limit, at})
return len(episodes), f.completeErr
}
func (f *fakeSubscriptionStore) RecordSubscriptionCheckFailure(_ context.Context, seriesID string, at time.Time, message string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.failures = append(f.failures, schedulerFailure{seriesID, message, at})
return nil
}
type fakeSeriesFetcher struct {
mu sync.Mutex
calls []string
episodes map[string][]tver.Episode
errors map[string]error
block bool
started chan string
}
func (f *fakeSeriesFetcher) FetchSeriesEpisodes(ctx context.Context, seriesID string) ([]tver.Episode, error) {
f.mu.Lock()
f.calls = append(f.calls, seriesID)
block := f.block
started := f.started
episodes := append([]tver.Episode(nil), f.episodes[seriesID]...)
err := f.errors[seriesID]
f.mu.Unlock()
if started != nil {
started <- seriesID
}
if block {
<-ctx.Done()
return nil, ctx.Err()
}
return episodes, err
}
func TestSubscriptionSchedulerChecksImmediatelyAndMapsEpisodes(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}, {SeriesID: "two"}}}
fetcher := &fakeSeriesFetcher{episodes: map[string][]tver.Episode{
"one": {{ID: "ep1", SeriesID: "one", SeriesTitle: "番組", Title: "回", BroadcasterName: "局", Duration: 180, EndAt: 1_800_000_000, ThumbnailPath: "/image"}},
"two": {{ID: "ep2", SeriesID: "two", Duration: 181}},
}, errors: map[string]error{}}
var waits []time.Duration
after := func(duration time.Duration) <-chan time.Time {
waits = append(waits, duration)
ch := make(chan time.Time, 1)
ch <- now
return ch
}
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
DueAfter: 24 * time.Hour, SeriesInterval: 4 * time.Second,
Now: func() time.Time { return now }, After: after,
}, log.New(&bytes.Buffer{}, "", 0))
if err != nil {
t.Fatal(err)
}
scheduler.checkDue(context.Background())
if len(subscriptions.cutoffs) != 1 || !subscriptions.cutoffs[0].Equal(now.Add(-24*time.Hour)) {
t.Fatalf("確認対象日時: %v", subscriptions.cutoffs)
}
if len(fetcher.calls) != 2 || fetcher.calls[0] != "one" || fetcher.calls[1] != "two" || len(waits) != 1 || waits[0] != 4*time.Second {
t.Fatalf("順次確認と間隔: calls=%v waits=%v", fetcher.calls, waits)
}
if len(subscriptions.completions) != 2 || subscriptions.completions[0].limit != tver.PreviewMaxSeconds || !subscriptions.completions[0].at.Equal(now) {
t.Fatalf("確認完了: %+v", subscriptions.completions)
}
got := subscriptions.completions[0].episodes[0]
if got.ID != "ep1" || got.DurationSeconds != 180 || got.EndAt == nil || got.EndAt.Unix() != 1_800_000_000 {
t.Fatalf("エピソード変換: %+v", got)
}
}
func TestShortErrorLimitsUnicodeByCharacters(t *testing.T) {
message := shortError(errors.New(strings.Repeat("界", 260) + "\n\t" + strings.Repeat("語", 300)))
if got := len([]rune(message)); got != 512 {
t.Fatalf("エラー文字数=%d want=512", got)
}
if strings.ContainsAny(message, "\r\n\t") || strings.Contains(message, " ") {
t.Fatalf("空白を1行へ正規化していない: %q", message)
}
if !strings.HasSuffix(message, strings.Repeat("語", 251)) {
t.Fatalf("UTF-8文字の途中で切断または境界が不正: %q", message)
}
}
func TestSubscriptionSchedulerRecordsRedactedFailureWithoutCompleting(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "failed"}}}
fetcher := &fakeSeriesFetcher{errors: map[string]error{
"failed": errors.New("request failed\nplatform_token=secret-token Cookie: session-secret"),
}}
var logs bytes.Buffer
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{Now: func() time.Time { return now }}, log.New(&logs, "", 0))
if err != nil {
t.Fatal(err)
}
scheduler.checkDue(context.Background())
if len(subscriptions.completions) != 0 || len(subscriptions.failures) != 1 || !subscriptions.failures[0].at.Equal(now) {
t.Fatalf("失敗の記録: completions=%v failures=%v", subscriptions.completions, subscriptions.failures)
}
message := subscriptions.failures[0].message
if strings.Contains(message, "secret-token") || strings.Contains(message, "session-secret") || strings.Contains(logs.String(), "secret") || strings.Contains(message, "\n") {
t.Fatalf("秘密値または改行が残った: message=%q logs=%q", message, logs.String())
}
}
func TestSubscriptionSchedulerRunStartsImmediatelyThenUsesScanInterval(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
subscriptions := &fakeSubscriptionStore{}
fetcher := &fakeSeriesFetcher{}
requested := make(chan time.Duration, 2)
tick := make(chan time.Time, 1)
after := func(duration time.Duration) <-chan time.Time {
requested <- duration
return tick
}
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
ScanInterval: time.Hour, Now: func() time.Time { return now }, After: after,
}, nil)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { scheduler.Run(ctx); close(done) }()
select {
case duration := <-requested:
if duration != time.Hour {
t.Fatalf("走査間隔: %v", duration)
}
case <-time.After(time.Second):
t.Fatal("起動直後の確認が行われなかった")
}
subscriptions.mu.Lock()
initialCalls := len(subscriptions.cutoffs)
subscriptions.mu.Unlock()
if initialCalls != 1 {
t.Fatalf("起動直後の確認回数: %d", initialCalls)
}
tick <- now
select {
case <-requested:
case <-time.After(time.Second):
t.Fatal("1時間後の確認が行われなかった")
}
cancel()
select {
case <-done:
case <-time.After(time.Second):
t.Fatal("終了できなかった")
}
subscriptions.mu.Lock()
defer subscriptions.mu.Unlock()
if len(subscriptions.cutoffs) != 2 {
t.Fatalf("確認回数: %d", len(subscriptions.cutoffs))
}
}
func TestSubscriptionSchedulerCancellationInterruptsFetchAndSeriesWait(t *testing.T) {
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
t.Run("fetch", func(t *testing.T) {
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}}}
fetcher := &fakeSeriesFetcher{block: true, started: make(chan string, 1)}
scheduler, _ := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{Now: func() time.Time { return now }}, nil)
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { scheduler.checkDue(ctx); close(done) }()
select {
case <-fetcher.started:
case <-time.After(time.Second):
t.Fatal("取得が開始されなかった")
}
cancel()
select {
case <-done:
case <-time.After(time.Second):
t.Fatal("取得中断で終了しなかった")
}
if len(subscriptions.failures) != 0 || len(subscriptions.completions) != 0 {
t.Fatalf("中断した番組を記録した: failures=%v completions=%v", subscriptions.failures, subscriptions.completions)
}
})
t.Run("between series", func(t *testing.T) {
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}, {SeriesID: "two"}}}
fetcher := &fakeSeriesFetcher{episodes: map[string][]tver.Episode{}, errors: map[string]error{}}
waiting := make(chan struct{})
never := make(chan time.Time)
scheduler, _ := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
Now: func() time.Time { return now }, After: func(time.Duration) <-chan time.Time { close(waiting); return never },
}, nil)
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() { scheduler.checkDue(ctx); close(done) }()
<-waiting
cancel()
<-done
if len(fetcher.calls) != 1 || fetcher.calls[0] != "one" {
t.Fatalf("番組間の中断: %v", fetcher.calls)
}
})
}
internal/worker/testdata/fake-ffprobe.sh
Vendored Executable
+17
View File
@@ -0,0 +1,17 @@
#!/bin/sh
set -eu
case "${FAKE_FFPROBE_MODE:-valid}" in
invalid|bad-container)
echo '{"format":{"format_name":"matroska"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
bad-video)
echo '{"format":{"format_name":"mp4"},"streams":[{"codec_type":"video","codec_name":"vp9"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
bad-audio)
echo '{"format":{"format_name":"mp4"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"opus"}]}' ;;
non-json)
echo 'not JSON' ;;
failure)
echo 'probe failed' >&2
exit 1 ;;
*)
echo '{"format":{"format_name":"mov,mp4,m4a,3gp,3g2,mj2"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
esac
internal/worker/testdata/fake-yt-dlp.sh
Vendored Executable
+67
View File
@@ -0,0 +1,67 @@
#!/bin/sh
set -eu
[ -z "${FAKE_ARGS_FILE:-}" ] || printf '%s\n' "$@" > "$FAKE_ARGS_FILE"
output_template=
episode_url=
while [ "$#" -gt 0 ]; do
if [ "$1" = "-o" ]; then
output_template=$2
shift 2
continue
fi
case "$1" in
https://tver.jp/episodes/*) episode_url=$1 ;;
esac
shift
done
mode=${FAKE_YTDLP_MODE:-success}
if [ "$mode" = "mixed" ]; then
case "$episode_url" in
*/ep-fail) mode=unknown ;;
*) mode=success ;;
esac
fi
case "$mode" in
drm) echo 'ERROR: This video is DRM protected token=secret-value' >&2; exit 1 ;;
geo) echo 'ERROR: This video is not available from your location due to geo restriction' >&2; exit 1 ;;
network) echo 'ERROR: Unable to download JSON metadata: [Errno 61] Connection refused' >&2; exit 1 ;;
unavailable) echo 'ERROR: MEDIA_NOT_FOUND: episode does not exist' >&2; exit 1 ;;
extract) echo 'ERROR: Unable to extract player data' >&2; exit 1 ;;
unknown) echo 'ERROR: surprising failure' >&2; exit 1 ;;
hang|hang-drm)
[ -z "${FAKE_READY_FILE:-}" ] || : > "$FAKE_READY_FILE"
if [ "$mode" = "hang-drm" ]; then
trap 'echo "ERROR: DRM protected" >&2; [ -z "${FAKE_TERM_FILE:-}" ] || : > "$FAKE_TERM_FILE"; exit 143' TERM INT
else
trap '[ -z "${FAKE_TERM_FILE:-}" ] || : > "$FAKE_TERM_FILE"; exit 143' TERM INT
fi
while :; do sleep 1; done
;;
esac
output=$(printf '%s' "$output_template" | sed 's/%(id)s/fake/g; s/%(ext)s/mp4/g')
mkdir -p "$(dirname "$output")"
printf 'fake mp4' > "$output"
if [ "$mode" = "symlink" ]; then
rm "$output"
ln -s "$FAKE_OUTSIDE_FILE" "$output"
fi
if [ "$mode" = "burst" ]; then
i=1
while [ "$i" -le 20 ]; do
printf '{"event":"download","format_id":"hls-698-1","progress":{"status":"downloading","downloaded_bytes":%s,"total_bytes":100,"speed":10,"eta":2,"fragment_index":%s,"fragment_count":20}}\n' "$i" "$i"
i=$((i + 1))
done
sleep 0.12
else
echo '{"event":"download","format_id":"hls-698-1","progress":{"status":"downloading","downloaded_bytes":1024,"fragment_index":0,"fragment_count":50,"filename":"/private/tmp/tver-spike/epq432nee4.fhls-698-1.mp4","tmpfilename":"/private/tmp/tver-spike/epq432nee4.fhls-698-1.mp4.part","max_progress":null,"progress_idx":null,"elapsed":0,"total_bytes_estimate":29798400.0,"speed":4834.271158821972,"eta":null,"_eta_str":"Unknown","_speed_str":" 4.72KiB/s","_percent":0.003436426116838488,"_percent_str":" 0.0%","_total_bytes_str":" N/A","_total_bytes_estimate_str":" 28.42MiB","_downloaded_bytes_str":" 1.00KiB","_elapsed_str":"00:00:00","_default_template":" 0.0% of ~ 28.42MiB at 4.72KiB/s ETA Unknown (frag 0/50)"}}'
sleep 0.02
echo '{"event":"download","format_id":"hls-ts_AUDIO-0_2","progress":{"status":"finished","downloaded_bytes":50,"total_bytes":50,"speed":20,"eta":0}}'
sleep 0.02
echo '{"event":"postprocess","progress":{"status":"started","postprocessor":"Merger"}}' >&2
fi
printf '{"event":"after_move","filepath":"%s"}\n' "$output"