Compare commits
36
Commits
797830b61f
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
 kazu634
|
80bd0eedbb | ||
|
kazu634
|
cb6fc1df4c | ||
|
kazu634
|
fd63a401f7 | ||
|
kazu634
|
6586d7edd2 | ||
|
kazu634
|
511365f0d4 | ||
|
kazu634
|
8e0b03483b | ||
|
kazu634
|
e305791423 | ||
|
kazu634
|
67f8f81fcf | ||
|
kazu634
|
fb3e128757 | ||
|
kazu634
|
d4346c06fe | ||
|
kazu634
|
a2dd67b8b6 | ||
|
kazu634
|
3fb3227aa7 | ||
|
kazu634
|
9b856d63fa | ||
|
kazu634
|
3ddd2063b6 | ||
|
kazu634
|
f963dea2d1 | ||
|
kazu634
|
37d42851ff | ||
|
kazu634
|
80d9606750 | ||
|
kazu634
|
944086c8ae | ||
|
kazu634
|
82536c6398 | ||
|
kazu634
|
6f150495d5 | ||
|
kazu634
|
ebf1cfcbc2 | ||
|
kazu634
|
276cd7b5b8 | ||
|
kazu634
|
637ee5b81a | ||
|
kazu634
|
787eda79ae | ||
|
kazu634
|
2e82a690b1 | ||
|
kazu634
|
4a02526e76 | ||
|
kazu634
|
a71b915c81 | ||
|
kazu634
|
edc8149202 | ||
|
kazu634
|
e2d3f83db4 | ||
|
kazu634
|
f0d73a14e1 | ||
|
kazu634
|
ffe7451d04 | ||
|
kazu634
|
5125614b93 | ||
|
kazu634
|
54de47fbfe | ||
 kazu634andClaude Opus 5.5
|
0d74a5d340 | ||
|
kazu634
|
4ad0d621e1 | ||
|
kazu634andClaude Opus 5.5
|
d7e01878e4 |
@@ -0,0 +1,6 @@
|
|||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
PROJECT.md
|
||||||
|
AGENTS.md
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
DATABASE_URL=
|
||||||
|
VIDEO_HOST_DIR=/mnt/nas-tver/tver
|
||||||
|
VIDEO_DIR=/videos
|
||||||
|
WORKER_CONCURRENCY=1
|
||||||
@@ -4,6 +4,7 @@
|
|||||||
# 秘匿情報（homelab 規約: .env に分離し、コミットしない）
|
# 秘匿情報（homelab 規約: .env に分離し、コミットしない）
|
||||||
.env
|
.env
|
||||||
.env.*
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
|
||||||
# エージェントのローカル設定（端末ごとの権限・agmsg のフック）
|
# エージェントのローカル設定（端末ごとの権限・agmsg のフック）
|
||||||
.claude/settings.local.json
|
.claude/settings.local.json
|
||||||
|
|||||||
@@ -7,9 +7,9 @@
|
|||||||
## チーム構成
|
## チーム構成
|
||||||
|
|
||||||
| エージェント | 種別 | 役割 |
|
| エージェント | 種別 | 役割 |
|
||||||
|---|---|---|
|
| --------------- | ----------- | --------------------------------------------- |
|
||||||
| **`Leader`** | claude-code | 人間の窓口。計画・タスク分解・依頼・検収 |
|
| **`Leader`** | claude-code | 人間の窓口。計画・タスク分解・依頼・検収 |
|
||||||
| **`Developer`** | codex | 実作業。実装・テスト実行・修正 |
|
| **`Developer`** | codex | 実作業。実装・テスト作成・修正 |
|
||||||
| **`QA`** | codex | テストの品質保証。テスト内容の検証・必要に応じたテストの修正・テスト実施・テスト結果の整理 |
|
| **`QA`** | codex | テストの品質保証。テスト内容の検証・必要に応じたテストの修正・テスト実施・テスト結果の整理 |
|
||||||
|
|
||||||
参加は人間の指示を受けてから行う。名前は上の表のとおりにする（`whoami.sh` が出す候補名は他チームのものを含むので使わない）。
|
参加は人間の指示を受けてから行う。名前は上の表のとおりにする（`whoami.sh` が出す候補名は他チームのものを含むので使わない）。
|
||||||
|
|||||||
+16
@@ -0,0 +1,16 @@
|
|||||||
|
FROM golang:1.27.1-alpine AS build
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
COPY cmd ./cmd
|
||||||
|
COPY internal ./internal
|
||||||
|
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /web ./cmd/web
|
||||||
|
|
||||||
|
FROM alpine:3.22
|
||||||
|
RUN apk add --no-cache ca-certificates tzdata
|
||||||
|
WORKDIR /app
|
||||||
|
COPY --from=build /web /app/web
|
||||||
|
USER 65532:65532
|
||||||
|
EXPOSE 8080
|
||||||
|
ENTRYPOINT ["/app/web"]
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
FROM golang:1.27.1-alpine AS build
|
||||||
|
|
||||||
|
WORKDIR /src
|
||||||
|
COPY go.mod go.sum ./
|
||||||
|
RUN go mod download
|
||||||
|
COPY cmd ./cmd
|
||||||
|
COPY internal ./internal
|
||||||
|
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /worker ./cmd/worker
|
||||||
|
|
||||||
|
FROM python:3.13-slim
|
||||||
|
ARG YT_DLP_VERSION=2026.08.19
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ca-certificates ffmpeg tzdata \
|
||||||
|
&& pip install --no-cache-dir "yt-dlp[default]==${YT_DLP_VERSION}" \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
WORKDIR /app
|
||||||
|
COPY --from=build /worker /app/worker
|
||||||
|
ENV HOME=/tmp \
|
||||||
|
XDG_CACHE_HOME=/tmp/yt-dlp-cache
|
||||||
|
USER 65532:65532
|
||||||
|
ENTRYPOINT ["/app/worker"]
|
||||||
+182
@@ -0,0 +1,182 @@
|
|||||||
|
---
|
||||||
|
title: tver の現状
|
||||||
|
tags:
|
||||||
|
- homelab
|
||||||
|
- tver
|
||||||
|
status: 本番稼働中
|
||||||
|
---
|
||||||
|
|
||||||
|
# tver の現状
|
||||||
|
|
||||||
|
> [!info] このファイルの位置づけ
|
||||||
|
> tver プロジェクトの現状（決定事項・要件・構成・課題）を積み上げていく。運用規約は [[AGENTS]] を参照。
|
||||||
|
|
||||||
|
## 今の状態
|
||||||
|
|
||||||
|
- Go の HTTP サーバー、TVer API クライアント、ドラマ・バラエティの新着表示、確認画面つきの動画複数削除、テスト、web・worker コンテナの定義がある
|
||||||
|
- **本番稼働中。** dockageのweb・worker、NASのPostgreSQL 17、NFS動画保存を構成し、実動画の取得・再生・Range配信まで確認済み
|
||||||
|
- リポジトリ: `https://gitea.kazu634.com/kazu634/tver.git`
|
||||||
|
|
||||||
|
## 目的
|
||||||
|
|
||||||
|
TVer（`https://tver.jp/`）で公開中の動画情報を取得し、選んだ動画を yt-dlp でダウンロードし、アプリ内で再生する Web アプリを作る。
|
||||||
|
|
||||||
|
## 決定事項
|
||||||
|
|
||||||
|
| 項目 | 決定 |
|
||||||
|
|---|---|
|
||||||
|
| TVer 利用規約との関係 | 鑑賞を前提とするダウンロードのため、問題無しと判断（ユーザー判断） |
|
||||||
|
| 利用者・公開範囲 | 自分のみ・**LAN 内だけ**。EC2 nginx では公開しない |
|
||||||
|
| 保存先 | **NAS（ds923p）の `/volume1/video/tver`**。既存の`anime`・`drama`・`misc`・`personal`・`variety`と分け、admin相当の書き込み範囲を限定する。`/volume1/video`をdockageの`/mnt/nas-tver`へNFSマウントし、`.env`は`VIDEO_HOST_DIR=/mnt/nas-tver/tver`とする |
|
||||||
|
| 自動ダウンロード | **作る**（番組を登録し、新しい回を自動でダウンロードする） |
|
||||||
|
| 再生 | **アプリの中で再生する** |
|
||||||
|
| ダウンロード | yt-dlp を使う |
|
||||||
|
| デプロイ | 読み取り専用のGiteaユーザー`tver-deploy`でdockage VMの`/opt/stacks/tver`へcloneし、`docker compose up -d --build`で起動する。Dockgeにも同じスタックが表示される。更新は`git pull --ff-only`後に再ビルドする |
|
||||||
|
| 配置用のGit認証 | `tver-deploy`を`tver`の読み取り共同作業者とし、repository読み取りトークンをdockageの権限`600`の専用credentialファイルに保存する。トークンの実値はGitや文書へ残さない |
|
||||||
|
| 開発言語 | **Go**（`web`・`worker` とも）。yt-dlp は外部コマンドとして呼び、進捗・結果は yt-dlp の出力から読み取る |
|
||||||
|
| DB | **NAS上のtver専用Postgres 17コンテナ**（Giteaと同じ方式）。ユーザー・DB名は`tver`、公開ポートは`15434`。dockageからの到達を確認済み |
|
||||||
|
| DB ドライバ | `github.com/jackc/pgx/v5` の `pgxpool` |
|
||||||
|
| DB テスト | `github.com/fergusstrange/embedded-postgres` v1.34.0 の Postgres 17 を起動する。Apple Silicon では Rosetta 2 が必要 |
|
||||||
|
| 定期実行 | **`worker` 内のスケジューラ** |
|
||||||
|
| NAS のマウント | **NFS**。dockageの`/etc/fstab`へ`_netdev,nofail`付きで登録し、DSMでは接続元`192.168.10.155`だけを許可する |
|
||||||
|
| NFS 権限 | 読み書き、Squash「すべてのユーザーをadminに割り当てる」、`sys`、非同期オン。DSMのadminアカウントが無効でもUID `65532`から書き込み確認済み |
|
||||||
|
| DBの通信制限 | DSMファイアウォールでTCP `15434`を`192.168.10.155`からだけ許可する |
|
||||||
|
| DBバックアップ | NASのDSMタスクスケジューラからrootで毎日実行する。`tver-postgres`内の`pg_dump`をgzip圧縮して`/volume1/backups/tver/tver_<YYYYmmdd_HHMMSS>.sql.gz`へ保存し、14日より古い世代を削除する。同名の完成ファイルは上書きせず失敗させる。初回は一時DBへの復元と主要4表の行数一致まで確認する |
|
||||||
|
| 保持期間 | **削除は手動のみ**。自動削除はしない |
|
||||||
|
| 待ち行列 | **Postgres のテーブルだけで実装する**。専用のキュー基盤は持たない |
|
||||||
|
| 定期確認の間隔 | 登録番組の新しい回を **24 時間ごと**に確認する |
|
||||||
|
| 定期確認の走査 | worker のスケジューラが起動直後と1時間ごとに、最終確認から24時間以上経過した有効な番組を探す。番組は1件ずつ確認し、間に数秒空ける |
|
||||||
|
| 定期確認の予告 | **180秒以下**の回は自動登録せず、既知としてだけ記録する。新着画面の予告判定と同じしきい値を使う |
|
||||||
|
| 定期確認の失敗 | `last_checked_at` を更新せず、失敗日時と秘匿値を伏せた短いメッセージを保存する。次の1時間走査で再試行し、成功時にエラーを消す |
|
||||||
|
| 登録時の既存回 | **登録の都度**「公開中の回もダウンロードする」かを選ぶ。選ばない場合も公開中の回を既知として記録する |
|
||||||
|
| 番組の一時停止 | 登録は残し、自動ダウンロードだけ止める。再開後は停止中に公開された未確認の回も対象にする |
|
||||||
|
| 動画のマウント先 | web と worker は `VIDEO_DIR`（既定 `/videos`）に同じ NAS 共有をマウントする。`download_jobs.file_path` は `VIDEO_DIR` からの相対パスを保存する |
|
||||||
|
| ファイル名 | `<番組名>/<番組名> <回タイトル> [<エピソードID>].mp4`。禁止文字と制御文字を `_` に置換し、空の要素は `名称未設定`とする。番組名はUTF-8で90バイト、回タイトルは140バイト、サニタイズ後のエピソードIDは64バイトまでとし、ファイル名全体を255バイト以下に収める。64バイトを超えるIDはworkerで`invalid_output`にする。実際のTVer IDは約10文字で、yt-dlpのextractorも英数字として扱うため64バイトを上限とした。既存ファイルは改名せず、新しい保存から適用する |
|
||||||
|
| 動画ファイルの境界 | webは`VIDEO_DIR`を`os.OpenRoot`で開き、DBの相対`file_path`を`Root.Open`・`Root.Remove`へ渡す。`VIDEO_DIR`外へのパストラバーサルと外向きシンボリックリンクをOSのディレクトリ境界で拒否する |
|
||||||
|
| 同名ファイル | 移動先に同名ファイルがあれば上書きする。ファイル名にエピソード ID が入り `episode_id` は一意なので、同名は同じ回の前回処理の残りに限られる。再実行で同じ結果になることを優先する |
|
||||||
|
| ライブラリ一覧 | DBの完了済みジョブから作り、NASのフォルダ走査やファイル存在確認は行わない。番組一覧、番組ごとの回一覧、再生の3段構成にする |
|
||||||
|
| 回の自然順 | 回タイトル中の半角・全角数字の並びを数値として比較し、同順位は完了日時の古い順、さらにジョブIDの小さい順にする |
|
||||||
|
| 動画の削除 | アプリからだけ行い、削除前にサーバーが描画する確認画面を必ず挟む。ライブラリでは番組単位、番組ページでは選択した回または番組全体を複数削除できる。確認画面で確定したジョブ ID だけを実行するため、確認後に増えた回は削除しない。各回はファイルを先に削除してからジョブ記録を削除し、親の番組フォルダが完全に空なら削除する。ファイルが既に無い場合も記録は削除し、1件の失敗で残りを止めない。登録番組の `known_episodes` は残す |
|
||||||
|
| 同時ダウンロード数 | 既定は **1**。`WORKER_CONCURRENCY` で **2まで**増やせる |
|
||||||
|
| ダウンロードの再試行 | 自動では再試行しない。失敗を `failed` で残し、画面から手動で再試行する |
|
||||||
|
| 実行中の取り消し | web で `running` を `canceled` にし、worker が検知して yt-dlp のプロセスグループを停止する |
|
||||||
|
| yt-dlp の作業場所 | `VIDEO_DIR/.incoming/<job id>/` に `%(id)s.%(ext)s` で保存し、検査後に Go が `media.RelativeVideoPath` の場所へ同一ファイルシステム内で移動する。終了状態にかかわらず作業場所を片付ける |
|
||||||
|
|
||||||
|
## 要件
|
||||||
|
|
||||||
|
### 機能要件
|
||||||
|
|
||||||
|
| # | 要件 | 補足 |
|
||||||
|
|---|---|---|
|
||||||
|
| F1 | ドラマ・バラエティの新着一覧を表示する | 番組名・回タイトル・放送局・長さ・配信終了日時・サムネイル |
|
||||||
|
| F2 | 一覧を絞り込む | 予告（「60秒PR」等）を除外できる。ダウンロード済み・待ち行列中を区別して表示する |
|
||||||
|
| F3 | 選んだ動画をダウンロードの待ち行列に入れる | 複数選択できる |
|
||||||
|
| F4 | ダウンロードの進捗・結果を表示する | 失敗理由（DRM・地域制限・TVer 側の変更による抽出失敗）を区別して表示する |
|
||||||
|
| F5 | 同じ回を二重にダウンロードしない | 手動・自動の両方で共通 |
|
||||||
|
| F6 | 番組（シリーズ）を登録し、新しい回を自動でダウンロードする | 登録・解除・一覧。定期的に新しい回を確認する。番組URL（`tver.jp/series/…`）から登録でき、登録前に確認画面を表示する |
|
||||||
|
| F7 | ダウンロード済みの動画をアプリ内で再生する | シーク（HTTP Range）に対応する |
|
||||||
|
| F8 | ダウンロード済みの動画を一覧・削除する | NAS の容量管理のため |
|
||||||
|
| F9 | 保存ファイルの名前と置き場所を規則で決める | 例: `<番組名>/<番組名> <回タイトル> [<エピソードID>].mp4` |
|
||||||
|
|
||||||
|
### 非機能要件
|
||||||
|
|
||||||
|
- **yt-dlp を素早く更新できること。** TVer 側の変更で抽出が壊れやすいので、worker のイメージを再ビルドするだけで更新できるようにする
|
||||||
|
- **TVer への負荷を抑える。** 一覧取得はキャッシュし、同時ダウンロード数は 1〜2 に制限する
|
||||||
|
- **ブラウザで再生できる形式で保存する。** 映像 H.264 + 音声を MP4 に結合する（ffmpeg が必要）
|
||||||
|
- 認証は持たない（LAN 内限定のため）
|
||||||
|
- homelab の規約に従う（[[homelab/docs/conventions|運用規約]]）: Compose v2、秘匿情報は `.env`、`TZ=Asia/Tokyo`
|
||||||
|
|
||||||
|
## 構成（案）
|
||||||
|
|
||||||
|
| コンテナ | 役割 |
|
||||||
|
| -------- | ----------------------------------------------------- |
|
||||||
|
| `web` | UI・API。新着一覧の取得、待ち行列への登録、番組URLを含む番組登録、動画の配信（再生） |
|
||||||
|
| `worker` | Go + yt-dlp + ffmpeg。待ち行列からダウンロードし、登録番組の新しい回を定期的に確認する |
|
||||||
|
|
||||||
|
- 現在の `web` は `/healthz`、新着（`/`）、待ち行列（`/queue`）、登録番組（`/subscriptions`）、番組単位のライブラリ（`/library`）、番組ごとの回一覧（`/library/series/{series_id}`）、再生ページ（`/library/{id}`）、Range 対応の MP4 配信（`/videos/{id}`）を提供する
|
||||||
|
- ライブラリはDBの完了済みジョブだけを基に、番組一覧から番組ページ、再生ページへ進む。番組ページは完了が新しい順と、回タイトルの半角・全角数字を数値比較する話の順を切り替えられる。再生ページは話の順で前後の回へ移動できる
|
||||||
|
- 動画配信はクライアントからジョブ ID だけを受け取り、DB の相対 `file_path` を `os.Root` の `Open`・`Remove` で扱う。絶対パス・`..`・`VIDEO_DIR` 外を指すシンボリックリンクをディレクトリ境界で拒否する。削除はライブラリまたは番組ページの選択、もしくは再生ページの1件削除からサーバー描画の確認画面へ進み、解決済みのジョブ ID を実行画面へ引き継ぐ。実行時は1リクエストにつき動画ルートを1回だけ開き、各回を独立して処理して削除・失敗・対象外の件数を結果画面へ表示する。動画削除後に実ディレクトリの親が空なら削除する。削除後は新着から手動で再登録できるが、登録番組の既知記録は残るため自動再登録しない
|
||||||
|
- 新着の「番組を登録」から、公開中の回をダウンロードするか選ぶ。web はサーバー側の新着キャッシュから番組 ID と名前を引き、TVer の season ごとの公開中エピソードを取得する。取得失敗時は登録しない
|
||||||
|
- `/subscriptions`では`https://tver.jp/series/<ID>`形式の番組URLを受け付け、検証した番組IDからTVer APIの接続先を組み立てる。登録前に番組名・番組ID・公開中の回数を確認し、登録時にもサーバー側で番組名と公開中の回を取り直す。クライアントから番組名は受け取らない
|
||||||
|
- 登録と公開中の回の既知記録・任意の待ち行列投入は1トランザクションで行い、登録時刻を最終確認日時に設定する。`known_episodes` は番組解除時に連動して削除する。一時停止中は投入を拒否する
|
||||||
|
- worker のスケジューラは起動直後と1時間ごとに、`enabled` かつ最終確認から24時間以上の番組を抽出する。TVer から公開中の回を順次取得し、未確認の180秒超の回だけを `source=subscription` で投入する。180秒以下も既知にし、成功時の最終確認更新・エラー消去まで1トランザクションで行う
|
||||||
|
- TVer の番組取得に失敗した場合は最終確認日時を変えず、失敗日時と伏字済みの短いメッセージを `subscriptions` に記録して画面に表示する。番組間には3秒空け、SIGTERM では取得中または待機中の確認を中断する
|
||||||
|
- 新着の登録ボタンはエピソード ID とジャンルだけを送る。web が有効な新着キャッシュから番組情報を引き、`episode_id` の一意制約で重複登録を防ぐ。新着一覧はジョブ状態を一括取得し、失敗・取り消しには再試行を表示する
|
||||||
|
- 待ち行列は実行中・待機中・失敗・取り消し・完了を状態別に直近50件ずつ表示する。段階別の進捗と失敗理由を表示し、待機中の取り消しと失敗・取り消しの再試行ができる。実行中または待機中のジョブがある間だけ5秒ごとに一覧を更新する
|
||||||
|
- 実行中のジョブにも取り消しボタンを表示する。worker は DB の `canceled` を定期確認し、yt-dlp をプロセスグループごと停止する
|
||||||
|
- 新着はジャンルごとにメモリへ 10 分間キャッシュする。キャッシュ中は TVer セッションの再発行と一覧の再取得を行わない。TVer の取得に失敗した場合、一覧の場所にエラーを表示して詳細をログへ記録する
|
||||||
|
- 「予告を隠す」は既定でオンとし、180 秒以下の件を非表示にする。長さは秒から分へ切り上げて表示し、配信終了日時は Asia/Tokyo で表示する。ジャンルと予告表示の状態は URL に残す
|
||||||
|
- 画面は `html/template` と htmx 2.0.11 を使い、htmx は Go の `embed` から `/static/` でローカル配信する
|
||||||
|
- `compose.yaml` は web と worker を定義する。web は dockage VM の LAN アドレス `192.168.10.155:8080` で公開し、待ち受けは `WEB_ADDR` で変更できる。両方へ `DATABASE_URL` と同じ動画ボリュームを渡し、worker の並行数は `WORKER_CONCURRENCY`（既定1、最大2）で指定する
|
||||||
|
- NAS用の`deploy/nas/tver-db/compose.yaml`は`postgres:17`を`15434:5432`で公開し、`/volume1/docker/tver-db/pgdata`へ保存する。パスワードは同ディレクトリのGit管理外`.env`から注入する
|
||||||
|
- NAS用の`backup-db.sh`はコンテナ内の`pg_dump`を検査後にgzip圧縮し、同一ディレクトリ内のハードリンクで完成ファイルを上書きせず公開する。`restore-test.sh`は最新ダンプを一時DBへ復元して`download_jobs`・`subscriptions`・`known_episodes`・`schema_migrations`の行数を本番DBと比較する。一時DBはEXIT trapで必ず削除し、復元SQLの通常出力はログへ記録しない
|
||||||
|
- 本番配置と更新の手順は[[docs/deploy|tver 本番配置手順]]にまとめる。dockageでは`/volume1/video`を`/mnt/nas-tver`へNFSマウントし、その配下の`tver`だけをコンテナ内の`/videos`へ渡す
|
||||||
|
- Giteaは匿名閲覧とSSHを無効にしているため、dockageのclone・pullは読み取り専用`tver-deploy`のrepository読み取りトークンを使う。Gitのcredential helperはGiteaのHTTPS URLに限定する
|
||||||
|
- DB は NAS 上に tver 専用 Postgres 17 コンテナを置く。`internal/store` の埋め込み SQL を `schema_migrations` で管理し、`pg_advisory_lock` で複数プロセスからの同時適用を直列化する。web は起動時に `pgxpool` で接続してマイグレーションを適用する
|
||||||
|
- `download_jobs` は `episode_id` を一意にして二重登録を防ぎ、ジョブの状態・段階別進捗・結果を持つ。`subscriptions` はシリーズ単位の登録と最終確認日時を持つ。worker からの操作と画面用の一覧取得を `internal/store` にまとめる
|
||||||
|
- worker は `yt-dlp[default]` と ffmpeg を外部コマンドとして使い、映像・音声・結合・移動の段階別進捗を1秒間隔に間引いて保存する。独立した heartbeat を5秒ごとに更新し、正確な全体パーセントは保存しない。`after_move:filepath` で実パスを取り、ffprobe で H.264 と AAC の MP4 を確認して成功とする。失敗は取り消し、DRM、地域制限、ネットワーク、配信不可、抽出失敗、出力不正、その他の順に扱い、自動再試行しない
|
||||||
|
- worker は起動時にマイグレーションと停止中ジョブの `RequeueStale` を行い、登録番組の確認も直ちに1回行う。SIGTERM では yt-dlp と確認処理を止め、担当中ジョブを `queued` に戻してから終了する。`Dockerfile.worker` は yt-dlp 2026.08.19 を ARG で固定し、ffmpeg・ffprobe とともに非 root で実行する
|
||||||
|
- 保存先の NAS 共有フォルダは dockage VM に NFS でマウントし、`worker`（書き込み）と `web`（読み取り・削除）に同じ `VIDEO_DIR` でバインドマウントする
|
||||||
|
|
||||||
|
## 外部とのつながり
|
||||||
|
|
||||||
|
| 相手 | 用途 | 備考 |
|
||||||
|
|---|---|---|
|
||||||
|
| TVer `platform-api.tver.jp` | 新着一覧（`/service/api/v1/callNewerDetail/{drama\|variety}`）・番組名（`/service/api/v2/callSeries/{series_id}`）・番組のエピソード一覧 | **非公開 API**。セッションは `/v2/api/platform_users/browser/create` で発行する（yt-dlp と同じ方式） |
|
||||||
|
| TVer `service-api.tver.jp` | 番組の season 一覧（`/api/v1/callSeriesSeasons/{series_id}`） | **非公開 API**。[yt-dlp の TVer extractor](https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/tver.py) と同じ取得方式 |
|
||||||
|
| yt-dlp | ダウンロード | TVer extractor を使う |
|
||||||
|
| Gitea `gitea.kazu634.com` | 配置元 | 匿名cloneとSSHは使えない。`tver-deploy`のHTTPSトークンでclone・pullする |
|
||||||
|
| NAS ds923p（NFS） | 動画の保存先 | `/volume1/video`をdockageの`/mnt/nas-tver`へNFSv3でマウントし、アプリは`tver`サブフォルダを使う。DSMのNFS権限は`192.168.10.155`のみ |
|
||||||
|
| NAS ds923p（Postgres 17） | DB | tver専用コンテナを`15434:5432`で公開し、DSMファイアウォールで`192.168.10.155`のみ許可する。dockageから接続確認済み |
|
||||||
|
|
||||||
|
## 確認済みの事実
|
||||||
|
|
||||||
|
- 新着 API は `drama` / `variety` それぞれ 100 件を返す。予告動画も含まれる。各件に `id`・`seriesID`・`seriesTitle`・`title`・`broadcasterName`・`duration`・`endAt`・`thumbnailPath` がある [実機] 2026-10-03 確認
|
||||||
|
- 番組 `sr0lhdrtdb` の `callSeriesSeasons` は `result.contents` に `type=season` の1件（`ss3i6njoq3`）を返し、`callSeasonEpisodes` は `type=episode` の5件を返した。エピソードには新着と同じ `id`・`seriesID`・`seriesTitle`・`title`・`broadcasterName`・`duration`・`endAt`・`thumbnailPath` が含まれる。他の番組での season 数や一覧件数は未確認 [実機] 2026-10-03 確認
|
||||||
|
- dockage VM から TVer API に到達でき（HTTP 200）、送信元は日本と判定される [実機] 2026-10-03 確認
|
||||||
|
- yt-dlp 2026.08.19 で TVer のエピソードを抽出できる（`--simulate`）。映像（H.264 / `avc1`）と音声が別ストリームで、結合に ffmpeg が要る。1080p・約 50 分で約 1.2GiB [実機] 2026-10-03 確認
|
||||||
|
- 5 分の `epq432nee4` を yt-dlp 2026.08.19 で取得した。`--newline` と `--progress-template` の 1 行 1 JSON は Go の外部プロセスから逐次読める。ただし `uvx yt-dlp` のみで `pycryptodomex` が無いと AES-128 HLS を ffmpeg に委譲し、途中のダウンロード進捗が出なかった。`yt-dlp[default]` に含まれる `pycryptodomex` を加えた試行では途中の進捗が出た [実機] 2026-10-03 確認
|
||||||
|
- 別ストリームの映像と音声は進捗が個別に始まり、`format_id` で区別できる。結合処理は `postprocess:` の出力で検知でき、`--print after_move:filepath` で実ファイルのパスを取得した。完成した MP4 は 24,613,787 バイトで、ffprobe では H.264 640×360 と AAC LC だった。初回取得は約 10 秒だった [実機] 2026-10-03 確認
|
||||||
|
- SIGTERM で止めると `.part` と `.ytdl` が残り、再実行で 437,066 バイトから 438,090 バイトへ続きから取得できた [実機] 2026-10-03 確認
|
||||||
|
- 到達不能のプロキシ経由の通信失敗と存在しないエピソードは、ともに yt-dlp の終了コード 1 だった。stderr の文言で分類が必要。DRM・地域制限の失敗文言と終了コードは実測していない（yt-dlp の extractor ソースのみ確認） [実機] 2026-10-03 確認
|
||||||
|
- yt-dlp の TVer extractor は日本のみ（`_GEO_COUNTRIES = ['JP']`）。DRM 付きの作品は取得できない [公式] https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/extractor/tver.py (取得日: 2026-10-03)
|
||||||
|
- DSMのNFS権限を接続元`192.168.10.155`、読み書き、全ユーザーをadminへ割り当て、`sys`、非同期オンにすると、dockageからNFSv3で`/volume1/video`をマウントできた。設定直後の1回目は`access denied`、直後の2回目は成功したため、権限の反映待ちと考えられる [実機] 2026-10-04 確認
|
||||||
|
- DSMのadminアカウントが無効でも、UID `65532`のコンテナから`tver`サブフォルダの作成・書き込み・読み込み・rename・削除に成功した。NAS上の所有者はUID `1024`・GID `100`。`/mnt/nas-tver/tver`は作成済み [実機] 2026-10-04 確認
|
||||||
|
- NFSマウントのファイル名上限は`namlen=255` [実機] 2026-10-04 確認
|
||||||
|
- Giteaは`REQUIRE_SIGNIN_VIEW = true`・`DISABLE_SSH = true`で、匿名HTTPSの`info/refs`はHTTP `401`だった。読み取り専用ユーザー`tver-deploy`とrepository読み取りトークンで`/opt/stacks/tver`へのcloneに成功した [実機] 2026-10-04 確認
|
||||||
|
- dockageで`docker compose up -d --build`を実行してweb・workerを起動でき、Dockgeにも`tver`スタックとして表示された。DSMファイアウォール設定後、dockageからNASのTCP `15434`へ到達できた [実機] 2026-10-04 確認
|
||||||
|
- 本番の`/healthz`はHTTP `200`、新着バラエティは98件だった。4分の本編`ep5hcgopp2`は登録から20秒以内に完了し、92,340,468バイトの1080p H.264/AAC MP4をNASへ保存した。所有者はUID `1024`・GID `100`、`.incoming`は空で、`/library`とHTTP Range `206`も確認した [実機] 2026-10-04 確認
|
||||||
|
- NASは番組フォルダ内に所有者UID `0`の`@eaDir`を作る。ライブラリ一覧はDBから作るため、アプリへの影響はない [実機] 2026-10-04 確認
|
||||||
|
- `callNewerDetail/drama`は毎回100件を返し、応答に続きや総件数を示す項目はなかった。`result.contents`は`type=newer`で、`content`は`id`・`version`・`title`のみ。`page=2`・`offset=100`・`limit=200`を付けても先頭・末尾IDが同じ100件となり、パラメータは無視された。100件を超えて取得する手段は見つからなかったが、非公開APIのため不可能とは断定しない。新着一覧は直近100件までとなる一方、登録番組の自動確認は番組ごとの`callSeriesSeasons`→`callSeasonEpisodes`を使うため、新着から外れた回も取りこぼさない [実機] 2026-10-04 確認
|
||||||
|
- dockage上の使い捨て`postgres:17`コンテナへマイグレーション001〜003を適用してテスト用の行を投入し、`backup-db.sh`と`restore-test.sh`を実行した。gzip圧縮・権限600のダンプ作成、同じ`STAMP`での2回目が`ln`の失敗で非0終了して既存ダンプを保持すること、主要4表の行数一致、一時DBの削除、ログに`psql`の標準出力が残らないことを確認した [実機] 2026-10-04 確認
|
||||||
|
- NASの`/volume1/docker/tver-db/`へスクリプトを配置し、DSMタスクスケジューラのrootタスクから実行して、`/volume1/backups/tver/`へ`tver_20261004_182838.sql.gz`（2,286バイト）と`tver_20261004_183256.sql.gz`（2,289バイト）を作成した。ダンプはroot所有・権限600で、バックアップディレクトリはroot所有・`drwxr-xr-x`だった。`restore-test.sh`を`sudo`で実行し、`download_jobs` 1件・`subscriptions` 1件・`known_episodes` 1件・`schema_migrations` 3件の一致、ログへの結果記録、一時DBの削除を確認した。一般ユーザーではroot所有のログへ書き込めず失敗する [実機] 2026-10-04 確認
|
||||||
|
- DSMのACLで全員書き込み可だった`/volume1/docker`配下について、`/volume1/docker/tver-db`を`chmod 755`、`backup-db.sh`と`restore-test.sh`を`chmod 700`にしてACLが外れたことを確認した。変更後も本番アプリの`/queue`と`/subscriptions`はHTTP `200`でDBを読み取れた。PostgreSQLコンテナ再起動後の動作は未確認 [実機] 2026-10-04 確認
|
||||||
|
- platform-apiの`v2/callSeries/{series_id}`は`result.content.content.title`に番組名を返し、存在しないIDにはHTTP `404`を返した。`sr8a864g0w`では番組名がエピソード側の`seriesTitle`と一致した。`v1/callEpisode/{episode_id}`は`seriesID`・`seriesTitle`を返した（今回は使わない） [実機] 2026-10-07 確認
|
||||||
|
|
||||||
|
## 課題
|
||||||
|
|
||||||
|
- 登録済みのDSM日次バックアップタスクが翌日以降に自動実行され、新しいダンプを作ることを確認する [要確認]
|
||||||
|
- DRM・地域制限の実例探しはTVerへの大量アクセスになるため行わず、運用中に観測する。workerは失敗時の`error_message`をDBへ保存し、既知の文言に一致しない失敗は`unknown`にする。実例が出た時点で分類を見直す
|
||||||
|
|
||||||
|
## 経緯
|
||||||
|
|
||||||
|
- ステップ13で、ライブラリの番組単位削除、番組ページの回選択・番組全体削除、再生ページの1件削除を、すべてサーバー描画の確認画面を経由する方式に統一した。確認時に解決した最大1,000件のジョブ ID だけを実行し、途中の失敗後も残りを処理して結果を表示する
|
||||||
|
- ステップ12でライブラリを番組一覧、番組ページ、再生ページの3段構成にした。番組単位の集計と登録状態はDBから取得し、番組ページの話の順と再生ページの前後移動には同じ自然順を使う。動画削除後は空になった番組フォルダだけを`os.Root`経由で削除する
|
||||||
|
|
||||||
|
## 確かめ方
|
||||||
|
|
||||||
|
- Codex では `go test -count=1 ./...` を環境変数の前置なしで実行する。`go test` を許可する rules に一致させ、Postgres の共有メモリ作成に必要な権限で実行するため。初回のモジュール取得にはネットワーク接続が必要
|
||||||
|
- web を起動するには `.env.example` を参考に、Git 管理外の `.env` に実際の `DATABASE_URL` と `VIDEO_HOST_DIR` を設定する。`VIDEO_DIR` の既定は `/videos`。DB 接続がある環境では起動時にマイグレーションが適用される
|
||||||
|
- DB テストは `embedded-postgres` v1.34.0 が Postgres 17 を一度起動し、テストごとに独立した DB を作る。バイナリキャッシュ、実行ファイル、データは `/private/tmp` 配下に置く。Apple Silicon での実行には Rosetta 2 の導入が必要
|
||||||
|
- TVer 応答のテストは `internal/tver/testdata/newer_drama.json`（2026-10-03 の実応答を 3 件に縮小）と `httptest` を使い、テストから本物の TVer にはアクセスしない
|
||||||
|
- 番組一覧の応答は `internal/tver/testdata/series_seasons.json` と `season_episodes.json`（2026-10-03 の実応答を各1件に縮小）に記録した。テストは `httptest` を使い、実 API にアクセスしない
|
||||||
|
- 番組情報の応答は`internal/tver/testdata/series.json`（2026-10-07の実応答を番組情報だけに縮小）に記録した。番組URLは純関数でscheme・host・path・IDを検証し、TVerクライアントのテストは`httptest`だけへ接続する
|
||||||
|
- 新着画面のテストは偽物の TVer クライアント・store と差し替え可能な時計を使い、表示項目、絞り込みの境界、10 分キャッシュ、取得失敗、JST 表示、登録・状態表示・待ち行列の操作を確認する
|
||||||
|
- ライブラリのテストは番組単位の集計、番組別・ID指定の完了済み回取得、3画面、半角・全角数字の自然順、前後移動を確認する。一時ディレクトリの小さな動画を使い、Rangeの206・範囲外の416、`os.Root`による相対パスとシンボリックリンクの拒否、404、サーバー確認画面の入力検証・重複除去・HTMLエスケープ、複数削除の部分失敗・対象外・確認後の新着保持、ファイル削除後の記録削除、空の番組フォルダだけを削除することも確認する。共有のファイル名規則は`internal/media`で90・140・255バイトの境界とUTF-8の完全性をテストする
|
||||||
|
- worker のテストは `internal/worker/testdata` の偽 yt-dlp・ffprobe を使い、本物の TVer へ接続せずに正常完了、失敗分類、取消、終了時の待機復帰、進捗の間引き、heartbeat、出力検査、移動先、作業場所の片付けを確認する
|
||||||
|
- 登録番組の定期確認は偽の TVer クライアントと差し替え可能な時計・タイマーを使い、24時間の境界、順次確認、1時間走査、失敗記録、予告の既知化、終了時の中断を確認する。store の抽出・既知化・投入・日時更新・エラー消去は embedded Postgres 17 で確認する
|
||||||
|
- NAS用Composeは`deploy/nas/tver-db/compose.yaml`をYAMLパーサーで読み、`services.postgres`、環境変数、ポート、保存先、healthcheckを確認する。Dockerが使える環境では`docker compose --env-file deploy/nas/tver-db/.env.example -f deploy/nas/tver-db/compose.yaml config`でも展開結果を確認する
|
||||||
|
- DBバックアップスクリプトは`deploy/nas/tver-db/scripts_test.go`からPATH上の偽dockerを使って実行し、正常な圧縮、同名・同時実行時の上書き防止、空ダンプと`pg_dump`失敗、14日の世代境界、復元失敗時の一時DB削除、主要表の行数不一致を確認する。NAS実機では[[docs/deploy#5. PostgreSQLをバックアップする|配置手順]]に従って手動バックアップと復元確認を行う
|
||||||
|
- 本番の確認手順は[[docs/deploy|tver 本番配置手順]]に従い、NFS上の`/mnt/nas-tver/tver`をUID `65532`で読み書き・rename・削除できること、DBがhealthyであること、webの`/healthz`、実動画の保存・再生・シークを確認する
|
||||||
|
- `env GOPATH=/private/tmp/tver-gopath GOCACHE=/private/tmp/tver-go-cache GOMODCACHE=/private/tmp/tver-go-modcache GOPROXY=off go vet ./...` で静的解析する
|
||||||
|
- `DATABASE_URL`、`VIDEO_DIR`、`WEB_ADDR=127.0.0.1:18080` を設定して `go run ./cmd/web` で web を起動できる。worker は同じ `DATABASE_URL` と `VIDEO_DIR` を設定して `go run ./cmd/worker` で起動する。worker の実行は TVer への実アクセスを伴うため手動確認は少数回に留める
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
_ "time/tzdata"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/web"
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
pool, err := openDatabase(context.Background(), os.Getenv("DATABASE_URL"), pgxpool.New, store.Migrate)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
defer pool.Close()
|
||||||
|
address := os.Getenv("WEB_ADDR")
|
||||||
|
if address == "" {
|
||||||
|
address = ":8080"
|
||||||
|
}
|
||||||
|
client, err := tver.NewClient("", nil)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
videoDir := os.Getenv("VIDEO_DIR")
|
||||||
|
if videoDir == "" {
|
||||||
|
videoDir = "/videos"
|
||||||
|
}
|
||||||
|
server := &http.Server{
|
||||||
|
Addr: address,
|
||||||
|
Handler: web.NewHandler(client, store.New(pool), videoDir, time.Now, log.Default()),
|
||||||
|
ReadHeaderTimeout: 5 * time.Second,
|
||||||
|
}
|
||||||
|
log.Printf("web listening on %s", address)
|
||||||
|
log.Fatal(server.ListenAndServe())
|
||||||
|
}
|
||||||
|
|
||||||
|
func openDatabase(ctx context.Context, databaseURL string,
|
||||||
|
connect func(context.Context, string) (*pgxpool.Pool, error),
|
||||||
|
migrate func(context.Context, *pgxpool.Pool) error) (*pgxpool.Pool, error) {
|
||||||
|
if strings.TrimSpace(databaseURL) == "" {
|
||||||
|
return nil, errors.New("DATABASE_URL が設定されていません")
|
||||||
|
}
|
||||||
|
pool, err := connect(ctx, databaseURL)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New("DATABASE_URL から接続プールを作成できません")
|
||||||
|
}
|
||||||
|
if err := migrate(ctx, pool); err != nil {
|
||||||
|
pool.Close()
|
||||||
|
return nil, fmt.Errorf("DB マイグレーションに失敗: %w", err)
|
||||||
|
}
|
||||||
|
return pool, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestOpenDatabaseRequiresURLAndRunsMigration(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
connectCalls, migrateCalls := 0, 0
|
||||||
|
connect := func(_ context.Context, url string) (*pgxpool.Pool, error) {
|
||||||
|
connectCalls++
|
||||||
|
if url != "postgres://test:secret@localhost/test" {
|
||||||
|
t.Errorf("URL = %q", url)
|
||||||
|
}
|
||||||
|
return pgxpool.New(ctx, url)
|
||||||
|
}
|
||||||
|
migrate := func(_ context.Context, _ *pgxpool.Pool) error {
|
||||||
|
migrateCalls++
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := openDatabase(ctx, " ", connect, migrate); err == nil || !strings.Contains(err.Error(), "DATABASE_URL") {
|
||||||
|
t.Fatalf("未設定 URL のエラー: %v", err)
|
||||||
|
}
|
||||||
|
if connectCalls != 0 || migrateCalls != 0 {
|
||||||
|
t.Fatalf("未設定でも接続・移行された: %d %d", connectCalls, migrateCalls)
|
||||||
|
}
|
||||||
|
pool, err := openDatabase(ctx, "postgres://test:secret@localhost/test", connect, migrate)
|
||||||
|
if err != nil || pool == nil {
|
||||||
|
t.Fatalf("DB 初期化: %v", err)
|
||||||
|
}
|
||||||
|
pool.Close()
|
||||||
|
if connectCalls != 1 || migrateCalls != 1 {
|
||||||
|
t.Fatalf("接続・移行回数: %d %d", connectCalls, migrateCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOpenDatabaseReportsMigrationFailure(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
pool, err := openDatabase(ctx, "postgres://test:secret@localhost/test", pgxpool.New,
|
||||||
|
func(context.Context, *pgxpool.Pool) error { return errors.New("migration failed") })
|
||||||
|
if pool != nil || err == nil || !strings.Contains(err.Error(), "migration failed") {
|
||||||
|
t.Fatalf("移行失敗: pool=%v err=%v", pool, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOpenDatabaseReportsConnectionFailureWithoutLeakingURL(t *testing.T) {
|
||||||
|
migrateCalled := false
|
||||||
|
pool, err := openDatabase(context.Background(), "postgres://user:secret@localhost/test",
|
||||||
|
func(context.Context, string) (*pgxpool.Pool, error) {
|
||||||
|
return nil, errors.New("postgres://user:secret@localhost/test: connection refused")
|
||||||
|
},
|
||||||
|
func(context.Context, *pgxpool.Pool) error {
|
||||||
|
migrateCalled = true
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if pool != nil || err == nil || strings.Contains(err.Error(), "secret") || migrateCalled {
|
||||||
|
t.Fatalf("接続失敗: pool=%v err=%v migrateCalled=%v", pool, err, migrateCalled)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestOpenDatabaseDoesNotExposeInvalidURL(t *testing.T) {
|
||||||
|
_, err := openDatabase(context.Background(), "postgres://user:secret@%", pgxpool.New,
|
||||||
|
func(context.Context, *pgxpool.Pool) error { t.Fatal("移行してはいけない"); return nil })
|
||||||
|
if err == nil || strings.Contains(err.Error(), "secret") {
|
||||||
|
t.Fatalf("接続 URL の秘匿: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
workerpkg "gitea.kazu634.com/kazu634/tver/internal/worker"
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
)
|
||||||
|
|
||||||
|
const staleAfter = 30 * time.Second
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
config, err := loadConfig(os.Getenv)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
pool, err := initializeDatabase(ctx, os.Getenv("DATABASE_URL"), time.Now(), pgxpool.New,
|
||||||
|
store.Migrate, func(ctx context.Context, pool *pgxpool.Pool, cutoff time.Time) (int64, error) {
|
||||||
|
return store.New(pool).RequeueStale(ctx, cutoff)
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
defer pool.Close()
|
||||||
|
jobStore := store.New(pool)
|
||||||
|
runner, err := workerpkg.New(jobStore, config, log.Default())
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
tverClient, err := tver.NewClient("", nil)
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
scheduler, err := workerpkg.NewSubscriptionScheduler(jobStore, tverClient,
|
||||||
|
workerpkg.SubscriptionSchedulerConfig{PreviewMaxSeconds: tver.PreviewMaxSeconds}, log.Default())
|
||||||
|
if err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
schedulerDone := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
defer close(schedulerDone)
|
||||||
|
scheduler.Run(ctx)
|
||||||
|
}()
|
||||||
|
log.Printf("worker started (concurrency=%d)", config.Concurrency)
|
||||||
|
if err := runner.Run(ctx); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
<-schedulerDone
|
||||||
|
log.Print("worker stopped")
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadConfig(getenv func(string) string) (workerpkg.Config, error) {
|
||||||
|
concurrency := 1
|
||||||
|
if value := strings.TrimSpace(getenv("WORKER_CONCURRENCY")); value != "" {
|
||||||
|
parsed, err := strconv.Atoi(value)
|
||||||
|
if err != nil || parsed < 1 || parsed > 2 {
|
||||||
|
return workerpkg.Config{}, errors.New("WORKER_CONCURRENCY は 1 または 2 を指定してください")
|
||||||
|
}
|
||||||
|
concurrency = parsed
|
||||||
|
}
|
||||||
|
return workerpkg.Config{VideoDir: getenv("VIDEO_DIR"), YTDLPPath: getenv("YT_DLP_PATH"),
|
||||||
|
FFprobePath: getenv("FFPROBE_PATH"), Concurrency: concurrency}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func initializeDatabase(ctx context.Context, databaseURL string, now time.Time,
|
||||||
|
connect func(context.Context, string) (*pgxpool.Pool, error),
|
||||||
|
migrate func(context.Context, *pgxpool.Pool) error,
|
||||||
|
requeue func(context.Context, *pgxpool.Pool, time.Time) (int64, error),
|
||||||
|
) (*pgxpool.Pool, error) {
|
||||||
|
if strings.TrimSpace(databaseURL) == "" {
|
||||||
|
return nil, errors.New("DATABASE_URL が設定されていません")
|
||||||
|
}
|
||||||
|
pool, err := connect(ctx, databaseURL)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New("DATABASE_URL から接続プールを作成できません")
|
||||||
|
}
|
||||||
|
if err := migrate(ctx, pool); err != nil {
|
||||||
|
pool.Close()
|
||||||
|
return nil, fmt.Errorf("DB マイグレーションに失敗: %w", err)
|
||||||
|
}
|
||||||
|
if _, err := requeue(ctx, pool, now.Add(-staleAfter)); err != nil {
|
||||||
|
pool.Close()
|
||||||
|
return nil, fmt.Errorf("停止中ジョブの復帰に失敗: %w", err)
|
||||||
|
}
|
||||||
|
return pool, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,97 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestLoadConfigConcurrency(t *testing.T) {
|
||||||
|
values := map[string]string{"VIDEO_DIR": "/media", "YT_DLP_PATH": "/bin/downloader", "FFPROBE_PATH": "/bin/probe"}
|
||||||
|
getenv := func(key string) string { return values[key] }
|
||||||
|
config, err := loadConfig(getenv)
|
||||||
|
if err != nil || config.Concurrency != 1 || config.VideoDir != "/media" || config.YTDLPPath != "/bin/downloader" || config.FFprobePath != "/bin/probe" {
|
||||||
|
t.Fatalf("既定設定: %+v %v", config, err)
|
||||||
|
}
|
||||||
|
values["WORKER_CONCURRENCY"] = "2"
|
||||||
|
config, err = loadConfig(getenv)
|
||||||
|
if err != nil || config.Concurrency != 2 {
|
||||||
|
t.Fatalf("並行数2: %+v %v", config, err)
|
||||||
|
}
|
||||||
|
for _, invalid := range []string{"0", "3", "bad"} {
|
||||||
|
values["WORKER_CONCURRENCY"] = invalid
|
||||||
|
if _, err := loadConfig(getenv); err == nil {
|
||||||
|
t.Errorf("不正な並行数 %q を許した", invalid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInitializeDatabaseRequiresURLMigratesAndRequeues(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||||
|
connectCalls, migrateCalls, requeueCalls := 0, 0, 0
|
||||||
|
connect := func(_ context.Context, url string) (*pgxpool.Pool, error) {
|
||||||
|
connectCalls++
|
||||||
|
if url != "postgres://test:secret@localhost/test" {
|
||||||
|
t.Errorf("URL=%q", url)
|
||||||
|
}
|
||||||
|
return pgxpool.New(ctx, url)
|
||||||
|
}
|
||||||
|
migrate := func(context.Context, *pgxpool.Pool) error { migrateCalls++; return nil }
|
||||||
|
requeue := func(_ context.Context, _ *pgxpool.Pool, cutoff time.Time) (int64, error) {
|
||||||
|
requeueCalls++
|
||||||
|
if want := now.Add(-staleAfter); !cutoff.Equal(want) {
|
||||||
|
t.Errorf("cutoff=%v want=%v", cutoff, want)
|
||||||
|
}
|
||||||
|
return 2, nil
|
||||||
|
}
|
||||||
|
if _, err := initializeDatabase(ctx, " ", now, connect, migrate, requeue); err == nil || !strings.Contains(err.Error(), "DATABASE_URL") {
|
||||||
|
t.Fatalf("未設定 URL: %v", err)
|
||||||
|
}
|
||||||
|
if connectCalls != 0 || migrateCalls != 0 || requeueCalls != 0 {
|
||||||
|
t.Fatalf("未設定時の呼び出し: %d %d %d", connectCalls, migrateCalls, requeueCalls)
|
||||||
|
}
|
||||||
|
pool, err := initializeDatabase(ctx, "postgres://test:secret@localhost/test", now, connect, migrate, requeue)
|
||||||
|
if err != nil || pool == nil {
|
||||||
|
t.Fatalf("初期化: %v", err)
|
||||||
|
}
|
||||||
|
pool.Close()
|
||||||
|
if connectCalls != 1 || migrateCalls != 1 || requeueCalls != 1 {
|
||||||
|
t.Fatalf("呼び出し回数: %d %d %d", connectCalls, migrateCalls, requeueCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInitializeDatabaseStopsOnMigrationOrRequeueFailure(t *testing.T) {
|
||||||
|
ctx := context.Background()
|
||||||
|
connect := func(context.Context, string) (*pgxpool.Pool, error) {
|
||||||
|
return pgxpool.New(ctx, "postgres://localhost/test")
|
||||||
|
}
|
||||||
|
requeueCalled := false
|
||||||
|
pool, err := initializeDatabase(ctx, "postgres://localhost/test", time.Now(), connect,
|
||||||
|
func(context.Context, *pgxpool.Pool) error { return errors.New("migration failed") },
|
||||||
|
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { requeueCalled = true; return 0, nil })
|
||||||
|
if pool != nil || err == nil || !strings.Contains(err.Error(), "migration failed") || requeueCalled {
|
||||||
|
t.Fatalf("migration failure: pool=%v err=%v requeue=%v", pool, err, requeueCalled)
|
||||||
|
}
|
||||||
|
pool, err = initializeDatabase(ctx, "postgres://localhost/test", time.Now(), connect,
|
||||||
|
func(context.Context, *pgxpool.Pool) error { return nil },
|
||||||
|
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { return 0, errors.New("requeue failed") })
|
||||||
|
if pool != nil || err == nil || !strings.Contains(err.Error(), "requeue failed") {
|
||||||
|
t.Fatalf("requeue failure: pool=%v err=%v", pool, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInitializeDatabaseDoesNotLeakURL(t *testing.T) {
|
||||||
|
_, err := initializeDatabase(context.Background(), "postgres://user:secret@localhost/test", time.Now(),
|
||||||
|
func(context.Context, string) (*pgxpool.Pool, error) {
|
||||||
|
return nil, errors.New("postgres://user:secret@localhost/test refused")
|
||||||
|
}, func(context.Context, *pgxpool.Pool) error { return nil },
|
||||||
|
func(context.Context, *pgxpool.Pool, time.Time) (int64, error) { return 0, nil })
|
||||||
|
if err == nil || strings.Contains(err.Error(), "secret") {
|
||||||
|
t.Fatalf("URL 漏えい: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
services:
|
||||||
|
web:
|
||||||
|
build: .
|
||||||
|
env_file: .env
|
||||||
|
environment:
|
||||||
|
TZ: Asia/Tokyo
|
||||||
|
WEB_ADDR: ":8080"
|
||||||
|
VIDEO_DIR: "${VIDEO_DIR:-/videos}"
|
||||||
|
volumes:
|
||||||
|
- "${VIDEO_HOST_DIR:?VIDEO_HOST_DIR must be set}:${VIDEO_DIR:-/videos}"
|
||||||
|
ports:
|
||||||
|
- "192.168.10.155:8080:8080"
|
||||||
|
restart: unless-stopped
|
||||||
|
worker:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile.worker
|
||||||
|
env_file: .env
|
||||||
|
environment:
|
||||||
|
TZ: Asia/Tokyo
|
||||||
|
VIDEO_DIR: "${VIDEO_DIR:-/videos}"
|
||||||
|
WORKER_CONCURRENCY: "${WORKER_CONCURRENCY:-1}"
|
||||||
|
volumes:
|
||||||
|
- "${VIDEO_HOST_DIR:?VIDEO_HOST_DIR must be set}:${VIDEO_DIR:-/videos}"
|
||||||
|
restart: unless-stopped
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
POSTGRES_PASSWORD=
|
||||||
Executable
+66
@@ -0,0 +1,66 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# tver DBバックアップ（Synology ds923p / DSMタスクスケジューラで日次実行）
|
||||||
|
#
|
||||||
|
# 配置先: /volume1/docker/tver-db/backup-db.sh
|
||||||
|
# DSM設定: コントロールパネル → タスクスケジューラ → 作成
|
||||||
|
# → スケジュールされたタスク → ユーザー定義のスクリプト
|
||||||
|
# ユーザーはroot、実行周期は毎日とする。
|
||||||
|
# 実行コマンド: bash /volume1/docker/tver-db/backup-db.sh
|
||||||
|
#
|
||||||
|
# コンテナ内のpg_dumpを使い、サーバーとクライアントのバージョンを合わせる。
|
||||||
|
# PostgreSQLのローカル接続を使うため、DBパスワードはスクリプトへ書かない。
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
BACKUP_DIR="${BACKUP_DIR:-/volume1/backups/tver}"
|
||||||
|
KEEP_DAYS="${KEEP_DAYS:-14}"
|
||||||
|
CONTAINER="${CONTAINER:-tver-postgres}"
|
||||||
|
DB_USER="${DB_USER:-tver}"
|
||||||
|
DB_NAME="${DB_NAME:-tver}"
|
||||||
|
DOCKER="${DOCKER:-$(command -v docker || printf '/usr/local/bin/docker')}"
|
||||||
|
STAMP="${STAMP:-$(date +%Y%m%d_%H%M%S)}"
|
||||||
|
OUT="${BACKUP_DIR}/tver_${STAMP}.sql.gz"
|
||||||
|
RAW_TMP="${BACKUP_DIR}/.tver_${STAMP}.$$.sql"
|
||||||
|
GZ_TMP="${OUT}.tmp.$$"
|
||||||
|
|
||||||
|
case "$KEEP_DAYS" in
|
||||||
|
''|*[!0-9]*)
|
||||||
|
echo "ERROR: KEEP_DAYS must be a non-negative integer: $KEEP_DAYS" >&2
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
mkdir -p "$BACKUP_DIR"
|
||||||
|
umask 077
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
rm -f "$RAW_TMP" "$GZ_TMP"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
# 先に非圧縮の一時ファイルへ出し、pg_dump成功と空でないことを確認する。
|
||||||
|
"$DOCKER" exec "$CONTAINER" pg_dump -U "$DB_USER" -d "$DB_NAME" > "$RAW_TMP"
|
||||||
|
if [[ ! -s "$RAW_TMP" ]]; then
|
||||||
|
echo "ERROR: dump is empty" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
gzip -9 -c "$RAW_TMP" > "$GZ_TMP"
|
||||||
|
if [[ ! -s "$GZ_TMP" ]]; then
|
||||||
|
echo "ERROR: compressed dump is empty" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
# 完成ファイルは上書きしない。同名が同時に作られても、ハードリンクの
|
||||||
|
# 作成に成功した一方だけを公開する。一時ファイルは同じディレクトリに
|
||||||
|
# あるため同一ファイルシステム内でリンクできる。
|
||||||
|
if ! ln "$GZ_TMP" "$OUT"; then
|
||||||
|
echo "ERROR: backup already exists or could not be published: $OUT" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
rm -f "$GZ_TMP" "$RAW_TMP"
|
||||||
|
|
||||||
|
# KEEP_DAYS日より古い完了済みバックアップだけを削除する。
|
||||||
|
find "$BACKUP_DIR" -name 'tver_*.sql.gz' -type f -mtime "+$KEEP_DAYS" -delete
|
||||||
|
|
||||||
|
SIZE_BYTES="$(wc -c < "$OUT" | tr -d '[:space:]')"
|
||||||
|
echo "OK: $OUT ($SIZE_BYTES bytes)"
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# tver 用 PostgreSQL スタック（Synology ds923p / 192.168.10.201）
|
||||||
|
#
|
||||||
|
# 配置先: ds923p の Container Manager「プロジェクト」
|
||||||
|
# /volume1/docker/tver-db/ に本ファイルを compose.yaml として配置する。
|
||||||
|
# パスワードは同じディレクトリの .env（POSTGRES_PASSWORD）から注入し、
|
||||||
|
# このファイルには実値を書かない。
|
||||||
|
#
|
||||||
|
# 注意（初回起動チェックポイント）:
|
||||||
|
# 起動前に /volume1/docker/tver-db/pgdata を作成する。
|
||||||
|
# 初回起動後は Container Manager のヘルス状態とログを確認し、
|
||||||
|
# "database system is ready to accept connections" が出てからアプリを起動する。
|
||||||
|
# bind mount の所有権・権限で初期化に失敗した場合は、原因を確認してから
|
||||||
|
# 名前付きボリュームへの切り替えを検討する。
|
||||||
|
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
# Debian ベースの PostgreSQL 17。initdb の UTF-8 ロケールを利用する。
|
||||||
|
image: postgres:17
|
||||||
|
container_name: tver-postgres
|
||||||
|
restart: always
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: tver
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||||
|
POSTGRES_DB: tver
|
||||||
|
# 初回の initdb でのみ有効。既存の pgdata には後から反映されない。
|
||||||
|
POSTGRES_INITDB_ARGS: "--encoding=UTF8 --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8"
|
||||||
|
ports:
|
||||||
|
# [要確認] Container Manager で 15434 が未使用であることを起動前に確認する。
|
||||||
|
# DSM ファイアウォールでは dockage（192.168.10.155）だけを許可する。
|
||||||
|
- "15434:5432"
|
||||||
|
volumes:
|
||||||
|
- /volume1/docker/tver-db/pgdata:/var/lib/postgresql/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U tver -d tver"]
|
||||||
|
interval: 10s
|
||||||
|
retries: 5
|
||||||
Executable
+73
@@ -0,0 +1,73 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# tver DB復元確認（Synology ds923pで手動実行）
|
||||||
|
#
|
||||||
|
# 配置先: /volume1/docker/tver-db/restore-test.sh
|
||||||
|
# 実行コマンド: bash /volume1/docker/tver-db/restore-test.sh
|
||||||
|
# 最新のバックアップを同じPostgreSQLコンテナ内の一時DBへ復元し、主要4表の
|
||||||
|
# 行数を本番DBと比較する。本番DBにはSELECTだけを実行する。
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
BACKUP_DIR="${BACKUP_DIR:-/volume1/backups/tver}"
|
||||||
|
LOG_FILE="${LOG_FILE:-${BACKUP_DIR}/restore-test.log}"
|
||||||
|
CONTAINER="${CONTAINER:-tver-postgres}"
|
||||||
|
DB_USER="${DB_USER:-tver}"
|
||||||
|
DB_NAME="${DB_NAME:-tver}"
|
||||||
|
RESTORE_DB="${RESTORE_DB:-tver_restore_test}"
|
||||||
|
DOCKER="${DOCKER:-$(command -v docker || printf '/usr/local/bin/docker')}"
|
||||||
|
TABLES=(download_jobs subscriptions known_episodes schema_migrations)
|
||||||
|
|
||||||
|
mkdir -p "$BACKUP_DIR"
|
||||||
|
touch "$LOG_FILE"
|
||||||
|
chmod 600 "$LOG_FILE"
|
||||||
|
exec > >(tee -a "$LOG_FILE") 2>&1
|
||||||
|
|
||||||
|
if [[ "$RESTORE_DB" == "$DB_NAME" ]]; then
|
||||||
|
echo "ERROR: RESTORE_DB must differ from DB_NAME" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# 終了状態にかかわらず、一時DBの削除を最後に試みる。
|
||||||
|
cleanup() {
|
||||||
|
status=$?
|
||||||
|
trap - EXIT
|
||||||
|
set +e
|
||||||
|
"$DOCKER" exec "$CONTAINER" dropdb --force --if-exists -U "$DB_USER" "$RESTORE_DB"
|
||||||
|
cleanup_status=$?
|
||||||
|
if ((status != 0)); then
|
||||||
|
echo "復元確認に失敗"
|
||||||
|
exit "$status"
|
||||||
|
fi
|
||||||
|
exit "$cleanup_status"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
LATEST="$(find "$BACKUP_DIR" -name 'tver_*.sql.gz' -type f -print | sort | tail -n 1)"
|
||||||
|
if [[ -z "$LATEST" ]]; then
|
||||||
|
echo "ERROR: backup dump was not found in $BACKUP_DIR" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "復元確認を開始: $LATEST"
|
||||||
|
"$DOCKER" exec "$CONTAINER" dropdb --force --if-exists -U "$DB_USER" "$RESTORE_DB"
|
||||||
|
"$DOCKER" exec "$CONTAINER" createdb -U "$DB_USER" "$RESTORE_DB"
|
||||||
|
|
||||||
|
gzip -dc "$LATEST" \
|
||||||
|
| "$DOCKER" exec -i "$CONTAINER" psql -q -o /dev/null -v ON_ERROR_STOP=1 -U "$DB_USER" -d "$RESTORE_DB"
|
||||||
|
|
||||||
|
for table in "${TABLES[@]}"; do
|
||||||
|
query="SELECT count(*) FROM public.${table};"
|
||||||
|
production_count="$("$DOCKER" exec "$CONTAINER" psql -A -t -q -U "$DB_USER" -d "$DB_NAME" -c "$query" | tr -d '[:space:]')"
|
||||||
|
restored_count="$("$DOCKER" exec "$CONTAINER" psql -A -t -q -U "$DB_USER" -d "$RESTORE_DB" -c "$query" | tr -d '[:space:]')"
|
||||||
|
if [[ ! "$production_count" =~ ^[0-9]+$ || ! "$restored_count" =~ ^[0-9]+$ ]]; then
|
||||||
|
echo "ERROR: $table の行数を取得できませんでした: production=$production_count restore=$restored_count" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [[ "$production_count" != "$restored_count" ]]; then
|
||||||
|
echo "ERROR: $table の行数が一致しません: production=$production_count restore=$restored_count" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "OK: $table production=$production_count restore=$restored_count"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "復元確認に成功: $LATEST"
|
||||||
@@ -0,0 +1,401 @@
|
|||||||
|
package tverdb_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"compress/gzip"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestBackupDB(t *testing.T) {
|
||||||
|
t.Run("success and retention boundary", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
old := filepath.Join(env.backupDir, "tver_20260901_000000.sql.gz")
|
||||||
|
boundary := filepath.Join(env.backupDir, "tver_20260920_000000.sql.gz")
|
||||||
|
unrelated := []string{
|
||||||
|
filepath.Join(env.backupDir, "restore-test.log"),
|
||||||
|
filepath.Join(env.backupDir, "other_20260901_000000.sql.gz"),
|
||||||
|
filepath.Join(env.backupDir, "tver_20260901_000000.sql"),
|
||||||
|
}
|
||||||
|
writeFile(t, old, "old")
|
||||||
|
writeFile(t, boundary, "boundary")
|
||||||
|
for _, path := range unrelated {
|
||||||
|
writeFile(t, path, "keep")
|
||||||
|
}
|
||||||
|
now := time.Now()
|
||||||
|
oldTime := now.Add(-(15*24*time.Hour + time.Hour))
|
||||||
|
boundaryTime := now.Add(-(15*24*time.Hour - time.Hour))
|
||||||
|
if err := os.Chtimes(old, oldTime, oldTime); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Chtimes(boundary, boundaryTime, boundaryTime); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, path := range unrelated {
|
||||||
|
if err := os.Chtimes(path, oldTime, oldTime); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203", "KEEP_DAYS=14", "CONTAINER=custom-postgres", "DB_USER=custom-user", "DB_NAME=custom-db")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("backup failed: %v\n%s", err, output)
|
||||||
|
}
|
||||||
|
created := filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")
|
||||||
|
if got := readGzip(t, created); !strings.Contains(got, "CREATE TABLE example") {
|
||||||
|
t.Fatalf("dump=%q", got)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(old); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("old backup remains: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(boundary); err != nil {
|
||||||
|
t.Fatalf("14-day backup was deleted: %v", err)
|
||||||
|
}
|
||||||
|
for _, path := range unrelated {
|
||||||
|
if got := readFile(t, path); got != "keep" {
|
||||||
|
t.Fatalf("unrelated file was changed: %s content=%q", path, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !strings.Contains(output, "OK: "+created) || !strings.Contains(output, "bytes") {
|
||||||
|
t.Fatalf("success output=%q", output)
|
||||||
|
}
|
||||||
|
if log := readFile(t, env.dockerLog); !strings.Contains(log, "exec custom-postgres pg_dump -U custom-user -d custom-db") {
|
||||||
|
t.Fatalf("docker log=%q", log)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("creates missing backup directory", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
if err := os.RemoveAll(env.backupDir); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("backup failed: %v\n%s", err, output)
|
||||||
|
}
|
||||||
|
if got := readGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")); !strings.Contains(got, "CREATE TABLE example") {
|
||||||
|
t.Fatalf("dump=%q", got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("does not replace an existing completed backup", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
completed := filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")
|
||||||
|
writeGzip(t, completed, "existing")
|
||||||
|
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("backup replaced an existing file: %s", output)
|
||||||
|
}
|
||||||
|
if got := readGzip(t, completed); got != "existing" {
|
||||||
|
t.Fatalf("existing backup was changed: %q", got)
|
||||||
|
}
|
||||||
|
assertNoBackupTemps(t, env.backupDir)
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("same stamp concurrently allows exactly one backup", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "concurrent")
|
||||||
|
barrier := filepath.Join(env.dir, "barrier")
|
||||||
|
extra := []string{"STAMP=20261004_010203", "FAKE_DOCKER_BARRIER=" + barrier}
|
||||||
|
commands := make([]*exec.Cmd, 2)
|
||||||
|
outputs := make([]bytes.Buffer, 2)
|
||||||
|
for i := range commands {
|
||||||
|
commands[i] = exec.Command("bash", "backup-db.sh")
|
||||||
|
commands[i].Env = append(append(append([]string{}, os.Environ()...), env.baseEnv...), extra...)
|
||||||
|
commands[i].Stdout = &outputs[i]
|
||||||
|
commands[i].Stderr = &outputs[i]
|
||||||
|
if err := commands[i].Start(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
succeeded := 0
|
||||||
|
for i, command := range commands {
|
||||||
|
if err := command.Wait(); err == nil {
|
||||||
|
succeeded++
|
||||||
|
} else if _, ok := err.(*exec.ExitError); !ok {
|
||||||
|
t.Fatalf("backup %d wait: %v output=%q", i, err, outputs[i].String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if succeeded != 1 {
|
||||||
|
t.Fatalf("concurrent successes=%d want=1 outputs=%q / %q", succeeded, outputs[0].String(), outputs[1].String())
|
||||||
|
}
|
||||||
|
if got := readGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz")); !strings.Contains(got, "CREATE TABLE example") {
|
||||||
|
t.Fatalf("completed dump=%q", got)
|
||||||
|
}
|
||||||
|
assertNoBackupTemps(t, env.backupDir)
|
||||||
|
})
|
||||||
|
|
||||||
|
for _, mode := range []string{"dump-fail", "dump-partial-fail", "empty"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, mode)
|
||||||
|
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("backup succeeded unexpectedly: %s", output)
|
||||||
|
}
|
||||||
|
matches, globErr := filepath.Glob(filepath.Join(env.backupDir, "tver_*.sql.gz"))
|
||||||
|
if globErr != nil || len(matches) != 0 {
|
||||||
|
t.Fatalf("partial backups=%v err=%v", matches, globErr)
|
||||||
|
}
|
||||||
|
assertNoBackupTemps(t, env.backupDir)
|
||||||
|
if mode == "empty" && !strings.Contains(output, "empty") {
|
||||||
|
t.Fatalf("empty dump error missing: %q", output)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, keepDays := range []string{"-1", "invalid"} {
|
||||||
|
t.Run("invalid retention "+keepDays, func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
output, err := env.run(t, "backup-db.sh", "STAMP=20261004_010203", "KEEP_DAYS="+keepDays)
|
||||||
|
if err == nil || !strings.Contains(output, "KEEP_DAYS") {
|
||||||
|
t.Fatalf("invalid KEEP_DAYS: err=%v output=%q", err, output)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(env.dockerLog); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("pg_dump was invoked: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRestoreTest(t *testing.T) {
|
||||||
|
t.Run("restores latest dump and compares rows", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
writeGzip(t, filepath.Join(env.backupDir, "tver_20261003_010203.sql.gz"), "older")
|
||||||
|
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "newest")
|
||||||
|
output, err := env.run(t, "restore-test.sh")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("restore test failed: %v\n%s", err, output)
|
||||||
|
}
|
||||||
|
for _, table := range []string{"download_jobs", "subscriptions", "known_episodes", "schema_migrations"} {
|
||||||
|
if !strings.Contains(output, "OK: "+table) {
|
||||||
|
t.Errorf("missing result for %s: %s", table, output)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := readFile(t, env.restoreInput); got != "newest" {
|
||||||
|
t.Fatalf("restored dump=%q", got)
|
||||||
|
}
|
||||||
|
log := readFile(t, env.dockerLog)
|
||||||
|
if !strings.Contains(log, "psql -q -o /dev/null -v ON_ERROR_STOP=1") {
|
||||||
|
t.Fatalf("psql output suppression is missing: %q", log)
|
||||||
|
}
|
||||||
|
cleanupCommand := "dropdb --force --if-exists -U tver tver_restore_test"
|
||||||
|
if strings.Count(log, cleanupCommand) != 2 || !strings.HasSuffix(strings.TrimSpace(log), cleanupCommand) {
|
||||||
|
t.Fatalf("temporary DB cleanup log=%q", log)
|
||||||
|
}
|
||||||
|
for _, line := range strings.Split(strings.TrimSpace(log), "\n") {
|
||||||
|
if (strings.Contains(line, " createdb ") || strings.Contains(line, " dropdb ")) && !strings.HasSuffix(line, "tver_restore_test") {
|
||||||
|
t.Fatalf("production DB mutation was attempted: %q", line)
|
||||||
|
}
|
||||||
|
if strings.Contains(line, " psql ") && strings.Contains(line, " -d tver ") && !strings.Contains(line, "SELECT count(*)") {
|
||||||
|
t.Fatalf("production DB received a non-count command: %q", line)
|
||||||
|
}
|
||||||
|
if strings.Contains(line, " ON_ERROR_STOP=1 ") && !strings.Contains(line, " -d tver_restore_test") {
|
||||||
|
t.Fatalf("dump was restored into a non-temporary DB: %q", line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if restoreLog := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(restoreLog, "復元確認に成功") || strings.Contains(restoreLog, "SET\n") || strings.Contains(restoreLog, "CREATE TABLE\n") {
|
||||||
|
t.Fatalf("restore log=%q", restoreLog)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
for _, mode := range []string{"restore-fail", "count-mismatch"} {
|
||||||
|
t.Run(mode+" cleans temporary database", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, mode)
|
||||||
|
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "dump")
|
||||||
|
output, err := env.run(t, "restore-test.sh")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("restore test succeeded unexpectedly: %s", output)
|
||||||
|
}
|
||||||
|
log := strings.TrimSpace(readFile(t, env.dockerLog))
|
||||||
|
if !strings.HasSuffix(log, "dropdb --force --if-exists -U tver tver_restore_test") {
|
||||||
|
t.Fatalf("cleanup was not last command: %q", log)
|
||||||
|
}
|
||||||
|
if mode == "count-mismatch" && !strings.Contains(output, "ERROR: known_episodes") {
|
||||||
|
t.Fatalf("mismatch not reported: %q", output)
|
||||||
|
}
|
||||||
|
if mode == "restore-fail" {
|
||||||
|
if !strings.Contains(output, "restore failed") {
|
||||||
|
t.Fatalf("restore error missing from output: %q", output)
|
||||||
|
}
|
||||||
|
if restoreLog := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(restoreLog, "restore failed") {
|
||||||
|
t.Fatalf("restore error missing from log: %q", restoreLog)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
t.Run("rejects production database as restore target", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "dump")
|
||||||
|
output, err := env.run(t, "restore-test.sh", "RESTORE_DB=tver")
|
||||||
|
if err == nil || !strings.Contains(output, "RESTORE_DB must differ from DB_NAME") {
|
||||||
|
t.Fatalf("production restore target was accepted: err=%v output=%q", err, output)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(env.dockerLog); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("docker was invoked: %v", err)
|
||||||
|
}
|
||||||
|
if log := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(log, "RESTORE_DB must differ from DB_NAME") {
|
||||||
|
t.Fatalf("rejection was not logged: %q", log)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("missing dump fails and cleans temporary database", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
output, err := env.run(t, "restore-test.sh")
|
||||||
|
if err == nil || !strings.Contains(output, "backup dump was not found") {
|
||||||
|
t.Fatalf("missing dump: err=%v output=%q", err, output)
|
||||||
|
}
|
||||||
|
log := strings.TrimSpace(readFile(t, env.dockerLog))
|
||||||
|
if log != "exec tver-postgres dropdb --force --if-exists -U tver tver_restore_test" {
|
||||||
|
t.Fatalf("unexpected cleanup: %q", log)
|
||||||
|
}
|
||||||
|
if restoreLog := readFile(t, filepath.Join(env.backupDir, "restore-test.log")); !strings.Contains(restoreLog, "backup dump was not found") {
|
||||||
|
t.Fatalf("failure was not logged: %q", restoreLog)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("honors database and container overrides", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
writeGzip(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "dump")
|
||||||
|
output, err := env.run(t, "restore-test.sh", "CONTAINER=custom-postgres", "DB_USER=custom-user", "DB_NAME=production", "RESTORE_DB=temporary")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("restore failed: %v\n%s", err, output)
|
||||||
|
}
|
||||||
|
for _, line := range strings.Split(strings.TrimSpace(readFile(t, env.dockerLog)), "\n") {
|
||||||
|
if !strings.Contains(line, " custom-postgres ") || strings.Contains(line, " tver-postgres ") || !strings.Contains(line, " -U custom-user ") {
|
||||||
|
t.Fatalf("override missing: %q", line)
|
||||||
|
}
|
||||||
|
if (strings.Contains(line, " createdb ") || strings.Contains(line, " dropdb ")) && !strings.HasSuffix(line, "temporary") {
|
||||||
|
t.Fatalf("wrong temporary DB: %q", line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("invalid gzip fails and cleans temporary database", func(t *testing.T) {
|
||||||
|
env := newScriptEnv(t, "success")
|
||||||
|
writeFile(t, filepath.Join(env.backupDir, "tver_20261004_010203.sql.gz"), "not gzip")
|
||||||
|
output, err := env.run(t, "restore-test.sh")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("invalid gzip succeeded: %s", output)
|
||||||
|
}
|
||||||
|
log := strings.TrimSpace(readFile(t, env.dockerLog))
|
||||||
|
if !strings.HasSuffix(log, "dropdb --force --if-exists -U tver tver_restore_test") {
|
||||||
|
t.Fatalf("cleanup was not last command: %q", log)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
type scriptEnv struct {
|
||||||
|
dir, backupDir, dockerLog, restoreInput string
|
||||||
|
baseEnv []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func newScriptEnv(t *testing.T, mode string) scriptEnv {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
backupDir := filepath.Join(dir, "backups")
|
||||||
|
if err := os.MkdirAll(backupDir, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
scriptDir, err := filepath.Abs(".")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
dockerDir := filepath.Join(scriptDir, "testdata")
|
||||||
|
return scriptEnv{
|
||||||
|
dir: dir,
|
||||||
|
backupDir: backupDir,
|
||||||
|
dockerLog: filepath.Join(dir, "docker.log"),
|
||||||
|
restoreInput: filepath.Join(dir, "restore.sql"),
|
||||||
|
baseEnv: []string{
|
||||||
|
"PATH=" + dockerDir + string(os.PathListSeparator) + os.Getenv("PATH"),
|
||||||
|
"BACKUP_DIR=" + backupDir,
|
||||||
|
"LOG_FILE=" + filepath.Join(backupDir, "restore-test.log"),
|
||||||
|
"FAKE_DOCKER_LOG=" + filepath.Join(dir, "docker.log"),
|
||||||
|
"FAKE_RESTORE_INPUT=" + filepath.Join(dir, "restore.sql"),
|
||||||
|
"FAKE_DOCKER_MODE=" + mode,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e scriptEnv) run(t *testing.T, script string, extraEnv ...string) (string, error) {
|
||||||
|
t.Helper()
|
||||||
|
cmd := exec.Command("bash", script)
|
||||||
|
cmd.Env = append(append(os.Environ(), e.baseEnv...), extraEnv...)
|
||||||
|
var output bytes.Buffer
|
||||||
|
cmd.Stdout = &output
|
||||||
|
cmd.Stderr = &output
|
||||||
|
err := cmd.Run()
|
||||||
|
return output.String(), err
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeFile(t *testing.T, path, content string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func readFile(t *testing.T, path string) string {
|
||||||
|
t.Helper()
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return string(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeGzip(t *testing.T, path, content string) {
|
||||||
|
t.Helper()
|
||||||
|
file, err := os.Create(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
writer := gzip.NewWriter(file)
|
||||||
|
if _, err := io.WriteString(writer, content); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := writer.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := file.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func readGzip(t *testing.T, path string) string {
|
||||||
|
t.Helper()
|
||||||
|
file, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer file.Close()
|
||||||
|
reader, err := gzip.NewReader(file)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer reader.Close()
|
||||||
|
data, err := io.ReadAll(reader)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return string(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func assertNoBackupTemps(t *testing.T, backupDir string) {
|
||||||
|
t.Helper()
|
||||||
|
entries, err := os.ReadDir(backupDir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, entry := range entries {
|
||||||
|
name := entry.Name()
|
||||||
|
if strings.HasPrefix(name, ".tver_") || strings.Contains(name, ".sql.gz.tmp.") {
|
||||||
|
t.Errorf("temporary backup remains: %s", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+97
@@ -0,0 +1,97 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
printf '%s\n' "$*" >> "${FAKE_DOCKER_LOG:?}"
|
||||||
|
|
||||||
|
command_line=" $* "
|
||||||
|
if [[ "$command_line" == *" pg_dump "* ]]; then
|
||||||
|
case "${FAKE_DOCKER_MODE:-success}" in
|
||||||
|
dump-fail)
|
||||||
|
printf 'pg_dump failed\n' >&2
|
||||||
|
exit 42
|
||||||
|
;;
|
||||||
|
dump-partial-fail)
|
||||||
|
printf '%s\n' '-- incomplete dump --'
|
||||||
|
printf 'pg_dump failed after output\n' >&2
|
||||||
|
exit 42
|
||||||
|
;;
|
||||||
|
empty)
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
concurrent)
|
||||||
|
barrier="${FAKE_DOCKER_BARRIER:?}"
|
||||||
|
mkdir -p "$barrier"
|
||||||
|
touch "$barrier/$$"
|
||||||
|
for _ in {1..500}; do
|
||||||
|
count="$(find "$barrier" -type f | wc -l | tr -d '[:space:]')"
|
||||||
|
if ((count >= 2)); then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 0.01
|
||||||
|
done
|
||||||
|
if ((count < 2)); then
|
||||||
|
printf 'concurrent backup barrier timed out\n' >&2
|
||||||
|
exit 46
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
printf '%s\n' '-- fake PostgreSQL dump --' 'CREATE TABLE example(id integer);'
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$command_line" == *" createdb "* ]]; then
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$command_line" == *" dropdb "* ]]; then
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$command_line" == *" psql "* && "$command_line" == *" ON_ERROR_STOP=1 "* ]]; then
|
||||||
|
cat > "${FAKE_RESTORE_INPUT:?}"
|
||||||
|
if [[ "${FAKE_DOCKER_MODE:-success}" == "restore-fail" ]]; then
|
||||||
|
printf 'restore failed\n' >&2
|
||||||
|
exit 43
|
||||||
|
fi
|
||||||
|
if [[ "$command_line" != *" -o /dev/null "* ]]; then
|
||||||
|
printf '%s\n' 'SET' 'CREATE TABLE'
|
||||||
|
fi
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "$command_line" == *" psql "* && "$command_line" == *" SELECT count(*) FROM public."* ]]; then
|
||||||
|
database=""
|
||||||
|
query=""
|
||||||
|
while (($#)); do
|
||||||
|
case "$1" in
|
||||||
|
-d)
|
||||||
|
database="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-c)
|
||||||
|
query="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
table="${query#*public.}"
|
||||||
|
table="${table%%;*}"
|
||||||
|
case "$table" in
|
||||||
|
download_jobs) count=3 ;;
|
||||||
|
subscriptions) count=2 ;;
|
||||||
|
known_episodes) count=5 ;;
|
||||||
|
schema_migrations) count=3 ;;
|
||||||
|
*) printf 'unexpected table: %s\n' "$table" >&2; exit 44 ;;
|
||||||
|
esac
|
||||||
|
if [[ "${FAKE_DOCKER_MODE:-success}" == "count-mismatch" && "$database" == "${RESTORE_DB:-tver_restore_test}" && "$table" == "known_episodes" ]]; then
|
||||||
|
count=4
|
||||||
|
fi
|
||||||
|
printf '%s\n' "$count"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
printf 'unexpected docker invocation: %s\n' "$*" >&2
|
||||||
|
exit 45
|
||||||
+339
@@ -0,0 +1,339 @@
|
|||||||
|
---
|
||||||
|
title: tver 本番配置手順
|
||||||
|
tags:
|
||||||
|
- homelab
|
||||||
|
- tver
|
||||||
|
- deployment
|
||||||
|
---
|
||||||
|
|
||||||
|
# tver 本番配置手順
|
||||||
|
|
||||||
|
> [!info] 対象
|
||||||
|
> Synology NAS `ds923p`（`192.168.10.201`）に PostgreSQL 17 と動画共有を用意し、dockage（`192.168.10.155`）の `/opt/stacks/tver` で web・worker を起動する。
|
||||||
|
|
||||||
|
> [!danger] 秘匿値を記録しない
|
||||||
|
> PostgreSQL のパスワードや完成した `DATABASE_URL` は、この文書や Git 管理下のファイルに書かない。NAS と dockage の `.env` にだけ保存し、権限を `600` にする。
|
||||||
|
|
||||||
|
## 事前に確認する値
|
||||||
|
|
||||||
|
| 項目 | 値 |
|
||||||
|
|---|---|
|
||||||
|
| NAS | `ds923p` / `192.168.10.201` |
|
||||||
|
| dockage | `192.168.10.155` |
|
||||||
|
| NAS のNFS共有 | `/volume1/video` |
|
||||||
|
| dockage のNFSマウント | `/mnt/nas-tver` |
|
||||||
|
| tverの動画保存先 | NAS: `/volume1/video/tver`、dockage: `/mnt/nas-tver/tver` |
|
||||||
|
| PostgreSQL 公開ポート | `15434` |
|
||||||
|
| PostgreSQL ユーザー / DB | `tver` / `tver` |
|
||||||
|
| アプリ配置先 | `/opt/stacks/tver` |
|
||||||
|
| Gitea配置用ユーザー | `tver-deploy`（リポジトリ読み取りのみ） |
|
||||||
|
|
||||||
|
## 1. NASを準備する
|
||||||
|
|
||||||
|
### 1.1 NFSサービスを有効にする
|
||||||
|
|
||||||
|
1. DSMの「コントロール パネル」→「ファイル サービス」→「NFS」でNFSサービスを有効にする。
|
||||||
|
2. 共有フォルダ `video` が `/volume1/video` にあることを確認する。
|
||||||
|
3. `video` の「編集」→「NFS 権限」で次を追加する。
|
||||||
|
|
||||||
|
| 設定 | 値 |
|
||||||
|
|---|---|
|
||||||
|
| ホスト名またはIP | `192.168.10.155` |
|
||||||
|
| 権限 | 読み書き |
|
||||||
|
| Squash | すべてのユーザーをadminに割り当てる |
|
||||||
|
| セキュリティ | `sys` |
|
||||||
|
| 非同期を有効にする | オン |
|
||||||
|
|
||||||
|
> [!success] adminへの割り当てを確認済み
|
||||||
|
> DSMの`admin`アカウントが無効な状態でも、上記設定でUID `65532`からフォルダ作成・書き込み・読み込み・rename・削除に成功した。NAS上の所有者はUID `1024`・GID `100`だった。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
### 1.2 PostgreSQLのファイルを用意する
|
||||||
|
|
||||||
|
1. NAS上に次のディレクトリを事前作成する。
|
||||||
|
|
||||||
|
```text
|
||||||
|
/volume1/docker/tver-db/
|
||||||
|
/volume1/docker/tver-db/pgdata/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. [[../deploy/nas/tver-db/compose.yaml|NAS用compose.yaml]]、`.env.example`、`backup-db.sh`、`restore-test.sh`を`/volume1/docker/tver-db/`に配置する。ディレクトリはroot以外がファイルを削除・置換できないようにし、スクリプトはrootだけが実行できるようにする。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
chmod 755 /volume1/docker/tver-db
|
||||||
|
chmod 700 /volume1/docker/tver-db/backup-db.sh
|
||||||
|
chmod 700 /volume1/docker/tver-db/restore-test.sh
|
||||||
|
ls -ld /volume1/docker/tver-db
|
||||||
|
ls -l /volume1/docker/tver-db/backup-db.sh /volume1/docker/tver-db/restore-test.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
DSMのACLが付いている場合、`chmod`後の`ls`でパーミッション表示の末尾から`+`が外れ、ディレクトリが`drwxr-xr-x`、スクリプトが`-rwx------`になったことを確認する。`/volume1/docker`では全員書き込み可のACLが確認されたため、スクリプトだけでなく配置ディレクトリにも`chmod 755`を適用する。
|
||||||
|
|
||||||
|
3. `.env.example`を`.env`へコピーし、`POSTGRES_PASSWORD`に十分長いランダムな値を設定する。値には、URLへの埋め込みで追加のエスケープが不要な16進文字列を使える。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /volume1/docker/tver-db
|
||||||
|
umask 077
|
||||||
|
cp .env.example .env
|
||||||
|
# openssl rand -hex 64 で値を生成し、.env の POSTGRES_PASSWORD に設定する。
|
||||||
|
chmod 600 .env
|
||||||
|
```
|
||||||
|
|
||||||
|
> [!important] DBとアプリで同じパスワードを使う
|
||||||
|
> ここで設定した値を、後述するdockage側の`DATABASE_URL`にも使う。値そのものは文書やGitへ残さない。
|
||||||
|
|
||||||
|
### 1.3 Container ManagerでPostgreSQLを起動する
|
||||||
|
|
||||||
|
1. Container Managerの「プロジェクト」で、`/volume1/docker/tver-db/compose.yaml`を使う`tver-db`プロジェクトを作成する。
|
||||||
|
2. 起動前に、他のプロジェクトとポート一覧を確認し、`15434`が空いていることを確認する。
|
||||||
|
3. プロジェクトをビルド・起動する。
|
||||||
|
4. `tver-postgres`が`healthy`になることを確認する。
|
||||||
|
5. コンテナログに`database system is ready to accept connections`があることを確認する。
|
||||||
|
|
||||||
|
初期化に失敗した場合は、先に`pgdata`の所有権・権限とログを確認する。空のDBを作り直す場合を除き、既存の`pgdata`を削除しない。
|
||||||
|
|
||||||
|
### 1.4 DSMファイアウォールを設定する
|
||||||
|
|
||||||
|
TCP `15434`について、送信元`192.168.10.155`だけを許可するルールを追加する。他の送信元を許可する広いルールより前に配置し、LAN内の別ホストからDBへ接続できない構成にする。
|
||||||
|
|
||||||
|
DSMファイアウォールを設定した状態で、dockageからNASのTCP `15434`へ到達できることを確認済みである。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
## 2. dockageを準備する
|
||||||
|
|
||||||
|
### 2.1 NFSクライアントとマウント先を用意する
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo apt update
|
||||||
|
sudo apt install -y nfs-common
|
||||||
|
sudo mkdir -p /mnt/nas-tver
|
||||||
|
```
|
||||||
|
|
||||||
|
`/etc/fstab`へ次の1行を追加する。`_netdev`でネットワーク後に扱い、`nofail`でNAS停止時もdockageの起動を妨げない。
|
||||||
|
|
||||||
|
```fstab
|
||||||
|
192.168.10.201:/volume1/video /mnt/nas-tver nfs defaults,_netdev,nofail 0 0
|
||||||
|
```
|
||||||
|
|
||||||
|
設定を反映し、想定したNFS共有がマウントされたことを確認する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo mount -a
|
||||||
|
findmnt --target /mnt/nas-tver
|
||||||
|
```
|
||||||
|
|
||||||
|
実機ではNFSv3でマウントできた。NFS権限を追加した直後の1回目は`access denied`となり、直後の2回目で成功した。NFS権限の反映待ちと考えられるため、設定内容が正しい場合は少し待って`sudo mount -a`を再実行する。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
### 2.2 workerのUIDで書き込みを確認する
|
||||||
|
|
||||||
|
既存の`anime`・`drama`・`misc`・`personal`・`variety`とアプリのファイルを分け、admin相当の書き込み範囲を限定するため、tver専用サブフォルダを使う。Dockerグループの`kazu634`で、workerコンテナと同じUID・GID `65532:65532`の確認用コンテナを実行し、フォルダ作成、書き込み、読み取り、rename、削除を確認する。`sudo`は不要である。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker run --rm --user 65532:65532 -v /mnt/nas-tver/tver:/videos busybox:stable sh -c 'mkdir /videos/.tver-write-test && echo ok > /videos/.tver-write-test/f && cat /videos/.tver-write-test/f && mv /videos/.tver-write-test/f /videos/.tver-write-test/g && ls -ln /videos/.tver-write-test && rm -r /videos/.tver-write-test && echo cleanup-ok'
|
||||||
|
```
|
||||||
|
|
||||||
|
期待する結果は、内容の`ok`、`ls -ln`で表示される`g`の所有者`1024 100`、最後の`cleanup-ok`である。確認後は確認用イメージを削除する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker rmi busybox:stable
|
||||||
|
```
|
||||||
|
|
||||||
|
`/mnt/nas-tver/tver`は作成済みで、上記のDockerコンテナによる操作は実機確認済みである。[実機] 2026-10-04 確認。別環境へ作り直す場合も、書き込み確認が成功するまでworkerを起動しない。
|
||||||
|
|
||||||
|
### 2.3 Giteaに配置用ユーザーを用意する
|
||||||
|
|
||||||
|
Giteaは`REQUIRE_SIGNIN_VIEW = true`かつ`DISABLE_SSH = true`で動作している。匿名HTTPSの`info/refs`は`401`となり、SSHも使えないため、配置専用の読み取り資格情報を使う。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
1. Giteaの管理画面で、配置専用ユーザー`tver-deploy`を作成する。
|
||||||
|
2. `kazu634/tver`リポジトリの「設定」→「共同作業者」で`tver-deploy`を追加し、権限を「読み取り」にする。
|
||||||
|
3. `tver-deploy`でGiteaへログインし、ユーザー設定の「アプリケーション」からrepository読み取り権限だけを持つトークンを作成する。
|
||||||
|
4. トークンは作成時に安全な場所へ控え、この文書、シェル履歴、Git管理下のファイルへ実値を書かない。
|
||||||
|
|
||||||
|
### 2.4 dockageにclone用トークンを保存する
|
||||||
|
|
||||||
|
Dockerホストの管理ユーザー`kazu634`で次を実行する。トークンは非表示で読み取り、権限`600`の専用credentialファイルへ保存する。credential helperはGiteaのHTTPS URLにだけ設定する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
install -d -m 700 ~/.config/git
|
||||||
|
read -rs -p 'Gitea token: ' TOKEN; echo
|
||||||
|
umask 077
|
||||||
|
printf 'https://tver-deploy:%s@gitea.kazu634.com\n' "$TOKEN" > ~/.config/git/credentials-tver
|
||||||
|
unset TOKEN
|
||||||
|
git config --global credential.https://gitea.kazu634.com.helper 'store --file ~/.config/git/credentials-tver'
|
||||||
|
git clone https://tver-deploy@gitea.kazu634.com/kazu634/tver.git /opt/stacks/tver
|
||||||
|
cd /opt/stacks/tver
|
||||||
|
```
|
||||||
|
|
||||||
|
この手順で認証付きcloneに成功した。[実機] 2026-10-04 確認。トークンを失効・更新した場合は、同じ手順で`credentials-tver`を作り直す。
|
||||||
|
|
||||||
|
### 2.5 アプリの環境変数を用意する
|
||||||
|
|
||||||
|
`.env.example`をGit管理外の`.env`へコピーし、`DATABASE_URL`へパスワード用の伏字を入れる。ファイルを`600`にしてから、パスワードを非表示で読み取り、伏字だけを置換する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/stacks/tver
|
||||||
|
umask 077
|
||||||
|
cp .env.example .env
|
||||||
|
sed -i 's|^DATABASE_URL=.*|DATABASE_URL=postgres://tver:__SET_PASSWORD__@192.168.10.201:15434/tver?sslmode=disable|' .env
|
||||||
|
chmod 600 .env
|
||||||
|
read -rs -p 'PostgreSQL password: ' DB_PASSWORD; echo
|
||||||
|
sed -i "s|__SET_PASSWORD__|${DB_PASSWORD}|" .env
|
||||||
|
unset DB_PASSWORD
|
||||||
|
if grep -q '__SET_PASSWORD__' .env; then echo 'DATABASE_URLのパスワードが未設定です' >&2; exit 1; fi
|
||||||
|
```
|
||||||
|
|
||||||
|
完成した`.env`は次の項目を持つ。`DATABASE_URL`の実値は表示したり、文書へ転記したりしない。
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
DATABASE_URL=<設定済み・表示しない>
|
||||||
|
VIDEO_HOST_DIR=/mnt/nas-tver/tver
|
||||||
|
VIDEO_DIR=/videos
|
||||||
|
WORKER_CONCURRENCY=1
|
||||||
|
```
|
||||||
|
|
||||||
|
> [!warning] パスワードの文字種
|
||||||
|
> この手順で使うPostgreSQLパスワードには、URLや`sed`の区切りに影響する`@`・`:`・`/`・`?`・`#`・`%`・`|`を含めない。NAS準備時に16進文字列で生成すれば、この制約を満たせる。
|
||||||
|
|
||||||
|
### 2.6 起動する
|
||||||
|
|
||||||
|
`/opt/stacks/tver`でwebとworkerをビルドして起動する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/stacks/tver
|
||||||
|
docker compose up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
この方法で起動すると、Dockgeの画面にも`tver`スタックとして表示された。[実機] 2026-10-04 確認。Dockge上でweb・workerがともにrunningになり、再起動を繰り返していないことを確認する。
|
||||||
|
|
||||||
|
コマンドでも状態とログを確認できる。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/stacks/tver
|
||||||
|
docker compose ps
|
||||||
|
docker compose logs --tail=100 web worker
|
||||||
|
```
|
||||||
|
|
||||||
|
## 3. 動作を確認する
|
||||||
|
|
||||||
|
1. ヘルスチェックが`200`と`ok`を返すことを確認する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl --fail --show-error http://192.168.10.155:8080/healthz
|
||||||
|
```
|
||||||
|
|
||||||
|
2. ブラウザで`http://192.168.10.155:8080/`を開き、新着一覧が表示されることを確認する。
|
||||||
|
3. 番組またはエピソードを1件登録し、`/queue`で待ち行列、進捗、完了を確認する。
|
||||||
|
4. `/library`から動画を再生し、シークできることを確認する。
|
||||||
|
5. NASの`/volume1/video/tver`に規則どおりのMP4が作成されたことを確認する。
|
||||||
|
6. `tver-postgres`のログとweb・workerのログに、接続失敗やマイグレーション失敗がないことを確認する。
|
||||||
|
|
||||||
|
> [!success] 本番の動作確認結果
|
||||||
|
> `/healthz`はHTTP `200`、新着バラエティは98件を表示した。4分の本編`ep5hcgopp2`を登録すると20秒以内に完了し、NASの`video/tver/<番組名>/`へ92,340,468バイトの1080p H.264/AAC MP4が保存された。所有者はUID `1024`・GID `100`、`.incoming`は空だった。`/library`の表示・再生とHTTP Range `206`も確認した。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
> [!note] Synologyの`@eaDir`
|
||||||
|
> NASが番組フォルダ内に所有者UID `0`の`@eaDir`を作る。アプリのライブラリ一覧はファイルシステムの走査ではなくDBから作るため、表示や再生には影響しない。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
## 4. 更新する
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/stacks/tver
|
||||||
|
git status --short
|
||||||
|
git pull --ff-only
|
||||||
|
```
|
||||||
|
|
||||||
|
`git status --short`に追跡対象ファイルの変更がある場合は、内容を確認してから更新する。pull後、Dockgeで`tver`スタックを再ビルドして再配置し、[[#3. 動作を確認する]]を繰り返す。
|
||||||
|
|
||||||
|
yt-dlpを更新するときは`Dockerfile.worker`の`YT_DLP_VERSION`の既定値を変更してリポジトリへ反映し、workerイメージをキャッシュなしで再ビルドする。更新後はworkerログと少数の実動画で抽出・保存を確認する。
|
||||||
|
|
||||||
|
## 5. PostgreSQLをバックアップする
|
||||||
|
|
||||||
|
バックアップはNAS上のDSMタスクスケジューラから毎日実行する。`tver-postgres`コンテナ内の`pg_dump`を使って`/volume1/backups/tver/tver_<YYYYmmdd_HHMMSS>.sql.gz`へ保存し、既定では14日より古い世代を削除する。
|
||||||
|
|
||||||
|
> [!warning] バックアップ先も同じNASにある
|
||||||
|
> このバックアップはDBの誤操作や破損からの復元用であり、NAS本体の故障には備えられない。`/volume1/backups/tver/`はSynology Hyper Backupの対象にも含める。
|
||||||
|
|
||||||
|
### 5.1 DSMの日次タスクを登録する
|
||||||
|
|
||||||
|
DSMの「コントロール パネル」→「タスク スケジューラ」で「スケジュールされたタスク」→「ユーザー定義のスクリプト」を作成する。
|
||||||
|
|
||||||
|
| 設定 | 値 |
|
||||||
|
|---|---|
|
||||||
|
| ユーザー | `root` |
|
||||||
|
| 実行周期 | 毎日 |
|
||||||
|
| ユーザー定義のスクリプト | `bash /volume1/docker/tver-db/backup-db.sh` |
|
||||||
|
|
||||||
|
`DOCKER`、`CONTAINER`、`BACKUP_DIR`、`KEEP_DAYS`、`DB_USER`、`DB_NAME`は環境変数で上書きできる。通常運用では既定値を使う。
|
||||||
|
|
||||||
|
> [!success] NASで手動実行を確認済み
|
||||||
|
> DSMタスクスケジューラへrootの日次タスクを登録し、手動実行で2つのgzipダンプを作成した。復元確認も成功している。翌日以降の日次自動実行は未確認である。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
> [!todo] 日次自動実行を確認する
|
||||||
|
> 翌日以降にDSMタスクの実行履歴と新しいダンプを確認する。[要確認]
|
||||||
|
|
||||||
|
### 5.2 手動でバックアップを確認する
|
||||||
|
|
||||||
|
NASへSSH接続し、初回は手動で実行する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo bash /volume1/docker/tver-db/backup-db.sh
|
||||||
|
LATEST=$(sudo find /volume1/backups/tver -name 'tver_*.sql.gz' -type f -print | sort | tail -n 1)
|
||||||
|
test -n "$LATEST"
|
||||||
|
sudo test -s "$LATEST"
|
||||||
|
sudo gzip -t "$LATEST"
|
||||||
|
sudo ls -lh "$LATEST"
|
||||||
|
```
|
||||||
|
|
||||||
|
スクリプトが`OK:`、保存先、バイト数を表示し、`gzip -t`が成功することを確認する。`pg_dump`の失敗や空のダンプでは非0で終了し、完成ファイルを残さない。同じ秒の完成ファイルが既にある場合も非0で終了し、既存ファイルを上書きしない。同時実行時は同一ディレクトリ内のハードリンク作成に成功した一方だけが完成ファイルを公開する。
|
||||||
|
|
||||||
|
> [!success] NASでのバックアップ結果
|
||||||
|
> DSMタスクスケジューラから実行し、`tver_20261004_182838.sql.gz`（2,286バイト）と`tver_20261004_183256.sql.gz`（2,289バイト）を`/volume1/backups/tver/`へ作成した。ダンプはroot所有・権限600で、バックアップディレクトリはroot所有・`drwxr-xr-x`だった。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
### 5.3 別DBへの復元を確認する
|
||||||
|
|
||||||
|
最新ダンプを本番DBへ戻す前に、同じコンテナ内の一時DB`tver_restore_test`へ復元する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo bash /volume1/docker/tver-db/restore-test.sh
|
||||||
|
sudo tail -n 100 /volume1/backups/tver/restore-test.log
|
||||||
|
```
|
||||||
|
|
||||||
|
`restore-test.sh`はrootで実行する。バックアップディレクトリとログがroot所有のため、一般ユーザーでは`restore-test.log`への書き込みに失敗する。`download_jobs`、`subscriptions`、`known_episodes`、`schema_migrations`の行数が本番DBと一致し、最後に「復元確認に成功」と表示されることを確認する。復元SQLの通常出力は抑制され、ログには開始、各表の結果、成功または失敗が残る。SQLエラーは標準エラーとしてログに残る。一時DBは成功・失敗のどちらでもEXIT trapで削除される。本番DBに対しては行数取得の`SELECT`だけを実行する。
|
||||||
|
|
||||||
|
> [!success] NASでの復元確認結果
|
||||||
|
> 一般ユーザー`kazu634`での実行は`restore-test.log`へ書き込めず、`touch: Permission denied`で終了した。`sudo`で実行すると、`download_jobs` 1件、`subscriptions` 1件、`known_episodes` 1件、`schema_migrations` 3件が本番DBと一時DBで一致し、「復元確認に成功」と表示された。同じ結果がログに残り、一時DBも削除された。[実機] 2026-10-04 確認
|
||||||
|
|
||||||
|
> [!note] 配置ディレクトリの権限変更後の状態
|
||||||
|
> `/volume1/docker/tver-db`を`chmod 755`にした後も、本番アプリの`/queue`と`/subscriptions`はHTTP `200`を返し、DBを読み取れた。PostgreSQLコンテナを再起動した後の動作は未確認である。[要確認]
|
||||||
|
|
||||||
|
### 5.4 障害時に本番DBへ復元する
|
||||||
|
|
||||||
|
> [!danger] 本番DBを置き換える操作
|
||||||
|
> 先に[[#5.3 別DBへの復元を確認する]]を成功させ、復元対象のダンプを明示してから実行する。`tver`以外のDB名へ読み替えない。
|
||||||
|
|
||||||
|
1. dockageの`/opt/stacks/tver`でwebとworkerを停止し、DBへの書き込みを止める。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/stacks/tver
|
||||||
|
docker compose stop web worker
|
||||||
|
```
|
||||||
|
|
||||||
|
2. NASで復元対象を指定し、存在・サイズ・gzip形式を確認する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
DUMP=/volume1/backups/tver/tver_<YYYYmmdd_HHMMSS>.sql.gz
|
||||||
|
test -s "$DUMP"
|
||||||
|
gzip -t "$DUMP"
|
||||||
|
```
|
||||||
|
|
||||||
|
3. NASで本番DBを作り直し、ダンプを復元する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec tver-postgres dropdb --if-exists -U tver tver
|
||||||
|
docker exec tver-postgres createdb -U tver tver
|
||||||
|
gzip -dc "$DUMP" | docker exec -i tver-postgres psql -v ON_ERROR_STOP=1 -U tver -d tver
|
||||||
|
```
|
||||||
|
|
||||||
|
4. dockageでwebとworkerを再開し、[[#3. 動作を確認する]]を実施する。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /opt/stacks/tver
|
||||||
|
docker compose start web worker
|
||||||
|
```
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
module gitea.kazu634.com/kazu634/tver
|
||||||
|
|
||||||
|
go 1.27.1
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/fergusstrange/embedded-postgres v1.34.0
|
||||||
|
github.com/jackc/pgx/v5 v5.11.0
|
||||||
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||||
|
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
|
||||||
|
github.com/jackc/puddle/v2 v2.2.2 // indirect
|
||||||
|
github.com/lib/pq v1.10.9 // indirect
|
||||||
|
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
|
||||||
|
golang.org/x/sync v0.17.0 // indirect
|
||||||
|
golang.org/x/text v0.29.0 // indirect
|
||||||
|
)
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/fergusstrange/embedded-postgres v1.34.0 h1:c6RKhPKFsLVU+Tdxsx8q0UxCHsvZZ/iShAnljRBXs6s=
|
||||||
|
github.com/fergusstrange/embedded-postgres v1.34.0/go.mod h1:w0YvnCgf19o6tskInrOOACtnqfVlOvluz3hlNLY7tRk=
|
||||||
|
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
|
||||||
|
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
|
||||||
|
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
|
||||||
|
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
|
||||||
|
github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg=
|
||||||
|
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
|
||||||
|
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||||
|
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||||
|
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||||
|
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||||
|
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||||
|
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||||
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
|
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||||
|
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
|
||||||
|
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
|
||||||
|
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
||||||
|
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
||||||
|
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
|
||||||
|
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
|
||||||
|
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
|
||||||
|
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"unicode"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
maxSeriesBytes = 90
|
||||||
|
maxEpisodeBytes = 140
|
||||||
|
maxEpisodeIDBytes = 64
|
||||||
|
maxFilenameBytes = 255
|
||||||
|
)
|
||||||
|
|
||||||
|
// RelativeVideoPath は worker と web が共有する保存先の相対パスを返す。
|
||||||
|
func RelativeVideoPath(seriesTitle, episodeTitle, episodeID string) (string, error) {
|
||||||
|
series := sanitizeComponent(seriesTitle, maxSeriesBytes)
|
||||||
|
episode := sanitizeComponent(episodeTitle, maxEpisodeBytes)
|
||||||
|
id := sanitizeComponent(episodeID, -1)
|
||||||
|
if len(id) > maxEpisodeIDBytes {
|
||||||
|
return "", fmt.Errorf("エピソードIDはサニタイズ後%dバイト以下である必要があります: %dバイト", maxEpisodeIDBytes, len(id))
|
||||||
|
}
|
||||||
|
|
||||||
|
// A directory entry must fit within the NFS namlen limit. Keep the episode
|
||||||
|
// ID intact and use the remaining bytes for the episode title.
|
||||||
|
const separators = " [].mp4"
|
||||||
|
seriesInFilename := series
|
||||||
|
titleBytes := maxFilenameBytes - len(seriesInFilename) - len(id) - len(separators)
|
||||||
|
if titleBytes < 0 {
|
||||||
|
titleBytes = 0
|
||||||
|
}
|
||||||
|
episode = truncateComponent(episode, titleBytes, false)
|
||||||
|
return filepath.Join(series, seriesInFilename+" "+episode+" ["+id+"].mp4"), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sanitizeComponent(value string, maxBytes int) string {
|
||||||
|
var result []rune
|
||||||
|
lastReplacement := false
|
||||||
|
for _, r := range value {
|
||||||
|
invalid := unicode.IsControl(r) || strings.ContainsRune(`<>:"/\|?*`, r)
|
||||||
|
if invalid {
|
||||||
|
if !lastReplacement {
|
||||||
|
result = append(result, '_')
|
||||||
|
lastReplacement = true
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if unicode.IsSpace(r) {
|
||||||
|
r = ' '
|
||||||
|
}
|
||||||
|
result = append(result, r)
|
||||||
|
lastReplacement = false
|
||||||
|
}
|
||||||
|
clean := strings.Trim(string(result), " .")
|
||||||
|
if clean == "" || clean == "." || clean == ".." {
|
||||||
|
return "名称未設定"
|
||||||
|
}
|
||||||
|
return truncateComponent(clean, maxBytes, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
func truncateComponent(value string, maxBytes int, fallback bool) string {
|
||||||
|
if maxBytes < 0 {
|
||||||
|
return value
|
||||||
|
}
|
||||||
|
var limited []rune
|
||||||
|
bytes := 0
|
||||||
|
for _, r := range value {
|
||||||
|
size := len(string(r))
|
||||||
|
if bytes+size > maxBytes {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
limited = append(limited, r)
|
||||||
|
bytes += size
|
||||||
|
}
|
||||||
|
clean := strings.Trim(string(limited), " .")
|
||||||
|
if fallback && (clean == "" || clean == "." || clean == "..") {
|
||||||
|
return "名称未設定"
|
||||||
|
}
|
||||||
|
return clean
|
||||||
|
}
|
||||||
@@ -0,0 +1,267 @@
|
|||||||
|
package media
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"unicode"
|
||||||
|
"unicode/utf8"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRelativeVideoPath(t *testing.T) {
|
||||||
|
got, err := RelativeVideoPath("番組名", "第1話", "ep123")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
want := filepath.Join("番組名", "番組名 第1話 [ep123].mp4")
|
||||||
|
if got != want {
|
||||||
|
t.Fatalf("path=%q want=%q", got, want)
|
||||||
|
}
|
||||||
|
other, err := RelativeVideoPath("番組名", "第1話", "ep456")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got == other || !strings.Contains(other, "[ep456]") {
|
||||||
|
t.Fatalf("ID による衝突回避: %q %q", got, other)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelativeVideoPathSanitizesComponents(t *testing.T) {
|
||||||
|
got, err := RelativeVideoPath(" 番組:/\\*?<>|\"\x00\n\t\x7f\u0085. ", "\x00\n", "../")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if filepath.IsAbs(got) || strings.ContainsAny(got, `:*?<>|"`) || strings.Contains(got, "..") {
|
||||||
|
t.Fatalf("禁止文字が残った: %q", got)
|
||||||
|
}
|
||||||
|
for _, r := range got {
|
||||||
|
if unicode.IsControl(r) {
|
||||||
|
t.Errorf("制御文字が残った: %U in %q", r, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
parts := strings.Split(filepath.ToSlash(got), "/")
|
||||||
|
if len(parts) != 2 || parts[0] == "" || !strings.HasSuffix(got, ".mp4") {
|
||||||
|
t.Fatalf("相対パスの形: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelativeVideoPathHandlesEmptyAndLongNames(t *testing.T) {
|
||||||
|
for _, name := range []string{"", " ", ".", "..", "..."} {
|
||||||
|
got, err := RelativeVideoPath(name, name, "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(got, "名称未設定") {
|
||||||
|
t.Errorf("空の代替名 %q: %q", name, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
long := strings.Repeat("界", 200)
|
||||||
|
got, err := RelativeVideoPath(long, long, "id")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
parts := strings.Split(filepath.ToSlash(got), "/")
|
||||||
|
if len(parts) != 2 || len([]byte(parts[0])) != maxSeriesBytes || !utf8.ValidString(parts[0]) {
|
||||||
|
t.Fatalf("番組名の%dバイト境界: %q (%d bytes)", maxSeriesBytes, parts[0], len([]byte(parts[0])))
|
||||||
|
}
|
||||||
|
if len([]byte(parts[1])) > maxFilenameBytes || !utf8.ValidString(parts[1]) {
|
||||||
|
t.Fatalf("ファイル名の上限: %q (%d bytes)", parts[1], len([]byte(parts[1])))
|
||||||
|
}
|
||||||
|
if got := sanitizeComponent(strings.Repeat("a", maxEpisodeBytes+1), maxEpisodeBytes); len([]byte(got)) != maxEpisodeBytes {
|
||||||
|
t.Fatalf("ASCII の%dバイト境界: %d %q", maxEpisodeBytes, len([]byte(got)), got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelativeVideoPathEpisodeIDLimit(t *testing.T) {
|
||||||
|
id := strings.Repeat("i", maxEpisodeIDBytes)
|
||||||
|
got, err := RelativeVideoPath(strings.Repeat("s", maxSeriesBytes), strings.Repeat("t", maxEpisodeBytes), id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
parts := strings.Split(filepath.ToSlash(got), "/")
|
||||||
|
if len(parts) != 2 || len([]byte(parts[1])) > maxFilenameBytes || !strings.Contains(parts[1], "["+id+"]") {
|
||||||
|
t.Fatalf("64バイトIDのファイル名=%q", got)
|
||||||
|
}
|
||||||
|
if _, err := RelativeVideoPath("series", "episode", strings.Repeat("i", maxEpisodeIDBytes+1)); err == nil {
|
||||||
|
t.Fatal("65バイトのエピソードIDを受理した")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelativeVideoPathUsesFilenameBudgetInRequiredOrder(t *testing.T) {
|
||||||
|
series := strings.Repeat("s", maxSeriesBytes)
|
||||||
|
title := strings.Repeat("t", maxEpisodeBytes)
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name string
|
||||||
|
idBytes int
|
||||||
|
wantSeriesInFile int
|
||||||
|
wantEpisodeInFile int
|
||||||
|
wantFilenameBytes int
|
||||||
|
}{
|
||||||
|
{"ちょうど255バイト", 17, 90, 140, 255},
|
||||||
|
{"回タイトルから1バイト削る", 18, 90, 139, 255},
|
||||||
|
{"64バイトIDでも番組名を維持", 64, 90, 93, 255},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
id := strings.Repeat("i", tt.idBytes)
|
||||||
|
path, err := RelativeVideoPath(series, title, id)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
parts := strings.Split(filepath.ToSlash(path), "/")
|
||||||
|
if len(parts) != 2 || parts[0] != series {
|
||||||
|
t.Fatalf("ディレクトリ名=%q", parts)
|
||||||
|
}
|
||||||
|
wantFilename := strings.Repeat("s", tt.wantSeriesInFile) + " " + strings.Repeat("t", tt.wantEpisodeInFile) + " [" + id + "].mp4"
|
||||||
|
if parts[1] != wantFilename || len([]byte(parts[1])) != tt.wantFilenameBytes {
|
||||||
|
t.Fatalf("filename=%q (%d bytes) want=%q (%d bytes)", parts[1], len([]byte(parts[1])), wantFilename, tt.wantFilenameBytes)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelativeVideoPathEpisodeIDNearAndBeyondFilenameLimit(t *testing.T) {
|
||||||
|
for _, idBytes := range []int{64, 65, 248, 255, 256} {
|
||||||
|
t.Run(fmt.Sprintf("id_%d_bytes", idBytes), func(t *testing.T) {
|
||||||
|
id := strings.Repeat("i", idBytes)
|
||||||
|
path, err := RelativeVideoPath("series", "episode", id)
|
||||||
|
if idBytes > 64 {
|
||||||
|
if err == nil || path != "" {
|
||||||
|
t.Fatalf("%dバイトID: path=%q err=%v, want error", idBytes, path, err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(err.Error(), "64バイト以下") {
|
||||||
|
t.Fatalf("%dバイトIDのエラー理由=%q", idBytes, err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
parts := strings.Split(filepath.ToSlash(path), "/")
|
||||||
|
if len(parts) != 2 {
|
||||||
|
t.Fatalf("parts=%q", parts)
|
||||||
|
}
|
||||||
|
if !strings.Contains(parts[1], "["+id+"]") {
|
||||||
|
t.Fatalf("episode ID was changed: %q", parts[1])
|
||||||
|
}
|
||||||
|
if got := len([]byte(parts[1])); got > maxFilenameBytes {
|
||||||
|
t.Fatalf("episode ID %d bytes produced %d-byte filename above %d: %q", idBytes, got, maxFilenameBytes, parts[1])
|
||||||
|
}
|
||||||
|
if !utf8.ValidString(parts[1]) {
|
||||||
|
t.Fatalf("filename is not valid UTF-8: %q", parts[1])
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRelativeVideoPathHasNoRuneLimit(t *testing.T) {
|
||||||
|
series := strings.Repeat("a", 90)
|
||||||
|
title := strings.Repeat("b", 100)
|
||||||
|
got, err := RelativeVideoPath(series, title, "id")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
parts := strings.Split(filepath.ToSlash(got), "/")
|
||||||
|
if parts[0] != series || !strings.Contains(parts[1], " "+title+" [id].mp4") {
|
||||||
|
t.Fatalf("80 rune limit remains: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSanitizeComponentTrimsWhitespaceBeforeLengthLimit(t *testing.T) {
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name, input, want string
|
||||||
|
}{
|
||||||
|
{"前後の空白", " \t 番組名\n ", "_ 番組名_"},
|
||||||
|
{"空白のみ", " ", "名称未設定"},
|
||||||
|
{"制御文字のみ", "\t\n", "_"},
|
||||||
|
{"多数の先頭空白", strings.Repeat(" ", 100) + "番組名", "番組名"},
|
||||||
|
{"先頭空白と長い名前", strings.Repeat(" ", 70) + strings.Repeat("a", 100), strings.Repeat("a", maxSeriesBytes)},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
if got := sanitizeComponent(tt.input, maxSeriesBytes); got != tt.want {
|
||||||
|
t.Errorf("sanitizeComponent()=%q want=%q", got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSanitizeComponentTrimsAgainAfterLengthLimit(t *testing.T) {
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name, input, want string
|
||||||
|
}{
|
||||||
|
{"90バイト目が空白", strings.Repeat("a", 89) + " " + "b", strings.Repeat("a", 89)},
|
||||||
|
{"90バイト目がドット", strings.Repeat("a", 89) + "." + "b", strings.Repeat("a", 89)},
|
||||||
|
{"切り詰め部分が空白のみ", "a" + strings.Repeat(" ", 89) + "b", "a"},
|
||||||
|
{"切り詰め部分がドットのみ", "a" + strings.Repeat(".", 89) + "b", "a"},
|
||||||
|
{"複数バイト文字の直後がドット", strings.Repeat("界", 29) + ".続", strings.Repeat("界", 29)},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
got := sanitizeComponent(tt.input, maxSeriesBytes)
|
||||||
|
if got != tt.want {
|
||||||
|
t.Errorf("sanitizeComponent()=%q want=%q", got, tt.want)
|
||||||
|
}
|
||||||
|
if strings.Trim(got, " .") != got || got == "" {
|
||||||
|
t.Errorf("切り詰め後の名前が不正: %q", got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSanitizeComponentSafetyProperties(t *testing.T) {
|
||||||
|
var controls strings.Builder
|
||||||
|
for r := rune(0); r <= 0x9f; r++ {
|
||||||
|
if unicode.IsControl(r) {
|
||||||
|
controls.WriteRune(r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
inputs := []string{
|
||||||
|
"", " ", ".", "..", "...", `<>:"/\|?*`, controls.String(),
|
||||||
|
" \t 番組名\n. ", "a/b\\c", strings.Repeat("a", 500),
|
||||||
|
strings.Repeat("界", 500), strings.Repeat("😀", 500),
|
||||||
|
strings.Repeat(".", 100) + "name" + strings.Repeat(" ", 100),
|
||||||
|
string([]byte{0xff, 0xfe, 'a', 0xc0, 0xaf}),
|
||||||
|
}
|
||||||
|
for i, input := range inputs {
|
||||||
|
got := sanitizeComponent(input, maxEpisodeBytes)
|
||||||
|
if got == "" || got == "." || got == ".." {
|
||||||
|
t.Errorf("case %d: 空またはdot segment: %q", i, got)
|
||||||
|
}
|
||||||
|
if strings.Trim(got, " .") != got {
|
||||||
|
t.Errorf("case %d: 前後に空白またはdot: %q", i, got)
|
||||||
|
}
|
||||||
|
if len([]byte(got)) > 140 {
|
||||||
|
t.Errorf("case %d: 上限超過: bytes=%d %q", i, len([]byte(got)), got)
|
||||||
|
}
|
||||||
|
if !utf8.ValidString(got) {
|
||||||
|
t.Errorf("case %d: 不正なUTF-8: %q", i, got)
|
||||||
|
}
|
||||||
|
if strings.ContainsAny(got, `<>:"/\|?*`) {
|
||||||
|
t.Errorf("case %d: 禁止文字が残った: %q", i, got)
|
||||||
|
}
|
||||||
|
for _, r := range got {
|
||||||
|
if unicode.IsControl(r) {
|
||||||
|
t.Errorf("case %d: 制御文字が残った: %U in %q", i, r, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if again := sanitizeComponent(got, maxEpisodeBytes); again != got {
|
||||||
|
t.Errorf("case %d: 再適用で変化: %q -> %q", i, got, again)
|
||||||
|
}
|
||||||
|
|
||||||
|
path, err := RelativeVideoPath(input, input, "id")
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("case %d: パス生成エラー: %v", i, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
parts := strings.Split(filepath.ToSlash(path), "/")
|
||||||
|
if filepath.IsAbs(path) || len(parts) != 2 || parts[0] == "" || parts[0] == "." || parts[0] == ".." || !strings.HasSuffix(parts[1], ".mp4") {
|
||||||
|
t.Errorf("case %d: 安全でない相対パス: %q parts=%q", i, path, parts)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len([]byte(parts[0])) > 90 || len([]byte(parts[1])) > 255 {
|
||||||
|
t.Errorf("case %d: パス要素の上限超過: dir=%d filename=%d path=%q", i, len([]byte(parts[0])), len([]byte(parts[1])), path)
|
||||||
|
}
|
||||||
|
if !utf8.ValidString(path) {
|
||||||
|
t.Errorf("case %d: パスが不正なUTF-8: %q", i, path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,205 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
func completeTestJob(t *testing.T, s *Store, id string) Job {
|
||||||
|
t.Helper()
|
||||||
|
ctx := context.Background()
|
||||||
|
job, _, err := s.Enqueue(ctx, sampleEpisode(id), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.ID != job.ID {
|
||||||
|
t.Fatalf("claim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := s.Complete(ctx, job.ID, "番組/video.mp4", 1234); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
result, err := s.GetSucceededJob(ctx, job.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibrarySeriesAggregationAndJobs(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
complete := func(ep Episode, size int64, finished time.Time) Job {
|
||||||
|
t.Helper()
|
||||||
|
job, _, err := s.Enqueue(ctx, ep, "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.ID != job.ID {
|
||||||
|
t.Fatalf("claim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := s.Complete(ctx, job.ID, ep.SeriesTitle+"/video.mp4", size); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.pool.Exec(ctx, `UPDATE download_jobs SET finished_at=$2 WHERE id=$1`, job.ID, finished); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
result, err := s.GetSucceededJob(ctx, job.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
older := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
newer := older.Add(time.Hour)
|
||||||
|
newest := newer.Add(time.Hour)
|
||||||
|
first := complete(Episode{ID: "aggregate-a1", SeriesID: "seriesA", SeriesTitle: "旧番組名", Title: "第1話", BroadcasterName: "旧局", ThumbnailPath: "/old.jpg"}, 100, older)
|
||||||
|
second := complete(Episode{ID: "aggregate-a2", SeriesID: "seriesA", SeriesTitle: "新番組名", Title: "第2話", BroadcasterName: "新局", ThumbnailPath: "/new.jpg"}, 250, newer)
|
||||||
|
complete(Episode{ID: "aggregate-b1", SeriesID: "seriesB", SeriesTitle: "別番組", Title: "第1話", BroadcasterName: "別局", ThumbnailPath: "/other.jpg"}, 500, newest)
|
||||||
|
if _, err := s.pool.Exec(ctx, `UPDATE download_jobs SET file_size_bytes=NULL WHERE id=$1`, first.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title) VALUES ('seriesA', '登録名')`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
series, err := s.ListLibrarySeries(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(series) != 2 || series[0].SeriesID != "seriesB" || series[1].SeriesID != "seriesA" {
|
||||||
|
t.Fatalf("番組の並び: %+v", series)
|
||||||
|
}
|
||||||
|
got := series[1]
|
||||||
|
if got.SeriesTitle != "新番組名" || got.LatestTitle != "第2話" || got.BroadcasterName != "新局" || got.ThumbnailPath != "/new.jpg" || got.EpisodeCount != 2 || got.TotalSizeBytes != 250 || got.LatestFinishedAt == nil || !got.LatestFinishedAt.Equal(newer) || !got.Subscribed {
|
||||||
|
t.Fatalf("番組集計: %+v", got)
|
||||||
|
}
|
||||||
|
if series[0].Subscribed || series[0].TotalSizeBytes != 500 {
|
||||||
|
t.Fatalf("未登録番組: %+v", series[0])
|
||||||
|
}
|
||||||
|
jobs, err := s.ListSucceededJobsBySeries(ctx, "seriesA")
|
||||||
|
if err != nil || len(jobs) != 2 || jobs[0].ID != second.ID || jobs[1].ID != first.ID || jobs[1].FileSizeBytes != nil {
|
||||||
|
t.Fatalf("番組の回: %+v %v", jobs, err)
|
||||||
|
}
|
||||||
|
if jobs, err := s.ListSucceededJobsBySeries(ctx, "missing"); err != nil || len(jobs) != 0 {
|
||||||
|
t.Fatalf("存在しない番組: %+v %v", jobs, err)
|
||||||
|
}
|
||||||
|
if _, err := s.ListSucceededJobsBySeries(ctx, ""); err == nil {
|
||||||
|
t.Fatal("空の番組IDを許した")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibraryStoreOperations(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
first := completeTestJob(t, s, "library-1")
|
||||||
|
second := completeTestJob(t, s, "library-2")
|
||||||
|
jobs, err := s.ListSucceededJobs(ctx, 1)
|
||||||
|
if err != nil || len(jobs) != 1 || jobs[0].ID != second.ID {
|
||||||
|
t.Fatalf("完了一覧: %+v %v", jobs, err)
|
||||||
|
}
|
||||||
|
if first.FilePath == nil || *first.FilePath != "番組/video.mp4" || first.FileSizeBytes == nil || *first.FileSizeBytes != 1234 || first.FinishedAt == nil {
|
||||||
|
t.Fatalf("完了情報: %+v", first)
|
||||||
|
}
|
||||||
|
queued, _, err := s.Enqueue(ctx, sampleEpisode("not-finished"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.GetSucceededJob(ctx, queued.ID); !errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
t.Fatalf("未完了の取得: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.DeleteSucceededJob(ctx, queued.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("未完了の削除: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.DeleteSucceededJob(ctx, first.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.GetJob(ctx, first.ID); !errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
t.Fatalf("削除後の取得: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.DeleteSucceededJob(ctx, first.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("再削除: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := s.ListSucceededJobs(ctx, 0); err == nil {
|
||||||
|
t.Fatal("0件の上限を許した")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteSucceededJobKeepsKnownEpisodeAndAllowsManualRequeue(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
ep := seriesEpisode("known-library", "known-series")
|
||||||
|
created, err := s.RegisterSubscription(ctx, "known-series", "番組", []Episode{ep}, false)
|
||||||
|
if err != nil || !created {
|
||||||
|
t.Fatalf("番組登録: %t %v", created, err)
|
||||||
|
}
|
||||||
|
job, created, err := s.Enqueue(ctx, ep, "manual")
|
||||||
|
if err != nil || !created {
|
||||||
|
t.Fatalf("手動投入: %+v %t %v", job, created, err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.ID != job.ID {
|
||||||
|
t.Fatalf("取得: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := s.Complete(ctx, job.ID, "番組/video.mp4", 10); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.DeleteSucceededJob(ctx, job.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
unknown, err := s.UnknownEpisodeIDs(ctx, "known-series", []string{ep.ID})
|
||||||
|
if err != nil || len(unknown) != 0 {
|
||||||
|
t.Fatalf("削除後の既知記録: %v %v", unknown, err)
|
||||||
|
}
|
||||||
|
requeued, created, err := s.Enqueue(ctx, ep, "manual")
|
||||||
|
if err != nil || !created || requeued.Status != "queued" || requeued.Source != "manual" {
|
||||||
|
t.Fatalf("削除後の手動再投入: %+v %t %v", requeued, created, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListSucceededJobsByIDsAndSeriesIDs(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
first := completeTestJob(t, s, "bulk-list-1")
|
||||||
|
|
||||||
|
ep := seriesEpisode("bulk-list-2", "series-2")
|
||||||
|
second, _, err := s.Enqueue(ctx, ep, "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.ID != second.ID {
|
||||||
|
t.Fatalf("claim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := s.Complete(ctx, second.ID, "別番組/video.mp4", 5678); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
second, err = s.GetSucceededJob(ctx, second.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
queued, _, err := s.Enqueue(ctx, seriesEpisode("bulk-list-queued", "series-2"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
byIDs, err := s.ListSucceededJobsByIDs(ctx, []int64{second.ID, queued.ID, first.ID})
|
||||||
|
if err != nil || len(byIDs) != 2 || byIDs[0].ID != first.ID || byIDs[1].ID != second.ID {
|
||||||
|
t.Fatalf("ID指定の完了済み取得: %+v %v", byIDs, err)
|
||||||
|
}
|
||||||
|
bySeries, err := s.ListSucceededJobsBySeriesIDs(ctx, []string{"series-2", "series-1"})
|
||||||
|
if err != nil || len(bySeries) != 2 || bySeries[0].ID != first.ID || bySeries[1].ID != second.ID {
|
||||||
|
t.Fatalf("番組指定の完了済み取得: %+v %v", bySeries, err)
|
||||||
|
}
|
||||||
|
if empty, err := s.ListSucceededJobsByIDs(ctx, nil); err != nil || len(empty) != 0 {
|
||||||
|
t.Fatalf("空のID指定: %+v %v", empty, err)
|
||||||
|
}
|
||||||
|
if empty, err := s.ListSucceededJobsBySeriesIDs(ctx, []string{}); err != nil || len(empty) != 0 {
|
||||||
|
t.Fatalf("空の番組指定: %+v %v", empty, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"embed"
|
||||||
|
"fmt"
|
||||||
|
"io/fs"
|
||||||
|
"sort"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
)
|
||||||
|
|
||||||
|
//go:embed migrations/*.sql
|
||||||
|
var migrations embed.FS
|
||||||
|
|
||||||
|
const migrationLockID int64 = 0x747665725f646231
|
||||||
|
|
||||||
|
// Migrate は同じ接続で排他ロックを保持し、未適用の SQL を順に実行する。
|
||||||
|
func Migrate(ctx context.Context, pool *pgxpool.Pool) error {
|
||||||
|
conn, err := pool.Acquire(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("migration connection: %w", err)
|
||||||
|
}
|
||||||
|
defer conn.Release()
|
||||||
|
if _, err := conn.Exec(ctx, "SELECT pg_advisory_lock($1)", migrationLockID); err != nil {
|
||||||
|
return fmt.Errorf("migration lock: %w", err)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
// 接続を返す前にロックを解放する。解放に失敗した接続は閉じる。
|
||||||
|
if _, err := conn.Exec(context.Background(), "SELECT pg_advisory_unlock($1)", migrationLockID); err != nil {
|
||||||
|
conn.Conn().Close(context.Background())
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
if _, err := conn.Exec(ctx, `CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||||
|
version TEXT PRIMARY KEY,
|
||||||
|
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||||
|
)`); err != nil {
|
||||||
|
return fmt.Errorf("migration metadata: %w", err)
|
||||||
|
}
|
||||||
|
files, err := fs.Glob(migrations, "migrations/*.sql")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
sort.Strings(files)
|
||||||
|
for _, name := range files {
|
||||||
|
tx, err := conn.Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var applied bool
|
||||||
|
err = tx.QueryRow(ctx, "SELECT EXISTS (SELECT 1 FROM schema_migrations WHERE version=$1)", name).Scan(&applied)
|
||||||
|
if err == nil && !applied {
|
||||||
|
var sql []byte
|
||||||
|
sql, err = migrations.ReadFile(name)
|
||||||
|
if err == nil {
|
||||||
|
_, err = tx.Exec(ctx, string(sql), pgx.QueryExecModeSimpleProtocol)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
_, err = tx.Exec(ctx, "INSERT INTO schema_migrations (version) VALUES ($1)", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
_ = tx.Rollback(ctx)
|
||||||
|
return fmt.Errorf("migration %s: %w", name, err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(ctx); err != nil {
|
||||||
|
return fmt.Errorf("migration %s commit: %w", name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
CREATE TABLE download_jobs (
|
||||||
|
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
|
||||||
|
episode_id TEXT NOT NULL UNIQUE,
|
||||||
|
series_id TEXT NOT NULL,
|
||||||
|
series_title TEXT NOT NULL,
|
||||||
|
title TEXT NOT NULL,
|
||||||
|
broadcaster_name TEXT NOT NULL,
|
||||||
|
duration_seconds INTEGER NOT NULL CHECK (duration_seconds >= 0),
|
||||||
|
end_at TIMESTAMPTZ,
|
||||||
|
thumbnail_path TEXT NOT NULL,
|
||||||
|
source TEXT NOT NULL CHECK (source IN ('manual', 'subscription')),
|
||||||
|
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'succeeded', 'failed', 'canceled')),
|
||||||
|
phase TEXT CHECK (phase IN ('video', 'audio', 'merging', 'moving')),
|
||||||
|
downloaded_bytes BIGINT CHECK (downloaded_bytes >= 0),
|
||||||
|
total_bytes BIGINT CHECK (total_bytes >= 0),
|
||||||
|
total_is_estimate BOOLEAN NOT NULL DEFAULT FALSE,
|
||||||
|
speed_bps DOUBLE PRECISION CHECK (speed_bps >= 0),
|
||||||
|
eta_seconds DOUBLE PRECISION CHECK (eta_seconds >= 0),
|
||||||
|
fragment_index INTEGER CHECK (fragment_index >= 0),
|
||||||
|
fragment_count INTEGER CHECK (fragment_count >= 0),
|
||||||
|
format_id TEXT,
|
||||||
|
error_kind TEXT CHECK (error_kind IN ('drm', 'geo', 'network', 'unavailable', 'extract', 'canceled', 'invalid_output', 'unknown')),
|
||||||
|
error_message TEXT,
|
||||||
|
file_path TEXT,
|
||||||
|
file_size_bytes BIGINT CHECK (file_size_bytes >= 0),
|
||||||
|
attempts INTEGER NOT NULL DEFAULT 0 CHECK (attempts >= 0),
|
||||||
|
heartbeat_at TIMESTAMPTZ,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||||
|
started_at TIMESTAMPTZ,
|
||||||
|
finished_at TIMESTAMPTZ,
|
||||||
|
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX download_jobs_claim_idx ON download_jobs (created_at, id) WHERE status = 'queued';
|
||||||
|
CREATE INDEX download_jobs_status_newest_idx ON download_jobs (status, created_at DESC, id DESC);
|
||||||
|
CREATE INDEX download_jobs_stale_idx ON download_jobs (heartbeat_at) WHERE status = 'running';
|
||||||
|
|
||||||
|
CREATE TABLE subscriptions (
|
||||||
|
series_id TEXT PRIMARY KEY,
|
||||||
|
series_title TEXT NOT NULL,
|
||||||
|
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||||
|
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||||
|
last_checked_at TIMESTAMPTZ
|
||||||
|
);
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
CREATE TABLE known_episodes (
|
||||||
|
series_id TEXT NOT NULL REFERENCES subscriptions(series_id) ON DELETE CASCADE,
|
||||||
|
episode_id TEXT NOT NULL,
|
||||||
|
first_seen_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||||
|
PRIMARY KEY (series_id, episode_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX download_jobs_series_idx ON download_jobs (series_id);
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
ALTER TABLE subscriptions
|
||||||
|
ADD COLUMN last_check_error_at TIMESTAMPTZ,
|
||||||
|
ADD COLUMN last_check_error TEXT;
|
||||||
|
|
||||||
|
CREATE INDEX subscriptions_due_idx
|
||||||
|
ON subscriptions (last_checked_at) WHERE enabled;
|
||||||
@@ -0,0 +1,424 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
)
|
||||||
|
|
||||||
|
var ErrInvalidTransition = errors.New("invalid job status transition")
|
||||||
|
|
||||||
|
type Store struct{ pool *pgxpool.Pool }
|
||||||
|
|
||||||
|
func New(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
|
||||||
|
|
||||||
|
type Episode struct {
|
||||||
|
ID, SeriesID, SeriesTitle, Title, BroadcasterName, ThumbnailPath string
|
||||||
|
DurationSeconds int
|
||||||
|
EndAt *time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type Job struct {
|
||||||
|
ID int64
|
||||||
|
EpisodeID, SeriesID, SeriesTitle string
|
||||||
|
Title, BroadcasterName string
|
||||||
|
DurationSeconds int
|
||||||
|
EndAt *time.Time
|
||||||
|
ThumbnailPath, Source, Status string
|
||||||
|
Phase, FormatID *string
|
||||||
|
DownloadedBytes, TotalBytes *int64
|
||||||
|
TotalIsEstimate bool
|
||||||
|
SpeedBPS, ETASeconds *float64
|
||||||
|
FragmentIndex, FragmentCount *int32
|
||||||
|
ErrorKind, ErrorMessage, FilePath *string
|
||||||
|
FileSizeBytes *int64
|
||||||
|
Attempts int
|
||||||
|
HeartbeatAt, StartedAt, FinishedAt *time.Time
|
||||||
|
CreatedAt, UpdatedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type LibrarySeries struct {
|
||||||
|
SeriesID, SeriesTitle, LatestTitle string
|
||||||
|
BroadcasterName, ThumbnailPath string
|
||||||
|
EpisodeCount, TotalSizeBytes int64
|
||||||
|
LatestFinishedAt *time.Time
|
||||||
|
Subscribed bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type Progress struct {
|
||||||
|
Phase string
|
||||||
|
DownloadedBytes *int64
|
||||||
|
TotalBytes *int64
|
||||||
|
TotalIsEstimate bool
|
||||||
|
SpeedBPS *float64
|
||||||
|
ETASeconds *float64
|
||||||
|
FragmentIndex *int32
|
||||||
|
FragmentCount *int32
|
||||||
|
FormatID *string
|
||||||
|
}
|
||||||
|
|
||||||
|
type Subscription struct {
|
||||||
|
SeriesID, SeriesTitle string
|
||||||
|
Enabled bool
|
||||||
|
CreatedAt time.Time
|
||||||
|
LastCheckedAt *time.Time
|
||||||
|
LastCheckErrorAt *time.Time
|
||||||
|
LastCheckError *string
|
||||||
|
JobCount int64
|
||||||
|
}
|
||||||
|
|
||||||
|
const jobColumns = `id, episode_id, series_id, series_title, title, broadcaster_name,
|
||||||
|
duration_seconds, end_at, thumbnail_path, source, status, phase,
|
||||||
|
downloaded_bytes, total_bytes, total_is_estimate, speed_bps, eta_seconds,
|
||||||
|
fragment_index, fragment_count, format_id, error_kind, error_message,
|
||||||
|
file_path, file_size_bytes, attempts, heartbeat_at, created_at,
|
||||||
|
started_at, finished_at, updated_at`
|
||||||
|
|
||||||
|
var claimedJobColumns = func() string {
|
||||||
|
columns := strings.Split(jobColumns, ",")
|
||||||
|
for i, column := range columns {
|
||||||
|
columns[i] = "j." + strings.TrimSpace(column)
|
||||||
|
}
|
||||||
|
return strings.Join(columns, ", ")
|
||||||
|
}()
|
||||||
|
|
||||||
|
func scanJob(row pgx.Row) (Job, error) {
|
||||||
|
var j Job
|
||||||
|
err := row.Scan(&j.ID, &j.EpisodeID, &j.SeriesID, &j.SeriesTitle, &j.Title,
|
||||||
|
&j.BroadcasterName, &j.DurationSeconds, &j.EndAt, &j.ThumbnailPath,
|
||||||
|
&j.Source, &j.Status, &j.Phase, &j.DownloadedBytes, &j.TotalBytes,
|
||||||
|
&j.TotalIsEstimate, &j.SpeedBPS, &j.ETASeconds, &j.FragmentIndex,
|
||||||
|
&j.FragmentCount, &j.FormatID, &j.ErrorKind, &j.ErrorMessage,
|
||||||
|
&j.FilePath, &j.FileSizeBytes, &j.Attempts, &j.HeartbeatAt,
|
||||||
|
&j.CreatedAt, &j.StartedAt, &j.FinishedAt, &j.UpdatedAt)
|
||||||
|
return j, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Enqueue は同じエピソードの既存ジョブを再利用する。
|
||||||
|
func (s *Store) Enqueue(ctx context.Context, ep Episode, source string) (Job, bool, error) {
|
||||||
|
row := s.pool.QueryRow(ctx, `INSERT INTO download_jobs
|
||||||
|
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9) ON CONFLICT (episode_id) DO NOTHING RETURNING `+jobColumns,
|
||||||
|
ep.ID, ep.SeriesID, ep.SeriesTitle, ep.Title, ep.BroadcasterName,
|
||||||
|
ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath, source)
|
||||||
|
job, err := scanJob(row)
|
||||||
|
if err == nil {
|
||||||
|
return job, true, nil
|
||||||
|
}
|
||||||
|
if !errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
return Job{}, false, err
|
||||||
|
}
|
||||||
|
job, err = scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE episode_id=$1`, ep.ID))
|
||||||
|
return job, false, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) GetJob(ctx context.Context, id int64) (Job, error) {
|
||||||
|
return scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE id=$1`, id))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) GetSucceededJob(ctx context.Context, id int64) (Job, error) {
|
||||||
|
return scanJob(s.pool.QueryRow(ctx, `SELECT `+jobColumns+` FROM download_jobs WHERE id=$1 AND status='succeeded'`, id))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) ListSucceededJobs(ctx context.Context, limit int) ([]Job, error) {
|
||||||
|
if limit <= 0 {
|
||||||
|
return nil, errors.New("limit must be positive")
|
||||||
|
}
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
|
||||||
|
WHERE status='succeeded' ORDER BY finished_at DESC NULLS LAST, id DESC LIMIT $1`, limit)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var jobs []Job
|
||||||
|
for rows.Next() {
|
||||||
|
job, err := scanJob(rows)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
jobs = append(jobs, job)
|
||||||
|
}
|
||||||
|
return jobs, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) ListLibrarySeries(ctx context.Context) ([]LibrarySeries, error) {
|
||||||
|
rows, err := s.pool.Query(ctx, `WITH totals AS (
|
||||||
|
SELECT series_id, count(*) AS episode_count,
|
||||||
|
COALESCE(sum(file_size_bytes), 0) AS total_size_bytes
|
||||||
|
FROM download_jobs WHERE status='succeeded' GROUP BY series_id
|
||||||
|
), latest AS (
|
||||||
|
SELECT DISTINCT ON (series_id) series_id, series_title, title,
|
||||||
|
broadcaster_name, thumbnail_path, finished_at, id
|
||||||
|
FROM download_jobs WHERE status='succeeded'
|
||||||
|
ORDER BY series_id, finished_at DESC NULLS LAST, id DESC
|
||||||
|
)
|
||||||
|
SELECT l.series_id, l.series_title, l.title, l.broadcaster_name,
|
||||||
|
l.thumbnail_path, t.episode_count, t.total_size_bytes, l.finished_at,
|
||||||
|
EXISTS (SELECT 1 FROM subscriptions s WHERE s.series_id=l.series_id)
|
||||||
|
FROM latest l JOIN totals t USING (series_id)
|
||||||
|
ORDER BY l.finished_at DESC NULLS LAST, l.id DESC`)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var result []LibrarySeries
|
||||||
|
for rows.Next() {
|
||||||
|
var item LibrarySeries
|
||||||
|
if err := rows.Scan(&item.SeriesID, &item.SeriesTitle, &item.LatestTitle,
|
||||||
|
&item.BroadcasterName, &item.ThumbnailPath, &item.EpisodeCount,
|
||||||
|
&item.TotalSizeBytes, &item.LatestFinishedAt, &item.Subscribed); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
result = append(result, item)
|
||||||
|
}
|
||||||
|
return result, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) ListSucceededJobsBySeries(ctx context.Context, seriesID string) ([]Job, error) {
|
||||||
|
if seriesID == "" {
|
||||||
|
return nil, errors.New("series ID must not be empty")
|
||||||
|
}
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
|
||||||
|
WHERE status='succeeded' AND series_id=$1
|
||||||
|
ORDER BY finished_at DESC NULLS LAST, id DESC`, seriesID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var jobs []Job
|
||||||
|
for rows.Next() {
|
||||||
|
job, err := scanJob(rows)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
jobs = append(jobs, job)
|
||||||
|
}
|
||||||
|
return jobs, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) ListSucceededJobsByIDs(ctx context.Context, ids []int64) ([]Job, error) {
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return []Job{}, nil
|
||||||
|
}
|
||||||
|
return s.listSucceededJobs(ctx, `id = ANY($1)`, ids)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) ListSucceededJobsBySeriesIDs(ctx context.Context, seriesIDs []string) ([]Job, error) {
|
||||||
|
if len(seriesIDs) == 0 {
|
||||||
|
return []Job{}, nil
|
||||||
|
}
|
||||||
|
return s.listSucceededJobs(ctx, `series_id = ANY($1)`, seriesIDs)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) listSucceededJobs(ctx context.Context, condition string, value any) ([]Job, error) {
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
|
||||||
|
WHERE status='succeeded' AND `+condition+`
|
||||||
|
ORDER BY id`, value)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var jobs []Job
|
||||||
|
for rows.Next() {
|
||||||
|
job, err := scanJob(rows)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
jobs = append(jobs, job)
|
||||||
|
}
|
||||||
|
return jobs, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) DeleteSucceededJob(ctx context.Context, id int64) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `DELETE FROM download_jobs WHERE id=$1 AND status='succeeded'`, id)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Retry は失敗・中断されたジョブの実行時情報を消して再投入する。
|
||||||
|
func (s *Store) Retry(ctx context.Context, id int64) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
|
||||||
|
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
|
||||||
|
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
|
||||||
|
error_kind=NULL, error_message=NULL, heartbeat_at=NULL, started_at=NULL,
|
||||||
|
finished_at=NULL, updated_at=now()
|
||||||
|
WHERE id=$1 AND status IN ('failed','canceled')`, id)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// RetryEpisode はエピソード ID で失敗・中断されたジョブを再投入する。
|
||||||
|
func (s *Store) RetryEpisode(ctx context.Context, episodeID string) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
|
||||||
|
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
|
||||||
|
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
|
||||||
|
error_kind=NULL, error_message=NULL, heartbeat_at=NULL, started_at=NULL,
|
||||||
|
finished_at=NULL, updated_at=now()
|
||||||
|
WHERE episode_id=$1 AND status IN ('failed','canceled')`, episodeID)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ClaimNext は最も古い未実行ジョブを同時 worker と重複せず取得する。
|
||||||
|
func (s *Store) ClaimNext(ctx context.Context) (Job, error) {
|
||||||
|
return scanJob(s.pool.QueryRow(ctx, `WITH next AS (
|
||||||
|
SELECT id FROM download_jobs WHERE status='queued'
|
||||||
|
ORDER BY created_at, id LIMIT 1 FOR UPDATE SKIP LOCKED
|
||||||
|
) UPDATE download_jobs AS j SET status='running', attempts=j.attempts+1,
|
||||||
|
started_at=now(), heartbeat_at=now(), updated_at=now()
|
||||||
|
FROM next WHERE j.id=next.id RETURNING `+claimedJobColumns))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) UpdateProgress(ctx context.Context, id int64, p Progress) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET phase=$2, downloaded_bytes=$3,
|
||||||
|
total_bytes=$4, total_is_estimate=$5, speed_bps=$6, eta_seconds=$7,
|
||||||
|
fragment_index=$8, fragment_count=$9, format_id=$10,
|
||||||
|
heartbeat_at=now(), updated_at=now() WHERE id=$1 AND status='running'`,
|
||||||
|
id, p.Phase, p.DownloadedBytes, p.TotalBytes, p.TotalIsEstimate,
|
||||||
|
p.SpeedBPS, p.ETASeconds, p.FragmentIndex, p.FragmentCount, p.FormatID)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Heartbeat は進捗出力がない間も実行中ジョブの生存を記録する。
|
||||||
|
func (s *Store) Heartbeat(ctx context.Context, id int64) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=now(), updated_at=now()
|
||||||
|
WHERE id=$1 AND status='running'`, id)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) Complete(ctx context.Context, id int64, filePath string, fileSizeBytes int64) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='succeeded', phase=NULL,
|
||||||
|
file_path=$2, file_size_bytes=$3, heartbeat_at=NULL, finished_at=now(), updated_at=now()
|
||||||
|
WHERE id=$1 AND status='running'`, id, filePath, fileSizeBytes)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) Fail(ctx context.Context, id int64, kind, message string) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='failed', phase=NULL,
|
||||||
|
error_kind=$2, error_message=$3, heartbeat_at=NULL, finished_at=now(), updated_at=now()
|
||||||
|
WHERE id=$1 AND status='running'`, id, kind, message)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) Cancel(ctx context.Context, id int64) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='canceled', phase=NULL,
|
||||||
|
error_kind='canceled', heartbeat_at=NULL, finished_at=now(), updated_at=now()
|
||||||
|
WHERE id=$1 AND status IN ('queued','running')`, id)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CancelQueued は worker 未着手のジョブだけを取り消す。
|
||||||
|
func (s *Store) CancelQueued(ctx context.Context, id int64) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='canceled', phase=NULL,
|
||||||
|
error_kind='canceled', heartbeat_at=NULL, finished_at=now(), updated_at=now()
|
||||||
|
WHERE id=$1 AND status='queued'`, id)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// RequeueStale は cutoff より前に heartbeat が止まった実行中ジョブを戻す。
|
||||||
|
func (s *Store) RequeueStale(ctx context.Context, cutoff time.Time) (int64, error) {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
|
||||||
|
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
|
||||||
|
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
|
||||||
|
heartbeat_at=NULL, started_at=NULL, updated_at=now()
|
||||||
|
WHERE status='running' AND heartbeat_at < $1`, cutoff)
|
||||||
|
return tag.RowsAffected(), err
|
||||||
|
}
|
||||||
|
|
||||||
|
// RequeueRunning は worker の正常終了時に担当中ジョブを待機中へ戻す。
|
||||||
|
func (s *Store) RequeueRunning(ctx context.Context, id int64) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE download_jobs SET status='queued', phase=NULL,
|
||||||
|
downloaded_bytes=NULL, total_bytes=NULL, total_is_estimate=false, speed_bps=NULL,
|
||||||
|
eta_seconds=NULL, fragment_index=NULL, fragment_count=NULL, format_id=NULL,
|
||||||
|
heartbeat_at=NULL, started_at=NULL, updated_at=now()
|
||||||
|
WHERE id=$1 AND status='running'`, id)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func transitionResult(count int64, err error) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if count == 0 {
|
||||||
|
return ErrInvalidTransition
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListJobs は指定状態のジョブを新しい順に返す。空の状態は全件を意味する。
|
||||||
|
func (s *Store) ListJobs(ctx context.Context, status string, limit int) ([]Job, error) {
|
||||||
|
if status != "" && status != "queued" && status != "running" && status != "succeeded" && status != "failed" && status != "canceled" {
|
||||||
|
return nil, fmt.Errorf("invalid status: %s", status)
|
||||||
|
}
|
||||||
|
if limit <= 0 {
|
||||||
|
return nil, errors.New("limit must be positive")
|
||||||
|
}
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT `+jobColumns+` FROM download_jobs
|
||||||
|
WHERE ($1='' OR status=$1) ORDER BY created_at DESC, id DESC LIMIT $2`, status, limit)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var jobs []Job
|
||||||
|
for rows.Next() {
|
||||||
|
job, err := scanJob(rows)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
jobs = append(jobs, job)
|
||||||
|
}
|
||||||
|
return jobs, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) EpisodeStatuses(ctx context.Context, ids []string) (map[string]string, error) {
|
||||||
|
result := make(map[string]string)
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT episode_id, status FROM download_jobs WHERE episode_id = ANY($1)`, ids)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
for rows.Next() {
|
||||||
|
var id, status string
|
||||||
|
if err := rows.Scan(&id, &status); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
result[id] = status
|
||||||
|
}
|
||||||
|
return result, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) DeleteSubscription(ctx context.Context, seriesID string) (bool, error) {
|
||||||
|
tag, err := s.pool.Exec(ctx, `DELETE FROM subscriptions WHERE series_id=$1`, seriesID)
|
||||||
|
return tag.RowsAffected() == 1, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) ListSubscriptions(ctx context.Context) ([]Subscription, error) {
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT s.series_id, s.series_title, s.enabled, s.created_at, s.last_checked_at,
|
||||||
|
s.last_check_error_at, s.last_check_error,
|
||||||
|
(SELECT count(*) FROM download_jobs j WHERE j.series_id=s.series_id)
|
||||||
|
FROM subscriptions s ORDER BY s.created_at DESC, s.series_id`)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var subs []Subscription
|
||||||
|
for rows.Next() {
|
||||||
|
var sub Subscription
|
||||||
|
if err := rows.Scan(&sub.SeriesID, &sub.SeriesTitle, &sub.Enabled, &sub.CreatedAt, &sub.LastCheckedAt,
|
||||||
|
&sub.LastCheckErrorAt, &sub.LastCheckError, &sub.JobCount); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
subs = append(subs, sub)
|
||||||
|
}
|
||||||
|
return subs, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) SetSubscriptionChecked(ctx context.Context, seriesID string, checkedAt time.Time) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, seriesID, checkedAt)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
@@ -0,0 +1,766 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
embeddedpostgres "github.com/fergusstrange/embedded-postgres"
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
|
)
|
||||||
|
|
||||||
|
var adminPool *pgxpool.Pool
|
||||||
|
var testPort int
|
||||||
|
var dbSequence atomic.Uint64
|
||||||
|
|
||||||
|
func TestMain(m *testing.M) {
|
||||||
|
root, err := os.MkdirTemp("/private/tmp", "tver-pg17-")
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
testPort = listener.Addr().(*net.TCPAddr).Port
|
||||||
|
listener.Close()
|
||||||
|
config := embeddedpostgres.DefaultConfig().Version(embeddedpostgres.V17).
|
||||||
|
Port(uint32(testPort)).CachePath("/private/tmp/tver-embedded-pg-cache").
|
||||||
|
RuntimePath(filepath.Join(root, "runtime")).DataPath(filepath.Join(root, "data")).
|
||||||
|
Logger(io.Discard).StartTimeout(60 * time.Second)
|
||||||
|
db := embeddedpostgres.NewDatabase(config)
|
||||||
|
if err := db.Start(); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "embedded Postgres 17 start: %v\n", err)
|
||||||
|
_ = os.RemoveAll(root)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
adminPool, err = pgxpool.New(context.Background(), config.GetConnectionURL())
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
_ = db.Stop()
|
||||||
|
_ = os.RemoveAll(root)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
code := m.Run()
|
||||||
|
adminPool.Close()
|
||||||
|
if err := db.Stop(); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "embedded Postgres 17 stop: %v\n", err)
|
||||||
|
code = 1
|
||||||
|
}
|
||||||
|
_ = os.RemoveAll(root)
|
||||||
|
os.Exit(code)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testPool(t *testing.T, migrate bool) *pgxpool.Pool {
|
||||||
|
t.Helper()
|
||||||
|
name := fmt.Sprintf("tver_test_%d", dbSequence.Add(1))
|
||||||
|
ctx := context.Background()
|
||||||
|
if _, err := adminPool.Exec(ctx, `CREATE DATABASE `+name); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
url := fmt.Sprintf("postgres://postgres:postgres@localhost:%d/%s", testPort, name)
|
||||||
|
pool, err := pgxpool.New(ctx, url)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
pool.Close()
|
||||||
|
if _, err := adminPool.Exec(ctx, `DROP DATABASE `+name); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
if migrate {
|
||||||
|
if err := Migrate(ctx, pool); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return pool
|
||||||
|
}
|
||||||
|
|
||||||
|
func sampleEpisode(id string) Episode {
|
||||||
|
end := time.Date(2026, 10, 4, 12, 0, 0, 0, time.UTC)
|
||||||
|
return Episode{ID: id, SeriesID: "series-1", SeriesTitle: "番組", Title: "第1話",
|
||||||
|
BroadcasterName: "放送局", DurationSeconds: 300, EndAt: &end, ThumbnailPath: "/thumb.jpg"}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMigrate(t *testing.T) {
|
||||||
|
pool := testPool(t, false)
|
||||||
|
ctx := context.Background()
|
||||||
|
if err := Migrate(ctx, pool); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var first time.Time
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT applied_at FROM schema_migrations WHERE version='migrations/001_initial.sql'`).Scan(&first); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := Migrate(ctx, pool); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var count int
|
||||||
|
var second time.Time
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*), min(applied_at) FROM schema_migrations`).Scan(&count, &second); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if count != 3 || !first.Equal(second) {
|
||||||
|
t.Fatalf("再適用で履歴が変わった: %d %v %v", count, first, second)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMigrateConcurrent(t *testing.T) {
|
||||||
|
pool := testPool(t, false)
|
||||||
|
const workers = 8
|
||||||
|
errCh := make(chan error, workers)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for range workers {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() { defer wg.Done(); errCh <- Migrate(context.Background(), pool) }()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
close(errCh)
|
||||||
|
for err := range errCh {
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var count int
|
||||||
|
if err := pool.QueryRow(context.Background(), `SELECT count(*) FROM schema_migrations`).Scan(&count); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if count != 3 {
|
||||||
|
t.Fatalf("適用件数: %d", count)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMigrateAddsKnownEpisodesToExistingDatabase(t *testing.T) {
|
||||||
|
pool := testPool(t, false)
|
||||||
|
ctx := context.Background()
|
||||||
|
initial, err := migrations.ReadFile("migrations/001_initial.sql")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, string(initial), pgx.QueryExecModeSimpleProtocol); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `CREATE TABLE schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `INSERT INTO schema_migrations (version) VALUES ('migrations/001_initial.sql')`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title) VALUES ('existing', '既存番組')`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := Migrate(ctx, pool); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var count int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM schema_migrations`).Scan(&count); err != nil || count != 3 {
|
||||||
|
t.Fatalf("移行履歴: %d %v", count, err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `INSERT INTO known_episodes (series_id, episode_id) VALUES ('existing', 'ep1')`); err != nil {
|
||||||
|
t.Fatalf("既存番組への既知記録: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `DELETE FROM subscriptions WHERE series_id='existing'`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id='existing'`).Scan(&count); err != nil || count != 0 {
|
||||||
|
t.Fatalf("アップグレード後の cascade: %d %v", count, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestMigrateAddsSubscriptionCheckErrorsToVersionTwoDatabase(t *testing.T) {
|
||||||
|
pool := testPool(t, false)
|
||||||
|
ctx := context.Background()
|
||||||
|
for _, name := range []string{"migrations/001_initial.sql", "migrations/002_known_episodes.sql"} {
|
||||||
|
sql, err := migrations.ReadFile(name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, string(sql), pgx.QueryExecModeSimpleProtocol); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `CREATE TABLE schema_migrations (version TEXT PRIMARY KEY, applied_at TIMESTAMPTZ NOT NULL DEFAULT now())`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `INSERT INTO schema_migrations (version) VALUES ('migrations/001_initial.sql'), ('migrations/002_known_episodes.sql')`); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
checked := time.Date(2026, 10, 1, 2, 3, 4, 0, time.UTC)
|
||||||
|
if _, err := pool.Exec(ctx, `INSERT INTO subscriptions (series_id, series_title, last_checked_at) VALUES ('existing-v2', '既存番組', $1)`, checked); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := Migrate(ctx, pool); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var title string
|
||||||
|
var last time.Time
|
||||||
|
var errorAt, errorMessage any
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT series_title, last_checked_at, last_check_error_at, last_check_error FROM subscriptions WHERE series_id='existing-v2'`).Scan(&title, &last, &errorAt, &errorMessage); err != nil || title != "既存番組" || !last.Equal(checked) || errorAt != nil || errorMessage != nil {
|
||||||
|
t.Fatalf("v2データ保持と追加列: title=%q last=%v errorAt=%v error=%v err=%v", title, last, errorAt, errorMessage, err)
|
||||||
|
}
|
||||||
|
var indexDef string
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT indexdef FROM pg_indexes WHERE schemaname=current_schema() AND indexname='subscriptions_due_idx'`).Scan(&indexDef); err != nil || !strings.Contains(indexDef, "last_checked_at") || !strings.Contains(indexDef, "WHERE enabled") {
|
||||||
|
t.Fatalf("期限抽出索引: %q %v", indexDef, err)
|
||||||
|
}
|
||||||
|
var versions int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM schema_migrations`).Scan(&versions); err != nil || versions != 3 {
|
||||||
|
t.Fatalf("移行履歴: %d %v", versions, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDownloadJobIndexes(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
rows, err := pool.Query(context.Background(), `SELECT indexname, indexdef FROM pg_indexes WHERE schemaname=current_schema() AND tablename='download_jobs'`)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
indexes := make(map[string]string)
|
||||||
|
for rows.Next() {
|
||||||
|
var name, definition string
|
||||||
|
if err := rows.Scan(&name, &definition); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
indexes[name] = definition
|
||||||
|
}
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
checks := map[string][]string{
|
||||||
|
"download_jobs_episode_id_key": {"UNIQUE", "(episode_id)"},
|
||||||
|
"download_jobs_claim_idx": {"(created_at, id)", "status = 'queued'"},
|
||||||
|
"download_jobs_status_newest_idx": {"(status, created_at DESC, id DESC)"},
|
||||||
|
"download_jobs_stale_idx": {"(heartbeat_at)", "status = 'running'"},
|
||||||
|
}
|
||||||
|
for name, parts := range checks {
|
||||||
|
definition := indexes[name]
|
||||||
|
for _, part := range parts {
|
||||||
|
if !strings.Contains(definition, part) {
|
||||||
|
t.Errorf("索引 %s に %q がない: %q", name, part, definition)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnqueueAndListing(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
first, created, err := s.Enqueue(ctx, sampleEpisode("ep1"), "manual")
|
||||||
|
if err != nil || !created || first.Status != "queued" || first.Source != "manual" || first.DurationSeconds != 300 || first.EndAt == nil {
|
||||||
|
t.Fatalf("enqueue: %+v %v %v", first, created, err)
|
||||||
|
}
|
||||||
|
again, created, err := s.Enqueue(ctx, sampleEpisode("ep1"), "subscription")
|
||||||
|
if err != nil || created || again.ID != first.ID || again.Source != "manual" {
|
||||||
|
t.Fatalf("重複: %+v %v %v", again, created, err)
|
||||||
|
}
|
||||||
|
second, _, err := s.Enqueue(ctx, sampleEpisode("ep2"), "subscription")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs, err := s.ListJobs(ctx, "queued", 10)
|
||||||
|
if err != nil || len(jobs) != 2 || jobs[0].ID != second.ID {
|
||||||
|
t.Fatalf("list: %+v %v", jobs, err)
|
||||||
|
}
|
||||||
|
statuses, err := s.EpisodeStatuses(ctx, []string{"ep1", "ep2", "absent"})
|
||||||
|
if err != nil || len(statuses) != 2 || statuses["ep1"] != "queued" {
|
||||||
|
t.Fatalf("statuses: %+v %v", statuses, err)
|
||||||
|
}
|
||||||
|
if _, err := s.ListJobs(ctx, "bad", 10); err == nil {
|
||||||
|
t.Fatal("不正な状態を許した")
|
||||||
|
}
|
||||||
|
if _, _, err := s.Enqueue(ctx, sampleEpisode("bad"), "invalid"); err == nil {
|
||||||
|
t.Fatal("source CHECK を許した")
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET phase='bad' WHERE id=$1`, first.ID); err == nil {
|
||||||
|
t.Fatal("phase CHECK を許した")
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET status='bad' WHERE id=$1`, first.ID); err == nil {
|
||||||
|
t.Fatal("status CHECK を許した")
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET error_kind='bad' WHERE id=$1`, first.ID); err == nil {
|
||||||
|
t.Fatal("error CHECK を許した")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTransitions(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
job, _, err := s.Enqueue(ctx, sampleEpisode("ep1"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "video"}); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("queued progress: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Complete(ctx, job.ID, "/file.mp4", 100); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("queued complete: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Fail(ctx, job.ID, "network", "failed"); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("queued fail: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Retry(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("queued retry: %v", err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.ID != job.ID || claimed.Status != "running" || claimed.Attempts != 1 || claimed.HeartbeatAt == nil || claimed.StartedAt == nil {
|
||||||
|
t.Fatalf("claim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
oldHeartbeat := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, job.ID, oldHeartbeat); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
bytes := int64(123)
|
||||||
|
total := int64(1000)
|
||||||
|
speed := 40.5
|
||||||
|
eta := 12.0
|
||||||
|
fragment := int32(2)
|
||||||
|
fragments := int32(10)
|
||||||
|
format := "video-1"
|
||||||
|
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "video", DownloadedBytes: &bytes,
|
||||||
|
TotalBytes: &total, TotalIsEstimate: true, SpeedBPS: &speed, ETASeconds: &eta,
|
||||||
|
FragmentIndex: &fragment, FragmentCount: &fragments, FormatID: &format}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
progressed, err := s.GetJob(ctx, job.ID)
|
||||||
|
if err != nil || progressed.Phase == nil || *progressed.Phase != "video" || progressed.DownloadedBytes == nil || *progressed.DownloadedBytes != bytes || progressed.TotalBytes == nil || *progressed.TotalBytes != total || !progressed.TotalIsEstimate || progressed.SpeedBPS == nil || *progressed.SpeedBPS != speed || progressed.ETASeconds == nil || *progressed.ETASeconds != eta || progressed.FragmentIndex == nil || *progressed.FragmentIndex != fragment || progressed.FragmentCount == nil || *progressed.FragmentCount != fragments || progressed.FormatID == nil || *progressed.FormatID != format || progressed.HeartbeatAt == nil || !progressed.HeartbeatAt.After(oldHeartbeat) {
|
||||||
|
t.Fatalf("progress: %+v %v", progressed, err)
|
||||||
|
}
|
||||||
|
if err := s.Fail(ctx, job.ID, "network", "connection lost"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
failed, err := s.GetJob(ctx, job.ID)
|
||||||
|
if err != nil || failed.Status != "failed" || failed.ErrorKind == nil || *failed.ErrorKind != "network" || failed.ErrorMessage == nil || *failed.ErrorMessage != "connection lost" || failed.HeartbeatAt != nil || failed.FinishedAt == nil {
|
||||||
|
t.Fatalf("fail: %+v %v", failed, err)
|
||||||
|
}
|
||||||
|
if err := s.UpdateProgress(ctx, job.ID, Progress{Phase: "audio"}); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("failed progress: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Retry(ctx, job.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
retried, err := s.GetJob(ctx, job.ID)
|
||||||
|
if err != nil || retried.Status != "queued" || retried.Phase != nil || retried.ErrorKind != nil || retried.ErrorMessage != nil || retried.DownloadedBytes != nil || retried.TotalBytes != nil || retried.TotalIsEstimate || retried.SpeedBPS != nil || retried.ETASeconds != nil || retried.FragmentIndex != nil || retried.FragmentCount != nil || retried.FormatID != nil || retried.HeartbeatAt != nil || retried.StartedAt != nil || retried.FinishedAt != nil || retried.Attempts != 1 {
|
||||||
|
t.Fatalf("retry: %+v %v", retried, err)
|
||||||
|
}
|
||||||
|
claimed, err = s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.Attempts != 2 {
|
||||||
|
t.Fatalf("reclaim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := s.Complete(ctx, job.ID, "/file.mp4", 100); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
completed, err := s.GetJob(ctx, job.ID)
|
||||||
|
if err != nil || completed.Status != "succeeded" || completed.Phase != nil || completed.HeartbeatAt != nil || completed.FinishedAt == nil || completed.FilePath == nil || *completed.FilePath != "/file.mp4" || completed.FileSizeBytes == nil || *completed.FileSizeBytes != 100 {
|
||||||
|
t.Fatalf("complete: %+v %v", completed, err)
|
||||||
|
}
|
||||||
|
if err := s.Cancel(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("succeeded cancel: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Retry(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("succeeded retry: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := s.ClaimNext(ctx); !errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
t.Fatalf("empty queue: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCancelAndStaleBoundary(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
queued, _, err := s.Enqueue(ctx, sampleEpisode("queued"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.Cancel(ctx, queued.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
canceled, err := s.GetJob(ctx, queued.ID)
|
||||||
|
if err != nil || canceled.Status != "canceled" || canceled.ErrorKind == nil || *canceled.ErrorKind != "canceled" {
|
||||||
|
t.Fatalf("cancel: %+v %v", canceled, err)
|
||||||
|
}
|
||||||
|
if err := s.Retry(ctx, queued.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.Cancel(ctx, claimed.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.Complete(ctx, claimed.ID, "/late.mp4", 1); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("canceled complete: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Retry(ctx, claimed.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err = s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.Attempts != 2 {
|
||||||
|
t.Fatalf("cancel retry claim: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
cutoff := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, claimed.ID, cutoff); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
fresh, _, err := s.Enqueue(ctx, sampleEpisode("fresh"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.ClaimNext(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, fresh.ID, cutoff.Add(time.Hour)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
n, err := s.RequeueStale(ctx, cutoff)
|
||||||
|
if err != nil || n != 0 {
|
||||||
|
t.Fatalf("境界上: %d %v", n, err)
|
||||||
|
}
|
||||||
|
n, err = s.RequeueStale(ctx, cutoff.Add(time.Microsecond))
|
||||||
|
if err != nil || n != 1 {
|
||||||
|
t.Fatalf("境界後: %d %v", n, err)
|
||||||
|
}
|
||||||
|
job, err := s.GetJob(ctx, claimed.ID)
|
||||||
|
if err != nil || job.Status != "queued" || job.HeartbeatAt != nil || job.Attempts != 2 {
|
||||||
|
t.Fatalf("requeue: %+v %v", job, err)
|
||||||
|
}
|
||||||
|
stillRunning, err := s.GetJob(ctx, fresh.ID)
|
||||||
|
if err != nil || stillRunning.Status != "running" || stillRunning.HeartbeatAt == nil || !stillRunning.HeartbeatAt.Equal(cutoff.Add(time.Hour)) {
|
||||||
|
t.Fatalf("新しい heartbeat を持つジョブ: %+v %v", stillRunning, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHeartbeatAndGracefulRequeue(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
job, _, err := s.Enqueue(ctx, sampleEpisode("heartbeat"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.Heartbeat(ctx, job.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("待機中の heartbeat: %v", err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
old := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
if _, err := s.pool.Exec(ctx, `UPDATE download_jobs SET heartbeat_at=$2 WHERE id=$1`, claimed.ID, old); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.Heartbeat(ctx, claimed.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
heartbeated, err := s.GetJob(ctx, claimed.ID)
|
||||||
|
if err != nil || heartbeated.HeartbeatAt == nil || !heartbeated.HeartbeatAt.After(old) {
|
||||||
|
t.Fatalf("heartbeat: %+v %v", heartbeated, err)
|
||||||
|
}
|
||||||
|
bytes := int64(10)
|
||||||
|
if err := s.UpdateProgress(ctx, claimed.ID, Progress{Phase: "video", DownloadedBytes: &bytes}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.RequeueRunning(ctx, claimed.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
requeued, err := s.GetJob(ctx, claimed.ID)
|
||||||
|
if err != nil || requeued.Status != "queued" || requeued.StartedAt != nil || requeued.HeartbeatAt != nil || requeued.Phase != nil || requeued.DownloadedBytes != nil || requeued.Attempts != 1 {
|
||||||
|
t.Fatalf("graceful requeue: %+v %v", requeued, err)
|
||||||
|
}
|
||||||
|
if err := s.RequeueRunning(ctx, claimed.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("待機中の再復帰: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRejectedTransitions(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
operations := map[string]func(int64) error{
|
||||||
|
"progress": func(id int64) error { return s.UpdateProgress(ctx, id, Progress{Phase: "video"}) },
|
||||||
|
"complete": func(id int64) error { return s.Complete(ctx, id, "/file.mp4", 1) },
|
||||||
|
"fail": func(id int64) error { return s.Fail(ctx, id, "network", "error") },
|
||||||
|
"cancel": func(id int64) error { return s.Cancel(ctx, id) },
|
||||||
|
"retry": func(id int64) error { return s.Retry(ctx, id) },
|
||||||
|
}
|
||||||
|
for status, rejected := range map[string][]string{
|
||||||
|
"queued": {"progress", "complete", "fail", "retry"},
|
||||||
|
"running": {"retry"},
|
||||||
|
"succeeded": {"progress", "complete", "fail", "cancel", "retry"},
|
||||||
|
"failed": {"progress", "complete", "fail", "cancel"},
|
||||||
|
"canceled": {"progress", "complete", "fail", "cancel"},
|
||||||
|
} {
|
||||||
|
t.Run(status, func(t *testing.T) {
|
||||||
|
job, _, err := s.Enqueue(ctx, sampleEpisode("rejected-"+status), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET status=$2 WHERE id=$1`, job.ID, status); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, name := range rejected {
|
||||||
|
if err := operations[name](job.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Errorf("%s on %s: %v", name, status, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
got, err := s.GetJob(ctx, job.ID)
|
||||||
|
if err != nil || got.Status != status {
|
||||||
|
t.Fatalf("拒否後の状態: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRetryEpisode(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
job, _, err := s.Enqueue(ctx, sampleEpisode("retry-by-episode"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.RetryEpisode(ctx, job.EpisodeID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("待機中の再試行: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Cancel(ctx, job.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.RetryEpisode(ctx, job.EpisodeID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
got, err := s.GetJob(ctx, job.ID)
|
||||||
|
if err != nil || got.Status != "queued" || got.ErrorKind != nil || got.FinishedAt != nil {
|
||||||
|
t.Fatalf("エピソード ID による再試行: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
downloaded, total := int64(10), int64(20)
|
||||||
|
format := "video-1"
|
||||||
|
if err := s.UpdateProgress(ctx, claimed.ID, Progress{
|
||||||
|
Phase: "video", DownloadedBytes: &downloaded, TotalBytes: &total,
|
||||||
|
TotalIsEstimate: true, FormatID: &format,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.Fail(ctx, claimed.ID, "network", "通信失敗"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.RetryEpisode(ctx, job.EpisodeID); err != nil {
|
||||||
|
t.Fatalf("失敗からの再試行: %v", err)
|
||||||
|
}
|
||||||
|
got, err = s.GetJob(ctx, job.ID)
|
||||||
|
if err != nil || got.Status != "queued" || got.Phase != nil || got.DownloadedBytes != nil || got.TotalBytes != nil || got.TotalIsEstimate || got.FormatID != nil || got.ErrorKind != nil || got.ErrorMessage != nil || got.HeartbeatAt != nil || got.StartedAt != nil || got.FinishedAt != nil || got.Attempts != 1 {
|
||||||
|
t.Fatalf("失敗情報の消去: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
if err := s.RetryEpisode(ctx, "unknown"); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("存在しないエピソード: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCancelQueuedDoesNotCancelRunning(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
queued, _, err := s.Enqueue(ctx, sampleEpisode("queue-cancel"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.CancelQueued(ctx, queued.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
canceled, err := s.GetJob(ctx, queued.ID)
|
||||||
|
if err != nil || canceled.Status != "canceled" || canceled.ErrorKind == nil || *canceled.ErrorKind != "canceled" || canceled.FinishedAt == nil {
|
||||||
|
t.Fatalf("待機中の取り消し: %+v %v", canceled, err)
|
||||||
|
}
|
||||||
|
if err := s.CancelQueued(ctx, queued.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("再取り消し: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.CancelQueued(ctx, -1); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("存在しないジョブの取り消し: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.Retry(ctx, queued.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.CancelQueued(ctx, claimed.ID); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("実行中の取り消し: %v", err)
|
||||||
|
}
|
||||||
|
job, err := s.GetJob(ctx, claimed.ID)
|
||||||
|
if err != nil || job.Status != "running" {
|
||||||
|
t.Fatalf("実行中の状態: %+v %v", job, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClaimNextConcurrent(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
const jobs = 12
|
||||||
|
for i := range jobs {
|
||||||
|
if _, _, err := s.Enqueue(ctx, sampleEpisode(fmt.Sprintf("ep%d", i)), "manual"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
results := make(chan int64, jobs)
|
||||||
|
errCh := make(chan error, jobs)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for range jobs {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
job, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil {
|
||||||
|
errCh <- err
|
||||||
|
return
|
||||||
|
}
|
||||||
|
results <- job.ID
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
close(results)
|
||||||
|
close(errCh)
|
||||||
|
for err := range errCh {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
seen := make(map[int64]bool)
|
||||||
|
for id := range results {
|
||||||
|
if seen[id] {
|
||||||
|
t.Errorf("二重 claim: %d", id)
|
||||||
|
}
|
||||||
|
seen[id] = true
|
||||||
|
}
|
||||||
|
if len(seen) != jobs {
|
||||||
|
t.Fatalf("claim 数: %d", len(seen))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestClaimNextSkipsLockedOldest(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
oldest, _, err := s.Enqueue(ctx, sampleEpisode("oldest"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
newer, _, err := s.Enqueue(ctx, sampleEpisode("newer"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
base := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, oldest.ID, base); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, newer.ID, base.Add(time.Hour)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
tx, err := pool.Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback(ctx)
|
||||||
|
if _, err := tx.Exec(ctx, `SELECT id FROM download_jobs WHERE id=$1 FOR UPDATE`, oldest.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
claimed, err := s.ClaimNext(claimCtx)
|
||||||
|
if err != nil || claimed.ID != newer.ID {
|
||||||
|
t.Fatalf("ロック中の最古ジョブを飛ばせない: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
if err := tx.Rollback(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claimed, err = s.ClaimNext(ctx)
|
||||||
|
if err != nil || claimed.ID != oldest.ID {
|
||||||
|
t.Fatalf("ロック解放後の最古ジョブ: %+v %v", claimed, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListJobsFiltersAndLimit(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
ids := make([]int64, 3)
|
||||||
|
for i := range ids {
|
||||||
|
job, _, err := s.Enqueue(ctx, sampleEpisode(fmt.Sprintf("list-%d", i)), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ids[i] = job.ID
|
||||||
|
created := time.Date(2026, 10, 3, i, 0, 0, 0, time.UTC)
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE download_jobs SET created_at=$2 WHERE id=$1`, job.ID, created); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.Cancel(ctx, ids[1]); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
all, err := s.ListJobs(ctx, "", 2)
|
||||||
|
if err != nil || len(all) != 2 || all[0].ID != ids[2] || all[1].ID != ids[1] {
|
||||||
|
t.Fatalf("全件の新しい順と上限: %+v %v", all, err)
|
||||||
|
}
|
||||||
|
queued, err := s.ListJobs(ctx, "queued", 10)
|
||||||
|
if err != nil || len(queued) != 2 || queued[0].ID != ids[2] || queued[1].ID != ids[0] {
|
||||||
|
t.Fatalf("状態別一覧: %+v %v", queued, err)
|
||||||
|
}
|
||||||
|
canceled, err := s.ListJobs(ctx, "canceled", 10)
|
||||||
|
if err != nil || len(canceled) != 1 || canceled[0].ID != ids[1] {
|
||||||
|
t.Fatalf("中断済み一覧: %+v %v", canceled, err)
|
||||||
|
}
|
||||||
|
if _, err := s.ListJobs(ctx, "", 0); err == nil {
|
||||||
|
t.Fatal("0 件の上限を許した")
|
||||||
|
}
|
||||||
|
statuses, err := s.EpisodeStatuses(ctx, []string{"list-0", "list-1", "absent"})
|
||||||
|
if err != nil || len(statuses) != 2 || statuses["list-0"] != "queued" || statuses["list-1"] != "canceled" {
|
||||||
|
t.Fatalf("状態の一括取得: %+v %v", statuses, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptions(t *testing.T) {
|
||||||
|
s := New(testPool(t, true))
|
||||||
|
ctx := context.Background()
|
||||||
|
created, err := s.RegisterSubscription(ctx, "series", "番組", nil, false)
|
||||||
|
if err != nil || !created {
|
||||||
|
t.Fatalf("add: %v %v", created, err)
|
||||||
|
}
|
||||||
|
created, err = s.RegisterSubscription(ctx, "series", "別タイトル", nil, false)
|
||||||
|
if err != nil || created {
|
||||||
|
t.Fatalf("duplicate: %v %v", created, err)
|
||||||
|
}
|
||||||
|
when := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
if err := s.SetSubscriptionChecked(ctx, "series", when); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
subs, err := s.ListSubscriptions(ctx)
|
||||||
|
if err != nil || len(subs) != 1 || subs[0].SeriesTitle != "番組" || !subs[0].Enabled || subs[0].LastCheckedAt == nil || !subs[0].LastCheckedAt.Equal(when) {
|
||||||
|
t.Fatalf("list: %+v %v", subs, err)
|
||||||
|
}
|
||||||
|
deleted, err := s.DeleteSubscription(ctx, "series")
|
||||||
|
if err != nil || !deleted {
|
||||||
|
t.Fatalf("delete: %v %v", deleted, err)
|
||||||
|
}
|
||||||
|
deleted, err = s.DeleteSubscription(ctx, "series")
|
||||||
|
if err != nil || deleted {
|
||||||
|
t.Fatalf("duplicate delete: %v %v", deleted, err)
|
||||||
|
}
|
||||||
|
if err := s.SetSubscriptionChecked(ctx, "series", when); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("missing: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,244 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
var ErrSubscriptionPaused = errors.New("subscription is paused")
|
||||||
|
|
||||||
|
// RegisterSubscription は登録と公開中の回の記録・任意の投入を一つの取引で行う。
|
||||||
|
func (s *Store) RegisterSubscription(ctx context.Context, seriesID, title string, episodes []Episode, downloadExisting bool) (bool, error) {
|
||||||
|
if seriesID == "" || title == "" {
|
||||||
|
return false, errors.New("series id and title are required")
|
||||||
|
}
|
||||||
|
tx, err := s.pool.Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
defer tx.Rollback(ctx)
|
||||||
|
var inserted string
|
||||||
|
err = tx.QueryRow(ctx, `INSERT INTO subscriptions (series_id, series_title, last_checked_at)
|
||||||
|
VALUES ($1,$2,now()) ON CONFLICT (series_id) DO NOTHING RETURNING series_id`, seriesID, title).Scan(&inserted)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if _, err := addKnownAndMaybeQueue(ctx, tx, seriesID, episodes, downloadExisting); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := tx.Commit(ctx); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// QueueUnseenEpisodes は未確認の回だけを既知に記録し、同じ取引で待ち行列へ入れる。
|
||||||
|
func (s *Store) QueueUnseenEpisodes(ctx context.Context, seriesID string, episodes []Episode) (int, error) {
|
||||||
|
tx, err := s.pool.Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer tx.Rollback(ctx)
|
||||||
|
var enabled bool
|
||||||
|
err = tx.QueryRow(ctx, `SELECT enabled FROM subscriptions WHERE series_id=$1 FOR UPDATE`, seriesID).Scan(&enabled)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
return 0, ErrInvalidTransition
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if !enabled {
|
||||||
|
return 0, ErrSubscriptionPaused
|
||||||
|
}
|
||||||
|
count, err := addKnownAndMaybeQueue(ctx, tx, seriesID, episodes, true)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if err := tx.Commit(ctx); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
return count, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListDueSubscriptions は有効で、cutoff 以前に確認された（または未確認の）番組を返す。
|
||||||
|
func (s *Store) ListDueSubscriptions(ctx context.Context, cutoff time.Time) ([]Subscription, error) {
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT series_id, series_title, enabled, created_at, last_checked_at,
|
||||||
|
last_check_error_at, last_check_error, 0::bigint
|
||||||
|
FROM subscriptions
|
||||||
|
WHERE enabled AND (last_checked_at IS NULL OR last_checked_at <= $1)
|
||||||
|
ORDER BY last_checked_at ASC NULLS FIRST, created_at, series_id`, cutoff)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var result []Subscription
|
||||||
|
for rows.Next() {
|
||||||
|
var sub Subscription
|
||||||
|
if err := rows.Scan(&sub.SeriesID, &sub.SeriesTitle, &sub.Enabled, &sub.CreatedAt,
|
||||||
|
&sub.LastCheckedAt, &sub.LastCheckErrorAt, &sub.LastCheckError, &sub.JobCount); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
result = append(result, sub)
|
||||||
|
}
|
||||||
|
return result, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// CompleteSubscriptionCheck は全公開回を既知にし、しきい値を超える未確認回だけを投入する。
|
||||||
|
// 最終確認日時の更新と前回エラーの消去も同じ取引で行う。
|
||||||
|
func (s *Store) CompleteSubscriptionCheck(ctx context.Context, seriesID string, episodes []Episode, previewMaxSeconds int, checkedAt time.Time) (int, error) {
|
||||||
|
if previewMaxSeconds < 0 {
|
||||||
|
return 0, errors.New("preview threshold must not be negative")
|
||||||
|
}
|
||||||
|
tx, err := s.pool.Begin(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
defer tx.Rollback(ctx)
|
||||||
|
var enabled bool
|
||||||
|
err = tx.QueryRow(ctx, `SELECT enabled FROM subscriptions WHERE series_id=$1 FOR UPDATE`, seriesID).Scan(&enabled)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
return 0, ErrInvalidTransition
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if !enabled {
|
||||||
|
return 0, ErrSubscriptionPaused
|
||||||
|
}
|
||||||
|
queued := 0
|
||||||
|
for _, ep := range episodes {
|
||||||
|
if ep.ID == "" || ep.SeriesID != seriesID || ep.DurationSeconds < 0 {
|
||||||
|
return 0, fmt.Errorf("invalid episode identity for series %s", seriesID)
|
||||||
|
}
|
||||||
|
var inserted string
|
||||||
|
err := tx.QueryRow(ctx, `INSERT INTO known_episodes (series_id, episode_id)
|
||||||
|
VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING episode_id`, seriesID, ep.ID).Scan(&inserted)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if ep.DurationSeconds <= previewMaxSeconds {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
tag, err := tx.Exec(ctx, `INSERT INTO download_jobs
|
||||||
|
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,'subscription')
|
||||||
|
ON CONFLICT (episode_id) DO NOTHING`, ep.ID, seriesID, ep.SeriesTitle, ep.Title,
|
||||||
|
ep.BroadcasterName, ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
queued += int(tag.RowsAffected())
|
||||||
|
}
|
||||||
|
if _, err := tx.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2,
|
||||||
|
last_check_error_at=NULL, last_check_error=NULL WHERE series_id=$1`, seriesID, checkedAt); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
if err := tx.Commit(ctx); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
return queued, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RecordSubscriptionCheckFailure は取得失敗を記録し、最終確認日時は変更しない。
|
||||||
|
func (s *Store) RecordSubscriptionCheckFailure(ctx context.Context, seriesID string, failedAt time.Time, message string) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET last_check_error_at=$2, last_check_error=$3
|
||||||
|
WHERE series_id=$1 AND enabled`, seriesID, failedAt, message)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func addKnownAndMaybeQueue(ctx context.Context, tx pgx.Tx, seriesID string, episodes []Episode, queue bool) (int, error) {
|
||||||
|
count := 0
|
||||||
|
for _, ep := range episodes {
|
||||||
|
if ep.ID == "" || ep.SeriesID != seriesID {
|
||||||
|
return 0, fmt.Errorf("invalid episode identity for series %s", seriesID)
|
||||||
|
}
|
||||||
|
var inserted string
|
||||||
|
err := tx.QueryRow(ctx, `INSERT INTO known_episodes (series_id, episode_id)
|
||||||
|
VALUES ($1,$2) ON CONFLICT DO NOTHING RETURNING episode_id`, seriesID, ep.ID).Scan(&inserted)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
count++
|
||||||
|
if queue {
|
||||||
|
_, err := tx.Exec(ctx, `INSERT INTO download_jobs
|
||||||
|
(episode_id, series_id, series_title, title, broadcaster_name, duration_seconds, end_at, thumbnail_path, source)
|
||||||
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,'subscription')
|
||||||
|
ON CONFLICT (episode_id) DO NOTHING`, ep.ID, seriesID, ep.SeriesTitle, ep.Title,
|
||||||
|
ep.BroadcasterName, ep.DurationSeconds, ep.EndAt, ep.ThumbnailPath)
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return count, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnknownEpisodeIDs は指定シリーズの未確認のエピソード ID を入力順で返す。
|
||||||
|
func (s *Store) UnknownEpisodeIDs(ctx context.Context, seriesID string, ids []string) ([]string, error) {
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return []string{}, nil
|
||||||
|
}
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT episode_id FROM known_episodes WHERE series_id=$1 AND episode_id=ANY($2)`, seriesID, ids)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
known := make(map[string]bool)
|
||||||
|
for rows.Next() {
|
||||||
|
var id string
|
||||||
|
if err := rows.Scan(&id); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
known[id] = true
|
||||||
|
}
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var unknown []string
|
||||||
|
emitted := make(map[string]bool)
|
||||||
|
for _, id := range ids {
|
||||||
|
if !known[id] && !emitted[id] {
|
||||||
|
unknown = append(unknown, id)
|
||||||
|
emitted[id] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return unknown, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) SetSubscriptionEnabled(ctx context.Context, seriesID string, enabled bool) error {
|
||||||
|
tag, err := s.pool.Exec(ctx, `UPDATE subscriptions SET enabled=$2 WHERE series_id=$1`, seriesID, enabled)
|
||||||
|
return transitionResult(tag.RowsAffected(), err)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Store) SubscribedSeries(ctx context.Context, ids []string) (map[string]bool, error) {
|
||||||
|
result := make(map[string]bool)
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
rows, err := s.pool.Query(ctx, `SELECT series_id FROM subscriptions WHERE series_id=ANY($1)`, ids)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
for rows.Next() {
|
||||||
|
var id string
|
||||||
|
if err := rows.Scan(&id); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
result[id] = true
|
||||||
|
}
|
||||||
|
return result, rows.Err()
|
||||||
|
}
|
||||||
@@ -0,0 +1,326 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestListDueSubscriptionsBoundaryAndPaused(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||||
|
for _, id := range []string{"null", "old", "boundary", "fresh", "paused"} {
|
||||||
|
if created, err := s.RegisterSubscription(ctx, id, "番組 "+id, nil, false); err != nil || !created {
|
||||||
|
t.Fatalf("登録 %s: %t %v", id, created, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
updates := map[string]*time.Time{
|
||||||
|
"null": nil, "old": timePointer(now.Add(-25 * time.Hour)),
|
||||||
|
"boundary": timePointer(now.Add(-24 * time.Hour)), "fresh": timePointer(now.Add(-24*time.Hour + time.Microsecond)),
|
||||||
|
"paused": timePointer(now.Add(-48 * time.Hour)),
|
||||||
|
}
|
||||||
|
for id, checked := range updates {
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, id, checked); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.SetSubscriptionEnabled(ctx, "paused", false); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
due, err := s.ListDueSubscriptions(ctx, now.Add(-24*time.Hour))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(due) != 3 || due[0].SeriesID != "null" || due[1].SeriesID != "old" || due[2].SeriesID != "boundary" {
|
||||||
|
t.Fatalf("24時間境界と一時停止: %+v", due)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompleteSubscriptionCheckKnownPreviewErrorAndAtomicity(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
if created, err := s.RegisterSubscription(ctx, "series-check", "番組", []Episode{seriesEpisode("known", "series-check")}, false); err != nil || !created {
|
||||||
|
t.Fatalf("登録: %t %v", created, err)
|
||||||
|
}
|
||||||
|
before := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
failedAt := time.Date(2026, 10, 3, 1, 2, 3, 0, time.UTC)
|
||||||
|
if _, err := pool.Exec(ctx, `UPDATE subscriptions SET last_checked_at=$2 WHERE series_id=$1`, "series-check", before); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := s.RecordSubscriptionCheckFailure(ctx, "series-check", failedAt, "network token=[REDACTED]"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var unchanged time.Time
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id='series-check'`).Scan(&unchanged); err != nil || !unchanged.Equal(before) {
|
||||||
|
t.Fatalf("失敗時に最終確認を更新: %v %v", unchanged, err)
|
||||||
|
}
|
||||||
|
checkedAt := time.Date(2026, 10, 3, 4, 5, 6, 0, time.UTC)
|
||||||
|
preview := seriesEpisode("preview", "series-check")
|
||||||
|
preview.DurationSeconds = 180
|
||||||
|
normal := seriesEpisode("normal", "series-check")
|
||||||
|
normal.DurationSeconds = 181
|
||||||
|
count, err := s.CompleteSubscriptionCheck(ctx, "series-check", []Episode{
|
||||||
|
seriesEpisode("known", "series-check"), preview, normal,
|
||||||
|
}, 180, checkedAt)
|
||||||
|
if err != nil || count != 1 {
|
||||||
|
t.Fatalf("確認完了: count=%d err=%v", count, err)
|
||||||
|
}
|
||||||
|
unknown, err := s.UnknownEpisodeIDs(ctx, "series-check", []string{"known", "preview", "normal"})
|
||||||
|
if err != nil || len(unknown) != 0 {
|
||||||
|
t.Fatalf("予告を含む既知化: %v %v", unknown, err)
|
||||||
|
}
|
||||||
|
var jobs int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-check'`).Scan(&jobs); err != nil || jobs != 1 {
|
||||||
|
t.Fatalf("予告の投入除外: %d %v", jobs, err)
|
||||||
|
}
|
||||||
|
var last time.Time
|
||||||
|
var errorAt, errorMessage any
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT last_checked_at, last_check_error_at, last_check_error FROM subscriptions WHERE series_id='series-check'`).Scan(&last, &errorAt, &errorMessage); err != nil || !last.Equal(checkedAt) || errorAt != nil || errorMessage != nil {
|
||||||
|
t.Fatalf("成功時の日時・エラー消去: %v %v %v %v", last, errorAt, errorMessage, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
bad := seriesEpisode("bad", "series-check")
|
||||||
|
bad.DurationSeconds = -1
|
||||||
|
if _, err := s.CompleteSubscriptionCheck(ctx, "series-check", []Episode{seriesEpisode("rollback", "series-check"), bad}, 180, checkedAt.Add(time.Hour)); err == nil {
|
||||||
|
t.Fatal("不正な回を含む確認が成功した")
|
||||||
|
}
|
||||||
|
unknown, err = s.UnknownEpisodeIDs(ctx, "series-check", []string{"rollback"})
|
||||||
|
if err != nil || len(unknown) != 1 {
|
||||||
|
t.Fatalf("失敗時の既知記録を巻き戻さない: %v %v", unknown, err)
|
||||||
|
}
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id='series-check'`).Scan(&last); err != nil || !last.Equal(checkedAt) {
|
||||||
|
t.Fatalf("失敗時の最終確認を巻き戻さない: %v %v", last, err)
|
||||||
|
}
|
||||||
|
if err := s.SetSubscriptionEnabled(ctx, "series-check", false); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.CompleteSubscriptionCheck(ctx, "series-check", nil, 180, checkedAt.Add(time.Hour)); !errors.Is(err, ErrSubscriptionPaused) {
|
||||||
|
t.Fatalf("一時停止中の確認完了: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.RecordSubscriptionCheckFailure(ctx, "series-check", failedAt, "ignored"); !errors.Is(err, ErrInvalidTransition) {
|
||||||
|
t.Fatalf("一時停止中のエラー記録: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompleteSubscriptionCheckKeepsExistingManualJob(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
if created, err := s.RegisterSubscription(ctx, "series-manual", "番組", nil, false); err != nil || !created {
|
||||||
|
t.Fatalf("登録: %t %v", created, err)
|
||||||
|
}
|
||||||
|
episode := seriesEpisode("manual-existing", "series-manual")
|
||||||
|
manual, _, err := s.Enqueue(ctx, episode, "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
checkedAt := time.Date(2026, 10, 3, 4, 5, 6, 0, time.UTC)
|
||||||
|
queued, err := s.CompleteSubscriptionCheck(ctx, "series-manual", []Episode{episode, episode}, 180, checkedAt)
|
||||||
|
if err != nil || queued != 0 {
|
||||||
|
t.Fatalf("既存手動ジョブの確認: queued=%d err=%v", queued, err)
|
||||||
|
}
|
||||||
|
got, err := s.GetJob(ctx, manual.ID)
|
||||||
|
if err != nil || got.Source != "manual" {
|
||||||
|
t.Fatalf("既存手動ジョブを変更した: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
var jobs, known int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE episode_id=$1`, episode.ID).Scan(&jobs); err != nil || jobs != 1 {
|
||||||
|
t.Fatalf("ジョブが二重化した: %d %v", jobs, err)
|
||||||
|
}
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id=$1 AND episode_id=$2`, episode.SeriesID, episode.ID).Scan(&known); err != nil || known != 1 {
|
||||||
|
t.Fatalf("既知記録: %d %v", known, err)
|
||||||
|
}
|
||||||
|
var last time.Time
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT last_checked_at FROM subscriptions WHERE series_id=$1`, episode.SeriesID).Scan(&last); err != nil || !last.Equal(checkedAt) {
|
||||||
|
t.Fatalf("最終確認日時: %v %v", last, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func timePointer(value time.Time) *time.Time { return &value }
|
||||||
|
|
||||||
|
func seriesEpisode(id, seriesID string) Episode {
|
||||||
|
ep := sampleEpisode(id)
|
||||||
|
ep.SeriesID = seriesID
|
||||||
|
return ep
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionSnapshotAndRollback(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
episodes := []Episode{seriesEpisode("ep1", "series-1"), seriesEpisode("ep2", "series-1")}
|
||||||
|
created, err := s.RegisterSubscription(ctx, "series-1", "番組", episodes, false)
|
||||||
|
if err != nil || !created {
|
||||||
|
t.Fatalf("既存回をダウンロードしない登録: %t %v", created, err)
|
||||||
|
}
|
||||||
|
subs, err := s.ListSubscriptions(ctx)
|
||||||
|
if err != nil || len(subs) != 1 || subs[0].LastCheckedAt == nil || subs[0].LastCheckedAt.Before(subs[0].CreatedAt) {
|
||||||
|
t.Fatalf("登録時の最終確認日時: %+v %v", subs, err)
|
||||||
|
}
|
||||||
|
unknown, err := s.UnknownEpisodeIDs(ctx, "series-1", []string{"ep1", "ep2", "ep3"})
|
||||||
|
if err != nil || len(unknown) != 1 || unknown[0] != "ep3" {
|
||||||
|
t.Fatalf("既知の記録: %v %v", unknown, err)
|
||||||
|
}
|
||||||
|
var jobs int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
|
||||||
|
t.Fatalf("既存回が投入された: %d %v", jobs, err)
|
||||||
|
}
|
||||||
|
created, err = s.RegisterSubscription(ctx, "series-1", "別の名前", episodes, true)
|
||||||
|
if err != nil || created {
|
||||||
|
t.Fatalf("重複登録: %t %v", created, err)
|
||||||
|
}
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
|
||||||
|
t.Fatalf("重複登録が投入した: %d %v", jobs, err)
|
||||||
|
}
|
||||||
|
bad := seriesEpisode("bad", "series-bad")
|
||||||
|
bad.DurationSeconds = -1
|
||||||
|
created, err = s.RegisterSubscription(ctx, "series-bad", "番組", []Episode{seriesEpisode("good", "series-bad"), bad}, true)
|
||||||
|
if err == nil || created {
|
||||||
|
t.Fatalf("壊れた回の登録が成功: %t %v", created, err)
|
||||||
|
}
|
||||||
|
for _, table := range []string{"subscriptions", "known_episodes", "download_jobs"} {
|
||||||
|
var count int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM `+table+` WHERE series_id='series-bad'`).Scan(&count); err != nil || count != 0 {
|
||||||
|
t.Errorf("取引の巻き戻し %s: %d %v", table, count, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
deleted, err := s.DeleteSubscription(ctx, "series-1")
|
||||||
|
if err != nil || !deleted {
|
||||||
|
t.Fatalf("解除: %t %v", deleted, err)
|
||||||
|
}
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM known_episodes WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 0 {
|
||||||
|
t.Fatalf("解除後に既知が残った: %d %v", jobs, err)
|
||||||
|
}
|
||||||
|
created, err = s.RegisterSubscription(ctx, "series-1", "番組", episodes, true)
|
||||||
|
if err != nil || !created {
|
||||||
|
t.Fatalf("解除後の再登録: %t %v", created, err)
|
||||||
|
}
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-1'`).Scan(&jobs); err != nil || jobs != 2 {
|
||||||
|
t.Fatalf("再登録で既存回を選び直せない: %d %v", jobs, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterAndQueueUnseenEpisodes(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
manual, _, err := s.Enqueue(ctx, seriesEpisode("manual", "series-2"), "manual")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
created, err := s.RegisterSubscription(ctx, "series-2", "番組", []Episode{seriesEpisode("manual", "series-2"), seriesEpisode("past", "series-2")}, true)
|
||||||
|
if err != nil || !created {
|
||||||
|
t.Fatalf("公開中の回を投入: %t %v", created, err)
|
||||||
|
}
|
||||||
|
manualAfter, err := s.GetJob(ctx, manual.ID)
|
||||||
|
if err != nil || manualAfter.Source != "manual" {
|
||||||
|
t.Fatalf("既存の手動ジョブを変更: %+v %v", manualAfter, err)
|
||||||
|
}
|
||||||
|
var subscriptionJobs int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM download_jobs WHERE series_id='series-2' AND source='subscription' AND status='queued'`).Scan(&subscriptionJobs); err != nil || subscriptionJobs != 1 {
|
||||||
|
t.Fatalf("登録時に作ったジョブ: %d %v", subscriptionJobs, err)
|
||||||
|
}
|
||||||
|
subs, err := s.ListSubscriptions(ctx)
|
||||||
|
if err != nil || len(subs) != 1 || subs[0].JobCount != 2 || !subs[0].Enabled || subs[0].LastCheckedAt == nil {
|
||||||
|
t.Fatalf("登録一覧と関連ジョブ数: %+v %v", subs, err)
|
||||||
|
}
|
||||||
|
unknown, err := s.UnknownEpisodeIDs(ctx, "series-2", []string{"past", "new", "manual", "new"})
|
||||||
|
if err != nil || len(unknown) != 1 || unknown[0] != "new" {
|
||||||
|
t.Fatalf("未確認の抽出: %v %v", unknown, err)
|
||||||
|
}
|
||||||
|
if err := s.SetSubscriptionEnabled(ctx, "series-2", false); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("new", "series-2")}); !errors.Is(err, ErrSubscriptionPaused) {
|
||||||
|
t.Fatalf("一時停止中の投入: %v", err)
|
||||||
|
}
|
||||||
|
unknown, err = s.UnknownEpisodeIDs(ctx, "series-2", []string{"new"})
|
||||||
|
if err != nil || len(unknown) != 1 {
|
||||||
|
t.Fatalf("一時停止中に既知となった: %v %v", unknown, err)
|
||||||
|
}
|
||||||
|
if err := s.SetSubscriptionEnabled(ctx, "series-2", true); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
invalid := seriesEpisode("invalid", "series-2")
|
||||||
|
invalid.DurationSeconds = -1
|
||||||
|
if _, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("newer", "series-2"), invalid}); err == nil {
|
||||||
|
t.Fatal("新規回の投入が途中失敗でも確定した")
|
||||||
|
}
|
||||||
|
unknown, err = s.UnknownEpisodeIDs(ctx, "series-2", []string{"newer"})
|
||||||
|
if err != nil || len(unknown) != 1 {
|
||||||
|
t.Fatalf("新規回の投入失敗が巻き戻らない: %v %v", unknown, err)
|
||||||
|
}
|
||||||
|
count, err := s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("past", "series-2"), seriesEpisode("new", "series-2")})
|
||||||
|
if err != nil || count != 1 {
|
||||||
|
t.Fatalf("再開後の新しい回: %d %v", count, err)
|
||||||
|
}
|
||||||
|
count, err = s.QueueUnseenEpisodes(ctx, "series-2", []Episode{seriesEpisode("new", "series-2")})
|
||||||
|
if err != nil || count != 0 {
|
||||||
|
t.Fatalf("重複した新しい回: %d %v", count, err)
|
||||||
|
}
|
||||||
|
var source string
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT source FROM download_jobs WHERE episode_id='new'`).Scan(&source); err != nil || source != "subscription" {
|
||||||
|
t.Fatalf("新しい回の登録元: %q %v", source, err)
|
||||||
|
}
|
||||||
|
statuses, err := s.SubscribedSeries(ctx, []string{"series-2", "missing"})
|
||||||
|
if err != nil || !statuses["series-2"] || statuses["missing"] {
|
||||||
|
t.Fatalf("登録済みシリーズの一括取得: %v %v", statuses, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueueUnseenEpisodesConcurrent(t *testing.T) {
|
||||||
|
pool := testPool(t, true)
|
||||||
|
s := New(pool)
|
||||||
|
ctx := context.Background()
|
||||||
|
created, err := s.RegisterSubscription(ctx, "series-concurrent", "番組", nil, false)
|
||||||
|
if err != nil || !created {
|
||||||
|
t.Fatalf("登録: %t %v", created, err)
|
||||||
|
}
|
||||||
|
episodes := []Episode{
|
||||||
|
seriesEpisode("concurrent-1", "series-concurrent"),
|
||||||
|
seriesEpisode("concurrent-2", "series-concurrent"),
|
||||||
|
}
|
||||||
|
const workers = 8
|
||||||
|
counts := make(chan int, workers)
|
||||||
|
errs := make(chan error, workers)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for range workers {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
count, err := s.QueueUnseenEpisodes(ctx, "series-concurrent", episodes)
|
||||||
|
counts <- count
|
||||||
|
errs <- err
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
close(counts)
|
||||||
|
close(errs)
|
||||||
|
for err := range errs {
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("同時投入: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
total := 0
|
||||||
|
positive := 0
|
||||||
|
for count := range counts {
|
||||||
|
total += count
|
||||||
|
if count > 0 {
|
||||||
|
positive++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if total != 2 || positive != 1 {
|
||||||
|
t.Fatalf("同時投入の記録件数: total=%d positive workers=%d", total, positive)
|
||||||
|
}
|
||||||
|
for table := range map[string]struct{}{"known_episodes": {}, "download_jobs": {}} {
|
||||||
|
var count int
|
||||||
|
if err := pool.QueryRow(ctx, `SELECT count(*) FROM `+table+` WHERE series_id='series-concurrent'`).Scan(&count); err != nil || count != 2 {
|
||||||
|
t.Errorf("%s の重複防止: %d %v", table, count, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,171 @@
|
|||||||
|
package tver
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
DefaultBaseURL = "https://platform-api.tver.jp"
|
||||||
|
PreviewMaxSeconds = 180
|
||||||
|
)
|
||||||
|
|
||||||
|
type Session struct {
|
||||||
|
PlatformUID string `json:"platform_uid"`
|
||||||
|
PlatformToken string `json:"platform_token"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Episode struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
SeriesID string `json:"seriesID"`
|
||||||
|
SeriesTitle string `json:"seriesTitle"`
|
||||||
|
Title string `json:"title"`
|
||||||
|
BroadcasterName string `json:"broadcasterName"`
|
||||||
|
Duration int64 `json:"duration"`
|
||||||
|
EndAt int64 `json:"endAt"`
|
||||||
|
ThumbnailPath string `json:"thumbnailPath"`
|
||||||
|
BroadcastDateLabel string `json:"broadcastDateLabel"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Client struct {
|
||||||
|
baseURL *url.URL
|
||||||
|
serviceURL *url.URL
|
||||||
|
httpClient *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
type httpStatusError struct {
|
||||||
|
statusCode int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *httpStatusError) Error() string {
|
||||||
|
return fmt.Sprintf("HTTP %d", e.statusCode)
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewClient はテスト用に API の接続先と HTTP クライアントを差し替えられる。
|
||||||
|
// 接続先が空、またはクライアントが nil の場合は本番用の既定値を使う。
|
||||||
|
func NewClient(baseURL string, httpClient *http.Client) (*Client, error) {
|
||||||
|
defaultAPI := baseURL == ""
|
||||||
|
if baseURL == "" {
|
||||||
|
baseURL = DefaultBaseURL
|
||||||
|
}
|
||||||
|
parsed, err := url.Parse(baseURL)
|
||||||
|
if err != nil || (parsed.Scheme != "https" && parsed.Scheme != "http") || parsed.Host == "" || parsed.RawQuery != "" || parsed.Fragment != "" {
|
||||||
|
return nil, fmt.Errorf("invalid TVer API base URL")
|
||||||
|
}
|
||||||
|
if httpClient == nil {
|
||||||
|
httpClient = &http.Client{Timeout: 15 * time.Second}
|
||||||
|
}
|
||||||
|
service := parsed
|
||||||
|
if defaultAPI {
|
||||||
|
service = &url.URL{Scheme: "https", Host: "service-api.tver.jp"}
|
||||||
|
}
|
||||||
|
return &Client{baseURL: parsed, serviceURL: service, httpClient: httpClient}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) CreateSession(ctx context.Context) (Session, error) {
|
||||||
|
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/v2/api/platform_users/browser/create"})
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint.String(), strings.NewReader("device_type=pc"))
|
||||||
|
if err != nil {
|
||||||
|
return Session{}, fmt.Errorf("create session request: %w", err)
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||||
|
var response struct {
|
||||||
|
Code int `json:"code"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
Result Session `json:"result"`
|
||||||
|
}
|
||||||
|
if err := c.getJSON(req, &response); err != nil {
|
||||||
|
return Session{}, fmt.Errorf("create session: %w", err)
|
||||||
|
}
|
||||||
|
if response.Code != 0 || response.Result.PlatformUID == "" || response.Result.PlatformToken == "" {
|
||||||
|
return Session{}, fmt.Errorf("create session: invalid API result (code %d)", response.Code)
|
||||||
|
}
|
||||||
|
return response.Result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) ListNewer(ctx context.Context, category string, session Session) ([]Episode, error) {
|
||||||
|
if category != "drama" && category != "variety" {
|
||||||
|
return nil, fmt.Errorf("unsupported category %q", category)
|
||||||
|
}
|
||||||
|
if session.PlatformUID == "" || session.PlatformToken == "" {
|
||||||
|
return nil, errors.New("missing TVer session credentials")
|
||||||
|
}
|
||||||
|
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v1/callNewerDetail/" + category})
|
||||||
|
query := endpoint.Query()
|
||||||
|
query.Set("platform_uid", session.PlatformUID)
|
||||||
|
query.Set("platform_token", session.PlatformToken)
|
||||||
|
endpoint.RawQuery = query.Encode()
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("newer list request: %w", err)
|
||||||
|
}
|
||||||
|
req.Header.Set("x-tver-platform-type", "web")
|
||||||
|
req.Header.Set("Origin", "https://tver.jp")
|
||||||
|
req.Header.Set("Referer", "https://tver.jp/")
|
||||||
|
var response struct {
|
||||||
|
Code int `json:"code"`
|
||||||
|
Result struct {
|
||||||
|
Contents *struct {
|
||||||
|
Contents []struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
Content Episode `json:"content"`
|
||||||
|
} `json:"contents"`
|
||||||
|
} `json:"contents"`
|
||||||
|
} `json:"result"`
|
||||||
|
}
|
||||||
|
if err := c.getJSON(req, &response); err != nil {
|
||||||
|
return nil, fmt.Errorf("newer list: %w", err)
|
||||||
|
}
|
||||||
|
if response.Code != 0 {
|
||||||
|
return nil, fmt.Errorf("newer list: API code %d", response.Code)
|
||||||
|
}
|
||||||
|
if response.Result.Contents == nil || response.Result.Contents.Contents == nil {
|
||||||
|
return nil, errors.New("newer list: missing contents")
|
||||||
|
}
|
||||||
|
var episodes []Episode
|
||||||
|
for _, item := range response.Result.Contents.Contents {
|
||||||
|
if item.Type != "episode" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if item.Content.ID == "" {
|
||||||
|
return nil, errors.New("newer list: episode without id")
|
||||||
|
}
|
||||||
|
episodes = append(episodes, item.Content)
|
||||||
|
}
|
||||||
|
return episodes, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) FetchNewer(ctx context.Context, category string) ([]Episode, error) {
|
||||||
|
session, err := c.CreateSession(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return c.ListNewer(ctx, category, session)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) getJSON(req *http.Request, dst any) error {
|
||||||
|
response, err := c.httpClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
// url.Error には platform_token を含むリクエスト URL が入るため、内部のエラーだけを返す。
|
||||||
|
var urlErr *url.Error
|
||||||
|
if errors.As(err, &urlErr) {
|
||||||
|
err = urlErr.Err
|
||||||
|
}
|
||||||
|
return fmt.Errorf("request failed: %w", err)
|
||||||
|
}
|
||||||
|
defer response.Body.Close()
|
||||||
|
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
||||||
|
return &httpStatusError{statusCode: response.StatusCode}
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(io.LimitReader(response.Body, 8<<20)).Decode(dst); err != nil {
|
||||||
|
return fmt.Errorf("decode JSON: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
package tver
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type localOnlyTransport struct {
|
||||||
|
host string
|
||||||
|
base http.RoundTripper
|
||||||
|
}
|
||||||
|
|
||||||
|
func (tr localOnlyTransport) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||||
|
if req.URL.Scheme != "http" || req.URL.Host != tr.host {
|
||||||
|
return nil, fmt.Errorf("test attempted external request to %s", req.URL.Host)
|
||||||
|
}
|
||||||
|
return tr.base.RoundTrip(req)
|
||||||
|
}
|
||||||
|
|
||||||
|
func testClient(t *testing.T, handler http.HandlerFunc) *Client {
|
||||||
|
t.Helper()
|
||||||
|
server := httptest.NewServer(handler)
|
||||||
|
t.Cleanup(server.Close)
|
||||||
|
serverURL, err := url.Parse(server.URL)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
httpClient := server.Client()
|
||||||
|
baseTransport := httpClient.Transport
|
||||||
|
if baseTransport == nil {
|
||||||
|
baseTransport = http.DefaultTransport
|
||||||
|
}
|
||||||
|
httpClient.Transport = localOnlyTransport{host: serverURL.Host, base: baseTransport}
|
||||||
|
client, err := NewClient(server.URL, httpClient)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return client
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTestClientRejectsExternalRequests(t *testing.T) {
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
t.Errorf("unexpected local request to %s", r.URL)
|
||||||
|
})
|
||||||
|
_, err := client.httpClient.Get(DefaultBaseURL)
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "test attempted external request") {
|
||||||
|
t.Errorf("external request error = %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchNewerFromRecordedResponse(t *testing.T) {
|
||||||
|
fixture, err := os.ReadFile("testdata/newer_drama.json")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var sessionCalls, newerCalls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/v2/api/platform_users/browser/create":
|
||||||
|
sessionCalls++
|
||||||
|
if r.Method != http.MethodPost || r.Header.Get("Content-Type") != "application/x-www-form-urlencoded" {
|
||||||
|
t.Errorf("session request method/Content-Type: %s %s", r.Method, r.Header.Get("Content-Type"))
|
||||||
|
}
|
||||||
|
if err := r.ParseForm(); err != nil || r.PostForm.Get("device_type") != "pc" {
|
||||||
|
t.Errorf("session form: %v, %v", r.PostForm, err)
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
|
||||||
|
case "/service/api/v1/callNewerDetail/drama":
|
||||||
|
newerCalls++
|
||||||
|
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" {
|
||||||
|
t.Errorf("newer request method/query: %s %v", r.Method, r.URL.Query())
|
||||||
|
}
|
||||||
|
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
|
||||||
|
if got := r.Header.Get(name); got != want {
|
||||||
|
t.Errorf("header %s = %q; want %q", name, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, _ = w.Write(fixture)
|
||||||
|
default:
|
||||||
|
t.Errorf("unexpected path %s", r.URL.Path)
|
||||||
|
http.NotFound(w, r)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
episodes, err := client.FetchNewer(context.Background(), "drama")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if sessionCalls != 1 || newerCalls != 1 {
|
||||||
|
t.Fatalf("calls: session=%d newer=%d", sessionCalls, newerCalls)
|
||||||
|
}
|
||||||
|
if len(episodes) != 3 {
|
||||||
|
t.Fatalf("episodes = %d; want 3", len(episodes))
|
||||||
|
}
|
||||||
|
want := Episode{
|
||||||
|
ID: "eps0z5761a",
|
||||||
|
SeriesID: "sr0lhdrtdb",
|
||||||
|
SeriesTitle: "韓国ドラマ「良いが悪い、ドンジェ」",
|
||||||
|
Title: "#10（最終話）",
|
||||||
|
BroadcasterName: "BS12トゥエルビ",
|
||||||
|
Duration: 2712,
|
||||||
|
EndAt: 1792209540,
|
||||||
|
ThumbnailPath: "/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51",
|
||||||
|
BroadcastDateLabel: "10月3日(土)放送分",
|
||||||
|
}
|
||||||
|
if episodes[0] != want {
|
||||||
|
t.Errorf("first episode = %+v; want %+v", episodes[0], want)
|
||||||
|
}
|
||||||
|
if episodes[2].Duration != 60 || episodes[2].BroadcastDateLabel != "" {
|
||||||
|
t.Errorf("third episode = %+v", episodes[2])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListNewerVarietySkipsOtherTypes(t *testing.T) {
|
||||||
|
var calls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
calls++
|
||||||
|
if r.URL.Path != "/service/api/v1/callNewerDetail/variety" {
|
||||||
|
t.Errorf("path = %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "u" || r.URL.Query().Get("platform_token") != "t" {
|
||||||
|
t.Errorf("variety request method/query: %s %v", r.Method, r.URL.Query())
|
||||||
|
}
|
||||||
|
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
|
||||||
|
if got := r.Header.Get(name); got != want {
|
||||||
|
t.Errorf("variety header %s = %q; want %q", name, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":{"contents":[{"type":"series","content":{"id":"series-1"}},{"type":"episode","content":{"id":"episode-1","duration":42}}]}}}`))
|
||||||
|
})
|
||||||
|
episodes, err := client.ListNewer(context.Background(), "variety", Session{PlatformUID: "u", PlatformToken: "t"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(episodes) != 1 || episodes[0].ID != "episode-1" || episodes[0].Duration != 42 {
|
||||||
|
t.Errorf("episodes = %+v", episodes)
|
||||||
|
}
|
||||||
|
if calls != 1 {
|
||||||
|
t.Errorf("variety calls = %d; want 1", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchNewerErrors(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
sessionStatus int
|
||||||
|
sessionBody string
|
||||||
|
newerStatus int
|
||||||
|
newerBody string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"session HTTP error", 503, "", 200, "", "create session: HTTP 503"},
|
||||||
|
{"session invalid JSON", 200, "{", 200, "", "create session: decode JSON"},
|
||||||
|
{"session API error", 200, `{"code":7,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, "", "create session: invalid API result"},
|
||||||
|
{"session missing credentials", 200, `{"code":0,"result":{}}`, 200, "", "create session: invalid API result"},
|
||||||
|
{"session missing token", 200, `{"code":0,"result":{"platform_uid":"u"}}`, 200, "", "create session: invalid API result"},
|
||||||
|
{"session missing uid", 200, `{"code":0,"result":{"platform_token":"t"}}`, 200, "", "create session: invalid API result"},
|
||||||
|
{"newer HTTP error", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 502, "", "newer list: HTTP 502"},
|
||||||
|
{"newer invalid JSON", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, "{", "newer list: decode JSON"},
|
||||||
|
{"newer API error", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":7}`, "newer list: API code 7"},
|
||||||
|
{"newer missing contents", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{}}`, "newer list: missing contents"},
|
||||||
|
{"newer missing items", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{"contents":{}}}`, "newer list: missing contents"},
|
||||||
|
{"episode missing id", 200, `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`, 200, `{"code":0,"result":{"contents":{"contents":[{"type":"episode","content":{"title":"untitled"}}]}}}`, "newer list: episode without id"},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var sessionCalls, newerCalls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/v2/api/platform_users/browser/create":
|
||||||
|
sessionCalls++
|
||||||
|
w.WriteHeader(tt.sessionStatus)
|
||||||
|
_, _ = w.Write([]byte(tt.sessionBody))
|
||||||
|
case "/service/api/v1/callNewerDetail/drama":
|
||||||
|
newerCalls++
|
||||||
|
w.WriteHeader(tt.newerStatus)
|
||||||
|
_, _ = w.Write([]byte(tt.newerBody))
|
||||||
|
default:
|
||||||
|
t.Errorf("unexpected path %s", r.URL.Path)
|
||||||
|
http.NotFound(w, r)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
_, err := client.FetchNewer(context.Background(), "drama")
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.want) {
|
||||||
|
t.Errorf("error = %v; want %q", err, tt.want)
|
||||||
|
}
|
||||||
|
wantNewerCalls := 1
|
||||||
|
if strings.HasPrefix(tt.name, "session ") {
|
||||||
|
wantNewerCalls = 0
|
||||||
|
}
|
||||||
|
if sessionCalls != 1 || newerCalls != wantNewerCalls {
|
||||||
|
t.Errorf("calls: session=%d newer=%d; want session=1 newer=%d", sessionCalls, newerCalls, wantNewerCalls)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListNewerRejectsInvalidInput(t *testing.T) {
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
t.Errorf("unexpected request to %s", r.URL)
|
||||||
|
})
|
||||||
|
if _, err := client.ListNewer(context.Background(), "news", Session{PlatformUID: "u", PlatformToken: "t"}); err == nil {
|
||||||
|
t.Error("unsupported category accepted")
|
||||||
|
}
|
||||||
|
if _, err := client.ListNewer(context.Background(), "drama", Session{}); err == nil {
|
||||||
|
t.Error("empty session accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
package tver
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"regexp"
|
||||||
|
)
|
||||||
|
|
||||||
|
var tverSeriesIDPattern = regexp.MustCompile(`^[A-Za-z0-9]+$`)
|
||||||
|
|
||||||
|
// ErrSeriesNotFound は callSeries が HTTP 404 を返したことを表す。
|
||||||
|
var ErrSeriesNotFound = errors.New("series not found")
|
||||||
|
|
||||||
|
type seriesContents struct {
|
||||||
|
Code int `json:"code"`
|
||||||
|
Result struct {
|
||||||
|
Contents []struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
Content Episode `json:"content"`
|
||||||
|
} `json:"contents"`
|
||||||
|
} `json:"result"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) FetchSeries(ctx context.Context, seriesID string) (string, error) {
|
||||||
|
if !tverSeriesIDPattern.MatchString(seriesID) {
|
||||||
|
return "", errors.New("invalid series id")
|
||||||
|
}
|
||||||
|
session, err := c.CreateSession(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v2/callSeries/" + seriesID})
|
||||||
|
query := endpoint.Query()
|
||||||
|
query.Set("platform_uid", session.PlatformUID)
|
||||||
|
query.Set("platform_token", session.PlatformToken)
|
||||||
|
endpoint.RawQuery = query.Encode()
|
||||||
|
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("series request: %w", err)
|
||||||
|
}
|
||||||
|
setTVerHeaders(request)
|
||||||
|
var response struct {
|
||||||
|
Code int `json:"code"`
|
||||||
|
Result struct {
|
||||||
|
Content struct {
|
||||||
|
Content struct {
|
||||||
|
Title string `json:"title"`
|
||||||
|
} `json:"content"`
|
||||||
|
} `json:"content"`
|
||||||
|
} `json:"result"`
|
||||||
|
}
|
||||||
|
if err := c.getJSON(request, &response); err != nil {
|
||||||
|
var statusErr *httpStatusError
|
||||||
|
if errors.As(err, &statusErr) && statusErr.statusCode == http.StatusNotFound {
|
||||||
|
return "", ErrSeriesNotFound
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("series: %w", err)
|
||||||
|
}
|
||||||
|
if response.Code != 0 || response.Result.Content.Content.Title == "" {
|
||||||
|
return "", fmt.Errorf("series: invalid API result (code %d)", response.Code)
|
||||||
|
}
|
||||||
|
return response.Result.Content.Content.Title, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *Client) FetchSeriesEpisodes(ctx context.Context, seriesID string) ([]Episode, error) {
|
||||||
|
if !tverSeriesIDPattern.MatchString(seriesID) {
|
||||||
|
return nil, errors.New("invalid series id")
|
||||||
|
}
|
||||||
|
seasonsURL := c.serviceURL.ResolveReference(&url.URL{Path: "/api/v1/callSeriesSeasons/" + seriesID})
|
||||||
|
request, err := http.NewRequestWithContext(ctx, http.MethodGet, seasonsURL.String(), nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("series seasons request: %w", err)
|
||||||
|
}
|
||||||
|
setTVerHeaders(request)
|
||||||
|
var seasons seriesContents
|
||||||
|
if err := c.getJSON(request, &seasons); err != nil {
|
||||||
|
return nil, fmt.Errorf("series seasons: %w", err)
|
||||||
|
}
|
||||||
|
if seasons.Code != 0 || seasons.Result.Contents == nil {
|
||||||
|
return nil, fmt.Errorf("series seasons: invalid API result (code %d)", seasons.Code)
|
||||||
|
}
|
||||||
|
var seasonIDs []string
|
||||||
|
for _, item := range seasons.Result.Contents {
|
||||||
|
if item.Type != "season" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !tverSeriesIDPattern.MatchString(item.Content.ID) {
|
||||||
|
return nil, errors.New("series seasons: invalid season id")
|
||||||
|
}
|
||||||
|
seasonIDs = append(seasonIDs, item.Content.ID)
|
||||||
|
}
|
||||||
|
if len(seasonIDs) == 0 {
|
||||||
|
return []Episode{}, nil
|
||||||
|
}
|
||||||
|
session, err := c.CreateSession(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
seen := make(map[string]bool)
|
||||||
|
var episodes []Episode
|
||||||
|
for _, seasonID := range seasonIDs {
|
||||||
|
endpoint := c.baseURL.ResolveReference(&url.URL{Path: "/service/api/v1/callSeasonEpisodes/" + seasonID})
|
||||||
|
query := endpoint.Query()
|
||||||
|
query.Set("platform_uid", session.PlatformUID)
|
||||||
|
query.Set("platform_token", session.PlatformToken)
|
||||||
|
endpoint.RawQuery = query.Encode()
|
||||||
|
request, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint.String(), nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("season episodes request: %w", err)
|
||||||
|
}
|
||||||
|
setTVerHeaders(request)
|
||||||
|
var response seriesContents
|
||||||
|
if err := c.getJSON(request, &response); err != nil {
|
||||||
|
return nil, fmt.Errorf("season episodes: %w", err)
|
||||||
|
}
|
||||||
|
if response.Code != 0 || response.Result.Contents == nil {
|
||||||
|
return nil, fmt.Errorf("season episodes: invalid API result (code %d)", response.Code)
|
||||||
|
}
|
||||||
|
for _, item := range response.Result.Contents {
|
||||||
|
if item.Type != "episode" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ep := item.Content
|
||||||
|
if !tverSeriesIDPattern.MatchString(ep.ID) || ep.SeriesID != seriesID {
|
||||||
|
return nil, errors.New("season episodes: invalid episode identity")
|
||||||
|
}
|
||||||
|
if !seen[ep.ID] {
|
||||||
|
episodes = append(episodes, ep)
|
||||||
|
seen[ep.ID] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return episodes, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func setTVerHeaders(request *http.Request) {
|
||||||
|
request.Header.Set("x-tver-platform-type", "web")
|
||||||
|
request.Header.Set("Origin", "https://tver.jp")
|
||||||
|
request.Header.Set("Referer", "https://tver.jp/")
|
||||||
|
}
|
||||||
@@ -0,0 +1,305 @@
|
|||||||
|
package tver
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestFetchSeriesEpisodesFromRecordedResponses(t *testing.T) {
|
||||||
|
seasons, err := os.ReadFile("testdata/series_seasons.json")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
episodes, err := os.ReadFile("testdata/season_episodes.json")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var seasonCalls, sessionCalls, episodeCalls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/api/v1/callSeriesSeasons/sr0lhdrtdb":
|
||||||
|
seasonCalls++
|
||||||
|
if r.Method != http.MethodGet || r.Header.Get("x-tver-platform-type") != "web" || r.Header.Get("Origin") != "https://tver.jp" || r.Header.Get("Referer") != "https://tver.jp/" {
|
||||||
|
t.Errorf("seasons request: method=%s headers=%v", r.Method, r.Header)
|
||||||
|
}
|
||||||
|
_, _ = w.Write(seasons)
|
||||||
|
case "/v2/api/platform_users/browser/create":
|
||||||
|
sessionCalls++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
|
||||||
|
case "/service/api/v1/callSeasonEpisodes/ss3i6njoq3":
|
||||||
|
episodeCalls++
|
||||||
|
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" || r.Header.Get("x-tver-platform-type") != "web" || r.Header.Get("Origin") != "https://tver.jp" || r.Header.Get("Referer") != "https://tver.jp/" {
|
||||||
|
t.Errorf("season episode request: method=%s query=%v headers=%v", r.Method, r.URL.Query(), r.Header)
|
||||||
|
}
|
||||||
|
_, _ = w.Write(episodes)
|
||||||
|
default:
|
||||||
|
t.Errorf("unexpected path: %s", r.URL.Path)
|
||||||
|
http.NotFound(w, r)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
got, err := client.FetchSeriesEpisodes(context.Background(), "sr0lhdrtdb")
|
||||||
|
if err != nil || len(got) != 1 {
|
||||||
|
t.Fatalf("series episodes: %+v %v", got, err)
|
||||||
|
}
|
||||||
|
ep := got[0]
|
||||||
|
if ep.ID != "eps0z5761a" || ep.SeriesID != "sr0lhdrtdb" || ep.Title != "#10（最終話）" || ep.SeriesTitle != "韓国ドラマ「良いが悪い、ドンジェ」" || ep.BroadcasterName != "BS12トゥエルビ" || ep.Duration != 2712 || ep.EndAt != 1792209540 || !strings.Contains(ep.ThumbnailPath, "eps0z5761a.jpg") {
|
||||||
|
t.Errorf("episode fields: %+v", ep)
|
||||||
|
}
|
||||||
|
if seasonCalls != 1 || sessionCalls != 1 || episodeCalls != 1 {
|
||||||
|
t.Errorf("calls: seasons=%d session=%d episodes=%d", seasonCalls, sessionCalls, episodeCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesEpisodesTransportAndDecodeErrors(t *testing.T) {
|
||||||
|
validSeasons := `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`
|
||||||
|
validSession := `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`
|
||||||
|
validEpisodes := `{"code":0,"result":{"contents":[]}}`
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name string
|
||||||
|
seasonStatus, sessionStatus, episodeStatus int
|
||||||
|
seasonBody, sessionBody, episodeBody string
|
||||||
|
want string
|
||||||
|
wantSession, wantEpisode int
|
||||||
|
}{
|
||||||
|
{"seasons HTTP", 503, 200, 200, "", validSession, validEpisodes, "series seasons: HTTP 503", 0, 0},
|
||||||
|
{"seasons JSON", 200, 200, 200, "{", validSession, validEpisodes, "series seasons: decode JSON", 0, 0},
|
||||||
|
{"session HTTP", 200, 503, 200, validSeasons, "", validEpisodes, "create session: HTTP 503", 1, 0},
|
||||||
|
{"episodes HTTP", 200, 200, 502, validSeasons, validSession, "", "season episodes: HTTP 502", 1, 1},
|
||||||
|
{"episodes JSON", 200, 200, 200, validSeasons, validSession, "{", "season episodes: decode JSON", 1, 1},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var seasonCalls, sessionCalls, episodeCalls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/api/v1/callSeriesSeasons/sr1":
|
||||||
|
seasonCalls++
|
||||||
|
w.WriteHeader(tt.seasonStatus)
|
||||||
|
_, _ = w.Write([]byte(tt.seasonBody))
|
||||||
|
case "/v2/api/platform_users/browser/create":
|
||||||
|
sessionCalls++
|
||||||
|
w.WriteHeader(tt.sessionStatus)
|
||||||
|
_, _ = w.Write([]byte(tt.sessionBody))
|
||||||
|
case "/service/api/v1/callSeasonEpisodes/ss1":
|
||||||
|
episodeCalls++
|
||||||
|
w.WriteHeader(tt.episodeStatus)
|
||||||
|
_, _ = w.Write([]byte(tt.episodeBody))
|
||||||
|
default:
|
||||||
|
t.Errorf("unexpected request: %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
_, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.want) {
|
||||||
|
t.Fatalf("error=%v; want %q", err, tt.want)
|
||||||
|
}
|
||||||
|
if seasonCalls != 1 || sessionCalls != tt.wantSession || episodeCalls != tt.wantEpisode {
|
||||||
|
t.Errorf("calls: seasons=%d session=%d episodes=%d", seasonCalls, sessionCalls, episodeCalls)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesEpisodesRejectsBadResponses(t *testing.T) {
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name, seasons, episodes string
|
||||||
|
}{
|
||||||
|
{"season API code", `{"code":1,"result":{"contents":[]}}`, ""},
|
||||||
|
{"missing seasons", `{"code":0,"result":{}}`, ""},
|
||||||
|
{"season without id", `{"code":0,"result":{"contents":[{"type":"season","content":{}}]}}`, ""},
|
||||||
|
{"episode API code", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":1,"result":{"contents":[]}}`},
|
||||||
|
{"wrong series", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"other"}}]}}`},
|
||||||
|
{"episode without id", `{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}}]}}`, `{"code":0,"result":{"contents":[{"type":"episode","content":{"seriesID":"sr1"}}]}}`},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch {
|
||||||
|
case strings.Contains(r.URL.Path, "callSeriesSeasons"):
|
||||||
|
_, _ = w.Write([]byte(tt.seasons))
|
||||||
|
case strings.Contains(r.URL.Path, "browser/create"):
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
|
||||||
|
default:
|
||||||
|
_, _ = w.Write([]byte(tt.episodes))
|
||||||
|
}
|
||||||
|
})
|
||||||
|
if _, err := client.FetchSeriesEpisodes(context.Background(), "sr1"); err == nil {
|
||||||
|
t.Fatal("invalid response accepted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesEpisodesMultipleSeasonsDeduplicates(t *testing.T) {
|
||||||
|
var sessionCalls, episodeCalls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/api/v1/callSeriesSeasons/sr1":
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss1"}},{"type":"season","content":{"id":"ss2"}}]}}`))
|
||||||
|
case "/v2/api/platform_users/browser/create":
|
||||||
|
sessionCalls++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
|
||||||
|
case "/service/api/v1/callSeasonEpisodes/ss1":
|
||||||
|
episodeCalls++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"sr1"}}]}}`))
|
||||||
|
case "/service/api/v1/callSeasonEpisodes/ss2":
|
||||||
|
episodeCalls++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"ep1","seriesID":"sr1"}},{"type":"episode","content":{"id":"ep2","seriesID":"sr1"}}]}}`))
|
||||||
|
default:
|
||||||
|
t.Errorf("unexpected request: %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
episodes, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
|
||||||
|
if err != nil || len(episodes) != 2 || episodes[0].ID != "ep1" || episodes[1].ID != "ep2" || sessionCalls != 1 || episodeCalls != 2 {
|
||||||
|
t.Fatalf("複数 season: %+v %v session=%d episode=%d", episodes, err, sessionCalls, episodeCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesEpisodesNoSeasonsSkipsSession(t *testing.T) {
|
||||||
|
var calls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
calls++
|
||||||
|
if r.URL.Path != "/api/v1/callSeriesSeasons/sr1" {
|
||||||
|
t.Errorf("unexpected request: %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"contents":[]}}`))
|
||||||
|
})
|
||||||
|
episodes, err := client.FetchSeriesEpisodes(context.Background(), "sr1")
|
||||||
|
if err != nil || len(episodes) != 0 || calls != 1 {
|
||||||
|
t.Fatalf("season 無し: %+v %v calls=%d", episodes, err, calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesEpisodesRejectsInvalidSeriesIDBeforeRequest(t *testing.T) {
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
t.Errorf("unexpected request: %s", r.URL.Path)
|
||||||
|
})
|
||||||
|
for _, id := range []string{"", "../other", "sr1?x=1"} {
|
||||||
|
if _, err := client.FetchSeriesEpisodes(context.Background(), id); err == nil {
|
||||||
|
t.Errorf("invalid series id accepted: %q", id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesFromRecordedResponse(t *testing.T) {
|
||||||
|
fixture, err := os.ReadFile("testdata/series.json")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var sessionCalls, seriesCalls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/v2/api/platform_users/browser/create":
|
||||||
|
sessionCalls++
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"uid-test","platform_token":"token-test"}}`))
|
||||||
|
case "/service/api/v2/callSeries/sr0lhdrtdb":
|
||||||
|
seriesCalls++
|
||||||
|
if r.Method != http.MethodGet || r.URL.Query().Get("platform_uid") != "uid-test" || r.URL.Query().Get("platform_token") != "token-test" {
|
||||||
|
t.Errorf("series request method/query: %s %v", r.Method, r.URL.Query())
|
||||||
|
}
|
||||||
|
for name, want := range map[string]string{"x-tver-platform-type": "web", "Origin": "https://tver.jp", "Referer": "https://tver.jp/"} {
|
||||||
|
if got := r.Header.Get(name); got != want {
|
||||||
|
t.Errorf("series header %s = %q; want %q", name, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, _ = w.Write(fixture)
|
||||||
|
default:
|
||||||
|
t.Errorf("unexpected request: %s", r.URL.Path)
|
||||||
|
http.NotFound(w, r)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
title, err := client.FetchSeries(context.Background(), "sr0lhdrtdb")
|
||||||
|
if err != nil || title != "韓国ドラマ「良いが悪い、ドンジェ」" {
|
||||||
|
t.Fatalf("series: title=%q error=%v", title, err)
|
||||||
|
}
|
||||||
|
if sessionCalls != 1 || seriesCalls != 1 {
|
||||||
|
t.Fatalf("calls: session=%d series=%d", sessionCalls, seriesCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesNotFound(t *testing.T) {
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/v2/api/platform_users/browser/create" {
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
http.NotFound(w, r)
|
||||||
|
})
|
||||||
|
_, err := client.FetchSeries(context.Background(), "missing")
|
||||||
|
if !errors.Is(err, ErrSeriesNotFound) {
|
||||||
|
t.Fatalf("error=%v; want ErrSeriesNotFound", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesTransportAndDecodeErrors(t *testing.T) {
|
||||||
|
validSession := `{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name string
|
||||||
|
sessionStatus, seriesStatus int
|
||||||
|
sessionBody, seriesBody string
|
||||||
|
want string
|
||||||
|
wantSeriesCalls int
|
||||||
|
}{
|
||||||
|
{"session HTTP is not series not found", http.StatusNotFound, http.StatusOK, "", "", "create session: HTTP 404", 0},
|
||||||
|
{"series HTTP", http.StatusOK, http.StatusBadGateway, validSession, "", "series: HTTP 502", 1},
|
||||||
|
{"series JSON", http.StatusOK, http.StatusOK, validSession, "{", "series: decode JSON", 1},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var seriesCalls int
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
switch r.URL.Path {
|
||||||
|
case "/v2/api/platform_users/browser/create":
|
||||||
|
w.WriteHeader(tt.sessionStatus)
|
||||||
|
_, _ = w.Write([]byte(tt.sessionBody))
|
||||||
|
case "/service/api/v2/callSeries/sr1":
|
||||||
|
seriesCalls++
|
||||||
|
w.WriteHeader(tt.seriesStatus)
|
||||||
|
_, _ = w.Write([]byte(tt.seriesBody))
|
||||||
|
default:
|
||||||
|
t.Errorf("unexpected request: %s", r.URL.Path)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
_, err := client.FetchSeries(context.Background(), "sr1")
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.want) || errors.Is(err, ErrSeriesNotFound) {
|
||||||
|
t.Fatalf("error=%v; want %q and not ErrSeriesNotFound", err, tt.want)
|
||||||
|
}
|
||||||
|
if seriesCalls != tt.wantSeriesCalls {
|
||||||
|
t.Fatalf("series calls=%d; want %d", seriesCalls, tt.wantSeriesCalls)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesRejectsBadResponses(t *testing.T) {
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name, body string
|
||||||
|
}{
|
||||||
|
{"empty title", `{"code":0,"result":{"content":{"content":{"title":""}}}}`},
|
||||||
|
{"API code", `{"code":7,"result":{"content":{"content":{"title":"番組"}}}}`},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/v2/api/platform_users/browser/create" {
|
||||||
|
_, _ = w.Write([]byte(`{"code":0,"result":{"platform_uid":"u","platform_token":"t"}}`))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(tt.body))
|
||||||
|
})
|
||||||
|
if _, err := client.FetchSeries(context.Background(), "sr1"); err == nil {
|
||||||
|
t.Fatal("invalid response accepted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFetchSeriesRejectsInvalidSeriesIDBeforeRequest(t *testing.T) {
|
||||||
|
client := testClient(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
t.Errorf("unexpected request: %s", r.URL.Path)
|
||||||
|
})
|
||||||
|
for _, id := range []string{"", "../other", "sr1?x=1", "sr_1"} {
|
||||||
|
if _, err := client.FetchSeries(context.Background(), id); err == nil {
|
||||||
|
t.Errorf("invalid series id accepted: %q", id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+91
@@ -0,0 +1,91 @@
|
|||||||
|
{
|
||||||
|
"api_version": "",
|
||||||
|
"code": 0,
|
||||||
|
"message": "",
|
||||||
|
"type": "hasharray",
|
||||||
|
"result": {
|
||||||
|
"contents": {
|
||||||
|
"type": "newer",
|
||||||
|
"content": {
|
||||||
|
"id": "drama",
|
||||||
|
"version": 1,
|
||||||
|
"title": "ドラマ"
|
||||||
|
},
|
||||||
|
"thumbnailType": "horizontal",
|
||||||
|
"contents": [
|
||||||
|
{
|
||||||
|
"type": "episode",
|
||||||
|
"content": {
|
||||||
|
"id": "eps0z5761a",
|
||||||
|
"version": 4,
|
||||||
|
"title": "#10（最終話）",
|
||||||
|
"seriesID": "sr0lhdrtdb",
|
||||||
|
"duration": 2712,
|
||||||
|
"endAt": 1792209540,
|
||||||
|
"broadcastDateLabel": "10月3日(土)放送分",
|
||||||
|
"isNHKContent": false,
|
||||||
|
"isSubtitle": false,
|
||||||
|
"ribbonID": 0,
|
||||||
|
"seriesTitle": "韓国ドラマ「良いが悪い、ドンジェ」",
|
||||||
|
"isAvailable": true,
|
||||||
|
"broadcasterName": "BS12トゥエルビ",
|
||||||
|
"productionProviderName": "BS12 トゥエルビ",
|
||||||
|
"isEndingSoon": false,
|
||||||
|
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51",
|
||||||
|
"isChannelDefaultSeries": false
|
||||||
|
},
|
||||||
|
"startAt": 1791007200,
|
||||||
|
"endAt": 1792209540
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "episode",
|
||||||
|
"content": {
|
||||||
|
"id": "epo4tfi10u",
|
||||||
|
"version": 4,
|
||||||
|
"title": "#9",
|
||||||
|
"seriesID": "sr0lhdrtdb",
|
||||||
|
"duration": 2712,
|
||||||
|
"endAt": 1792209540,
|
||||||
|
"broadcastDateLabel": "10月3日(土)放送分",
|
||||||
|
"isNHKContent": false,
|
||||||
|
"isSubtitle": false,
|
||||||
|
"ribbonID": 0,
|
||||||
|
"seriesTitle": "韓国ドラマ「良いが悪い、ドンジェ」",
|
||||||
|
"isAvailable": true,
|
||||||
|
"broadcasterName": "BS12トゥエルビ",
|
||||||
|
"productionProviderName": "BS12 トゥエルビ",
|
||||||
|
"isEndingSoon": false,
|
||||||
|
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/epo4tfi10u.jpg?v=2d8ff444",
|
||||||
|
"isChannelDefaultSeries": false
|
||||||
|
},
|
||||||
|
"startAt": 1791007200,
|
||||||
|
"endAt": 1792209540
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "episode",
|
||||||
|
"content": {
|
||||||
|
"id": "ep8cxuessg",
|
||||||
|
"version": 6,
|
||||||
|
"title": "【予告】10月12日(月)24時24分 ~ 放送予定 60秒PR！",
|
||||||
|
"seriesID": "srne6nwihe",
|
||||||
|
"duration": 60,
|
||||||
|
"endAt": 1800282240,
|
||||||
|
"broadcastDateLabel": "",
|
||||||
|
"isNHKContent": false,
|
||||||
|
"isSubtitle": false,
|
||||||
|
"ribbonID": 1,
|
||||||
|
"seriesTitle": "十六夜膳房 -いざよいぜんぼう-",
|
||||||
|
"isAvailable": true,
|
||||||
|
"broadcasterName": "日テレ",
|
||||||
|
"productionProviderName": "日テレ",
|
||||||
|
"isEndingSoon": false,
|
||||||
|
"thumbnailPath": "/images/content/thumbnail/episode/xlarge/ep8cxuessg.jpg?v=58408db2",
|
||||||
|
"isChannelDefaultSeries": false
|
||||||
|
},
|
||||||
|
"startAt": 1791007200,
|
||||||
|
"endAt": 1800282240
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+1
@@ -0,0 +1 @@
|
|||||||
|
{"code":0,"result":{"contents":[{"type":"episode","content":{"id":"eps0z5761a","version":4,"title":"#10（最終話）","seriesID":"sr0lhdrtdb","duration":2712,"endAt":1792209540,"broadcastDateLabel":"10月3日(土)放送分","isNHKContent":false,"isSubtitle":false,"ribbonID":0,"seriesTitle":"韓国ドラマ「良いが悪い、ドンジェ」","isAvailable":true,"broadcasterName":"BS12トゥエルビ","productionProviderName":"BS12 トゥエルビ","isEndingSoon":false,"thumbnailPath":"/images/content/thumbnail/episode/xlarge/eps0z5761a.jpg?v=b45ddb51","isChannelDefaultSeries":false}}]}}
|
||||||
Vendored
+1
@@ -0,0 +1 @@
|
|||||||
|
{"code":0,"result":{"content":{"type":"series","content":{"id":"sr0lhdrtdb","title":"韓国ドラマ「良いが悪い、ドンジェ」"}}}}
|
||||||
+1
@@ -0,0 +1 @@
|
|||||||
|
{"code":0,"result":{"contents":[{"type":"season","content":{"id":"ss3i6njoq3","version":1,"title":"本編","season_extended":null}}]}}
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type episodeActionView struct {
|
||||||
|
ID, Genre, Status, StatusText string
|
||||||
|
}
|
||||||
|
|
||||||
|
func statusText(status string) string {
|
||||||
|
switch status {
|
||||||
|
case "queued":
|
||||||
|
return "待機中"
|
||||||
|
case "running":
|
||||||
|
return "ダウンロード中"
|
||||||
|
case "succeeded":
|
||||||
|
return "完了"
|
||||||
|
case "failed":
|
||||||
|
return "失敗"
|
||||||
|
case "canceled":
|
||||||
|
return "取り消し"
|
||||||
|
default:
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) enqueue(w http.ResponseWriter, r *http.Request) {
|
||||||
|
genre, id, ok := episodeForm(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ep, found := s.cachedEpisode(genre, id)
|
||||||
|
if !found {
|
||||||
|
http.Error(w, "新着キャッシュにエピソードがありません。新着を再表示してください", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
job, _, err := s.jobs.Enqueue(r.Context(), storeEpisode(ep), "manual")
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("待ち行列への登録に失敗: %v", err)
|
||||||
|
http.Error(w, "待ち行列に登録できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.episodeActionResponse(w, r, episodeActionView{ID: id, Genre: genre, Status: job.Status, StatusText: statusText(job.Status)})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) retryEpisode(w http.ResponseWriter, r *http.Request) {
|
||||||
|
genre, id, ok := episodeForm(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if _, found := s.cachedEpisode(genre, id); !found {
|
||||||
|
http.Error(w, "新着キャッシュにエピソードがありません。新着を再表示してください", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := s.jobs.RetryEpisode(r.Context(), id); err != nil {
|
||||||
|
if errors.Is(err, store.ErrInvalidTransition) {
|
||||||
|
http.Error(w, "再試行できる状態ではありません", http.StatusConflict)
|
||||||
|
} else {
|
||||||
|
s.logger.Printf("ジョブの再試行に失敗: %v", err)
|
||||||
|
http.Error(w, "再試行できませんでした", http.StatusInternalServerError)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.episodeActionResponse(w, r, episodeActionView{ID: id, Genre: genre, Status: "queued", StatusText: statusText("queued")})
|
||||||
|
}
|
||||||
|
|
||||||
|
func episodeForm(w http.ResponseWriter, r *http.Request) (string, string, bool) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, "フォームを読み取れませんでした", http.StatusBadRequest)
|
||||||
|
return "", "", false
|
||||||
|
}
|
||||||
|
genre, id := r.Form.Get("genre"), r.Form.Get("episode_id")
|
||||||
|
if (genre != "drama" && genre != "variety") || id == "" {
|
||||||
|
http.Error(w, "ジャンルまたはエピソード ID が不正です", http.StatusBadRequest)
|
||||||
|
return "", "", false
|
||||||
|
}
|
||||||
|
return genre, id, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) episodeActionResponse(w http.ResponseWriter, r *http.Request, view episodeActionView) {
|
||||||
|
if r.Header.Get("HX-Request") != "true" {
|
||||||
|
http.Redirect(w, r, "/?genre="+view.Genre, http.StatusSeeOther)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "episode_action", view)
|
||||||
|
}
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"embed"
|
||||||
|
"html/template"
|
||||||
|
"io/fs"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
//go:embed templates/*.html static/*
|
||||||
|
var assets embed.FS
|
||||||
|
|
||||||
|
type page struct {
|
||||||
|
Title string
|
||||||
|
Newer *newerView
|
||||||
|
Queue *queueView
|
||||||
|
Subscriptions *subscriptionsView
|
||||||
|
Library *libraryView
|
||||||
|
Series *seriesView
|
||||||
|
Video *videoView
|
||||||
|
DeleteConfirm *deleteConfirmView
|
||||||
|
DeleteResult *deleteResultView
|
||||||
|
}
|
||||||
|
|
||||||
|
type JobStore interface {
|
||||||
|
Enqueue(context.Context, store.Episode, string) (store.Job, bool, error)
|
||||||
|
EpisodeStatuses(context.Context, []string) (map[string]string, error)
|
||||||
|
ListJobs(context.Context, string, int) ([]store.Job, error)
|
||||||
|
GetJob(context.Context, int64) (store.Job, error)
|
||||||
|
Retry(context.Context, int64) error
|
||||||
|
RetryEpisode(context.Context, string) error
|
||||||
|
Cancel(context.Context, int64) error
|
||||||
|
SubscribedSeries(context.Context, []string) (map[string]bool, error)
|
||||||
|
RegisterSubscription(context.Context, string, string, []store.Episode, bool) (bool, error)
|
||||||
|
ListSubscriptions(context.Context) ([]store.Subscription, error)
|
||||||
|
SetSubscriptionEnabled(context.Context, string, bool) error
|
||||||
|
DeleteSubscription(context.Context, string) (bool, error)
|
||||||
|
ListSucceededJobs(context.Context, int) ([]store.Job, error)
|
||||||
|
ListLibrarySeries(context.Context) ([]store.LibrarySeries, error)
|
||||||
|
ListSucceededJobsBySeries(context.Context, string) ([]store.Job, error)
|
||||||
|
ListSucceededJobsByIDs(context.Context, []int64) ([]store.Job, error)
|
||||||
|
ListSucceededJobsBySeriesIDs(context.Context, []string) ([]store.Job, error)
|
||||||
|
GetSucceededJob(context.Context, int64) (store.Job, error)
|
||||||
|
DeleteSucceededJob(context.Context, int64) error
|
||||||
|
}
|
||||||
|
|
||||||
|
type server struct {
|
||||||
|
fetcher NewerFetcher
|
||||||
|
jobs JobStore
|
||||||
|
videoDir string
|
||||||
|
now func() time.Time
|
||||||
|
logger *log.Logger
|
||||||
|
zone *time.Location
|
||||||
|
tmpl *template.Template
|
||||||
|
cacheMu sync.Mutex
|
||||||
|
cache map[string]cacheEntry
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewHandler(fetcher NewerFetcher, jobs JobStore, videoDir string, now func() time.Time, logger *log.Logger) http.Handler {
|
||||||
|
tmpl := template.Must(template.ParseFS(assets, "templates/*.html"))
|
||||||
|
staticFiles, err := fs.Sub(assets, "static")
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
zone, err := time.LoadLocation("Asia/Tokyo")
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
if videoDir == "" {
|
||||||
|
videoDir = "/videos"
|
||||||
|
}
|
||||||
|
s := &server{fetcher: fetcher, jobs: jobs, videoDir: videoDir, now: now, logger: logger, zone: zone, tmpl: tmpl, cache: make(map[string]cacheEntry)}
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServerFS(staticFiles)))
|
||||||
|
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||||
|
_, _ = w.Write([]byte("ok\n"))
|
||||||
|
})
|
||||||
|
mux.HandleFunc("GET /{$}", s.home)
|
||||||
|
mux.HandleFunc("GET /newer", s.newerFragment)
|
||||||
|
mux.HandleFunc("POST /episodes/enqueue", s.enqueue)
|
||||||
|
mux.HandleFunc("POST /episodes/retry", s.retryEpisode)
|
||||||
|
mux.HandleFunc("GET /queue", s.queuePage)
|
||||||
|
mux.HandleFunc("GET /queue/list", s.queueFragment)
|
||||||
|
mux.HandleFunc("POST /jobs/{id}/retry", s.retryJob)
|
||||||
|
mux.HandleFunc("POST /jobs/{id}/cancel", s.cancelJob)
|
||||||
|
mux.HandleFunc("GET /subscriptions", s.subscriptionsPage)
|
||||||
|
mux.HandleFunc("GET /subscriptions/list", s.subscriptionsFragment)
|
||||||
|
mux.HandleFunc("POST /subscriptions/from-url/confirm", s.confirmSubscriptionURL)
|
||||||
|
mux.HandleFunc("POST /subscriptions/from-url/register", s.registerSubscriptionURL)
|
||||||
|
mux.HandleFunc("GET /subscriptions/from-url/cancel", s.cancelSubscriptionURL)
|
||||||
|
mux.HandleFunc("GET /subscriptions/choice", s.subscriptionChoice)
|
||||||
|
mux.HandleFunc("POST /subscriptions/register", s.registerSubscription)
|
||||||
|
mux.HandleFunc("POST /subscriptions/{id}/pause", s.pauseSubscription)
|
||||||
|
mux.HandleFunc("POST /subscriptions/{id}/resume", s.resumeSubscription)
|
||||||
|
mux.HandleFunc("POST /subscriptions/{id}/delete", s.deleteSubscription)
|
||||||
|
mux.HandleFunc("GET /library", s.libraryPage)
|
||||||
|
mux.HandleFunc("GET /library/series/{series_id}", s.librarySeriesPage)
|
||||||
|
mux.HandleFunc("GET /library/{id}", s.videoPage)
|
||||||
|
mux.HandleFunc("GET /videos/{id}", s.serveVideo)
|
||||||
|
mux.HandleFunc("POST /library/delete/confirm", s.confirmDeleteVideos)
|
||||||
|
mux.HandleFunc("POST /library/delete", s.deleteVideos)
|
||||||
|
return mux
|
||||||
|
}
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestHealthz(t *testing.T) {
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
newTestHandler(&fakeFetcher{}, nil).ServeHTTP(response, httptest.NewRequest(http.MethodGet, "/healthz", nil))
|
||||||
|
if response.Code != http.StatusOK || response.Body.String() != "ok\n" {
|
||||||
|
t.Errorf("healthz: status=%d body=%q", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPagesAndEmbeddedAssets(t *testing.T) {
|
||||||
|
handler := newTestHandler(&fakeFetcher{}, nil)
|
||||||
|
for _, tt := range []struct{ path, title string }{
|
||||||
|
{"/", "新着"}, {"/queue", "待ち行列"}, {"/subscriptions", "登録番組"}, {"/library", "ライブラリ"},
|
||||||
|
} {
|
||||||
|
t.Run(tt.path, func(t *testing.T) {
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, tt.path, nil))
|
||||||
|
body := response.Body.String()
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Header().Get("Content-Type"), "text/html") {
|
||||||
|
t.Fatalf("status=%d Content-Type=%q", response.Code, response.Header().Get("Content-Type"))
|
||||||
|
}
|
||||||
|
for _, want := range []string{"<h1>" + tt.title + "</h1>", "新着", "待ち行列", "登録番組", "ライブラリ", `hx-boost="true"`, "/static/htmx-2.0.11.min.js"} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("page missing %q", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, "/static/htmx-2.0.11.min.js", nil))
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Header().Get("Content-Type"), "javascript") || response.Body.Len() < 1000 || !strings.Contains(response.Body.String(), "htmx") {
|
||||||
|
t.Errorf("embedded htmx: status=%d Content-Type=%q bytes=%d", response.Code, response.Header().Get("Content-Type"), response.Body.Len())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUnknownPathsAndMethods(t *testing.T) {
|
||||||
|
handler := newTestHandler(&fakeFetcher{}, nil)
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name, method, path string
|
||||||
|
wantStatus int
|
||||||
|
}{
|
||||||
|
{"unknown page", http.MethodGet, "/missing", http.StatusNotFound},
|
||||||
|
{"root prefix", http.MethodGet, "/missing/nested", http.StatusNotFound},
|
||||||
|
{"missing static asset", http.MethodGet, "/static/missing.js", http.StatusNotFound},
|
||||||
|
{"unsupported page method", http.MethodPost, "/queue", http.StatusMethodNotAllowed},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, httptest.NewRequest(tt.method, tt.path, nil))
|
||||||
|
if response.Code != tt.wantStatus {
|
||||||
|
t.Errorf("%s %s: status=%d; want %d", tt.method, tt.path, response.Code, tt.wantStatus)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,492 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
type libraryItemView struct {
|
||||||
|
ID int64
|
||||||
|
ThumbnailURL, SeriesTitle, Title string
|
||||||
|
BroadcasterName, DurationText string
|
||||||
|
FileSizeText, FinishedAtText string
|
||||||
|
}
|
||||||
|
|
||||||
|
type librarySeriesView struct {
|
||||||
|
SeriesID, SeriesTitle, LatestTitle string
|
||||||
|
ThumbnailURL, BroadcasterName string
|
||||||
|
EpisodeCount int64
|
||||||
|
TotalSizeText, LatestFinishedText string
|
||||||
|
Subscribed bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type libraryView struct {
|
||||||
|
Series []librarySeriesView
|
||||||
|
SeriesCount, EpisodeCount int
|
||||||
|
TotalSizeText string
|
||||||
|
}
|
||||||
|
|
||||||
|
type seriesView struct {
|
||||||
|
SeriesID, SeriesTitle, BroadcasterName string
|
||||||
|
EpisodeCount int
|
||||||
|
TotalSizeText, Sort string
|
||||||
|
Subscribed bool
|
||||||
|
Items []libraryItemView
|
||||||
|
}
|
||||||
|
|
||||||
|
type episodeLinkView struct {
|
||||||
|
ID int64
|
||||||
|
Title string
|
||||||
|
}
|
||||||
|
|
||||||
|
type videoView struct {
|
||||||
|
libraryItemView
|
||||||
|
VideoURL, SeriesURL string
|
||||||
|
Previous, Next *episodeLinkView
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteGroupView struct {
|
||||||
|
SeriesID, SeriesTitle, TotalSizeText string
|
||||||
|
Items []libraryItemView
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteConfirmView struct {
|
||||||
|
Groups []deleteGroupView
|
||||||
|
IDs []int64
|
||||||
|
Count int
|
||||||
|
TotalSizeText string
|
||||||
|
CancelURL string
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteFailureView struct {
|
||||||
|
Title, Reason string
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteSeriesLinkView struct {
|
||||||
|
SeriesID, SeriesTitle string
|
||||||
|
}
|
||||||
|
|
||||||
|
type deleteResultView struct {
|
||||||
|
DeletedCount, ExcludedCount int
|
||||||
|
Failures []deleteFailureView
|
||||||
|
SeriesLinks []deleteSeriesLinkView
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) libraryPage(w http.ResponseWriter, r *http.Request) {
|
||||||
|
series, err := s.jobs.ListLibrarySeries(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("ライブラリの取得に失敗: %v", err)
|
||||||
|
http.Error(w, "ライブラリを取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
view := libraryView{Series: make([]librarySeriesView, 0, len(series)), SeriesCount: len(series)}
|
||||||
|
var totalBytes int64
|
||||||
|
for _, item := range series {
|
||||||
|
view.EpisodeCount += int(item.EpisodeCount)
|
||||||
|
totalBytes += item.TotalSizeBytes
|
||||||
|
viewItem := librarySeriesView{
|
||||||
|
SeriesID: item.SeriesID, SeriesTitle: item.SeriesTitle, LatestTitle: item.LatestTitle,
|
||||||
|
ThumbnailURL: thumbnailURL(item.ThumbnailPath), BroadcasterName: item.BroadcasterName,
|
||||||
|
EpisodeCount: item.EpisodeCount, TotalSizeText: formatBytes(float64(item.TotalSizeBytes)),
|
||||||
|
Subscribed: item.Subscribed,
|
||||||
|
}
|
||||||
|
if item.LatestFinishedAt != nil {
|
||||||
|
viewItem.LatestFinishedText = item.LatestFinishedAt.In(s.zone).Format("2006-01-02 15:04 MST")
|
||||||
|
}
|
||||||
|
view.Series = append(view.Series, viewItem)
|
||||||
|
}
|
||||||
|
view.TotalSizeText = fmt.Sprintf("%.2f GB", float64(totalBytes)/1_000_000_000)
|
||||||
|
s.render(w, "layout", page{Title: "ライブラリ", Library: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) librarySeriesPage(w http.ResponseWriter, r *http.Request) {
|
||||||
|
seriesID := r.PathValue("series_id")
|
||||||
|
if !seriesIDPattern.MatchString(seriesID) {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sortOrder := r.URL.Query().Get("sort")
|
||||||
|
if sortOrder == "" {
|
||||||
|
sortOrder = "latest"
|
||||||
|
}
|
||||||
|
if sortOrder != "latest" && sortOrder != "natural" {
|
||||||
|
http.Error(w, "並び順が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jobs, err := s.jobs.ListSucceededJobsBySeries(r.Context(), seriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組のライブラリ取得に失敗: %v", err)
|
||||||
|
http.Error(w, "番組を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(jobs) == 0 {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
latest := latestFinishedJob(jobs)
|
||||||
|
if sortOrder == "natural" {
|
||||||
|
sortJobsNaturally(jobs)
|
||||||
|
} else {
|
||||||
|
sortJobsLatest(jobs)
|
||||||
|
}
|
||||||
|
subscribed, err := s.jobs.SubscribedSeries(r.Context(), []string{seriesID})
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組登録状態の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "番組を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
view := seriesView{SeriesID: seriesID, SeriesTitle: latest.SeriesTitle,
|
||||||
|
BroadcasterName: latest.BroadcasterName, EpisodeCount: len(jobs),
|
||||||
|
Subscribed: subscribed[seriesID], Sort: sortOrder, Items: make([]libraryItemView, 0, len(jobs))}
|
||||||
|
var totalBytes int64
|
||||||
|
for _, job := range jobs {
|
||||||
|
if job.FileSizeBytes != nil {
|
||||||
|
totalBytes += *job.FileSizeBytes
|
||||||
|
}
|
||||||
|
view.Items = append(view.Items, s.libraryItem(job))
|
||||||
|
}
|
||||||
|
view.TotalSizeText = formatBytes(float64(totalBytes))
|
||||||
|
s.render(w, "layout", page{Title: latest.SeriesTitle, Series: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) videoPage(w http.ResponseWriter, r *http.Request) {
|
||||||
|
job, ok := s.succeededJob(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jobs, err := s.jobs.ListSucceededJobsBySeries(r.Context(), job.SeriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("前後の回の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "動画を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sortJobsNaturally(jobs)
|
||||||
|
view := videoView{libraryItemView: s.libraryItem(job), VideoURL: "/videos/" + strconv.FormatInt(job.ID, 10),
|
||||||
|
SeriesURL: "/library/series/" + job.SeriesID}
|
||||||
|
for i := range jobs {
|
||||||
|
if jobs[i].ID != job.ID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if i > 0 {
|
||||||
|
view.Previous = &episodeLinkView{ID: jobs[i-1].ID, Title: jobs[i-1].Title}
|
||||||
|
}
|
||||||
|
if i+1 < len(jobs) {
|
||||||
|
view.Next = &episodeLinkView{ID: jobs[i+1].ID, Title: jobs[i+1].Title}
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
s.render(w, "layout", page{Title: job.SeriesTitle + " " + job.Title, Video: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) libraryItem(job store.Job) libraryItemView {
|
||||||
|
item := libraryItemView{ID: job.ID, ThumbnailURL: thumbnailURL(job.ThumbnailPath),
|
||||||
|
SeriesTitle: job.SeriesTitle, Title: job.Title, BroadcasterName: job.BroadcasterName,
|
||||||
|
DurationText: fmt.Sprintf("%d分", durationMinutes(int64(job.DurationSeconds)))}
|
||||||
|
if job.FileSizeBytes != nil {
|
||||||
|
item.FileSizeText = formatBytes(float64(*job.FileSizeBytes))
|
||||||
|
}
|
||||||
|
if job.FinishedAt != nil {
|
||||||
|
item.FinishedAtText = job.FinishedAt.In(s.zone).Format("2006-01-02 15:04 MST")
|
||||||
|
}
|
||||||
|
return item
|
||||||
|
}
|
||||||
|
|
||||||
|
func thumbnailURL(path string) string {
|
||||||
|
if path == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return "https://statics.tver.jp" + path
|
||||||
|
}
|
||||||
|
|
||||||
|
func latestFinishedJob(jobs []store.Job) store.Job {
|
||||||
|
latest := jobs[0]
|
||||||
|
for _, job := range jobs[1:] {
|
||||||
|
if jobFinishedLater(job, latest) {
|
||||||
|
latest = job
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return latest
|
||||||
|
}
|
||||||
|
|
||||||
|
func sortJobsLatest(jobs []store.Job) {
|
||||||
|
sort.SliceStable(jobs, func(i, j int) bool { return jobFinishedLater(jobs[i], jobs[j]) })
|
||||||
|
}
|
||||||
|
|
||||||
|
func jobFinishedLater(left, right store.Job) bool {
|
||||||
|
if left.FinishedAt == nil || right.FinishedAt == nil {
|
||||||
|
if left.FinishedAt != right.FinishedAt {
|
||||||
|
return left.FinishedAt != nil
|
||||||
|
}
|
||||||
|
return left.ID > right.ID
|
||||||
|
}
|
||||||
|
if !left.FinishedAt.Equal(*right.FinishedAt) {
|
||||||
|
return left.FinishedAt.After(*right.FinishedAt)
|
||||||
|
}
|
||||||
|
return left.ID > right.ID
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) serveVideo(w http.ResponseWriter, r *http.Request) {
|
||||||
|
job, ok := s.succeededJob(w, r)
|
||||||
|
if !ok || job.FilePath == nil {
|
||||||
|
if ok {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
root, err := os.OpenRoot(s.videoDir)
|
||||||
|
if err != nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer root.Close()
|
||||||
|
file, err := root.Open(*job.FilePath)
|
||||||
|
if err != nil {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer file.Close()
|
||||||
|
info, err := file.Stat()
|
||||||
|
if err != nil || !info.Mode().IsRegular() {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "video/mp4")
|
||||||
|
http.ServeContent(w, r, filepath.Base(*job.FilePath), info.ModTime(), file)
|
||||||
|
}
|
||||||
|
|
||||||
|
const maxDeleteSelections = 1000
|
||||||
|
|
||||||
|
func (s *server) confirmDeleteVideos(w http.ResponseWriter, r *http.Request) {
|
||||||
|
ids, seriesIDs, ok := parseDeleteSelection(w, r, true)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jobsByID, err := s.jobs.ListSucceededJobsByIDs(r.Context(), ids)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("削除対象IDの取得に失敗: %v", err)
|
||||||
|
http.Error(w, "削除対象を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jobsBySeries, err := s.jobs.ListSucceededJobsBySeriesIDs(r.Context(), seriesIDs)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("削除対象番組の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "削除対象を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
byID := make(map[int64]store.Job, len(jobsByID)+len(jobsBySeries))
|
||||||
|
for _, job := range append(jobsByID, jobsBySeries...) {
|
||||||
|
byID[job.ID] = job
|
||||||
|
}
|
||||||
|
if len(byID) == 0 {
|
||||||
|
http.Error(w, "削除対象がありません", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(byID) > maxDeleteSelections {
|
||||||
|
http.Error(w, fmt.Sprintf("削除対象が多すぎます（%d本）。番組を分けて選んでください", len(byID)), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jobs := make([]store.Job, 0, len(byID))
|
||||||
|
for _, job := range byID {
|
||||||
|
jobs = append(jobs, job)
|
||||||
|
}
|
||||||
|
sort.Slice(jobs, func(i, j int) bool {
|
||||||
|
if jobs[i].SeriesTitle != jobs[j].SeriesTitle {
|
||||||
|
return jobs[i].SeriesTitle < jobs[j].SeriesTitle
|
||||||
|
}
|
||||||
|
return jobs[i].ID < jobs[j].ID
|
||||||
|
})
|
||||||
|
view := deleteConfirmView{IDs: make([]int64, 0, len(jobs)), Count: len(jobs), CancelURL: "/library"}
|
||||||
|
groupIndexes := make(map[string]int)
|
||||||
|
groupBytes := make(map[string]int64)
|
||||||
|
var totalBytes int64
|
||||||
|
for _, job := range jobs {
|
||||||
|
view.IDs = append(view.IDs, job.ID)
|
||||||
|
index, exists := groupIndexes[job.SeriesID]
|
||||||
|
if !exists {
|
||||||
|
index = len(view.Groups)
|
||||||
|
groupIndexes[job.SeriesID] = index
|
||||||
|
view.Groups = append(view.Groups, deleteGroupView{SeriesID: job.SeriesID, SeriesTitle: job.SeriesTitle})
|
||||||
|
}
|
||||||
|
view.Groups[index].Items = append(view.Groups[index].Items, s.libraryItem(job))
|
||||||
|
if job.FileSizeBytes != nil {
|
||||||
|
totalBytes += *job.FileSizeBytes
|
||||||
|
groupBytes[job.SeriesID] += *job.FileSizeBytes
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for i := range view.Groups {
|
||||||
|
view.Groups[i].TotalSizeText = formatBytes(float64(groupBytes[view.Groups[i].SeriesID]))
|
||||||
|
}
|
||||||
|
view.TotalSizeText = formatBytes(float64(totalBytes))
|
||||||
|
if len(view.Groups) == 1 {
|
||||||
|
view.CancelURL = "/library/series/" + view.Groups[0].SeriesID
|
||||||
|
}
|
||||||
|
s.render(w, "layout", page{Title: "動画の削除確認", DeleteConfirm: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseDeleteSelection(w http.ResponseWriter, r *http.Request, includeSeries bool) ([]int64, []string, bool) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, "入力が不正です", http.StatusBadRequest)
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
idSet := make(map[int64]struct{})
|
||||||
|
for _, raw := range r.Form["id"] {
|
||||||
|
id, err := strconv.ParseInt(raw, 10, 64)
|
||||||
|
if err != nil || id <= 0 {
|
||||||
|
http.Error(w, "動画IDが不正です", http.StatusBadRequest)
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
idSet[id] = struct{}{}
|
||||||
|
}
|
||||||
|
seriesSet := make(map[string]struct{})
|
||||||
|
if includeSeries {
|
||||||
|
for _, seriesID := range r.Form["series_id"] {
|
||||||
|
if !seriesIDPattern.MatchString(seriesID) {
|
||||||
|
http.Error(w, "番組IDが不正です", http.StatusBadRequest)
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
seriesSet[seriesID] = struct{}{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(idSet)+len(seriesSet) == 0 {
|
||||||
|
http.Error(w, "削除対象がありません", http.StatusBadRequest)
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
if len(idSet)+len(seriesSet) > maxDeleteSelections {
|
||||||
|
http.Error(w, "削除対象が多すぎます", http.StatusBadRequest)
|
||||||
|
return nil, nil, false
|
||||||
|
}
|
||||||
|
ids := make([]int64, 0, len(idSet))
|
||||||
|
for id := range idSet {
|
||||||
|
ids = append(ids, id)
|
||||||
|
}
|
||||||
|
seriesIDs := make([]string, 0, len(seriesSet))
|
||||||
|
for seriesID := range seriesSet {
|
||||||
|
seriesIDs = append(seriesIDs, seriesID)
|
||||||
|
}
|
||||||
|
return ids, seriesIDs, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) deleteVideos(w http.ResponseWriter, r *http.Request) {
|
||||||
|
ids, _, ok := parseDeleteSelection(w, r, false)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jobs, err := s.jobs.ListSucceededJobsByIDs(r.Context(), ids)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("削除対象の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "削除対象を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
view := deleteResultView{ExcludedCount: len(ids) - len(jobs)}
|
||||||
|
var root *os.Root
|
||||||
|
needsRoot := false
|
||||||
|
for _, job := range jobs {
|
||||||
|
needsRoot = needsRoot || job.FilePath != nil
|
||||||
|
}
|
||||||
|
if needsRoot {
|
||||||
|
root, err = os.OpenRoot(s.videoDir)
|
||||||
|
if err == nil {
|
||||||
|
defer root.Close()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
affectedSeries := make(map[string]string)
|
||||||
|
for _, job := range jobs {
|
||||||
|
affectedSeries[job.SeriesID] = job.SeriesTitle
|
||||||
|
if err != nil && job.FilePath != nil {
|
||||||
|
s.logger.Printf("動画ルートを開けないため削除に失敗 id=%d: %v", job.ID, err)
|
||||||
|
view.Failures = append(view.Failures, deleteFailureView{Title: job.Title, Reason: "動画ファイルを削除できませんでした"})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if deleteErr := s.deleteJob(r.Context(), root, job); deleteErr != nil {
|
||||||
|
s.logger.Printf("動画の削除に失敗 id=%d: %v", job.ID, deleteErr)
|
||||||
|
view.Failures = append(view.Failures, deleteFailureView{Title: job.Title, Reason: "削除できませんでした"})
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
view.DeletedCount++
|
||||||
|
}
|
||||||
|
for seriesID, title := range affectedSeries {
|
||||||
|
remaining, listErr := s.jobs.ListSucceededJobsBySeries(r.Context(), seriesID)
|
||||||
|
if listErr != nil {
|
||||||
|
s.logger.Printf("削除後の番組取得に失敗 series_id=%s: %v", seriesID, listErr)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(remaining) > 0 {
|
||||||
|
view.SeriesLinks = append(view.SeriesLinks, deleteSeriesLinkView{SeriesID: seriesID, SeriesTitle: title})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(view.SeriesLinks, func(i, j int) bool { return view.SeriesLinks[i].SeriesTitle < view.SeriesLinks[j].SeriesTitle })
|
||||||
|
s.render(w, "layout", page{Title: "動画の削除結果", DeleteResult: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) deleteJob(ctx context.Context, root *os.Root, job store.Job) error {
|
||||||
|
if job.FilePath != nil {
|
||||||
|
if !filepath.IsLocal(*job.FilePath) {
|
||||||
|
return errors.New("動画ファイルの場所が不正")
|
||||||
|
}
|
||||||
|
if root == nil {
|
||||||
|
return errors.New("動画ルートを開けない")
|
||||||
|
}
|
||||||
|
file, err := root.Open(*job.FilePath)
|
||||||
|
if err == nil {
|
||||||
|
_ = file.Close()
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return fmt.Errorf("動画ファイルの検査: %w", err)
|
||||||
|
}
|
||||||
|
if err := root.Remove(*job.FilePath); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return fmt.Errorf("動画ファイルの削除: %w", err)
|
||||||
|
}
|
||||||
|
s.removeEmptyVideoParent(root, *job.FilePath)
|
||||||
|
}
|
||||||
|
if err := s.jobs.DeleteSucceededJob(ctx, job.ID); err != nil {
|
||||||
|
return fmt.Errorf("動画の記録削除: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) removeEmptyVideoParent(root *os.Root, filePath string) {
|
||||||
|
parent := filepath.Dir(filePath)
|
||||||
|
if parent == "." || !filepath.IsLocal(parent) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
info, err := root.Lstat(parent)
|
||||||
|
if err != nil {
|
||||||
|
if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
s.logger.Printf("動画フォルダの確認に失敗: %v", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !info.IsDir() {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := root.Remove(parent); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
s.logger.Printf("空の動画フォルダを削除できませんでした: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) succeededJob(w http.ResponseWriter, r *http.Request) (store.Job, bool) {
|
||||||
|
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
|
||||||
|
if err != nil || id <= 0 {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return store.Job{}, false
|
||||||
|
}
|
||||||
|
job, err := s.jobs.GetSucceededJob(r.Context(), id)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
http.NotFound(w, r)
|
||||||
|
return store.Job{}, false
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("完了ジョブの取得に失敗: %v", err)
|
||||||
|
http.Error(w, "動画を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return store.Job{}, false
|
||||||
|
}
|
||||||
|
return job, true
|
||||||
|
}
|
||||||
@@ -0,0 +1,639 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func libraryJob(id int64, relative string) store.Job {
|
||||||
|
size := int64(10)
|
||||||
|
finished := time.Date(2026, 10, 3, 1, 2, 3, 0, time.UTC)
|
||||||
|
return store.Job{ID: id, EpisodeID: "ep1", SeriesID: "series1", SeriesTitle: `<script>番組</script>`, Title: `<b>第1話</b>`,
|
||||||
|
BroadcasterName: "放送局", DurationSeconds: 61, ThumbnailPath: "/thumb.jpg", Status: "succeeded",
|
||||||
|
FilePath: &relative, FileSizeBytes: &size, FinishedAt: &finished}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibraryAndPlaybackPage(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
page := getPage(handler, "/library")
|
||||||
|
if page.Code != http.StatusOK {
|
||||||
|
t.Fatalf("library status=%d", page.Code)
|
||||||
|
}
|
||||||
|
body := page.Body.String()
|
||||||
|
for _, want := range []string{"<script>番組</script>", "<b>第1話</b>", "放送局", "1番組・1本・0.00 GB", "10.0 B", "2026-10-03 10:02 JST", "https://statics.tver.jp/thumb.jpg", `/library/series/series1`, "1本"} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("library missing %q", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(body, "<script>番組</script>") {
|
||||||
|
t.Fatal("library fields were not escaped")
|
||||||
|
}
|
||||||
|
video := getPage(handler, "/library/1")
|
||||||
|
if video.Code != http.StatusOK || !strings.Contains(video.Body.String(), `<video controls`) || !strings.Contains(video.Body.String(), `src="/videos/1"`) || !strings.Contains(video.Body.String(), `action="/library/delete/confirm"`) || strings.Contains(video.Body.String(), `hx-confirm`) || !strings.Contains(video.Body.String(), `/library/series/series1`) || !strings.Contains(video.Body.String(), `前の回はありません`) || !strings.Contains(video.Body.String(), `次の回はありません`) {
|
||||||
|
t.Fatalf("video page: %d %s", video.Code, video.Body.String())
|
||||||
|
}
|
||||||
|
for _, path := range []string{"/library/bad", "/library/99", "/library/1/delete"} {
|
||||||
|
response := getPage(handler, path)
|
||||||
|
want := http.StatusNotFound
|
||||||
|
if response.Code != want {
|
||||||
|
t.Errorf("%s status=%d want=%d", path, response.Code, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibrarySeriesPageSortingSubscriptionAndNotFound(t *testing.T) {
|
||||||
|
old := libraryJob(2, "show/old.mp4")
|
||||||
|
old.EpisodeID = "ep2"
|
||||||
|
old.Title = "第2話"
|
||||||
|
old.SeriesTitle = "旧番組名"
|
||||||
|
old.BroadcasterName = "旧局"
|
||||||
|
oldFinished := time.Date(2026, 10, 1, 0, 0, 0, 0, time.UTC)
|
||||||
|
old.FinishedAt = &oldFinished
|
||||||
|
latest := libraryJob(10, "show/latest.mp4")
|
||||||
|
latest.EpisodeID = "ep10"
|
||||||
|
latest.Title = "第10話"
|
||||||
|
latest.SeriesTitle = `<em>新番組名</em>`
|
||||||
|
latest.BroadcasterName = "新局"
|
||||||
|
latestFinished := oldFinished.Add(time.Hour)
|
||||||
|
latest.FinishedAt = &latestFinished
|
||||||
|
jobs := &fakeJobStore{
|
||||||
|
jobs: map[string]store.Job{"ep2": old, "ep10": latest},
|
||||||
|
subscriptions: map[string]store.Subscription{"series1": {SeriesID: "series1"}},
|
||||||
|
}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
|
||||||
|
library := getPage(handler, "/library")
|
||||||
|
if library.Code != http.StatusOK || !strings.Contains(library.Body.String(), "登録中") || !strings.Contains(library.Body.String(), "2本") {
|
||||||
|
t.Fatalf("library series card: status=%d body=%s", library.Code, library.Body.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
page := getPage(handler, "/library/series/series1")
|
||||||
|
if page.Code != http.StatusOK {
|
||||||
|
t.Fatalf("series status=%d body=%s", page.Code, page.Body.String())
|
||||||
|
}
|
||||||
|
body := page.Body.String()
|
||||||
|
for _, want := range []string{"ライブラリ", "<em>新番組名</em>", "登録中", "新局・2本・20.0 B", `/library/10`, `/library/2`, `?sort=natural`} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("series page missing %q", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Index(body, `/library/10`) > strings.Index(body, `/library/2`) {
|
||||||
|
t.Fatal("default order is not latest first")
|
||||||
|
}
|
||||||
|
natural := getPage(handler, "/library/series/series1?sort=natural")
|
||||||
|
if natural.Code != http.StatusOK || strings.Index(natural.Body.String(), `/library/2`) > strings.Index(natural.Body.String(), `/library/10`) {
|
||||||
|
t.Fatalf("natural order: status=%d body=%s", natural.Code, natural.Body.String())
|
||||||
|
}
|
||||||
|
if got := getPage(handler, "/library/series/missing").Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("missing series status=%d", got)
|
||||||
|
}
|
||||||
|
if got := getPage(handler, "/library/series/bad-id").Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("invalid series status=%d", got)
|
||||||
|
}
|
||||||
|
if got := getPage(handler, "/library/series/series1?sort=bad").Code; got != http.StatusBadRequest {
|
||||||
|
t.Fatalf("invalid sort status=%d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibraryOmitsSubscriptionMarkerForUnregisteredSeries(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
|
||||||
|
page := getPage(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil), "/library")
|
||||||
|
if page.Code != http.StatusOK || strings.Contains(page.Body.String(), "登録中") {
|
||||||
|
t.Fatalf("unregistered series: status=%d body=%s", page.Code, page.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibraryPagesDoNotInspectVideoDirectory(t *testing.T) {
|
||||||
|
missingVideoDir := filepath.Join(t.TempDir(), "missing-video-root")
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "missing/video.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, missingVideoDir, nil)
|
||||||
|
for _, path := range []string{"/library", "/library/series/series1"} {
|
||||||
|
response := getPage(handler, path)
|
||||||
|
if response.Code != http.StatusOK {
|
||||||
|
t.Errorf("DB由来の一覧 %s: status=%d body=%s", path, response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(missingVideoDir); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("一覧表示が動画ルートを作成または走査した: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPlaybackPreviousAndNextNaturalOrder(t *testing.T) {
|
||||||
|
jobs := make(map[string]store.Job)
|
||||||
|
for _, data := range []struct {
|
||||||
|
id int64
|
||||||
|
epid string
|
||||||
|
title string
|
||||||
|
}{{2, "ep2", "第2話"}, {10, "ep10", "第10話"}, {9, "ep9", "第9話"}} {
|
||||||
|
job := libraryJob(data.id, "show/video.mp4")
|
||||||
|
job.EpisodeID = data.epid
|
||||||
|
job.Title = data.title
|
||||||
|
jobs[data.epid] = job
|
||||||
|
}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, &fakeJobStore{jobs: jobs}, t.TempDir(), nil)
|
||||||
|
middle := getPage(handler, "/library/9")
|
||||||
|
for _, want := range []string{`href="/library/2"`, `前の回: 第2話`, `href="/library/10"`, `次の回: 第10話`} {
|
||||||
|
if !strings.Contains(middle.Body.String(), want) {
|
||||||
|
t.Errorf("middle playback missing %q", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
first := getPage(handler, "/library/2")
|
||||||
|
if !strings.Contains(first.Body.String(), "前の回はありません") || !strings.Contains(first.Body.String(), `href="/library/9"`) {
|
||||||
|
t.Fatalf("first playback navigation: %s", first.Body.String())
|
||||||
|
}
|
||||||
|
last := getPage(handler, "/library/10")
|
||||||
|
if !strings.Contains(last.Body.String(), "次の回はありません") || !strings.Contains(last.Body.String(), `href="/library/9"`) {
|
||||||
|
t.Fatalf("last playback navigation: %s", last.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibraryRejectsInvalidJobIDs(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
|
||||||
|
for _, id := range []string{"0", "-1", "bad", "999999999999999999999999999999999999"} {
|
||||||
|
for _, path := range []string{"/library/" + id, "/videos/" + id} {
|
||||||
|
if got := getPage(handler, path).Code; got != http.StatusNotFound {
|
||||||
|
t.Errorf("GET %s: %d", path, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/library/delete", url.Values{"id": {id}}).Code; got != http.StatusBadRequest {
|
||||||
|
t.Errorf("POST delete id=%q: %d", id, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVideoRangeAndMissingFile(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
content := []byte("0123456789")
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "show", "video.mp4"), content, 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
request := httptest.NewRequest(http.MethodGet, "/videos/1?file_path=../../outside.mp4", nil)
|
||||||
|
request.Header.Set("Range", "bytes=2-5")
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, request)
|
||||||
|
if response.Code != http.StatusPartialContent || response.Body.String() != "2345" || response.Header().Get("Content-Range") != "bytes 2-5/10" || response.Header().Get("Content-Type") != "video/mp4" {
|
||||||
|
t.Fatalf("range: status=%d range=%q type=%q body=%q", response.Code, response.Header().Get("Content-Range"), response.Header().Get("Content-Type"), response.Body.String())
|
||||||
|
}
|
||||||
|
full := getPage(handler, "/videos/1")
|
||||||
|
if full.Code != http.StatusOK || full.Body.String() != string(content) || full.Header().Get("Content-Type") != "video/mp4" || full.Header().Get("Accept-Ranges") != "bytes" {
|
||||||
|
t.Fatalf("full video: status=%d type=%q ranges=%q body=%q", full.Code, full.Header().Get("Content-Type"), full.Header().Get("Accept-Ranges"), full.Body.String())
|
||||||
|
}
|
||||||
|
request = httptest.NewRequest(http.MethodGet, "/videos/1", nil)
|
||||||
|
request.Header.Set("Range", "bytes=20-30")
|
||||||
|
response = httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, request)
|
||||||
|
if response.Code != http.StatusRequestedRangeNotSatisfiable || response.Header().Get("Content-Range") != "bytes */10" {
|
||||||
|
t.Fatalf("range outside: status=%d range=%q", response.Code, response.Header().Get("Content-Range"))
|
||||||
|
}
|
||||||
|
if err := os.Remove(filepath.Join(dir, "show", "video.mp4")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("missing file status=%d", got)
|
||||||
|
}
|
||||||
|
jobs.jobs["ep1"] = store.Job{ID: 1, Status: "queued"}
|
||||||
|
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("unfinished status=%d", got)
|
||||||
|
}
|
||||||
|
if got := getPage(handler, "/library/1").Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("unfinished playback page status=%d", got)
|
||||||
|
}
|
||||||
|
missingPath := libraryJob(1, "show/video.mp4")
|
||||||
|
missingPath.FilePath = nil
|
||||||
|
jobs.jobs["ep1"] = missingPath
|
||||||
|
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("missing DB file path status=%d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVideoPathTraversalAndSymlinkAreRejected(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
outside := t.TempDir()
|
||||||
|
outsideFile := filepath.Join(outside, "outside.mp4")
|
||||||
|
if err := os.WriteFile(outsideFile, []byte("secret"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(outsideFile, filepath.Join(dir, "link.mp4")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(outside, filepath.Join(dir, "linked-directory")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
relativeOutsideFile, err := filepath.Rel(dir, outsideFile)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
relativeOutsideDir, err := filepath.Rel(dir, outside)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(relativeOutsideFile, filepath.Join(dir, "relative-link.mp4")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(relativeOutsideDir, filepath.Join(dir, "relative-linked-directory")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, relative := range []string{"../outside.mp4", "show/../../outside.mp4", outsideFile,
|
||||||
|
"link.mp4", "linked-directory/outside.mp4", "linked-directory/missing.mp4",
|
||||||
|
"relative-link.mp4", "relative-linked-directory/outside.mp4", "relative-linked-directory/missing.mp4"} {
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, relative)}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
|
||||||
|
t.Errorf("path %q status=%d", relative, got)
|
||||||
|
}
|
||||||
|
deleted := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
|
||||||
|
if deleted.Code != http.StatusOK || len(jobs.jobs) != 1 || !strings.Contains(deleted.Body.String(), "失敗: 1件") {
|
||||||
|
t.Errorf("delete path %q status=%d jobs=%d", relative, deleted.Code, len(jobs.jobs))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got, err := os.ReadFile(outsideFile); err != nil || string(got) != "secret" {
|
||||||
|
t.Fatalf("outside target was changed: content=%q err=%v", got, err)
|
||||||
|
}
|
||||||
|
for _, link := range []string{"link.mp4", "linked-directory", "relative-link.mp4", "relative-linked-directory"} {
|
||||||
|
if _, err := os.Lstat(filepath.Join(dir, link)); err != nil {
|
||||||
|
t.Errorf("rejected link %q was changed: %v", link, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVideoSymlinkWithinRootCanBeServedAndDeleted(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
target := filepath.Join(dir, "target.mp4")
|
||||||
|
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink("target.mp4", filepath.Join(dir, "link.mp4")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "link.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
if response := getPage(handler, "/videos/1"); response.Code != http.StatusOK || response.Body.String() != "video" {
|
||||||
|
t.Fatalf("serve: status=%d body=%q", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
if response := postForm(handler, "/library/delete", url.Values{"id": {"1"}}); response.Code != http.StatusOK || len(jobs.jobs) != 0 {
|
||||||
|
t.Fatalf("delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(target); err != nil {
|
||||||
|
t.Fatalf("symlink target was removed: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(filepath.Join(dir, "link.mp4")); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("symlink itself remains: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVideoSymlinkedDirectoryWithinRootCanBeServedAndDeleted(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.Mkdir(filepath.Join(dir, "target"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
target := filepath.Join(dir, "target", "video.mp4")
|
||||||
|
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink("target", filepath.Join(dir, "linked-directory")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "linked-directory/video.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
if response := getPage(handler, "/videos/1"); response.Code != http.StatusOK || response.Body.String() != "video" {
|
||||||
|
t.Fatalf("serve: status=%d body=%q", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
if response := postForm(handler, "/library/delete", url.Values{"id": {"1"}}); response.Code != http.StatusOK || len(jobs.jobs) != 0 {
|
||||||
|
t.Fatalf("delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(target); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("target file remains: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(filepath.Join(dir, "linked-directory")); err != nil {
|
||||||
|
t.Fatalf("directory symlink was removed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVideoAbsoluteSymlinkWithinRootIsRejectedByOSRoot(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
target := filepath.Join(dir, "target.mp4")
|
||||||
|
if err := os.WriteFile(target, []byte("video"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(target, filepath.Join(dir, "absolute-link.mp4")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "absolute-link.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
if got := getPage(handler, "/videos/1").Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("absolute symlink serve status=%d", got)
|
||||||
|
}
|
||||||
|
if response := postForm(handler, "/library/delete", url.Values{"id": {"1"}}); response.Code != http.StatusOK || len(jobs.jobs) != 1 || !strings.Contains(response.Body.String(), "失敗: 1件") {
|
||||||
|
t.Fatalf("absolute symlink delete: status=%d jobs=%d", response.Code, len(jobs.jobs))
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(target); err != nil {
|
||||||
|
t.Fatalf("target was changed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteVideoFileThenRecord(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
path := filepath.Join(dir, "show", "video.mp4")
|
||||||
|
if err := os.WriteFile(path, []byte("video"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
|
||||||
|
fileWasGoneBeforeRecordDelete := false
|
||||||
|
jobs.beforeDeleteSucceeded = func() {
|
||||||
|
_, err := os.Stat(path)
|
||||||
|
fileWasGoneBeforeRecordDelete = errors.Is(err, os.ErrNotExist)
|
||||||
|
}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
response := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
|
||||||
|
if response.Code != http.StatusOK || len(jobs.jobs) != 0 || !fileWasGoneBeforeRecordDelete || !strings.Contains(response.Body.String(), "削除: 1件") {
|
||||||
|
t.Fatalf("delete: status=%d location=%q jobs=%d", response.Code, response.Header().Get("Location"), len(jobs.jobs))
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(path); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("file remains: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "show")); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("empty series folder remains: %v", err)
|
||||||
|
}
|
||||||
|
jobs.jobs["ep1"] = libraryJob(1, "show/missing.mp4")
|
||||||
|
if got := postForm(handler, "/library/delete", url.Values{"id": {"1"}}).Code; got != http.StatusOK || len(jobs.jobs) != 0 {
|
||||||
|
t.Fatalf("missing file delete: status=%d jobs=%d", got, len(jobs.jobs))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteRedirectsToSeriesWhenEpisodesRemainAndKeepsNonemptyFolder(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.Mkdir(filepath.Join(dir, "show"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "show", "one.mp4"), []byte("one"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "show", "two.mp4"), []byte("two"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
first := libraryJob(1, "show/one.mp4")
|
||||||
|
second := libraryJob(2, "show/two.mp4")
|
||||||
|
second.EpisodeID = "ep2"
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": first, "ep2": second}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
response := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "/library/series/series1") || len(jobs.jobs) != 1 {
|
||||||
|
t.Fatalf("delete: status=%d location=%q jobs=%d", response.Code, response.Header().Get("Location"), len(jobs.jobs))
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "show")); err != nil {
|
||||||
|
t.Fatalf("nonempty series folder removed: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "show", "two.mp4")); err != nil {
|
||||||
|
t.Fatalf("remaining video removed: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteVideoAtRootNeverRemovesVideoRoot(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "video.mp4"), []byte("video"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "video.mp4")}}
|
||||||
|
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil), "/library/delete", url.Values{"id": {"1"}})
|
||||||
|
if response.Code != http.StatusOK {
|
||||||
|
t.Fatalf("delete status=%d", response.Code)
|
||||||
|
}
|
||||||
|
if info, err := os.Stat(dir); err != nil || !info.IsDir() {
|
||||||
|
t.Fatalf("video root was removed: info=%v err=%v", info, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteFailureKeepsRecord(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
blocked := filepath.Join(dir, "blocked")
|
||||||
|
if err := os.Mkdir(blocked, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(blocked, "child"), []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "blocked")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
response := postForm(handler, "/library/delete", url.Values{"id": {"1"}})
|
||||||
|
if response.Code != http.StatusOK || len(jobs.jobs) != 1 || !strings.Contains(response.Body.String(), "失敗: 1件") {
|
||||||
|
t.Fatalf("failure: status=%d jobs=%d body=%s", response.Code, len(jobs.jobs), response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteConfirmationSelectionValidationAndDeduplication(t *testing.T) {
|
||||||
|
first := libraryJob(1, "show/one.mp4")
|
||||||
|
second := libraryJob(2, "show/two.mp4")
|
||||||
|
second.EpisodeID = "ep2"
|
||||||
|
second.Title = `<img src=x onerror=alert(1)>`
|
||||||
|
third := libraryJob(3, "other/three.mp4")
|
||||||
|
third.EpisodeID = "ep3"
|
||||||
|
third.SeriesID = "series2"
|
||||||
|
third.SeriesTitle = "別番組"
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": first, "ep2": second, "ep3": third}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
|
||||||
|
|
||||||
|
for name, values := range map[string]url.Values{
|
||||||
|
"IDのみ": {"id": {"1"}},
|
||||||
|
"番組のみ": {"series_id": {"series2"}},
|
||||||
|
"両方と重複除去": {"id": {"1"}, "series_id": {"series1"}},
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
response := postForm(handler, "/library/delete/confirm", values)
|
||||||
|
if response.Code != http.StatusOK || len(jobs.jobs) != 3 {
|
||||||
|
t.Fatalf("status=%d jobs=%d body=%s", response.Code, len(jobs.jobs), response.Body.String())
|
||||||
|
}
|
||||||
|
if strings.Contains(response.Body.String(), `<img src=x`) || strings.Contains(response.Body.String(), `<script>番組</script>`) {
|
||||||
|
t.Fatal("確認画面でHTMLがエスケープされていない")
|
||||||
|
}
|
||||||
|
if name == "両方と重複除去" && (!strings.Contains(response.Body.String(), `<script>番組</script>`) || !strings.Contains(response.Body.String(), `<img src=x onerror=alert(1)>`)) {
|
||||||
|
t.Fatalf("番組名または回タイトルのエスケープ結果がない: %s", response.Body.String())
|
||||||
|
}
|
||||||
|
if name == "両方と重複除去" && strings.Count(response.Body.String(), `name="id"`) != 2 {
|
||||||
|
t.Fatalf("重複除去後のhidden ID: %s", response.Body.String())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
for name, values := range map[string]url.Values{
|
||||||
|
"対象なし": {},
|
||||||
|
"該当なし": {"id": {"99"}},
|
||||||
|
"不正ID": {"id": {"bad"}},
|
||||||
|
"ゼロID": {"id": {"0"}},
|
||||||
|
"不正番組ID": {"series_id": {"bad-id"}},
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
if got := postForm(handler, "/library/delete/confirm", values).Code; got != http.StatusBadRequest {
|
||||||
|
t.Fatalf("status=%d", got)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
tooMany := url.Values{}
|
||||||
|
for i := 1; i <= maxDeleteSelections+1; i++ {
|
||||||
|
tooMany.Add("id", strconv.Itoa(i))
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/library/delete/confirm", tooMany).Code; got != http.StatusBadRequest {
|
||||||
|
t.Fatalf("上限超過 status=%d", got)
|
||||||
|
}
|
||||||
|
atLimit := url.Values{}
|
||||||
|
for i := 1; i <= maxDeleteSelections; i++ {
|
||||||
|
atLimit.Add("id", strconv.Itoa(i))
|
||||||
|
}
|
||||||
|
if response := postForm(handler, "/library/delete/confirm", atLimit); response.Code != http.StatusOK || len(jobs.jobs) != 3 {
|
||||||
|
t.Fatalf("入力上限ちょうど: status=%d jobs=%d", response.Code, len(jobs.jobs))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteConfirmationRejectsExpandedSelectionOverLimit(t *testing.T) {
|
||||||
|
makeJobs := func(count int) map[string]store.Job {
|
||||||
|
jobs := make(map[string]store.Job, count)
|
||||||
|
for i := 1; i <= count; i++ {
|
||||||
|
job := libraryJob(int64(i), "show/video.mp4")
|
||||||
|
job.EpisodeID = "ep" + strconv.Itoa(i)
|
||||||
|
jobs[job.EpisodeID] = job
|
||||||
|
}
|
||||||
|
return jobs
|
||||||
|
}
|
||||||
|
|
||||||
|
atLimit := &fakeJobStore{jobs: makeJobs(maxDeleteSelections)}
|
||||||
|
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, atLimit, t.TempDir(), nil),
|
||||||
|
"/library/delete/confirm", url.Values{"series_id": {"series1"}})
|
||||||
|
if response.Code != http.StatusOK || len(atLimit.jobs) != maxDeleteSelections {
|
||||||
|
t.Fatalf("上限ちょうど: status=%d jobs=%d", response.Code, len(atLimit.jobs))
|
||||||
|
}
|
||||||
|
|
||||||
|
overLimit := &fakeJobStore{jobs: makeJobs(maxDeleteSelections + 1)}
|
||||||
|
response = postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, overLimit, t.TempDir(), nil),
|
||||||
|
"/library/delete/confirm", url.Values{"series_id": {"series1"}})
|
||||||
|
if response.Code != http.StatusBadRequest || len(overLimit.jobs) != maxDeleteSelections+1 ||
|
||||||
|
!strings.Contains(response.Body.String(), strconv.Itoa(maxDeleteSelections+1)+"本") ||
|
||||||
|
!strings.Contains(response.Body.String(), "番組を分けて選んでください") {
|
||||||
|
t.Fatalf("上限超過: status=%d jobs=%d body=%s", response.Code, len(overLimit.jobs), response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBulkDeleteContinuesAfterFailureAndDoesNotExpandSeries(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.MkdirAll(filepath.Join(dir, "show"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "show", "one.mp4"), []byte("one"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Mkdir(filepath.Join(dir, "show", "blocked"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "show", "blocked", "child"), []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
first := libraryJob(1, "show/one.mp4")
|
||||||
|
blocked := libraryJob(2, "show/blocked")
|
||||||
|
blocked.EpisodeID = "ep2"
|
||||||
|
blocked.Title = "削除失敗回"
|
||||||
|
newAfterConfirmation := libraryJob(3, "show/new.mp4")
|
||||||
|
newAfterConfirmation.EpisodeID = "ep3"
|
||||||
|
newAfterConfirmation.Title = "確認後の新着"
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": first, "ep2": blocked, "ep3": newAfterConfirmation}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, dir, nil)
|
||||||
|
|
||||||
|
response := postForm(handler, "/library/delete", url.Values{
|
||||||
|
"id": {"1", "2", "99"},
|
||||||
|
"series_id": {"series1"},
|
||||||
|
})
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "削除: 1件") || !strings.Contains(response.Body.String(), "失敗: 1件") || !strings.Contains(response.Body.String(), "対象外: 1件") {
|
||||||
|
t.Fatalf("status=%d body=%s", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
if _, ok := jobs.jobs["ep1"]; ok {
|
||||||
|
t.Fatal("正常な回の記録が残った")
|
||||||
|
}
|
||||||
|
if _, ok := jobs.jobs["ep2"]; !ok {
|
||||||
|
t.Fatal("失敗した回の記録が消えた")
|
||||||
|
}
|
||||||
|
if _, ok := jobs.jobs["ep3"]; !ok {
|
||||||
|
t.Fatal("確認後に増えた回が消えた")
|
||||||
|
}
|
||||||
|
if strings.Contains(response.Body.String(), filepath.Join(dir, "show", "blocked")) {
|
||||||
|
t.Fatal("内部パスが結果画面に出た")
|
||||||
|
}
|
||||||
|
for _, detail := range []string{"directory not empty", "show/blocked"} {
|
||||||
|
if strings.Contains(response.Body.String(), detail) {
|
||||||
|
t.Fatalf("エラー詳細が結果画面に出た: %q", detail)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBulkDeleteWhenVideoRootCannotOpenDeletesOnlyRecordWithoutFile(t *testing.T) {
|
||||||
|
withFile := libraryJob(1, "show/video.mp4")
|
||||||
|
withFile.Title = `<img src=x onerror=alert(1)>`
|
||||||
|
withoutFile := libraryJob(2, "")
|
||||||
|
withoutFile.EpisodeID = "ep2"
|
||||||
|
withoutFile.Title = "記録だけの回"
|
||||||
|
withoutFile.FilePath = nil
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": withFile, "ep2": withoutFile}}
|
||||||
|
missingRoot := filepath.Join(t.TempDir(), "missing-video-root")
|
||||||
|
|
||||||
|
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, missingRoot, nil),
|
||||||
|
"/library/delete", url.Values{"id": {"1", "2"}})
|
||||||
|
if response.Code != http.StatusOK ||
|
||||||
|
!strings.Contains(response.Body.String(), "削除: 1件") ||
|
||||||
|
!strings.Contains(response.Body.String(), "失敗: 1件") {
|
||||||
|
t.Fatalf("status=%d body=%s", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
if _, ok := jobs.jobs["ep1"]; !ok {
|
||||||
|
t.Fatal("動画ルートを開けない回の記録が消えた")
|
||||||
|
}
|
||||||
|
if _, ok := jobs.jobs["ep2"]; ok {
|
||||||
|
t.Fatal("file_pathがNULLの回の記録が残った")
|
||||||
|
}
|
||||||
|
for _, secret := range []string{missingRoot, "no such file or directory", `<img src=x`} {
|
||||||
|
if strings.Contains(response.Body.String(), secret) {
|
||||||
|
t.Fatalf("内部情報または未エスケープHTMLが結果画面に出た: %q", secret)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !strings.Contains(response.Body.String(), `<img src=x onerror=alert(1)>`) {
|
||||||
|
t.Fatalf("失敗した回のタイトルが安全に表示されていない: %s", response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBulkDeleteRejectsEmptySelection(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
|
||||||
|
response := postForm(newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil), "/library/delete", nil)
|
||||||
|
if response.Code != http.StatusBadRequest || len(jobs.jobs) != 1 {
|
||||||
|
t.Fatalf("status=%d jobs=%d", response.Code, len(jobs.jobs))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLibraryDeleteEntryFormsAndOldRoute(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep1": libraryJob(1, "show/video.mp4")}}
|
||||||
|
handler := newTestHandlerWithVideoDir(&fakeFetcher{}, jobs, t.TempDir(), nil)
|
||||||
|
for _, path := range []string{"/library", "/library/series/series1", "/library/1"} {
|
||||||
|
response := getPage(handler, path)
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), `action="/library/delete/confirm"`) {
|
||||||
|
t.Fatalf("%s: status=%d body=%s", path, response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if body := getPage(handler, "/library/1").Body.String(); strings.Contains(body, "hx-confirm") {
|
||||||
|
t.Fatal("再生ページにhx-confirmが残っている")
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/library/1/delete", nil).Code; got != http.StatusNotFound && got != http.StatusMethodNotAllowed {
|
||||||
|
t.Fatalf("旧経路 status=%d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func sortJobsNaturally(jobs []store.Job) {
|
||||||
|
sort.SliceStable(jobs, func(i, j int) bool {
|
||||||
|
if comparison := compareNaturalTitle(jobs[i].Title, jobs[j].Title); comparison != 0 {
|
||||||
|
return comparison < 0
|
||||||
|
}
|
||||||
|
left, right := jobs[i].FinishedAt, jobs[j].FinishedAt
|
||||||
|
if left == nil || right == nil {
|
||||||
|
if left != right {
|
||||||
|
return left == nil
|
||||||
|
}
|
||||||
|
} else if !left.Equal(*right) {
|
||||||
|
return left.Before(*right)
|
||||||
|
}
|
||||||
|
return jobs[i].ID < jobs[j].ID
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func compareNaturalTitle(left, right string) int {
|
||||||
|
a, b := []rune(left), []rune(right)
|
||||||
|
ai, bi := 0, 0
|
||||||
|
for ai < len(a) && bi < len(b) {
|
||||||
|
_, aIsDigit := decimalDigit(a[ai])
|
||||||
|
_, bIsDigit := decimalDigit(b[bi])
|
||||||
|
if aIsDigit && bIsDigit {
|
||||||
|
var an, bn strings.Builder
|
||||||
|
for ai < len(a) {
|
||||||
|
digit, ok := decimalDigit(a[ai])
|
||||||
|
if !ok {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
an.WriteByte(byte('0' + digit))
|
||||||
|
ai++
|
||||||
|
}
|
||||||
|
for bi < len(b) {
|
||||||
|
digit, ok := decimalDigit(b[bi])
|
||||||
|
if !ok {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
bn.WriteByte(byte('0' + digit))
|
||||||
|
bi++
|
||||||
|
}
|
||||||
|
if comparison := compareDecimal(an.String(), bn.String()); comparison != 0 {
|
||||||
|
return comparison
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if a[ai] < b[bi] {
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
if a[ai] > b[bi] {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
ai++
|
||||||
|
bi++
|
||||||
|
}
|
||||||
|
if ai == len(a) && bi < len(b) {
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
if bi == len(b) && ai < len(a) {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func decimalDigit(r rune) (int, bool) {
|
||||||
|
switch {
|
||||||
|
case '0' <= r && r <= '9':
|
||||||
|
return int(r - '0'), true
|
||||||
|
case '０' <= r && r <= '９':
|
||||||
|
return int(r - '０'), true
|
||||||
|
default:
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func compareDecimal(left, right string) int {
|
||||||
|
left = strings.TrimLeft(left, "0")
|
||||||
|
right = strings.TrimLeft(right, "0")
|
||||||
|
if left == "" {
|
||||||
|
left = "0"
|
||||||
|
}
|
||||||
|
if right == "" {
|
||||||
|
right = "0"
|
||||||
|
}
|
||||||
|
if len(left) < len(right) {
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
if len(left) > len(right) {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return strings.Compare(left, right)
|
||||||
|
}
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSortJobsNaturally(t *testing.T) {
|
||||||
|
base := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
|
||||||
|
jobs := []store.Job{
|
||||||
|
{ID: 8, Title: "第10話", FinishedAt: timePointer(base)},
|
||||||
|
{ID: 7, Title: "第２話", FinishedAt: timePointer(base)},
|
||||||
|
{ID: 6, Title: "#10", FinishedAt: timePointer(base)},
|
||||||
|
{ID: 5, Title: "#9", FinishedAt: timePointer(base)},
|
||||||
|
{ID: 4, Title: "第2話", FinishedAt: timePointer(base.Add(time.Hour))},
|
||||||
|
{ID: 3, Title: "第２話", FinishedAt: timePointer(base.Add(-time.Hour))},
|
||||||
|
{ID: 2, Title: "第02話", FinishedAt: timePointer(base.Add(-time.Hour))},
|
||||||
|
{ID: 1, Title: "おまけ", FinishedAt: timePointer(base)},
|
||||||
|
}
|
||||||
|
sortJobsNaturally(jobs)
|
||||||
|
got := make([]int64, len(jobs))
|
||||||
|
for i := range jobs {
|
||||||
|
got[i] = jobs[i].ID
|
||||||
|
}
|
||||||
|
want := []int64{5, 6, 1, 2, 3, 7, 4, 8}
|
||||||
|
for i := range want {
|
||||||
|
if got[i] != want[i] {
|
||||||
|
t.Fatalf("自然順=%v want=%v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNaturalOrderUsesIDAfterEqualTitleAndFinishedAt(t *testing.T) {
|
||||||
|
finished := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
|
||||||
|
jobs := []store.Job{
|
||||||
|
{ID: 20, Title: "#９", FinishedAt: &finished},
|
||||||
|
{ID: 10, Title: "#9", FinishedAt: &finished},
|
||||||
|
}
|
||||||
|
sortJobsNaturally(jobs)
|
||||||
|
if jobs[0].ID != 10 || jobs[1].ID != 20 {
|
||||||
|
t.Fatalf("同順位のID順: %+v", jobs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNaturalOrderTreatsLeadingZerosAsTheSameNumber(t *testing.T) {
|
||||||
|
base := time.Date(2026, 10, 10, 0, 0, 0, 0, time.UTC)
|
||||||
|
jobs := []store.Job{
|
||||||
|
{ID: 30, Title: "数字なし", FinishedAt: timePointer(base.Add(-time.Hour))},
|
||||||
|
{ID: 20, Title: "#09", FinishedAt: timePointer(base)},
|
||||||
|
{ID: 10, Title: "#9", FinishedAt: timePointer(base.Add(time.Hour))},
|
||||||
|
}
|
||||||
|
sortJobsNaturally(jobs)
|
||||||
|
if jobs[0].ID != 20 || jobs[1].ID != 10 || jobs[2].ID != 30 {
|
||||||
|
t.Fatalf("先頭ゼロと数字なしの自然順: %+v", jobs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func timePointer(value time.Time) *time.Time { return &value }
|
||||||
@@ -0,0 +1,240 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"regexp"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
newerCacheTTL = 10 * time.Minute
|
||||||
|
)
|
||||||
|
|
||||||
|
var platformTokenPattern = regexp.MustCompile(`(?i)(platform_token=)[^&\s]+`)
|
||||||
|
|
||||||
|
type NewerFetcher interface {
|
||||||
|
FetchNewer(context.Context, string) ([]tver.Episode, error)
|
||||||
|
FetchSeries(context.Context, string) (string, error)
|
||||||
|
FetchSeriesEpisodes(context.Context, string) ([]tver.Episode, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type cacheEntry struct {
|
||||||
|
episodes []tver.Episode
|
||||||
|
expiresAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type newerOptions struct {
|
||||||
|
genre string
|
||||||
|
hidePreviews bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type episodeView struct {
|
||||||
|
ID string
|
||||||
|
SeriesID string
|
||||||
|
SubscriptionAction subscriptionActionView
|
||||||
|
Genre string
|
||||||
|
Status string
|
||||||
|
StatusText string
|
||||||
|
URL string
|
||||||
|
ThumbnailURL string
|
||||||
|
SeriesTitle string
|
||||||
|
Title string
|
||||||
|
BroadcasterName string
|
||||||
|
DurationMinutes int64
|
||||||
|
BroadcastDateLabel string
|
||||||
|
EndAtText string
|
||||||
|
EndAtISO string
|
||||||
|
}
|
||||||
|
|
||||||
|
type newerView struct {
|
||||||
|
Genre string
|
||||||
|
HidePreviews bool
|
||||||
|
Episodes []episodeView
|
||||||
|
Failed bool
|
||||||
|
ErrorText string
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseNewerOptions(r *http.Request) (newerOptions, error) {
|
||||||
|
options := newerOptions{genre: "drama", hidePreviews: true}
|
||||||
|
query := r.URL.Query()
|
||||||
|
if values, ok := query["genre"]; ok {
|
||||||
|
options.genre = values[len(values)-1]
|
||||||
|
}
|
||||||
|
if options.genre != "drama" && options.genre != "variety" {
|
||||||
|
return newerOptions{}, fmt.Errorf("invalid genre")
|
||||||
|
}
|
||||||
|
if values, ok := query["hide_previews"]; ok {
|
||||||
|
switch values[len(values)-1] {
|
||||||
|
case "0":
|
||||||
|
options.hidePreviews = false
|
||||||
|
case "1":
|
||||||
|
options.hidePreviews = true
|
||||||
|
default:
|
||||||
|
return newerOptions{}, fmt.Errorf("invalid hide_previews")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return options, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (options newerOptions) pageURL() string {
|
||||||
|
query := url.Values{}
|
||||||
|
query.Set("genre", options.genre)
|
||||||
|
if options.hidePreviews {
|
||||||
|
query.Set("hide_previews", "1")
|
||||||
|
} else {
|
||||||
|
query.Set("hide_previews", "0")
|
||||||
|
}
|
||||||
|
return "/?" + query.Encode()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) home(w http.ResponseWriter, r *http.Request) {
|
||||||
|
options, err := parseNewerOptions(r)
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
view := s.newerView(r.Context(), options)
|
||||||
|
s.render(w, "layout", page{Title: "新着", Newer: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) newerFragment(w http.ResponseWriter, r *http.Request) {
|
||||||
|
options, err := parseNewerOptions(r)
|
||||||
|
if err != nil {
|
||||||
|
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
view := s.newerView(r.Context(), options)
|
||||||
|
w.Header().Set("HX-Push-Url", options.pageURL())
|
||||||
|
s.render(w, "newer_results", view)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) render(w http.ResponseWriter, name string, value any) {
|
||||||
|
var body bytes.Buffer
|
||||||
|
if err := s.tmpl.ExecuteTemplate(&body, name, value); err != nil {
|
||||||
|
s.logger.Printf("テンプレートの描画に失敗: %v", err)
|
||||||
|
http.Error(w, "template error", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||||
|
_, _ = body.WriteTo(w)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) newerView(ctx context.Context, options newerOptions) newerView {
|
||||||
|
view := newerView{Genre: options.genre, HidePreviews: options.hidePreviews}
|
||||||
|
episodes, err := s.cachedNewer(ctx, options.genre)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("TVer の新着取得に失敗（genre=%s）: %s", options.genre, platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
|
||||||
|
view.Failed = true
|
||||||
|
view.ErrorText = "新着を取得できませんでした"
|
||||||
|
return view
|
||||||
|
}
|
||||||
|
visible := make([]tver.Episode, 0, len(episodes))
|
||||||
|
for _, episode := range episodes {
|
||||||
|
if options.hidePreviews && episode.Duration <= tver.PreviewMaxSeconds {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
visible = append(visible, episode)
|
||||||
|
}
|
||||||
|
ids := make([]string, 0, len(visible))
|
||||||
|
for _, episode := range visible {
|
||||||
|
ids = append(ids, episode.ID)
|
||||||
|
}
|
||||||
|
statuses, err := s.jobs.EpisodeStatuses(ctx, ids)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("新着のジョブ状態取得に失敗: %v", err)
|
||||||
|
view.Failed = true
|
||||||
|
view.ErrorText = "状態を取得できませんでした"
|
||||||
|
return view
|
||||||
|
}
|
||||||
|
seriesIDs := make([]string, 0, len(visible))
|
||||||
|
seenSeries := make(map[string]bool)
|
||||||
|
for _, episode := range visible {
|
||||||
|
if episode.SeriesID != "" && !seenSeries[episode.SeriesID] {
|
||||||
|
seriesIDs = append(seriesIDs, episode.SeriesID)
|
||||||
|
seenSeries[episode.SeriesID] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
subscribed, err := s.jobs.SubscribedSeries(ctx, seriesIDs)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("登録番組の状態取得に失敗: %v", err)
|
||||||
|
view.Failed = true
|
||||||
|
view.ErrorText = "登録番組の状態を取得できませんでした"
|
||||||
|
return view
|
||||||
|
}
|
||||||
|
for _, episode := range visible {
|
||||||
|
endAt := time.Unix(episode.EndAt, 0).In(s.zone)
|
||||||
|
view.Episodes = append(view.Episodes, episodeView{
|
||||||
|
ID: episode.ID,
|
||||||
|
SeriesID: episode.SeriesID,
|
||||||
|
SubscriptionAction: subscriptionActionView{ID: episode.ID, Genre: options.genre, Subscribed: subscribed[episode.SeriesID]},
|
||||||
|
Genre: options.genre,
|
||||||
|
Status: statuses[episode.ID],
|
||||||
|
StatusText: statusText(statuses[episode.ID]),
|
||||||
|
URL: "https://tver.jp/episodes/" + url.PathEscape(episode.ID),
|
||||||
|
ThumbnailURL: "https://statics.tver.jp" + episode.ThumbnailPath,
|
||||||
|
SeriesTitle: episode.SeriesTitle,
|
||||||
|
Title: episode.Title,
|
||||||
|
BroadcasterName: episode.BroadcasterName,
|
||||||
|
DurationMinutes: durationMinutes(episode.Duration),
|
||||||
|
BroadcastDateLabel: episode.BroadcastDateLabel,
|
||||||
|
EndAtText: endAt.Format("2006-01-02 15:04 MST"),
|
||||||
|
EndAtISO: endAt.Format(time.RFC3339),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return view
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) cachedEpisode(genre, id string) (tver.Episode, bool) {
|
||||||
|
s.cacheMu.Lock()
|
||||||
|
defer s.cacheMu.Unlock()
|
||||||
|
entry, ok := s.cache[genre]
|
||||||
|
if !ok || !s.now().Before(entry.expiresAt) {
|
||||||
|
return tver.Episode{}, false
|
||||||
|
}
|
||||||
|
for _, ep := range entry.episodes {
|
||||||
|
if ep.ID == id {
|
||||||
|
return ep, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return tver.Episode{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
func storeEpisode(ep tver.Episode) store.Episode {
|
||||||
|
var endAt *time.Time
|
||||||
|
if ep.EndAt > 0 {
|
||||||
|
end := time.Unix(ep.EndAt, 0)
|
||||||
|
endAt = &end
|
||||||
|
}
|
||||||
|
return store.Episode{ID: ep.ID, SeriesID: ep.SeriesID, SeriesTitle: ep.SeriesTitle,
|
||||||
|
Title: ep.Title, BroadcasterName: ep.BroadcasterName, DurationSeconds: int(ep.Duration),
|
||||||
|
EndAt: endAt, ThumbnailPath: ep.ThumbnailPath}
|
||||||
|
}
|
||||||
|
|
||||||
|
func durationMinutes(seconds int64) int64 {
|
||||||
|
minutes := seconds / 60
|
||||||
|
if seconds%60 != 0 {
|
||||||
|
minutes++
|
||||||
|
}
|
||||||
|
return max(1, minutes)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) cachedNewer(ctx context.Context, genre string) ([]tver.Episode, error) {
|
||||||
|
s.cacheMu.Lock()
|
||||||
|
defer s.cacheMu.Unlock()
|
||||||
|
if cached, ok := s.cache[genre]; ok && s.now().Before(cached.expiresAt) {
|
||||||
|
return cached.episodes, nil
|
||||||
|
}
|
||||||
|
episodes, err := s.fetcher.FetchNewer(ctx, genre)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
s.cache[genre] = cacheEntry{episodes: episodes, expiresAt: s.now().Add(newerCacheTTL)}
|
||||||
|
return episodes, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,271 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeFetcher struct {
|
||||||
|
episodes []tver.Episode
|
||||||
|
episodesByGenre map[string][]tver.Episode
|
||||||
|
err error
|
||||||
|
calls []string
|
||||||
|
seriesEpisodes []tver.Episode
|
||||||
|
seriesErr error
|
||||||
|
seriesCalls []string
|
||||||
|
seriesTitle string
|
||||||
|
seriesTitleErr error
|
||||||
|
seriesTitleCalls []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeFetcher) FetchSeries(_ context.Context, seriesID string) (string, error) {
|
||||||
|
f.seriesTitleCalls = append(f.seriesTitleCalls, seriesID)
|
||||||
|
return f.seriesTitle, f.seriesTitleErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeFetcher) FetchNewer(_ context.Context, genre string) ([]tver.Episode, error) {
|
||||||
|
f.calls = append(f.calls, genre)
|
||||||
|
if f.episodesByGenre != nil {
|
||||||
|
return f.episodesByGenre[genre], f.err
|
||||||
|
}
|
||||||
|
return f.episodes, f.err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeFetcher) FetchSeriesEpisodes(_ context.Context, seriesID string) ([]tver.Episode, error) {
|
||||||
|
f.seriesCalls = append(f.seriesCalls, seriesID)
|
||||||
|
return f.seriesEpisodes, f.seriesErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func newTestHandler(fetcher *fakeFetcher, now func() time.Time) http.Handler {
|
||||||
|
return newTestHandlerWithStore(fetcher, &fakeJobStore{}, now)
|
||||||
|
}
|
||||||
|
|
||||||
|
func newTestHandlerWithStore(fetcher *fakeFetcher, jobs *fakeJobStore, now func() time.Time) http.Handler {
|
||||||
|
return newTestHandlerWithVideoDir(fetcher, jobs, "", now)
|
||||||
|
}
|
||||||
|
|
||||||
|
func newTestHandlerWithVideoDir(fetcher *fakeFetcher, jobs *fakeJobStore, videoDir string, now func() time.Time) http.Handler {
|
||||||
|
if now == nil {
|
||||||
|
now = func() time.Time { return time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC) }
|
||||||
|
}
|
||||||
|
return NewHandler(fetcher, jobs, videoDir, now, log.New(io.Discard, "", 0))
|
||||||
|
}
|
||||||
|
|
||||||
|
func getPage(handler http.Handler, path string) *httptest.ResponseRecorder {
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, httptest.NewRequest(http.MethodGet, path, nil))
|
||||||
|
return response
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerPageShowsEpisodeFieldsAndJST(t *testing.T) {
|
||||||
|
endAt := time.Date(2026, 10, 3, 21, 34, 0, 0, time.UTC).Unix()
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{
|
||||||
|
ID: "ep123", SeriesTitle: "番組名", Title: "第1話", BroadcasterName: "放送局",
|
||||||
|
Duration: 2712, EndAt: endAt, ThumbnailPath: "/images/ep123.jpg",
|
||||||
|
BroadcastDateLabel: "10月3日放送分",
|
||||||
|
}}}
|
||||||
|
response := getPage(newTestHandler(fetcher, nil), "/")
|
||||||
|
if response.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d", response.Code)
|
||||||
|
}
|
||||||
|
body := response.Body.String()
|
||||||
|
for _, want := range []string{
|
||||||
|
"番組名", "第1話", "放送局", "10月3日放送分",
|
||||||
|
"長さ: 46分",
|
||||||
|
"https://statics.tver.jp/images/ep123.jpg",
|
||||||
|
"https://tver.jp/episodes/ep123",
|
||||||
|
"2026-10-04 06:34 JST", `datetime="2026-10-04T06:34:00+09:00"`,
|
||||||
|
`value="drama" selected`, `name="hide_previews" value="1" checked`,
|
||||||
|
`hx-get="/newer"`, `hx-target="#newer-results"`,
|
||||||
|
} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("page missing %q", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(fetcher.calls) != 1 || fetcher.calls[0] != "drama" {
|
||||||
|
t.Errorf("calls = %v", fetcher.calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDurationMinutesRoundsUp(t *testing.T) {
|
||||||
|
for _, tt := range []struct {
|
||||||
|
seconds int64
|
||||||
|
minutes int64
|
||||||
|
}{
|
||||||
|
{0, 1}, {1, 1}, {59, 1}, {60, 1}, {61, 2}, {2712, 46},
|
||||||
|
} {
|
||||||
|
if got := durationMinutes(tt.seconds); got != tt.minutes {
|
||||||
|
t.Errorf("%d seconds = %d minutes; want %d", tt.seconds, got, tt.minutes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerFragmentUpdatesURLAndFullPageRestoresState(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "variety-1", Duration: 200}}}
|
||||||
|
handler := newTestHandler(fetcher, nil)
|
||||||
|
response := getPage(handler, "/newer?genre=variety&hide_previews=0")
|
||||||
|
if response.Code != http.StatusOK {
|
||||||
|
t.Fatalf("fragment status = %d", response.Code)
|
||||||
|
}
|
||||||
|
if got := response.Header().Get("HX-Push-Url"); got != "/?genre=variety&hide_previews=0" {
|
||||||
|
t.Errorf("HX-Push-Url = %q", got)
|
||||||
|
}
|
||||||
|
if body := response.Body.String(); !strings.Contains(body, `id="newer-results"`) || !strings.Contains(body, "variety-1") || strings.Contains(body, "<html") {
|
||||||
|
t.Errorf("not an episode fragment: %s", body)
|
||||||
|
}
|
||||||
|
reload := getPage(handler, "/?genre=variety&hide_previews=0")
|
||||||
|
if reload.Code != http.StatusOK || !strings.Contains(reload.Body.String(), `value="variety" selected`) || strings.Contains(reload.Body.String(), `value="1" checked`) || !strings.Contains(reload.Body.String(), "variety-1") {
|
||||||
|
t.Errorf("reload did not restore state: status=%d body=%s", reload.Code, reload.Body.String())
|
||||||
|
}
|
||||||
|
if len(fetcher.calls) != 1 || fetcher.calls[0] != "variety" {
|
||||||
|
t.Errorf("calls = %v", fetcher.calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerInvalidGenreReturns400WithoutFetch(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{}
|
||||||
|
handler := newTestHandler(fetcher, nil)
|
||||||
|
for _, path := range []string{"/?genre=news", "/newer?genre=news", "/?genre=", "/?hide_previews=invalid", "/newer?hide_previews="} {
|
||||||
|
response := getPage(handler, path)
|
||||||
|
if response.Code != http.StatusBadRequest {
|
||||||
|
t.Errorf("%s: status = %d", path, response.Code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(fetcher.calls) != 0 {
|
||||||
|
t.Errorf("unexpected fetch calls: %v", fetcher.calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPreviewFilterBoundaryAndToggle(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{
|
||||||
|
{ID: "preview-180", Duration: 180},
|
||||||
|
{ID: "episode-181", Duration: 181},
|
||||||
|
}}
|
||||||
|
handler := newTestHandler(fetcher, nil)
|
||||||
|
hidden := getPage(handler, "/")
|
||||||
|
if strings.Contains(hidden.Body.String(), "preview-180") || !strings.Contains(hidden.Body.String(), "episode-181") {
|
||||||
|
t.Errorf("default filter: %s", hidden.Body.String())
|
||||||
|
}
|
||||||
|
shown := getPage(handler, "/newer?genre=drama&hide_previews=0")
|
||||||
|
if !strings.Contains(shown.Body.String(), "preview-180") || !strings.Contains(shown.Body.String(), "episode-181") {
|
||||||
|
t.Errorf("show previews: %s", shown.Body.String())
|
||||||
|
}
|
||||||
|
checked := getPage(handler, "/newer?genre=drama&hide_previews=0&hide_previews=1")
|
||||||
|
if strings.Contains(checked.Body.String(), "preview-180") || checked.Header().Get("HX-Push-Url") != "/?genre=drama&hide_previews=1" {
|
||||||
|
t.Errorf("checked form state: %s", checked.Body.String())
|
||||||
|
}
|
||||||
|
if len(fetcher.calls) != 1 {
|
||||||
|
t.Errorf("filter caused %d TVer fetches; want 1", len(fetcher.calls))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerCacheExpiresPerGenre(t *testing.T) {
|
||||||
|
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}
|
||||||
|
handler := newTestHandler(fetcher, func() time.Time { return now })
|
||||||
|
getPage(handler, "/")
|
||||||
|
now = now.Add(9*time.Minute + 59*time.Second)
|
||||||
|
getPage(handler, "/?genre=drama&hide_previews=0")
|
||||||
|
getPage(handler, "/?genre=variety")
|
||||||
|
if got := strings.Join(fetcher.calls, ","); got != "drama,variety" {
|
||||||
|
t.Fatalf("before expiry calls = %q", got)
|
||||||
|
}
|
||||||
|
now = now.Add(2 * time.Second)
|
||||||
|
getPage(handler, "/")
|
||||||
|
getPage(handler, "/?genre=variety")
|
||||||
|
if got := strings.Join(fetcher.calls, ","); got != "drama,variety,drama" {
|
||||||
|
t.Errorf("after expiry calls = %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerCacheKeepsGenreResultsSeparateAndExpiresAtTenMinutes(t *testing.T) {
|
||||||
|
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
fetcher := &fakeFetcher{episodesByGenre: map[string][]tver.Episode{
|
||||||
|
"drama": {{ID: "drama-only", Duration: 200}},
|
||||||
|
"variety": {{ID: "variety-only", Duration: 200}},
|
||||||
|
}}
|
||||||
|
handler := newTestHandler(fetcher, func() time.Time { return now })
|
||||||
|
checkPage := func(path, wantID, absentID string) {
|
||||||
|
t.Helper()
|
||||||
|
response := getPage(handler, path)
|
||||||
|
body := response.Body.String()
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(body, "/episodes/"+wantID) || strings.Contains(body, "/episodes/"+absentID) {
|
||||||
|
t.Errorf("%s at %s: status=%d, wrong genre contents: %s", path, now.Format(time.RFC3339), response.Code, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
checkCalls := func(want string) {
|
||||||
|
t.Helper()
|
||||||
|
if got := strings.Join(fetcher.calls, ","); got != want {
|
||||||
|
t.Errorf("fetch calls = %q; want %q", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
checkPage("/", "drama-only", "variety-only")
|
||||||
|
now = now.Add(5 * time.Minute)
|
||||||
|
checkPage("/?genre=variety", "variety-only", "drama-only")
|
||||||
|
now = now.Add(4*time.Minute + 59*time.Second)
|
||||||
|
checkPage("/", "drama-only", "variety-only")
|
||||||
|
checkCalls("drama,variety")
|
||||||
|
now = now.Add(time.Second)
|
||||||
|
checkPage("/", "drama-only", "variety-only")
|
||||||
|
checkPage("/?genre=variety", "variety-only", "drama-only")
|
||||||
|
checkCalls("drama,variety,drama")
|
||||||
|
now = now.Add(5 * time.Minute)
|
||||||
|
checkPage("/?genre=variety", "variety-only", "drama-only")
|
||||||
|
checkCalls("drama,variety,drama,variety")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerPageEscapesUntrustedEpisodeFields(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{
|
||||||
|
ID: "safe-id", SeriesTitle: `<script>alert(1)</script>`, Title: `<b>title</b>`,
|
||||||
|
BroadcasterName: `<img src=x onerror=alert(1)>`, BroadcastDateLabel: `<svg onload=alert(1)>`,
|
||||||
|
ThumbnailPath: `/image.jpg" onerror="alert(1)`, Duration: 200,
|
||||||
|
}}}
|
||||||
|
response := getPage(newTestHandler(fetcher, nil), "/")
|
||||||
|
if response.Code != http.StatusOK {
|
||||||
|
t.Fatalf("status = %d", response.Code)
|
||||||
|
}
|
||||||
|
body := response.Body.String()
|
||||||
|
for _, unsafe := range []string{"<script>alert(1)</script>", "<b>title</b>", "<img src=x", "<svg onload", `onerror="alert(1)"`} {
|
||||||
|
if strings.Contains(body, unsafe) {
|
||||||
|
t.Errorf("untrusted markup was not escaped: %q", unsafe)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, escaped := range []string{"<script>", "<b>title</b>", "<img src=x onerror=alert(1)>", "<svg onload=alert(1)>"} {
|
||||||
|
if !strings.Contains(body, escaped) {
|
||||||
|
t.Errorf("escaped field missing: %q", escaped)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerFailureShowsFriendlyMessageAndRedactsToken(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{err: errors.New("GET https://platform-api.tver.jp/?platform_token=secret-123&x=1: upstream unavailable")}
|
||||||
|
var logs bytes.Buffer
|
||||||
|
handler := NewHandler(fetcher, &fakeJobStore{}, "", time.Now, log.New(&logs, "", 0))
|
||||||
|
response := getPage(handler, "/")
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "新着を取得できませんでした") {
|
||||||
|
t.Errorf("error page: status=%d body=%s", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
if strings.Contains(response.Body.String(), "upstream") || strings.Contains(response.Body.String(), "secret-123") {
|
||||||
|
t.Errorf("error detail leaked into page: %s", response.Body.String())
|
||||||
|
}
|
||||||
|
fragment := getPage(handler, "/newer?genre=variety&hide_previews=0")
|
||||||
|
if fragment.Code != http.StatusOK || !strings.Contains(fragment.Body.String(), `id="newer-results"`) || !strings.Contains(fragment.Body.String(), "新着を取得できませんでした") || strings.Contains(fragment.Body.String(), "<html") {
|
||||||
|
t.Errorf("error fragment: status=%d body=%s", fragment.Code, fragment.Body.String())
|
||||||
|
}
|
||||||
|
if fragment.Header().Get("HX-Push-Url") != "/?genre=variety&hide_previews=0" || strings.Contains(fragment.Body.String(), "secret-123") {
|
||||||
|
t.Errorf("error fragment URL or token leak: header=%q body=%s", fragment.Header().Get("HX-Push-Url"), fragment.Body.String())
|
||||||
|
}
|
||||||
|
if !strings.Contains(logs.String(), "upstream unavailable") || strings.Contains(logs.String(), "secret-123") {
|
||||||
|
t.Errorf("log detail or token redaction failed: %s", logs.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,194 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
const queueListLimit = 50
|
||||||
|
|
||||||
|
type queueJobView struct {
|
||||||
|
store.Job
|
||||||
|
SourceText, CreatedAtText, PhaseText string
|
||||||
|
PercentText, SpeedText, ETAText string
|
||||||
|
FragmentText, ErrorKindText string
|
||||||
|
}
|
||||||
|
|
||||||
|
type queueView struct {
|
||||||
|
Running, Queued, Failed, Canceled, Succeeded []queueJobView
|
||||||
|
Poll bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) queuePage(w http.ResponseWriter, r *http.Request) {
|
||||||
|
view, err := s.loadQueue(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("待ち行列の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "待ち行列を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "layout", page{Title: "待ち行列", Queue: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) queueFragment(w http.ResponseWriter, r *http.Request) {
|
||||||
|
view, err := s.loadQueue(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("待ち行列の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "待ち行列を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "queue_list", view)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) loadQueue(ctx context.Context) (queueView, error) {
|
||||||
|
var view queueView
|
||||||
|
for _, item := range []struct {
|
||||||
|
status string
|
||||||
|
assign func([]queueJobView)
|
||||||
|
}{
|
||||||
|
{"running", func(j []queueJobView) { view.Running = j }},
|
||||||
|
{"queued", func(j []queueJobView) { view.Queued = j }},
|
||||||
|
{"failed", func(j []queueJobView) { view.Failed = j }},
|
||||||
|
{"canceled", func(j []queueJobView) { view.Canceled = j }},
|
||||||
|
{"succeeded", func(j []queueJobView) { view.Succeeded = j }},
|
||||||
|
} {
|
||||||
|
jobs, err := s.jobs.ListJobs(ctx, item.status, queueListLimit)
|
||||||
|
if err != nil {
|
||||||
|
return queueView{}, err
|
||||||
|
}
|
||||||
|
views := make([]queueJobView, 0, len(jobs))
|
||||||
|
for _, job := range jobs {
|
||||||
|
views = append(views, s.queueJob(job))
|
||||||
|
}
|
||||||
|
item.assign(views)
|
||||||
|
}
|
||||||
|
view.Poll = len(view.Running) > 0 || len(view.Queued) > 0
|
||||||
|
return view, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) queueJob(job store.Job) queueJobView {
|
||||||
|
view := queueJobView{Job: job, CreatedAtText: job.CreatedAt.In(s.zone).Format("2006-01-02 15:04 MST")}
|
||||||
|
if job.Source == "subscription" {
|
||||||
|
view.SourceText = "自動"
|
||||||
|
} else {
|
||||||
|
view.SourceText = "手動"
|
||||||
|
}
|
||||||
|
if job.Phase != nil {
|
||||||
|
switch *job.Phase {
|
||||||
|
case "video":
|
||||||
|
view.PhaseText = "映像"
|
||||||
|
case "audio":
|
||||||
|
view.PhaseText = "音声"
|
||||||
|
case "merging":
|
||||||
|
view.PhaseText = "結合"
|
||||||
|
case "moving":
|
||||||
|
view.PhaseText = "移動"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if job.DownloadedBytes != nil && job.TotalBytes != nil && *job.TotalBytes > 0 {
|
||||||
|
percent := 100 * float64(*job.DownloadedBytes) / float64(*job.TotalBytes)
|
||||||
|
percent = max(0, min(percent, 100))
|
||||||
|
if job.TotalIsEstimate {
|
||||||
|
view.PercentText = fmt.Sprintf("約%.0f%%", percent)
|
||||||
|
} else {
|
||||||
|
view.PercentText = fmt.Sprintf("%.0f%%", percent)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if job.SpeedBPS != nil && *job.SpeedBPS > 0 {
|
||||||
|
view.SpeedText = formatBytes(*job.SpeedBPS) + "/秒"
|
||||||
|
}
|
||||||
|
if job.ETASeconds != nil && *job.ETASeconds >= 0 {
|
||||||
|
seconds := int64(*job.ETASeconds)
|
||||||
|
view.ETAText = fmt.Sprintf("残り%d分%d秒", seconds/60, seconds%60)
|
||||||
|
}
|
||||||
|
if job.FragmentIndex != nil && job.FragmentCount != nil && *job.FragmentCount > 0 {
|
||||||
|
view.FragmentText = fmt.Sprintf("断片 %d/%d", *job.FragmentIndex, *job.FragmentCount)
|
||||||
|
}
|
||||||
|
if job.ErrorKind != nil {
|
||||||
|
view.ErrorKindText = errorKindText(*job.ErrorKind)
|
||||||
|
}
|
||||||
|
return view
|
||||||
|
}
|
||||||
|
|
||||||
|
func formatBytes(n float64) string {
|
||||||
|
for _, unit := range []string{"B", "KiB", "MiB", "GiB"} {
|
||||||
|
if n < 1024 || unit == "GiB" {
|
||||||
|
return fmt.Sprintf("%.1f %s", n, unit)
|
||||||
|
}
|
||||||
|
n /= 1024
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func errorKindText(kind string) string {
|
||||||
|
switch kind {
|
||||||
|
case "drm":
|
||||||
|
return "DRM のため取得不可"
|
||||||
|
case "geo":
|
||||||
|
return "地域制限"
|
||||||
|
case "network":
|
||||||
|
return "ネットワーク"
|
||||||
|
case "unavailable":
|
||||||
|
return "配信終了・存在しない"
|
||||||
|
case "extract":
|
||||||
|
return "抽出失敗"
|
||||||
|
case "invalid_output":
|
||||||
|
return "出力形式の不正"
|
||||||
|
default:
|
||||||
|
return "不明"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) retryJob(w http.ResponseWriter, r *http.Request) {
|
||||||
|
s.queueMutation(w, r, s.jobs.Retry)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) cancelJob(w http.ResponseWriter, r *http.Request) {
|
||||||
|
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
|
||||||
|
if err != nil || id <= 0 {
|
||||||
|
http.Error(w, "ジョブ ID が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
job, err := s.jobs.GetJob(r.Context(), id)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
http.Error(w, "ジョブが見つかりません", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("ジョブ状態の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "操作できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if job.Status != "queued" && job.Status != "running" {
|
||||||
|
http.Error(w, "待機中またはダウンロード中のジョブだけ取り消せます", http.StatusConflict)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.queueMutation(w, r, s.jobs.Cancel)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) queueMutation(w http.ResponseWriter, r *http.Request, change func(context.Context, int64) error) {
|
||||||
|
id, err := strconv.ParseInt(r.PathValue("id"), 10, 64)
|
||||||
|
if err != nil || id <= 0 {
|
||||||
|
http.Error(w, "ジョブ ID が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := change(r.Context(), id); err != nil {
|
||||||
|
if errors.Is(err, store.ErrInvalidTransition) {
|
||||||
|
http.Error(w, "操作できる状態ではありません", http.StatusConflict)
|
||||||
|
} else {
|
||||||
|
s.logger.Printf("待ち行列の操作に失敗: %v", err)
|
||||||
|
http.Error(w, "操作できませんでした", http.StatusInternalServerError)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.Header.Get("HX-Request") != "true" {
|
||||||
|
http.Redirect(w, r, "/queue", http.StatusSeeOther)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.queueFragment(w, r)
|
||||||
|
}
|
||||||
@@ -0,0 +1,350 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"regexp"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
func postForm(handler http.Handler, path string, values url.Values) *httptest.ResponseRecorder {
|
||||||
|
r := httptest.NewRequest(http.MethodPost, path, strings.NewReader(values.Encode()))
|
||||||
|
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||||
|
r.Header.Set("HX-Request", "true")
|
||||||
|
w := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(w, r)
|
||||||
|
return w
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnqueueUsesCachedEpisodeAndDuplicateStatus(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "series1", SeriesTitle: "正しい番組", Title: "正しい回", BroadcasterName: "正しい局", Duration: 1234, EndAt: 1791061200, ThumbnailPath: "/correct.jpg"}}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
getPage(handler, "/")
|
||||||
|
if jobs.statusCalls != 1 || len(jobs.statusIDs) != 1 || jobs.statusIDs[0] != "ep1" {
|
||||||
|
t.Fatalf("状態の一括取得: calls=%d ids=%v", jobs.statusCalls, jobs.statusIDs)
|
||||||
|
}
|
||||||
|
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}, "series_title": {"偽番組"}, "title": {"偽回"}}
|
||||||
|
response := postForm(handler, "/episodes/enqueue", values)
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "待機中") || !strings.Contains(response.Body.String(), `class="episode-action"`) || strings.Contains(response.Body.String(), "<html") || strings.Contains(response.Body.String(), "episode-card") {
|
||||||
|
t.Fatalf("登録応答: %d %s", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
if jobs.lastEpisode.SeriesTitle != "正しい番組" || jobs.lastEpisode.Title != "正しい回" || jobs.lastEpisode.BroadcasterName != "正しい局" || jobs.lastEpisode.DurationSeconds != 1234 || jobs.lastEpisode.ThumbnailPath != "/correct.jpg" || jobs.lastEpisode.EndAt == nil {
|
||||||
|
t.Fatalf("キャッシュ外の情報を使用: %+v", jobs.lastEpisode)
|
||||||
|
}
|
||||||
|
if len(fetcher.calls) != 1 {
|
||||||
|
t.Fatalf("POST が TVer に再問い合わせ: %v", fetcher.calls)
|
||||||
|
}
|
||||||
|
duplicate := postForm(handler, "/episodes/enqueue", values)
|
||||||
|
if duplicate.Code != http.StatusOK || !strings.Contains(duplicate.Body.String(), "待機中") || len(jobs.jobs) != 1 {
|
||||||
|
t.Fatalf("二重登録: %d %s jobs=%d", duplicate.Code, duplicate.Body.String(), len(jobs.jobs))
|
||||||
|
}
|
||||||
|
page := getPage(handler, "/")
|
||||||
|
if !strings.Contains(page.Body.String(), "待機中") || strings.Contains(page.Body.String(), ">ダウンロード</button>") {
|
||||||
|
t.Fatalf("登録後の一覧: %s", page.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnqueueRejectsInvalidOrExpiredCache(t *testing.T) {
|
||||||
|
now := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", Duration: 200}}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, func() time.Time { return now })
|
||||||
|
valid := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
|
||||||
|
if got := postForm(handler, "/episodes/enqueue", valid).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("未取得キャッシュ: %d", got)
|
||||||
|
}
|
||||||
|
getPage(handler, "/")
|
||||||
|
for _, values := range []url.Values{
|
||||||
|
{"genre": {"news"}, "episode_id": {"ep1"}},
|
||||||
|
{"genre": {"drama"}},
|
||||||
|
} {
|
||||||
|
if got := postForm(handler, "/episodes/enqueue", values).Code; got != http.StatusBadRequest {
|
||||||
|
t.Errorf("不正なフォーム: %d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"drama"}, "episode_id": {"unknown"}}).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("同じジャンルの未収録 ID: %d", got)
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"variety"}, "episode_id": {"ep1"}}).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("別ジャンルのキャッシュ: %d", got)
|
||||||
|
}
|
||||||
|
now = now.Add(newerCacheTTL)
|
||||||
|
if got := postForm(handler, "/episodes/enqueue", valid).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("期限切れキャッシュ: %d", got)
|
||||||
|
}
|
||||||
|
if jobs.enqueueCalls != 0 || len(fetcher.calls) != 1 {
|
||||||
|
t.Fatalf("無効な入力で登録・再取得: enqueue=%d fetch=%v", jobs.enqueueCalls, fetcher.calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnqueueUsesSelectedGenreCacheForSameEpisodeID(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodesByGenre: map[string][]tver.Episode{
|
||||||
|
"drama": {{ID: "shared", SeriesTitle: "ドラマの番組", Duration: 200}},
|
||||||
|
"variety": {{ID: "shared", SeriesTitle: "バラエティの番組", Duration: 200}},
|
||||||
|
}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
getPage(handler, "/")
|
||||||
|
getPage(handler, "/?genre=variety")
|
||||||
|
response := postForm(handler, "/episodes/enqueue", url.Values{
|
||||||
|
"genre": {"variety"}, "episode_id": {"shared"}, "series_title": {"偽タイトル"},
|
||||||
|
})
|
||||||
|
if response.Code != http.StatusOK || jobs.lastEpisode.SeriesTitle != "バラエティの番組" || jobs.enqueueCalls != 1 || !slices.Equal(fetcher.calls, []string{"drama", "variety"}) {
|
||||||
|
t.Fatalf("選択ジャンルのキャッシュ: status=%d episode=%+v calls=%v", response.Code, jobs.lastEpisode, fetcher.calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerStatusesAndRetry(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "failed", Duration: 200}, {ID: "canceled", Duration: 200}, {ID: "running", Duration: 200}, {ID: "done", Duration: 200}}}
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{
|
||||||
|
"failed": {ID: 1, Status: "failed"}, "canceled": {ID: 2, Status: "canceled"},
|
||||||
|
"running": {ID: 3, Status: "running"}, "done": {ID: 4, Status: "succeeded"},
|
||||||
|
}}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
body := getPage(handler, "/").Body.String()
|
||||||
|
for _, want := range []string{"失敗", "取り消し", "ダウンロード中", "完了"} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("状態表示がない: %s", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if got := strings.Count(body, ">再試行</button>"); got != 2 {
|
||||||
|
t.Fatalf("再試行ボタン数: %d", got)
|
||||||
|
}
|
||||||
|
response := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"failed"}})
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "待機中") || jobs.jobs["failed"].Status != "queued" {
|
||||||
|
t.Fatalf("新着から再試行: %d %s", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"running"}}).Code; got != http.StatusConflict {
|
||||||
|
t.Fatalf("実行中の再試行: %d", got)
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/episodes/retry", url.Values{"genre": {"drama"}, "episode_id": {"missing"}}).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("キャッシュにない ID の再試行: %d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueueGroupsProgressAndPolling(t *testing.T) {
|
||||||
|
phase, errorKind, errorMessage := "video", "drm", `<script>alert(1)</script>`
|
||||||
|
downloaded, total := int64(25), int64(100)
|
||||||
|
speed, eta := float64(2048), float64(63)
|
||||||
|
index, count := int32(2), int32(10)
|
||||||
|
created := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{
|
||||||
|
"running": {ID: 1, Status: "running", Source: "subscription", SeriesTitle: "番組A", Title: "第1話", BroadcasterName: "局A", CreatedAt: created, Phase: &phase, DownloadedBytes: &downloaded, TotalBytes: &total, TotalIsEstimate: true, SpeedBPS: &speed, ETASeconds: &eta, FragmentIndex: &index, FragmentCount: &count},
|
||||||
|
"queued": {ID: 2, Status: "queued", Source: "manual", SeriesTitle: "番組B", CreatedAt: created},
|
||||||
|
"failed": {ID: 3, Status: "failed", ErrorKind: &errorKind, ErrorMessage: &errorMessage, CreatedAt: created},
|
||||||
|
"canceled": {ID: 4, Status: "canceled", CreatedAt: created},
|
||||||
|
"done": {ID: 5, Status: "succeeded", CreatedAt: created},
|
||||||
|
}}
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
|
||||||
|
response := getPage(handler, "/queue")
|
||||||
|
if response.Code != http.StatusOK {
|
||||||
|
t.Fatalf("待ち行列: %d", response.Code)
|
||||||
|
}
|
||||||
|
body := response.Body.String()
|
||||||
|
for _, want := range []string{"ダウンロード中", "待機中", "失敗", "取り消し", "完了（直近50件）", "番組A", "第1話", "局A", "自動", "手動", "2026-10-03 09:00 JST", "映像", "約25%", "2.0 KiB/秒", "残り1分3秒", "断片 2/10", "DRM のため取得不可", "<details>", "<script>", `hx-get="/queue/list"`, `hx-trigger="every 5s"`} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("待ち行列に %q がない", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(body, errorMessage) || !strings.Contains(body, `action="/jobs/1/cancel"`) || strings.Contains(body, "全体") {
|
||||||
|
t.Errorf("不正な詳細・操作・全体進捗: %s", body)
|
||||||
|
}
|
||||||
|
fragment := getPage(handler, "/queue/list")
|
||||||
|
if fragment.Code != http.StatusOK || strings.Contains(fragment.Body.String(), "<html") || !strings.Contains(fragment.Body.String(), `id="queue-list"`) {
|
||||||
|
t.Errorf("待ち行列断片: %d %s", fragment.Code, fragment.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueueJobsAppearInCorrectSectionsWithLimit(t *testing.T) {
|
||||||
|
statuses := []string{"running", "queued", "failed", "canceled", "succeeded"}
|
||||||
|
labels := []string{"ダウンロード中", "待機中", "失敗", "取り消し", "完了"}
|
||||||
|
jobs := &fakeJobStore{jobs: make(map[string]store.Job)}
|
||||||
|
for i, status := range statuses {
|
||||||
|
jobs.jobs[status] = store.Job{ID: int64(i + 1), Status: status, SeriesTitle: "番組-" + status}
|
||||||
|
}
|
||||||
|
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, jobs, nil), "/queue")
|
||||||
|
if response.Code != http.StatusOK {
|
||||||
|
t.Fatalf("待ち行列: %d", response.Code)
|
||||||
|
}
|
||||||
|
if !slices.Equal(jobs.listStatuses, statuses) {
|
||||||
|
t.Errorf("状態別取得: %v", jobs.listStatuses)
|
||||||
|
}
|
||||||
|
if !slices.Equal(jobs.listLimits, []int{50, 50, 50, 50, 50}) {
|
||||||
|
t.Errorf("状態ごとの取得上限: %v", jobs.listLimits)
|
||||||
|
}
|
||||||
|
for i, label := range labels {
|
||||||
|
section := regexp.MustCompile(`(?s)<section aria-label="` + label + `">(.*?)</section>`).FindStringSubmatch(response.Body.String())
|
||||||
|
if len(section) != 2 || !strings.Contains(section[1], "番組-"+statuses[i]) {
|
||||||
|
t.Errorf("%s に %s がない: %v", label, statuses[i], section)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for j, other := range statuses {
|
||||||
|
if j != i && strings.Contains(section[1], "番組-"+other) {
|
||||||
|
t.Errorf("%s に %s が混入", label, other)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cancelAction := `action="/jobs/` + string(rune('1'+i)) + `/cancel"`
|
||||||
|
retryAction := `action="/jobs/` + string(rune('1'+i)) + `/retry"`
|
||||||
|
switch statuses[i] {
|
||||||
|
case "running", "queued":
|
||||||
|
if !strings.Contains(section[1], cancelAction) || strings.Contains(section[1], retryAction) {
|
||||||
|
t.Errorf("%s の操作が不正: %s", statuses[i], section[1])
|
||||||
|
}
|
||||||
|
case "failed", "canceled":
|
||||||
|
if !strings.Contains(section[1], retryAction) || strings.Contains(section[1], cancelAction) {
|
||||||
|
t.Errorf("%s の操作が不正: %s", statuses[i], section[1])
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
if strings.Contains(section[1], "/cancel") || strings.Contains(section[1], "/retry") {
|
||||||
|
t.Errorf("%s に操作がある: %s", statuses[i], section[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueueEscapesEpisodeFields(t *testing.T) {
|
||||||
|
attack := `<script>alert(1)</script>`
|
||||||
|
message := `<img src=x onerror=alert(1)>`
|
||||||
|
kind := "extract"
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{
|
||||||
|
"bad": {ID: 1, Status: "failed", SeriesTitle: attack, Title: attack,
|
||||||
|
BroadcasterName: attack, ErrorKind: &kind, ErrorMessage: &message},
|
||||||
|
}}
|
||||||
|
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, jobs, nil), "/queue")
|
||||||
|
body := response.Body.String()
|
||||||
|
if response.Code != http.StatusOK || strings.Contains(body, attack) || strings.Contains(body, message) || strings.Count(body, "<script>") != 3 || !strings.Contains(body, "<img src=x onerror=alert(1)>") {
|
||||||
|
t.Fatalf("待ち行列の HTML エスケープ: %d %s", response.Code, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueueCancelRetryAndStopPolling(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep": {ID: 1, Status: "queued"}}}
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
|
||||||
|
if !strings.Contains(getPage(handler, "/queue").Body.String(), `hx-get="/queue/list"`) {
|
||||||
|
t.Fatal("待機中なのに更新がない")
|
||||||
|
}
|
||||||
|
canceled := postForm(handler, "/jobs/1/cancel", nil)
|
||||||
|
if canceled.Code != http.StatusOK || !strings.Contains(canceled.Body.String(), `action="/jobs/1/retry"`) || strings.Contains(canceled.Body.String(), `hx-get="/queue/list"`) {
|
||||||
|
t.Fatalf("取り消し後: %d %s", canceled.Code, canceled.Body.String())
|
||||||
|
}
|
||||||
|
retried := postForm(handler, "/jobs/1/retry", nil)
|
||||||
|
if retried.Code != http.StatusOK || !strings.Contains(retried.Body.String(), `action="/jobs/1/cancel"`) || !strings.Contains(retried.Body.String(), `hx-get="/queue/list"`) {
|
||||||
|
t.Fatalf("再試行後: %d %s", retried.Code, retried.Body.String())
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/jobs/1/retry", nil).Code; got != http.StatusConflict {
|
||||||
|
t.Fatalf("待機中の再試行: %d", got)
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/jobs/bad/cancel", nil).Code; got != http.StatusBadRequest {
|
||||||
|
t.Fatalf("不正な ID: %d", got)
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/jobs/99/cancel", nil).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("存在しないジョブ: %d", got)
|
||||||
|
}
|
||||||
|
jobs.jobs["ep"] = store.Job{ID: 1, Status: "running"}
|
||||||
|
runningCanceled := postForm(handler, "/jobs/1/cancel", nil)
|
||||||
|
if runningCanceled.Code != http.StatusOK || jobs.jobs["ep"].Status != "canceled" || !strings.Contains(runningCanceled.Body.String(), `action="/jobs/1/retry"`) {
|
||||||
|
t.Fatalf("実行中の取り消し: %d %+v", runningCanceled.Code, jobs.jobs["ep"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStateChangingGETIsRejected(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{jobs: map[string]store.Job{"ep": {ID: 1, Status: "queued"}}}
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
|
||||||
|
getPage(handler, "/")
|
||||||
|
for _, path := range []string{
|
||||||
|
"/episodes/enqueue?episode_id=ep&genre=drama",
|
||||||
|
"/episodes/retry?episode_id=ep&genre=drama",
|
||||||
|
"/jobs/1/cancel", "/jobs/1/retry",
|
||||||
|
} {
|
||||||
|
response := getPage(handler, path)
|
||||||
|
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
|
||||||
|
t.Errorf("GET %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if jobs.jobs["ep"].Status != "queued" || jobs.enqueueCalls != 0 {
|
||||||
|
t.Fatalf("GET が状態を変更した: jobs=%+v enqueue=%d", jobs.jobs, jobs.enqueueCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestStateChangingPOSTWithoutHtmxRedirects(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
|
||||||
|
getPage(handler, "/")
|
||||||
|
form := url.Values{"episode_id": {"ep"}, "genre": {"drama"}}
|
||||||
|
request := httptest.NewRequest(http.MethodPost, "/episodes/enqueue", strings.NewReader(form.Encode()))
|
||||||
|
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, request)
|
||||||
|
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/?genre=drama" || jobs.jobs["ep"].Status != "queued" {
|
||||||
|
t.Fatalf("通常 POST 後の遷移: status=%d Location=%q jobs=%+v", response.Code, response.Header().Get("Location"), jobs.jobs)
|
||||||
|
}
|
||||||
|
request = httptest.NewRequest(http.MethodPost, "/jobs/1/cancel", nil)
|
||||||
|
response = httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, request)
|
||||||
|
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/queue" || jobs.jobs["ep"].Status != "canceled" {
|
||||||
|
t.Fatalf("待ち行列操作後の遷移: status=%d Location=%q jobs=%+v", response.Code, response.Header().Get("Location"), jobs.jobs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerStoreFailureDoesNotOfferDownload(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{err: errors.New("db unavailable")}
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
|
||||||
|
response := getPage(handler, "/")
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "状態を取得できませんでした") || strings.Contains(response.Body.String(), ">ダウンロード</button>") {
|
||||||
|
t.Errorf("状態取得失敗: %d %s", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnqueueStoreFailureShowsShortError(t *testing.T) {
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{episodes: []tver.Episode{{ID: "ep", Duration: 200}}}, jobs, nil)
|
||||||
|
getPage(handler, "/")
|
||||||
|
jobs.err = errors.New("secret database detail")
|
||||||
|
response := postForm(handler, "/episodes/enqueue", url.Values{"genre": {"drama"}, "episode_id": {"ep"}})
|
||||||
|
if response.Code != http.StatusInternalServerError || !strings.Contains(response.Body.String(), "待ち行列に登録できませんでした") || strings.Contains(response.Body.String(), "secret") {
|
||||||
|
t.Errorf("DB エラーの表示: %d %s", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQueueProgressAndFailureLabels(t *testing.T) {
|
||||||
|
s := &server{zone: time.FixedZone("JST", 9*60*60)}
|
||||||
|
for _, tt := range []struct{ kind, label string }{
|
||||||
|
{"drm", "DRM のため取得不可"}, {"geo", "地域制限"},
|
||||||
|
{"network", "ネットワーク"}, {"unavailable", "配信終了・存在しない"},
|
||||||
|
{"extract", "抽出失敗"}, {"invalid_output", "出力形式の不正"},
|
||||||
|
{"unknown", "不明"},
|
||||||
|
} {
|
||||||
|
if got := errorKindText(tt.kind); got != tt.label {
|
||||||
|
t.Errorf("%s: %q", tt.kind, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, tt := range []struct{ phase, label string }{
|
||||||
|
{"video", "映像"}, {"audio", "音声"}, {"merging", "結合"}, {"moving", "移動"},
|
||||||
|
} {
|
||||||
|
if got := s.queueJob(store.Job{Phase: &tt.phase}).PhaseText; got != tt.label {
|
||||||
|
t.Errorf("%s: %q", tt.phase, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
downloaded, total := int64(50), int64(100)
|
||||||
|
job := store.Job{DownloadedBytes: &downloaded, TotalBytes: &total}
|
||||||
|
if got := s.queueJob(job).PercentText; got != "50%" {
|
||||||
|
t.Errorf("確定した段階の割合: %q", got)
|
||||||
|
}
|
||||||
|
job.TotalIsEstimate = true
|
||||||
|
if got := s.queueJob(job).PercentText; got != "約50%" {
|
||||||
|
t.Errorf("推定した段階の割合: %q", got)
|
||||||
|
}
|
||||||
|
total = 0
|
||||||
|
if got := s.queueJob(job).PercentText; got != "" {
|
||||||
|
t.Errorf("総量ゼロでの割合: %q", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
body { font-family: system-ui, sans-serif; margin: 0; color: #17212b; }
|
||||||
|
header { display: flex; align-items: center; gap: 2rem; padding: 1rem 2rem; border-bottom: 1px solid #d6dee4; }
|
||||||
|
nav { display: flex; flex-wrap: wrap; gap: 1.5rem; }
|
||||||
|
nav a { color: #145a86; }
|
||||||
|
main { max-width: 72rem; margin: 2rem auto; padding: 0 2rem; }
|
||||||
|
#newer-controls { display: flex; align-items: center; flex-wrap: wrap; gap: 1rem; margin-bottom: 2rem; }
|
||||||
|
#newer-controls select, #newer-controls button { font: inherit; padding: .35rem .6rem; }
|
||||||
|
.episode-grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(17rem, 1fr)); gap: 1.25rem; }
|
||||||
|
.episode-card { border: 1px solid #d6dee4; border-radius: .5rem; overflow: hidden; }
|
||||||
|
.library-selection { display: flex; align-items: center; gap: .4rem; padding: .6rem .8rem; }
|
||||||
|
.library-select-grid .episode-card { display: flex; flex-direction: column; }
|
||||||
|
.library-select-grid .episode-card > a { flex: 1; }
|
||||||
|
.episode-card a { color: inherit; text-decoration: none; }
|
||||||
|
.episode-card a:hover h2 { text-decoration: underline; }
|
||||||
|
.episode-card img { display: block; width: 100%; aspect-ratio: 16 / 9; object-fit: cover; background: #e5ebef; }
|
||||||
|
.library-card-image { position: relative; }
|
||||||
|
.episode-count-badge, .subscription-badge { display: inline-block; border-radius: 999px; padding: .15rem .5rem; font-size: .8rem; line-height: 1.2; }
|
||||||
|
.episode-count-badge { position: absolute; right: .5rem; bottom: .5rem; color: #fff; background: rgba(23, 33, 43, .85); }
|
||||||
|
.subscription-badge { color: #145a86; background: #e1f1fb; vertical-align: middle; }
|
||||||
|
.episode-copy { padding: .8rem 1rem; }
|
||||||
|
.episode-copy h2 { font-size: 1.1rem; margin: 0 0 .4rem; }
|
||||||
|
.episode-copy p { margin: .3rem 0; }
|
||||||
|
.episode-action { padding: 0 .8rem .8rem; }
|
||||||
|
.subscription-action { padding: 0 .8rem .8rem; }
|
||||||
|
.subscription-action button, .subscription-item button { font: inherit; padding: .35rem .7rem; }
|
||||||
|
.subscription-item { border: 1px solid #d6dee4; border-radius: .5rem; margin: .75rem 0; padding: .8rem 1rem; }
|
||||||
|
.subscription-item form { display: inline-block; margin-right: .5rem; }
|
||||||
|
.subscription-url-entry { border: 1px solid #d6dee4; border-radius: .5rem; margin-bottom: 1.5rem; padding: .8rem 1rem; }
|
||||||
|
.subscription-url-entry input[type="url"] { font: inherit; min-width: min(32rem, 70vw); padding: .35rem .5rem; }
|
||||||
|
.subscription-url-entry button { font: inherit; padding: .35rem .7rem; }
|
||||||
|
.episode-action form { display: inline; }
|
||||||
|
.episode-action button, .queue-job button { font: inherit; padding: .35rem .7rem; }
|
||||||
|
.queue-job { border: 1px solid #d6dee4; border-radius: .5rem; margin: .75rem 0; padding: .8rem 1rem; }
|
||||||
|
.queue-job h3 { margin: 0 0 .4rem; }
|
||||||
|
.queue-job p { margin: .4rem 0; }
|
||||||
|
.queue-job pre { white-space: pre-wrap; overflow-wrap: anywhere; }
|
||||||
|
video { display: block; width: 100%; max-height: 75vh; background: #000; }
|
||||||
|
.breadcrumbs { display: flex; flex-wrap: wrap; gap: .45rem; margin-bottom: 1rem; overflow-wrap: anywhere; }
|
||||||
|
.library-sort { margin: 1rem 0; }
|
||||||
|
.library-episode-list { border-top: 1px solid #d6dee4; }
|
||||||
|
.library-episode-select-row { display: grid; grid-template-columns: auto minmax(0, 1fr); align-items: center; border-bottom: 1px solid #d6dee4; }
|
||||||
|
.library-episode-row { display: grid; grid-template-columns: 8rem minmax(0, 1fr) auto auto auto; gap: .8rem; align-items: center; padding: .7rem 0; border-bottom: 1px solid #d6dee4; color: inherit; text-decoration: none; }
|
||||||
|
.library-episode-select-row .library-episode-row { border-bottom: 0; }
|
||||||
|
.library-episode-row:hover .library-episode-title { text-decoration: underline; }
|
||||||
|
.library-episode-row img { display: block; width: 8rem; aspect-ratio: 16 / 9; object-fit: cover; background: #e5ebef; }
|
||||||
|
.library-episode-title { min-width: 0; overflow-wrap: anywhere; font-weight: 600; }
|
||||||
|
.episode-navigation { display: flex; justify-content: space-between; gap: 1rem; margin: 1rem 0; }
|
||||||
|
.episode-navigation a, .episode-navigation span { min-width: 0; overflow-wrap: anywhere; }
|
||||||
|
.delete-warning { border-left: .3rem solid #b42318; background: #fff1f0; padding: .8rem 1rem; }
|
||||||
|
.delete-group { overflow-wrap: anywhere; }
|
||||||
|
|
||||||
|
@media (max-width: 36rem) {
|
||||||
|
header { align-items: flex-start; flex-direction: column; gap: .75rem; padding: 1rem; }
|
||||||
|
main { margin: 1.25rem auto; padding: 0 1rem; }
|
||||||
|
.episode-grid { grid-template-columns: minmax(0, 1fr); }
|
||||||
|
.library-episode-row { grid-template-columns: 6rem minmax(0, 1fr); gap: .35rem .7rem; }
|
||||||
|
.library-episode-select-row { grid-template-columns: 1fr; }
|
||||||
|
.library-episode-select-row > .library-selection { padding: .7rem 0 0; }
|
||||||
|
.library-episode-row img { grid-row: 1 / span 4; width: 6rem; }
|
||||||
|
.library-episode-row > span:not(.library-episode-title) { grid-column: 2; font-size: .9rem; }
|
||||||
|
.episode-navigation { flex-direction: column; }
|
||||||
|
}
|
||||||
+1
File diff suppressed because one or more lines are too long
@@ -0,0 +1,13 @@
|
|||||||
|
Zero-Clause BSD
|
||||||
|
=============
|
||||||
|
|
||||||
|
Permission to use, copy, modify, and/or distribute this software for
|
||||||
|
any purpose with or without fee is hereby granted.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED “AS IS” AND THE AUTHOR DISCLAIMS ALL
|
||||||
|
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES
|
||||||
|
OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE
|
||||||
|
FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY
|
||||||
|
DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN
|
||||||
|
AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
|
||||||
|
OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
|
||||||
@@ -0,0 +1,361 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"sort"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeJobStore struct {
|
||||||
|
jobs map[string]store.Job
|
||||||
|
subscriptions map[string]store.Subscription
|
||||||
|
nextID int64
|
||||||
|
lastEpisode store.Episode
|
||||||
|
enqueueCalls, statusCalls int
|
||||||
|
statusIDs []string
|
||||||
|
listStatuses []string
|
||||||
|
listLimits []int
|
||||||
|
registerCalls int
|
||||||
|
registerEpisodes []store.Episode
|
||||||
|
registerDownload bool
|
||||||
|
registerErr error
|
||||||
|
subscribedCalls int
|
||||||
|
subscribedIDs []string
|
||||||
|
beforeDeleteSucceeded func()
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ensure() {
|
||||||
|
if f.jobs == nil {
|
||||||
|
f.jobs = make(map[string]store.Job)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) Enqueue(_ context.Context, ep store.Episode, source string) (store.Job, bool, error) {
|
||||||
|
f.enqueueCalls++
|
||||||
|
f.lastEpisode = ep
|
||||||
|
if f.err != nil {
|
||||||
|
return store.Job{}, false, f.err
|
||||||
|
}
|
||||||
|
f.ensure()
|
||||||
|
if job, ok := f.jobs[ep.ID]; ok {
|
||||||
|
return job, false, nil
|
||||||
|
}
|
||||||
|
f.nextID++
|
||||||
|
job := store.Job{ID: f.nextID, EpisodeID: ep.ID, SeriesTitle: ep.SeriesTitle,
|
||||||
|
Title: ep.Title, BroadcasterName: ep.BroadcasterName, Source: source,
|
||||||
|
Status: "queued", CreatedAt: time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)}
|
||||||
|
f.jobs[ep.ID] = job
|
||||||
|
return job, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) EpisodeStatuses(_ context.Context, ids []string) (map[string]string, error) {
|
||||||
|
f.statusCalls++
|
||||||
|
f.statusIDs = append([]string(nil), ids...)
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
result := make(map[string]string)
|
||||||
|
for _, id := range ids {
|
||||||
|
if job, ok := f.jobs[id]; ok {
|
||||||
|
result[id] = job.Status
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ListJobs(_ context.Context, status string, limit int) ([]store.Job, error) {
|
||||||
|
f.listStatuses = append(f.listStatuses, status)
|
||||||
|
f.listLimits = append(f.listLimits, limit)
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
var result []store.Job
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if job.Status == status {
|
||||||
|
result = append(result, job)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(result, func(i, j int) bool { return result[i].ID > result[j].ID })
|
||||||
|
if len(result) > limit {
|
||||||
|
result = result[:limit]
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) GetJob(_ context.Context, id int64) (store.Job, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return store.Job{}, f.err
|
||||||
|
}
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if job.ID == id {
|
||||||
|
return job, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.Job{}, pgx.ErrNoRows
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) Retry(_ context.Context, id int64) error {
|
||||||
|
for ep, job := range f.jobs {
|
||||||
|
if job.ID == id && (job.Status == "failed" || job.Status == "canceled") {
|
||||||
|
job.Status = "queued"
|
||||||
|
f.jobs[ep] = job
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) RetryEpisode(_ context.Context, id string) error {
|
||||||
|
job, ok := f.jobs[id]
|
||||||
|
if !ok || (job.Status != "failed" && job.Status != "canceled") {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
job.Status = "queued"
|
||||||
|
f.jobs[id] = job
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) Cancel(_ context.Context, id int64) error {
|
||||||
|
if f.err != nil {
|
||||||
|
return f.err
|
||||||
|
}
|
||||||
|
for ep, job := range f.jobs {
|
||||||
|
if job.ID == id && (job.Status == "queued" || job.Status == "running") {
|
||||||
|
job.Status = "canceled"
|
||||||
|
f.jobs[ep] = job
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) CancelQueued(ctx context.Context, id int64) error {
|
||||||
|
return f.Cancel(ctx, id)
|
||||||
|
}
|
||||||
|
|
||||||
|
var _ JobStore = (*fakeJobStore)(nil)
|
||||||
|
|
||||||
|
func (f *fakeJobStore) SubscribedSeries(_ context.Context, ids []string) (map[string]bool, error) {
|
||||||
|
f.subscribedCalls++
|
||||||
|
f.subscribedIDs = append([]string(nil), ids...)
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
result := make(map[string]bool)
|
||||||
|
for _, id := range ids {
|
||||||
|
_, result[id] = f.subscriptions[id]
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) RegisterSubscription(_ context.Context, seriesID, title string, episodes []store.Episode, download bool) (bool, error) {
|
||||||
|
f.registerCalls++
|
||||||
|
f.registerEpisodes = append([]store.Episode(nil), episodes...)
|
||||||
|
f.registerDownload = download
|
||||||
|
if f.registerErr != nil {
|
||||||
|
return false, f.registerErr
|
||||||
|
}
|
||||||
|
if f.err != nil {
|
||||||
|
return false, f.err
|
||||||
|
}
|
||||||
|
if f.subscriptions == nil {
|
||||||
|
f.subscriptions = make(map[string]store.Subscription)
|
||||||
|
}
|
||||||
|
if _, exists := f.subscriptions[seriesID]; exists {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
var count int64
|
||||||
|
if download {
|
||||||
|
count = int64(len(episodes))
|
||||||
|
}
|
||||||
|
checked := time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC)
|
||||||
|
f.subscriptions[seriesID] = store.Subscription{SeriesID: seriesID, SeriesTitle: title,
|
||||||
|
Enabled: true, CreatedAt: checked, LastCheckedAt: &checked, JobCount: count}
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ListSubscriptions(_ context.Context) ([]store.Subscription, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
var result []store.Subscription
|
||||||
|
for _, sub := range f.subscriptions {
|
||||||
|
result = append(result, sub)
|
||||||
|
}
|
||||||
|
sort.Slice(result, func(i, j int) bool { return result[i].SeriesID < result[j].SeriesID })
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) SetSubscriptionEnabled(_ context.Context, seriesID string, enabled bool) error {
|
||||||
|
if f.err != nil {
|
||||||
|
return f.err
|
||||||
|
}
|
||||||
|
sub, exists := f.subscriptions[seriesID]
|
||||||
|
if !exists {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
sub.Enabled = enabled
|
||||||
|
f.subscriptions[seriesID] = sub
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) DeleteSubscription(_ context.Context, seriesID string) (bool, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return false, f.err
|
||||||
|
}
|
||||||
|
if _, exists := f.subscriptions[seriesID]; !exists {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
delete(f.subscriptions, seriesID)
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ListSucceededJobs(_ context.Context, limit int) ([]store.Job, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
var result []store.Job
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if job.Status == "succeeded" {
|
||||||
|
result = append(result, job)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(result, func(i, j int) bool { return result[i].ID > result[j].ID })
|
||||||
|
if len(result) > limit {
|
||||||
|
result = result[:limit]
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ListLibrarySeries(_ context.Context) ([]store.LibrarySeries, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
bySeries := make(map[string]*store.LibrarySeries)
|
||||||
|
latestIDs := make(map[string]int64)
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if job.Status != "succeeded" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
item := bySeries[job.SeriesID]
|
||||||
|
if item == nil {
|
||||||
|
item = &store.LibrarySeries{SeriesID: job.SeriesID}
|
||||||
|
bySeries[job.SeriesID] = item
|
||||||
|
}
|
||||||
|
item.EpisodeCount++
|
||||||
|
if job.FileSizeBytes != nil {
|
||||||
|
item.TotalSizeBytes += *job.FileSizeBytes
|
||||||
|
}
|
||||||
|
current := store.Job{ID: latestIDs[job.SeriesID], FinishedAt: item.LatestFinishedAt}
|
||||||
|
if item.SeriesTitle == "" || jobFinishedLater(job, current) {
|
||||||
|
item.SeriesTitle = job.SeriesTitle
|
||||||
|
item.LatestTitle = job.Title
|
||||||
|
item.BroadcasterName = job.BroadcasterName
|
||||||
|
item.ThumbnailPath = job.ThumbnailPath
|
||||||
|
if job.FinishedAt != nil {
|
||||||
|
finished := *job.FinishedAt
|
||||||
|
item.LatestFinishedAt = &finished
|
||||||
|
}
|
||||||
|
latestIDs[job.SeriesID] = job.ID
|
||||||
|
}
|
||||||
|
_, item.Subscribed = f.subscriptions[job.SeriesID]
|
||||||
|
}
|
||||||
|
result := make([]store.LibrarySeries, 0, len(bySeries))
|
||||||
|
for _, item := range bySeries {
|
||||||
|
result = append(result, *item)
|
||||||
|
}
|
||||||
|
sort.Slice(result, func(i, j int) bool {
|
||||||
|
left, right := result[i].LatestFinishedAt, result[j].LatestFinishedAt
|
||||||
|
if left == nil || right == nil {
|
||||||
|
if left != right {
|
||||||
|
return left != nil
|
||||||
|
}
|
||||||
|
} else if !left.Equal(*right) {
|
||||||
|
return left.After(*right)
|
||||||
|
}
|
||||||
|
return result[i].SeriesID < result[j].SeriesID
|
||||||
|
})
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ListSucceededJobsBySeries(_ context.Context, seriesID string) ([]store.Job, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
var result []store.Job
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if job.Status == "succeeded" && job.SeriesID == seriesID {
|
||||||
|
result = append(result, job)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sortJobsLatest(result)
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ListSucceededJobsByIDs(_ context.Context, ids []int64) ([]store.Job, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
wanted := make(map[int64]struct{}, len(ids))
|
||||||
|
for _, id := range ids {
|
||||||
|
wanted[id] = struct{}{}
|
||||||
|
}
|
||||||
|
var result []store.Job
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if _, ok := wanted[job.ID]; ok && job.Status == "succeeded" {
|
||||||
|
result = append(result, job)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sortJobsLatest(result)
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) ListSucceededJobsBySeriesIDs(_ context.Context, seriesIDs []string) ([]store.Job, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return nil, f.err
|
||||||
|
}
|
||||||
|
wanted := make(map[string]struct{}, len(seriesIDs))
|
||||||
|
for _, id := range seriesIDs {
|
||||||
|
wanted[id] = struct{}{}
|
||||||
|
}
|
||||||
|
var result []store.Job
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if _, ok := wanted[job.SeriesID]; ok && job.Status == "succeeded" {
|
||||||
|
result = append(result, job)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sortJobsLatest(result)
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) GetSucceededJob(_ context.Context, id int64) (store.Job, error) {
|
||||||
|
if f.err != nil {
|
||||||
|
return store.Job{}, f.err
|
||||||
|
}
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if job.ID == id && job.Status == "succeeded" {
|
||||||
|
return job, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.Job{}, pgx.ErrNoRows
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeJobStore) DeleteSucceededJob(_ context.Context, id int64) error {
|
||||||
|
if f.beforeDeleteSucceeded != nil {
|
||||||
|
f.beforeDeleteSucceeded()
|
||||||
|
}
|
||||||
|
if f.err != nil {
|
||||||
|
return f.err
|
||||||
|
}
|
||||||
|
for episodeID, job := range f.jobs {
|
||||||
|
if job.ID == id && job.Status == "succeeded" {
|
||||||
|
delete(f.jobs, episodeID)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
@@ -0,0 +1,161 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"regexp"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
const seriesURLInputError = "番組ページ（https://tver.jp/series/…）の URL を入力してください"
|
||||||
|
|
||||||
|
var seriesURLPathPattern = regexp.MustCompile(`^/series/([A-Za-z0-9]+)/?$`)
|
||||||
|
var seriesIDPattern = regexp.MustCompile(`^[A-Za-z0-9]+$`)
|
||||||
|
|
||||||
|
type subscriptionURLResultView struct {
|
||||||
|
SeriesID, Title, ErrorText string
|
||||||
|
EpisodeCount int
|
||||||
|
Subscribed bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type subscriptionURLRegisteredView struct {
|
||||||
|
Subscriptions subscriptionsView
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseSeriesURL(raw string) (string, error) {
|
||||||
|
parsed, err := url.Parse(strings.TrimSpace(raw))
|
||||||
|
if err != nil || parsed.Scheme != "https" || parsed.User != nil || (!strings.EqualFold(parsed.Host, "tver.jp") && !strings.EqualFold(parsed.Host, "www.tver.jp")) {
|
||||||
|
return "", errors.New(seriesURLInputError)
|
||||||
|
}
|
||||||
|
match := seriesURLPathPattern.FindStringSubmatch(parsed.Path)
|
||||||
|
if len(match) != 2 {
|
||||||
|
return "", errors.New(seriesURLInputError)
|
||||||
|
}
|
||||||
|
return match[1], nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) confirmSubscriptionURL(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Header.Get("HX-Request") != "true" {
|
||||||
|
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, "入力が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
seriesID, err := parseSeriesURL(r.Form.Get("series_url"))
|
||||||
|
if err != nil {
|
||||||
|
s.render(w, "subscription_url_result", subscriptionURLResultView{ErrorText: seriesURLInputError})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
title, episodes, ok := s.fetchSubscriptionSeries(w, r, seriesID)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
subscribed, err := s.isSubscribed(r.Context(), seriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
|
||||||
|
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "subscription_url_result", subscriptionURLResultView{
|
||||||
|
SeriesID: seriesID, Title: title, EpisodeCount: len(episodes), Subscribed: subscribed,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) registerSubscriptionURL(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if err := r.ParseForm(); err != nil {
|
||||||
|
http.Error(w, "入力が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
seriesID := r.Form.Get("series_id")
|
||||||
|
if !seriesIDPattern.MatchString(seriesID) {
|
||||||
|
http.Error(w, "番組 ID が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
choice := r.Form.Get("download_existing")
|
||||||
|
if choice != "" && choice != "1" {
|
||||||
|
http.Error(w, "公開中の回の選択が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
subscribed, err := s.isSubscribed(r.Context(), seriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
|
||||||
|
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if subscribed {
|
||||||
|
s.subscriptionURLRegisteredResponse(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
title, episodes, ok := s.fetchSubscriptionSeries(w, r, seriesID)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
items := make([]store.Episode, 0, len(episodes))
|
||||||
|
for _, episode := range episodes {
|
||||||
|
items = append(items, storeEpisode(episode))
|
||||||
|
}
|
||||||
|
_, err = s.jobs.RegisterSubscription(r.Context(), seriesID, title, items, choice == "1")
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組URLからの登録に失敗: %v", err)
|
||||||
|
s.subscriptionURLFailure(w, r, "番組を登録できませんでした。再試行してください")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.subscriptionURLRegisteredResponse(w, r)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) cancelSubscriptionURL(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
s.render(w, "subscription_url_result", subscriptionURLResultView{})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) fetchSubscriptionSeries(w http.ResponseWriter, r *http.Request, seriesID string) (string, []tver.Episode, bool) {
|
||||||
|
title, err := s.fetcher.FetchSeries(r.Context(), seriesID)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, tver.ErrSeriesNotFound) {
|
||||||
|
s.subscriptionURLFailure(w, r, "番組が見つかりません")
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
s.logSubscriptionFetchError(err)
|
||||||
|
s.subscriptionURLFailure(w, r, "番組情報を取得できませんでした。再試行してください")
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
episodes, err := s.fetcher.FetchSeriesEpisodes(r.Context(), seriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logSubscriptionFetchError(err)
|
||||||
|
s.subscriptionURLFailure(w, r, "番組情報を取得できませんでした。再試行してください")
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
return title, episodes, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) logSubscriptionFetchError(err error) {
|
||||||
|
s.logger.Printf("番組URLの情報取得に失敗: %s", platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) subscriptionURLFailure(w http.ResponseWriter, r *http.Request, message string) {
|
||||||
|
if r.Header.Get("HX-Request") != "true" {
|
||||||
|
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "subscription_url_result", subscriptionURLResultView{ErrorText: message})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) subscriptionURLRegisteredResponse(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Header.Get("HX-Request") != "true" {
|
||||||
|
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
view, err := s.loadSubscriptions(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("登録番組の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
view.OutOfBand = true
|
||||||
|
s.render(w, "subscription_url_registered", subscriptionURLRegisteredView{Subscriptions: view})
|
||||||
|
}
|
||||||
@@ -0,0 +1,260 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"errors"
|
||||||
|
"log"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseSeriesURL(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name, raw, want string
|
||||||
|
valid bool
|
||||||
|
}{
|
||||||
|
{"without www", "https://tver.jp/series/sr123", "sr123", true},
|
||||||
|
{"with www", "https://www.tver.jp/series/ABC123", "ABC123", true},
|
||||||
|
{"uppercase host", "https://TVER.JP/series/sr123", "sr123", true},
|
||||||
|
{"trailing slash", "https://tver.jp/series/sr123/", "sr123", true},
|
||||||
|
{"query and fragment", "https://tver.jp/series/sr123/?foo=bar#section", "sr123", true},
|
||||||
|
{"surrounding whitespace", " \thttps://tver.jp/series/sr123\n", "sr123", true},
|
||||||
|
{"http", "http://tver.jp/series/sr123", "", false},
|
||||||
|
{"subdomain suffix", "https://tver.jp.example.com/series/sr123", "", false},
|
||||||
|
{"evil prefix", "https://eviltver.jp/series/sr123", "", false},
|
||||||
|
{"userinfo on allowed host", "https://evil@tver.jp/series/sr123", "", false},
|
||||||
|
{"allowed host as userinfo", "https://tver.jp@evil.example/series/sr123", "", false},
|
||||||
|
{"episode", "https://tver.jp/episodes/ep123", "", false},
|
||||||
|
{"lp", "https://tver.jp/lp/example", "", false},
|
||||||
|
{"corner", "https://tver.jp/corner/example", "", false},
|
||||||
|
{"feature", "https://tver.jp/feature/example", "", false},
|
||||||
|
{"missing id", "https://tver.jp/series/", "", false},
|
||||||
|
{"invalid id", "https://tver.jp/series/sr_123", "", false},
|
||||||
|
{"extra path", "https://tver.jp/series/sr123/episodes", "", false},
|
||||||
|
{"encoded slash in id", "https://tver.jp/series/sr123%2Fevil", "", false},
|
||||||
|
{"encoded dot segments", "https://tver.jp/series/%2E%2E/episodes", "", false},
|
||||||
|
{"duplicate slash", "https://tver.jp//series/sr123", "", false},
|
||||||
|
{"port", "https://tver.jp:443/series/sr123", "", false},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
got, err := parseSeriesURL(tt.raw)
|
||||||
|
if tt.valid {
|
||||||
|
if err != nil || got != tt.want {
|
||||||
|
t.Fatalf("parseSeriesURL(%q)=%q, %v; want %q", tt.raw, got, err, tt.want)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err == nil || err.Error() != seriesURLInputError {
|
||||||
|
t.Fatalf("parseSeriesURL(%q)=%q, %v; want input error", tt.raw, got, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionsPageShowsSeriesURLForm(t *testing.T) {
|
||||||
|
page := getPage(newTestHandlerWithStore(&fakeFetcher{}, &fakeJobStore{}, nil), "/subscriptions")
|
||||||
|
body := page.Body.String()
|
||||||
|
if page.Code != http.StatusOK || !strings.Contains(body, `name="series_url"`) || !strings.Contains(body, `hx-post="/subscriptions/from-url/confirm"`) || !strings.Contains(body, `id="subscriptions-list"`) {
|
||||||
|
t.Fatalf("URL入力フォーム: status=%d body=%s", page.Code, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestConfirmSubscriptionURL(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{
|
||||||
|
seriesTitle: `<script>alert("title")</script>`,
|
||||||
|
seriesEpisodes: []tver.Episode{
|
||||||
|
{ID: "ep1", SeriesID: "sr1"},
|
||||||
|
{ID: "ep2", SeriesID: "sr1"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, &fakeJobStore{}, nil)
|
||||||
|
response := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1?x=1#fragment"}})
|
||||||
|
body := response.Body.String()
|
||||||
|
for _, want := range []string{"<script>", "番組ID: sr1", "公開中の回数: 2件", `name="series_id" value="sr1"`, "公開中の回もダウンロードする", ">登録</button>", ">キャンセル</button>"} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("確認画面に %q がない: %s", want, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if response.Code != http.StatusOK || strings.Contains(body, `<script>`) || strings.Contains(body, `name="series_title"`) || strings.Contains(body, `checked`) {
|
||||||
|
t.Fatalf("確認画面: status=%d body=%s", response.Code, body)
|
||||||
|
}
|
||||||
|
if len(fetcher.seriesTitleCalls) != 1 || fetcher.seriesTitleCalls[0] != "sr1" || len(fetcher.seriesCalls) != 1 || fetcher.seriesCalls[0] != "sr1" {
|
||||||
|
t.Fatalf("取得したID: title=%v episodes=%v", fetcher.seriesTitleCalls, fetcher.seriesCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestConfirmSubscriptionURLAlreadyRegisteredIncludingPaused(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{seriesTitle: "番組", seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1"}}}
|
||||||
|
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{"sr1": {SeriesID: "sr1", SeriesTitle: "番組", Enabled: false}}}
|
||||||
|
response := postForm(newTestHandlerWithStore(fetcher, jobs, nil), "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1"}})
|
||||||
|
body := response.Body.String()
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(body, "番組") || !strings.Contains(body, "公開中の回数: 1件") || !strings.Contains(body, "登録済み") || strings.Contains(body, ">登録</button>") || len(fetcher.seriesTitleCalls) != 1 || len(fetcher.seriesCalls) != 1 {
|
||||||
|
t.Fatalf("登録済み確認: status=%d body=%s title=%v episodes=%v", response.Code, body, fetcher.seriesTitleCalls, fetcher.seriesCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionURLNotFoundAndInvalidURL(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{seriesTitleErr: tver.ErrSeriesNotFound}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
notFound := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/missing"}})
|
||||||
|
if notFound.Code != http.StatusOK || !strings.Contains(notFound.Body.String(), "番組が見つかりません") || len(fetcher.seriesCalls) != 0 {
|
||||||
|
t.Fatalf("404: status=%d body=%s episodeCalls=%v", notFound.Code, notFound.Body.String(), fetcher.seriesCalls)
|
||||||
|
}
|
||||||
|
registerNotFound := postForm(handler, "/subscriptions/from-url/register", url.Values{"series_id": {"missing"}})
|
||||||
|
if registerNotFound.Code != http.StatusOK || !strings.Contains(registerNotFound.Body.String(), "番組が見つかりません") || jobs.registerCalls != 0 {
|
||||||
|
t.Fatalf("登録時404: status=%d body=%s registerCalls=%d", registerNotFound.Code, registerNotFound.Body.String(), jobs.registerCalls)
|
||||||
|
}
|
||||||
|
invalid := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://evil.example/series/sr1"}})
|
||||||
|
if invalid.Code != http.StatusOK || !strings.Contains(invalid.Body.String(), seriesURLInputError) || len(fetcher.seriesTitleCalls) != 2 {
|
||||||
|
t.Fatalf("不正URL: status=%d body=%s titleCalls=%v", invalid.Code, invalid.Body.String(), fetcher.seriesTitleCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionURLWithAndWithoutExistingEpisodes(t *testing.T) {
|
||||||
|
for _, download := range []bool{false, true} {
|
||||||
|
t.Run(map[bool]string{false: "without existing", true: "with existing"}[download], func(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{seriesTitle: "番組", seriesEpisodes: []tver.Episode{}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
confirmation := postForm(handler, "/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1"}})
|
||||||
|
if confirmation.Code != http.StatusOK || !strings.Contains(confirmation.Body.String(), "公開中の回はありません") {
|
||||||
|
t.Fatalf("0件の確認: %d %s", confirmation.Code, confirmation.Body.String())
|
||||||
|
}
|
||||||
|
values := url.Values{"series_id": {"sr1"}, "series_title": {"偽番組"}}
|
||||||
|
if download {
|
||||||
|
values.Set("download_existing", "1")
|
||||||
|
}
|
||||||
|
registered := postForm(handler, "/subscriptions/from-url/register", values)
|
||||||
|
body := registered.Body.String()
|
||||||
|
if registered.Code != http.StatusOK || !strings.Contains(body, "登録済み") || !strings.Contains(body, `id="subscriptions-list"`) || !strings.Contains(body, `hx-swap-oob="outerHTML"`) || !strings.Contains(body, `id="subscription-url-input"`) {
|
||||||
|
t.Fatalf("登録応答: %d %s", registered.Code, body)
|
||||||
|
}
|
||||||
|
if jobs.registerCalls != 1 || jobs.registerDownload != download || len(jobs.registerEpisodes) != 0 || jobs.subscriptions["sr1"].SeriesTitle != "番組" {
|
||||||
|
t.Fatalf("登録内容: calls=%d download=%t episodes=%v subscriptions=%v", jobs.registerCalls, jobs.registerDownload, jobs.registerEpisodes, jobs.subscriptions)
|
||||||
|
}
|
||||||
|
if len(fetcher.seriesTitleCalls) != 2 || len(fetcher.seriesCalls) != 2 {
|
||||||
|
t.Fatalf("確認と登録で再取得していない: title=%v episodes=%v", fetcher.seriesTitleCalls, fetcher.seriesCalls)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionURLUsesServerFetchedData(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{seriesTitle: "本当の番組", seriesEpisodes: []tver.Episode{{
|
||||||
|
ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Title: "公開中の回", Duration: 200,
|
||||||
|
}}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
response := postForm(handler, "/subscriptions/from-url/register", url.Values{
|
||||||
|
"series_id": {"sr1"}, "series_title": {"偽番組"}, "download_existing": {"1"},
|
||||||
|
})
|
||||||
|
if response.Code != http.StatusOK || jobs.registerCalls != 1 || !jobs.registerDownload || len(jobs.registerEpisodes) != 1 || jobs.registerEpisodes[0].ID != "ep1" || jobs.subscriptions["sr1"].SeriesTitle != "本当の番組" {
|
||||||
|
t.Fatalf("サーバー取得値での登録: status=%d calls=%d download=%t episodes=%+v subscriptions=%+v", response.Code, jobs.registerCalls, jobs.registerDownload, jobs.registerEpisodes, jobs.subscriptions)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionURLAlreadyRegisteredDoesNothing(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{seriesTitleErr: errors.New("must not fetch"), seriesErr: errors.New("must not fetch")}
|
||||||
|
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{"sr1": {SeriesID: "sr1", SeriesTitle: "番組", Enabled: false}}}
|
||||||
|
response := postForm(newTestHandlerWithStore(fetcher, jobs, nil), "/subscriptions/from-url/register", url.Values{"series_id": {"sr1"}})
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "登録済み") || jobs.registerCalls != 0 || len(fetcher.seriesTitleCalls) != 0 || len(fetcher.seriesCalls) != 0 {
|
||||||
|
t.Fatalf("登録済みの再登録: status=%d body=%s register=%d title=%v episodes=%v", response.Code, response.Body.String(), jobs.registerCalls, fetcher.seriesTitleCalls, fetcher.seriesCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionURLFetchFailuresDoNotRegisterAndRedactToken(t *testing.T) {
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name string
|
||||||
|
seriesErr error
|
||||||
|
episodesErr error
|
||||||
|
wantEpisodeCall int
|
||||||
|
}{
|
||||||
|
{"series", errors.New("platform_token=secret-series& upstream"), nil, 0},
|
||||||
|
{"episodes", nil, errors.New("platform_token=secret-episodes& upstream"), 1},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{seriesTitle: "番組", seriesTitleErr: tt.seriesErr, seriesErr: tt.episodesErr}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
var logs bytes.Buffer
|
||||||
|
handler := NewHandler(fetcher, jobs, "", time.Now, log.New(&logs, "", 0))
|
||||||
|
response := postForm(handler, "/subscriptions/from-url/register", url.Values{"series_id": {"sr1"}})
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(response.Body.String(), "番組情報を取得できませんでした。再試行してください") || jobs.registerCalls != 0 || len(jobs.subscriptions) != 0 {
|
||||||
|
t.Fatalf("取得失敗: status=%d body=%s calls=%d subscriptions=%v", response.Code, response.Body.String(), jobs.registerCalls, jobs.subscriptions)
|
||||||
|
}
|
||||||
|
if strings.Contains(logs.String(), "secret-") || !strings.Contains(logs.String(), "platform_token=[REDACTED]") || len(fetcher.seriesCalls) != tt.wantEpisodeCall {
|
||||||
|
t.Fatalf("ログまたは呼出回数: logs=%s episodeCalls=%v", logs.String(), fetcher.seriesCalls)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionURLRejectsInvalidChoice(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
for _, values := range []url.Values{
|
||||||
|
{},
|
||||||
|
{"series_id": {"bad_id"}},
|
||||||
|
{"series_id": {"../sr1"}},
|
||||||
|
{"series_id": {"sr1"}, "download_existing": {"yes"}},
|
||||||
|
} {
|
||||||
|
response := postForm(handler, "/subscriptions/from-url/register", values)
|
||||||
|
if response.Code != http.StatusBadRequest {
|
||||||
|
t.Errorf("不正値 %v: status=%d body=%s", values, response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(fetcher.seriesTitleCalls) != 0 || len(fetcher.seriesCalls) != 0 || jobs.subscribedCalls != 0 || jobs.registerCalls != 0 {
|
||||||
|
t.Fatalf("不正な hidden 値で外部取得または DB 操作を実行: title=%v episodes=%v subscribed=%d register=%d", fetcher.seriesTitleCalls, fetcher.seriesCalls, jobs.subscribedCalls, jobs.registerCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCancelSubscriptionURLClearsConfirmation(t *testing.T) {
|
||||||
|
response := getPage(newTestHandlerWithStore(&fakeFetcher{}, &fakeJobStore{}, nil), "/subscriptions/from-url/cancel")
|
||||||
|
body := response.Body.String()
|
||||||
|
if response.Code != http.StatusOK || !strings.Contains(body, `id="subscription-url-result"`) || strings.Contains(body, `name="series_id"`) || strings.Contains(body, `role="alert"`) || strings.Contains(body, `role="status"`) {
|
||||||
|
t.Fatalf("キャンセル応答: status=%d body=%q", response.Code, response.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionURLPostsWithoutHtmxRedirect(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{seriesTitle: "番組", seriesEpisodes: []tver.Episode{}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
for _, tt := range []struct {
|
||||||
|
path string
|
||||||
|
form url.Values
|
||||||
|
}{
|
||||||
|
{"/subscriptions/from-url/confirm", url.Values{"series_url": {"https://tver.jp/series/sr1"}}},
|
||||||
|
{"/subscriptions/from-url/register", url.Values{"series_id": {"sr1"}}},
|
||||||
|
} {
|
||||||
|
request := httptest.NewRequest(http.MethodPost, tt.path, strings.NewReader(tt.form.Encode()))
|
||||||
|
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, request)
|
||||||
|
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/subscriptions" {
|
||||||
|
t.Errorf("通常POST %s: status=%d Location=%q", tt.path, response.Code, response.Header().Get("Location"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(fetcher.seriesTitleCalls) != 1 || jobs.registerCalls != 1 {
|
||||||
|
t.Fatalf("登録POSTが処理されていない: fetch=%v register=%d", fetcher.seriesTitleCalls, jobs.registerCalls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionURLPostRoutesRejectGet(t *testing.T) {
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{}, &fakeJobStore{}, nil)
|
||||||
|
for _, path := range []string{"/subscriptions/from-url/confirm", "/subscriptions/from-url/register"} {
|
||||||
|
response := getPage(handler, path)
|
||||||
|
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
|
||||||
|
t.Errorf("GET %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,192 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type subscriptionActionView struct {
|
||||||
|
ID, Genre, ErrorText string
|
||||||
|
Subscribed, Choosing, DownloadExisting bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type subscriptionView struct {
|
||||||
|
ID, Title, CreatedAtText, LastCheckedText string
|
||||||
|
CheckErrorAtText, CheckErrorText string
|
||||||
|
Enabled bool
|
||||||
|
Jobs int64
|
||||||
|
}
|
||||||
|
|
||||||
|
type subscriptionsView struct {
|
||||||
|
Items []subscriptionView
|
||||||
|
OutOfBand bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) subscriptionChoice(w http.ResponseWriter, r *http.Request) {
|
||||||
|
genre, id, ok := episodeForm(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ep, found := s.cachedEpisode(genre, id)
|
||||||
|
if !found || ep.SeriesID == "" {
|
||||||
|
http.Error(w, "新着キャッシュに番組がありません。新着を再表示してください", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
registered, err := s.isSubscribed(r.Context(), ep.SeriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
|
||||||
|
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if registered {
|
||||||
|
s.subscriptionRegisteredResponse(w, r, id, genre)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) registerSubscription(w http.ResponseWriter, r *http.Request) {
|
||||||
|
genre, id, ok := episodeForm(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
choice := r.Form.Get("download_existing")
|
||||||
|
if choice != "" && choice != "1" {
|
||||||
|
http.Error(w, "公開中の回の選択が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ep, found := s.cachedEpisode(genre, id)
|
||||||
|
if !found || ep.SeriesID == "" || ep.SeriesTitle == "" {
|
||||||
|
http.Error(w, "新着キャッシュに番組がありません。新着を再表示してください", http.StatusNotFound)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
registered, err := s.isSubscribed(r.Context(), ep.SeriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組の登録状態取得に失敗: %v", err)
|
||||||
|
http.Error(w, "登録状態を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if registered {
|
||||||
|
s.subscriptionRegisteredResponse(w, r, id, genre)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
episodes, err := s.fetcher.FetchSeriesEpisodes(r.Context(), ep.SeriesID)
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組の公開中エピソード取得に失敗: %s", platformTokenPattern.ReplaceAllString(err.Error(), "${1}[REDACTED]"))
|
||||||
|
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true, DownloadExisting: choice == "1", ErrorText: "公開中の回を取得できませんでした。再試行してください"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
items := make([]store.Episode, 0, len(episodes))
|
||||||
|
for _, episode := range episodes {
|
||||||
|
items = append(items, storeEpisode(episode))
|
||||||
|
}
|
||||||
|
_, err = s.jobs.RegisterSubscription(r.Context(), ep.SeriesID, ep.SeriesTitle, items, choice == "1")
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("番組登録に失敗: %v", err)
|
||||||
|
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Choosing: true, DownloadExisting: choice == "1", ErrorText: "番組を登録できませんでした。再試行してください"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.subscriptionRegisteredResponse(w, r, id, genre)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) subscriptionRegisteredResponse(w http.ResponseWriter, r *http.Request, id, genre string) {
|
||||||
|
if r.Header.Get("HX-Request") != "true" && r.Method == http.MethodPost {
|
||||||
|
http.Redirect(w, r, "/?genre="+genre, http.StatusSeeOther)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "subscription_action", subscriptionActionView{ID: id, Genre: genre, Subscribed: true})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) isSubscribed(ctx context.Context, seriesID string) (bool, error) {
|
||||||
|
result, err := s.jobs.SubscribedSeries(ctx, []string{seriesID})
|
||||||
|
return result[seriesID], err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) subscriptionsPage(w http.ResponseWriter, r *http.Request) {
|
||||||
|
view, err := s.loadSubscriptions(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("登録番組の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "layout", page{Title: "登録番組", Subscriptions: &view})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) subscriptionsFragment(w http.ResponseWriter, r *http.Request) {
|
||||||
|
view, err := s.loadSubscriptions(r.Context())
|
||||||
|
if err != nil {
|
||||||
|
s.logger.Printf("登録番組の取得に失敗: %v", err)
|
||||||
|
http.Error(w, "登録番組を取得できませんでした", http.StatusInternalServerError)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.render(w, "subscriptions_list", view)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) loadSubscriptions(ctx context.Context) (subscriptionsView, error) {
|
||||||
|
subs, err := s.jobs.ListSubscriptions(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return subscriptionsView{}, err
|
||||||
|
}
|
||||||
|
view := subscriptionsView{Items: make([]subscriptionView, 0, len(subs))}
|
||||||
|
for _, sub := range subs {
|
||||||
|
item := subscriptionView{ID: sub.SeriesID, Title: sub.SeriesTitle,
|
||||||
|
CreatedAtText: sub.CreatedAt.In(s.zone).Format("2006-01-02 15:04 MST"),
|
||||||
|
LastCheckedText: "未確認", Enabled: sub.Enabled, Jobs: sub.JobCount}
|
||||||
|
if sub.LastCheckedAt != nil {
|
||||||
|
item.LastCheckedText = sub.LastCheckedAt.In(s.zone).Format("2006-01-02 15:04 MST")
|
||||||
|
}
|
||||||
|
if sub.LastCheckErrorAt != nil && sub.LastCheckError != nil {
|
||||||
|
item.CheckErrorAtText = sub.LastCheckErrorAt.In(s.zone).Format("2006-01-02 15:04 MST")
|
||||||
|
item.CheckErrorText = *sub.LastCheckError
|
||||||
|
}
|
||||||
|
view.Items = append(view.Items, item)
|
||||||
|
}
|
||||||
|
return view, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) pauseSubscription(w http.ResponseWriter, r *http.Request) {
|
||||||
|
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
|
||||||
|
return s.jobs.SetSubscriptionEnabled(ctx, id, false)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) resumeSubscription(w http.ResponseWriter, r *http.Request) {
|
||||||
|
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
|
||||||
|
return s.jobs.SetSubscriptionEnabled(ctx, id, true)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) deleteSubscription(w http.ResponseWriter, r *http.Request) {
|
||||||
|
s.subscriptionMutation(w, r, func(ctx context.Context, id string) error {
|
||||||
|
deleted, err := s.jobs.DeleteSubscription(ctx, id)
|
||||||
|
if err == nil && !deleted {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *server) subscriptionMutation(w http.ResponseWriter, r *http.Request, change func(context.Context, string) error) {
|
||||||
|
id := r.PathValue("id")
|
||||||
|
if id == "" {
|
||||||
|
http.Error(w, "番組 ID が不正です", http.StatusBadRequest)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := change(r.Context(), id); err != nil {
|
||||||
|
if errors.Is(err, store.ErrInvalidTransition) {
|
||||||
|
http.Error(w, "番組が見つかりません", http.StatusNotFound)
|
||||||
|
} else {
|
||||||
|
s.logger.Printf("登録番組の操作に失敗: %v", err)
|
||||||
|
http.Error(w, "登録番組を変更できませんでした", http.StatusInternalServerError)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.Header.Get("HX-Request") != "true" {
|
||||||
|
http.Redirect(w, r, "/subscriptions", http.StatusSeeOther)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
s.subscriptionsFragment(w, r)
|
||||||
|
}
|
||||||
@@ -0,0 +1,199 @@
|
|||||||
|
package web
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionChoicesAndServerOwnedSeries(t *testing.T) {
|
||||||
|
for _, download := range []bool{false, true} {
|
||||||
|
t.Run(map[bool]string{false: "without existing", true: "with existing"}[download], func(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Duration: 200}},
|
||||||
|
seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "本当の番組", Title: "公開中", Duration: 200}}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
page := getPage(handler, "/")
|
||||||
|
if page.Code != http.StatusOK || !strings.Contains(page.Body.String(), "番組を登録") || jobs.subscribedCalls != 1 || len(jobs.subscribedIDs) != 1 || jobs.subscribedIDs[0] != "sr1" {
|
||||||
|
t.Fatalf("新着の登録状態: %d ids=%v body=%s", page.Code, jobs.subscribedIDs, page.Body.String())
|
||||||
|
}
|
||||||
|
choice := getPage(handler, "/subscriptions/choice?genre=drama&episode_id=ep1")
|
||||||
|
if choice.Code != http.StatusOK || !strings.Contains(choice.Body.String(), "公開中の回もダウンロードする") || !strings.Contains(choice.Body.String(), ">登録</button>") {
|
||||||
|
t.Fatalf("登録時の選択: %d %s", choice.Code, choice.Body.String())
|
||||||
|
}
|
||||||
|
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}, "series_id": {"forged"}, "series_title": {"偽番組"}}
|
||||||
|
if download {
|
||||||
|
values.Set("download_existing", "1")
|
||||||
|
}
|
||||||
|
registered := postForm(handler, "/subscriptions/register", values)
|
||||||
|
if registered.Code != http.StatusOK || !strings.Contains(registered.Body.String(), "登録済み") {
|
||||||
|
t.Fatalf("登録応答: %d %s", registered.Code, registered.Body.String())
|
||||||
|
}
|
||||||
|
if jobs.registerCalls != 1 || jobs.registerDownload != download || len(jobs.registerEpisodes) != 1 || jobs.registerEpisodes[0].ID != "ep1" || jobs.registerEpisodes[0].SeriesID != "sr1" || jobs.subscriptions["sr1"].SeriesTitle != "本当の番組" {
|
||||||
|
t.Fatalf("登録内容: calls=%d download=%t episodes=%+v subs=%+v", jobs.registerCalls, jobs.registerDownload, jobs.registerEpisodes, jobs.subscriptions)
|
||||||
|
}
|
||||||
|
if len(fetcher.seriesCalls) != 1 || fetcher.seriesCalls[0] != "sr1" {
|
||||||
|
t.Fatalf("番組一覧取得: %v", fetcher.seriesCalls)
|
||||||
|
}
|
||||||
|
if !strings.Contains(getPage(handler, "/subscriptions").Body.String(), "最終確認: 2026-10-03 09:00 JST") {
|
||||||
|
t.Fatal("登録直後の最終確認日時が表示されない")
|
||||||
|
}
|
||||||
|
fetcher.seriesErr = errors.New("API unavailable")
|
||||||
|
duplicate := postForm(handler, "/subscriptions/register", values)
|
||||||
|
if duplicate.Code != http.StatusOK || !strings.Contains(duplicate.Body.String(), "登録済み") || len(fetcher.seriesCalls) != 1 || jobs.registerCalls != 1 {
|
||||||
|
t.Fatalf("重複登録が API を再取得: %d %s calls=%v register=%d", duplicate.Code, duplicate.Body.String(), fetcher.seriesCalls, jobs.registerCalls)
|
||||||
|
}
|
||||||
|
reloaded := getPage(handler, "/")
|
||||||
|
if !strings.Contains(reloaded.Body.String(), "登録済み") || strings.Contains(reloaded.Body.String(), ">番組を登録</button>") {
|
||||||
|
t.Fatalf("登録済みの表示: %s", reloaded.Body.String())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRegisterSubscriptionRejectsMissingCacheAndFetchFailure(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}}}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
values := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
|
||||||
|
if got := postForm(handler, "/subscriptions/register", values).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("キャッシュ無しの登録: %d", got)
|
||||||
|
}
|
||||||
|
getPage(handler, "/")
|
||||||
|
for _, path := range []string{"/subscriptions/register", "/subscriptions/sr1/pause", "/subscriptions/sr1/resume", "/subscriptions/sr1/delete"} {
|
||||||
|
response := getPage(handler, path)
|
||||||
|
if response.Code != http.StatusMethodNotAllowed || !strings.Contains(response.Header().Get("Allow"), "POST") {
|
||||||
|
t.Errorf("GET での変更 %s: status=%d Allow=%q", path, response.Code, response.Header().Get("Allow"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, tt := range []struct {
|
||||||
|
path string
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{"/subscriptions/choice?genre=news&episode_id=ep1", http.StatusBadRequest},
|
||||||
|
{"/subscriptions/choice?genre=drama", http.StatusBadRequest},
|
||||||
|
{"/subscriptions/choice?genre=drama&episode_id=missing", http.StatusNotFound},
|
||||||
|
} {
|
||||||
|
response := getPage(handler, tt.path)
|
||||||
|
if response.Code != tt.want {
|
||||||
|
t.Errorf("不正な選択 %s: got=%d want=%d", tt.path, response.Code, tt.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, tt := range []struct {
|
||||||
|
values url.Values
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{url.Values{"genre": {"news"}, "episode_id": {"ep1"}}, http.StatusBadRequest},
|
||||||
|
{url.Values{"genre": {"drama"}}, http.StatusBadRequest},
|
||||||
|
{url.Values{"genre": {"drama"}, "episode_id": {"missing"}}, http.StatusNotFound},
|
||||||
|
} {
|
||||||
|
response := postForm(handler, "/subscriptions/register", tt.values)
|
||||||
|
if response.Code != tt.want {
|
||||||
|
t.Errorf("不正な登録 %v: got=%d want=%d", tt.values, response.Code, tt.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
values.Set("download_existing", "bad")
|
||||||
|
if got := postForm(handler, "/subscriptions/register", values).Code; got != http.StatusBadRequest {
|
||||||
|
t.Fatalf("不正な選択: %d", got)
|
||||||
|
}
|
||||||
|
values.Del("download_existing")
|
||||||
|
fetcher.seriesErr = errors.New("platform_token=secret& upstream failure")
|
||||||
|
failed := postForm(handler, "/subscriptions/register", values)
|
||||||
|
if failed.Code != http.StatusOK || !strings.Contains(failed.Body.String(), "公開中の回を取得できませんでした") || strings.Contains(failed.Body.String(), "secret") || jobs.registerCalls != 0 {
|
||||||
|
t.Fatalf("取得失敗時に未登録: %d %s calls=%d", failed.Code, failed.Body.String(), jobs.registerCalls)
|
||||||
|
}
|
||||||
|
fetcher.seriesErr = nil
|
||||||
|
fetcher.seriesEpisodes = []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}}
|
||||||
|
jobs.registerErr = errors.New("secret database detail")
|
||||||
|
failed = postForm(handler, "/subscriptions/register", values)
|
||||||
|
if failed.Code != http.StatusOK || !strings.Contains(failed.Body.String(), "番組を登録できませんでした") || strings.Contains(failed.Body.String(), "secret") || len(jobs.subscriptions) != 0 {
|
||||||
|
t.Fatalf("DB 失敗時に未登録: %d %s subscriptions=%v", failed.Code, failed.Body.String(), jobs.subscriptions)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewerShowsInitiallySubscribedSeriesOncePerSeriesLookup(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{episodes: []tver.Episode{
|
||||||
|
{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200},
|
||||||
|
{ID: "ep2", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200},
|
||||||
|
}}
|
||||||
|
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{
|
||||||
|
"sr1": {SeriesID: "sr1", SeriesTitle: "番組", Enabled: true},
|
||||||
|
}}
|
||||||
|
response := getPage(newTestHandlerWithStore(fetcher, jobs, nil), "/")
|
||||||
|
body := response.Body.String()
|
||||||
|
if response.Code != http.StatusOK || strings.Count(body, "登録済み") != 2 || strings.Contains(body, ">番組を登録</button>") || jobs.subscribedCalls != 1 || len(jobs.subscribedIDs) != 1 || jobs.subscribedIDs[0] != "sr1" {
|
||||||
|
t.Fatalf("登録済み番組の初期表示: status=%d calls=%d ids=%v body=%s", response.Code, jobs.subscribedCalls, jobs.subscribedIDs, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionMutationsWithoutHtmxRedirect(t *testing.T) {
|
||||||
|
fetcher := &fakeFetcher{
|
||||||
|
episodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}},
|
||||||
|
seriesEpisodes: []tver.Episode{{ID: "ep1", SeriesID: "sr1", SeriesTitle: "番組", Duration: 200}},
|
||||||
|
}
|
||||||
|
jobs := &fakeJobStore{}
|
||||||
|
handler := newTestHandlerWithStore(fetcher, jobs, nil)
|
||||||
|
getPage(handler, "/")
|
||||||
|
form := url.Values{"genre": {"drama"}, "episode_id": {"ep1"}}
|
||||||
|
request := httptest.NewRequest(http.MethodPost, "/subscriptions/register", strings.NewReader(form.Encode()))
|
||||||
|
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||||
|
response := httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, request)
|
||||||
|
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/?genre=drama" {
|
||||||
|
t.Fatalf("通常登録 POST: status=%d Location=%q", response.Code, response.Header().Get("Location"))
|
||||||
|
}
|
||||||
|
request = httptest.NewRequest(http.MethodPost, "/subscriptions/sr1/pause", nil)
|
||||||
|
response = httptest.NewRecorder()
|
||||||
|
handler.ServeHTTP(response, request)
|
||||||
|
if response.Code != http.StatusSeeOther || response.Header().Get("Location") != "/subscriptions" || jobs.subscriptions["sr1"].Enabled {
|
||||||
|
t.Fatalf("通常操作 POST: status=%d Location=%q subscription=%+v", response.Code, response.Header().Get("Location"), jobs.subscriptions["sr1"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionsPagePauseResumeDeleteAndEscaping(t *testing.T) {
|
||||||
|
checked := time.Date(2026, 10, 4, 2, 30, 0, 0, time.UTC)
|
||||||
|
failedAt := time.Date(2026, 10, 4, 3, 45, 0, 0, time.UTC)
|
||||||
|
checkError := `<script>alert("error")</script> token=[REDACTED]`
|
||||||
|
jobs := &fakeJobStore{subscriptions: map[string]store.Subscription{
|
||||||
|
"sr1": {SeriesID: "sr1", SeriesTitle: `<script>alert(1)</script>`, Enabled: true,
|
||||||
|
CreatedAt: time.Date(2026, 10, 3, 0, 0, 0, 0, time.UTC), LastCheckedAt: &checked,
|
||||||
|
LastCheckErrorAt: &failedAt, LastCheckError: &checkError, JobCount: 3},
|
||||||
|
"sr2": {SeriesID: "sr2", SeriesTitle: "別番組", Enabled: false, JobCount: 0},
|
||||||
|
}}
|
||||||
|
handler := newTestHandlerWithStore(&fakeFetcher{}, jobs, nil)
|
||||||
|
page := getPage(handler, "/subscriptions")
|
||||||
|
if page.Code != http.StatusOK {
|
||||||
|
t.Fatalf("登録番組ページ: %d", page.Code)
|
||||||
|
}
|
||||||
|
body := page.Body.String()
|
||||||
|
for _, want := range []string{"<script>", "登録日時: 2026-10-03 09:00 JST", "最終確認: 2026-10-04 11:30 JST", "最終確認: 未確認", "最後の確認エラー（2026-10-04 12:45 JST）", `<script>alert("error")</script> token=[REDACTED]`, "関連ジョブ: 3件", "有効", "一時停止", `hx-confirm="番組の登録を解除しますか？"`} {
|
||||||
|
if !strings.Contains(body, want) {
|
||||||
|
t.Errorf("登録番組の情報 %q がない", want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(body, `<script>`) {
|
||||||
|
t.Fatal("番組名または確認エラーがエスケープされていない")
|
||||||
|
}
|
||||||
|
paused := postForm(handler, "/subscriptions/sr1/pause", nil)
|
||||||
|
if paused.Code != http.StatusOK || jobs.subscriptions["sr1"].Enabled || !strings.Contains(paused.Body.String(), `action="/subscriptions/sr1/resume"`) || strings.Contains(paused.Body.String(), "<html") {
|
||||||
|
t.Fatalf("一時停止: %d %s", paused.Code, paused.Body.String())
|
||||||
|
}
|
||||||
|
resumed := postForm(handler, "/subscriptions/sr1/resume", nil)
|
||||||
|
if resumed.Code != http.StatusOK || !jobs.subscriptions["sr1"].Enabled || !strings.Contains(resumed.Body.String(), `action="/subscriptions/sr1/pause"`) {
|
||||||
|
t.Fatalf("再開: %d %s", resumed.Code, resumed.Body.String())
|
||||||
|
}
|
||||||
|
deleted := postForm(handler, "/subscriptions/sr1/delete", nil)
|
||||||
|
if deleted.Code != http.StatusOK || len(jobs.subscriptions) != 1 || strings.Contains(deleted.Body.String(), "<script>") {
|
||||||
|
t.Fatalf("解除: %d %s", deleted.Code, deleted.Body.String())
|
||||||
|
}
|
||||||
|
if got := postForm(handler, "/subscriptions/sr1/delete", nil).Code; got != http.StatusNotFound {
|
||||||
|
t.Fatalf("存在しない番組の解除: %d", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
{{define "layout"}}<!doctype html>
|
||||||
|
<html lang="ja">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
|
<title>{{.Title}} | TVer</title>
|
||||||
|
<link rel="stylesheet" href="/static/app.css">
|
||||||
|
<script src="/static/htmx-2.0.11.min.js" defer></script>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<header>
|
||||||
|
<strong>TVer</strong>
|
||||||
|
<nav aria-label="メイン" hx-boost="true">
|
||||||
|
<a href="/">新着</a>
|
||||||
|
<a href="/queue">待ち行列</a>
|
||||||
|
<a href="/subscriptions">登録番組</a>
|
||||||
|
<a href="/library">ライブラリ</a>
|
||||||
|
</nav>
|
||||||
|
</header>
|
||||||
|
<main>
|
||||||
|
{{if .Newer}}
|
||||||
|
{{template "newer_page" .Newer}}
|
||||||
|
{{else if .Queue}}
|
||||||
|
{{template "queue_page" .Queue}}
|
||||||
|
{{else if .Subscriptions}}
|
||||||
|
{{template "subscriptions_page" .Subscriptions}}
|
||||||
|
{{else if .Library}}
|
||||||
|
{{template "library_page" .Library}}
|
||||||
|
{{else if .Series}}
|
||||||
|
{{template "series_page" .Series}}
|
||||||
|
{{else if .Video}}
|
||||||
|
{{template "video_page" .Video}}
|
||||||
|
{{else if .DeleteConfirm}}
|
||||||
|
{{template "delete_confirm_page" .DeleteConfirm}}
|
||||||
|
{{else if .DeleteResult}}
|
||||||
|
{{template "delete_result_page" .DeleteResult}}
|
||||||
|
{{else}}
|
||||||
|
<h1>{{.Title}}</h1>
|
||||||
|
{{end}}
|
||||||
|
</main>
|
||||||
|
</body>
|
||||||
|
</html>{{end}}
|
||||||
@@ -0,0 +1,115 @@
|
|||||||
|
{{define "library_page"}}
|
||||||
|
<h1>ライブラリ</h1>
|
||||||
|
{{if .Series}}
|
||||||
|
<p>{{.SeriesCount}}番組・{{.EpisodeCount}}本・{{.TotalSizeText}}</p>
|
||||||
|
<form action="/library/delete/confirm" method="post">
|
||||||
|
<div class="episode-grid library-select-grid">
|
||||||
|
{{range .Series}}
|
||||||
|
<article class="episode-card">
|
||||||
|
<label class="library-selection"><input type="checkbox" name="series_id" value="{{.SeriesID}}"> この番組を選択</label>
|
||||||
|
<a href="/library/series/{{.SeriesID}}">
|
||||||
|
<div class="library-card-image">
|
||||||
|
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
|
||||||
|
<span class="episode-count-badge">{{.EpisodeCount}}本</span>
|
||||||
|
</div>
|
||||||
|
<div class="episode-copy">
|
||||||
|
<h2>{{.SeriesTitle}} {{if .Subscribed}}<span class="subscription-badge">登録中</span>{{end}}</h2>
|
||||||
|
<p>放送局: {{.BroadcasterName}}</p>
|
||||||
|
<p>最新: {{.LatestTitle}}</p>
|
||||||
|
<p>最終完了: {{.LatestFinishedText}}</p>
|
||||||
|
<p>合計サイズ: {{.TotalSizeText}}</p>
|
||||||
|
</div>
|
||||||
|
</a>
|
||||||
|
</article>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
<p><button type="submit">選択した番組を削除</button></p>
|
||||||
|
</form>
|
||||||
|
{{else}}
|
||||||
|
<p>ダウンロード済みの動画はありません</p>
|
||||||
|
{{end}}
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "series_page"}}
|
||||||
|
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><span>{{.SeriesTitle}}</span></nav>
|
||||||
|
<h1>{{.SeriesTitle}} {{if .Subscribed}}<span class="subscription-badge">登録中</span>{{end}}</h1>
|
||||||
|
<p>{{.BroadcasterName}}・{{.EpisodeCount}}本・{{.TotalSizeText}}</p>
|
||||||
|
<p class="library-sort">並び順:
|
||||||
|
{{if eq .Sort "latest"}}<strong>完了が新しい順</strong>{{else}}<a href="/library/series/{{.SeriesID}}">完了が新しい順</a>{{end}}
|
||||||
|
／
|
||||||
|
{{if eq .Sort "natural"}}<strong>話の順</strong>{{else}}<a href="/library/series/{{.SeriesID}}?sort=natural">話の順</a>{{end}}
|
||||||
|
</p>
|
||||||
|
<form action="/library/delete/confirm" method="post">
|
||||||
|
<div class="library-episode-list">
|
||||||
|
{{range .Items}}
|
||||||
|
<div class="library-episode-select-row">
|
||||||
|
<label class="library-selection"><input type="checkbox" name="id" value="{{.ID}}"> 選択</label>
|
||||||
|
<a class="library-episode-row" href="/library/{{.ID}}">
|
||||||
|
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
|
||||||
|
<span class="library-episode-title">{{.Title}}</span>
|
||||||
|
<span>{{.FinishedAtText}}</span>
|
||||||
|
<span>{{.DurationText}}</span>
|
||||||
|
<span>{{.FileSizeText}}</span>
|
||||||
|
</a>
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
<p><button type="submit">選択した回を削除</button></p>
|
||||||
|
</form>
|
||||||
|
<form action="/library/delete/confirm" method="post">
|
||||||
|
<input type="hidden" name="series_id" value="{{.SeriesID}}">
|
||||||
|
<button type="submit">この番組をすべて削除</button>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "video_page"}}
|
||||||
|
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><a href="{{.SeriesURL}}">{{.SeriesTitle}}</a><span aria-hidden="true">›</span><span>{{.Title}}</span></nav>
|
||||||
|
<h1>{{.SeriesTitle}}</h1>
|
||||||
|
<h2>{{.Title}}</h2>
|
||||||
|
<video controls preload="metadata" src="{{.VideoURL}}"></video>
|
||||||
|
<p>放送局: {{.BroadcasterName}}／長さ: {{.DurationText}}／サイズ: {{.FileSizeText}}／完了日時: {{.FinishedAtText}}</p>
|
||||||
|
<nav class="episode-navigation" aria-label="前後の回">
|
||||||
|
{{if .Previous}}<a href="/library/{{.Previous.ID}}">← 前の回: {{.Previous.Title}}</a>{{else}}<span aria-disabled="true">前の回はありません</span>{{end}}
|
||||||
|
{{if .Next}}<a href="/library/{{.Next.ID}}">次の回: {{.Next.Title}} →</a>{{else}}<span aria-disabled="true">次の回はありません</span>{{end}}
|
||||||
|
</nav>
|
||||||
|
<form action="/library/delete/confirm" method="post">
|
||||||
|
<input type="hidden" name="id" value="{{.ID}}">
|
||||||
|
<button type="submit">削除</button>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "delete_confirm_page"}}
|
||||||
|
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><span>削除確認</span></nav>
|
||||||
|
<h1>動画の削除確認</h1>
|
||||||
|
<p class="delete-warning">ファイルと記録を削除します。この操作は取り消せません。</p>
|
||||||
|
<p>{{.Count}}本・合計{{.TotalSizeText}}</p>
|
||||||
|
{{range .Groups}}
|
||||||
|
<section class="delete-group">
|
||||||
|
<h2>{{.SeriesTitle}}</h2>
|
||||||
|
<p>{{len .Items}}本・{{.TotalSizeText}}</p>
|
||||||
|
<ul>
|
||||||
|
{{range .Items}}<li>{{.Title}}（{{.FileSizeText}}）</li>{{end}}
|
||||||
|
</ul>
|
||||||
|
</section>
|
||||||
|
{{end}}
|
||||||
|
<form action="/library/delete" method="post">
|
||||||
|
{{range .IDs}}<input type="hidden" name="id" value="{{.}}">{{end}}
|
||||||
|
<button type="submit">削除を実行</button>
|
||||||
|
<a href="{{.CancelURL}}">取り消す</a>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "delete_result_page"}}
|
||||||
|
<nav class="breadcrumbs" aria-label="パンくず"><a href="/library">ライブラリ</a><span aria-hidden="true">›</span><span>削除結果</span></nav>
|
||||||
|
<h1>動画の削除結果</h1>
|
||||||
|
<p>削除: {{.DeletedCount}}件／失敗: {{len .Failures}}件／対象外: {{.ExcludedCount}}件</p>
|
||||||
|
{{if .Failures}}
|
||||||
|
<h2>失敗した回</h2>
|
||||||
|
<ul>{{range .Failures}}<li>{{.Title}}: {{.Reason}}</li>{{end}}</ul>
|
||||||
|
{{end}}
|
||||||
|
<p><a href="/library">ライブラリへ戻る</a></p>
|
||||||
|
{{if .SeriesLinks}}
|
||||||
|
<h2>回が残っている番組</h2>
|
||||||
|
<ul>{{range .SeriesLinks}}<li><a href="/library/series/{{.SeriesID}}">{{.SeriesTitle}}</a></li>{{end}}</ul>
|
||||||
|
{{end}}
|
||||||
|
{{end}}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
{{define "newer_page"}}
|
||||||
|
<h1>新着</h1>
|
||||||
|
<form id="newer-controls" action="/" method="get"
|
||||||
|
hx-get="/newer" hx-trigger="change, submit"
|
||||||
|
hx-target="#newer-results" hx-swap="outerHTML">
|
||||||
|
<label for="genre">ジャンル</label>
|
||||||
|
<select id="genre" name="genre">
|
||||||
|
<option value="drama" {{if eq .Genre "drama"}}selected{{end}}>ドラマ</option>
|
||||||
|
<option value="variety" {{if eq .Genre "variety"}}selected{{end}}>バラエティ</option>
|
||||||
|
</select>
|
||||||
|
<input type="hidden" name="hide_previews" value="0">
|
||||||
|
<label for="hide-previews">
|
||||||
|
<input id="hide-previews" type="checkbox" name="hide_previews" value="1" {{if .HidePreviews}}checked{{end}}>
|
||||||
|
予告を隠す
|
||||||
|
</label>
|
||||||
|
<button type="submit">表示</button>
|
||||||
|
</form>
|
||||||
|
{{template "newer_results" .}}
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "newer_results"}}
|
||||||
|
<section id="newer-results" aria-label="新着一覧" aria-live="polite">
|
||||||
|
{{if .Failed}}
|
||||||
|
<p role="status">{{.ErrorText}}</p>
|
||||||
|
{{else if not .Episodes}}
|
||||||
|
<p>表示する新着はありません</p>
|
||||||
|
{{else}}
|
||||||
|
<div class="episode-grid">
|
||||||
|
{{range .Episodes}}
|
||||||
|
<article class="episode-card">
|
||||||
|
<a href="{{.URL}}">
|
||||||
|
<img src="{{.ThumbnailURL}}" alt="" loading="lazy">
|
||||||
|
<div class="episode-copy">
|
||||||
|
<h2>{{.SeriesTitle}}</h2>
|
||||||
|
<p>{{.Title}}</p>
|
||||||
|
<p>放送局: {{.BroadcasterName}}</p>
|
||||||
|
<p>長さ: {{.DurationMinutes}}分</p>
|
||||||
|
<p>放送日: {{.BroadcastDateLabel}}</p>
|
||||||
|
<p>配信終了: <time datetime="{{.EndAtISO}}">{{.EndAtText}}</time></p>
|
||||||
|
</div>
|
||||||
|
</a>
|
||||||
|
{{template "episode_action" .}}
|
||||||
|
{{template "subscription_action" .SubscriptionAction}}
|
||||||
|
</article>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
</section>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "episode_action"}}
|
||||||
|
<div class="episode-action">
|
||||||
|
{{if not .Status}}
|
||||||
|
<form action="/episodes/enqueue" method="post" hx-post="/episodes/enqueue" hx-target="closest .episode-action" hx-swap="outerHTML">
|
||||||
|
<input type="hidden" name="episode_id" value="{{.ID}}">
|
||||||
|
<input type="hidden" name="genre" value="{{.Genre}}">
|
||||||
|
<button type="submit">ダウンロード</button>
|
||||||
|
</form>
|
||||||
|
{{else}}
|
||||||
|
<span role="status">{{.StatusText}}</span>
|
||||||
|
{{if or (eq .Status "failed") (eq .Status "canceled")}}
|
||||||
|
<form action="/episodes/retry" method="post" hx-post="/episodes/retry" hx-target="closest .episode-action" hx-swap="outerHTML">
|
||||||
|
<input type="hidden" name="episode_id" value="{{.ID}}">
|
||||||
|
<input type="hidden" name="genre" value="{{.Genre}}">
|
||||||
|
<button type="submit">再試行</button>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
{{define "queue_page"}}
|
||||||
|
<h1>待ち行列</h1>
|
||||||
|
{{template "queue_list" .}}
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "queue_list"}}
|
||||||
|
<div id="queue-list" {{if .Poll}}hx-get="/queue/list" hx-trigger="every 5s" hx-target="this" hx-swap="outerHTML"{{end}}>
|
||||||
|
<section aria-label="ダウンロード中">
|
||||||
|
<h2>ダウンロード中</h2>
|
||||||
|
{{range .Running}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
|
||||||
|
</section>
|
||||||
|
<section aria-label="待機中">
|
||||||
|
<h2>待機中</h2>
|
||||||
|
{{range .Queued}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
|
||||||
|
</section>
|
||||||
|
<section aria-label="失敗">
|
||||||
|
<h2>失敗</h2>
|
||||||
|
{{range .Failed}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
|
||||||
|
</section>
|
||||||
|
<section aria-label="取り消し">
|
||||||
|
<h2>取り消し</h2>
|
||||||
|
{{range .Canceled}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
|
||||||
|
</section>
|
||||||
|
<section aria-label="完了">
|
||||||
|
<h2>完了（直近50件）</h2>
|
||||||
|
{{range .Succeeded}}{{template "queue_job" .}}{{else}}<p>ありません</p>{{end}}
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "queue_job"}}
|
||||||
|
<article class="queue-job">
|
||||||
|
<h3>{{.SeriesTitle}} — {{.Title}}</h3>
|
||||||
|
<p>放送局: {{.BroadcasterName}}／登録元: {{.SourceText}}／登録日時: {{.CreatedAtText}}</p>
|
||||||
|
{{if eq .Status "running"}}
|
||||||
|
<p>段階: {{if .PhaseText}}{{.PhaseText}}{{else}}準備中{{end}}
|
||||||
|
{{if .PercentText}}／{{.PercentText}}{{end}}
|
||||||
|
{{if .SpeedText}}／{{.SpeedText}}{{end}}
|
||||||
|
{{if .ETAText}}／{{.ETAText}}{{end}}
|
||||||
|
{{if .FragmentText}}／{{.FragmentText}}{{end}}</p>
|
||||||
|
<form action="/jobs/{{.ID}}/cancel" method="post" hx-post="/jobs/{{.ID}}/cancel" hx-target="#queue-list" hx-swap="outerHTML">
|
||||||
|
<button type="submit">取り消し</button>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
{{if eq .Status "failed"}}
|
||||||
|
<p>失敗理由: {{.ErrorKindText}}</p>
|
||||||
|
{{if .ErrorMessage}}<details><summary>詳細</summary><pre>{{.ErrorMessage}}</pre></details>{{end}}
|
||||||
|
{{end}}
|
||||||
|
{{if eq .Status "queued"}}
|
||||||
|
<form action="/jobs/{{.ID}}/cancel" method="post" hx-post="/jobs/{{.ID}}/cancel" hx-target="#queue-list" hx-swap="outerHTML">
|
||||||
|
<button type="submit">取り消し</button>
|
||||||
|
</form>
|
||||||
|
{{else if or (eq .Status "failed") (eq .Status "canceled")}}
|
||||||
|
<form action="/jobs/{{.ID}}/retry" method="post" hx-post="/jobs/{{.ID}}/retry" hx-target="#queue-list" hx-swap="outerHTML">
|
||||||
|
<button type="submit">再試行</button>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
</article>
|
||||||
|
{{end}}
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
{{define "subscription_action"}}
|
||||||
|
<div class="subscription-action">
|
||||||
|
{{if .Subscribed}}
|
||||||
|
<span role="status">登録済み</span>
|
||||||
|
{{else if .Choosing}}
|
||||||
|
{{if .ErrorText}}<p role="alert">{{.ErrorText}}</p>{{end}}
|
||||||
|
<form action="/subscriptions/register" method="post" hx-post="/subscriptions/register" hx-target="closest .subscription-action" hx-swap="outerHTML">
|
||||||
|
<input type="hidden" name="episode_id" value="{{.ID}}">
|
||||||
|
<input type="hidden" name="genre" value="{{.Genre}}">
|
||||||
|
<label><input type="checkbox" name="download_existing" value="1" {{if .DownloadExisting}}checked{{end}}>公開中の回もダウンロードする</label>
|
||||||
|
<button type="submit">登録</button>
|
||||||
|
</form>
|
||||||
|
{{else}}
|
||||||
|
<button type="button" hx-get="/subscriptions/choice?genre={{.Genre}}&episode_id={{.ID}}" hx-target="closest .subscription-action" hx-swap="outerHTML">番組を登録</button>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "subscriptions_page"}}
|
||||||
|
<h1>登録番組</h1>
|
||||||
|
<section class="subscription-url-entry">
|
||||||
|
<h2>番組URLから登録</h2>
|
||||||
|
<form action="/subscriptions/from-url/confirm" method="post" hx-post="/subscriptions/from-url/confirm" hx-target="#subscription-url-result" hx-swap="outerHTML">
|
||||||
|
<label>番組URL <input id="subscription-url-input" type="url" name="series_url" placeholder="https://tver.jp/series/…" required></label>
|
||||||
|
<button type="submit">確認</button>
|
||||||
|
</form>
|
||||||
|
<div id="subscription-url-result"></div>
|
||||||
|
</section>
|
||||||
|
{{template "subscriptions_list" .}}
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "subscription_url_result"}}
|
||||||
|
<div id="subscription-url-result">
|
||||||
|
{{if .ErrorText}}
|
||||||
|
<p role="alert">{{.ErrorText}}</p>
|
||||||
|
{{else if .SeriesID}}
|
||||||
|
<h3>{{.Title}}</h3>
|
||||||
|
<p>番組ID: {{.SeriesID}}</p>
|
||||||
|
{{if eq .EpisodeCount 0}}<p>公開中の回はありません</p>{{else}}<p>公開中の回数: {{.EpisodeCount}}件</p>{{end}}
|
||||||
|
{{if .Subscribed}}
|
||||||
|
<p role="status">登録済み</p>
|
||||||
|
<button type="button" hx-get="/subscriptions/from-url/cancel" hx-target="#subscription-url-result" hx-swap="outerHTML">キャンセル</button>
|
||||||
|
{{else}}
|
||||||
|
<form action="/subscriptions/from-url/register" method="post" hx-post="/subscriptions/from-url/register" hx-target="#subscription-url-result" hx-swap="outerHTML">
|
||||||
|
<input type="hidden" name="series_id" value="{{.SeriesID}}">
|
||||||
|
<label><input type="checkbox" name="download_existing" value="1">公開中の回もダウンロードする</label>
|
||||||
|
<button type="submit">登録</button>
|
||||||
|
<button type="button" hx-get="/subscriptions/from-url/cancel" hx-target="#subscription-url-result" hx-swap="outerHTML">キャンセル</button>
|
||||||
|
</form>
|
||||||
|
{{end}}
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "subscription_url_registered"}}
|
||||||
|
<div id="subscription-url-result"><p role="status">登録済み</p></div>
|
||||||
|
<input id="subscription-url-input" type="url" name="series_url" placeholder="https://tver.jp/series/…" required hx-swap-oob="outerHTML">
|
||||||
|
{{template "subscriptions_list" .Subscriptions}}
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
{{define "subscriptions_list"}}
|
||||||
|
<div id="subscriptions-list" {{if .OutOfBand}}hx-swap-oob="outerHTML"{{end}}>
|
||||||
|
{{range .Items}}
|
||||||
|
<article class="subscription-item">
|
||||||
|
<h2>{{.Title}}</h2>
|
||||||
|
<p>登録日時: {{.CreatedAtText}}／最終確認: {{.LastCheckedText}}／状態: {{if .Enabled}}有効{{else}}一時停止{{end}}／関連ジョブ: {{.Jobs}}件</p>
|
||||||
|
{{if .CheckErrorText}}
|
||||||
|
<details class="error"><summary>最後の確認エラー（{{.CheckErrorAtText}}）</summary><p>{{.CheckErrorText}}</p></details>
|
||||||
|
{{end}}
|
||||||
|
{{if .Enabled}}
|
||||||
|
<form action="/subscriptions/{{.ID}}/pause" method="post" hx-post="/subscriptions/{{.ID}}/pause" hx-target="#subscriptions-list" hx-swap="outerHTML"><button type="submit">一時停止</button></form>
|
||||||
|
{{else}}
|
||||||
|
<form action="/subscriptions/{{.ID}}/resume" method="post" hx-post="/subscriptions/{{.ID}}/resume" hx-target="#subscriptions-list" hx-swap="outerHTML"><button type="submit">再開</button></form>
|
||||||
|
{{end}}
|
||||||
|
<form action="/subscriptions/{{.ID}}/delete" method="post" hx-post="/subscriptions/{{.ID}}/delete" hx-target="#subscriptions-list" hx-swap="outerHTML" hx-confirm="番組の登録を解除しますか？"><button type="submit">解除</button></form>
|
||||||
|
</article>
|
||||||
|
{{else}}
|
||||||
|
<p>登録番組はありません</p>
|
||||||
|
{{end}}
|
||||||
|
</div>
|
||||||
|
{{end}}
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
package worker
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type outputEvent struct {
|
||||||
|
Event string `json:"event"`
|
||||||
|
FormatID string `json:"format_id"`
|
||||||
|
Filepath string `json:"filepath"`
|
||||||
|
Progress struct {
|
||||||
|
DownloadedBytes *float64 `json:"downloaded_bytes"`
|
||||||
|
TotalBytes *float64 `json:"total_bytes"`
|
||||||
|
TotalBytesEstimate *float64 `json:"total_bytes_estimate"`
|
||||||
|
Speed *float64 `json:"speed"`
|
||||||
|
ETA *float64 `json:"eta"`
|
||||||
|
FragmentIndex *float64 `json:"fragment_index"`
|
||||||
|
FragmentCount *float64 `json:"fragment_count"`
|
||||||
|
} `json:"progress"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseProgress(line string) (store.Progress, bool) {
|
||||||
|
var event outputEvent
|
||||||
|
if err := json.Unmarshal([]byte(line), &event); err != nil {
|
||||||
|
return store.Progress{}, false
|
||||||
|
}
|
||||||
|
phase := ""
|
||||||
|
switch event.Event {
|
||||||
|
case "download":
|
||||||
|
if strings.Contains(strings.ToLower(event.FormatID), "audio") {
|
||||||
|
phase = "audio"
|
||||||
|
} else {
|
||||||
|
phase = "video"
|
||||||
|
}
|
||||||
|
case "postprocess":
|
||||||
|
phase = "merging"
|
||||||
|
default:
|
||||||
|
return store.Progress{}, false
|
||||||
|
}
|
||||||
|
total := int64Pointer(event.Progress.TotalBytes)
|
||||||
|
estimate := false
|
||||||
|
if total == nil && event.Progress.TotalBytesEstimate != nil {
|
||||||
|
total = int64Pointer(event.Progress.TotalBytesEstimate)
|
||||||
|
estimate = true
|
||||||
|
}
|
||||||
|
var formatID *string
|
||||||
|
if event.FormatID != "" {
|
||||||
|
formatID = &event.FormatID
|
||||||
|
}
|
||||||
|
return store.Progress{Phase: phase, DownloadedBytes: int64Pointer(event.Progress.DownloadedBytes),
|
||||||
|
TotalBytes: total, TotalIsEstimate: estimate, SpeedBPS: event.Progress.Speed,
|
||||||
|
ETASeconds: event.Progress.ETA, FragmentIndex: int32Pointer(event.Progress.FragmentIndex),
|
||||||
|
FragmentCount: int32Pointer(event.Progress.FragmentCount), FormatID: formatID}, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func int64Pointer(value *float64) *int64 {
|
||||||
|
if value == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
result := int64(*value)
|
||||||
|
return &result
|
||||||
|
}
|
||||||
|
|
||||||
|
func int32Pointer(value *float64) *int32 {
|
||||||
|
if value == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
result := int32(*value)
|
||||||
|
return &result
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseAfterMove(line string) (string, bool) {
|
||||||
|
var event outputEvent
|
||||||
|
if err := json.Unmarshal([]byte(line), &event); err != nil || event.Event != "after_move" || event.Filepath == "" {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return event.Filepath, true
|
||||||
|
}
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
package worker
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestParseProgress(t *testing.T) {
|
||||||
|
video, ok := parseProgress(`{"event":"download","format_id":"hls-698-1","progress":{"downloaded_bytes":25,"total_bytes_estimate":100.0,"speed":10.5,"eta":3,"fragment_index":1,"fragment_count":4}}`)
|
||||||
|
if !ok || video.Phase != "video" || video.DownloadedBytes == nil || *video.DownloadedBytes != 25 || video.TotalBytes == nil || *video.TotalBytes != 100 || !video.TotalIsEstimate || video.SpeedBPS == nil || *video.SpeedBPS != 10.5 || video.ETASeconds == nil || *video.ETASeconds != 3 || video.FragmentIndex == nil || *video.FragmentIndex != 1 || video.FragmentCount == nil || *video.FragmentCount != 4 || video.FormatID == nil || *video.FormatID != "hls-698-1" {
|
||||||
|
t.Fatalf("映像進捗: %+v ok=%v", video, ok)
|
||||||
|
}
|
||||||
|
audio, ok := parseProgress(`{"event":"download","format_id":"hls-ts_AUDIO-0_2","progress":{"downloaded_bytes":50,"total_bytes":50}}`)
|
||||||
|
if !ok || audio.Phase != "audio" || audio.TotalIsEstimate || audio.TotalBytes == nil || *audio.TotalBytes != 50 {
|
||||||
|
t.Fatalf("音声進捗: %+v ok=%v", audio, ok)
|
||||||
|
}
|
||||||
|
merging, ok := parseProgress(`{"event":"postprocess","progress":{"status":"started","postprocessor":"Merger"}}`)
|
||||||
|
if !ok || merging.Phase != "merging" {
|
||||||
|
t.Fatalf("結合進捗: %+v ok=%v", merging, ok)
|
||||||
|
}
|
||||||
|
path, ok := parseAfterMove(`{"event":"after_move","filepath":"/videos/incoming/ep.mp4"}`)
|
||||||
|
if !ok || path != "/videos/incoming/ep.mp4" {
|
||||||
|
t.Fatalf("移動後パス: %q ok=%v", path, ok)
|
||||||
|
}
|
||||||
|
for _, invalid := range []string{"noise", `{"event":"other"}`} {
|
||||||
|
if _, ok := parseProgress(invalid); ok {
|
||||||
|
t.Errorf("不正な進捗を許した: %q", invalid)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,504 @@
|
|||||||
|
package worker
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/media"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
errCanceled = errors.New("job canceled")
|
||||||
|
errShutdown = errors.New("worker shutting down")
|
||||||
|
errInvalidOutput = errors.New("invalid output")
|
||||||
|
headerSecretRE = regexp.MustCompile(`(?im)(authorization|cookie)([=:][ \t]*)([^\r\n]+)`)
|
||||||
|
paramSecretRE = regexp.MustCompile(`(?i)(token|platform_uid|platform_token)([=:][[:space:]]*)([^&[:space:]]+)`)
|
||||||
|
)
|
||||||
|
|
||||||
|
type JobStore interface {
|
||||||
|
ClaimNext(context.Context) (store.Job, error)
|
||||||
|
UpdateProgress(context.Context, int64, store.Progress) error
|
||||||
|
Heartbeat(context.Context, int64) error
|
||||||
|
GetJob(context.Context, int64) (store.Job, error)
|
||||||
|
Complete(context.Context, int64, string, int64) error
|
||||||
|
Fail(context.Context, int64, string, string) error
|
||||||
|
RequeueRunning(context.Context, int64) error
|
||||||
|
}
|
||||||
|
|
||||||
|
type Config struct {
|
||||||
|
VideoDir string
|
||||||
|
YTDLPPath string
|
||||||
|
FFprobePath string
|
||||||
|
Concurrency int
|
||||||
|
PollInterval time.Duration
|
||||||
|
ProgressInterval time.Duration
|
||||||
|
HeartbeatInterval time.Duration
|
||||||
|
CancelInterval time.Duration
|
||||||
|
}
|
||||||
|
|
||||||
|
type Runner struct {
|
||||||
|
store JobStore
|
||||||
|
config Config
|
||||||
|
logger *log.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
type cappedBuffer struct {
|
||||||
|
buffer bytes.Buffer
|
||||||
|
limit int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (b *cappedBuffer) Write(p []byte) (int, error) {
|
||||||
|
written := len(p)
|
||||||
|
remaining := b.limit - b.buffer.Len()
|
||||||
|
if remaining > 0 {
|
||||||
|
if remaining < len(p) {
|
||||||
|
p = p[:remaining]
|
||||||
|
}
|
||||||
|
_, _ = b.buffer.Write(p)
|
||||||
|
}
|
||||||
|
return written, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (b *cappedBuffer) String() string { return b.buffer.String() }
|
||||||
|
|
||||||
|
func scanLines(reader io.Reader, capture *cappedBuffer) <-chan string {
|
||||||
|
lines := make(chan string)
|
||||||
|
go func() {
|
||||||
|
defer close(lines)
|
||||||
|
scanner := bufio.NewScanner(reader)
|
||||||
|
scanner.Buffer(make([]byte, 4096), 1<<20)
|
||||||
|
for scanner.Scan() {
|
||||||
|
line := scanner.Text()
|
||||||
|
if capture != nil {
|
||||||
|
_, _ = capture.Write([]byte(line + "\n"))
|
||||||
|
}
|
||||||
|
lines <- line
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
return lines
|
||||||
|
}
|
||||||
|
|
||||||
|
func New(jobs JobStore, config Config, logger *log.Logger) (*Runner, error) {
|
||||||
|
if config.VideoDir == "" {
|
||||||
|
config.VideoDir = "/videos"
|
||||||
|
}
|
||||||
|
if config.YTDLPPath == "" {
|
||||||
|
config.YTDLPPath = "yt-dlp"
|
||||||
|
}
|
||||||
|
if config.FFprobePath == "" {
|
||||||
|
config.FFprobePath = "ffprobe"
|
||||||
|
}
|
||||||
|
if config.Concurrency == 0 {
|
||||||
|
config.Concurrency = 1
|
||||||
|
}
|
||||||
|
if config.Concurrency < 1 || config.Concurrency > 2 {
|
||||||
|
return nil, errors.New("worker concurrency must be 1 or 2")
|
||||||
|
}
|
||||||
|
if config.PollInterval <= 0 {
|
||||||
|
config.PollInterval = time.Second
|
||||||
|
}
|
||||||
|
if config.ProgressInterval <= 0 {
|
||||||
|
config.ProgressInterval = time.Second
|
||||||
|
}
|
||||||
|
if config.HeartbeatInterval <= 0 {
|
||||||
|
config.HeartbeatInterval = 5 * time.Second
|
||||||
|
}
|
||||||
|
if config.CancelInterval <= 0 {
|
||||||
|
config.CancelInterval = 500 * time.Millisecond
|
||||||
|
}
|
||||||
|
if logger == nil {
|
||||||
|
logger = log.Default()
|
||||||
|
}
|
||||||
|
return &Runner{store: jobs, config: config, logger: logger}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) Run(ctx context.Context) error {
|
||||||
|
sem := make(chan struct{}, r.config.Concurrency)
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
ticker := time.NewTicker(r.config.PollInterval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
for len(sem) < cap(sem) && ctx.Err() == nil {
|
||||||
|
job, err := r.store.ClaimNext(ctx)
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
r.logger.Printf("ジョブ取得に失敗: %v", err)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
sem <- struct{}{}
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
defer func() { <-sem }()
|
||||||
|
r.ProcessJob(ctx, job)
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
wg.Wait()
|
||||||
|
return nil
|
||||||
|
case <-ticker.C:
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) ProcessJob(ctx context.Context, job store.Job) {
|
||||||
|
workDir := filepath.Join(r.config.VideoDir, ".incoming", strconv.FormatInt(job.ID, 10))
|
||||||
|
if err := os.RemoveAll(workDir); err != nil {
|
||||||
|
r.fail(job.ID, "unknown", fmt.Sprintf("作業ディレクトリを初期化できません: %v", err))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(workDir, 0o755); err != nil {
|
||||||
|
r.fail(job.ID, "unknown", fmt.Sprintf("作業ディレクトリを作成できません: %v", err))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer os.RemoveAll(workDir)
|
||||||
|
relative, err := media.RelativeVideoPath(job.SeriesTitle, job.Title, job.EpisodeID)
|
||||||
|
if err != nil {
|
||||||
|
r.failIfRunning(job.ID, "invalid_output", redact(err.Error()))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
downloaded, stderr, err := r.download(ctx, job, workDir)
|
||||||
|
if errors.Is(err, errShutdown) {
|
||||||
|
r.requeue(job.ID)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if errors.Is(err, errCanceled) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
message := redact(stderr)
|
||||||
|
if message == "" {
|
||||||
|
message = redact(err.Error())
|
||||||
|
}
|
||||||
|
kind := classifyFailure(message)
|
||||||
|
if errors.Is(err, errInvalidOutput) {
|
||||||
|
kind = "invalid_output"
|
||||||
|
}
|
||||||
|
r.failIfRunning(job.ID, kind, message)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.canceled(job.ID) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := r.probe(ctx, downloaded); err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
r.requeue(job.ID)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
r.failIfRunning(job.ID, "invalid_output", redact(err.Error()))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if r.canceled(job.ID) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := r.store.UpdateProgress(ctx, job.ID, store.Progress{Phase: "moving"}); err != nil {
|
||||||
|
if r.canceled(job.ID) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
r.failIfRunning(job.ID, "unknown", "移動状態を記録できません: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
destination := filepath.Join(r.config.VideoDir, relative)
|
||||||
|
if err := os.MkdirAll(filepath.Dir(destination), 0o755); err != nil {
|
||||||
|
r.failIfRunning(job.ID, "unknown", "保存先を作成できません: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := os.Rename(downloaded, destination); err != nil {
|
||||||
|
r.failIfRunning(job.ID, "unknown", "動画を保存先へ移動できません: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
info, err := os.Stat(destination)
|
||||||
|
if err != nil {
|
||||||
|
r.failIfRunning(job.ID, "unknown", "保存後の動画を確認できません: "+err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := r.store.Complete(ctx, job.ID, filepath.ToSlash(relative), info.Size()); err != nil {
|
||||||
|
if r.canceled(job.ID) {
|
||||||
|
if removeErr := os.Remove(destination); removeErr != nil && !errors.Is(removeErr, os.ErrNotExist) {
|
||||||
|
r.logger.Printf("取り消されたジョブ %d の保存済みファイル削除に失敗: %v", job.ID, removeErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
r.logger.Printf("ジョブ %d の完了記録に失敗: %v", job.ID, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) download(ctx context.Context, job store.Job, workDir string) (string, string, error) {
|
||||||
|
output := filepath.Join(workDir, "%(id)s.%(ext)s")
|
||||||
|
args := []string{"--no-playlist", "--no-simulate", "--quiet", "--progress", "--newline",
|
||||||
|
"--continue", "--retries", "0", "--fragment-retries", "0", "--extractor-retries", "0",
|
||||||
|
"-f", "bv*[vcodec^=avc1]+ba/b", "--merge-output-format", "mp4",
|
||||||
|
"--progress-template", `download:{"event":"download","format_id":%(info.format_id)j,"progress":%(progress)j}`,
|
||||||
|
"--progress-template", `postprocess:{"event":"postprocess","progress":%(progress)j}`,
|
||||||
|
"--print", `after_move:{"event":"after_move","filepath":%(filepath)j}`, "-o", output,
|
||||||
|
"https://tver.jp/episodes/" + job.EpisodeID}
|
||||||
|
cmd := exec.Command(r.config.YTDLPPath, args...)
|
||||||
|
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||||
|
stdout, err := cmd.StdoutPipe()
|
||||||
|
if err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
stderrPipe, err := cmd.StderrPipe()
|
||||||
|
if err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
stderr := cappedBuffer{limit: 64 << 10}
|
||||||
|
stdoutLines := scanLines(stdout, nil)
|
||||||
|
stderrLines := scanLines(stderrPipe, &stderr)
|
||||||
|
waited := make(chan error, 1)
|
||||||
|
go func() { waited <- cmd.Wait() }()
|
||||||
|
heartbeat := time.NewTicker(r.config.HeartbeatInterval)
|
||||||
|
cancelCheck := time.NewTicker(r.config.CancelInterval)
|
||||||
|
defer heartbeat.Stop()
|
||||||
|
defer cancelCheck.Stop()
|
||||||
|
_ = r.store.Heartbeat(ctx, job.ID)
|
||||||
|
var afterMove string
|
||||||
|
var pending *store.Progress
|
||||||
|
nextProgress := time.Time{}
|
||||||
|
lastPhase := ""
|
||||||
|
recordProgress := func(progress store.Progress) {
|
||||||
|
pending = &progress
|
||||||
|
now := time.Now()
|
||||||
|
if progress.Phase != lastPhase || now.After(nextProgress) {
|
||||||
|
_ = r.store.UpdateProgress(ctx, job.ID, progress)
|
||||||
|
nextProgress = now.Add(r.config.ProgressInterval)
|
||||||
|
lastPhase = progress.Phase
|
||||||
|
pending = nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
stopping := ""
|
||||||
|
var forceKill <-chan time.Time
|
||||||
|
ctxDone := ctx.Done()
|
||||||
|
waitComplete := false
|
||||||
|
var waitErr error
|
||||||
|
for {
|
||||||
|
if waitComplete && stdoutLines == nil && stderrLines == nil {
|
||||||
|
if pending != nil && stopping == "" {
|
||||||
|
updateCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
_ = r.store.UpdateProgress(updateCtx, job.ID, *pending)
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
if stopping == "canceled" {
|
||||||
|
return "", stderr.String(), errCanceled
|
||||||
|
}
|
||||||
|
if stopping == "shutdown" {
|
||||||
|
return "", stderr.String(), errShutdown
|
||||||
|
}
|
||||||
|
if waitErr != nil {
|
||||||
|
r.logger.Printf("yt-dlp ジョブ %d 失敗: %s", job.ID, redact(stderr.String()))
|
||||||
|
return "", stderr.String(), waitErr
|
||||||
|
}
|
||||||
|
path, err := pathInside(workDir, afterMove)
|
||||||
|
if err != nil {
|
||||||
|
return "", stderr.String(), err
|
||||||
|
}
|
||||||
|
return path, stderr.String(), nil
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case line, ok := <-stdoutLines:
|
||||||
|
if !ok {
|
||||||
|
stdoutLines = nil
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if path, ok := parseAfterMove(line); ok {
|
||||||
|
afterMove = path
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if progress, ok := parseProgress(line); ok {
|
||||||
|
recordProgress(progress)
|
||||||
|
}
|
||||||
|
case line, ok := <-stderrLines:
|
||||||
|
if !ok {
|
||||||
|
stderrLines = nil
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if progress, ok := parseProgress(line); ok {
|
||||||
|
recordProgress(progress)
|
||||||
|
}
|
||||||
|
case <-heartbeat.C:
|
||||||
|
_ = r.store.Heartbeat(ctx, job.ID)
|
||||||
|
case <-cancelCheck.C:
|
||||||
|
if r.canceled(job.ID) && stopping == "" {
|
||||||
|
stopping = "canceled"
|
||||||
|
if !waitComplete {
|
||||||
|
stopProcessGroup(cmd)
|
||||||
|
forceKill = time.After(2 * time.Second)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case <-ctxDone:
|
||||||
|
if stopping == "" {
|
||||||
|
stopping = "shutdown"
|
||||||
|
if !waitComplete {
|
||||||
|
stopProcessGroup(cmd)
|
||||||
|
forceKill = time.After(2 * time.Second)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ctxDone = nil
|
||||||
|
case <-forceKill:
|
||||||
|
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL)
|
||||||
|
forceKill = nil
|
||||||
|
case waitErr = <-waited:
|
||||||
|
waitComplete = true
|
||||||
|
waited = nil
|
||||||
|
forceKill = nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func stopProcessGroup(cmd *exec.Cmd) {
|
||||||
|
if cmd.Process == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_ = syscall.Kill(-cmd.Process.Pid, syscall.SIGTERM)
|
||||||
|
}
|
||||||
|
|
||||||
|
func pathInside(root, candidate string) (string, error) {
|
||||||
|
if candidate == "" {
|
||||||
|
return "", errors.New("yt-dlp did not report after_move filepath")
|
||||||
|
}
|
||||||
|
rootAbs, err := filepath.Abs(root)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
candidateAbs, err := filepath.Abs(candidate)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
rel, err := filepath.Rel(rootAbs, candidateAbs)
|
||||||
|
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
|
return "", fmt.Errorf("%w: yt-dlp filepath leaves working directory", errInvalidOutput)
|
||||||
|
}
|
||||||
|
info, err := os.Lstat(candidateAbs)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("%w: yt-dlp output file is missing: %v", errInvalidOutput, err)
|
||||||
|
}
|
||||||
|
if !info.Mode().IsRegular() {
|
||||||
|
return "", fmt.Errorf("%w: yt-dlp output is not a regular file", errInvalidOutput)
|
||||||
|
}
|
||||||
|
rootReal, err := filepath.EvalSymlinks(rootAbs)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("%w: resolve working directory: %v", errInvalidOutput, err)
|
||||||
|
}
|
||||||
|
candidateReal, err := filepath.EvalSymlinks(candidateAbs)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("%w: resolve yt-dlp output: %v", errInvalidOutput, err)
|
||||||
|
}
|
||||||
|
realRelative, err := filepath.Rel(rootReal, candidateReal)
|
||||||
|
if err != nil || realRelative == ".." || strings.HasPrefix(realRelative, ".."+string(filepath.Separator)) {
|
||||||
|
return "", fmt.Errorf("%w: yt-dlp output resolves outside working directory", errInvalidOutput)
|
||||||
|
}
|
||||||
|
return candidateAbs, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) probe(ctx context.Context, path string) error {
|
||||||
|
cmd := exec.CommandContext(ctx, r.config.FFprobePath, "-v", "error", "-show_entries",
|
||||||
|
"format=format_name", "-show_entries", "stream=codec_type,codec_name", "-of", "json", path)
|
||||||
|
output, err := cmd.Output()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("ffprobe failed: %w", err)
|
||||||
|
}
|
||||||
|
var result struct {
|
||||||
|
Format struct {
|
||||||
|
FormatName string `json:"format_name"`
|
||||||
|
} `json:"format"`
|
||||||
|
Streams []struct {
|
||||||
|
CodecType string `json:"codec_type"`
|
||||||
|
CodecName string `json:"codec_name"`
|
||||||
|
} `json:"streams"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(output, &result); err != nil {
|
||||||
|
return fmt.Errorf("ffprobe output is not JSON: %w", err)
|
||||||
|
}
|
||||||
|
video, audio := false, false
|
||||||
|
for _, stream := range result.Streams {
|
||||||
|
video = video || stream.CodecType == "video" && stream.CodecName == "h264"
|
||||||
|
audio = audio || stream.CodecType == "audio" && stream.CodecName == "aac"
|
||||||
|
}
|
||||||
|
if !strings.Contains(result.Format.FormatName, "mp4") || !video || !audio {
|
||||||
|
return fmt.Errorf("expected MP4 with H.264 and AAC, got format=%s video=%t audio=%t", result.Format.FormatName, video, audio)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) canceled(id int64) bool {
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
job, err := r.store.GetJob(ctx, id)
|
||||||
|
return err == nil && job.Status == "canceled"
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) failIfRunning(id int64, kind, message string) {
|
||||||
|
if r.canceled(id) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
r.fail(id, kind, message)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) fail(id int64, kind, message string) {
|
||||||
|
if len(message) > 8192 {
|
||||||
|
message = message[:8192]
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if err := r.store.Fail(ctx, id, kind, message); err != nil {
|
||||||
|
r.logger.Printf("ジョブ %d の失敗記録に失敗: %v", id, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *Runner) requeue(id int64) {
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if err := r.store.RequeueRunning(ctx, id); err != nil && !errors.Is(err, store.ErrInvalidTransition) {
|
||||||
|
r.logger.Printf("ジョブ %d の待機中への復帰に失敗: %v", id, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func classifyFailure(message string) string {
|
||||||
|
lower := strings.ToLower(message)
|
||||||
|
for _, test := range []struct {
|
||||||
|
kind string
|
||||||
|
patterns []string
|
||||||
|
}{
|
||||||
|
{"drm", []string{"drm", "digital rights management", "protected by"}},
|
||||||
|
{"geo", []string{"geo-restricted", "not available in your country", "not available from your location", "geo restriction", "region restriction", "geographic restriction"}},
|
||||||
|
{"network", []string{"network", "connection", "timed out", "temporary failure", "unable to download", "proxy error", "http error 5"}},
|
||||||
|
{"unavailable", []string{"media_not_found", "video unavailable", "not available", "no longer available", "http error 404", "episode does not exist"}},
|
||||||
|
{"extract", []string{"extractor error", "unable to extract", "unsupported url", "no video formats", "after_move", "output file is missing"}},
|
||||||
|
} {
|
||||||
|
for _, pattern := range test.patterns {
|
||||||
|
if strings.Contains(lower, pattern) {
|
||||||
|
return test.kind
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "unknown"
|
||||||
|
}
|
||||||
|
|
||||||
|
func redact(message string) string {
|
||||||
|
message = headerSecretRE.ReplaceAllString(message, "$1$2[REDACTED]")
|
||||||
|
return strings.TrimSpace(paramSecretRE.ReplaceAllString(message, "$1$2[REDACTED]"))
|
||||||
|
}
|
||||||
@@ -0,0 +1,678 @@
|
|||||||
|
package worker
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/media"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeStore struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
job store.Job
|
||||||
|
progress []store.Progress
|
||||||
|
heartbeats int
|
||||||
|
completed bool
|
||||||
|
completePath string
|
||||||
|
completeSize int64
|
||||||
|
failedKind, failedMessage string
|
||||||
|
requeued bool
|
||||||
|
claimed bool
|
||||||
|
}
|
||||||
|
|
||||||
|
type queueStore struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
jobs []store.Job
|
||||||
|
claimCounts map[int64]int
|
||||||
|
active int
|
||||||
|
maxActive int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) ClaimNext(context.Context) (store.Job, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
for i := range f.jobs {
|
||||||
|
if f.jobs[i].Status != "queued" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
f.jobs[i].Status = "running"
|
||||||
|
f.claimCounts[f.jobs[i].ID]++
|
||||||
|
f.active++
|
||||||
|
if f.active > f.maxActive {
|
||||||
|
f.maxActive = f.active
|
||||||
|
}
|
||||||
|
return f.jobs[i], nil
|
||||||
|
}
|
||||||
|
return store.Job{}, pgx.ErrNoRows
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) UpdateProgress(_ context.Context, id int64, _ store.Progress) error {
|
||||||
|
return f.requireRunning(id)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) Heartbeat(_ context.Context, id int64) error {
|
||||||
|
return f.requireRunning(id)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) requireRunning(id int64) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
for i := range f.jobs {
|
||||||
|
if f.jobs[i].ID == id && f.jobs[i].Status == "running" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) GetJob(_ context.Context, id int64) (store.Job, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
for _, job := range f.jobs {
|
||||||
|
if job.ID == id {
|
||||||
|
return job, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.Job{}, pgx.ErrNoRows
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) Complete(_ context.Context, id int64, _ string, _ int64) error {
|
||||||
|
return f.finish(id, "succeeded")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) Fail(_ context.Context, id int64, _, _ string) error {
|
||||||
|
return f.finish(id, "failed")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) RequeueRunning(_ context.Context, id int64) error {
|
||||||
|
return f.finish(id, "queued")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) finish(id int64, status string) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
for i := range f.jobs {
|
||||||
|
if f.jobs[i].ID == id && f.jobs[i].Status == "running" {
|
||||||
|
f.jobs[i].Status = status
|
||||||
|
f.active--
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *queueStore) state() ([]store.Job, map[int64]int, int) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
jobs := append([]store.Job(nil), f.jobs...)
|
||||||
|
counts := make(map[int64]int, len(f.claimCounts))
|
||||||
|
for id, count := range f.claimCounts {
|
||||||
|
counts[id] = count
|
||||||
|
}
|
||||||
|
return jobs, counts, f.maxActive
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) ClaimNext(context.Context) (store.Job, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
if f.claimed || f.job.Status != "queued" {
|
||||||
|
return store.Job{}, pgx.ErrNoRows
|
||||||
|
}
|
||||||
|
f.claimed = true
|
||||||
|
f.job.Status = "running"
|
||||||
|
return f.job, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) UpdateProgress(_ context.Context, _ int64, p store.Progress) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
if f.job.Status != "running" {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
f.progress = append(f.progress, p)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) Heartbeat(context.Context, int64) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
if f.job.Status != "running" {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
f.heartbeats++
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) GetJob(context.Context, int64) (store.Job, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
return f.job, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) Complete(_ context.Context, _ int64, path string, size int64) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
if f.job.Status != "running" {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
f.job.Status = "succeeded"
|
||||||
|
f.completed, f.completePath, f.completeSize = true, path, size
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) Fail(_ context.Context, _ int64, kind, message string) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
if f.job.Status != "running" {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
f.job.Status = "failed"
|
||||||
|
f.failedKind, f.failedMessage = kind, message
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) RequeueRunning(context.Context, int64) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
if f.job.Status != "running" {
|
||||||
|
return store.ErrInvalidTransition
|
||||||
|
}
|
||||||
|
f.job.Status = "queued"
|
||||||
|
f.requeued = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) setStatus(status string) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.job.Status = status
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeStore) heartbeatCount() int {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
return f.heartbeats
|
||||||
|
}
|
||||||
|
|
||||||
|
func testRunner(t *testing.T, jobs *fakeStore, videoDir string) *Runner {
|
||||||
|
t.Helper()
|
||||||
|
ytdlp := filepath.Join("testdata", "fake-yt-dlp.sh")
|
||||||
|
ffprobe := filepath.Join("testdata", "fake-ffprobe.sh")
|
||||||
|
for _, path := range []string{ytdlp, ffprobe} {
|
||||||
|
info, err := os.Stat(path)
|
||||||
|
if err != nil || info.Mode().Perm()&0o111 == 0 {
|
||||||
|
t.Fatalf("偽コマンドが実行可能ではありません: %s: %v", path, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
runner, err := New(jobs, Config{VideoDir: videoDir, YTDLPPath: ytdlp, FFprobePath: ffprobe,
|
||||||
|
Concurrency: 1, PollInterval: 5 * time.Millisecond, ProgressInterval: time.Millisecond,
|
||||||
|
HeartbeatInterval: 10 * time.Millisecond, CancelInterval: 5 * time.Millisecond}, log.New(io.Discard, "", 0))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return runner
|
||||||
|
}
|
||||||
|
|
||||||
|
func runningJob() store.Job {
|
||||||
|
return store.Job{ID: 7, EpisodeID: "ep123", SeriesTitle: "番組名", Title: "第1話", Status: "running"}
|
||||||
|
}
|
||||||
|
|
||||||
|
func relativeVideoPath(t *testing.T, seriesTitle, episodeTitle, episodeID string) string {
|
||||||
|
t.Helper()
|
||||||
|
path, err := media.RelativeVideoPath(seriesTitle, episodeTitle, episodeID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProcessJobDownloadsProbesMovesAndCleans(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "success")
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", "valid")
|
||||||
|
dir := t.TempDir()
|
||||||
|
argsFile := filepath.Join(dir, "args")
|
||||||
|
t.Setenv("FAKE_ARGS_FILE", argsFile)
|
||||||
|
jobs := &fakeStore{job: runningJob()}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
|
||||||
|
want := filepath.ToSlash(relativeVideoPath(t, "番組名", "第1話", "ep123"))
|
||||||
|
if !jobs.completed || jobs.completePath != want || jobs.completeSize == 0 {
|
||||||
|
t.Fatalf("complete: done=%v path=%q size=%d", jobs.completed, jobs.completePath, jobs.completeSize)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, filepath.FromSlash(want))); err != nil {
|
||||||
|
t.Fatalf("保存先: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("作業ディレクトリが残った: %v", err)
|
||||||
|
}
|
||||||
|
args, err := os.ReadFile(argsFile)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
argumentText := string(args)
|
||||||
|
for _, want := range []string{"--newline", "--no-playlist", "--no-simulate", "--quiet", "--progress", "--continue", "--merge-output-format", "bv*[vcodec^=avc1]+ba/b", `download:{"event":"download","format_id":%(info.format_id)j,"progress":%(progress)j}`, `postprocess:{"event":"postprocess","progress":%(progress)j}`, `after_move:{"event":"after_move","filepath":%(filepath)j}`, filepath.Join(dir, ".incoming", "7", "%(id)s.%(ext)s")} {
|
||||||
|
if !strings.Contains(argumentText, want) {
|
||||||
|
t.Errorf("yt-dlp 引数に %q がない: %s", want, argumentText)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(argumentText, "番組名") || strings.Contains(argumentText, "第1話") {
|
||||||
|
t.Errorf("出力テンプレートに番組情報が入った: %s", argumentText)
|
||||||
|
}
|
||||||
|
phases := make(map[string]bool)
|
||||||
|
for _, progress := range jobs.progress {
|
||||||
|
phases[progress.Phase] = true
|
||||||
|
}
|
||||||
|
for _, phase := range []string{"video", "audio", "merging", "moving"} {
|
||||||
|
if !phases[phase] {
|
||||||
|
t.Errorf("進捗段階 %s がない: %+v", phase, jobs.progress)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if jobs.heartbeats < 2 {
|
||||||
|
t.Errorf("heartbeat が少ない: %d", jobs.heartbeats)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProcessJobRejectsLongEpisodeIDAndCleans(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
argsFile := filepath.Join(dir, "yt-dlp-args")
|
||||||
|
t.Setenv("FAKE_ARGS_FILE", argsFile)
|
||||||
|
workDir := filepath.Join(dir, ".incoming", "7")
|
||||||
|
if err := os.MkdirAll(workDir, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(workDir, "stale"), []byte("stale"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
job := runningJob()
|
||||||
|
job.EpisodeID = strings.Repeat("i", 65)
|
||||||
|
jobs := &fakeStore{job: job}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
|
||||||
|
if jobs.completed || jobs.failedKind != "invalid_output" || !strings.Contains(jobs.failedMessage, "64バイト以下") {
|
||||||
|
t.Fatalf("長いIDの失敗: kind=%q message=%q completed=%v", jobs.failedKind, jobs.failedMessage, jobs.completed)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(argsFile); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("yt-dlp が起動された: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(workDir); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("作業ディレクトリが残った: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProcessJobKeepsPercentNamesOutOfYTDLPTemplate(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "success")
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", "valid")
|
||||||
|
dir := t.TempDir()
|
||||||
|
argsFile := filepath.Join(dir, "args")
|
||||||
|
t.Setenv("FAKE_ARGS_FILE", argsFile)
|
||||||
|
job := runningJob()
|
||||||
|
job.SeriesTitle = "100%番組"
|
||||||
|
job.Title = "第%1話"
|
||||||
|
jobs := &fakeStore{job: job}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
|
||||||
|
if !jobs.completed {
|
||||||
|
t.Fatalf("%% を含む番組を完了できなかった: %+v", jobs)
|
||||||
|
}
|
||||||
|
args, err := os.ReadFile(argsFile)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(args), job.SeriesTitle) || strings.Contains(string(args), job.Title) {
|
||||||
|
t.Fatalf("番組情報が yt-dlp 引数に入った: %s", args)
|
||||||
|
}
|
||||||
|
want := filepath.ToSlash(relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID))
|
||||||
|
if jobs.completePath != want {
|
||||||
|
t.Fatalf("保存先=%q want=%q", jobs.completePath, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProcessJobReplacesExistingDestination(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "success")
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", "valid")
|
||||||
|
dir := t.TempDir()
|
||||||
|
job := runningJob()
|
||||||
|
relative := relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID)
|
||||||
|
destination := filepath.Join(dir, relative)
|
||||||
|
if err := os.MkdirAll(filepath.Dir(destination), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(destination, []byte("stale file from previous attempt"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeStore{job: job}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
|
||||||
|
got, err := os.ReadFile(destination)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !jobs.completed || string(got) != "fake mp4" || jobs.completeSize != int64(len(got)) {
|
||||||
|
t.Fatalf("既存ファイルの置換: completed=%v content=%q size=%d", jobs.completed, got, jobs.completeSize)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProcessJobRejectsAfterMoveSymlinkOutsideWorkDirectory(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "symlink")
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", "valid")
|
||||||
|
dir := t.TempDir()
|
||||||
|
outside := filepath.Join(t.TempDir(), "outside.mp4")
|
||||||
|
if err := os.WriteFile(outside, []byte("outside video"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("FAKE_OUTSIDE_FILE", outside)
|
||||||
|
jobs := &fakeStore{job: runningJob()}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
|
||||||
|
if jobs.completed || jobs.failedKind != "invalid_output" {
|
||||||
|
t.Fatalf("作業ディレクトリ外を指すシンボリックリンクを受理した: %+v", jobs)
|
||||||
|
}
|
||||||
|
if got, err := os.ReadFile(outside); err != nil || string(got) != "outside video" {
|
||||||
|
t.Fatalf("作業ディレクトリ外のファイルを変更した: %q %v", got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPathInsideSymlinkDerivedCases(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
insideDir := filepath.Join(root, "real")
|
||||||
|
if err := os.Mkdir(insideDir, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
insideFile := filepath.Join(insideDir, "video.mp4")
|
||||||
|
if err := os.WriteFile(insideFile, []byte("video"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
outsideDir := t.TempDir()
|
||||||
|
outsideFile := filepath.Join(outsideDir, "outside.mp4")
|
||||||
|
if err := os.WriteFile(outsideFile, []byte("outside"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
internalFileLink := filepath.Join(root, "internal-file-link.mp4")
|
||||||
|
internalDirLink := filepath.Join(root, "internal-dir-link")
|
||||||
|
externalDirLink := filepath.Join(root, "external-dir-link")
|
||||||
|
rootLink := filepath.Join(t.TempDir(), "root-link")
|
||||||
|
for link, target := range map[string]string{
|
||||||
|
internalFileLink: insideFile,
|
||||||
|
internalDirLink: insideDir,
|
||||||
|
externalDirLink: outsideDir,
|
||||||
|
rootLink: root,
|
||||||
|
} {
|
||||||
|
if err := os.Symlink(target, link); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range []struct {
|
||||||
|
name, testRoot, candidate string
|
||||||
|
wantOK bool
|
||||||
|
}{
|
||||||
|
{"通常ファイル", root, insideFile, true},
|
||||||
|
{"作業ディレクトリ内を指すファイルリンク", root, internalFileLink, false},
|
||||||
|
{"after_moveがディレクトリ", root, insideDir, false},
|
||||||
|
{"作業ディレクトリ内を指すディレクトリリンク配下", root, filepath.Join(internalDirLink, "video.mp4"), true},
|
||||||
|
{"作業ディレクトリ外を指すディレクトリリンク配下", root, filepath.Join(externalDirLink, "outside.mp4"), false},
|
||||||
|
{"作業ディレクトリ自体がリンク", rootLink, filepath.Join(rootLink, "real", "video.mp4"), true},
|
||||||
|
} {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
got, err := pathInside(tt.testRoot, tt.candidate)
|
||||||
|
if tt.wantOK {
|
||||||
|
want, absErr := filepath.Abs(tt.candidate)
|
||||||
|
if absErr != nil {
|
||||||
|
t.Fatal(absErr)
|
||||||
|
}
|
||||||
|
if err != nil || got != want {
|
||||||
|
t.Fatalf("pathInside()=(%q, %v) want=(%q, nil)", got, err, want)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !errors.Is(err, errInvalidOutput) || got != "" {
|
||||||
|
t.Fatalf("pathInside()=(%q, %v), invalid_output を期待", got, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProcessJobDestinationDirectoryFailureCleansIncoming(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "success")
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", "valid")
|
||||||
|
dir := t.TempDir()
|
||||||
|
job := runningJob()
|
||||||
|
seriesDir := filepath.Join(dir, filepath.Dir(relativeVideoPath(t, job.SeriesTitle, job.Title, job.EpisodeID)))
|
||||||
|
if err := os.WriteFile(seriesDir, []byte("blocks directory"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
jobs := &fakeStore{job: job}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), job)
|
||||||
|
if jobs.completed || jobs.failedKind != "unknown" {
|
||||||
|
t.Fatalf("保存先作成失敗: %+v", jobs)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("作業ディレクトリが残った: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFailureClassificationAndRedaction(t *testing.T) {
|
||||||
|
for _, tt := range []struct{ mode, kind string }{
|
||||||
|
{"drm", "drm"}, {"geo", "geo"}, {"network", "network"},
|
||||||
|
{"unavailable", "unavailable"}, {"extract", "extract"}, {"unknown", "unknown"},
|
||||||
|
} {
|
||||||
|
t.Run(tt.mode, func(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", tt.mode)
|
||||||
|
dir := t.TempDir()
|
||||||
|
jobs := &fakeStore{job: runningJob()}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
|
||||||
|
if jobs.failedKind != tt.kind || jobs.completed || strings.Contains(jobs.failedMessage, "secret-value") {
|
||||||
|
t.Fatalf("failure: kind=%q message=%q completed=%v", jobs.failedKind, jobs.failedMessage, jobs.completed)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("作業ディレクトリが残った: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestInvalidProbeFailsAndCleans(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "success")
|
||||||
|
for _, mode := range []string{"bad-container", "bad-video", "bad-audio", "non-json", "failure"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", mode)
|
||||||
|
dir := t.TempDir()
|
||||||
|
jobs := &fakeStore{job: runningJob()}
|
||||||
|
testRunner(t, jobs, dir).ProcessJob(context.Background(), jobs.job)
|
||||||
|
if jobs.failedKind != "invalid_output" || jobs.completed {
|
||||||
|
t.Fatalf("ffprobe 不合格: kind=%q complete=%v", jobs.failedKind, jobs.completed)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("作業ディレクトリが残った: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProgressIsThrottledButHeartbeatContinues(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "burst")
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", "valid")
|
||||||
|
dir := t.TempDir()
|
||||||
|
jobs := &fakeStore{job: runningJob()}
|
||||||
|
runner := testRunner(t, jobs, dir)
|
||||||
|
runner.config.ProgressInterval = 50 * time.Millisecond
|
||||||
|
runner.ProcessJob(context.Background(), jobs.job)
|
||||||
|
downloadUpdates := 0
|
||||||
|
for _, progress := range jobs.progress {
|
||||||
|
if progress.Phase == "video" {
|
||||||
|
downloadUpdates++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if downloadUpdates > 2 || downloadUpdates == 0 {
|
||||||
|
t.Fatalf("進捗の間引き: %d updates (%+v)", downloadUpdates, jobs.progress)
|
||||||
|
}
|
||||||
|
if jobs.heartbeats < 5 {
|
||||||
|
t.Fatalf("進捗が止まった間の heartbeat: %d", jobs.heartbeats)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunConcurrencyTwoDoesNotDuplicateAndIsolatesFailure(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "mixed")
|
||||||
|
t.Setenv("FAKE_FFPROBE_MODE", "valid")
|
||||||
|
dir := t.TempDir()
|
||||||
|
jobs := &queueStore{
|
||||||
|
jobs: []store.Job{
|
||||||
|
{ID: 7, EpisodeID: "ep-ok", SeriesTitle: "成功番組", Title: "成功回", Status: "queued"},
|
||||||
|
{ID: 8, EpisodeID: "ep-fail", SeriesTitle: "失敗番組", Title: "失敗回", Status: "queued"},
|
||||||
|
},
|
||||||
|
claimCounts: make(map[int64]int),
|
||||||
|
}
|
||||||
|
runner, err := New(jobs, Config{VideoDir: dir,
|
||||||
|
YTDLPPath: filepath.Join("testdata", "fake-yt-dlp.sh"), FFprobePath: filepath.Join("testdata", "fake-ffprobe.sh"),
|
||||||
|
Concurrency: 2, PollInterval: 5 * time.Millisecond, ProgressInterval: time.Millisecond,
|
||||||
|
HeartbeatInterval: 10 * time.Millisecond, CancelInterval: 5 * time.Millisecond}, log.New(io.Discard, "", 0))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() { done <- runner.Run(ctx) }()
|
||||||
|
deadline := time.Now().Add(3 * time.Second)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
state, _, _ := jobs.state()
|
||||||
|
if state[0].Status == "succeeded" && state[1].Status == "failed" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Millisecond)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
if err := <-done; err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
state, counts, maxActive := jobs.state()
|
||||||
|
if state[0].Status != "succeeded" || state[1].Status != "failed" {
|
||||||
|
t.Fatalf("片方の失敗がもう片方へ波及した: %+v", state)
|
||||||
|
}
|
||||||
|
if counts[7] != 1 || counts[8] != 1 {
|
||||||
|
t.Fatalf("ジョブを二重取得した: %+v", counts)
|
||||||
|
}
|
||||||
|
if maxActive != 2 {
|
||||||
|
t.Fatalf("並行実行数の最大値=%d want=2", maxActive)
|
||||||
|
}
|
||||||
|
for _, id := range []int64{7, 8} {
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, ".incoming", strconv.FormatInt(id, 10))); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Errorf("ジョブ %d の作業ディレクトリが残った: %v", id, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCancelStopsProcessGroupAndLeavesCanceled(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "hang-drm")
|
||||||
|
dir := t.TempDir()
|
||||||
|
ready := filepath.Join(dir, "ready")
|
||||||
|
terminated := filepath.Join(dir, "terminated")
|
||||||
|
t.Setenv("FAKE_READY_FILE", ready)
|
||||||
|
t.Setenv("FAKE_TERM_FILE", terminated)
|
||||||
|
jobs := &fakeStore{job: runningJob()}
|
||||||
|
runner := testRunner(t, jobs, dir)
|
||||||
|
job := jobs.job
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { runner.ProcessJob(context.Background(), job); close(done) }()
|
||||||
|
waitForFile(t, ready)
|
||||||
|
waitForHeartbeatCount(t, jobs, 2)
|
||||||
|
jobs.setStatus("canceled")
|
||||||
|
waitDone(t, done)
|
||||||
|
if jobs.failedKind != "" || jobs.requeued || jobs.completed || jobs.job.Status != "canceled" {
|
||||||
|
t.Fatalf("取り消し後: %+v", jobs)
|
||||||
|
}
|
||||||
|
waitForFile(t, terminated)
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("作業ディレクトリが残った: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestShutdownStopsProcessAndRequeues(t *testing.T) {
|
||||||
|
t.Setenv("FAKE_YTDLP_MODE", "hang")
|
||||||
|
dir := t.TempDir()
|
||||||
|
ready := filepath.Join(dir, "ready")
|
||||||
|
t.Setenv("FAKE_READY_FILE", ready)
|
||||||
|
jobs := &fakeStore{job: runningJob()}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
runner := testRunner(t, jobs, dir)
|
||||||
|
job := jobs.job
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { runner.ProcessJob(ctx, job); close(done) }()
|
||||||
|
waitForFile(t, ready)
|
||||||
|
cancel()
|
||||||
|
waitDone(t, done)
|
||||||
|
if !jobs.requeued || jobs.job.Status != "queued" || jobs.failedKind != "" {
|
||||||
|
t.Fatalf("終了時の復帰: %+v", jobs)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, ".incoming", "7")); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatalf("作業ディレクトリが残った: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunnerRejectsConcurrencyOutsideRange(t *testing.T) {
|
||||||
|
for _, concurrency := range []int{-1, 3} {
|
||||||
|
if _, err := New(&fakeStore{}, Config{Concurrency: concurrency}, nil); err == nil {
|
||||||
|
t.Errorf("concurrency=%d を許した", concurrency)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFailureClassificationOrderAndRedaction(t *testing.T) {
|
||||||
|
for _, tt := range []struct{ message, want string }{
|
||||||
|
{"DRM geo-restricted connection video unavailable unable to extract", "drm"},
|
||||||
|
{"geo-restricted connection video unavailable unable to extract", "geo"},
|
||||||
|
{"connection video unavailable unable to extract", "network"},
|
||||||
|
{"video unavailable unable to extract", "unavailable"},
|
||||||
|
{"unable to extract", "extract"},
|
||||||
|
{"different error", "unknown"},
|
||||||
|
} {
|
||||||
|
if got := classifyFailure(tt.message); got != tt.want {
|
||||||
|
t.Errorf("classifyFailure(%q)=%q want=%q", tt.message, got, tt.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
redacted := redact("Authorization: Bearer bearer-secret\nCookie: session=session-secret\ntoken=token-secret platform_uid=uid-secret")
|
||||||
|
for _, secret := range []string{"bearer-secret", "session-secret", "token-secret", "uid-secret"} {
|
||||||
|
if strings.Contains(redacted, secret) {
|
||||||
|
t.Errorf("秘密が残った: %q", redacted)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func waitForFile(t *testing.T, path string) {
|
||||||
|
t.Helper()
|
||||||
|
deadline := time.Now().Add(2 * time.Second)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
if _, err := os.Stat(path); err == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Millisecond)
|
||||||
|
}
|
||||||
|
t.Fatalf("ファイル待ちがタイムアウト: %s", path)
|
||||||
|
}
|
||||||
|
|
||||||
|
func waitDone(t *testing.T, done <-chan struct{}) {
|
||||||
|
t.Helper()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(3 * time.Second):
|
||||||
|
t.Fatal("worker の終了待ちがタイムアウト")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func waitForHeartbeatCount(t *testing.T, jobs *fakeStore, want int) {
|
||||||
|
t.Helper()
|
||||||
|
deadline := time.Now().Add(2 * time.Second)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
if jobs.heartbeatCount() >= want {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Millisecond)
|
||||||
|
}
|
||||||
|
t.Fatalf("heartbeat 待ちがタイムアウト: got=%d want=%d", jobs.heartbeatCount(), want)
|
||||||
|
}
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
package worker
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
type SubscriptionStore interface {
|
||||||
|
ListDueSubscriptions(context.Context, time.Time) ([]store.Subscription, error)
|
||||||
|
CompleteSubscriptionCheck(context.Context, string, []store.Episode, int, time.Time) (int, error)
|
||||||
|
RecordSubscriptionCheckFailure(context.Context, string, time.Time, string) error
|
||||||
|
}
|
||||||
|
|
||||||
|
type SeriesFetcher interface {
|
||||||
|
FetchSeriesEpisodes(context.Context, string) ([]tver.Episode, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
type SubscriptionSchedulerConfig struct {
|
||||||
|
ScanInterval time.Duration
|
||||||
|
DueAfter time.Duration
|
||||||
|
SeriesInterval time.Duration
|
||||||
|
PreviewMaxSeconds int
|
||||||
|
Now func() time.Time
|
||||||
|
After func(time.Duration) <-chan time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type SubscriptionScheduler struct {
|
||||||
|
store SubscriptionStore
|
||||||
|
fetcher SeriesFetcher
|
||||||
|
config SubscriptionSchedulerConfig
|
||||||
|
logger *log.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewSubscriptionScheduler(subscriptions SubscriptionStore, fetcher SeriesFetcher, config SubscriptionSchedulerConfig, logger *log.Logger) (*SubscriptionScheduler, error) {
|
||||||
|
if subscriptions == nil || fetcher == nil {
|
||||||
|
return nil, errors.New("subscription store and fetcher are required")
|
||||||
|
}
|
||||||
|
if config.ScanInterval <= 0 {
|
||||||
|
config.ScanInterval = time.Hour
|
||||||
|
}
|
||||||
|
if config.DueAfter <= 0 {
|
||||||
|
config.DueAfter = 24 * time.Hour
|
||||||
|
}
|
||||||
|
if config.SeriesInterval <= 0 {
|
||||||
|
config.SeriesInterval = 3 * time.Second
|
||||||
|
}
|
||||||
|
if config.PreviewMaxSeconds == 0 {
|
||||||
|
config.PreviewMaxSeconds = tver.PreviewMaxSeconds
|
||||||
|
}
|
||||||
|
if config.PreviewMaxSeconds < 0 {
|
||||||
|
return nil, errors.New("preview threshold must not be negative")
|
||||||
|
}
|
||||||
|
if config.Now == nil {
|
||||||
|
config.Now = time.Now
|
||||||
|
}
|
||||||
|
if config.After == nil {
|
||||||
|
config.After = time.After
|
||||||
|
}
|
||||||
|
if logger == nil {
|
||||||
|
logger = log.Default()
|
||||||
|
}
|
||||||
|
return &SubscriptionScheduler{store: subscriptions, fetcher: fetcher, config: config, logger: logger}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run は起動直後に確認し、その後は ScanInterval ごとに繰り返す。
|
||||||
|
func (s *SubscriptionScheduler) Run(ctx context.Context) {
|
||||||
|
for {
|
||||||
|
s.checkDue(ctx)
|
||||||
|
if !s.wait(ctx, s.config.ScanInterval) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *SubscriptionScheduler) checkDue(ctx context.Context) {
|
||||||
|
now := s.config.Now()
|
||||||
|
subscriptions, err := s.store.ListDueSubscriptions(ctx, now.Add(-s.config.DueAfter))
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
s.logger.Printf("確認対象の登録番組を取得できません: %v", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for index, subscription := range subscriptions {
|
||||||
|
if index > 0 && !s.wait(ctx, s.config.SeriesInterval) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
episodes, err := s.fetcher.FetchSeriesEpisodes(ctx, subscription.SeriesID)
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
message := shortError(err)
|
||||||
|
if recordErr := s.store.RecordSubscriptionCheckFailure(ctx, subscription.SeriesID, s.config.Now(), message); recordErr != nil &&
|
||||||
|
!errors.Is(recordErr, store.ErrInvalidTransition) {
|
||||||
|
s.logger.Printf("登録番組 %s の確認エラーを記録できません: %v", subscription.SeriesID, recordErr)
|
||||||
|
}
|
||||||
|
s.logger.Printf("登録番組 %s の公開中エピソードを取得できません: %s", subscription.SeriesID, message)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
items := make([]store.Episode, 0, len(episodes))
|
||||||
|
for _, episode := range episodes {
|
||||||
|
items = append(items, subscriptionEpisode(episode))
|
||||||
|
}
|
||||||
|
queued, err := s.store.CompleteSubscriptionCheck(ctx, subscription.SeriesID, items,
|
||||||
|
s.config.PreviewMaxSeconds, s.config.Now())
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !errors.Is(err, store.ErrSubscriptionPaused) && !errors.Is(err, store.ErrInvalidTransition) {
|
||||||
|
s.logger.Printf("登録番組 %s の確認結果を保存できません: %v", subscription.SeriesID, err)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
s.logger.Printf("登録番組 %s を確認しました（新規ジョブ=%d）", subscription.SeriesID, queued)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *SubscriptionScheduler) wait(ctx context.Context, duration time.Duration) bool {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return false
|
||||||
|
case <-s.config.After(duration):
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func subscriptionEpisode(episode tver.Episode) store.Episode {
|
||||||
|
item := store.Episode{ID: episode.ID, SeriesID: episode.SeriesID, SeriesTitle: episode.SeriesTitle,
|
||||||
|
Title: episode.Title, BroadcasterName: episode.BroadcasterName,
|
||||||
|
DurationSeconds: int(episode.Duration), ThumbnailPath: episode.ThumbnailPath}
|
||||||
|
if episode.EndAt > 0 {
|
||||||
|
end := time.Unix(episode.EndAt, 0)
|
||||||
|
item.EndAt = &end
|
||||||
|
}
|
||||||
|
return item
|
||||||
|
}
|
||||||
|
|
||||||
|
func shortError(err error) string {
|
||||||
|
message := redact(err.Error())
|
||||||
|
message = strings.Join(strings.Fields(message), " ")
|
||||||
|
const maxRunes = 512
|
||||||
|
if utf8.RuneCountInString(message) <= maxRunes {
|
||||||
|
return message
|
||||||
|
}
|
||||||
|
runes := []rune(message)
|
||||||
|
return string(runes[:maxRunes])
|
||||||
|
}
|
||||||
@@ -0,0 +1,253 @@
|
|||||||
|
package worker
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/store"
|
||||||
|
"gitea.kazu634.com/kazu634/tver/internal/tver"
|
||||||
|
)
|
||||||
|
|
||||||
|
type schedulerCompletion struct {
|
||||||
|
seriesID string
|
||||||
|
episodes []store.Episode
|
||||||
|
limit int
|
||||||
|
at time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type schedulerFailure struct {
|
||||||
|
seriesID, message string
|
||||||
|
at time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
type fakeSubscriptionStore struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
due []store.Subscription
|
||||||
|
cutoffs []time.Time
|
||||||
|
completions []schedulerCompletion
|
||||||
|
failures []schedulerFailure
|
||||||
|
listErr error
|
||||||
|
completeErr error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSubscriptionStore) ListDueSubscriptions(_ context.Context, cutoff time.Time) ([]store.Subscription, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.cutoffs = append(f.cutoffs, cutoff)
|
||||||
|
return append([]store.Subscription(nil), f.due...), f.listErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSubscriptionStore) CompleteSubscriptionCheck(_ context.Context, seriesID string, episodes []store.Episode, limit int, at time.Time) (int, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.completions = append(f.completions, schedulerCompletion{seriesID, append([]store.Episode(nil), episodes...), limit, at})
|
||||||
|
return len(episodes), f.completeErr
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSubscriptionStore) RecordSubscriptionCheckFailure(_ context.Context, seriesID string, at time.Time, message string) error {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
f.failures = append(f.failures, schedulerFailure{seriesID, message, at})
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type fakeSeriesFetcher struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
calls []string
|
||||||
|
episodes map[string][]tver.Episode
|
||||||
|
errors map[string]error
|
||||||
|
block bool
|
||||||
|
started chan string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeSeriesFetcher) FetchSeriesEpisodes(ctx context.Context, seriesID string) ([]tver.Episode, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
f.calls = append(f.calls, seriesID)
|
||||||
|
block := f.block
|
||||||
|
started := f.started
|
||||||
|
episodes := append([]tver.Episode(nil), f.episodes[seriesID]...)
|
||||||
|
err := f.errors[seriesID]
|
||||||
|
f.mu.Unlock()
|
||||||
|
if started != nil {
|
||||||
|
started <- seriesID
|
||||||
|
}
|
||||||
|
if block {
|
||||||
|
<-ctx.Done()
|
||||||
|
return nil, ctx.Err()
|
||||||
|
}
|
||||||
|
return episodes, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionSchedulerChecksImmediatelyAndMapsEpisodes(t *testing.T) {
|
||||||
|
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||||
|
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}, {SeriesID: "two"}}}
|
||||||
|
fetcher := &fakeSeriesFetcher{episodes: map[string][]tver.Episode{
|
||||||
|
"one": {{ID: "ep1", SeriesID: "one", SeriesTitle: "番組", Title: "回", BroadcasterName: "局", Duration: 180, EndAt: 1_800_000_000, ThumbnailPath: "/image"}},
|
||||||
|
"two": {{ID: "ep2", SeriesID: "two", Duration: 181}},
|
||||||
|
}, errors: map[string]error{}}
|
||||||
|
var waits []time.Duration
|
||||||
|
after := func(duration time.Duration) <-chan time.Time {
|
||||||
|
waits = append(waits, duration)
|
||||||
|
ch := make(chan time.Time, 1)
|
||||||
|
ch <- now
|
||||||
|
return ch
|
||||||
|
}
|
||||||
|
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
|
||||||
|
DueAfter: 24 * time.Hour, SeriesInterval: 4 * time.Second,
|
||||||
|
Now: func() time.Time { return now }, After: after,
|
||||||
|
}, log.New(&bytes.Buffer{}, "", 0))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
scheduler.checkDue(context.Background())
|
||||||
|
if len(subscriptions.cutoffs) != 1 || !subscriptions.cutoffs[0].Equal(now.Add(-24*time.Hour)) {
|
||||||
|
t.Fatalf("確認対象日時: %v", subscriptions.cutoffs)
|
||||||
|
}
|
||||||
|
if len(fetcher.calls) != 2 || fetcher.calls[0] != "one" || fetcher.calls[1] != "two" || len(waits) != 1 || waits[0] != 4*time.Second {
|
||||||
|
t.Fatalf("順次確認と間隔: calls=%v waits=%v", fetcher.calls, waits)
|
||||||
|
}
|
||||||
|
if len(subscriptions.completions) != 2 || subscriptions.completions[0].limit != tver.PreviewMaxSeconds || !subscriptions.completions[0].at.Equal(now) {
|
||||||
|
t.Fatalf("確認完了: %+v", subscriptions.completions)
|
||||||
|
}
|
||||||
|
got := subscriptions.completions[0].episodes[0]
|
||||||
|
if got.ID != "ep1" || got.DurationSeconds != 180 || got.EndAt == nil || got.EndAt.Unix() != 1_800_000_000 {
|
||||||
|
t.Fatalf("エピソード変換: %+v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestShortErrorLimitsUnicodeByCharacters(t *testing.T) {
|
||||||
|
message := shortError(errors.New(strings.Repeat("界", 260) + "\n\t" + strings.Repeat("語", 300)))
|
||||||
|
if got := len([]rune(message)); got != 512 {
|
||||||
|
t.Fatalf("エラー文字数=%d want=512", got)
|
||||||
|
}
|
||||||
|
if strings.ContainsAny(message, "\r\n\t") || strings.Contains(message, " ") {
|
||||||
|
t.Fatalf("空白を1行へ正規化していない: %q", message)
|
||||||
|
}
|
||||||
|
if !strings.HasSuffix(message, strings.Repeat("語", 251)) {
|
||||||
|
t.Fatalf("UTF-8文字の途中で切断または境界が不正: %q", message)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionSchedulerRecordsRedactedFailureWithoutCompleting(t *testing.T) {
|
||||||
|
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||||
|
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "failed"}}}
|
||||||
|
fetcher := &fakeSeriesFetcher{errors: map[string]error{
|
||||||
|
"failed": errors.New("request failed\nplatform_token=secret-token Cookie: session-secret"),
|
||||||
|
}}
|
||||||
|
var logs bytes.Buffer
|
||||||
|
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{Now: func() time.Time { return now }}, log.New(&logs, "", 0))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
scheduler.checkDue(context.Background())
|
||||||
|
if len(subscriptions.completions) != 0 || len(subscriptions.failures) != 1 || !subscriptions.failures[0].at.Equal(now) {
|
||||||
|
t.Fatalf("失敗の記録: completions=%v failures=%v", subscriptions.completions, subscriptions.failures)
|
||||||
|
}
|
||||||
|
message := subscriptions.failures[0].message
|
||||||
|
if strings.Contains(message, "secret-token") || strings.Contains(message, "session-secret") || strings.Contains(logs.String(), "secret") || strings.Contains(message, "\n") {
|
||||||
|
t.Fatalf("秘密値または改行が残った: message=%q logs=%q", message, logs.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionSchedulerRunStartsImmediatelyThenUsesScanInterval(t *testing.T) {
|
||||||
|
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||||
|
subscriptions := &fakeSubscriptionStore{}
|
||||||
|
fetcher := &fakeSeriesFetcher{}
|
||||||
|
requested := make(chan time.Duration, 2)
|
||||||
|
tick := make(chan time.Time, 1)
|
||||||
|
after := func(duration time.Duration) <-chan time.Time {
|
||||||
|
requested <- duration
|
||||||
|
return tick
|
||||||
|
}
|
||||||
|
scheduler, err := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
|
||||||
|
ScanInterval: time.Hour, Now: func() time.Time { return now }, After: after,
|
||||||
|
}, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { scheduler.Run(ctx); close(done) }()
|
||||||
|
select {
|
||||||
|
case duration := <-requested:
|
||||||
|
if duration != time.Hour {
|
||||||
|
t.Fatalf("走査間隔: %v", duration)
|
||||||
|
}
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("起動直後の確認が行われなかった")
|
||||||
|
}
|
||||||
|
subscriptions.mu.Lock()
|
||||||
|
initialCalls := len(subscriptions.cutoffs)
|
||||||
|
subscriptions.mu.Unlock()
|
||||||
|
if initialCalls != 1 {
|
||||||
|
t.Fatalf("起動直後の確認回数: %d", initialCalls)
|
||||||
|
}
|
||||||
|
tick <- now
|
||||||
|
select {
|
||||||
|
case <-requested:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("1時間後の確認が行われなかった")
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("終了できなかった")
|
||||||
|
}
|
||||||
|
subscriptions.mu.Lock()
|
||||||
|
defer subscriptions.mu.Unlock()
|
||||||
|
if len(subscriptions.cutoffs) != 2 {
|
||||||
|
t.Fatalf("確認回数: %d", len(subscriptions.cutoffs))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSubscriptionSchedulerCancellationInterruptsFetchAndSeriesWait(t *testing.T) {
|
||||||
|
now := time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)
|
||||||
|
t.Run("fetch", func(t *testing.T) {
|
||||||
|
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}}}
|
||||||
|
fetcher := &fakeSeriesFetcher{block: true, started: make(chan string, 1)}
|
||||||
|
scheduler, _ := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{Now: func() time.Time { return now }}, nil)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { scheduler.checkDue(ctx); close(done) }()
|
||||||
|
select {
|
||||||
|
case <-fetcher.started:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("取得が開始されなかった")
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("取得中断で終了しなかった")
|
||||||
|
}
|
||||||
|
if len(subscriptions.failures) != 0 || len(subscriptions.completions) != 0 {
|
||||||
|
t.Fatalf("中断した番組を記録した: failures=%v completions=%v", subscriptions.failures, subscriptions.completions)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("between series", func(t *testing.T) {
|
||||||
|
subscriptions := &fakeSubscriptionStore{due: []store.Subscription{{SeriesID: "one"}, {SeriesID: "two"}}}
|
||||||
|
fetcher := &fakeSeriesFetcher{episodes: map[string][]tver.Episode{}, errors: map[string]error{}}
|
||||||
|
waiting := make(chan struct{})
|
||||||
|
never := make(chan time.Time)
|
||||||
|
scheduler, _ := NewSubscriptionScheduler(subscriptions, fetcher, SubscriptionSchedulerConfig{
|
||||||
|
Now: func() time.Time { return now }, After: func(time.Duration) <-chan time.Time { close(waiting); return never },
|
||||||
|
}, nil)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { scheduler.checkDue(ctx); close(done) }()
|
||||||
|
<-waiting
|
||||||
|
cancel()
|
||||||
|
<-done
|
||||||
|
if len(fetcher.calls) != 1 || fetcher.calls[0] != "one" {
|
||||||
|
t.Fatalf("番組間の中断: %v", fetcher.calls)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
+17
@@ -0,0 +1,17 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
case "${FAKE_FFPROBE_MODE:-valid}" in
|
||||||
|
invalid|bad-container)
|
||||||
|
echo '{"format":{"format_name":"matroska"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
|
||||||
|
bad-video)
|
||||||
|
echo '{"format":{"format_name":"mp4"},"streams":[{"codec_type":"video","codec_name":"vp9"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
|
||||||
|
bad-audio)
|
||||||
|
echo '{"format":{"format_name":"mp4"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"opus"}]}' ;;
|
||||||
|
non-json)
|
||||||
|
echo 'not JSON' ;;
|
||||||
|
failure)
|
||||||
|
echo 'probe failed' >&2
|
||||||
|
exit 1 ;;
|
||||||
|
*)
|
||||||
|
echo '{"format":{"format_name":"mov,mp4,m4a,3gp,3g2,mj2"},"streams":[{"codec_type":"video","codec_name":"h264"},{"codec_type":"audio","codec_name":"aac"}]}' ;;
|
||||||
|
esac
|
||||||
+67
@@ -0,0 +1,67 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
[ -z "${FAKE_ARGS_FILE:-}" ] || printf '%s\n' "$@" > "$FAKE_ARGS_FILE"
|
||||||
|
|
||||||
|
output_template=
|
||||||
|
episode_url=
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
if [ "$1" = "-o" ]; then
|
||||||
|
output_template=$2
|
||||||
|
shift 2
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
case "$1" in
|
||||||
|
https://tver.jp/episodes/*) episode_url=$1 ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
|
||||||
|
mode=${FAKE_YTDLP_MODE:-success}
|
||||||
|
if [ "$mode" = "mixed" ]; then
|
||||||
|
case "$episode_url" in
|
||||||
|
*/ep-fail) mode=unknown ;;
|
||||||
|
*) mode=success ;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
case "$mode" in
|
||||||
|
drm) echo 'ERROR: This video is DRM protected token=secret-value' >&2; exit 1 ;;
|
||||||
|
geo) echo 'ERROR: This video is not available from your location due to geo restriction' >&2; exit 1 ;;
|
||||||
|
network) echo 'ERROR: Unable to download JSON metadata: [Errno 61] Connection refused' >&2; exit 1 ;;
|
||||||
|
unavailable) echo 'ERROR: MEDIA_NOT_FOUND: episode does not exist' >&2; exit 1 ;;
|
||||||
|
extract) echo 'ERROR: Unable to extract player data' >&2; exit 1 ;;
|
||||||
|
unknown) echo 'ERROR: surprising failure' >&2; exit 1 ;;
|
||||||
|
hang|hang-drm)
|
||||||
|
[ -z "${FAKE_READY_FILE:-}" ] || : > "$FAKE_READY_FILE"
|
||||||
|
if [ "$mode" = "hang-drm" ]; then
|
||||||
|
trap 'echo "ERROR: DRM protected" >&2; [ -z "${FAKE_TERM_FILE:-}" ] || : > "$FAKE_TERM_FILE"; exit 143' TERM INT
|
||||||
|
else
|
||||||
|
trap '[ -z "${FAKE_TERM_FILE:-}" ] || : > "$FAKE_TERM_FILE"; exit 143' TERM INT
|
||||||
|
fi
|
||||||
|
while :; do sleep 1; done
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
output=$(printf '%s' "$output_template" | sed 's/%(id)s/fake/g; s/%(ext)s/mp4/g')
|
||||||
|
mkdir -p "$(dirname "$output")"
|
||||||
|
printf 'fake mp4' > "$output"
|
||||||
|
if [ "$mode" = "symlink" ]; then
|
||||||
|
rm "$output"
|
||||||
|
ln -s "$FAKE_OUTSIDE_FILE" "$output"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$mode" = "burst" ]; then
|
||||||
|
i=1
|
||||||
|
while [ "$i" -le 20 ]; do
|
||||||
|
printf '{"event":"download","format_id":"hls-698-1","progress":{"status":"downloading","downloaded_bytes":%s,"total_bytes":100,"speed":10,"eta":2,"fragment_index":%s,"fragment_count":20}}\n' "$i" "$i"
|
||||||
|
i=$((i + 1))
|
||||||
|
done
|
||||||
|
sleep 0.12
|
||||||
|
else
|
||||||
|
echo '{"event":"download","format_id":"hls-698-1","progress":{"status":"downloading","downloaded_bytes":1024,"fragment_index":0,"fragment_count":50,"filename":"/private/tmp/tver-spike/epq432nee4.fhls-698-1.mp4","tmpfilename":"/private/tmp/tver-spike/epq432nee4.fhls-698-1.mp4.part","max_progress":null,"progress_idx":null,"elapsed":0,"total_bytes_estimate":29798400.0,"speed":4834.271158821972,"eta":null,"_eta_str":"Unknown","_speed_str":" 4.72KiB/s","_percent":0.003436426116838488,"_percent_str":" 0.0%","_total_bytes_str":" N/A","_total_bytes_estimate_str":" 28.42MiB","_downloaded_bytes_str":" 1.00KiB","_elapsed_str":"00:00:00","_default_template":" 0.0% of ~ 28.42MiB at 4.72KiB/s ETA Unknown (frag 0/50)"}}'
|
||||||
|
sleep 0.02
|
||||||
|
echo '{"event":"download","format_id":"hls-ts_AUDIO-0_2","progress":{"status":"finished","downloaded_bytes":50,"total_bytes":50,"speed":20,"eta":0}}'
|
||||||
|
sleep 0.02
|
||||||
|
echo '{"event":"postprocess","progress":{"status":"started","postprocessor":"Merger"}}' >&2
|
||||||
|
fi
|
||||||
|
printf '{"event":"after_move","filepath":"%s"}\n' "$output"
|
||||||
Reference in New Issue
Block a user