feat: Unify Driveのコピー処理を実装

This commit is contained in:
2026-10-12 16:12:17 +09:00
parent 375030c4e9
commit aa11347fa3
37 changed files with 6759 additions and 8 deletions
+14
View File
@@ -0,0 +1,14 @@
__pycache__/
*.py[cod]
*.sqlite
*.sqlite3
*.sqlite3-*
*.db
*.lock
.claude/
.codex/
.coverage
htmlcov/
.DS_Store
tmp/
work/
+65 -8
View File
@@ -3,7 +3,7 @@ title: UnifyDrive2NAS の現状
tags: tags:
- homelab - homelab
- UnifyDrive2NAS - UnifyDrive2NAS
status: 方式検討中 status: コピーコア実装済み
--- ---
# UnifyDrive2NAS の現状 # UnifyDrive2NAS の現状
@@ -13,8 +13,9 @@ status: 方式検討中
## 今の状態 ## 今の状態
- **方式検討中。** コード・テストはまだ無い - Python 3.8 の標準ライブラリと同梱した `exifread` 3.5.1 だけで動くコピーコアを実装済み
- リポジトリ: 未作成 - コピー済み一覧、初期化、ドライラン、二重起動防止、マウント確認を含む
- NAS への配置と Dagu の DAG 定義は未実施
## 目的 ## 目的
@@ -41,6 +42,8 @@ status: 方式検討中
| `YYYYMMDD` の日付 | **EXIF の撮影日時 → 動画のメタデータ(QuickTime の作成日時等)→ ファイルの日時**の順に、取れたものを使う(ユーザー判断、2026-10-10) | | `YYYYMMDD` の日付 | **EXIF の撮影日時 → 動画のメタデータ(QuickTime の作成日時等)→ ファイルの日時**の順に、取れたものを使う(ユーザー判断、2026-10-10) |
| 動画の日時の優先順 | **Apple の作成日時(`moov/meta`、タイムゾーン付き)→ `moov/mvhd` の作成日時(UTC を JST に変換)**(ユーザー判断、2026-10-12) | | 動画の日時の優先順 | **Apple の作成日時(`moov/meta`、タイムゾーン付き)→ `moov/mvhd` の作成日時(UTC を JST に変換)**(ユーザー判断、2026-10-12) |
| 隠しファイル | **先頭が `.` のファイル・フォルダはコピーしない**(`.<元の名前>.z<16 進 32 桁>` の書きかけと推定されるファイルを含む)(ユーザー判断、2026-10-12) | | 隠しファイル | **先頭が `.` のファイル・フォルダはコピーしない**(`.<元の名前>.z<16 進 32 桁>` の書きかけと推定されるファイルを含む)(ユーザー判断、2026-10-12) |
| カメラの管理フォルダ内の動画 | **コピー対象とする**。Sony の動画は `PlugBackup/<回>/.../PRIVATE/M4ROOT/CLIP/*.MP4` にあり(2026-10-12 時点で 9 本・約 5.4 GB)、`mvhd` を JST に変換した時刻が同じフォルダの `*M01.XML` の `CreationDate` と一致することを確認済み(ユーザー判断、2026-10-12) |
| コピー済み DB の既定値 | `~/.local/state/unifydrive2nas/copied.sqlite3`。`--database` で変更できる |
## 要件 ## 要件
@@ -51,11 +54,41 @@ status: 方式検討中
### 非機能要件 ### 非機能要件
- 未定 - Python 3.8.15 で動作し、`exifread` 以外は標準ライブラリだけを使う
- 1 ファイルの失敗で全体を止めず、終了時に非 0 を返す
- コピー元は読み取りだけとし、コピー先は一時ファイルからの rename で確定する
- コピー元がマウントポイントでない場合または空の場合は処理を開始しない
- `fcntl.flock` により同じ DB を使う処理の二重起動を防ぐ
## 構成(案) ## 構成
未定。[[#未決事項]] を決めてから書く。 | パス | 内容 |
|---|---|
| `unifydrive2nas.py` | ファイル列挙、日時判定、コピー、SQLite 記録、安全装置を持つ CLI |
| `vendor/exifread/` | `exifread` 3.5.1 と BSD-2-Clause ライセンス |
| `tests/test_unifydrive2nas.py` | 最小の画像・動画バイト列を生成して実行する標準ライブラリ `unittest` |
SQLite の `copied_files` は、コピー元からの相対パス・サイズ・更新日時(nanosecond)を複合主キーとする。コピー先、日付の出どころ、処理種別、処理日時も記録する。
### CLI
既定値は次のとおり。
| 引数 | 既定値 |
|---|---|
| `--source` | `/volume1/homes/kazu634/UnifyDrive` |
| `--destination` | `/volume1/homes/kazu634/Photos/PhotoLibrary` |
| `--database` | `~/.local/state/unifydrive2nas/copied.sqlite3` |
> [!warning] 初回運用
> 本番開始の前日に `python3 unifydrive2nas.py --init` を 1 回実行する。`--init` は現在の対象を DB に登録するだけで、コピーしない。事前確認には `--dry-run` を付ける。
通常実行とドライランの例:
```bash
python3 unifydrive2nas.py
python3 unifydrive2nas.py --dry-run
```
## 外部とのつながり ## 外部とのつながり
@@ -212,6 +245,10 @@ status: 方式検討中
- **先頭が `.` の隠しファイル `.<元の名前>.z<16 進 32 桁>` が 14 件ある**(例: `.IMG_2584.MOV.zb0bad79...`、`.DSC07166.JPG.z6d2802...`)。iPhone からのバックアップの途中のファイルか一時ファイルと推定。**コピー対象から外さないと、書きかけのファイルを取り込むおそれがある** - **先頭が `.` の隠しファイル `.<元の名前>.z<16 進 32 桁>` が 14 件ある**(例: `.IMG_2584.MOV.zb0bad79...`、`.DSC07166.JPG.z6d2802...`)。iPhone からのバックアップの途中のファイルか一時ファイルと推定。**コピー対象から外さないと、書きかけのファイルを取り込むおそれがある**
- iPhone・iPad・frankel の全 2,331 件を読むのに約 78 秒(ほぼ iPhone)。日々の実行では新しいファイルだけを読むので、時間は問題にならない見込み - iPhone・iPad・frankel の全 2,331 件を読むのに約 78 秒(ほぼ iPhone)。日々の実行では新しいファイルだけを読むので、時間は問題にならない見込み
### コピーコアの実機検証
[実機] 2026-10-12、NAS の Python 3.8.15 で単体テスト 37 件がすべて成功した。実マウントに対する `--dry-run` は、対象 6,786 件・失敗 0 件、日付の出どころは exif 6,081 件・apple 119 件・mvhd 14 件・mtime 572 件で、所要時間は約 3 分 47 秒だった。DB には `/tmp` の使い捨てパスを指定した。
### コピー先の現状 ### コピー先の現状
[実機] 2026-10-12 確認。 [実機] 2026-10-12 確認。
@@ -232,7 +269,8 @@ status: 方式検討中
## 課題 ## 課題
- なし([[#未決事項]] を参照) - NAS への配置と Dagu の DAG 定義を行う
- マウントが NAS 再起動後に自動復旧するか確認する([[#未決事項]])
## 経緯 ## 経緯
@@ -240,7 +278,26 @@ status: 方式検討中
- 2026-10-10: Unify Drive を DHCP(`192.168.10.37`)から固定 IP `192.168.10.200/24` に変更 - 2026-10-10: Unify Drive を DHCP(`192.168.10.37`)から固定 IP `192.168.10.200/24` に変更
- 2026-10-10: 踏み台から WebDAV でコピー元のディレクトリ構成を調査 - 2026-10-10: 踏み台から WebDAV でコピー元のディレクトリ構成を調査
- 2026-10-12: Dagu → NAS の SSH 鍵を登録。NAS 上の純 Python で撮影日時が取れることを確認 - 2026-10-12: Dagu → NAS の SSH 鍵を登録。NAS 上の純 Python で撮影日時が取れることを確認
- 2026-10-12: Python のコピーコア、SQLite のコピー済み一覧、安全装置、単体テストを実装
- 2026-10-12: Apple 作成日時の読み取りを、version/flags の無い QuickTime 形式と FullBox 形式の両方の `meta` に対応
## 確かめ方 ## 確かめ方
未定(実装後に書く)。 ローカルの単体テスト:
```bash
python3 -m unittest discover -s tests -v
```
CLI と引数の確認:
```bash
python3 unifydrive2nas.py --help
```
NAS では配置先で同じ単体テストを実行した後、実マウントを使うドライランで対象と日付の出どころを確認する。
```bash
python3 -m unittest discover -s tests -v
python3 unifydrive2nas.py --dry-run
```
+1
View File
@@ -0,0 +1 @@
"""UnifyDrive2NAS のテスト。"""
+554
View File
@@ -0,0 +1,554 @@
import contextlib
import datetime
import os
from pathlib import Path
import sqlite3
import struct
import tempfile
import unittest
from unittest import mock
import unifydrive2nas as app
def atom(kind, payload):
return struct.pack(">I4s", len(payload) + 8, kind) + payload
def atoms_in(content, start=0, end=None):
if end is None:
end = len(content)
result = []
position = start
while position + 8 <= end:
size, kind = struct.unpack(">I4s", content[position:position + 8])
result.append((kind, position + 8, position + size))
position += size
return result
def tiff_with_exif_datetime(text, tag=0x9003):
return tiff_with_exif_datetimes(((tag, text),))
def tiff_with_exif_datetimes(values):
encoded_values = [(tag, text.encode("ascii") + b"\x00") for tag, text in values]
# IFD0 の ExifIFDPointer は TIFF 先頭から 26、値は Exif IFD の後に置く。
ifd0 = struct.pack("<H", 1)
ifd0 += struct.pack("<HHI", 0x8769, 4, 1) + struct.pack("<I", 26)
ifd0 += struct.pack("<I", 0)
value_offset = 26 + 2 + 12 * len(encoded_values) + 4
exif_ifd = struct.pack("<H", len(encoded_values))
payload = b""
for tag, encoded in encoded_values:
exif_ifd += struct.pack("<HHI", tag, 2, len(encoded)) + struct.pack("<I", value_offset)
payload += encoded
value_offset += len(encoded)
exif_ifd += struct.pack("<I", 0)
return b"II*\x00" + struct.pack("<I", 8) + ifd0 + exif_ifd + payload
def jpeg_with_exif(text):
exif = b"Exif\x00\x00" + tiff_with_exif_datetime(text)
return b"\xff\xd8\xff\xe1" + struct.pack(">H", len(exif) + 2) + exif + b"\xff\xd9"
def jfif_without_exif():
payload = b"JFIF\x00\x01\x01\x00\x00\x01\x00\x01\x00\x00"
return b"\xff\xd8\xff\xe0" + struct.pack(">H", len(payload) + 2) + payload + b"\xff\xd9"
def png_with_exif(text):
signature = b"\x89PNG\r\n\x1a\n"
exif = tiff_with_exif_datetime(text)
# exifread は日時テストでは CRC を参照しないため最小値でよい。
return signature + struct.pack(">I", len(exif)) + b"eXIf" + exif + b"\x00\x00\x00\x00"
def movie_with_metadata(
apple_text=None,
mvhd_seconds=None,
mvhd_version=0,
quicktime_meta=False,
metadata_key=b"com.apple.quicktime.creationdate",
item_index=1,
):
children = b""
if mvhd_seconds is not None:
if mvhd_version == 0:
payload = b"\x00\x00\x00\x00" + struct.pack(">I", mvhd_seconds)
else:
payload = b"\x01\x00\x00\x00" + struct.pack(">Q", mvhd_seconds)
children += atom(b"mvhd", payload)
if apple_text is not None:
key = metadata_key
keys = atom(b"keys", b"\x00\x00\x00\x00" + struct.pack(">I", 1) + struct.pack(">I4s", len(key) + 8, b"mdta") + key)
data = atom(b"data", b"\x00\x00\x00\x01" + b"\x00\x00\x00\x00" + apple_text.encode("utf-8"))
item = atom(struct.pack(">I", item_index), data)
ilst = atom(b"ilst", item)
if quicktime_meta:
# 実際の iPhone MOV と同じく、version/flags 無しで hdlr から始める。
meta_payload = atom(b"hdlr", b"\x00" * 26) + keys + ilst
else:
meta_payload = b"\x00\x00\x00\x00" + keys + ilst
children += atom(b"meta", meta_payload)
return atom(b"ftyp", b"qt \x00\x00\x00\x00") + atom(b"moov", children)
class Workspace:
def __init__(self, testcase):
self.temporary = tempfile.TemporaryDirectory()
testcase.addCleanup(self.temporary.cleanup)
root = Path(self.temporary.name)
self.source = root / "source"
self.destination = root / "destination"
self.database = root / "state" / "copied.sqlite3"
self.source.mkdir()
def config(self, initialize=False, dry_run=False):
return app.Config(
source=self.source,
destination=self.destination,
database=self.database,
initialize=initialize,
dry_run=dry_run,
)
def run(self, **kwargs):
return app.run(self.config(**kwargs), mount_checker=lambda unused: True)
class DateDetectionTests(unittest.TestCase):
def setUp(self):
self.workspace = Workspace(self)
def write(self, name, content, mtime=None):
path = self.workspace.source / name
path.write_bytes(content)
if mtime is not None:
os.utime(str(path), (mtime, mtime))
return path
def test_exif_original_is_used_for_jpeg(self):
path = self.write("photo.jpg", jpeg_with_exif("2024:02:03 04:05:06"))
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("exif", source)
self.assertEqual("20240203", value.strftime("%Y%m%d"))
def test_exif_is_read_from_png_chunk(self):
path = self.write("screen.PNG", png_with_exif("2023:12:31 23:59:58"))
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("exif", source)
self.assertEqual("20231231", value.strftime("%Y%m%d"))
def test_digitized_date_is_used_when_original_is_absent(self):
exif = b"Exif\x00\x00" + tiff_with_exif_datetime("2020:06:07 08:09:10", tag=0x9004)
content = b"\xff\xd8\xff\xe1" + struct.pack(">H", len(exif) + 2) + exif + b"\xff\xd9"
path = self.write("digitized.jpg", content)
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("exif", source)
self.assertEqual("20200607", value.strftime("%Y%m%d"))
def test_original_date_precedes_digitized_when_both_exist(self):
exif = b"Exif\x00\x00" + tiff_with_exif_datetimes(
(
(0x9003, "2024:01:02 03:04:05"),
(0x9004, "1999:12:31 23:59:59"),
)
)
content = b"\xff\xd8\xff\xe1" + struct.pack(">H", len(exif) + 2) + exif + b"\xff\xd9"
path = self.write("both.jpg", content)
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("exif", source)
self.assertEqual("20240102 030405", value.strftime("%Y%m%d %H%M%S"))
def test_apple_date_precedes_mvhd_and_is_converted_to_jst(self):
seconds = int((datetime.datetime(2020, 1, 1, tzinfo=app.UTC) - app.QUICKTIME_EPOCH).total_seconds())
path = self.write("clip.mov", movie_with_metadata("2025-11-08T20:46:34+0900", seconds))
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("apple", source)
self.assertEqual("20251108 204634", value.strftime("%Y%m%d %H%M%S"))
def test_apple_date_in_quicktime_meta_without_fullbox_header(self):
seconds = int((datetime.datetime(2020, 1, 1, tzinfo=app.UTC) - app.QUICKTIME_EPOCH).total_seconds())
path = self.write(
"iphone.mov",
movie_with_metadata(
"2025-11-08T20:46:34+0900", seconds, quicktime_meta=True
),
)
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("apple", source)
self.assertEqual("20251108 204634", value.strftime("%Y%m%d %H%M%S"))
def test_quicktime_meta_fixture_matches_observed_atom_layout(self):
apple_text = "2025-11-08T20:46:34+0900"
content = movie_with_metadata(apple_text, quicktime_meta=True)
top = atoms_in(content)
moov = next(item for item in top if item[0] == b"moov")
meta = next(item for item in atoms_in(content, moov[1], moov[2]) if item[0] == b"meta")
children = atoms_in(content, meta[1], meta[2])
self.assertEqual([b"hdlr", b"keys", b"ilst"], [item[0] for item in children])
keys = children[1]
self.assertEqual(b"\x00\x00\x00\x00", content[keys[1]:keys[1] + 4])
self.assertEqual(1, struct.unpack(">I", content[keys[1] + 4:keys[1] + 8])[0])
entry_size, namespace = struct.unpack(">I4s", content[keys[1] + 8:keys[1] + 16])
self.assertEqual(b"mdta", namespace)
self.assertEqual(
b"com.apple.quicktime.creationdate",
content[keys[1] + 16:keys[1] + 8 + entry_size],
)
ilst = children[2]
item = atoms_in(content, ilst[1], ilst[2])[0]
self.assertEqual(struct.pack(">I", 1), item[0])
data = atoms_in(content, item[1], item[2])[0]
self.assertEqual(b"data", data[0])
self.assertEqual(b"\x00\x00\x00\x01\x00\x00\x00\x00", content[data[1]:data[1] + 8])
self.assertEqual(apple_text.encode("utf-8"), content[data[1] + 8:data[2]])
def test_missing_apple_key_or_ilst_item_falls_back_to_mvhd_in_both_meta_forms(self):
instant = datetime.datetime(2024, 5, 6, 7, 8, 9, tzinfo=app.UTC)
seconds = int((instant - app.QUICKTIME_EPOCH).total_seconds())
cases = (
("missing-key", {"metadata_key": b"com.apple.quicktime.location.ISO6709"}),
("missing-item", {"item_index": 2}),
)
for quicktime_meta in (False, True):
for label, options in cases:
with self.subTest(quicktime_meta=quicktime_meta, case=label):
path = self.write(
"{0}-{1}.mov".format("quicktime" if quicktime_meta else "fullbox", label),
movie_with_metadata(
"2025-11-08T20:46:34+0900",
seconds,
quicktime_meta=quicktime_meta,
**options
),
)
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("mvhd", source)
self.assertEqual("20240506 160809", value.strftime("%Y%m%d %H%M%S"))
def test_apple_date_uses_its_own_non_jst_offset(self):
seconds = int((datetime.datetime(2020, 1, 1, tzinfo=app.UTC) - app.QUICKTIME_EPOCH).total_seconds())
path = self.write("offset.mov", movie_with_metadata("2025-01-01T20:30:00-0500", seconds))
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("apple", source)
self.assertEqual("20250102 103000", value.strftime("%Y%m%d %H%M%S"))
def test_mvhd_versions_zero_and_one_are_supported(self):
instant = datetime.datetime(2024, 12, 31, 18, 0, tzinfo=app.UTC)
seconds = int((instant - app.QUICKTIME_EPOCH).total_seconds())
for version in (0, 1):
with self.subTest(version=version):
path = self.write("clip{0}.mp4".format(version), movie_with_metadata(mvhd_seconds=seconds, mvhd_version=version))
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("mvhd", source)
self.assertEqual("20250101 030000", value.strftime("%Y%m%d %H%M%S"))
def test_mvhd_jst_day_boundary(self):
cases = (
(datetime.datetime(2024, 12, 31, 14, 59, 59, tzinfo=app.UTC), "20241231 235959"),
(datetime.datetime(2024, 12, 31, 15, 0, 0, tzinfo=app.UTC), "20250101 000000"),
)
for index, (instant, expected) in enumerate(cases):
with self.subTest(instant=instant):
seconds = int((instant - app.QUICKTIME_EPOCH).total_seconds())
path = self.write("boundary{0}.mp4".format(index), movie_with_metadata(mvhd_seconds=seconds))
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("mvhd", source)
self.assertEqual(expected, value.strftime("%Y%m%d %H%M%S"))
def test_metadata_parser_exceptions_fall_through_candidates(self):
expected = datetime.datetime(2022, 5, 7, 0, 30, tzinfo=app.JST)
path = self.write("error.mov", b"broken", expected.timestamp())
mvhd = datetime.datetime(2020, 2, 3, 4, 5, tzinfo=app.JST)
with mock.patch.object(app, "parse_exif_datetime", side_effect=ValueError("bad exif")):
with mock.patch.object(app, "parse_video_datetimes", return_value=(None, mvhd)):
value, source = app.determine_datetime(path, path.stat())
self.assertEqual((mvhd, "mvhd"), (value, source))
with mock.patch.object(app, "parse_exif_datetime", side_effect=ValueError("bad exif")):
with mock.patch.object(app, "parse_video_datetimes", side_effect=ValueError("bad movie")):
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("mtime", source)
self.assertEqual("20220507 003000", value.strftime("%Y%m%d %H%M%S"))
def test_zero_mvhd_and_broken_file_fall_back_to_mtime(self):
expected = datetime.datetime(2022, 5, 6, 12, 0, tzinfo=app.JST)
cases = (
("zero-v0.mp4", movie_with_metadata(mvhd_seconds=0, mvhd_version=0)),
("zero-v1.mp4", movie_with_metadata(mvhd_seconds=0, mvhd_version=1)),
("broken.mov", b"broken"),
)
for name, content in cases:
with self.subTest(name=name):
path = self.write(name, content, expected.timestamp())
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("mtime", source)
self.assertEqual("20220506", value.strftime("%Y%m%d"))
def test_mtime_is_interpreted_in_jst_across_day_boundary(self):
cases = (
(datetime.datetime(2024, 12, 31, 14, 59, 59, tzinfo=app.UTC), "20241231 235959"),
(datetime.datetime(2024, 12, 31, 15, 0, 0, tzinfo=app.UTC), "20250101 000000"),
)
for index, (instant, expected) in enumerate(cases):
with self.subTest(instant=instant):
path = self.write("mtime{0}.jpg".format(index), jfif_without_exif(), instant.timestamp())
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("mtime", source)
self.assertEqual(expected, value.strftime("%Y%m%d %H%M%S"))
def test_jfif_without_exif_falls_back_to_mtime(self):
expected = datetime.datetime(2021, 7, 8, 1, 2, tzinfo=app.JST)
path = self.write("plain.jpeg", jfif_without_exif(), expected.timestamp())
value, source = app.determine_datetime(path, path.stat())
self.assertEqual("mtime", source)
self.assertEqual("20210708", value.strftime("%Y%m%d"))
class CopyTests(unittest.TestCase):
def setUp(self):
self.workspace = Workspace(self)
self.timestamp = datetime.datetime(2026, 10, 12, 9, 0, tzinfo=app.JST).timestamp()
def write(self, relative, content=b"content"):
path = self.workspace.source / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(content)
os.utime(str(path), (self.timestamp, self.timestamp))
return path
def copied_path(self, name):
return self.workspace.destination / "2026" / "20261012" / name
def test_exclusions_and_case_insensitive_extensions(self):
self.write("visible/PHOTO.JpG")
self.write("visible/raw.ARW")
self.write("visible/note.txt")
self.write(".hidden/photo.jpg")
self.write("visible/.nested/deeper/photo.jpg")
self.write("visible/.partial.JPG.z0123456789abcdef0123456789abcdef")
self.write("visible/@eaDir/thumb.jpg")
summary = self.workspace.run()
self.assertEqual((1, 1), (summary.targets, summary.copied))
self.assertTrue(self.copied_path("PHOTO.JpG").is_file())
def test_all_supported_extensions_are_case_insensitive(self):
names = ("a.JPG", "b.Jpeg", "c.HEIC", "d.HeIf", "e.PNG", "f.MOV", "g.Mp4")
for name in names:
self.write("album/" + name)
summary = self.workspace.run()
self.assertEqual((len(names), len(names)), (summary.targets, summary.copied))
self.assertEqual(sorted(names), sorted(path.name for path in self.copied_path("a.JPG").parent.iterdir()))
def test_jpeg_content_is_detected_even_with_png_extension(self):
source = self.write("mismatch.PNG", jpeg_with_exif("2024:03:04 05:06:07"))
value, date_source = app.determine_datetime(source, source.stat())
self.assertEqual("exif", date_source)
self.assertEqual("20240304", value.strftime("%Y%m%d"))
def test_same_name_same_content_is_recorded_without_copy(self):
self.write("album/photo.jpg", b"same")
existing = self.copied_path("photo.jpg")
existing.parent.mkdir(parents=True)
existing.write_bytes(b"same")
summary = self.workspace.run()
self.assertEqual(1, summary.identical)
self.assertEqual(0, summary.copied)
self.assertEqual(b"same", existing.read_bytes())
def test_same_name_different_content_gets_sequence_number(self):
self.write("album/photo.jpg", b"new")
existing = self.copied_path("photo.jpg")
existing.parent.mkdir(parents=True)
existing.write_bytes(b"old")
summary = self.workspace.run()
self.assertEqual((1, 1), (summary.copied, summary.numbered))
self.assertEqual(b"old", existing.read_bytes())
self.assertEqual(b"new", self.copied_path("photo_1.jpg").read_bytes())
def test_sequence_number_skips_multiple_different_collisions(self):
self.write("album/photo.jpg", b"new")
directory = self.copied_path("photo.jpg").parent
directory.mkdir(parents=True)
self.copied_path("photo.jpg").write_bytes(b"old-zero")
self.copied_path("photo_1.jpg").write_bytes(b"old-one")
summary = self.workspace.run()
self.assertEqual((1, 1), (summary.copied, summary.numbered))
self.assertEqual(b"new", self.copied_path("photo_2.jpg").read_bytes())
def test_second_run_does_not_copy_again(self):
self.write("photo.jpg")
first = self.workspace.run()
second = self.workspace.run()
self.assertEqual(1, first.copied)
self.assertEqual((1, 0, 0), (second.targets, second.new, second.copied))
def test_size_or_mtime_change_makes_source_new_again(self):
source = self.write("photo.jpg", b"first")
self.workspace.run()
source.write_bytes(b"second-content")
os.utime(str(source), ns=(source.stat().st_atime_ns, source.stat().st_mtime_ns + 1_000_000_000))
changed_content = self.workspace.run()
self.assertEqual((1, 1, 1), (changed_content.new, changed_content.copied, changed_content.numbered))
self.assertEqual(b"second-content", self.copied_path("photo_1.jpg").read_bytes())
current = source.stat()
os.utime(str(source), ns=(current.st_atime_ns, current.st_mtime_ns + 1_000_000_000))
changed_mtime = self.workspace.run()
self.assertEqual((1, 0), (changed_mtime.new, changed_mtime.copied))
self.assertEqual(1, changed_mtime.identical)
with contextlib.closing(sqlite3.connect(str(self.workspace.database))) as connection:
self.assertEqual(3, connection.execute("SELECT COUNT(*) FROM copied_files").fetchone()[0])
def test_database_records_key_destination_and_date_source(self):
source = self.write("album/photo.jpg", b"plain")
self.workspace.run()
with contextlib.closing(sqlite3.connect(str(self.workspace.database))) as connection:
row = connection.execute(
"""SELECT source_relative_path, source_size, source_mtime_ns,
destination_path, date_source, action
FROM copied_files"""
).fetchone()
self.assertEqual("album/photo.jpg", row[0])
self.assertEqual(source.stat().st_size, row[1])
self.assertEqual(source.stat().st_mtime_ns, row[2])
self.assertEqual(str(self.copied_path("photo.jpg")), row[3])
self.assertEqual(("mtime", "copied"), row[4:6])
def test_initialization_records_without_copying(self):
self.write("photo.jpg")
initialized = self.workspace.run(initialize=True)
regular = self.workspace.run()
self.assertEqual((1, 1, 0), (initialized.targets, initialized.new, initialized.copied))
self.assertFalse(self.workspace.destination.exists())
self.assertEqual(0, regular.new)
with contextlib.closing(sqlite3.connect(str(self.workspace.database))) as connection:
row = connection.execute("SELECT date_source, action FROM copied_files").fetchone()
self.assertEqual(("init", "initialized"), row)
def test_unmounted_or_empty_source_stops_before_writing(self):
self.write("photo.jpg")
with self.assertRaisesRegex(RuntimeError, "マウントポイント"):
app.run(self.workspace.config(), mount_checker=lambda unused: False)
self.assertFalse(self.workspace.database.exists())
empty = Workspace(self)
with self.assertRaisesRegex(RuntimeError, "空です"):
empty.run()
self.assertFalse(empty.database.exists())
def test_one_copy_failure_does_not_stop_later_files(self):
self.write("a.jpg", b"a")
self.write("b.jpg", b"b")
original = app.atomic_copy
calls = []
def fail_first(source, destination):
calls.append(source.name)
if source.name == "a.jpg":
raise OSError("test failure")
original(source, destination)
summary = app.run(
self.workspace.config(), mount_checker=lambda unused: True, copy_function=fail_first
)
self.assertEqual((1, 1), (summary.failed, summary.copied))
self.assertEqual(["a.jpg", "b.jpg"], calls)
self.assertTrue(self.copied_path("b.jpg").exists())
with contextlib.closing(sqlite3.connect(str(self.workspace.database))) as connection:
rows = connection.execute("SELECT source_relative_path FROM copied_files ORDER BY source_relative_path").fetchall()
self.assertEqual([("b.jpg",)], rows)
def test_source_files_are_not_changed(self):
source = self.write("photo.jpg", b"immutable")
before = (source.read_bytes(), source.stat().st_mtime_ns, source.stat().st_mode)
self.workspace.run()
after = (source.read_bytes(), source.stat().st_mtime_ns, source.stat().st_mode)
self.assertEqual(before, after)
def test_dry_run_changes_neither_destination_nor_database_rows(self):
self.write("photo.jpg")
summary = self.workspace.run(dry_run=True)
self.assertEqual((1, 1, 0), (summary.targets, summary.new, summary.copied))
self.assertFalse(self.workspace.destination.exists())
self.assertFalse(self.workspace.database.exists())
self.assertFalse(Path(str(self.workspace.database) + ".lock").exists())
def test_dry_run_does_not_change_an_existing_database(self):
self.write("existing.jpg", b"existing")
self.workspace.run()
before = self.workspace.database.read_bytes()
self.write("new.jpg", b"new")
summary = self.workspace.run(dry_run=True)
self.assertEqual(1, summary.new)
self.assertEqual(before, self.workspace.database.read_bytes())
self.assertFalse(self.copied_path("new.jpg").exists())
def test_init_dry_run_writes_nothing(self):
self.write("photo.jpg")
summary = self.workspace.run(initialize=True, dry_run=True)
self.assertEqual((1, 1, 0), (summary.targets, summary.new, summary.copied))
self.assertFalse(self.workspace.destination.exists())
self.assertFalse(self.workspace.database.exists())
def test_atomic_copy_leaves_no_temporary_file(self):
source = self.write("photo.jpg")
self.workspace.run()
copied = self.copied_path("photo.jpg")
directory = copied.parent
self.assertEqual(["photo.jpg"], sorted(path.name for path in directory.iterdir()))
self.assertEqual(source.stat().st_mtime_ns, copied.stat().st_mtime_ns)
def test_atomic_copy_failure_removes_partial_temporary_file(self):
source = self.write("photo.jpg")
destination = self.copied_path("photo.jpg")
destination.parent.mkdir(parents=True)
def write_partial_then_fail(unused_source, temporary):
Path(temporary).write_bytes(b"partial")
raise OSError("copy failed")
with mock.patch.object(app.shutil, "copy2", side_effect=write_partial_then_fail):
with self.assertRaisesRegex(OSError, "copy failed"):
app.atomic_copy(source, destination)
self.assertEqual([], list(destination.parent.iterdir()))
class LockTests(unittest.TestCase):
def test_lock_rejects_a_second_holder(self):
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / "process.lock"
with app.process_lock(path):
with self.assertRaisesRegex(RuntimeError, "実行中"):
with app.process_lock(path):
self.fail("二重ロックを取得してはならない")
class MainTests(unittest.TestCase):
def test_success_prints_all_summary_counts_and_returns_zero(self):
summary = app.Summary(targets=6, new=5, copied=3, identical=1, numbered=2, failed=0)
with mock.patch.object(app, "run", return_value=summary):
with mock.patch("builtins.print") as output:
self.assertEqual(0, app.main([]))
output.assert_called_once_with(
"まとめ: 対象=6 新規=5 コピー=3 同一スキップ=1 連番=2 失敗=0"
)
def test_file_failure_returns_one(self):
with mock.patch.object(app, "run", return_value=app.Summary(failed=1)):
with mock.patch("builtins.print"):
self.assertEqual(1, app.main([]))
def test_startup_failure_returns_two(self):
with mock.patch.object(app, "run", side_effect=RuntimeError("test failure")):
self.assertEqual(2, app.main([]))
if __name__ == "__main__":
unittest.main()
+505
View File
@@ -0,0 +1,505 @@
#!/usr/bin/env python3
"""Unify Drive の画像・動画を日付別に NAS へコピーする。"""
import argparse
import contextlib
import datetime
import fcntl
import hashlib
import logging
import os
from pathlib import Path
import shutil
import sqlite3
import struct
import sys
import tempfile
from dataclasses import dataclass
from typing import BinaryIO, Callable, Iterator, List, Optional, Sequence, Tuple
# exifread は内部で絶対 import を使うため、同梱先を import 検索パスに加える。
VENDOR_DIRECTORY = Path(__file__).resolve().parent / "vendor"
sys.path.insert(0, str(VENDOR_DIRECTORY))
import exifread # noqa: E402
logging.getLogger("exifread").setLevel(logging.ERROR)
DEFAULT_SOURCE = Path("/volume1/homes/kazu634/UnifyDrive")
DEFAULT_DESTINATION = Path("/volume1/homes/kazu634/Photos/PhotoLibrary")
DEFAULT_DATABASE = Path.home() / ".local" / "state" / "unifydrive2nas" / "copied.sqlite3"
SUPPORTED_SUFFIXES = frozenset((".jpg", ".jpeg", ".heic", ".heif", ".png", ".mov", ".mp4"))
VIDEO_SUFFIXES = frozenset((".mov", ".mp4"))
JST = datetime.timezone(datetime.timedelta(hours=9))
UTC = datetime.timezone.utc
QUICKTIME_EPOCH = datetime.datetime(1904, 1, 1, tzinfo=UTC)
APPLE_CREATION_KEY = b"com.apple.quicktime.creationdate"
LOGGER = logging.getLogger("unifydrive2nas")
@dataclass
class Config:
source: Path
destination: Path
database: Path
initialize: bool = False
dry_run: bool = False
@dataclass
class Summary:
targets: int = 0
new: int = 0
copied: int = 0
identical: int = 0
numbered: int = 0
failed: int = 0
@dataclass(frozen=True)
class Atom:
kind: bytes
payload_start: int
end: int
def iter_target_files(source: Path, on_error: Optional[Callable[[OSError], None]] = None) -> Iterator[Path]:
"""隠し項目と @eaDir を除外し、対象拡張子のファイルを列挙する。"""
for root, directories, files in os.walk(str(source), topdown=True, onerror=on_error):
directories[:] = sorted(
name for name in directories if not name.startswith(".") and name != "@eaDir"
)
for name in sorted(files):
if name.startswith("."):
continue
path = Path(root) / name
if path.suffix.lower() in SUPPORTED_SUFFIXES:
yield path
def parse_exif_datetime(path: Path) -> Optional[datetime.datetime]:
"""EXIF の原日時、デジタル化日時の順に JST として読む。"""
with path.open("rb") as handle:
tags = exifread.process_file(handle, details=False)
for key in ("EXIF DateTimeOriginal", "EXIF DateTimeDigitized"):
tag = tags.get(key)
if tag is None:
continue
try:
value = datetime.datetime.strptime(str(tag).strip(), "%Y:%m:%d %H:%M:%S")
return value.replace(tzinfo=JST)
except (TypeError, ValueError):
continue
return None
def _iter_atoms(handle: BinaryIO, start: int, end: int) -> Iterator[Atom]:
"""指定範囲の ISO Base Media File Format atom を安全に走査する。"""
position = start
while position + 8 <= end:
handle.seek(position)
header = handle.read(8)
if len(header) != 8:
return
size, kind = struct.unpack(">I4s", header)
header_size = 8
if size == 1:
extended = handle.read(8)
if len(extended) != 8:
return
size = struct.unpack(">Q", extended)[0]
header_size = 16
elif size == 0:
size = end - position
if size < header_size or position + size > end:
return
yield Atom(kind=kind, payload_start=position + header_size, end=position + size)
position += size
def _find_child(handle: BinaryIO, start: int, end: int, kind: bytes) -> Optional[Atom]:
for atom in _iter_atoms(handle, start, end):
if atom.kind == kind:
return atom
return None
def _read_apple_creation_date(handle: BinaryIO, meta: Atom) -> Optional[datetime.datetime]:
# QuickTime 形式は payload 直下、ISO BMFF FullBox 形式は version/flags の後に子がある。
keys = None
ilst = None
for children_start in (meta.payload_start, meta.payload_start + 4):
keys = _find_child(handle, children_start, meta.end, b"keys")
ilst = _find_child(handle, children_start, meta.end, b"ilst")
if keys is not None and ilst is not None:
break
if keys is None or ilst is None:
return None
handle.seek(keys.payload_start)
keys_header = handle.read(8)
if len(keys_header) != 8:
return None
key_count = struct.unpack(">I", keys_header[4:8])[0]
wanted_index = None
position = keys.payload_start + 8
for index in range(1, key_count + 1):
if position + 8 > keys.end:
return None
handle.seek(position)
entry_header = handle.read(8)
entry_size, namespace = struct.unpack(">I4s", entry_header)
if entry_size < 8 or position + entry_size > keys.end:
return None
key = handle.read(entry_size - 8)
if namespace == b"mdta" and key == APPLE_CREATION_KEY:
wanted_index = index
break
position += entry_size
if wanted_index is None:
return None
item = _find_child(handle, ilst.payload_start, ilst.end, struct.pack(">I", wanted_index))
if item is None:
return None
data = _find_child(handle, item.payload_start, item.end, b"data")
if data is None or data.payload_start + 8 > data.end:
return None
handle.seek(data.payload_start + 8)
raw_value = handle.read(data.end - data.payload_start - 8)
try:
text = raw_value.rstrip(b"\x00").decode("utf-8").strip()
except UnicodeDecodeError:
return None
try:
value = datetime.datetime.strptime(text, "%Y-%m-%dT%H:%M:%S%z")
except ValueError:
return None
return value.astimezone(JST)
def _read_mvhd_datetime(handle: BinaryIO, mvhd: Atom) -> Optional[datetime.datetime]:
handle.seek(mvhd.payload_start)
version_bytes = handle.read(4)
if len(version_bytes) != 4:
return None
version = version_bytes[0]
if version == 0:
raw = handle.read(4)
if len(raw) != 4:
return None
seconds = struct.unpack(">I", raw)[0]
elif version == 1:
raw = handle.read(8)
if len(raw) != 8:
return None
seconds = struct.unpack(">Q", raw)[0]
else:
return None
if seconds == 0:
return None
try:
return (QUICKTIME_EPOCH + datetime.timedelta(seconds=seconds)).astimezone(JST)
except (OverflowError, ValueError):
return None
def parse_video_datetimes(path: Path) -> Tuple[Optional[datetime.datetime], Optional[datetime.datetime]]:
"""Apple 作成日時と mvhd 作成日時を読み、両方を返す。"""
with path.open("rb") as handle:
file_size = path.stat().st_size
moov = _find_child(handle, 0, file_size, b"moov")
if moov is None:
return None, None
meta = _find_child(handle, moov.payload_start, moov.end, b"meta")
mvhd = _find_child(handle, moov.payload_start, moov.end, b"mvhd")
apple = _read_apple_creation_date(handle, meta) if meta is not None else None
mvhd_value = _read_mvhd_datetime(handle, mvhd) if mvhd is not None else None
return apple, mvhd_value
def determine_datetime(path: Path, stat_result: os.stat_result) -> Tuple[datetime.datetime, str]:
"""仕様で定めた優先順で日時と出どころを決める。"""
try:
value = parse_exif_datetime(path)
if value is not None:
return value, "exif"
except Exception as exc: # 壊れたファイルでも後続候補へ進む。
LOGGER.debug("EXIF の解析に失敗: %s: %s", path, exc)
if path.suffix.lower() in VIDEO_SUFFIXES:
try:
apple, mvhd = parse_video_datetimes(path)
if apple is not None:
return apple, "apple"
if mvhd is not None:
return mvhd, "mvhd"
except Exception as exc: # 壊れた atom でも mtime へ進む。
LOGGER.debug("動画メタデータの解析に失敗: %s: %s", path, exc)
return datetime.datetime.fromtimestamp(stat_result.st_mtime, JST), "mtime"
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
while True:
chunk = handle.read(1024 * 1024)
if not chunk:
break
digest.update(chunk)
return digest.hexdigest()
def same_content(left: Path, right: Path) -> bool:
if left.stat().st_size != right.stat().st_size:
return False
return sha256_file(left) == sha256_file(right)
def select_destination(source: Path, directory: Path) -> Tuple[Path, bool, bool]:
"""コピー先、同一内容か、連番名かを返す。"""
candidate = directory / source.name
number = 0
while candidate.exists():
if candidate.is_file() and same_content(source, candidate):
return candidate, True, number > 0
number += 1
candidate = directory / (source.stem + "_" + str(number) + source.suffix)
return candidate, False, number > 0
def atomic_copy(source: Path, destination: Path) -> None:
"""同じディレクトリの一時ファイルへ copy2 し、rename で確定する。"""
file_descriptor, temporary_name = tempfile.mkstemp(
prefix="." + destination.name + ".tmp-", dir=str(destination.parent)
)
os.close(file_descriptor)
temporary = Path(temporary_name)
try:
shutil.copy2(str(source), str(temporary))
# rename 前に内容とメタデータを確定し、本来の名前に不完全なファイルを出さない。
with temporary.open("rb") as handle:
os.fsync(handle.fileno())
os.replace(str(temporary), str(destination))
finally:
try:
temporary.unlink()
except FileNotFoundError:
pass
def open_database(path: Path, read_only: bool = False) -> sqlite3.Connection:
if read_only and path.exists():
connection = sqlite3.connect(path.resolve().as_uri() + "?mode=ro", uri=True)
return connection
if read_only:
connection = sqlite3.connect(":memory:")
else:
path.parent.mkdir(parents=True, exist_ok=True)
connection = sqlite3.connect(str(path))
connection.execute(
"""
CREATE TABLE IF NOT EXISTS copied_files (
source_relative_path TEXT NOT NULL,
source_size INTEGER NOT NULL,
source_mtime_ns INTEGER NOT NULL,
destination_path TEXT,
date_source TEXT NOT NULL,
action TEXT NOT NULL,
processed_at TEXT NOT NULL,
PRIMARY KEY (source_relative_path, source_size, source_mtime_ns)
)
"""
)
connection.commit()
return connection
def is_recorded(connection: sqlite3.Connection, relative_path: str, size: int, mtime_ns: int) -> bool:
row = connection.execute(
"""
SELECT 1 FROM copied_files
WHERE source_relative_path = ? AND source_size = ? AND source_mtime_ns = ?
""",
(relative_path, size, mtime_ns),
).fetchone()
return row is not None
def record_file(
connection: sqlite3.Connection,
relative_path: str,
size: int,
mtime_ns: int,
destination_path: Optional[str],
date_source: str,
action: str,
) -> None:
connection.execute(
"""
INSERT OR REPLACE INTO copied_files (
source_relative_path, source_size, source_mtime_ns,
destination_path, date_source, action, processed_at
) VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(
relative_path,
size,
mtime_ns,
destination_path,
date_source,
action,
datetime.datetime.now(UTC).isoformat(),
),
)
connection.commit()
@contextlib.contextmanager
def process_lock(lock_path: Path) -> Iterator[None]:
lock_path.parent.mkdir(parents=True, exist_ok=True)
with lock_path.open("a+") as handle:
try:
fcntl.flock(handle.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
raise RuntimeError("別の処理が実行中です: " + str(lock_path))
try:
yield
finally:
fcntl.flock(handle.fileno(), fcntl.LOCK_UN)
def validate_source(source: Path, mount_checker: Callable[[str], bool]) -> None:
if not source.is_dir():
raise RuntimeError("コピー元がディレクトリではありません: " + str(source))
if not mount_checker(str(source)):
raise RuntimeError("コピー元がマウントポイントではありません: " + str(source))
try:
next(source.iterdir())
except StopIteration:
raise RuntimeError("コピー元が空です: " + str(source))
def run(
config: Config,
mount_checker: Callable[[str], bool] = os.path.ismount,
copy_function: Callable[[Path, Path], None] = atomic_copy,
) -> Summary:
validate_source(config.source, mount_checker)
summary = Summary()
lock_path = Path(str(config.database) + ".lock")
lock_context = contextlib.nullcontext() if config.dry_run else process_lock(lock_path)
with lock_context:
connection = open_database(config.database, read_only=config.dry_run)
try:
walk_errors = [] # type: List[OSError]
def remember_walk_error(error: OSError) -> None:
walk_errors.append(error)
LOGGER.error("ディレクトリを読めません: %s", error)
for source_path in iter_target_files(config.source, remember_walk_error):
summary.targets += 1
try:
stat_result = source_path.stat()
relative_path = source_path.relative_to(config.source).as_posix()
if is_recorded(
connection, relative_path, stat_result.st_size, stat_result.st_mtime_ns
):
continue
summary.new += 1
if config.initialize:
if not config.dry_run:
record_file(
connection,
relative_path,
stat_result.st_size,
stat_result.st_mtime_ns,
None,
"init",
"initialized",
)
LOGGER.info("初期化%s: %s", "予定" if config.dry_run else "", relative_path)
continue
captured_at, date_source = determine_datetime(source_path, stat_result)
day = captured_at.astimezone(JST).strftime("%Y%m%d")
destination_directory = config.destination / day[:4] / day
if config.dry_run:
LOGGER.info("コピー予定 [%s]: %s -> %s", date_source, source_path, destination_directory)
continue
destination_directory.mkdir(parents=True, exist_ok=True)
destination_path, identical, numbered = select_destination(
source_path, destination_directory
)
if identical:
summary.identical += 1
action = "identical"
LOGGER.info("同一内容のためスキップ [%s]: %s -> %s", date_source, source_path, destination_path)
else:
copy_function(source_path, destination_path)
summary.copied += 1
if numbered:
summary.numbered += 1
action = "copied"
LOGGER.info("コピー [%s]: %s -> %s", date_source, source_path, destination_path)
record_file(
connection,
relative_path,
stat_result.st_size,
stat_result.st_mtime_ns,
str(destination_path),
date_source,
action,
)
except Exception as exc:
summary.failed += 1
LOGGER.error("処理失敗: %s: %s", source_path, exc)
summary.failed += len(walk_errors)
finally:
connection.close()
return summary
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--source", type=Path, default=DEFAULT_SOURCE, help="コピー元マウントポイント")
parser.add_argument("--destination", type=Path, default=DEFAULT_DESTINATION, help="コピー先 PhotoLibrary")
parser.add_argument("--database", type=Path, default=DEFAULT_DATABASE, help="コピー済み記録の SQLite DB")
parser.add_argument("--init", action="store_true", dest="initialize", help="既存対象をコピー済みとして登録")
parser.add_argument("--dry-run", action="store_true", help="コピー・DB 登録をせず予定だけを表示")
parser.add_argument("--verbose", action="store_true", help="解析失敗の詳細も表示")
return parser
def main(arguments: Optional[Sequence[str]] = None) -> int:
parser = build_parser()
namespace = parser.parse_args(arguments)
logging.basicConfig(
level=logging.DEBUG if namespace.verbose else logging.INFO,
format="%(asctime)s %(levelname)s %(message)s",
)
config = Config(
source=namespace.source,
destination=namespace.destination,
database=namespace.database,
initialize=namespace.initialize,
dry_run=namespace.dry_run,
)
try:
summary = run(config)
except Exception as exc:
LOGGER.error("開始できません: %s", exc)
return 2
print(
"まとめ: 対象={0.targets} 新規={0.new} コピー={0.copied} "
"同一スキップ={0.identical} 連番={0.numbered} 失敗={0.failed}".format(summary)
)
return 1 if summary.failed else 0
if __name__ == "__main__":
sys.exit(main())
+1
View File
@@ -0,0 +1 @@
"""NAS に同梱する外部ライブラリ。"""
+31
View File
@@ -0,0 +1,31 @@
Copyright (c) 2002-2007 Gene Cash
Copyright (c) 2007-2025 Ianaré Sévi and contributors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.
3. Neither the name of the authors nor the names of its contributors
may be used to endorse or promote products derived from this
software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+140
View File
@@ -0,0 +1,140 @@
"""
Read Exif metadata from image files
Supported formats: TIFF, JPEG, PNG, Webp, HEIC
"""
from typing import Any, BinaryIO, Dict
from exifread.core.exceptions import ExifNotFound, InvalidExif
from exifread.core.exif_header import ExifHeader
from exifread.core.find_exif import determine_type, get_endian_str
from exifread.core.xmp import find_xmp_data
from exifread.exif_log import get_logger
from exifread.serialize import convert_types
from exifread.tags import DEFAULT_STOP_TAG
__version__ = "3.5.1"
logger = get_logger()
def _extract_xmp_data(hdr: ExifHeader, fh: BinaryIO):
# Easy we already have them
xmp_tag = hdr.tags.get("Image ApplicationNotes")
if xmp_tag:
logger.debug("XMP present in Exif")
xmp_bytes = bytes(xmp_tag.values)
# We need to look in the entire file for the XML
else:
xmp_bytes = find_xmp_data(fh)
if xmp_bytes:
hdr.parse_xmp(xmp_bytes)
def process_file(
fh: BinaryIO,
stop_tag: str = DEFAULT_STOP_TAG,
details=True,
strict=False,
debug=False,
truncate_tags=True,
auto_seek=True,
extract_thumbnail=True,
builtin_types=False,
) -> Dict[str, Any]:
"""
Process an image file to extract EXIF metadata.
This is the function that has to deal with all the arbitrary nasty bits
of the EXIF standard.
:param fh: the file to process, must be opened in binary mode.
:param stop_tag: Stop processing when the given tag is retrieved.
:param details: If `True`, process MakerNotes.
:param strict: If `True`, raise exceptions on errors.
:param debug: Output debug information.
:param truncate_tags: If `True`, truncate the `printable` tag output.
There is no effect on tag `values`.
:param auto_seek: If `True`, automatically `seek` to the start of the file.
:param extract_thumbnail: If `True`, extract the JPEG thumbnail.
The thumbnail is not always present in the EXIF metadata.
:param builtin_types: If `True`, convert tags to standard Python types.
:returns: A `dict` containing the EXIF metadata.
The keys are a string in the format `"IFD_NAME TAG_NAME"`.
If `builtin_types` is `False`, the value will be a `IfdTag` class, or bytes.
IF `builtin_types` is `True`, the value will be a standard Python type.
"""
if auto_seek:
fh.seek(0)
try:
offset, endian_bytes, fake_exif = determine_type(fh)
except ExifNotFound as err:
logger.warning(err)
return {}
except InvalidExif as err:
logger.debug(err)
return {}
endian_str, endian_type = get_endian_str(endian_bytes)
# deal with the EXIF info we found
logger.debug("Endian format is %s (%s)", endian_str, endian_type)
hdr = ExifHeader(
fh, endian_str, offset, fake_exif, strict, debug, details, truncate_tags
)
thumb_ifd = 0
ctr = 0
for ifd in hdr.list_ifd():
if ctr == 0:
ifd_name = "Image"
elif ctr == 1:
ifd_name = "Thumbnail"
thumb_ifd = ifd
else:
ifd_name = "IFD %d" % ctr
logger.debug("IFD %d (%s) at offset %s:", ctr, ifd_name, ifd)
hdr.dump_ifd(ifd=ifd, ifd_name=ifd_name, stop_tag=stop_tag)
ctr += 1
# EXIF IFD
exif_off = hdr.tags.get("Image ExifOffset")
if exif_off:
logger.debug("Exif SubIFD at offset %s:", exif_off.values[0])
hdr.dump_ifd(ifd=exif_off.values[0], ifd_name="EXIF", stop_tag=stop_tag)
# EXIF SubIFD
sub_ifds = hdr.tags.get("Image SubIFDs")
if details and sub_ifds:
for subifd_id, subifd_offset in enumerate(sub_ifds.values):
logger.debug("Exif SubIFD%d at offset %d:", subifd_id, subifd_offset)
hdr.dump_ifd(
ifd=subifd_offset, ifd_name=f"EXIF SubIFD{subifd_id}", stop_tag=stop_tag
)
# deal with MakerNote contained in EXIF IFD
# (Some apps use MakerNote tags but do not use a format for which we
# have a description, do not process these).
if details and "EXIF MakerNote" in hdr.tags and "Image Make" in hdr.tags:
try:
hdr.decode_maker_note()
except ValueError as err:
if not strict:
logger.debug("Failed to decode EXIF MakerNote: %s", str(err))
else:
raise err
# extract thumbnails
if thumb_ifd and extract_thumbnail:
hdr.extract_tiff_thumbnail(thumb_ifd)
hdr.extract_jpeg_thumbnail()
# parse XMP tags (experimental)
if debug and details:
_extract_xmp_data(hdr=hdr, fh=fh)
if builtin_types:
return convert_types(hdr.tags)
return hdr.tags
+6
View File
@@ -0,0 +1,6 @@
"""Main entrypoint for the module."""
from exifread.cli import main
if __name__ == "__main__":
main()
+167
View File
@@ -0,0 +1,167 @@
#!/usr/bin/env python3
#
#
# Library to extract Exif information from digital camera image files.
# https://github.com/ianare/exif-py
#
#
# Copyright (c) 2002-2007 Gene Cash
# Copyright (c) 2007-2025 Ianaré Sévi and contributors
#
# See LICENSE.txt file for licensing information
# See ChangeLog.rst file for all contributors and changes
#
"""
Runs Exif tag extraction in command line.
"""
import argparse
import sys
import timeit
from exifread import __version__, exif_log, process_file
from exifread.core.exceptions import ExifError
from exifread.tags.fields import FIELD_DEFINITIONS
logger = exif_log.get_logger()
def get_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog="EXIF.py", description="Extract EXIF information from digital image files."
)
parser.add_argument(
"files",
metavar="FILE",
type=str,
nargs="+",
help="files to process",
)
parser.add_argument(
"-v",
"--version",
action="version",
version="EXIF.py Version %s on Python%s" % (__version__, sys.version_info[0]),
help="Display version information and exit",
)
parser.add_argument(
"-q",
"--quick",
action="store_false",
dest="detailed",
help="Do not process MakerNotes and do not extract thumbnails",
)
parser.add_argument(
"-t",
"--tag",
type=str,
dest="stop_tag",
help="Stop processing when this tag is retrieved.",
)
parser.add_argument(
"-s",
"--strict",
action="store_true",
dest="strict",
help="Run in strict mode (stop on errors).",
)
parser.add_argument(
"-b",
"--builtin",
action="store_true",
dest="builtin_types",
help="Convert IfdTag values to built-in Python variable types",
)
parser.add_argument(
"-d",
"--debug",
action="store_true",
dest="debug",
help="Run in debug mode (display extra info).",
)
parser.add_argument(
"-c",
"--color",
action="store_true",
dest="color",
help="Output in color (only works with debug on POSIX).",
)
args = parser.parse_args()
return args
def run_cli(args: argparse.Namespace) -> None:
"""Extract tags based on options (args)."""
exif_log.setup_logger(args.debug, args.color)
# output info for each file
for filename in args.files:
# avoid errors when printing to console
escaped_fn = filename.encode(
sys.getfilesystemencoding(), "surrogateescape"
).decode()
file_start = timeit.default_timer()
try:
with open(escaped_fn, "rb") as img_file:
logger.info("Opening: %s", escaped_fn)
tag_start = timeit.default_timer()
# get the tags
data = process_file(
img_file,
stop_tag=args.stop_tag,
details=args.detailed,
strict=args.strict,
debug=args.debug,
extract_thumbnail=args.detailed,
builtin_types=args.builtin_types,
)
tag_stop = timeit.default_timer()
except IOError:
logger.error("'%s' is unreadable", escaped_fn)
continue
if not data:
logger.warning("No EXIF information found")
print()
continue
if "JPEGThumbnail" in data:
logger.info("File has JPEG thumbnail")
del data["JPEGThumbnail"]
if "TIFFThumbnail" in data:
logger.info("File has TIFF thumbnail")
del data["TIFFThumbnail"]
for field in sorted(data):
value = data[field]
try:
if args.builtin_types:
logger.info("%s (%s): %r", field, type(value).__name__, value)
else:
logger.info(
"%s (%s): %s",
field,
FIELD_DEFINITIONS[value.field_type][1],
value.printable,
)
except (ExifError, ValueError):
logger.error("%s: %s", field, str(value))
file_stop = timeit.default_timer()
logger.debug("Tags processed in %s seconds", tag_stop - tag_start)
logger.debug("File processed in %s seconds", file_stop - file_start)
print()
def main() -> None:
run_cli(get_args())
if __name__ == "__main__":
main()
View File
+13
View File
@@ -0,0 +1,13 @@
"""Exception classes."""
class ExifError(Exception):
"""Base class for all errors."""
class InvalidExif(ExifError):
"""The EXIF is invalid."""
class ExifNotFound(ExifError):
"""The EXIF could not be found."""
+721
View File
@@ -0,0 +1,721 @@
"""
Base classes.
"""
import re
import struct
from typing import Any, BinaryIO, Dict, List, Optional, Tuple, Union
from exifread.core.exceptions import ExifError
from exifread.core.ifd_tag import IfdTag
from exifread.core.xmp import xmp_bytes_to_str
from exifread.exif_log import get_logger
from exifread.tags import (
DEFAULT_STOP_TAG,
IGNORE_TAGS,
IfdDictValue,
SubIfdTagDict,
SubIfdTagDictValue,
)
from exifread.tags.exif import EXIF_TAGS
from exifread.tags.fields import (
FIELD_DEFINITIONS,
FLOAT_FIELD_TYPES,
RATIO_FIELD_TYPES,
SIGNED_FIELD_TYPES,
FieldType,
)
from exifread.tags.makernote import (
apple,
canon,
casio,
dji,
fujifilm,
nikon,
olympus,
sony,
)
from exifread.utils import Ratio
logger = get_logger()
class ExifHeader:
"""
Handle an EXIF header.
"""
def __init__(
self,
file_handle: BinaryIO,
endian: str,
offset: int,
fake_exif: int,
strict: bool,
debug=False,
detailed=True,
truncate_tags=True,
) -> None:
self.file_handle = file_handle
self.endian = endian
self.offset = offset
self.fake_exif = fake_exif
self.strict = strict
self.debug = debug
self.detailed = detailed
self.truncate_tags = truncate_tags
self.tags: Dict[str, Any] = {}
def s2n(self, offset: int, length: int, signed=False) -> int:
"""
Convert slice to integer, based on sign and endian flags.
Usually this offset is assumed to be relative to the beginning of the
start of the EXIF information.
For some cameras that use relative tags, this offset may be relative
to some other starting point.
"""
# Little-endian if Intel, big-endian if Motorola
fmt = "<" if self.endian == "I" else ">"
# Construct a format string from the requested length and signedness;
# raise a ValueError if length is something silly like 3
try:
fmt += {
(1, False): "B",
(1, True): "b",
(2, False): "H",
(2, True): "h",
(4, False): "I",
(4, True): "i",
(8, False): "L",
(8, True): "l",
}[(length, signed)]
except KeyError as err:
raise ValueError("unexpected unpacking length: %d" % length) from err
self.file_handle.seek(self.offset + offset)
buf = self.file_handle.read(length)
if buf:
# Make sure the buffer is the proper length.
# Allows bypassing of corrupt slices.
if len(buf) != length:
logger.warning("Unexpected slice length: %d", len(buf))
return 0
return struct.unpack(fmt, buf)[0]
return 0
def n2b(self, offset: int, length: int) -> bytes:
"""Convert offset to bytes."""
s = b""
for _ in range(length):
if self.endian == "I":
s += bytes([offset & 0xFF])
else:
s = bytes([offset & 0xFF]) + s
offset = offset >> 8
return s
def _first_ifd(self) -> int:
"""Return first IFD."""
return self.s2n(4, 4)
def _next_ifd(self, ifd: int) -> int:
"""Return the pointer to next IFD."""
entries = self.s2n(ifd, 2)
next_ifd = self.s2n(ifd + 2 + 12 * entries, 4)
if next_ifd == ifd:
return 0
return next_ifd
def list_ifd(self) -> List[int]:
"""Return the list of IFDs in the header."""
i = self._first_ifd()
ifds = []
set_ifds = set()
while i:
if i in set_ifds:
logger.warning("IFD loop detected.")
break
set_ifds.add(i)
ifds.append(i)
i = self._next_ifd(i)
return ifds
def _process_field(
self,
tag_name: str,
count: int,
field_type: int,
type_length: int,
offset: int,
) -> list:
values: List[Any] = []
signed = field_type in SIGNED_FIELD_TYPES
# TODO: investigate
# some entries get too big to handle could be malformed
# file or problem with self.s2n
if count < 1000:
for _ in range(count):
if field_type in RATIO_FIELD_TYPES:
# a ratio
ratio_value = Ratio(
self.s2n(offset, 4, signed), self.s2n(offset + 4, 4, signed)
)
values.append(ratio_value)
elif field_type in FLOAT_FIELD_TYPES:
# a float or double
unpack_format = ""
if self.endian == "I":
unpack_format += "<"
else:
unpack_format += ">"
if field_type == FieldType.FLOAT_32:
unpack_format += "f"
else:
unpack_format += "d"
self.file_handle.seek(self.offset + offset)
byte_str = self.file_handle.read(type_length)
try:
values.append(struct.unpack(unpack_format, byte_str))
except struct.error:
logger.warning("Possibly corrupted field %s", tag_name)
else:
value = self.s2n(offset, type_length, signed)
values.append(value)
offset = offset + type_length
# The test above causes problems with tags that are
# supposed to have long values! Fix up one important case.
elif tag_name in ("MakerNote", canon.CAMERA_INFO_TAG_NAME):
for _ in range(count):
value = self.s2n(offset, type_length, signed)
values.append(value)
offset = offset + type_length
return values
def _process_ascii_field(
self, ifd_name: str, tag_name: str, count: int, offset: int
):
values: Union[str, bytes] = ""
# special case: null-terminated ASCII string
# XXX investigate
# sometimes gets too big to fit in int value
if count != 0: # and count < (2**31): # 2E31 is hardware dependent. --gd
file_position = self.offset + offset
try:
self.file_handle.seek(file_position)
values = self.file_handle.read(count)
# Drop any garbage after a null.
values = values.split(b"\x00", 1)[0]
if isinstance(values, bytes):
try:
values = values.decode("utf-8")
except UnicodeDecodeError:
logger.warning(
"Possibly corrupted field %s in %s IFD", tag_name, ifd_name
)
except OverflowError:
logger.warning(
"OverflowError at position: %s, length: %s", file_position, count
)
values = ""
except MemoryError:
logger.warning(
"MemoryError at position: %s, length: %s", file_position, count
)
values = ""
return values
def _get_printable_for_field(
self,
count: int,
values: Union[str, list],
field_type: FieldType,
tag_entry: IfdDictValue,
stop_tag: str,
) -> Tuple[str, bool]:
# TODO: use only one type
if count == 1 and field_type != FieldType.ASCII:
printable = str(values[0])
elif count > 50 and len(values) > 20 and not isinstance(values, str):
if self.truncate_tags:
printable = str(values[0:20])[0:-1] + ", ... ]"
else:
printable = str(values[0:-1])
else:
printable = str(values)
prefer_printable = False
# compute printable version of values
if tag_entry:
# optional 2nd tag element is present
if tag_entry[1] is not None:
prefer_printable = True
# call mapping function
if callable(tag_entry[1]):
printable = tag_entry[1](values)
# handle sub-ifd
elif isinstance(tag_entry[1], tuple):
ifd_info = tag_entry[1]
try:
logger.debug("%s SubIFD at offset %d:", ifd_info[0], values[0])
self.dump_ifd(
ifd=values[0], # type: ignore
stop_tag=stop_tag,
ifd_name=ifd_info[0],
tag_dict=ifd_info[1],
)
except IndexError:
logger.warning("No values found for %s SubIFD", ifd_info[0])
else:
printable = ""
for val in values:
# use lookup table for this tag
printable += tag_entry[1].get(val, repr(val))
return printable, prefer_printable
def _process_tag(
self,
ifd: int,
ifd_name: str,
tag_entry: SubIfdTagDictValue,
entry: int,
tag: int,
tag_name: str,
relative: bool,
stop_tag: str,
) -> None:
field_type_id = self.s2n(entry + 2, 2)
try:
field_type = FieldType(field_type_id)
except ValueError as err:
if self.strict:
raise ValueError(
"Unknown field type %d in tag 0x%04X" % (field_type_id, tag)
) from err
return
# Not handled
if field_type == FieldType.PROPRIETARY:
if self.strict:
raise ValueError(
"Unknown field type %d in tag 0x%04X" % (field_type_id, tag)
)
return
type_length = FIELD_DEFINITIONS[field_type][0]
count = self.s2n(entry + 4, 4)
# Adjust for tag id/type/count (2+2+4 bytes)
# Now we point at either the data or the 2nd level offset
offset = entry + 8
# If the value fits in 4 bytes, it is inlined, else we
# need to jump ahead again.
if count * type_length > 4:
# offset is not the value; it's a pointer to the value
# if relative we set things up so s2n will seek to the right
# place when it adds self.offset. Note that this 'relative'
# is for the Nikon type 3 makernote. Other cameras may use
# other relative offsets, which would have to be computed here
# slightly differently.
if relative:
tmp_offset = self.s2n(offset, 4)
offset = tmp_offset + ifd - 8
if self.fake_exif:
offset += 18
else:
offset = self.s2n(offset, 4)
field_offset = offset
if field_type == FieldType.ASCII:
values = self._process_ascii_field(ifd_name, tag_name, count, offset)
else:
values = self._process_field(
tag_name, count, field_type, type_length, offset
)
printable, prefer_printable = self._get_printable_for_field(
count, values, field_type, tag_entry, stop_tag
)
self.tags[ifd_name + " " + tag_name] = IfdTag(
printable,
tag,
field_type,
values,
field_offset,
count * type_length,
prefer_printable,
)
tag_value = repr(self.tags[ifd_name + " " + tag_name])
logger.debug(" %s: %s", tag_name, tag_value)
def dump_ifd(
self,
ifd: int,
ifd_name: str,
tag_dict=None,
relative=0,
stop_tag=DEFAULT_STOP_TAG,
) -> None:
"""Return a list of entries in the given IFD."""
# make sure we can process the entries
if tag_dict is None:
tag_dict = EXIF_TAGS
try:
entries = self.s2n(ifd, 2)
except TypeError:
logger.warning("Possibly corrupted IFD: %s", ifd_name)
return
for i in range(entries):
# entry is index of start of this IFD in the file
entry = ifd + 2 + 12 * i
tag = self.s2n(entry, 2)
# get tag name early to avoid errors, help debug
tag_entry = tag_dict.get(tag)
if tag_entry:
tag_name = tag_entry[0]
else:
tag_name = f"Tag 0x{tag:04X}"
# ignore certain tags for faster processing
if not (not self.detailed and tag in IGNORE_TAGS):
self._process_tag(
ifd, ifd_name, tag_entry, entry, tag, tag_name, relative, stop_tag
)
if tag_name == stop_tag:
break
def extract_tiff_thumbnail(self, thumb_ifd: int) -> None:
"""
Extract uncompressed TIFF thumbnail.
Take advantage of the pre-existing layout in the thumbnail IFD as
much as possible
"""
thumb = self.tags.get("Thumbnail Compression")
if not thumb or thumb.printable != "Uncompressed TIFF":
return
entries = self.s2n(thumb_ifd, 2)
# this is header plus offset to IFD ...
if self.endian == "M":
tiff = b"MM\x00*\x00\x00\x00\x08"
else:
tiff = b"II*\x00\x08\x00\x00\x00"
# ... plus thumbnail IFD data plus a null "next IFD" pointer
self.file_handle.seek(self.offset + thumb_ifd)
tiff += self.file_handle.read(entries * 12 + 2) + b"\x00\x00\x00\x00"
# fix up large value offset pointers into data area
for i in range(entries):
entry = thumb_ifd + 2 + 12 * i
tag = self.s2n(entry, 2)
field_type = FieldType(self.s2n(entry + 2, 2))
type_length = FIELD_DEFINITIONS[field_type][0]
count = self.s2n(entry + 4, 4)
old_offset = self.s2n(entry + 8, 4)
# start of the 4-byte pointer area in entry
ptr = i * 12 + 18
# remember strip offsets
strip_len = 0
if tag == 0x0111:
strip_off = ptr
strip_len = count * type_length
# is it in the data area?
if count * type_length > 4:
# update offset pointer (nasty "strings are immutable" crap)
# should be able to say "tiff[ptr:ptr+4]=newoff"
newoff = len(tiff)
tiff = tiff[:ptr] + self.n2b(newoff, 4) + tiff[ptr + 4 :]
# remember strip offsets location
if tag == 0x0111:
strip_off = newoff
strip_len = 4
# get original data and store it
self.file_handle.seek(self.offset + old_offset)
tiff += self.file_handle.read(count * type_length)
# add pixel strips and update strip offset info
old_offsets = self.tags["Thumbnail StripOffsets"].values
old_counts = self.tags["Thumbnail StripByteCounts"].values
for i, old_offset in enumerate(old_offsets):
# update offset pointer (more nasty "strings are immutable" crap)
offset = self.n2b(len(tiff), strip_len)
tiff = tiff[:strip_off] + offset + tiff[strip_off + strip_len :]
strip_off += strip_len
# add pixel strip to end
self.file_handle.seek(self.offset + old_offset)
tiff += self.file_handle.read(old_counts[i])
self.tags["TIFFThumbnail"] = tiff
def extract_jpeg_thumbnail(self) -> None:
"""
Extract JPEG thumbnail.
(Thankfully the JPEG data is stored as a unit.)
"""
thumb_offset = self.tags.get("Thumbnail JPEGInterchangeFormat")
thumb_length = self.tags.get("Thumbnail JPEGInterchangeFormatLength")
if thumb_offset and thumb_length:
self.file_handle.seek(self.offset + thumb_offset.values[0])
size = thumb_length.values[0]
self.tags["JPEGThumbnail"] = self.file_handle.read(size)
# Sometimes in a TIFF file, a JPEG thumbnail is hidden in the MakerNote
# since it's not allowed in a uncompressed TIFF IFD
if "JPEGThumbnail" not in self.tags:
thumb_offset = self.tags.get("MakerNote JPEGThumbnail")
if thumb_offset:
self.file_handle.seek(self.offset + thumb_offset.values[0])
self.tags["JPEGThumbnail"] = self.file_handle.read(
thumb_offset.field_length
)
def decode_maker_note(self) -> None:
"""
Decode all the camera-specific MakerNote formats
Note is the data that comprises this MakerNote.
The MakerNote will likely have pointers in it that point to other
parts of the file. We'll use self.offset as the starting point for
most of those pointers, since they are relative to the beginning
of the file.
If the MakerNote is in a newer format, it may use relative addressing
within the MakerNote. In that case we'll use relative addresses for
the pointers.
As an aside: it's not just to be annoying that the manufacturers use
relative offsets. It's so that if the makernote has to be moved by the
picture software all of the offsets don't have to be adjusted. Overall,
this is probably the right strategy for makernotes, though the spec is
ambiguous.
The spec does not appear to imagine that makernotes would
follow EXIF format internally. Once they did, it's ambiguous whether
the offsets should be from the header at the start of all the EXIF info,
or from the header at the start of the makernote.
TODO: look into splitting this up
"""
note = self.tags["EXIF MakerNote"]
# Some apps use MakerNote tags but do not use a format for which we
# have a description, so just do a raw dump for these.
make = self.tags["Image Make"].printable
# Nikon
# The maker note usually starts with the word Nikon, followed by the
# type of the makernote (1 or 2, as a short). If the word Nikon is
# not at the start of the makernote, it's probably type 2, since some
# cameras work that way.
if "NIKON" in make:
if note.values[0:7] == [78, 105, 107, 111, 110, 0, 1]:
logger.debug("Looks like a type 1 Nikon MakerNote.")
self.dump_ifd(
ifd=note.field_offset + 8,
ifd_name="MakerNote",
tag_dict=nikon.TAGS_OLD,
)
elif note.values[0:7] == [78, 105, 107, 111, 110, 0, 2]:
logger.debug("Looks like a labeled type 2 Nikon MakerNote")
if note.values[12:14] != [0, 42] and note.values[12:14] != [42, 0]:
raise ValueError("Missing marker tag 42 in MakerNote.")
# skip the Makernote label and the TIFF header
self.dump_ifd(
ifd=note.field_offset + 10 + 8,
ifd_name="MakerNote",
tag_dict=nikon.TAGS_NEW,
relative=1,
)
else:
# E99x or D1
logger.debug("Looks like an unlabeled type 2 Nikon MakerNote")
self.dump_ifd(
ifd=note.field_offset, ifd_name="MakerNote", tag_dict=nikon.TAGS_NEW
)
return
# Olympus
if make.startswith("OLYMPUS"):
self.dump_ifd(
ifd=note.field_offset + 8, ifd_name="MakerNote", tag_dict=olympus.TAGS
)
# TODO
# for i in (('MakerNote Tag 0x2020', makernote.OLYMPUS_TAG_0x2020),):
# self.decode_olympus_tag(self.tags[i[0]].values, i[1])
# return
# Casio
if "CASIO" in make or "Casio" in make:
self.dump_ifd(
ifd=note.field_offset, ifd_name="MakerNote", tag_dict=casio.TAGS
)
return
if "SONY" in make:
self.dump_ifd(
ifd=note.field_offset, ifd_name="MakerNote", tag_dict=sony.TAGS
)
return
# Fujifilm
if make == "FUJIFILM":
# bug: everything else is "Motorola" endian, but the MakerNote
# is "Intel" endian
endian = self.endian
self.endian = "I"
# bug: IFD offsets are from beginning of MakerNote, not
# beginning of file header
offset = self.offset
self.offset += note.field_offset
# process note with bogus values (note is actually at offset 12)
self.dump_ifd(ifd=12, ifd_name="MakerNote", tag_dict=fujifilm.TAGS)
# reset to correct values
self.endian = endian
self.offset = offset
return
# Apple
if make == "Apple" and note.values[0:10] == [
65,
112,
112,
108,
101,
32,
105,
79,
83,
0,
]:
offset = self.offset
self.offset += note.field_offset + 14
self.dump_ifd(ifd=0, ifd_name="MakerNote", tag_dict=apple.TAGS)
self.offset = offset
return
if make == "DJI":
endian = self.endian
self.endian = "I"
offset = self.offset
self.offset += note.field_offset
self.dump_ifd(ifd=0, ifd_name="MakerNote", tag_dict=dji.TAGS)
self.offset = offset
self.endian = endian
return
# Canon
if make == "Canon":
self.dump_ifd(
ifd=note.field_offset, ifd_name="MakerNote", tag_dict=canon.TAGS
)
for tag_id, tags_dict in canon.OFFSET_TAGS.items():
tag_str = f"MakerNote Tag 0x{tag_id:04X}"
if tag_str in self.tags:
logger.debug("Canon %s", tag_str)
self._canon_decode_tag(self.tags[tag_str].values, tags_dict)
del self.tags[tag_str]
if canon.CAMERA_INFO_TAG_NAME in self.tags:
tag = self.tags[canon.CAMERA_INFO_TAG_NAME]
logger.debug("Canon CameraInfo")
self._canon_decode_camera_info(tag)
del self.tags[canon.CAMERA_INFO_TAG_NAME]
return
# TODO Decode Olympus MakerNote tag based on offset within tag.
# def _olympus_decode_tag(self, value, mn_tags):
# pass
def _canon_decode_tag(self, value, mn_tags: SubIfdTagDict) -> None:
"""
Decode Canon MakerNote tag based on offset within tag.
See http://www.burren.cx/david/canon.html by David Burren
"""
for tag_idx in range(1, len(value)):
tag_name, tag_format = mn_tags.get(tag_idx, ("Unknown", None))
if tag_format is not None:
if callable(tag_format):
val = tag_format(value[tag_idx])
elif isinstance(tag_format, dict):
val = tag_format.get(value[tag_idx], "Unknown")
else:
raise ExifError(f"Invalid tag type for Canon: {type(tag_format)}")
else:
val = value[tag_idx]
try:
logger.debug(" %s %s %s", tag_idx, tag_name, hex(value[tag_idx]))
except TypeError:
logger.debug(" %s %s %s", tag_idx, tag_name, value[tag_idx])
# It's not a real IFD Tag, but we fake one to make everybody happy.
# This will have a "proprietary" type
self.tags["MakerNote " + tag_name] = IfdTag(
printable=str(val),
tag=0,
field_type=FieldType.PROPRIETARY,
values=val,
field_offset=0,
field_length=0,
)
def _canon_decode_camera_info(self, camera_info_tag: IfdTag) -> None:
"""
Decode the variable length encoded camera info section.
"""
model_tag: Optional[IfdTag] = self.tags.get("Image Model", None)
if model_tag is None:
return
model = model_tag.printable
for model_name_re, tag_desc in canon.CAMERA_INFO_MODEL_MAP.items():
if re.search(model_name_re, model):
camera_info_tags = tag_desc
break
else:
return
# We are assuming here that these are all unsigned bytes
if camera_info_tag.field_type not in (FieldType.BYTE, FieldType.UNDEFINED):
return
camera_info = struct.pack(
"<%dB" % len(camera_info_tag.values), *camera_info_tag.values
)
# Look for each data value and decode it appropriately.
for offset, tag in camera_info_tags.items():
tag_name, tag_format, tag_func = tag
tag_size = struct.calcsize(tag_format)
if len(camera_info) < offset + tag_size:
continue
packed_tag_value = camera_info[offset : offset + tag_size]
tag_value = tag_func(struct.unpack(tag_format, packed_tag_value)[0])
logger.debug(" %s %s", tag_name, tag_value)
self.tags["MakerNote " + tag_name] = IfdTag(
printable=str(tag_value),
tag=0,
field_type=FieldType.PROPRIETARY,
values=tag_value,
field_offset=0,
field_length=0,
)
def parse_xmp(self, xmp_bytes: bytes):
"""Adobe's Extensible Metadata Platform, just dump the pretty XML."""
logger.debug("XMP cleaning data")
self.tags["Image ApplicationNotes"] = IfdTag(
xmp_bytes_to_str(xmp_bytes), 0, FieldType.BYTE, xmp_bytes, 0, 0
)
+118
View File
@@ -0,0 +1,118 @@
"""Utilities to find the EXIF offset and endian."""
import struct
from typing import BinaryIO, Dict, Tuple
from exifread.core.exceptions import ExifNotFound, InvalidExif
from exifread.core.heic import HEICExifFinder, find_heic_tiff
from exifread.core.jpeg import find_jpeg_exif
from exifread.core.jxl import JXLExifFinder
from exifread.core.utils import ord_
from exifread.exif_log import get_logger
logger = get_logger()
ENDIAN_TYPES: Dict[str, str] = {
"I": "Intel",
"M": "Motorola",
"\x01": "Adobe Ducky",
"b": "XMP/Adobe unknown",
}
def get_endian_str(endian_bytes) -> Tuple[str, str]:
endian_str = chr(ord_(endian_bytes[0]))
return endian_str, ENDIAN_TYPES.get(endian_str, "Unknown")
def find_tiff_exif(fh: BinaryIO) -> Tuple[int, bytes]:
logger.debug("TIFF format recognized in data[0:2]")
fh.seek(0)
endian = fh.read(1)
fh.read(1)
offset = 0
return offset, endian
def find_webp_exif(fh: BinaryIO) -> Tuple[int, bytes]:
logger.debug("WebP format recognized in data[0:4], data[8:12]")
# file specification: https://developers.google.com/speed/webp/docs/riff_container
data = fh.read(5)
if data[0:4] == b"VP8X" and data[4] & 8:
# https://developers.google.com/speed/webp/docs/riff_container#extended_file_format
fh.seek(13, 1)
while True:
data = fh.read(8) # Chunk FourCC (32 bits) and Chunk Size (32 bits)
if len(data) != 8:
raise InvalidExif("Invalid webp file chunk header.")
if data[0:4] == b"EXIF":
fh.seek(6, 1)
offset = fh.tell()
endian = fh.read(1)
return offset, endian
size = struct.unpack("<L", data[4:8])[0]
fh.seek(size, 1)
raise ExifNotFound("Webp file does not have exif data.")
def find_png_exif(fh: BinaryIO, data: bytes) -> Tuple[int, bytes]:
logger.debug("PNG format recognized in data[0:8]=%s", data[:8].hex())
fh.seek(8)
while True:
data = fh.read(8)
chunk = data[4:8]
logger.debug("PNG found chunk %s", chunk.decode("ascii"))
if chunk in (b"", b"IEND"):
break
if chunk == b"eXIf":
offset = fh.tell()
return offset, fh.read(1)
chunk_size = int.from_bytes(data[:4], "big")
fh.seek(fh.tell() + chunk_size + 4)
raise ExifNotFound("PNG file does not have exif data.")
def find_jxl_exif(fh: BinaryIO) -> Tuple[int, bytes]:
logger.debug("JPEG XL format recognized in data[0:12]")
fh.seek(0)
jxl = JXLExifFinder(fh)
offset, endian = jxl.find_exif()
if offset > 0:
return offset, endian
raise ExifNotFound("JPEG XL file does not have exif data.")
def determine_type(fh: BinaryIO) -> Tuple[int, bytes, int]:
# by default do not fake an EXIF beginning
fake_exif = 0
data = fh.read(12)
if data[0:2] in [b"II", b"MM"]:
# it's a TIFF file
offset, endian = find_tiff_exif(fh)
elif data[4:12] in [b"ftypheic", b"ftypavif", b"ftypmif1"]:
fh.seek(0)
heic = HEICExifFinder(fh)
offset, endian = heic.find_exif()
if offset == 0:
offset, endian = find_heic_tiff(fh)
# It's a HEIC file with a TIFF header
elif data[0:4] == b"RIFF" and data[8:12] == b"WEBP":
offset, endian = find_webp_exif(fh)
elif data[0:2] == b"\xff\xd8":
# it's a JPEG file
offset, endian, fake_exif = find_jpeg_exif(fh, data, fake_exif)
elif data[0:8] == b"\x89PNG\r\n\x1a\n":
offset, endian = find_png_exif(fh, data)
elif data == b"\0\0\0\x0cJXL\x20\x0d\x0a\x87\x0a":
offset, endian = find_jxl_exif(fh)
else:
raise ExifNotFound("File format not recognized.")
return offset, endian, fake_exif
+368
View File
@@ -0,0 +1,368 @@
"""
Find Exif data in an HEIC file.
As of 2019, the latest standard seems to be "ISO/IEC 14496-12:2015"
There are many different related standards. (quicktime, mov, mp4, etc...)
See https://en.wikipedia.org/wiki/ISO_base_media_file_format for more details.
We parse just enough of the ISO format to locate the Exif data in the file.
Inside the 'meta' box are two directories we need:
1) the 'iinf' box contains 'infe' records, we look for the item_id for 'Exif'.
2) once we have the item_id, we find a matching entry in the 'iloc' box, which
gives us position and size information.
"""
import struct
from typing import Any, BinaryIO, Callable, Dict, List, Optional, Tuple
from exifread.core.exceptions import ExifError, InvalidExif
from exifread.exif_log import get_logger
logger = get_logger()
def find_heic_tiff(fh: BinaryIO) -> Tuple[int, bytes]:
"""
Look for TIFF header in HEIC files.
In some HEIC files, the Exif offset is 0,
and yet there is a plain TIFF header near end of the file.
"""
data = fh.read(4)
if data[0:2] in [b"II", b"MM"] and data[2] == 42 and data[3] == 0:
offset = fh.tell() - 4
fh.seek(offset)
endian = data[0:2]
offset = fh.tell()
logger.debug("Found TIFF header in Exif, offset = %0xH", offset)
else:
raise InvalidExif(
"Exif pointer to zeros, but found "
+ str(data)
+ " instead of a TIFF header."
)
return offset, endian
class BoxVersion(ExifError):
"""Wrong box version."""
class BadSize(ExifError):
"""Wrong box size."""
class Box:
"""A HEIC Box."""
version = 0
minor_version = 0
item_count = 0
size = 0
after = 0
pos = 0
compat: List[bytes] = []
base_offset = 0
# this is full of boxes, but not in a predictable order.
subs: Dict[str, "Box"] = {}
locs: Dict = {}
exif_infe: Optional["Box"] = None
item_id = 0
item_type = b""
item_name = b""
item_protection_index = 0
major_brand = b""
offset_size = 0
length_size = 0
base_offset_size = 0
index_size = 0
flags = 0
name: str
def __init__(self, name: str) -> None:
self.name = name
def __repr__(self) -> str:
return "<box '%s'>" % self.name
def set_sizes(self, offset: int, length: int, base_offset: int, index: int) -> None:
self.offset_size = offset
self.length_size = length
self.base_offset_size = base_offset
self.index_size = index
def set_full(self, vflags: int) -> None:
"""
ISO boxes come in 'old' and 'full' variants.
The 'full' variant contains version and flags information.
"""
self.version = vflags >> 24
self.flags = vflags & 0x00FFFFFF
class HEICExifFinder:
"""Find HEIC EXIF tags."""
file_handle: BinaryIO
def __init__(self, file_handle: BinaryIO) -> None:
self.file_handle = file_handle
def get(self, nbytes: int) -> bytes:
read = self.file_handle.read(nbytes)
if not read:
raise EOFError
if len(read) != nbytes:
msg = "get(nbytes={nbytes}) found {read} bytes at position {pos}".format(
nbytes=nbytes, read=len(read), pos=self.file_handle.tell()
)
raise BadSize(msg)
return read
def get16(self) -> int:
return struct.unpack(">H", self.get(2))[0]
def get32(self) -> int:
return struct.unpack(">L", self.get(4))[0]
def get64(self) -> int:
return struct.unpack(">Q", self.get(8))[0]
def get_int4x2(self) -> tuple:
num = struct.unpack(">B", self.get(1))[0]
num0 = num >> 4
num1 = num & 0xF
return num0, num1
def get_int(self, size: int) -> int:
"""some fields have variant-sized data."""
if size == 2:
return self.get16()
if size == 4:
return self.get32()
if size == 8:
return self.get64()
if size == 0:
return 0
raise BadSize(size)
def get_string(self) -> bytes:
read = []
while 1:
char = self.get(1)
if char == b"\x00":
break
read.append(char)
return b"".join(read)
def next_box(self) -> Box:
pos = self.file_handle.tell()
size = self.get32()
kind = self.get(4).decode("ascii")
box = Box(kind)
if size == 0:
# signifies 'to the end of the file', we shouldn't see this.
raise NotImplementedError
if size == 1:
# 64-bit size follows type.
size = self.get64()
box.size = size - 16
box.after = pos + size
else:
box.size = size - 8
box.after = pos + size
box.pos = self.file_handle.tell()
return box
def get_full(self, box: Box) -> None:
box.set_full(self.get32())
def skip(self, box: Box) -> None:
self.file_handle.seek(box.after)
def expect_parse(self, name: str) -> Box:
while True:
box = self.next_box()
if box.name == name:
return self.parse_box(box)
self.skip(box)
def get_parser(self, box: Box) -> Optional[Callable[[Box], Any]]:
defs = {
"ftyp": self._parse_ftyp,
"meta": self._parse_meta,
"infe": self._parse_infe,
"iinf": self._parse_iinf,
"iloc": self._parse_iloc,
"hdlr": self._parse_hdlr, # HEIC/AVIF hdlr = Handler
"pitm": self._parse_pitm, # HEIC/AVIF pitm = Primary Item
"iref": self._parse_iref, # HEIC/AVIF idat = Item Reference
"idat": self._parse_idat, # HEIC/AVIF idat = Item Data Box
"dinf": self._parse_dinf, # HEIC/AVIF dinf = Data Information Box
"iprp": self._parse_iprp, # HEIC/AVIF iprp = Item Protection Box
}
return defs.get(box.name)
def parse_box(self, box: Box) -> Box:
probe = self.get_parser(box)
if probe is not None:
probe(box)
# in case anything is left unread
self.file_handle.seek(box.after)
return box
def _parse_ftyp(self, box: Box) -> None:
box.major_brand = self.get(4)
box.minor_version = self.get32()
box.compat = []
size = box.size - 8
while size > 0:
box.compat.append(self.get(4))
size -= 4
def _parse_meta(self, meta: Box) -> None:
self.get_full(meta)
while self.file_handle.tell() < meta.after:
box = self.next_box()
psub = self.get_parser(box)
if psub is not None:
psub(box)
meta.subs[box.name] = box
else:
logger.debug("HEIC: skipping %r", box)
# skip any unparsed data
self.skip(box)
def _parse_infe(self, box: Box) -> None:
self.get_full(box)
if box.version >= 2:
if box.version == 2:
box.item_id = self.get16()
elif box.version == 3:
box.item_id = self.get32()
box.item_protection_index = self.get16()
box.item_type = self.get(4)
box.item_name = self.get_string()
# ignore the rest
def _parse_iinf(self, box: Box) -> None:
self.get_full(box)
count = self.get16()
box.exif_infe = None
for _ in range(count):
infe = self.expect_parse("infe")
if infe.item_type == b"Exif":
logger.debug("HEIC: found Exif 'infe' box")
box.exif_infe = infe
break
def _parse_iloc(self, box: Box) -> None:
self.get_full(box)
size0, size1 = self.get_int4x2()
size2, size3 = self.get_int4x2()
box.set_sizes(size0, size1, size2, size3)
if box.version < 2:
box.item_count = self.get16()
elif box.version == 2:
box.item_count = self.get32()
else:
raise BoxVersion(2, box.version)
box.locs = {}
logger.debug("HEIC: %d iloc items", box.item_count)
for _ in range(box.item_count):
if box.version < 2:
item_id = self.get16()
elif box.version == 2:
item_id = self.get32()
else:
# notreached
raise BoxVersion(2, box.version)
if box.version in (1, 2):
# ignore construction_method
self.get16()
# ignore data_reference_index
self.get16()
box.base_offset = self.get_int(box.base_offset_size)
extent_count = self.get16()
extents = []
for _ in range(extent_count):
if box.version in (1, 2) and box.index_size > 0:
self.get_int(box.index_size)
extent_offset = self.get_int(box.offset_size)
extent_length = self.get_int(box.length_size)
extents.append((extent_offset, extent_length))
box.locs[item_id] = extents
# Added a few box names, which as unhandled aborted data extraction:
# hdlr, pitm, dinf, iprp, idat, iref
#
# Handling is initially `None`.
# They were found in .heif photo files produced by Nokia 8.3 5G.
#
# They are part of the standard, referring to:
# - ISO/IEC 14496-12 fifth edition 2015-02-20 (chapter 8.10 Metadata)
# found in:
# https://mpeg.chiariglione.org/standards/mpeg-4/iso-base-media-file-format/text-isoiec-14496-12-5th-edition
# (The newest is ISO/IEC 14496-12:2022, but would cost 208 Swiss Francs at iso.org)
# - A C++ example: https://exiv2.org/book/#BMFF
def _parse_hdlr(self, box: Box) -> None:
logger.debug("HEIC: found 'hdlr' Box %s, skipped", box.name)
def _parse_pitm(self, box: Box) -> None:
logger.debug("HEIC: found 'pitm' Box %s, skipped", box.name)
def _parse_dinf(self, box: Box) -> None:
logger.debug("HEIC: found 'dinf' Box %s, skipped", box.name)
def _parse_iprp(self, box: Box) -> None:
logger.debug("HEIC: found 'iprp' Box %s, skipped", box.name)
def _parse_idat(self, box: Box) -> None:
logger.debug("HEIC: found 'idat' Box %s, skipped", box.name)
def _parse_iref(self, box: Box) -> None:
logger.debug("HEIC: found 'iref' Box %s, skipped", box.name)
def find_exif(self) -> Tuple[int, bytes]:
ftyp = self.expect_parse("ftyp")
if (
ftyp.major_brand not in [b"heic", b"avif", b"mif1"]
or ftyp.minor_version != 0
):
return 0, b""
meta = self.expect_parse("meta")
if meta.subs["iinf"].exif_infe is None:
return 0, b""
item_id = meta.subs["iinf"].exif_infe.item_id
extents = meta.subs["iloc"].locs[item_id]
logger.debug("HEIC: found Exif location.")
# we expect the Exif data to be in one piece.
assert len(extents) == 1
pos, _ = extents[0]
# looks like there's a kind of pseudo-box here.
self.file_handle.seek(pos)
# the payload of "Exif" item may be start with either
# b'\xFF\xE1\xSS\xSSExif\x00\x00' (with APP1 marker, e.g. Android Q)
# or
# b'Exif\x00\x00' (without APP1 marker, e.g. iOS)
# according to "ISO/IEC 23008-12, 2017-12", both of them are legal
exif_tiff_header_offset = self.get32()
if exif_tiff_header_offset == 0:
# This case was found in HMD Nokia 8.3 5G heic photos.
# The TIFF header just sits there without any 'Exif'.
offset = 0
endian = b"?" # Haven't got Endian info yet
else:
assert exif_tiff_header_offset >= 6
assert self.get(exif_tiff_header_offset)[-6:] == b"Exif\x00\x00"
offset = self.file_handle.tell()
endian = self.file_handle.read(1)
return offset, endian
+57
View File
@@ -0,0 +1,57 @@
"""
Eases dealing with tags.
"""
from exifread.tags.fields import FIELD_DEFINITIONS, FieldType
class IfdTag:
"""
Represents an IFD tag.
"""
def __init__(
self,
printable: str,
tag: int,
field_type: FieldType,
values,
field_offset: int,
field_length: int,
prefer_printable: bool = True,
) -> None:
# printable version of data
self.printable = printable
# tag ID number
self.tag = tag
# field type as index into FIELD_TYPES
self.field_type = field_type
# offset of start of field in bytes from beginning of IFD
self.field_offset = field_offset
# length of data field in bytes
self.field_length = field_length
# either string, bytes or list of data items
# TODO: sort out this type mess!
self.values = values
# indication if printable version should be used upon serialization
self.prefer_printable = prefer_printable
def __str__(self) -> str:
return self.printable
def __repr__(self) -> str:
try:
tag = "(0x%04X) %s=%s @ %d" % (
self.tag,
FIELD_DEFINITIONS[self.field_type][1],
self.printable,
self.field_offset,
)
except TypeError:
tag = "(%s) %s=%s @ %s" % (
str(self.tag),
FIELD_DEFINITIONS[self.field_type][1],
self.printable,
str(self.field_offset),
)
return tag
+200
View File
@@ -0,0 +1,200 @@
"""Extract EXIF from JPEG files."""
from typing import BinaryIO, Tuple
from exifread.core.exceptions import InvalidExif
from exifread.core.utils import ord_
from exifread.exif_log import get_logger
logger = get_logger()
def _increment_base(data, base) -> int:
return ord_(data[base + 2]) * 256 + ord_(data[base + 3]) + 2
def _get_initial_base(fh: BinaryIO, data: bytes, fake_exif: int) -> Tuple[int, int]:
base = 2
logger.debug(
"data[2]=0x%X data[3]=0x%X data[6:10]=%s",
ord_(data[2]),
ord_(data[3]),
data[6:10],
)
while ord_(data[2]) == 0xFF and data[6:10] in (b"JFIF", b"JFXX", b"OLYM", b"Phot"):
length = ord_(data[4]) * 256 + ord_(data[5])
logger.debug(" Length offset is %s", length)
fh.read(length - 8)
# fake an EXIF beginning of file
# I don't think this is used. --gd
data = b"\xff\x00" + fh.read(10)
fake_exif = 1
if base > 2:
logger.debug(" Added to base")
base = base + length + 4 - 2
else:
logger.debug(" Added to zero")
base = length + 4
logger.debug(" Set segment base to 0x%X", base)
return base, fake_exif
def _get_base(base: int, data: bytes) -> int:
# pylint: disable=too-many-statements
while True:
logger.debug(" Segment base 0x%X", base)
if data[base : base + 2] == b"\xff\xe1":
# APP1
logger.debug(" APP1 at base 0x%X", base)
logger.debug(
" Length: 0x%X 0x%X", ord_(data[base + 2]), ord_(data[base + 3])
)
logger.debug(" Code: %s", data[base + 4 : base + 8])
if data[base + 4 : base + 8] == b"Exif":
logger.debug(
" Decrement base by 2 to get to pre-segment header (for compatibility with later code)"
)
base -= 2
break
increment = _increment_base(data, base)
logger.debug(" Increment base by %s", increment)
base += increment
elif data[base : base + 2] == b"\xff\xe0":
# APP0
logger.debug(" APP0 at base 0x%X", base)
logger.debug(
" Length: 0x%X 0x%X", ord_(data[base + 2]), ord_(data[base + 3])
)
logger.debug(" Code: %s", data[base + 4 : base + 8])
increment = _increment_base(data, base)
logger.debug(" Increment base by %s", increment)
base += increment
elif data[base : base + 2] == b"\xff\xe2":
# APP2
logger.debug(" APP2 at base 0x%X", base)
logger.debug(
" Length: 0x%X 0x%X", ord_(data[base + 2]), ord_(data[base + 3])
)
logger.debug(" Code: %s", data[base + 4 : base + 8])
increment = _increment_base(data, base)
logger.debug(" Increment base by %s", increment)
base += increment
elif data[base : base + 2] == b"\xff\xee":
# APP14
logger.debug(" APP14 Adobe segment at base 0x%X", base)
logger.debug(
" Length: 0x%X 0x%X", ord_(data[base + 2]), ord_(data[base + 3])
)
logger.debug(" Code: %s", data[base + 4 : base + 8])
increment = _increment_base(data, base)
logger.debug(" Increment base by %s", increment)
base += increment
logger.debug(
" There is useful EXIF-like data here, but we have no parser for it."
)
elif data[base : base + 2] == b"\xff\xdb":
logger.debug(
" JPEG image data at base 0x%X No more segments are expected.", base
)
break
elif data[base : base + 2] == b"\xff\xd8":
# APP12
logger.debug(" FFD8 segment at base 0x%X", base)
logger.debug(
" Got 0x%X 0x%X and %s instead",
ord_(data[base]),
ord_(data[base + 1]),
data[4 + base : 10 + base],
)
logger.debug(
" Length: 0x%X 0x%X", ord_(data[base + 2]), ord_(data[base + 3])
)
logger.debug(" Code: %s", data[base + 4 : base + 8])
increment = _increment_base(data, base)
logger.debug(" Increment base by %s", increment)
base += increment
elif data[base : base + 2] == b"\xff\xec":
# APP12
logger.debug(
" APP12 XMP (Ducky) or Pictureinfo segment at base 0x%X", base
)
logger.debug(
" Got 0x%X and 0x%X instead", ord_(data[base]), ord_(data[base + 1])
)
logger.debug(
" Length: 0x%X 0x%X", ord_(data[base + 2]), ord_(data[base + 3])
)
logger.debug("Code: %s", data[base + 4 : base + 8])
increment = _increment_base(data, base)
logger.debug(" Increment base by %s", increment)
base += increment
logger.debug(
(
" There is useful EXIF-like data here (quality, comment, copyright), "
"but we have no parser for it."
)
)
else:
try:
increment = _increment_base(data, base)
logger.debug(
" Got 0x%X and 0x%X instead",
ord_(data[base]),
ord_(data[base + 1]),
)
except IndexError as err:
raise InvalidExif(
"Unexpected/unhandled segment type or file content."
) from err
logger.debug(" Increment base by %s", increment)
base += increment
return base
def find_jpeg_exif(fh: BinaryIO, data: bytes, fake_exif: int) -> Tuple[int, bytes, int]:
logger.debug(
"JPEG format recognized data[0:2]=0x%X%X", ord_(data[0]), ord_(data[1])
)
base, fake_exif = _get_initial_base(fh, data, fake_exif)
# Big ugly patch to deal with APP2 (or other) data coming before APP1
fh.seek(0)
# in theory, this could be insufficient since 64K is the maximum size--gd
data = fh.read(base + 4000)
base = _get_base(base, data)
fh.seek(base + 12)
if ord_(data[2 + base]) == 0xFF and data[6 + base : 10 + base] == b"Exif":
# detected EXIF header
offset = fh.tell()
endian = fh.read(1)
# HACK TEST: endian = 'M'
elif ord_(data[2 + base]) == 0xFF and data[6 + base : 10 + base + 1] == b"Ducky":
# detected Ducky header.
logger.debug(
"EXIF-like header (normally 0xFF and code): 0x%X and %s",
ord_(data[2 + base]),
data[6 + base : 10 + base + 1],
)
offset = fh.tell()
endian = fh.read(1)
elif ord_(data[2 + base]) == 0xFF and data[6 + base : 10 + base + 1] == b"Adobe":
# detected APP14 (Adobe)
logger.debug(
"EXIF-like header (normally 0xFF and code): 0x%X and %s",
ord_(data[2 + base]),
data[6 + base : 10 + base + 1],
)
offset = fh.tell()
endian = fh.read(1)
else:
# no EXIF information
msg = "No EXIF header expected data[2+base]==0xFF and data[6+base:10+base]===Exif (or Duck)"
msg += " Did get 0x%X and %r" % (
ord_(data[2 + base]),
data[6 + base : 10 + base + 1],
)
raise InvalidExif(msg)
return offset, endian, fake_exif
+23
View File
@@ -0,0 +1,23 @@
"""
Find Exif data in a JPEG XL file
"""
from typing import Tuple
from exifread.core.heic import HEICExifFinder
class JXLExifFinder(HEICExifFinder):
"""Find JPEG XL EXIF tags."""
def find_exif(self) -> Tuple[int, bytes]:
ftyp = self.expect_parse("ftyp")
assert ftyp.major_brand == b"jxl "
assert ftyp.minor_version == 0
exif = self.expect_parse("Exif")
offset = exif.pos + 4
self.file_handle.seek(offset - 8)
assert self.get(8)[:6] == b"Exif\x00\x00"
endian = self.file_handle.read(1)
return offset, endian
+11
View File
@@ -0,0 +1,11 @@
"""
Misc utilities.
"""
from typing import Union
def ord_(dta: Union[str, int]) -> int:
if isinstance(dta, str):
return ord(dta)
return dta
+57
View File
@@ -0,0 +1,57 @@
"""XMP related utilities.."""
from pyexpat import ExpatError
from typing import BinaryIO
from xml.dom.minidom import parseString
from exifread.exif_log import get_logger
logger = get_logger()
def find_xmp_data(fh: BinaryIO) -> bytes:
xmp_bytes = b""
logger.debug("XMP not in Exif, searching file for XMP info...")
xml_started = False
xml_finished = False
for line in fh:
open_tag = line.find(b"<x:xmpmeta")
close_tag = line.find(b"</x:xmpmeta>")
if open_tag != -1:
xml_started = True
line = line[open_tag:]
logger.debug("XMP found opening tag at line position %s", open_tag)
if close_tag != -1:
logger.debug("XMP found closing tag at line position %s", close_tag)
line_offset = 0
if open_tag != -1:
line_offset = open_tag
line = line[: (close_tag - line_offset) + 12]
xml_finished = True
if xml_started:
xmp_bytes += line
if xml_finished:
break
logger.debug("Found %s XMP bytes", len(xmp_bytes))
return xmp_bytes
def xmp_bytes_to_str(xmp_bytes: bytes) -> str:
"""Adobe's Extensible Metadata Platform, just dump the pretty XML."""
logger.debug("Cleaning XMP data ...")
# Pray that it's encoded in UTF-8
# TODO: allow user to specify encoding
xmp_string = xmp_bytes.decode("utf-8")
try:
pretty = parseString(xmp_string).toprettyxml()
except ExpatError:
logger.warning("XMP: XML is not well formed")
return xmp_string
cleaned = []
for line in pretty.splitlines():
if line.strip():
cleaned.append(line)
return "\n".join(cleaned)
+96
View File
@@ -0,0 +1,96 @@
"""
Custom log output.
"""
import logging
import sys
TEXT_NORMAL = 0
TEXT_BOLD = 1
TEXT_RED = 31
TEXT_GREEN = 32
TEXT_YELLOW = 33
TEXT_BLUE = 34
TEXT_MAGENTA = 35
TEXT_CYAN = 36
def get_logger() -> logging.Logger:
"""Use this from all files needing to log."""
return logging.getLogger("exifread")
def setup_logger(debug: bool, color: bool) -> None:
"""Configure the logger."""
if debug:
log_level = logging.DEBUG
else:
log_level = logging.INFO
logger = get_logger()
stream = Handler(log_level, debug, color)
logger.addHandler(stream)
logger.setLevel(log_level)
class Formatter(logging.Formatter):
"""
Custom formatter, we like colors!
"""
color: bool
debug: bool
def __init__(self, debug: bool = False, color: bool = False) -> None:
self.color = color
self.debug = debug
if self.debug:
log_format = "%(levelname)-6s %(message)s"
else:
log_format = "%(message)s"
logging.Formatter.__init__(self, log_format)
def format(self, record):
if self.debug and self.color:
if record.levelno >= logging.CRITICAL:
color = TEXT_RED
elif record.levelno >= logging.ERROR:
color = TEXT_RED
elif record.levelno >= logging.WARNING:
color = TEXT_YELLOW
elif record.levelno >= logging.INFO:
color = TEXT_GREEN
elif record.levelno >= logging.DEBUG:
color = TEXT_CYAN
else:
color = TEXT_NORMAL
record.levelname = "\x1b[%sm%s\x1b[%sm" % (
color,
record.levelname,
TEXT_NORMAL,
)
return logging.Formatter.format(self, record)
class Handler(logging.StreamHandler):
"""
Custom StreamHandler so we can use the Formatter.
"""
color: bool
debug: bool
def __init__(
self, log_level: int, debug: bool = False, color: bool = False
) -> None:
self.color = color
self.debug = debug
logging.StreamHandler.__init__(self, sys.stdout)
self.setFormatter(Formatter(debug, color))
self.setLevel(log_level)
# def emit(self, record):
# record.msg = "\x1b[%sm%s\x1b[%sm" % (TEXT_BOLD, record.msg, TEXT_NORMAL)
# logging.StreamHandler.emit(self, record)
View File
+254
View File
@@ -0,0 +1,254 @@
"""
Enable conversion of Exif IfdTags to native Python types
"""
from typing import Callable, Dict, List, Union
from exifread.core.exif_header import IfdTag
from exifread.exif_log import get_logger
from exifread.tags.fields import FieldType
logger = get_logger()
SerializedTagValue = Union[int, float, str, bytes, List[int], List[float], None]
SerializedTagDict = Dict[str, SerializedTagValue]
def convert_types(
exif_tags: Dict[str, Union[IfdTag, bytes]],
) -> SerializedTagDict:
"""
Convert Exif IfdTags to built-in Python types for easier serialization and programmatic use.
- If the printable value of the IfdTag is relevant (e.g. enum type), it is preserved.
- Otherwise, values are processed based on their field type, with some cleanups applied.
- Single-element lists are unpacked to return the item directly.
"""
output: SerializedTagDict = {}
for tag_name, ifd_tag in exif_tags.items():
# JPEGThumbnail and TIFFThumbnail are the only values
# in Exif Tags dict that do not have the IfdTag type.
if isinstance(ifd_tag, bytes):
output[tag_name] = ifd_tag
continue
convert_func: Callable[[IfdTag, str], SerializedTagValue]
if ifd_tag.prefer_printable:
# Prioritize the printable value if prefer_printable is set
convert_func = convert_proprietary
else:
# Get the conversion function based on field type
try:
convert_func = conversion_map[ifd_tag.field_type]
except KeyError:
logger.error(
"Type conversion for field type %s not explicitly supported",
ifd_tag.field_type,
)
convert_func = convert_proprietary # Fallback to printable
output[tag_name] = convert_func(ifd_tag, tag_name)
# Useful only for testing
# logger.warning(
# f"{convert_func.__name__}: {ifd_tag.field_type} to {type(output[tag_name]).__name__}\n"
# f"{tag_name} --> {str(output[tag_name])[:30]!r}"
# )
return output
def convert_ascii(ifd_tag: IfdTag, tag_name: str) -> Union[str, bytes, None]:
"""
Handle ASCII conversion, including special date formats.
Returns:
- str
- bytes for rare ascii sequences that aren't Unicode
- None for empty values
"""
out = ifd_tag.values
# Handle DateTime formatting; often formatted in a way that cannot
# be parsed by Python dateutil (%Y:%m:%d %H:%M:%S).
if "DateTime" in tag_name and len(out) == 19 and out.count(":") == 4:
out = out.replace(":", "-", 2)
# Handle GPSDate formatting; these are proper dates with the wrong
# delimiter (':' rather than '-'). Invalid values have been found
# in test images: '' and '2014:09:259'
elif tag_name == "GPS GPSDate" and len(out) == 10 and out.count(":") == 2:
out = out.replace(":", "-")
# Strip occasional trailing whitespaces
out = out.strip()
if not out:
return None
# Attempt to decode bytes if unicode
if isinstance(out, bytes):
try:
return out.decode()
except UnicodeDecodeError:
pass
return out
def convert_undefined(ifd_tag: IfdTag, _tag_name: str) -> Union[bytes, str, int, None]:
"""
Handle Undefined type conversion.
Returns:
- bytes if not Unicode such as Exif MakerNote
- str for Unicode
- int for rare MakerNote Tags containing a single value
- None for empty values such as some MakerNote Tags
"""
out = ifd_tag.values
if len(out) == 1:
# Return integer from single-element list
return out[0]
# These contain bytes represented as a list of integers, sometimes with surrounded by spaces and/or null bytes
out = bytes(out).strip(b" \x00")
if not out:
return None
# Empty byte sequences or Unicode values should be decoded as strings
try:
return out.decode()
except UnicodeDecodeError:
return out
def convert_numeric(ifd_tag: IfdTag, _tag_name: str) -> Union[int, List[int], None]:
"""
Handle numeric types conversion.
Returns:
- int in most cases
- list of int
- None for empty values such as some MakerNote Tags
Note: All Floating Point tags seen were empty.
"""
out = ifd_tag.values
if not out: # Empty lists, seen in floating point numbers
return None
return out[0] if len(out) == 1 else out
def convert_ratio(
ifd_tag: IfdTag, _tag_name: str
) -> Union[int, float, List[int], List[float], None]:
"""
Handle Ratio and Signed Ratio conversion.
Returns:
- int when the denominator is 1 or unused
- float otherwise
- a list of int or float, such as GPS Latitude/Longitude/TimeStamp
- None for empty values such as some MakerNote Tags
Ratios can be re-created with `Ratio(float_value).limit_denominator()`.
"""
out = []
for ratio in ifd_tag.values:
# Prevent division by 0. Sometimes, EXIF data is full of 0s when a feature is unused.
if ratio.denominator == 0:
ratio = ratio.numerator
ratio = float(ratio)
if ratio.is_integer():
ratio = int(ratio)
out.append(ratio)
if not out:
return None
return out[0] if len(out) == 1 else out
def convert_bytes(ifd_tag: IfdTag, tag_name: str) -> Union[bytes, str, int, None]:
"""
Handle Byte and Signed Byte conversion.
Returns:
- bytes
- str for Unicode such as GPSVersionID and Image ApplicationNotes (XML)
- int for single byte values such as GPSAltitudeRef or some MakerNote fields
- None for empty values such as some MakerNote Tags
"""
out = ifd_tag.values
if len(out) == 1:
# Byte can be a single integer, such as GPSAltitudeRef (0 or 1)
return out[0]
if tag_name == "GPS GPSVersionID":
return ".".join(map(str, out)) # e.g. [2, 3, 0, 0] --> '2.3.0.0'
# Byte sequences are often surrounded by or only composed of spaces and/or null bytes
out = bytes(out).strip(b" \x00")
if not out:
return None
# Unicode values should be decoded as strings (e.g. XML)
try:
return out.decode()
except UnicodeDecodeError:
return out
def convert_proprietary(ifd_tag: IfdTag, _tag_name: str) -> Union[str, None]:
"""
Handle Proprietary type conversion.
Returns:
- str as all tags of this made-up type (e.g. enums) prefer printable
- None for very rare empty printable values
"""
out = ifd_tag.printable
if not out or out == "[]":
return None
return out
# Mapping of field type to conversion function
conversion_map: Dict[FieldType, Callable] = {
FieldType.PROPRIETARY: convert_proprietary,
FieldType.BYTE: convert_bytes,
FieldType.ASCII: convert_ascii,
FieldType.SHORT: convert_numeric,
FieldType.LONG: convert_numeric,
FieldType.RATIO: convert_ratio,
FieldType.SIGNED_BYTE: convert_numeric,
FieldType.UNDEFINED: convert_undefined,
FieldType.SIGNED_SHORT: convert_numeric,
FieldType.SIGNED_LONG: convert_numeric,
FieldType.SIGNED_RATIO: convert_ratio,
FieldType.FLOAT_32: convert_numeric,
FieldType.FLOAT_64: convert_numeric,
FieldType.IFD: convert_bytes,
}
+22
View File
@@ -0,0 +1,22 @@
"""
Tag definitions
"""
from typing import Callable, Dict, List, Tuple, Union
DEFAULT_STOP_TAG = "UNDEF"
# To ignore when quick processing
IGNORE_TAGS: List[int] = [
0x02BC, # XPM
0x927C, # MakerNote Tags
0x9286, # user comment
]
SubIfdTagDictValue = Tuple[str, Union[dict, Callable, None]]
SubIfdTagDict = Dict[int, SubIfdTagDictValue]
SubIfdInfoTuple = Tuple[str, SubIfdTagDict]
IfdDictValue = Union[Tuple[str, SubIfdInfoTuple], SubIfdTagDictValue]
IfdTagDict = Dict[int, IfdDictValue]
+443
View File
@@ -0,0 +1,443 @@
"""
Standard tag definitions.
"""
from exifread.tags import IfdTagDict, SubIfdInfoTuple, SubIfdTagDict
from exifread.tags.str_utils import make_string, make_string_uc
# Interoperability tags
INTEROP_TAGS: SubIfdTagDict = {
0x0001: ("InteroperabilityIndex", None),
0x0002: ("InteroperabilityVersion", None),
0x1000: ("RelatedImageFileFormat", None),
0x1001: ("RelatedImageWidth", None),
0x1002: ("RelatedImageLength", None),
}
INTEROP_INFO: SubIfdInfoTuple = ("Interoperability", INTEROP_TAGS)
# GPS tags
GPS_TAGS: SubIfdTagDict = {
0x0000: ("GPSVersionID", None),
0x0001: ("GPSLatitudeRef", None),
0x0002: ("GPSLatitude", None),
0x0003: ("GPSLongitudeRef", None),
0x0004: ("GPSLongitude", None),
0x0005: ("GPSAltitudeRef", None),
0x0006: ("GPSAltitude", None),
0x0007: ("GPSTimeStamp", None),
0x0008: ("GPSSatellites", None),
0x0009: ("GPSStatus", None),
0x000A: ("GPSMeasureMode", None),
0x000B: ("GPSDOP", None),
0x000C: ("GPSSpeedRef", None),
0x000D: ("GPSSpeed", None),
0x000E: ("GPSTrackRef", None),
0x000F: ("GPSTrack", None),
0x0010: ("GPSImgDirectionRef", None),
0x0011: ("GPSImgDirection", None),
0x0012: ("GPSMapDatum", None),
0x0013: ("GPSDestLatitudeRef", None),
0x0014: ("GPSDestLatitude", None),
0x0015: ("GPSDestLongitudeRef", None),
0x0016: ("GPSDestLongitude", None),
0x0017: ("GPSDestBearingRef", None),
0x0018: ("GPSDestBearing", None),
0x0019: ("GPSDestDistanceRef", None),
0x001A: ("GPSDestDistance", None),
0x001B: ("GPSProcessingMethod", None),
0x001C: ("GPSAreaInformation", None),
0x001D: ("GPSDate", None),
0x001E: ("GPSDifferential", None),
}
GPS_INFO: SubIfdInfoTuple = ("GPS", GPS_TAGS)
# Main Exif tag names
EXIF_TAGS: IfdTagDict = {
0x00FE: (
"SubfileType",
{
0x0: "Full-resolution Image",
0x1: "Reduced-resolution image",
0x2: "Single page of multi-page image",
0x3: "Single page of multi-page reduced-resolution image",
0x4: "Transparency mask",
0x5: "Transparency mask of reduced-resolution image",
0x6: "Transparency mask of multi-page image",
0x7: "Transparency mask of reduced-resolution multi-page image",
0x10001: "Alternate reduced-resolution image",
0xFFFFFFFF: "invalid ",
},
),
0x00FF: (
"OldSubfileType",
{
1: "Full-resolution image",
2: "Reduced-resolution image",
3: "Single page of multi-page image",
},
),
0x0100: ("ImageWidth", None),
0x0101: ("ImageLength", None),
0x0102: ("BitsPerSample", None),
0x0103: (
"Compression",
{
1: "Uncompressed",
2: "CCITT 1D",
3: "T4/Group 3 Fax",
4: "T6/Group 4 Fax",
5: "LZW",
6: "JPEG (old-style)",
7: "JPEG",
8: "Adobe Deflate",
9: "JBIG B&W",
10: "JBIG Color",
32766: "Next",
32769: "Epson ERF Compressed",
32771: "CCIRLEW",
32773: "PackBits",
32809: "Thunderscan",
32895: "IT8CTPAD",
32896: "IT8LW",
32897: "IT8MP",
32898: "IT8BL",
32908: "PixarFilm",
32909: "PixarLog",
32946: "Deflate",
32947: "DCS",
34661: "JBIG",
34676: "SGILog",
34677: "SGILog24",
34712: "JPEG 2000",
34713: "Nikon NEF Compressed",
65000: "Kodak DCR Compressed",
65535: "Pentax PEF Compressed",
},
),
0x0106: ("PhotometricInterpretation", None),
0x0107: ("Thresholding", None),
0x0108: ("CellWidth", None),
0x0109: ("CellLength", None),
0x010A: ("FillOrder", None),
0x010D: ("DocumentName", None),
0x010E: ("ImageDescription", None),
0x010F: ("Make", None),
0x0110: ("Model", None),
0x0111: ("StripOffsets", None),
0x0112: (
"Orientation",
{
1: "Horizontal (normal)",
2: "Mirrored horizontal",
3: "Rotated 180",
4: "Mirrored vertical",
5: "Mirrored horizontal then rotated 90 CCW",
6: "Rotated 90 CW",
7: "Mirrored horizontal then rotated 90 CW",
8: "Rotated 90 CCW",
},
),
0x0115: ("SamplesPerPixel", None),
0x0116: ("RowsPerStrip", None),
0x0117: ("StripByteCounts", None),
0x0118: ("MinSampleValue", None),
0x0119: ("MaxSampleValue", None),
0x011A: ("XResolution", None),
0x011B: ("YResolution", None),
0x011C: ("PlanarConfiguration", None),
0x011D: ("PageName", make_string),
0x011E: ("XPosition", None),
0x011F: ("YPosition", None),
0x0122: (
"GrayResponseUnit",
{
1: "0.1",
2: "0.001",
3: "0.0001",
4: "1e-05",
5: "1e-06",
},
),
0x0123: ("GrayResponseCurve", None),
0x0124: ("T4Options", None),
0x0125: ("T6Options", None),
0x0128: (
"ResolutionUnit",
{1: "Not Absolute", 2: "Pixels/Inch", 3: "Pixels/Centimeter"},
),
0x0129: ("PageNumber", None),
0x012C: ("ColorResponseUnit", None),
0x012D: ("TransferFunction", None),
0x0131: ("Software", None),
0x0132: ("DateTime", None),
0x013B: ("Artist", None),
0x013C: ("HostComputer", None),
0x013D: ("Predictor", {1: "None", 2: "Horizontal differencing"}),
0x013E: ("WhitePoint", None),
0x013F: ("PrimaryChromaticities", None),
0x0140: ("ColorMap", None),
0x0141: ("HalftoneHints", None),
0x0142: ("TileWidth", None),
0x0143: ("TileLength", None),
0x0144: ("TileOffsets", None),
0x0145: ("TileByteCounts", None),
0x0146: ("BadFaxLines", None),
0x0147: ("CleanFaxData", {0: "Clean", 1: "Regenerated", 2: "Unclean"}),
0x014A: ("SubIFDs", None),
0x0148: ("ConsecutiveBadFaxLines", None),
0x014C: ("InkSet", {1: "CMYK", 2: "Not CMYK"}),
0x014D: ("InkNames", None),
0x014E: ("NumberofInks", None),
0x0150: ("DotRange", None),
0x0151: ("TargetPrinter", None),
0x0152: (
"ExtraSamples",
{0: "Unspecified", 1: "Associated Alpha", 2: "Unassociated Alpha"},
),
0x0153: (
"SampleFormat",
{
1: "Unsigned",
2: "Signed",
3: "Float",
4: "Undefined",
5: "Complex int",
6: "Complex float",
},
),
0x0154: ("SMinSampleValue", None),
0x0155: ("SMaxSampleValue", None),
0x0156: ("TransferRange", None),
0x0157: ("ClipPath", None),
0x015B: ("JPEGTables", None),
0x0200: ("JPEGProc", None),
0x0201: ("JPEGInterchangeFormat", None), # JpegIFOffset
0x0202: ("JPEGInterchangeFormatLength", None), # JpegIFByteCount
0x0211: ("YCbCrCoefficients", None),
0x0212: ("YCbCrSubSampling", None),
0x0213: ("YCbCrPositioning", {1: "Centered", 2: "Co-sited"}),
0x0214: ("ReferenceBlackWhite", None),
0x02BC: ("ApplicationNotes", None), # XPM Info
0x4746: ("Rating", None),
0x828D: ("CFARepeatPatternDim", None),
0x828E: ("CFAPattern", None),
0x828F: ("BatteryLevel", None),
0x8298: ("Copyright", None),
0x829A: ("ExposureTime", None),
0x829D: ("FNumber", None),
0x83BB: ("IPTC/NAA", None),
0x8769: ("ExifOffset", None), # Exif Tags
0x8773: ("InterColorProfile", None),
0x8822: (
"ExposureProgram",
{
0: "Unidentified",
1: "Manual",
2: "Program Normal",
3: "Aperture Priority",
4: "Shutter Priority",
5: "Program Creative",
6: "Program Action",
7: "Portrait Mode",
8: "Landscape Mode",
},
),
0x8824: ("SpectralSensitivity", None),
0x8825: ("GPSInfo", GPS_INFO), # GPS tags
0x8827: ("ISOSpeedRatings", None),
0x8828: ("OECF", None),
0x8829: ("Interlace", None),
0x882A: ("TimeZoneOffset", None),
0x882B: ("SelfTimerMode", None),
0x8830: (
"SensitivityType",
{
0: "Unknown",
1: "Standard Output Sensitivity",
2: "Recommended Exposure Index",
3: "ISO Speed",
4: "Standard Output Sensitivity and Recommended Exposure Index",
5: "Standard Output Sensitivity and ISO Speed",
6: "Recommended Exposure Index and ISO Speed",
7: "Standard Output Sensitivity, Recommended Exposure Index and ISO Speed",
},
),
0x8832: ("RecommendedExposureIndex", None),
0x8833: ("ISOSpeed", None),
0x9000: ("ExifVersion", make_string),
0x9003: ("DateTimeOriginal", None),
0x9004: ("DateTimeDigitized", None),
0x9010: ("OffsetTime", None),
0x9011: ("OffsetTimeOriginal", None),
0x9012: ("OffsetTimeDigitized", None),
0x9101: (
"ComponentsConfiguration",
{0: "", 1: "Y", 2: "Cb", 3: "Cr", 4: "Red", 5: "Green", 6: "Blue"},
),
0x9102: ("CompressedBitsPerPixel", None),
0x9201: ("ShutterSpeedValue", None),
0x9202: ("ApertureValue", None),
0x9203: ("BrightnessValue", None),
0x9204: ("ExposureBiasValue", None),
0x9205: ("MaxApertureValue", None),
0x9206: ("SubjectDistance", None),
0x9207: (
"MeteringMode",
{
0: "Unidentified",
1: "Average",
2: "CenterWeightedAverage",
3: "Spot",
4: "MultiSpot",
5: "Pattern",
6: "Partial",
255: "other",
},
),
0x9208: (
"LightSource",
{
0: "Unknown",
1: "Daylight",
2: "Fluorescent",
3: "Tungsten (incandescent light)",
4: "Flash",
9: "Fine weather",
10: "Cloudy weather",
11: "Shade",
12: "Daylight fluorescent (D 5700 - 7100K)",
13: "Day white fluorescent (N 4600 - 5400K)",
14: "Cool white fluorescent (W 3900 - 4500K)",
15: "White fluorescent (WW 3200 - 3700K)",
17: "Standard light A",
18: "Standard light B",
19: "Standard light C",
20: "D55",
21: "D65",
22: "D75",
23: "D50",
24: "ISO studio tungsten",
255: "other light source",
},
),
0x9209: (
"Flash",
{
0: "Flash did not fire",
1: "Flash fired",
5: "Strobe return light not detected",
7: "Strobe return light detected",
9: "Flash fired, compulsory flash mode",
13: "Flash fired, compulsory flash mode, return light not detected",
15: "Flash fired, compulsory flash mode, return light detected",
16: "Flash did not fire, compulsory flash mode",
24: "Flash did not fire, auto mode",
25: "Flash fired, auto mode",
29: "Flash fired, auto mode, return light not detected",
31: "Flash fired, auto mode, return light detected",
32: "No flash function",
65: "Flash fired, red-eye reduction mode",
69: "Flash fired, red-eye reduction mode, return light not detected",
71: "Flash fired, red-eye reduction mode, return light detected",
73: "Flash fired, compulsory flash mode, red-eye reduction mode",
77: "Flash fired, compulsory flash mode, red-eye reduction mode, return light not detected",
79: "Flash fired, compulsory flash mode, red-eye reduction mode, return light detected",
89: "Flash fired, auto mode, red-eye reduction mode",
93: "Flash fired, auto mode, return light not detected, red-eye reduction mode",
95: "Flash fired, auto mode, return light detected, red-eye reduction mode",
},
),
0x920A: ("FocalLength", None),
0x920B: ("FlashEnergy", None),
0x920C: ("SpatialFrequencyResponse", None),
0x920D: ("Noise", None),
0x9211: ("ImageNumber", None),
0x9212: ("SecurityClassification", None),
0x9213: ("ImageHistory", None),
0x9214: ("SubjectArea", None),
0x9215: ("ExposureIndex", None),
0x9216: ("TIFF/EPStandardID", None),
0x927C: ("MakerNote", None),
0x9286: ("UserComment", make_string_uc),
0x9290: ("SubSecTime", None),
0x9291: ("SubSecTimeOriginal", None),
0x9292: ("SubSecTimeDigitized", None),
# used by Windows Explorer
0x9C9B: ("XPTitle", None),
0x9C9C: ("XPComment", None),
0x9C9D: ("XPAuthor", make_string), # (ignored by Windows Explorer if Artist exists)
0x9C9E: ("XPKeywords", None),
0x9C9F: ("XPSubject", None),
0xA000: ("FlashPixVersion", make_string),
0xA001: ("ColorSpace", {1: "sRGB", 2: "Adobe RGB", 65535: "Uncalibrated"}),
0xA002: ("ExifImageWidth", None),
0xA003: ("ExifImageLength", None),
0xA004: ("RelatedSoundFile", None),
0xA005: ("InteroperabilityOffset", INTEROP_INFO),
0xA20B: ("FlashEnergy", None), # 0x920B in TIFF/EP
0xA20C: ("SpatialFrequencyResponse", None), # 0x920C
0xA20E: ("FocalPlaneXResolution", None), # 0x920E
0xA20F: ("FocalPlaneYResolution", None), # 0x920F
0xA210: ("FocalPlaneResolutionUnit", None), # 0x9210
0xA214: ("SubjectLocation", None), # 0x9214
0xA215: ("ExposureIndex", None), # 0x9215
0xA217: (
"SensingMethod",
{ # 0x9217
1: "Not defined",
2: "One-chip color area",
3: "Two-chip color area",
4: "Three-chip color area",
5: "Color sequential area",
7: "Trilinear",
8: "Color sequential linear",
},
),
0xA300: (
"FileSource",
{1: "Film Scanner", 2: "Reflection Print Scanner", 3: "Digital Camera"},
),
0xA301: ("SceneType", {1: "Directly Photographed"}),
0xA302: ("CVAPattern", None),
0xA401: ("CustomRendered", {0: "Normal", 1: "Custom"}),
0xA402: (
"ExposureMode",
{0: "Auto Exposure", 1: "Manual Exposure", 2: "Auto Bracket"},
),
0xA403: ("WhiteBalance", {0: "Auto", 1: "Manual"}),
0xA404: ("DigitalZoomRatio", None),
0xA405: ("FocalLengthIn35mmFilm", None),
0xA406: (
"SceneCaptureType",
{0: "Standard", 1: "Landscape", 2: "Portrait", 3: "Night"},
),
0xA407: (
"GainControl",
{
0: "None",
1: "Low gain up",
2: "High gain up",
3: "Low gain down",
4: "High gain down",
},
),
0xA408: ("Contrast", {0: "Normal", 1: "Soft", 2: "Hard"}),
0xA409: ("Saturation", {0: "Normal", 1: "Soft", 2: "Hard"}),
0xA40A: ("Sharpness", {0: "Normal", 1: "Soft", 2: "Hard"}),
0xA40B: ("DeviceSettingDescription", None),
0xA40C: ("SubjectDistanceRange", None),
0xA420: ("ImageUniqueID", None),
0xA430: ("CameraOwnerName", None),
0xA431: ("BodySerialNumber", None),
0xA432: ("LensSpecification", None),
0xA433: ("LensMake", None),
0xA434: ("LensModel", None),
0xA435: ("LensSerialNumber", None),
0xA500: ("Gamma", None),
0xC4A5: ("PrintIM", None),
0xC61A: ("BlackLevel", None),
0xEA1C: ("Padding", None),
0xEA1D: ("OffsetSchema", None),
0xFDE8: ("OwnerName", None),
0xFDE9: ("SerialNumber", None),
}
+53
View File
@@ -0,0 +1,53 @@
"""Field type definitions."""
from enum import IntEnum
from typing import Dict, List, Tuple
class FieldType(IntEnum):
"""Field types mapped to their integer representation."""
PROPRIETARY = 0
BYTE = 1
ASCII = 2
SHORT = 3
LONG = 4
RATIO = 5
SIGNED_BYTE = 6
UNDEFINED = 7
SIGNED_SHORT = 8
SIGNED_LONG = 9
SIGNED_RATIO = 10
FLOAT_32 = 11
FLOAT_64 = 12
IFD = 13
SIGNED_FIELD_TYPES: List[FieldType] = [
FieldType.SIGNED_BYTE,
FieldType.SIGNED_SHORT,
FieldType.SIGNED_LONG,
FieldType.SIGNED_RATIO,
]
RATIO_FIELD_TYPES: List[FieldType] = [FieldType.RATIO, FieldType.SIGNED_RATIO]
FLOAT_FIELD_TYPES: List[FieldType] = [FieldType.FLOAT_32, FieldType.FLOAT_64]
# Field type descriptions as (length, full name) tuples
FIELD_DEFINITIONS: Dict[FieldType, Tuple[int, str]] = {
FieldType.PROPRIETARY: (0, "Proprietary"),
FieldType.BYTE: (1, "Byte"),
FieldType.ASCII: (1, "ASCII"),
FieldType.SHORT: (2, "Short"),
FieldType.LONG: (4, "Long"),
FieldType.RATIO: (8, "Ratio"),
FieldType.SIGNED_BYTE: (1, "Signed Byte"),
FieldType.UNDEFINED: (1, "Undefined"),
FieldType.SIGNED_SHORT: (2, "Signed Short"),
FieldType.SIGNED_LONG: (4, "Signed Long"),
FieldType.SIGNED_RATIO: (8, "Signed Ratio"),
FieldType.FLOAT_32: (4, "Single-Precision Floating Point (32-bit)"),
FieldType.FLOAT_64: (8, "Double-Precision Floating Point (64-bit)"),
FieldType.IFD: (4, "IFD"),
}
+3
View File
@@ -0,0 +1,3 @@
"""
Makernote tag definitions.
"""
+54
View File
@@ -0,0 +1,54 @@
"""
Makernote (proprietary) tag definitions for Apple iOS
Based on version 1.01 of ExifTool -> Image/ExifTool/Apple.pm
http://owl.phy.queensu.ca/~phil/exiftool/
"""
from exifread.tags import SubIfdTagDict
TAGS: SubIfdTagDict = {
0x0001: ("MakerNoteVersion", None),
0x0004: (
"AEStable",
{
0: "No",
1: "Yes",
},
),
0x0005: ("AETarget", None),
0x0006: ("AEAverage", None),
0x0007: (
"AFStable",
{
0: "No",
1: "Yes",
},
),
0x000A: (
"HDRImageType",
{
3: "HDR Image",
4: "Original Image",
},
),
0x0014: (
"ImageCaptureType",
{
1: "ProRAW",
2: "Portrait",
10: "Photo",
11: "Manual Focus",
12: "Scene",
},
),
0x0015: ("ImageUniqueID", None),
0x002E: (
"CameraType",
{
0: "Back Wide Angle",
1: "Back Normal",
6: "Front",
},
),
}
File diff suppressed because it is too large Load Diff
+101
View File
@@ -0,0 +1,101 @@
"""
Makernote (proprietary) tag definitions for Casio.
"""
from exifread.tags import SubIfdTagDict
TAGS: SubIfdTagDict = {
0x0001: (
"RecordingMode",
{
1: "Single Shutter",
2: "Panorama",
3: "Night Scene",
4: "Portrait",
5: "Landscape",
},
),
0x0002: (
"Quality",
{
1: "Economy",
2: "Normal",
3: "Fine",
},
),
0x0003: (
"FocusingMode",
{
2: "Macro",
3: "Auto Focus",
4: "Manual Focus",
5: "Infinity",
},
),
0x0004: (
"FlashMode",
{
1: "Auto",
2: "On",
3: "Off",
4: "Red Eye Reduction",
},
),
0x0005: (
"FlashIntensity",
{
11: "Weak",
13: "Normal",
15: "Strong",
},
),
0x0006: ("Object Distance", None),
0x0007: (
"WhiteBalance",
{
1: "Auto",
2: "Tungsten",
3: "Daylight",
4: "Fluorescent",
5: "Shade",
129: "Manual",
},
),
0x000A: ("DigitalZoom", None),
0x000B: (
"Sharpness",
{
0: "Normal",
1: "Soft",
2: "Hard",
},
),
0x000C: (
"Contrast",
{
0: "Normal",
1: "Low",
2: "High",
},
),
0x000D: (
"Saturation",
{
0: "Normal",
1: "Low",
2: "High",
},
),
0x0014: (
"CCDSpeed",
{
64: "Normal",
80: "Normal",
100: "High",
125: "+1.0",
244: "+3.0",
250: "+2.0",
},
),
0x0015: ("FirmwareDate", None),
}
+20
View File
@@ -0,0 +1,20 @@
"""
Makernote (proprietary) tag definitions for DJI cameras
Based on https://github.com/exiftool/exiftool/blob/master/lib/Image/ExifTool/DJI.pm
"""
from exifread.tags import SubIfdTagDict
TAGS: SubIfdTagDict = {
0x01: ("Make", None),
0x03: ("SpeedX", None),
0x04: ("SpeedY", None),
0x05: ("SpeedZ", None),
0x06: ("Pitch", None),
0x07: ("Yaw", None),
0x08: ("Roll", None),
0x09: ("CameraPitch", None),
0x0A: ("CameraYaw", None),
0x0B: ("CameraRoll", None),
}
+129
View File
@@ -0,0 +1,129 @@
"""
Makernote (proprietary) tag definitions for FujiFilm.
http://www.sno.phy.queensu.ca/~phil/exiftool/TagNames/FujiFilm.html
"""
from exifread.tags import SubIfdTagDict
from exifread.tags.str_utils import make_string
TAGS: SubIfdTagDict = {
0x0000: ("NoteVersion", make_string),
0x0010: ("InternalSerialNumber", None),
0x1000: ("Quality", None),
0x1001: (
"Sharpness",
{
0x1: "Soft",
0x2: "Soft",
0x3: "Normal",
0x4: "Hard",
0x5: "Hard2",
0x82: "Medium Soft",
0x84: "Medium Hard",
0x8000: "Film Simulation",
},
),
0x1002: (
"WhiteBalance",
{
0x0: "Auto",
0x100: "Daylight",
0x200: "Cloudy",
0x300: "Daylight Fluorescent",
0x301: "Day White Fluorescent",
0x302: "White Fluorescent",
0x303: "Warm White Fluorescent",
0x304: "Living Room Warm White Fluorescent",
0x400: "Incandescent",
0x500: "Flash",
0x600: "Underwater",
0xF00: "Custom",
0xF01: "Custom2",
0xF02: "Custom3",
0xF03: "Custom4",
0xF04: "Custom5",
0xFF0: "Kelvin",
},
),
0x1003: (
"Saturation",
{
0x0: "Normal",
0x80: "Medium High",
0x100: "High",
0x180: "Medium Low",
0x200: "Low",
0x300: "None (B&W)",
0x301: "B&W Red Filter",
0x302: "B&W Yellow Filter",
0x303: "B&W Green Filter",
0x310: "B&W Sepia",
0x400: "Low 2",
0x8000: "Film Simulation",
},
),
0x1004: (
"Contrast",
{
0x0: "Normal",
0x80: "Medium High",
0x100: "High",
0x180: "Medium Low",
0x200: "Low",
0x8000: "Film Simulation",
},
),
0x1005: ("ColorTemperature", None),
0x1006: (
"Contrast",
{
0x0: "Normal",
0x100: "High",
0x300: "Low",
},
),
0x100A: ("WhiteBalanceFineTune", None),
0x1010: (
"FlashMode",
{
0: "Auto",
1: "On",
2: "Off",
3: "Red Eye Reduction",
},
),
0x1011: ("FlashStrength", None),
0x1020: ("Macro", {0: "Off", 1: "On"}),
0x1021: ("FocusMode", {0: "Auto", 1: "Manual"}),
0x1022: ("AFPointSet", {0: "Yes", 1: "No"}),
0x1023: ("FocusPixel", None),
0x1030: ("SlowSync", {0: "Off", 1: "On"}),
0x1031: (
"PictureMode",
{
0: "Auto",
1: "Portrait",
2: "Landscape",
4: "Sports",
5: "Night",
6: "Program AE",
256: "Aperture Priority AE",
512: "Shutter Priority AE",
768: "Manual Exposure",
},
),
0x1032: ("ExposureCount", None),
0x1100: ("MotorOrBracket", {0: "Off", 1: "On"}),
0x1210: (
"ColorMode",
{
0x0: "Standard",
0x10: "Chrome",
0x30: "B & W",
},
),
0x1300: ("BlurWarning", {0: "Off", 1: "On"}),
0x1301: ("FocusWarning", {0: "Off", 1: "On"}),
0x1302: ("ExposureWarning", {0: "Off", 1: "On"}),
}
+424
View File
@@ -0,0 +1,424 @@
"""
Makernote (proprietary) tag definitions for Nikon.
"""
from exifread.tags import SubIfdTagDict
from exifread.tags.str_utils import make_string
from exifread.utils import Ratio
def ev_bias(seq) -> str:
"""
First digit seems to be in steps of 1/6 EV.
Does the third value mean the step size? It is usually 6,
but it is 12 for the ExposureDifference.
Check for an error condition that could cause a crash.
This only happens if something has gone really wrong in
reading the Nikon MakerNote.
http://tomtia.plala.jp/DigitalCamera/MakerNote/index.asp
"""
if len(seq) < 4:
return ""
if seq == [252, 1, 6, 0]:
return "-2/3 EV"
if seq == [253, 1, 6, 0]:
return "-1/2 EV"
if seq == [254, 1, 6, 0]:
return "-1/3 EV"
if seq == [0, 1, 6, 0]:
return "0 EV"
if seq == [2, 1, 6, 0]:
return "+1/3 EV"
if seq == [3, 1, 6, 0]:
return "+1/2 EV"
if seq == [4, 1, 6, 0]:
return "+2/3 EV"
# Handle combinations not in the table.
i = seq[0]
# Causes headaches for the +/- logic, so special case it.
if i == 0:
return "0 EV"
if i > 127:
i = 256 - i
ret_str = "-"
else:
ret_str = "+"
step = seq[2] # Assume third value means the step size
whole = i / step
i = i % step
if whole != 0:
ret_str = "%s%s " % (ret_str, str(whole))
if i == 0:
ret_str += "EV"
else:
ratio = Ratio(i, step)
ret_str = ret_str + str(ratio) + " EV"
return ret_str
# Nikon E99x MakerNote Tags
TAGS_NEW: SubIfdTagDict = {
0x0001: ("MakernoteVersion", make_string), # Sometimes binary
0x0002: ("ISOSetting", None),
0x0003: ("ColorMode", None),
0x0004: ("Quality", None),
0x0005: ("Whitebalance", None),
0x0006: ("ImageSharpening", None),
0x0007: ("FocusMode", None),
0x0008: ("FlashSetting", None),
0x0009: ("AutoFlashMode", None),
0x000A: ("LensMount", None), # According to LibRAW
0x000B: ("WhiteBalanceBias", None),
0x000C: ("WhiteBalanceRBCoeff", None),
0x000D: ("ProgramShift", ev_bias),
# Nearly the same as the other EV vals, but step size is 1/12 EV (?)
0x000E: ("ExposureDifference", ev_bias),
0x000F: ("ISOSelection", None),
0x0010: ("DataDump", None),
0x0011: ("NikonPreview", None),
0x0012: ("FlashCompensation", ev_bias),
0x0013: ("ISOSpeedRequested", None),
0x0014: ("ColorBalance", None), # According to LibRAW
0x0016: ("PhotoCornerCoordinates", None),
0x0017: ("ExternalFlashExposureComp", ev_bias),
0x0018: ("FlashBracketCompensationApplied", ev_bias),
0x0019: ("AEBracketCompensationApplied", None),
0x001A: ("ImageProcessing", None),
0x001B: (
"CropHiSpeed",
None,
# TODO: investigate, returns incoherent results
#
# {
# 0: "Off",
# 1: "1.3x Crop",
# 2: "DX Crop",
# 3: "5:4 Crop",
# 4: "3:2 Crop",
# 6: "16:9 Crop",
# 8: "2.7x Crop",
# 9: "DX Movie Crop",
# 10: "1.3x Movie Crop",
# 11: "FX Uncropped",
# 12: "DX Uncropped",
# 13: "2.8x Movie Crop",
# 14: "1.4x Movie Crop",
# 15: "1.5x Movie Crop",
# 17: "1:1 Crop",
# }
),
0x001C: ("ExposureTuning", None),
0x001D: ("SerialNumber", None), # Conflict with 0x00A0 ?
0x001E: (
"ColorSpace",
{
1: "sRGB",
2: "Adobe RGB",
4: "BT.2100",
},
),
0x001F: ("VRInfo", None),
0x0020: ("ImageAuthentication", None),
0x0021: ("FaceDetect", None),
0x0022: (
"ActiveDLighting",
{
0: "Off",
1: "Low",
3: "Normal",
5: "High",
7: "Extra High",
8: "Extra High 1",
9: "Extra High 2",
10: "Extra High 3",
11: "Extra High 4",
65535: "Auto",
},
),
0x0023: ("PictureControl", None),
0x0024: ("WorldTime", None),
0x0025: ("ISOInfo", None),
0x002A: (
"VignetteControl",
{
0: "Off",
1: "Low",
3: "Normal",
5: "High",
},
),
0x002B: ("DistortInfo", None),
0x002C: ("UnknownInfo", None),
0x0032: ("UnknownInfo2", None),
0x0034: (
"ShutterMode",
{
0: "Mechanical",
16: "Electronic",
48: "Electronic Front Curtain",
64: "Electronic (Movie)",
80: "Auto (Mechanical)",
81: "Auto (Electronic Front Curtain)",
96: "Electronic (High Speed)",
},
),
0x0035: ("HDRInfo", None),
0x0037: ("MechanicalShutterCount", None),
0x0039: ("LocationInfo", None),
0x003B: ("MultiExposureWhiteBalance", None),
0x003D: ("BlackLevel", None),
0x003E: ("ImageSizeRAW", None),
0x0045: ("CropArea", None),
0x004E: ("NikonSettings", None),
0x004F: ("ColorTemperatureAuto", None),
0x0080: ("ImageAdjustment", None),
0x0081: ("ToneCompensation", None),
0x0082: ("AuxiliaryLens", None),
0x0083: ("LensType", None),
0x0084: ("LensMinMaxFocalMaxAperture", None),
0x0085: ("ManualFocusDistance", None),
0x0086: ("DigitalZoomFactor", None),
0x0087: (
"FlashMode",
{
0: "Did Not Fire",
1: "Fired, Manual",
3: "Not Ready",
7: "Fired, External",
8: "Fired, Commander Mode ",
9: "Fired, TTL Mode",
18: "LED Light",
},
),
0x0088: (
"AFFocusPosition",
{
0x0000: "Center",
0x0100: "Top",
0x0200: "Bottom",
0x0300: "Left",
0x0400: "Right",
},
),
0x0089: (
"BracketingMode",
{
0x00: "Single frame, no bracketing",
0x01: "Continuous, no bracketing",
0x02: "Timer, no bracketing",
0x10: "Single frame, exposure bracketing",
0x11: "Continuous, exposure bracketing",
0x12: "Timer, exposure bracketing",
0x40: "Single frame, white balance bracketing",
0x41: "Continuous, white balance bracketing",
0x42: "Timer, white balance bracketing",
},
),
0x008A: ("AutoBracketRelease", None),
0x008B: ("LensFStops", None),
0x008C: ("NEFCurve1", None), # ExifTool calls this 'ContrastCurve'
0x008D: ("ColorMode", None),
0x008F: ("SceneMode", None),
0x0090: ("LightingType", None),
0x0091: ("ShotInfo", None), # First 4 bytes are a version number in ASCII
0x0092: ("HueAdjustment", None),
0x0093: (
"NEFCompression",
{
1: "Lossy (type 1)",
2: "Uncompressed",
3: "Lossless",
4: "Lossy (type 2)",
5: "Striped packed 12 bits",
6: "Uncompressed (reduced to 12 bit)",
7: "Unpacked 12 bits",
8: "Small",
9: "Packed 12 bits",
10: "Packed 14 bits",
13: "High Efficiency",
14: "High Efficiency*",
},
),
0x0094: (
"Saturation",
{
-3: "B&W",
-2: "-2",
-1: "-1",
0: "0",
1: "1",
2: "2",
},
),
0x0095: ("NoiseReduction", None),
0x0096: ("NEFCurve2", None), # ExifTool calls this 'LinearizationTable'
0x0097: ("ColorBalance", None), # First 4 bytes are a version number in ASCII
0x0098: ("LensData", None), # First 4 bytes are a version number in ASCII
0x0099: ("RawImageCenter", None),
0x009A: ("SensorPixelSize", None),
0x009C: ("Scene Assist", None),
0x009D: (
"DateStampMode",
{0: "Off", 1: "Date & Time", 2: "Date", 3: "Date Counter"},
),
0x009E: (
"RetouchHistory",
None,
# TODO: investigate, returns incoherent results
#
# {
# 0: "None",
# 3: "B & W",
# 4: "Sepia",
# 5: "Trim",
# 6: "Small Picture",
# 7: "D-Lighting",
# 8: "Red Eye",
# 9: "Cyanotype",
# 10: "Sky Light",
# 11: "Warm Tone",
# 12: "Color Custom",
# 13: "Image Overlay",
# 14: "Red Intensifier",
# 15: "Green Intensifier",
# 16: "Blue Intensifier",
# 17: "Cross Screen",
# 18: "Quick Retouch",
# 19: "NEF Processing",
# 23: "Distortion Control",
# 25: "Fisheye",
# 26: "Straighten",
# 29: "Perspective Control",
# 30: "Color Outline",
# 31: "Soft Filter",
# 32: "Resize",
# 33: "Miniature Effect",
# 34: "Skin Softening",
# 35: "Selected Frame",
# 37: "Color Sketch",
# 38: "Selective Color",
# 39: "Glamour",
# 40: "Drawing",
# 44: "Pop",
# 45: "Toy Camera Effect 1",
# 46: "Toy Camera Effect 2",
# 47: "Cross Process (red)",
# 48: "Cross Process (blue)",
# 49: "Cross Process (green)",
# 50: "Cross Process (yellow)",
# 51: "Super Vivid",
# 52: "High-contrast Monochrome",
# 53: "High Key",
# 54: "Low Key",
# },
),
0x00A0: ("SerialNumber", None),
0x00A2: ("ImageDataSize", None),
# 00A3: unknown - a single byte 0
# 00A4: In NEF, looks like a 4 byte ASCII version number ('0200')
0x00A5: ("ImageCount", None),
0x00A6: ("DeletedImageCount", None),
0x00A7: ("TotalShutterReleases", None),
# First 4 bytes are a version number in ASCII, with version specific
# info to follow. It's hard to treat it as a string due to embedded nulls.
0x00A8: ("FlashInfo", None),
0x00A9: ("ImageOptimization", None),
0x00AA: ("Saturation", None),
0x00AB: ("DigitalVariProgram", None),
0x00AC: ("ImageStabilization", None),
0x00AD: ("AFResponse", None),
0x00B0: ("MultiExposure", None),
0x00B1: (
"HighISONoiseReduction",
{
0: "Off",
1: "Minimal",
2: "Low",
3: "Medium Low",
4: "Normal",
5: "Medium High",
6: "High",
},
),
0x00B3: ("ToningEffect", None),
0x00B6: ("PowerUpTime", None),
0x00B7: ("AFInfo2", None),
0x00B8: ("FileInfo", None),
0x00B9: ("AFTune", None),
0x00BB: ("RetouchInfo", None),
0x00BD: ("PictureControlData", None),
0x00BF: ("SilentPhotography", {0: "Off", 1: "On"}),
0x00C3: ("BarometerInfo", None),
0x0100: ("DigitalICE", None),
0x0201: ("PreviewImageStart", None),
0x0202: ("PreviewImageLength", None),
0x0213: (
"PreviewYCbCrPositioning",
{
1: "Centered",
2: "Co-sited",
},
),
0x0E00: ("PrintIM", None),
0x0E01: ("InCameraEditNote", None),
0x0E09: ("NikonCaptureVersion", None),
0x0E0E: ("NikonCaptureOffsets", None),
0x0E10: ("NikonScan", None),
0x0E13: ("NikonCaptureEditVersions", None),
0x0E1D: ("NikonICCProfile", None),
0x0E1E: ("NikonCaptureOutput", None),
0x0E22: ("NEFBitDepth", None),
}
TAGS_OLD: SubIfdTagDict = {
0x0003: (
"Quality",
{
1: "VGA Basic",
2: "VGA Normal",
3: "VGA Fine",
4: "SXGA Basic",
5: "SXGA Normal",
6: "SXGA Fine",
},
),
0x0004: (
"ColorMode",
{
1: "Color",
2: "Monochrome",
},
),
0x0005: (
"ImageAdjustment",
{
0: "Normal",
1: "Bright+",
2: "Bright-",
3: "Contrast+",
4: "Contrast-",
},
),
0x0006: (
"CCDSpeed",
{
0: "ISO 80",
2: "ISO 160",
4: "ISO 320",
5: "ISO 100",
},
),
0x0007: (
"WhiteBalance",
{
0: "Auto",
1: "Preset",
2: "Daylight",
3: "Incandescent",
4: "Fluorescent",
5: "Cloudy",
6: "Speed Light",
},
),
}
+309
View File
@@ -0,0 +1,309 @@
"""
Makernote (proprietary) tag definitions for Olympus.
"""
from typing import Dict
from exifread.tags import SubIfdTagDictValue
from exifread.tags.str_utils import make_string
def special_mode(val: bytes) -> str:
"""Decode Olympus SpecialMode tag in MakerNote"""
mode1 = {
0: "Normal",
1: "Unknown",
2: "Fast",
3: "Panorama",
}
mode2 = {
0: "Non-panoramic",
1: "Left to right",
2: "Right to left",
3: "Bottom to top",
4: "Top to bottom",
}
if not val:
return ""
mode1_val = mode1.get(val[0], "Unknown")
mode2_val = mode2.get(val[2], "Unknown")
return "%s - Sequence %d - %s" % (mode1_val, val[1], mode2_val)
TAGS: Dict[int, SubIfdTagDictValue] = {
# Ah HAH! those sneeeeeaky bastids! this is how they get past the fact
# that a JPEG thumbnail is not allowed in an uncompressed TIFF file
0x0100: ("JPEGThumbnail", None),
0x0200: ("SpecialMode", special_mode),
0x0201: (
"JPEGQual",
{
1: "SQ",
2: "HQ",
3: "SHQ",
},
),
0x0202: (
"Macro",
{
0: "Normal",
1: "Macro",
2: "SuperMacro",
},
),
0x0203: ("BWMode", {0: "Off", 1: "On"}),
0x0204: ("DigitalZoom", None),
0x0205: ("FocalPlaneDiagonal", None),
0x0206: ("LensDistortionParams", None),
0x0207: ("SoftwareRelease", None),
0x0208: ("PictureInfo", None),
0x0209: ("CameraID", make_string), # print as string
0x0F00: ("DataDump", None),
0x0300: ("PreCaptureFrames", None),
0x0404: ("SerialNumber", None),
0x1000: ("ShutterSpeedValue", None),
0x1001: ("ISOValue", None),
0x1002: ("ApertureValue", None),
0x1003: ("BrightnessValue", None),
0x1004: ("FlashMode", {2: "On", 3: "Off"}),
0x1005: (
"FlashDevice",
{
0: "None",
1: "Internal",
4: "External",
5: "Internal + External",
},
),
0x1006: ("ExposureCompensation", None),
0x1007: ("SensorTemperature", None),
0x1008: ("LensTemperature", None),
0x100B: ("FocusMode", {0: "Auto", 1: "Manual"}),
0x1017: ("RedBalance", None),
0x1018: ("BlueBalance", None),
0x101A: ("SerialNumber", None),
0x1023: ("FlashExposureComp", None),
0x1026: ("ExternalFlashBounce", {0: "No", 1: "Yes"}),
0x1027: ("ExternalFlashZoom", None),
0x1028: ("ExternalFlashMode", None),
0x1029: ("Contrast int16u", {0: "High", 1: "Normal", 2: "Low"}),
0x102A: ("SharpnessFactor", None),
0x102B: ("ColorControl", None),
0x102C: ("ValidBits", None),
0x102D: ("CoringFilter", None),
0x102E: ("OlympusImageWidth", None),
0x102F: ("OlympusImageHeight", None),
0x1034: ("CompressionRatio", None),
0x1035: ("PreviewImageValid", {0: "No", 1: "Yes"}),
0x1036: ("PreviewImageStart", None),
0x1037: ("PreviewImageLength", None),
0x1039: ("CCDScanMode", {0: "Interlaced", 1: "Progressive"}),
0x103A: ("NoiseReduction", {0: "Off", 1: "On"}),
0x103B: ("InfinityLensStep", None),
0x103C: ("NearLensStep", None),
# TODO - these need extra definitions
# http://search.cpan.org/src/EXIFTOOL/Image-ExifTool-6.90/html/TagNames/Olympus.html
0x2010: ("Equipment", None),
0x2020: ("CameraSettings", None),
0x2030: ("RawDevelopment", None),
0x2040: ("ImageProcessing", None),
0x2050: ("FocusInfo", None),
0x3000: ("RawInfo ", None),
}
# 0x2020 CameraSettings
TAG_0x2020: Dict[int, SubIfdTagDictValue] = {
0x0100: ("PreviewImageValid", {0: "No", 1: "Yes"}),
0x0101: ("PreviewImageStart", None),
0x0102: ("PreviewImageLength", None),
0x0200: (
"ExposureMode",
{
1: "Manual",
2: "Program",
3: "Aperture-priority AE",
4: "Shutter speed priority AE",
5: "Program-shift",
},
),
0x0201: ("AELock", {0: "Off", 1: "On"}),
0x0202: (
"MeteringMode",
{
2: "Center Weighted",
3: "Spot",
5: "ESP",
261: "Pattern+AF",
515: "Spot+Highlight control",
1027: "Spot+Shadow control",
},
),
0x0300: ("MacroMode", {0: "Off", 1: "On"}),
0x0301: (
"FocusMode",
{
0: "Single AF",
1: "Sequential shooting AF",
2: "Continuous AF",
3: "Multi AF",
10: "MF",
},
),
0x0302: (
"FocusProcess",
{
0: "AF Not Used",
1: "AF Used",
},
),
0x0303: ("AFSearch", {0: "Not Ready", 1: "Ready"}),
0x0304: ("AFAreas", None),
0x0401: ("FlashExposureCompensation", None),
0x0500: (
"WhiteBalance2",
{
0: "Auto",
16: "7500K (Fine Weather with Shade)",
17: "6000K (Cloudy)",
18: "5300K (Fine Weather)",
20: "3000K (Tungsten light)",
21: "3600K (Tungsten light-like)",
33: "6600K (Daylight fluorescent)",
34: "4500K (Neutral white fluorescent)",
35: "4000K (Cool white fluorescent)",
48: "3600K (Tungsten light-like)",
256: "Custom WB 1",
257: "Custom WB 2",
258: "Custom WB 3",
259: "Custom WB 4",
512: "Custom WB 5400K",
513: "Custom WB 2900K",
514: "Custom WB 8000K",
},
),
0x0501: ("WhiteBalanceTemperature", None),
0x0502: ("WhiteBalanceBracket", None),
0x0503: ("CustomSaturation", None), # (3 numbers: 1. CS Value, 2. Min, 3. Max)
0x0504: (
"ModifiedSaturation",
{
0: "Off",
1: "CM1 (Red Enhance)",
2: "CM2 (Green Enhance)",
3: "CM3 (Blue Enhance)",
4: "CM4 (Skin Tones)",
},
),
0x0505: ("ContrastSetting", None), # (3 numbers: 1. Contrast, 2. Min, 3. Max)
0x0506: ("SharpnessSetting", None), # (3 numbers: 1. Sharpness, 2. Min, 3. Max)
0x0507: (
"ColorSpace",
{
0: "sRGB",
1: "Adobe RGB",
2: "Pro Photo RGB",
},
),
0x0509: (
"SceneMode",
{
0: "Standard",
6: "Auto",
7: "Sport",
8: "Portrait",
9: "Landscape+Portrait",
10: "Landscape",
11: "Night scene",
13: "Panorama",
16: "Landscape+Portrait",
17: "Night+Portrait",
19: "Fireworks",
20: "Sunset",
22: "Macro",
25: "Documents",
26: "Museum",
28: "Beach&Snow",
30: "Candle",
35: "Underwater Wide1",
36: "Underwater Macro",
39: "High Key",
40: "Digital Image Stabilization",
44: "Underwater Wide2",
45: "Low Key",
46: "Children",
48: "Nature Macro",
},
),
0x050A: (
"NoiseReduction",
{
0: "Off",
1: "Noise Reduction",
2: "Noise Filter",
3: "Noise Reduction + Noise Filter",
4: "Noise Filter (ISO Boost)",
5: "Noise Reduction + Noise Filter (ISO Boost)",
},
),
0x050B: ("DistortionCorrection", {0: "Off", 1: "On"}),
0x050C: ("ShadingCompensation", {0: "Off", 1: "On"}),
0x050D: ("CompressionFactor", None),
0x050F: (
"Gradation",
{
"-1 -1 1": "Low Key",
"0 -1 1": "Normal",
"1 -1 1": "High Key",
},
),
0x0520: (
"PictureMode",
{
1: "Vivid",
2: "Natural",
3: "Muted",
256: "Monotone",
512: "Sepia",
},
),
0x0521: ("PictureModeSaturation", None),
0x0522: ("PictureModeHue?", None),
0x0523: ("PictureModeContrast", None),
0x0524: ("PictureModeSharpness", None),
0x0525: (
"PictureModeBWFilter",
{
0: "n/a",
1: "Neutral",
2: "Yellow",
3: "Orange",
4: "Red",
5: "Green",
},
),
0x0526: (
"PictureModeTone",
{
0: "n/a",
1: "Neutral",
2: "Sepia",
3: "Blue",
4: "Purple",
5: "Green",
},
),
0x0600: ("Sequence", None), # 2 or 3 numbers: 1. Mode, 2. Shot number, 3. Mode bits
0x0601: ("PanoramaMode", None), # (2 numbers: 1. Mode, 2. Shot number)
0x0603: (
"ImageQuality2",
{
1: "SQ",
2: "HQ",
3: "SHQ",
4: "RAW",
},
),
0x0901: ("ManometerReading", None),
}
+605
View File
@@ -0,0 +1,605 @@
"""
Makernote (proprietary) tag definitions for Sony.
Based on https://www.exiftool.org/TagNames/Sony.html
"""
from exifread.tags import SubIfdTagDict
TAGS: SubIfdTagDict = {
# depends on model
# 0x0010: (),
# 0x0020: (),
0x0102: (
"Quality",
{
0: "RAW",
1: "Super Fine",
2: "Fine",
3: "Standard",
4: "Economy",
5: "Extra Fine",
6: "RAW + JPEG/HEIF",
7: "Compressed RAW",
8: "Compressed RAW + JPEG",
9: "Light",
},
),
0x0104: ("FlashExposureComp", None),
0x0105: (
"Teleconverter",
{
0x0: "None",
0x4: "Minolta/Sony AF 1.4x APO (D) (0x04)",
0x5: "Minolta/Sony AF 2x APO (D) (0x05)",
0x48: "Minolta/Sony AF 2x APO (D)",
0x50: "Minolta AF 2x APO II",
0x60: "Minolta AF 2x APO",
0x88: "Minolta/Sony AF 1.4x APO (D)",
0x90: "Minolta AF 1.4x APO II",
0xA0: "Minolta AF 1.4x APO",
},
),
0x0112: ("WhiteBalanceFineTune", None),
# depends on model
# 0x0114: (),
0x0115: (
"WhiteBalance",
{
0x0: "Auto",
0x1: "Color Temperature / Color Filter",
0x10: "Daylight",
0x20: "Cloudy",
0x30: "Shade",
0x40: "Tungsten",
0x50: "Flash",
0x60: "Fluorescent",
0x70: "Custom",
0x80: "Underwater",
},
),
# depends on model
# 0x0116: (),
0x0E00: ("PrintIM", None),
0x1000: ("MultiBurstMode", {0: "Off", 1: "On"}),
0x1001: ("MultiBurstImageWidth", None),
0x1002: ("MultiBurstImageHeight", None),
# needs specific processing?
# 0x1003: ("Panorama", {}),
0x2001: ("PreviewImage", None),
0x2002: ("Rating", None),
0x2004: ("Contrast", None),
0x2005: ("Saturation", None),
0x2006: ("Sharpness", None),
0x2007: ("Brightness", None),
0x2008: (
"LongExposureNoiseReduction",
{
0x0: "Off",
0x1: "On (unused)",
0x10001: "On (dark subtracted)",
0xFFFF0000: "Off (65535)",
0xFFFF0001: "On (65535)",
0xFFFFFFFF: "N/A",
},
),
0x2009: (
"HighISONoiseReduction",
{
0: "Off",
1: "Low",
2: "Normal",
3: "High",
256: "Auto",
65535: "N/A",
},
),
# Needs specific processing?
# 0x200A: ("HDR", {}),
0x200B: (
"MultiFrameNoiseReduction",
{
0: "Off",
1: "On",
255: "N/A",
},
),
0x200E: (
"PictureEffect",
{
0: "Off",
1: "Toy Camera",
2: "Pop Color",
3: "Posterization",
4: "Posterization B/W",
5: "Retro Photo",
6: "Soft High Key",
7: "Partial Color (red)",
8: "Partial Color (green)",
9: "Partial Color (blue)",
10: "Partial Color (yellow)",
13: "High Contrast Monochrome",
16: "Toy Camera (normal)",
17: "Toy Camera (cool)",
18: "Toy Camera (warm)",
19: "Toy Camera (green)",
20: "Toy Camera (magenta)",
32: "Soft Focus (low)",
33: "Soft Focus",
34: "Soft Focus (high)",
48: "Miniature (auto)",
49: "Miniature (top)",
50: "Miniature (middle horizontal)",
51: "Miniature (bottom)",
52: "Miniature (left)",
53: "Miniature (middle vertical)",
54: "Miniature (right)",
64: "HDR Painting (low)",
65: "HDR Painting",
66: "HDR Painting (high)",
80: "Rich-tone Monochrome",
97: "Water Color",
98: "Water Color 2",
112: "Illustration (low)",
113: "Illustration",
114: "Illustration (high)",
},
),
0x200F: (
"SoftSkinEffect",
{
0: "Off",
1: "Low",
2: "Mid",
3: "High",
4294967295: "N/A",
},
),
# depends on model
# 0x2010: (),
0x2011: (
"VignettingCorrection",
{
0: "Off",
2: "Auto",
4294967295: "N/A",
},
),
0x2012: (
"LateralChromaticAberration",
{
0: "Off",
2: "Auto",
4294967295: "N/A",
},
),
0x2013: (
"DistortionCorrectionSetting",
{
0: "Off",
2: "Auto",
4294967295: "N/A",
},
),
0x2014: ("WBShiftAB_GM", None),
0x2016: ("AutoPortraitFramed", {0: "No", 1: "Yes"}),
0x2017: (
"FlashAction",
{
0: "Did not fire",
1: "Flash Fired",
2: "External Flash Fired",
3: "Wireless Controlled Flash Fired",
},
),
0x201A: ("ElectronicFrontCurtainShutter", {0: "Off", 1: "On"}),
0x201B: (
"FocusMode",
{
0: "Manual",
2: "AF-S",
3: "AF-C",
4: "AF-A",
6: "DMF",
7: "AF-D",
},
),
0x2021: (
"FlashExposureComp",
{
0: "Off",
1: "Face tracking",
2: "Lock On AF",
},
),
0xB001: (
"SonyModelID",
{
2: "DSC-R1",
256: "DSLR-A100",
257: "DSLR-A900",
258: "DSLR-A700",
259: "DSLR-A200",
260: "DSLR-A350",
261: "DSLR-A300",
262: "DSLR-A900 (APS-C mode)",
263: "DSLR-A380/A390",
264: "DSLR-A330",
265: "DSLR-A230",
266: "DSLR-A290",
269: "DSLR-A850",
270: "DSLR-A850 (APS-C mode)",
273: "DSLR-A550",
274: "DSLR-A500",
275: "DSLR-A450",
278: "NEX-5",
279: "NEX-3",
280: "SLT-A33",
281: "SLT-A55 / SLT-A55V",
282: "DSLR-A560",
283: "DSLR-A580",
284: "NEX-C3",
285: "SLT-A35",
286: "SLT-A65 / SLT-A65V",
287: "SLT-A77 / SLT-A77V",
288: "NEX-5N",
289: "NEX-7",
290: "NEX-VG20E",
291: "SLT-A37",
292: "SLT-A57",
293: "NEX-F3",
294: "SLT-A99 / SLT-A99V",
295: "NEX-6",
296: "NEX-5R",
297: "DSC-RX100",
298: "DSC-RX1",
299: "NEX-VG900",
300: "NEX-VG30E",
302: "ILCE-3000 / ILCE-3500",
303: "SLT-A58",
305: "NEX-3N",
306: "ILCE-7",
307: "NEX-5T",
308: "DSC-RX100M2",
309: "DSC-RX10",
310: "DSC-RX1R",
311: "ILCE-7R",
312: "ILCE-6000",
313: "ILCE-5000",
317: "DSC-RX100M3",
318: "ILCE-7S",
319: "ILCA-77M2",
339: "ILCE-5100",
340: "ILCE-7M2",
341: "DSC-RX100M4",
342: "DSC-RX10M2",
344: "DSC-RX1RM2",
346: "ILCE-QX1",
347: "ILCE-7RM2",
350: "ILCE-7SM2",
353: "ILCA-68",
354: "ILCA-99M2",
355: "DSC-RX10M3",
356: "DSC-RX100M5",
357: "ILCE-6300",
358: "ILCE-9",
360: "ILCE-6500",
362: "ILCE-7RM3",
363: "ILCE-7M3",
364: "DSC-RX0",
365: "DSC-RX10M4",
366: "DSC-RX100M6",
367: "DSC-HX99",
369: "DSC-RX100M5A",
371: "ILCE-6400",
372: "DSC-RX0M2",
373: "DSC-HX95",
374: "DSC-RX100M7",
375: "ILCE-7RM4",
376: "ILCE-9M2",
378: "ILCE-6600",
379: "ILCE-6100",
380: "ZV-1",
381: "ILCE-7C",
382: "ZV-E10",
383: "ILCE-7SM3",
384: "ILCE-1",
385: "ILME-FX3",
386: "ILCE-7RM3A",
387: "ILCE-7RM4A",
388: "ILCE-7M4",
389: "ZV-1F",
390: "ILCE-7RM5",
391: "ILME-FX30",
392: "ILCE-9M3",
393: "ZV-E1",
394: "ILCE-6700",
395: "ZV-1M2",
396: "ILCE-7CR",
397: "ILCE-7CM2",
398: "ILX-LR1",
399: "ZV-E10M2",
400: "ILCE-1M2",
},
),
0xB020: ("CreativeStyle", None),
0xB021: ("ColorTemperature", None),
0xB022: ("ColorCompensationFilter", None),
0xB023: (
"SceneMode",
{
0: "Standard",
1: "Portrait",
2: "Text",
3: "Night Scene",
4: "Sunset",
5: "Sports",
6: "Landscape",
7: "Night Portrait",
8: "Macro",
9: "Super Macro",
16: "Auto",
17: "Night View / Portrait",
18: "Sweep Panorama",
19: "Handheld Night Shot",
20: "Anti Motion Blur",
21: "Cont. Priority AE",
22: "Auto+",
23: "3D Sweep Panorama",
24: "Superior Auto",
25: "High Sensitivity",
26: "Fireworks",
27: "Food",
28: "Pet",
33: "HDR",
},
),
0xB024: (
"ZoneMatching",
{
0: "ISO Setting Used",
1: "High Key",
2: "Low Key",
},
),
0xB025: (
"DynamicRangeOptimizer",
{
0: "Off",
1: "Standard",
2: "Advanced Auto",
3: "Auto",
8: "Advanced Lv1",
9: "Advanced Lv2",
10: "Advanced Lv3",
11: "Advanced Lv4",
12: "Advanced Lv5",
16: "Lv1",
17: "Lv2",
18: "Lv3",
19: "Lv4",
20: "Lv5",
},
),
0xB026: (
"ImageStabilization",
{
0: "Off",
1: "On",
4294967295: "N/A",
},
),
# TODO
# 0xB027: ("LensType", {}),
0xB029: (
"ColorMode",
{
0: "Standard",
1: "Vivid",
2: "Portrait",
3: "Landscape",
4: "Sunset",
5: "Night View / Portrait",
6: "B & W",
7: "Adobe RGB",
12: "Neutral",
13: "Clear",
14: "Deep",
15: "Light",
16: "Autumn Leaves",
17: "Sepia",
18: "FL",
19: "Vivid 2",
20: "IN",
21: "SH",
100: "Neutral",
101: "Clear",
102: "Deep",
103: "Light",
104: "Night View",
105: "Autumn Leaves",
255: "Off",
4294967295: "N/A",
},
),
0xB040: (
"Macro",
{
0: "Off",
1: "On",
2: "Close Focus",
65535: "N/A",
},
),
0xB041: (
"ExposureMode",
{
0: "Program AE",
1: "Portrait",
2: "Beach",
3: "Sports",
4: "Snow",
5: "Landscape",
6: "Auto",
7: "Aperture-priority AE",
8: "Shutter speed priority AE",
9: "Night Scene / Twilight",
10: "Hi-Speed Shutter",
11: "Twilight Portrait",
12: "Soft Snap/Portrait",
13: "Fireworks",
14: "Smile Shutter",
15: "Manual",
18: "High Sensitivity",
19: "Macro",
20: "Advanced Sports Shooting",
29: "Underwater",
33: "Food",
34: "Sweep Panorama",
35: "Handheld Night Shot",
36: "Anti Motion Blur",
37: "Pet",
38: "Backlight Correction HDR",
39: "Superior Auto",
40: "Background Defocus",
41: "Soft Skin",
42: "3D Image",
65535: "N/A",
},
),
0xB042: (
"FocusMode",
{
1: "AF-S",
2: "AF-C",
4: "Permanent-AF",
65535: "N/A",
},
),
# depends on model
# 0xb043: (),
0xB044: (
"AFIlluminator",
{
0: "Off",
1: "On",
65535: "N/A",
},
),
0xB047: (
"JPEGQuality",
{
0: "Standard",
1: "Fine",
2: "Extra Fine",
65535: "N/A",
},
),
0xB048: (
"FlashLevel",
{
-32768: "Low",
-9: "-9/3",
-8: "-8/3",
-7: "-7/3",
-6: "-6/3",
-5: "-5/3",
-4: "-4/3",
-3: "-3/3",
-2: "-2/3",
-1: "-1/3",
0: "Normal",
1: "+1/3",
2: "+2/3",
3: "+3/3",
4: "+4/3",
5: "+5/3",
6: "+6/3",
9: "+9/3",
128: "N/A",
32767: "High",
},
),
0xB049: (
"ReleaseMode",
{
0: "Normal",
2: "Continuous",
5: " Exposure Bracketing",
6: "White Balance Bracketing",
8: "DRO Bracketing",
65535: "N/A",
},
),
0xB04A: (
"SequenceNumber",
{
0: "Single",
65535: "N/A",
},
),
0xB04B: (
"Anti-Blur",
{
0: "Off",
1: "On - Continuous",
2: "On - Shooting",
65535: "N/A",
},
),
0xB04E: (
"FocusMode",
{
0: "Manual",
2: "AF-S",
3: "AF-C",
5: "Semi-manual",
6: "DMF",
},
),
0xB04F: (
"DynamicRangeOptimizer",
{
0: "Off",
1: "Standard",
2: "Plus",
},
),
0xB050: (
"HighISONoiseReduction2",
{
0: "Normal",
1: "High",
2: "Low",
3: "Off",
65535: "N/A",
},
),
0xB052: (
"IntelligentAuto",
{
0: "Off",
1: "On",
2: "Advanced",
},
),
0xB054: (
"WhiteBalance",
{
0: "Auto",
4: "Custom",
5: "Daylight",
6: "Cloudy",
7: "Cool White Fluorescent",
8: "Day White Fluorescent",
9: "Daylight Fluorescent",
10: "Incandescent2",
11: "Warm White Fluorescent",
14: "Incandescent",
15: "Flash",
17: "Underwater 1 (Blue Water)",
18: "Underwater 2 (Green Water)",
19: "Underwater Auto",
},
),
}
+44
View File
@@ -0,0 +1,44 @@
"""
Misc utilities.
"""
def make_string_uc(seq) -> str:
"""
Special version to deal with the code in the first 8 bytes of a user comment.
First 8 bytes gives coding system e.g. ASCII vs. JIS vs Unicode.
"""
if not isinstance(seq, str):
# Remove code from sequence only if it is valid
if make_string(seq[:8]).upper() in ("ASCII", "UNICODE", "JIS", ""):
seq = seq[8:]
# Of course, this is only correct if ASCII, and the standard explicitly
# allows JIS and Unicode.
return make_string(seq)
def make_string(seq) -> str:
"""
Don't throw an exception when given an out of range character.
"""
string = ""
for char in seq:
# Screen out non-printing characters
try:
if 32 <= char < 256:
string += chr(char)
except TypeError:
pass
# If no printing chars
if not string:
if isinstance(seq, list):
string = "".join(map(str, seq))
# Some UserComment lists only contain null bytes, nothing valuable to return
if set(string) == {"0"}:
return ""
else:
string = str(seq)
# Clean undesirable characters on any end
return string.strip(" \x00")
+83
View File
@@ -0,0 +1,83 @@
"""
Misc utilities.
"""
from fractions import Fraction
from typing import Optional, Tuple
def _degrees_to_decimal(degrees: float, minutes: float, seconds: float) -> float:
"""
Converts coordinates from a degrees minutes seconds format to a decimal degrees format.
Reference: https://en.wikipedia.org/wiki/Geographic_coordinate_conversion
"""
return degrees + minutes / 60 + seconds / 3600
def get_gps_coords(tags: dict) -> Optional[Tuple[float, float]]:
"""
Extract tuple of latitude and longitude values in decimal degrees format from EXIF tags.
Return None if no GPS coordinates are found.
Handles regular and serialized Exif tags.
"""
gps = {
"lat_coord": "GPS GPSLatitude",
"lat_ref": "GPS GPSLatitudeRef",
"lng_coord": "GPS GPSLongitude",
"lng_ref": "GPS GPSLongitudeRef",
}
# Verify if required keys are a subset of provided tags
if not set(gps.values()) <= tags.keys():
return None
# If tags have not been converted to native Python types, do it
if not isinstance(tags[gps["lat_coord"]], list):
tags[gps["lat_coord"]] = [c.decimal() for c in tags[gps["lat_coord"]].values]
tags[gps["lng_coord"]] = [c.decimal() for c in tags[gps["lng_coord"]].values]
tags[gps["lat_ref"]] = tags[gps["lat_ref"]].values
tags[gps["lng_ref"]] = tags[gps["lng_ref"]].values
lat = _degrees_to_decimal(*tags[gps["lat_coord"]])
if tags[gps["lat_ref"]] == "S":
lat *= -1
lng = _degrees_to_decimal(*tags[gps["lng_coord"]])
if tags[gps["lng_ref"]] == "W":
lng *= -1
return lat, lng
class Ratio(Fraction):
"""
Ratio object that eventually will be able to reduce itself to lowest
common denominator for printing.
"""
_numerator: Optional[int]
_denominator: Optional[int]
# We're immutable, so use __new__ not __init__
def __new__(cls, numerator: int = 0, denominator: Optional[int] = None):
try:
self = super(Ratio, cls).__new__(cls, numerator, denominator)
except ZeroDivisionError:
self = super(Ratio, cls).__new__(cls)
self._numerator = numerator
self._denominator = denominator
return self
def __repr__(self) -> str:
return str(self)
@property
def num(self) -> int:
return self.numerator
@property
def den(self) -> int:
return self.denominator
def decimal(self) -> float:
return float(self)