feat(v1): add verified pose adapter

This commit is contained in:
ila
2026-07-21 09:53:03 +08:00
parent 2805905486
commit f80fb51a85
8 changed files with 267 additions and 15 deletions
+1 -1
View File
@@ -31,7 +31,7 @@ V1 的同一数据流既可接 RTSP,也可回放本地录像。V2 复用同一
| 应用入口 | `v1/app.py` | 装配配置、窗口、线程和依赖 | 推理细节、事件判定 |
| 配置 | `v1/config.py` | 解析示例和本地配置,校验非敏感字段 | 保存真实凭证 |
| 视频源 | `v1/video_source.py` | 打开、读取、重连 RTSP 或录像;输出帧、单调回放时间戳和显式来源状态 | Pose、报警 |
| Pose 适配器 | `v1/pose.py` | 统一返回 box、关键点、置信度 | 跟踪、摔倒业务结论 |
| Pose 适配器 | `v1/pose.py` | 校验锁定模型的 SHA-256、pose/person/17×3 契约,统一返回 box、关键点、置信度 | 跟踪、摔倒业务结论 |
| 跟踪 | `v1/tracking.py` | 为连续人员输出 `track_id` | 根据姿态报警 |
| 质量与证据 | `v1/evidence.py` | 过滤低质量点,计算水平姿态、下移和持续性证据 | GUI 状态 |
| 状态机 | `v1/fall_state.py` | 管理每个 ID 的 NORMAL、SUSPECT、CONFIRMED、RECOVERING | 播放声音或存文件 |
+1 -1
View File
@@ -16,7 +16,7 @@
| --- | --- | --- | --- | --- |
| T-101 | 创建 V1 包、依赖清单、示例配置和忽略规则 | T-000 | `v1/` 可导入;真实 RTSP 凭证被拒绝提交;`pytest` 能运行。 | DONE |
| T-102 | 实现可重连的视频源与录像回放适配器 | T-101 | 有效本地录像可按时间戳产帧;无效源进入连接错误状态且不崩溃。 | DONE |
| T-103 | 实现 Pose 适配器与模型来源校验 | T-102 | 输出 person box、17 点和置信度;错误模型或哈希不符时给出明确错误。 | DOING |
| T-103 | 实现 Pose 适配器与模型来源校验 | T-102 | 输出 person box、17 点和置信度;错误模型或哈希不符时给出明确错误。 | DONE |
| T-104 | 实现人员跟踪与姿态质量门控 | T-103 | 连续人员维持 ID;低质量、缺失膝踝或空帧不会产生倒地候选。 | TODO |
| T-105 | 实现按 ID 的时序摔倒状态机 | T-104 | 正例在配置秒数内确认;坐下、弯腰、短时低姿态回到 NORMAL;事件副作用只触发一次。 | TODO |
+5 -5
View File
@@ -190,7 +190,7 @@ git commit -m "feat(v1): add replayable video source"
- Create: `v1/pose.py`
- Create: `v1/tests/test_pose.py`
- [ ] **Step 1: Write adapter shape tests**
- [x] **Step 1: Write adapter shape tests**
```python
def test_pose_adapter_rejects_non_pose_model(tmp_path):
@@ -202,12 +202,12 @@ def test_person_pose_has_seventeen_keypoints(fake_yolo_result):
assert len(poses[0].keypoints) == 17
```
- [ ] **Step 2: Run tests to verify failure**
- [x] **Step 2: Run tests to verify failure**
Run: `python -m pytest v1/tests/test_pose.py -v`
Expected: FAIL because `PoseAdapter` is missing.
- [ ] **Step 3: Implement only the adapter contract**
- [x] **Step 3: Implement only the adapter contract**
```python
from typing import Sequence, Tuple
@@ -226,12 +226,12 @@ class PoseAdapter:
At construction, hash the model, require task `pose`, class `person`, and exactly 17 three-value keypoints.
- [ ] **Step 4: Run tests and a real model smoke**
- [x] **Step 4: Run tests and a real model smoke**
Run: `python -m pytest v1/tests/test_pose.py -v; python -c "from ultralytics import YOLO; assert YOLO('demo/best.pt').task == 'pose'"`
Expected: PASS and no assertion error.
- [ ] **Step 5: Commit**
- [x] **Step 5: Commit**
```powershell
git add v1/pose.py v1/tests/test_pose.py docs progress.md
+7 -2
View File
@@ -40,12 +40,15 @@
```text
Keypoint = { x: float, y: float, confidence: float }
PersonPose = {
track_id: string,
detected_at_monotonic: float,
box_xyxy: [float, float, float, float],
box_confidence: float,
keypoints: Keypoint[17]
}
TrackedPersonPose = {
track_id: string,
detected_at_monotonic: float,
pose: PersonPose
}
PoseQuality = {
accepted: bool,
reason: string,
@@ -71,6 +74,8 @@ FallEvent = {
`FallEvent` 只在状态首次进入 `CONFIRMED` 时创建一次。连续帧更新 UI 状态,但不重复创建事件。
`PersonPose` 是 T-103 的纯模型输出,不带人员 ID;T-104 的跟踪模块产生 `TrackedPersonPose` 后,才允许事件证据按人员连续积累。
## 视频来源帧合约
```text
+6 -6
View File
@@ -5,13 +5,13 @@
## 当前快照
- 日期:2026-07-21
- 阶段:V1 工程化起步;T-103 进行中。
- 阶段:V1 工程化起步;T-103 已验收,等待 T-104。
- 已验证环境:Windows PowerShell;Python 3.8.10;Ultralytics 8.3.205;PyQt5 可导入。
- 旧生产基线:`demo/main.py`、`demo/fall_detection_gui.py`、`demo/detect_fall.py`、`demo/best.pt`。
- V1 代码:已建立安全配置基线,以及 `v1/video_source.py` 的 OpenCV 回放/重连适配器;它输出显式状态和单调时间戳。Pose、跟踪、证据、状态机和 GUI 尚未实现。
- V1 代码:已建立安全配置基线、OpenCV 回放/重连适配器,以及 `v1/pose.py` 的 SHA-256 锁定 Pose 适配器;真实模型 smoke 已验证首帧输出 person box 和 17 点。跟踪、证据、状态机和 GUI 尚未实现。
- V2 代码:`v2/` 目录存在但尚无实现。
- 非代码设计工件:docs/ui/silver-pose-ui-ux-spec.md、docs/ui/2026-07-20-html-prototype-plan.md、docs/ui/silver-pose-v1-prototype.html 与 docs/ui/silver-pose-v2-prototype.html 已建立。v2 HTML 是符合正式浅色 Windows 规范的当前视觉参考:浅灰蓝底、白色卡片,红色只表示确认摔倒、其弹窗和事件证据;文件名中的 v2 只表示原型设计修订,不能理解为 Go V2 实现已开始。v1 HTML 保留为历史深色对照。两者均使用顶部双 Tab、设置草稿与状态交互,且画面、事件和时间都是模拟数据,不连接真实摄像头、模型或网络,也不改变 Phase 1 任务顺序。
- 测试:`python -m compileall -q demo` 已通过;`python -m pytest v1/tests -v` 当前有 6 项配置/视频源测试并已通过。`demo/1.mp4` 的首两帧回放时间戳已验证为 0.000000 与 0.033333 秒。`init.ps1` 会检查运行时依赖、编译旧基线并运行 V1 测试,但不会安装软件包。
- 测试:`python -m compileall -q demo` 已通过;`python -m pytest v1/tests -v` 当前有 9 项配置/视频源/Pose 测试并已通过。`demo/1.mp4` 的首两帧回放时间戳已验证为 0.000000 与 0.033333 秒,首帧 Pose smoke 得到 2 名人员、每人 17 点。`init.ps1` 会检查运行时依赖、编译旧基线并运行 V1 测试,但不会安装软件包。
- 模型:`demo/best.pt` 可加载为 YOLO Pose,类别 `person`,`kpt_shape=[17, 3]`;与 `D:\PythonP\fall_detection\best.pt` 哈希一致。
- 当前标准启动:`./init.ps1`。
- 当前标准验证:`python -m compileall -q demo`。
@@ -32,9 +32,9 @@
## 任务状态
- 已完成:T-000(Harness 文档与旧基线快照)、T-101(V1 安全配置基线)、T-102(视频源与录像回放)。
- 正在进行:T-103(实现 Pose 适配器与模型来源校验)。
- 下一个可领取:完成 T-103 后为 T-104。
- 已完成:T-000(Harness 文档与旧基线快照)、T-101(V1 安全配置基线)、T-102(视频源与录像回放)、T-103(Pose 适配器与模型校验)。
- 正在进行:无。
- 下一个可领取:T-104。
## 当前可运行内容
+9
View File
@@ -116,3 +116,12 @@
- 阻塞:无。
- 决策:构造适配器时必须校验模型 SHA-256、任务为 pose、类别为 person、关键点形状为 17×3;不把任何输出命名为摔倒概率。
- 下一步:写入模型来源错误与 person Pose 输出形状的失败测试。
## 【2026-07-21】T-103 实现 Pose 适配器与模型来源校验(完成)
- 状态:DONE
- 变更:新增 `v1/pose.py`,以 SHA-256 锁定模型来源,要求任务为 pose、存在 person 类且关键点形状严格为 17×3;输出不含跟踪 ID 的 `PersonPose`(box、box 置信度、17 个三元关键点)。
- 验证:先运行 `python -m pytest v1/tests/test_pose.py -v`,确认因缺少 `v1.pose` 导入失败;实现后 Pose 测试 3 passed,完整 V1 测试在本轮最后一次运行时为 9 passed。对 `demo/best.pt` 的只读 smoke 使用实际 SHA-256 加载模型,并从 `demo/1.mp4` 首帧得到 2 名人员、每人 17 点;`python -m compileall -q v1 demo` 通过。
- 阻塞:无。
- 决策:模型输出绝不称为摔倒概率;人员 ID 由 T-104 在 Pose 输出之后分配,模型不匹配或结果形状损坏时必须显式报错而不是继续推理。
- 下一步:T-104,先写缺失下肢关键点拒绝和水平姿态只作为证据的失败测试。
+172
View File
@@ -0,0 +1,172 @@
"""Verified Ultralytics Pose adapter for the V1 event pipeline."""
import hashlib
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Callable, List, Optional, Sequence, Tuple
import numpy as np
from ultralytics import YOLO
class ModelValidationError(ValueError):
"""Raised when a model cannot satisfy the locked V1 Pose contract."""
@dataclass(frozen=True)
class Keypoint:
x: float
y: float
confidence: float
@dataclass(frozen=True)
class PersonPose:
box_xyxy: Tuple[float, float, float, float]
box_confidence: float
keypoints: Sequence[Keypoint]
ModelFactory = Callable[[str], Any]
_SHA256 = re.compile(r"^[0-9a-fA-F]{64}$")
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with Path(path).open("rb") as stream:
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
class PoseAdapter:
"""Load only the locked 17-keypoint person Pose model."""
def __init__(
self,
model_path: Path,
expected_sha256: str,
confidence_threshold: float = 0.25,
model_factory: Optional[ModelFactory] = None,
) -> None:
self._model_path = Path(model_path)
self._confidence = float(confidence_threshold)
if not 0.0 <= self._confidence <= 1.0:
raise ModelValidationError("confidence threshold must be between 0 and 1")
self._validate_hash(expected_sha256)
factory = model_factory or YOLO
try:
self._model = factory(str(self._model_path))
except Exception as exc:
raise ModelValidationError("unable to load Pose model: {0}".format(exc))
self._person_class_ids = self._validate_model_contract(self._model)
@property
def model_path(self) -> Path:
return self._model_path
def infer(self, image: np.ndarray) -> Sequence[PersonPose]:
results = self._model(image, conf=self._confidence, verbose=False)
return self.from_results(results, self._person_class_ids)
@classmethod
def from_results(
cls, results: Any, person_class_ids: Optional[Sequence[int]] = None
) -> Sequence[PersonPose]:
"""Convert Ultralytics result objects into framework-independent poses."""
if not isinstance(results, (list, tuple)):
result_items = [results]
else:
result_items = results
poses: List[PersonPose] = []
for result in result_items:
boxes = getattr(result, "boxes", None)
keypoints = getattr(result, "keypoints", None)
if boxes is None or keypoints is None:
raise ModelValidationError("Pose result is missing boxes or keypoints")
names = getattr(result, "names", {})
allowed_ids = (
set(person_class_ids)
if person_class_ids is not None
else cls._person_ids_from_names(names)
)
box_rows = cls._rows(getattr(boxes, "xyxy", None), "boxes.xyxy")
confidence_rows = cls._rows(getattr(boxes, "conf", None), "boxes.conf")
class_rows = cls._rows(getattr(boxes, "cls", None), "boxes.cls")
keypoint_rows = cls._rows(getattr(keypoints, "data", None), "keypoints.data")
if not (
len(box_rows)
== len(confidence_rows)
== len(class_rows)
== len(keypoint_rows)
):
raise ModelValidationError("Pose result arrays have inconsistent lengths")
for index, box in enumerate(box_rows):
class_id = int(class_rows[index])
if class_id not in allowed_ids:
continue
if len(box) != 4:
raise ModelValidationError("person box must contain four coordinates")
points = keypoint_rows[index]
if len(points) != 17:
raise ModelValidationError("person Pose result must contain exactly 17 keypoints")
parsed_points = []
for point in points:
if len(point) != 3:
raise ModelValidationError("keypoint must contain x, y and confidence")
parsed_points.append(
Keypoint(float(point[0]), float(point[1]), float(point[2]))
)
poses.append(
PersonPose(
box_xyxy=tuple(float(value) for value in box),
box_confidence=float(confidence_rows[index]),
keypoints=tuple(parsed_points),
)
)
return tuple(poses)
def _validate_hash(self, expected_sha256: str) -> None:
if not _SHA256.match(expected_sha256 or ""):
raise ModelValidationError("expected SHA-256 must be a 64-character hex value")
if not self._model_path.is_file():
raise ModelValidationError("Pose model file does not exist: {0}".format(self._model_path))
actual_sha256 = sha256_file(self._model_path)
if actual_sha256.lower() != expected_sha256.lower():
raise ModelValidationError("Pose model SHA-256 does not match expected value")
@classmethod
def _validate_model_contract(cls, model: Any) -> Sequence[int]:
if getattr(model, "task", None) != "pose":
raise ModelValidationError("model task must be pose")
person_ids = cls._person_ids_from_names(getattr(model, "names", {}))
raw_shape = getattr(getattr(model, "model", None), "kpt_shape", None)
if raw_shape is None:
raw_shape = getattr(model, "kpt_shape", None)
if tuple(raw_shape or ()) != (17, 3):
raise ModelValidationError("model keypoint shape must be exactly (17, 3)")
return tuple(person_ids)
@staticmethod
def _person_ids_from_names(names: Any) -> Sequence[int]:
items = names.items() if isinstance(names, dict) else enumerate(names or ())
person_ids = [int(index) for index, name in items if str(name).lower() == "person"]
if not person_ids:
raise ModelValidationError("model must expose a person class")
return tuple(person_ids)
@staticmethod
def _rows(value: Any, field_name: str) -> List[Any]:
if value is None:
raise ModelValidationError("Pose result is missing {0}".format(field_name))
if hasattr(value, "cpu"):
value = value.cpu()
if hasattr(value, "numpy"):
value = value.numpy()
if isinstance(value, np.ndarray):
return value.tolist()
if hasattr(value, "tolist"):
return value.tolist()
return list(value)
+66
View File
@@ -0,0 +1,66 @@
import hashlib
import numpy as np
import pytest
from v1.pose import ModelValidationError, PoseAdapter
class _FakeKeypoints:
def __init__(self, data):
self.data = data
class _FakeBoxes:
def __init__(self):
self.xyxy = np.array([[10.0, 20.0, 50.0, 90.0], [1.0, 2.0, 3.0, 4.0]])
self.conf = np.array([0.9, 0.8])
self.cls = np.array([0, 1])
class _FakeResult:
names = {0: "person", 1: "chair"}
def __init__(self):
person = [[float(index), float(index + 1), 0.9] for index in range(17)]
chair = [[float(index), float(index + 1), 0.8] for index in range(17)]
self.boxes = _FakeBoxes()
self.keypoints = _FakeKeypoints(np.array([person, chair]))
def test_pose_adapter_rejects_hash_mismatch_before_loading_model(tmp_path):
model_path = tmp_path / "pose.pt"
model_path.write_bytes(b"not a real model")
with pytest.raises(ModelValidationError, match="SHA-256"):
PoseAdapter(model_path, expected_sha256="0" * 64)
def test_from_results_extracts_person_box_confidence_and_seventeen_keypoints():
poses = PoseAdapter.from_results(_FakeResult())
assert len(poses) == 1
assert poses[0].box_xyxy == (10.0, 20.0, 50.0, 90.0)
assert poses[0].box_confidence == 0.9
assert len(poses[0].keypoints) == 17
assert poses[0].keypoints[5].confidence == 0.9
def test_pose_adapter_rejects_non_pose_model_after_hash_validation(tmp_path):
model_path = tmp_path / "model.pt"
model_path.write_bytes(b"model bytes")
expected_sha256 = hashlib.sha256(model_path.read_bytes()).hexdigest()
class _WrongTaskModel:
task = "detect"
names = {0: "person"}
class model:
kpt_shape = (17, 3)
with pytest.raises(ModelValidationError, match="task"):
PoseAdapter(
model_path,
expected_sha256=expected_sha256,
model_factory=lambda _path: _WrongTaskModel(),
)