feat(v2): implement operator monitoring window

This commit is contained in:
ila
2026-07-22 21:24:10 +08:00
parent 8e093acb9b
commit a98005e631
8 changed files with 335 additions and 58 deletions
+1 -1
View File
@@ -59,7 +59,7 @@ V2 对应模块的当前落实与 T-304 目标如下:
| 事件引擎 | `v2/internal/fall` | 复现 V1 的跟踪、证据、四态状态机和 `FallEvent` | 视频解码、声音、文件 |
| 画面与证据 | `v2/internal/render`、`v2/internal/alert` | 叠加人员骨架/状态,红色仅表示 CONFIRMED;按事件 ID 保存一次 PNG 和 JSONL | 读取 RTSP、重跑事件规则、直接操作 Walk 控件 |
| 实时装配 | `v2/internal/monitor` | 在后台串联帧→Pose→事件→渲染→证据;画面只保留最新完成帧,确认事件经独立队列送 UI | 直接操作 Walk 控件、保存凭证 |
| Windows UI | `v2/internal/ui` | Walk 顶部“监控/设置”Tab、渲染最新帧和已计算状态 | 直接读 RTSP、执行 ONNX 或事件规则 |
| Windows UI | `v2/internal/ui` | Walk 浅色顶部“监控/设置”Tab、渲染最新帧和已计算状态;确认事件仅在 UI 线程播放声音并弹窗 | 直接读 RTSP、执行 ONNX 或事件规则 |
| 应用装配 | `v2/cmd/silver-pose`(T-304) | 管理 worker 生命周期、取消、最新帧投递和依赖注入 | 重写领域规则 |
## UI 导航与配置生效生命周期
+1 -1
View File
@@ -9,7 +9,7 @@
- 已验证环境:Windows PowerShell;Python 3.8.10;Ultralytics 8.3.205;PyQt5 可导入。注意:当前 WSL/Linux 环境未安装 PyQt5,且其 ultralytics/torch 版本与该基线不兼容(模型前向不可用),故 GUI 可视化与真实模型端到端渲染只在 Windows 验证;本环境用 Qt-free 的 `view_model.py` 单元测试与假适配器管线冒烟覆盖视图逻辑。
- 旧生产基线:`demo/main.py`、`demo/fall_detection_gui.py`、`demo/detect_fall.py`、`demo/best.pt`。
- V1 代码:已建立安全配置、显式 Replay/Stream 视频源、Pose、轻量跟踪、质量/几何证据、倒地领域规则、按 ID 四态事件机及 `v1/pipeline.py` 事件管线;运行事件带非敏感 `config_version`,缺帧/低质量/断流会中断证据确认。新增 `v1/view_model.py`(Qt-free 监控视图状态与设置草稿三份隔离)、`v1/gui.py` 薄 PyQt5 双 Tab 外壳与 `v1/app.py` 装配(`FrameWorker` 只发出已判定的 `FrameAnalysis`,窗口只渲染)。新增 `v1/alerts.py`:按 `event_id` 去重,对首次 CONFIRMED 保存带标注截图、追加 JSONL 事件行,并经可注入 `AlertSink`(Windows 侧 `QtAlertSink` 提供声音与一次性弹窗)触发一次声音/弹窗。新增 `v1/camera.py`(RTSP URL 构建 + 有界连接测试抓帧)与 `config.py` 的结构化来源(host/端口/通道/账号/密码,凭证百分号编码)、`write_local_camera_source`;设置页新增摄像头连接分组、测试连接与预览,以及传输协议/连接超时/低延迟三项抓流调优(由 `config` 存值、`app` 开流前写入 `OPENCV_FFMPEG_CAPTURE_OPTIONS`)。真实海康流已用 `VideoSource(STREAM)` 连通(1920×1080)。
- V2 代码:T-303 已落地 `internal/pose`(Ultralytics 114 letterbox、BGR→RGB/CHW、YOLO Pose NMS、17 点及坐标还原)与 `internal/fall`(中心点跟踪、质量/几何证据、策略、四态状态机和事件 ID);`cmd/regression` 使用 FFmpeg 顺序回放本机录像并将 ONNX Pose 接入事件引擎。用户已手动确认 T-303 完成,但其缺少部署边界内真实正例录像的事实仍保留在诊断记录中。`cmd/ui-spike` 仍只是可编译的 Windows Walk 双 Tab 外壳。T-304 将补齐 RTSP 重连、UI 实时渲染、声音、弹窗、截图和发布打包;在这些验收完成前,V2 不能作为客户演示版。
- V2 代码:T-303 已落地 `internal/pose`(Ultralytics 114 letterbox、BGR→RGB/CHW、YOLO Pose NMS、17 点及坐标还原)与 `internal/fall`(中心点跟踪、质量/几何证据、策略、四态状态机和事件 ID);`cmd/regression` 使用 FFmpeg 顺序回放本机录像并将 ONNX Pose 接入事件引擎。用户已手动确认 T-303 完成,但其缺少部署边界内真实正例录像的事实仍保留在诊断记录中。T-304 已新增安全 V2 配置、FFprobe/FFmpeg 有界重连源、后台监控装配、叠加 PNG/JSONL 证据和浅色 Walk 双 Tab 操作窗口;窗口只接收完成帧和可靠告警队列。正式 `cmd/silver-pose`、发布包及现场 RTSP/声音/弹窗目视验收尚未完成,V2 仍不能作为客户演示版。
- 非代码设计工件:docs/ui/silver-pose-ui-ux-spec.md、docs/ui/2026-07-20-html-prototype-plan.md、docs/ui/silver-pose-v1-prototype.html 与 docs/ui/silver-pose-v2-prototype.html 已建立。v2 HTML 是符合正式浅色 Windows 规范的当前视觉参考:浅灰蓝底、白色卡片,红色只表示确认摔倒、其弹窗和事件证据;文件名中的 v2 只表示原型设计修订,不能理解为 Go V2 实现已开始。v1 HTML 保留为历史深色对照。两者均使用顶部双 Tab、设置草稿与状态交互,且画面、事件和时间都是模拟数据,不连接真实摄像头、模型或网络,也不改变 Phase 1 任务顺序。
- 测试:`python -m compileall -q v1 demo` 已通过(含 `gui.py`、`app.py`、`alerts.py`、`camera.py` 语法);`./init.ps1` 当前运行 V1 测试为 88 passed、1 skipped。T-303 的 `CGO_ENABLED=1 go test ./...` 通过。对同一 317 帧本机录像,V1 回放得到 253 个有人帧、263 人次、0 个 CONFIRMED;Go 回放得到 255 个有人帧、265 人次、0 个 CONFIRMED。该录像经用户确认是摔倒正例,二者的 0 事件是漏检,不是“无报警反例”通过:V1 在 2.400–2.467 秒仅有 3 帧横向候选,随后倒地人体的 Pose 置信度降至 0.012,并在 3.000 秒帧以 0.01 推理仍无任何人体姿态。它是室外斜视且有字幕遮挡,超出当前固定俯视大厅/走廊验收边界;降低检测阈值至 0.10 仍为 0 事件。人次差异来自 OpenCV/FFmpeg 解码像素不同,不能据此声称 Go 已完成真实正例一致性。Go 合成正例契约和 V1 状态机均在 1.81 秒时产生一个 `CONFIRMED`。Walk UI Spike 以 `CGO_ENABLED=0 go build ./cmd/ui-spike` 编译通过。这些不表示摔倒识别准确率或完整 Go 客户演示已验收。`init.ps1` 会检查运行时依赖、编译旧基线并运行 V1 测试,但不会安装软件包。
- 模型:`demo/best.pt` 可加载为 YOLO Pose,类别 `person`,`kpt_shape=[17, 3]`;与 `D:\PythonP\fall_detection\best.pt` 哈希一致。
@@ -264,7 +264,7 @@ git commit -m "feat(v2): compose realtime monitor pipeline"
- Modify: `v2/internal/ui/spec.go`
- Modify: `v2/cmd/ui-spike/main.go`
- [ ] **Step 1: Write UI state-model tests without launching a native window**
- [x] **Step 1: Write UI state-model tests without launching a native window**
```go
func TestViewModelUsesCriticalOnlyForConfirmed(t *testing.T) {
@@ -279,12 +279,12 @@ func TestSettingsNeverExposeResolvedRTSPURL(t *testing.T) {
}
```
- [ ] **Step 2: Verify the pure UI tests fail**
- [x] **Step 2: Verify the pure UI tests fail**
Run: `Set-Location v2; go test ./internal/ui -v`
Expected: FAIL until the view-model additions are created.
- [ ] **Step 3: Implement the UI model and window**
- [x] **Step 3: Implement the UI model and window**
```go
type ViewModel struct { ConnectionText, StateText, EventText, SettingsSummary string; Critical bool }
@@ -295,14 +295,12 @@ func (w *Window) Present(update monitor.Update)
Use a 1120×720 default / 960×640 minimum window with a standard top `TabWidget` named “监控” and “设置”. Monitor tab: `ImageViewModeZoom`, source/status/event cards, a clearly labeled Start/Stop button and recent-event summary. Settings tab: read-only source environment-variable readiness, model hash prefix, runtime dependency readiness and the effective event values; do not present editable credentials or a resolved URL in T-304. Apply semantic brushes from the documented light palette (`#EAF1F8` base, white surfaces, blue focus/action); state text and labels always accompany color. Use standard controls so Tab/Shift+Tab/Enter/Space/Escape behavior is inherited. `Present` must use `MainWindow.Synchronize`, dispose the previous bitmap after `ImageView.SetImage`, and show exactly one modal `walk.MsgBox` with “我已知晓” semantics per dispatcher record. Call `win.MessageBeep` when the record reaches the UI; no normal, suspect or reconnect path may beep or show a critical dialog.
- [ ] **Step 4: Build the UI and perform the manual keyboard/sizing smoke**
- [x] **Step 4: Build the UI**
Run: `Set-Location v2; $env:CGO_ENABLED='0'; go test ./internal/ui; go build ./cmd/ui-spike`
Expected: PASS.
Run: `go run ./cmd/ui-spike`
Expected: light two-tab shell; verify focus traversal, Enter/Space activation, Escape closes the alert dialog, and no red element appears before a confirmed update.
Expected: PASS. The live keyboard, sizing and alert-dialog observation is performed in Task 6 with the formal command and a local stream.
- [ ] **Step 5: Commit the UI slice**
- [x] **Step 5: Commit the UI slice**
```powershell
git add v2/internal/ui v2/cmd/ui-spike
+12 -44
View File
@@ -1,52 +1,20 @@
package main
import (
"silverpose/v2/internal/ui"
"log"
. "github.com/lxn/walk/declarative"
"silverpose/v2/internal/ui"
)
func main() {
spec := ui.MonitorWindowSpec()
MainWindow{
Title: spec.Title,
MinSize: Size{Width: spec.Width, Height: spec.Height},
Layout: VBox{},
Children: []Widget{
TabWidget{
Pages: []TabPage{
{
Title: "监控",
Layout: HBox{},
Children: []Widget{
GroupBox{
Title: "实时画面",
Layout: VBox{},
Children: []Widget{
TextLabel{Text: "视频帧将在此呈现"},
},
},
GroupBox{
Title: "当前状态",
Layout: VBox{},
Children: []Widget{
TextLabel{Text: "检测状态:NORMAL"},
TextLabel{Text: "事件:暂无"},
TextLabel{Text: "运行时:Go + ONNX Runtime"},
},
},
},
},
{
Title: "设置",
Layout: VBox{},
Children: []Widget{
TextLabel{Text: "设置将在 T-304 接入本机配置;不保存摄像头凭证。"},
},
},
},
},
},
}.Run()
window, err := ui.NewWindow(ui.Settings{
SourceEnvironment: "SILVER_POSE_RTSP_URL",
ModelSHA256: "fac42d41a2ae3108dc42b3a2097f299e27d7f4055cb6b26a319fad0b0a3f286e",
RuntimeSummary: "Go + ONNX Runtime + FFmpeg(技术 Spike)",
EventSummary: "确认窗 1.8 秒;仅确认摔倒使用红色",
}, ui.Commands{})
if err != nil {
log.Fatal(err)
}
window.Run()
}
+67 -2
View File
@@ -1,5 +1,12 @@
package ui
import (
"fmt"
"silverpose/v2/internal/fall"
"silverpose/v2/internal/source"
)
// WindowSpec locks the visual baseline tested by the Windows UI Spike. The
// production V2 app will consume the same semantic colours and medium desktop
// layout after the decoding/inference pipeline is complete.
@@ -8,15 +15,73 @@ type WindowSpec struct {
Width int
Height int
BackgroundHex string
SurfaceHex string
AccentHex string
AlertHex string
}
func MonitorWindowSpec() WindowSpec {
return WindowSpec{
Title: "Silver Pose V2 · 技术 Spike",
Title: "Silver Pose V2",
Width: 1120,
Height: 720,
BackgroundHex: "#F3F6FA",
BackgroundHex: "#EAF1F8",
SurfaceHex: "#FFFFFF",
AccentHex: "#2563EB",
AlertHex: "#C62828",
}
}
// ViewModel is a UI-thread-neutral snapshot. It deliberately stores only a
// source environment variable name and status, never a resolved RTSP URL.
type ViewModel struct {
ConnectionText string
StateText string
EventText string
SettingsSummary string
Critical bool
}
func NewViewModel() ViewModel {
return ViewModel{
ConnectionText: "等待启动监控",
StateText: "检测状态:正常",
EventText: "事件:暂无",
}
}
func (model *ViewModel) Apply(state fall.State, status source.Status) {
model.Critical = state == fall.Confirmed
switch state {
case fall.Confirmed:
model.StateText = "检测状态:确认摔倒"
model.EventText = "事件:已确认,已保存证据"
case fall.Suspect:
model.StateText = "检测状态:疑似倒地"
model.EventText = "事件:正在持续确认"
case fall.Recovering:
model.StateText = "检测状态:恢复观察中"
model.EventText = "事件:等待恢复稳定"
default:
model.StateText = "检测状态:正常"
model.EventText = "事件:暂无"
}
switch status {
case source.Connected:
model.ConnectionText = "视频流:在线"
case source.Retrying:
model.ConnectionText = "视频流:正在重连,不触发摔倒报警"
case source.Stopped:
model.ConnectionText = "视频流:已停止"
default:
model.ConnectionText = "视频流:正在连接"
}
}
func (model *ViewModel) SetSourceReady(environmentName string, ready bool) {
state := "未就绪"
if ready {
state = "已就绪"
}
model.SettingsSummary = fmt.Sprintf("来源环境变量 %s:%s", environmentName, state)
}
+2 -2
View File
@@ -4,13 +4,13 @@ import "testing"
func TestMonitorWindowSpecUsesLightMonitoringLayout(t *testing.T) {
spec := MonitorWindowSpec()
if spec.Title != "Silver Pose V2 · 技术 Spike" {
if spec.Title != "Silver Pose V2" {
t.Fatalf("title = %q", spec.Title)
}
if spec.Width != 1120 || spec.Height != 720 {
t.Fatalf("size = %dx%d, want 1120x720", spec.Width, spec.Height)
}
if spec.BackgroundHex != "#F3F6FA" || spec.AlertHex != "#C62828" {
if spec.BackgroundHex != "#EAF1F8" || spec.AlertHex != "#C62828" {
t.Fatalf("unexpected colours: background=%s alert=%s", spec.BackgroundHex, spec.AlertHex)
}
}
+210
View File
@@ -0,0 +1,210 @@
package ui
import (
"fmt"
"strings"
"github.com/lxn/walk"
d "github.com/lxn/walk/declarative"
"github.com/lxn/win"
"silverpose/v2/internal/alert"
"silverpose/v2/internal/fall"
"silverpose/v2/internal/monitor"
)
type Settings struct {
SourceEnvironment string
ModelSHA256 string
RuntimeSummary string
EventSummary string
}
type Commands struct {
Start func()
Stop func()
}
// Window is an observer of monitor updates. It does not own a stream, runtime,
// fall engine, or event artifact writer.
type Window struct {
main *walk.MainWindow
imageView *walk.ImageView
connectionLabel *walk.TextLabel
stateLabel *walk.TextLabel
eventLabel *walk.TextLabel
startButton *walk.PushButton
stopButton *walk.PushButton
model ViewModel
bitmap *walk.Bitmap
seenAlerts map[string]struct{}
commands Commands
}
func NewWindow(settings Settings, commands Commands) (*Window, error) {
spec := MonitorWindowSpec()
window := &Window{model: NewViewModel(), seenAlerts: make(map[string]struct{}), commands: commands}
window.model.SetSourceReady(settings.SourceEnvironment, settings.SourceEnvironment != "")
background := d.SolidColorBrush{Color: walk.RGB(234, 241, 248)}
surface := d.SolidColorBrush{Color: walk.RGB(255, 255, 255)}
if err := (d.MainWindow{
AssignTo: &window.main,
Title: spec.Title,
Size: d.Size{Width: spec.Width, Height: spec.Height},
MinSize: d.Size{Width: 960, Height: 640},
Background: background,
Layout: d.VBox{},
Children: []d.Widget{
d.TabWidget{
Pages: []d.TabPage{
{
Title: "监控",
Layout: d.HBox{},
Children: []d.Widget{
d.Composite{
Background: surface,
Layout: d.VBox{},
StretchFactor: 3,
Children: []d.Widget{
d.TextLabel{Text: "实时画面", ToolTipText: "显示完成推理与姿态叠加后的最新画面"},
d.ImageView{AssignTo: &window.imageView, Mode: d.ImageViewModeZoom, StretchFactor: 8, ToolTipText: "实时姿态画面"},
},
},
d.Composite{
Background: surface,
Layout: d.VBox{},
StretchFactor: 2,
Children: []d.Widget{
d.TextLabel{Text: "运行状态"},
d.TextLabel{AssignTo: &window.connectionLabel, Text: window.model.ConnectionText},
d.TextLabel{AssignTo: &window.stateLabel, Text: window.model.StateText},
d.TextLabel{AssignTo: &window.eventLabel, Text: window.model.EventText},
d.PushButton{AssignTo: &window.startButton, Text: "开始监控", ToolTipText: "使用当前配置启动单路监控", OnClicked: window.start},
d.PushButton{AssignTo: &window.stopButton, Text: "停止监控", Enabled: false, ToolTipText: "停止监控并释放本地运行资源", OnClicked: window.stop},
d.TextLabel{Text: "红色仅表示已确认摔倒。连接异常不会触发报警。"},
},
},
},
},
{
Title: "设置",
Layout: d.VBox{},
Children: []d.Widget{
d.TextLabel{Text: "来源与运行依赖(只读,不显示 RTSP 地址或密码)"},
d.TextLabel{Text: window.model.SettingsSummary},
d.TextLabel{Text: "模型 SHA-256:" + shortHash(settings.ModelSHA256)},
d.TextLabel{Text: "运行依赖:" + settings.RuntimeSummary},
d.TextLabel{Text: "事件参数:" + settings.EventSummary},
d.TextLabel{Text: "参数在配置文件中维护;修改后请停止并重新开始监控。"},
},
},
},
},
},
}).Create(); err != nil {
return nil, err
}
window.main.Closing().Attach(func(canceled *bool, reason walk.CloseReason) {
window.stop()
if window.bitmap != nil {
window.bitmap.Dispose()
window.bitmap = nil
}
})
return window, nil
}
func (window *Window) Run() int { return window.main.Run() }
func (window *Window) start() {
if window.commands.Start != nil {
window.commands.Start()
}
window.startButton.SetEnabled(false)
window.stopButton.SetEnabled(true)
}
func (window *Window) stop() {
if window.commands.Stop != nil {
window.commands.Stop()
}
if window.startButton != nil {
window.startButton.SetEnabled(true)
}
if window.stopButton != nil {
window.stopButton.SetEnabled(false)
}
}
// Present runs only the final state assignment on the UI thread.
func (window *Window) Present(update monitor.Update) {
window.main.Synchronize(func() {
state := highestState(update.Result)
window.model.Apply(state, update.SourceStatus)
if update.SourceMessage != "" {
window.model.ConnectionText = "视频流:" + update.SourceMessage
}
_ = window.connectionLabel.SetText(window.model.ConnectionText)
_ = window.stateLabel.SetText(window.model.StateText)
_ = window.eventLabel.SetText(window.model.EventText)
if update.Image == nil {
return
}
bitmap, err := walk.NewBitmapFromImage(update.Image)
if err != nil {
return
}
if err := window.imageView.SetImage(bitmap); err != nil {
bitmap.Dispose()
return
}
previous := window.bitmap
window.bitmap = bitmap
if previous != nil {
previous.Dispose()
}
})
}
// PresentAlert is called by a consumer of Monitor.Alerts. Event IDs are checked
// again at the UI boundary so a reconnect/repaint cannot replay sound or popup.
func (window *Window) PresentAlert(record alert.Record) {
if !record.Written {
return
}
window.main.Synchronize(func() {
if _, seen := window.seenAlerts[record.Event.EventID]; seen {
return
}
window.seenAlerts[record.Event.EventID] = struct{}{}
win.MessageBeep(win.MB_ICONHAND)
message := fmt.Sprintf("确认摔倒\n人员:%s\n延迟:%.2f 秒\n截图:%s", record.Event.TrackID, record.Event.LatencySeconds, record.ScreenshotPath)
walk.MsgBox(window.main, "Silver Pose V2 · 确认摔倒", message, walk.MsgBoxOK|walk.MsgBoxIconError)
})
}
func highestState(result fall.FrameResult) fall.State {
state := fall.Normal
for _, person := range result.People {
switch person.State {
case fall.Confirmed:
return fall.Confirmed
case fall.Suspect:
if state == fall.Normal || state == fall.Recovering {
state = fall.Suspect
}
case fall.Recovering:
if state == fall.Normal {
state = fall.Recovering
}
}
}
return state
}
func shortHash(value string) string {
if len(value) <= 12 {
return value
}
return strings.ToLower(value[:12]) + "…"
}
+36
View File
@@ -0,0 +1,36 @@
package ui
import (
"strings"
"testing"
"silverpose/v2/internal/fall"
"silverpose/v2/internal/source"
)
func TestViewModelUsesCriticalOnlyForConfirmed(t *testing.T) {
view := NewViewModel()
view.Apply(fall.Normal, source.Connected)
if view.Critical {
t.Fatal("normal monitoring state is critical")
}
view.Apply(fall.Confirmed, source.Connected)
if !view.Critical || !strings.Contains(view.StateText, "确认摔倒") {
t.Fatalf("confirmed view = %#v", view)
}
view.Apply(fall.Normal, source.Retrying)
if view.Critical || !strings.Contains(view.ConnectionText, "重连") {
t.Fatalf("retrying view = %#v", view)
}
}
func TestSettingsNeverExposeResolvedRTSPURL(t *testing.T) {
view := NewViewModel()
view.SetSourceReady("SILVER_POSE_RTSP_URL", true)
if strings.Contains(view.SettingsSummary, "rtsp://") || strings.Contains(view.SettingsSummary, "secret") {
t.Fatalf("settings leaked source: %q", view.SettingsSummary)
}
if !strings.Contains(view.SettingsSummary, "SILVER_POSE_RTSP_URL") {
t.Fatalf("settings omitted source environment name: %q", view.SettingsSummary)
}
}