26 Commits
Author SHA1 Message Date
ila 2e04737922 Merge T-017 issue mapping
Maps T-017 to Gitea issue 59.
2026-08-11 10:30:30 +08:00
QiuSW 2768cfeacb docs(tasks): map T-017 to issue 59
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-11 10:18:40 +08:00
ila 5c9318ceef Merge T-017 task definition
Defines the scoped Brain single-stream visual prototype task.
2026-08-11 10:17:33 +08:00
QiuSW 5de6f0c2bb docs(tasks): define T-017 Brain visual prototype
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-11 10:17:07 +08:00
ila 2b9a59fe8a Merge T-016 Sense audit relay
Harness governance / validate (push) Has been cancelled
Implements immutable Bell audit storage and fenced Sense Outbox delivery.
2026-08-11 00:26:08 +08:00
QiuSW 4be2386421 feat: deliver Sense audits to Bell (T-016)
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-11 00:24:32 +08:00
ila aca22f4667 Merge pull request '[T-016] 回填 Gitea Issue 映射' (#56) from governance/T-016-map into main
Harness governance / validate (push) Has been cancelled
2026-08-10 23:59:15 +08:00
QiuSW 88b92d8811 docs(tasks): map T-016 to issue 55
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-10 23:59:00 +08:00
ila 325e6ec255 Merge pull request '[T-016] 定义 Sense 到 Bell 审计 relay 任务' (#54) from bootstrap/T-016 into main
Harness governance / validate (push) Has been cancelled
2026-08-10 23:58:23 +08:00
QiuSW e3cb1c5bc5 docs(tasks): define T-016 audit relay
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-10 23:58:03 +08:00
ila 7e66fab042 Merge pull request '[T-015] 建立 Bell 不可变事件存储' (#53) from agent/codex/T-015 into main
Harness governance / validate (push) Has been cancelled
2026-08-10 23:54:27 +08:00
QiuSW 236964d242 feat(bell): add immutable event store [T-015]
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-10 23:53:11 +08:00
ila 5b42f58f23 Merge pull request '[T-015] 回填 Gitea Issue 映射' (#52) from governance/T-015-map into main
Harness governance / validate (push) Has been cancelled
2026-08-10 23:37:22 +08:00
QiuSW 8b11b39093 docs(tasks): map T-015 to issue 51
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-10 23:37:08 +08:00
ila 31b08454f9 Merge pull request '[T-015] 定义 Bell 不可变事件存储任务' (#50) from bootstrap/T-015 into main
Harness governance / validate (push) Has been cancelled
2026-08-10 23:36:19 +08:00
QiuSW cefa6275bf docs(tasks): define T-015 Bell immutable events
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-10 23:36:01 +08:00
ila 5ab212cb5c Merge pull request '[T-014] 建立 Sense 本地 16 路批量收敛与容量基线' (#49)
Harness governance / validate (push) Has been cancelled
Closes #47
2026-08-10 23:18:52 +08:00
QiuSW b9e21a95d1 docs(task): record T-014 16-stream baseline
Harness governance / validate (push) Has been cancelled
Harness governance / validate (pull_request) Has been cancelled
2026-08-10 23:16:12 +08:00
QiuSW d029067aa7 fix(lab): refresh first T-014 batch ETags [T-014]
Harness governance / validate (push) Has been cancelled
2026-08-10 22:39:13 +08:00
QiuSW d355f3d069 fix(lab): report frozen Sense Go toolchain [T-014]
Harness governance / validate (push) Has been cancelled
2026-08-10 22:35:25 +08:00
QiuSW 71e5e6e599 fix(lab): schedule all T-014 stability samples [T-014]
Harness governance / validate (push) Has been cancelled
2026-08-08 11:18:46 +08:00
QiuSW c3cfaff614 fix(lab): release T-014 process handles before cleanup [T-014]
Harness governance / validate (push) Has been cancelled
2026-08-08 10:45:23 +08:00
QiuSW 707ca8e508 fix(lab): avoid PostgreSQL pipeline deadlock [T-014]
Harness governance / validate (push) Has been cancelled
2026-08-08 10:11:36 +08:00
QiuSW 403fd6e554 fix(lab): bound T-014 process cleanup [T-014]
Harness governance / validate (push) Has been cancelled
2026-08-08 09:52:28 +08:00
QiuSW 5d222436f1 feat(lab): add 16-stream capacity harness [T-014]
Harness governance / validate (push) Has been cancelled
2026-08-08 09:10:40 +08:00
QiuSW 3af6a16740 chore(task): claim T-014
Harness governance / validate (push) Has been cancelled
2026-08-08 08:56:00 +08:00
57 changed files with 5086 additions and 54 deletions
+4 -1
View File
@@ -6,7 +6,7 @@
YoVision 是智能视频事件平台:统一接入 ONVIF/RTSP 摄像头与后续异构传感器,完成检测、规则判定、事件留证和分级预警。
当前为 **M0:摄像头兼容性验证与需求定稿**。默认交付 16 路,单站点按 32/64/128 路横向扩展;16 只能是默认配额,不能成为代码、数据库、数组、分页或批量操作的硬上限。
当前已完成 M0、M1 和 M2 的本地基线,进入 **M3:Bell/Brain 最小事件闭环**。默认交付 16 路,单站点按 32/64/128 路横向扩展;16 只能是默认配额,不能成为代码、数据库、数组、分页或批量操作的硬上限。
## 固定阅读顺序
@@ -83,6 +83,9 @@ Gitea 不可用时,只能继续已经确认属于自己的任务;不得领
python scripts/validate_agent_context.py
python -m unittest discover -s tests -p "test_*.py"
python scripts/validate_harness_governance.py
go -C Bell test ./...
go -C Bell vet ./...
go -C Bell build ./...
```
代码出现后,还必须执行 `docs/03-tech-stack.md` 中与本任务命中的模块测试;完整门禁、设备验收和容量验收按任务文件触发。提交前检查 `git status --short`、`git diff`、`git diff --cached` 与 `git diff --check`。
-1
View File
@@ -1 +0,0 @@
+30
View File
@@ -0,0 +1,30 @@
# Bell 事件存储与内部审计入口
Bell 当前实现 M3 的事件域基础及 Sense 审计 relay 的最小内部 HTTP 服务:
- Bell 在可信 ingress 内为不含 `id` 的候选事实生成 `evt_` ULID。
- 最终事件同时通过冻结 v0.1 JSON Schema 与六项代码级断言。
- PostgreSQL `bell.events` 保存不可变事实;后续 outcome 追加到 `bell.event_outcomes`。
- `bell_runtime` 对三张不可变事实表只有 `SELECT/INSERT`,没有 `UPDATE/DELETE/TRUNCATE` 或 migration owner 权限;仅可在短期 `audit_relay_receipts` 表查询、插入和清理过期收据。
- `cmd/bell-api` 默认只监听 `127.0.0.1:8081`,接收 HMAC 签名的 `/internal/v1/audit-events:batch`,把脱敏设备操作事实追加到 `bell.audit_events`。
- `(key_id, nonce)` 收据保存 10 分钟;相同摘要重放原结果,不同摘要返回冲突。非回环监听必须配置 TLS 证书和私钥。
Brain→Bell transport、公共认证/事件 API、规则、Alert 和证据对象存储仍需后续任务冻结。审计 relay 只服务 Sense,不得把 `internal/event` 的 Go 类型或该 HMAC 适配器当成公共协议。
启动内部 receiver 前必须私下设置 `BELL_DB_DSN` 和仓库外绝对路径 `BELL_AUDIT_KEYS_FILE`。远端监听还必须设置 `BELL_TLS_CERT_FILE`、`BELL_TLS_KEY_FILE`;仓库不保存 DSN、key 或证书:
```powershell
go -C Bell run ./cmd/bell-api
```
## 验证
```powershell
go -C Bell mod download
go -C Bell test ./...
go -C Bell vet ./...
go -C Bell build ./...
./scripts/test_postgres.ps1 -PgRoot D:\pgsql17
```
隔离 PostgreSQL harness 会创建临时 `bell_runtime` 登录成员,运行真实 repository、幂等冲突和不可变权限测试,然后停止并删除临时集群;不会读取或修改现有 `D:\pgsql17\data` 或 5432 服务。
+152
View File
@@ -0,0 +1,152 @@
package main
import (
"context"
"crypto/tls"
"errors"
"fmt"
"log/slog"
"net"
"net/http"
"os"
"os/signal"
"path/filepath"
"syscall"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/stdlib"
"yovision/bell/internal/audit"
"yovision/bell/internal/store"
)
var version = "dev"
type configuration struct {
address string
dsn string
keyFile string
tlsCert string
tlsKey string
}
func main() {
logger := slog.New(slog.NewJSONHandler(os.Stdout, nil))
if err := run(logger); err != nil {
logger.Error("Bell stopped", "error", err)
os.Exit(1)
}
}
func loadConfiguration() (configuration, error) {
value := configuration{
address: envOr("BELL_HTTP_ADDR", "127.0.0.1:8081"),
dsn: os.Getenv("BELL_DB_DSN"),
keyFile: os.Getenv("BELL_AUDIT_KEYS_FILE"),
tlsCert: os.Getenv("BELL_TLS_CERT_FILE"),
tlsKey: os.Getenv("BELL_TLS_KEY_FILE"),
}
if value.dsn == "" {
return configuration{}, errors.New("BELL_DB_DSN is required")
}
if value.keyFile == "" || !filepath.IsAbs(value.keyFile) {
return configuration{}, errors.New("BELL_AUDIT_KEYS_FILE must be an absolute external path")
}
host, _, err := net.SplitHostPort(value.address)
if err != nil {
return configuration{}, errors.New("invalid BELL_HTTP_ADDR")
}
ip := net.ParseIP(host)
loopback := host == "localhost" || (ip != nil && ip.IsLoopback())
if !loopback && (value.tlsCert == "" || value.tlsKey == "" || !filepath.IsAbs(value.tlsCert) || !filepath.IsAbs(value.tlsKey)) {
return configuration{}, errors.New("non-loopback Bell bind requires absolute TLS certificate and key paths")
}
if (value.tlsCert == "") != (value.tlsKey == "") {
return configuration{}, errors.New("Bell TLS certificate and key must be configured together")
}
return value, nil
}
func run(logger *slog.Logger) error {
cfg, err := loadConfiguration()
if err != nil {
return err
}
pgConfig, err := pgx.ParseConfig(cfg.dsn)
if err != nil {
return errors.New("invalid Bell postgres DSN")
}
if pgConfig.RuntimeParams == nil {
pgConfig.RuntimeParams = make(map[string]string)
}
pgConfig.RuntimeParams["application_name"] = "yovision-bell"
db := stdlib.OpenDB(*pgConfig)
db.SetMaxOpenConns(16)
db.SetMaxIdleConns(4)
db.SetConnMaxLifetime(30 * time.Minute)
defer db.Close()
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
repository, err := store.OpenPostgres(ctx, db)
if err != nil {
return err
}
if err := repository.AuditRelayReady(ctx); err != nil {
return err
}
keys, err := audit.LoadKeys(cfg.keyFile)
if err != nil {
return err
}
handler, err := audit.NewHandler(repository, keys)
if err != nil {
return err
}
mux := http.NewServeMux()
mux.Handle(audit.RelayPath, handler)
mux.HandleFunc("GET /healthz", func(writer http.ResponseWriter, _ *http.Request) {
writeStatus(writer, http.StatusOK, "ok")
})
mux.HandleFunc("GET /readyz", func(writer http.ResponseWriter, request *http.Request) {
if err := repository.AuditRelayReady(request.Context()); err != nil {
writeStatus(writer, http.StatusServiceUnavailable, "not_ready")
return
}
writeStatus(writer, http.StatusOK, "ready")
})
server := &http.Server{Addr: cfg.address, Handler: mux, ReadHeaderTimeout: 5 * time.Second, ReadTimeout: 15 * time.Second, WriteTimeout: 15 * time.Second, IdleTimeout: 60 * time.Second, TLSConfig: &tls.Config{MinVersion: tls.VersionTLS12}}
serverErrors := make(chan error, 1)
go func() {
logger.Info("Bell listening", "address", cfg.address, "version", version, "tls_enabled", cfg.tlsCert != "")
if cfg.tlsCert != "" {
serverErrors <- server.ListenAndServeTLS(cfg.tlsCert, cfg.tlsKey)
return
}
serverErrors <- server.ListenAndServe()
}()
select {
case <-ctx.Done():
case serverErr := <-serverErrors:
if !errors.Is(serverErr, http.ErrServerClosed) {
return fmt.Errorf("serve Bell HTTP: %w", serverErr)
}
}
shutdownContext, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
return server.Shutdown(shutdownContext)
}
func writeStatus(writer http.ResponseWriter, status int, value string) {
writer.Header().Set("Content-Type", "application/json")
writer.WriteHeader(status)
_, _ = fmt.Fprintf(writer, `{"status":%q}`, value)
}
func envOr(name, fallback string) string {
if value := os.Getenv(name); value != "" {
return value
}
return fallback
}
+33
View File
@@ -0,0 +1,33 @@
package main
import (
"path/filepath"
"testing"
)
func TestConfigurationRequiresDatabaseAndExternalKey(t *testing.T) {
t.Setenv("BELL_DB_DSN", "")
t.Setenv("BELL_AUDIT_KEYS_FILE", "")
if _, err := loadConfiguration(); err == nil {
t.Fatal("missing Bell database was accepted")
}
t.Setenv("BELL_DB_DSN", "postgres://bell@127.0.0.1/yovision")
t.Setenv("BELL_AUDIT_KEYS_FILE", "relative.json")
if _, err := loadConfiguration(); err == nil {
t.Fatal("relative Bell key file was accepted")
}
}
func TestConfigurationRequiresTLSOutsideLoopback(t *testing.T) {
t.Setenv("BELL_DB_DSN", "postgres://bell@127.0.0.1/yovision")
t.Setenv("BELL_AUDIT_KEYS_FILE", filepath.Join(t.TempDir(), "keys.json"))
t.Setenv("BELL_HTTP_ADDR", "0.0.0.0:8081")
if _, err := loadConfiguration(); err == nil {
t.Fatal("remote plaintext Bell bind was accepted")
}
t.Setenv("BELL_TLS_CERT_FILE", filepath.Join(t.TempDir(), "server.crt"))
t.Setenv("BELL_TLS_KEY_FILE", filepath.Join(t.TempDir(), "server.key"))
if _, err := loadConfiguration(); err != nil {
t.Fatalf("remote TLS Bell bind rejected: %v", err)
}
}
+9
View File
@@ -0,0 +1,9 @@
// Package contracts embeds the frozen event contract used by Bell.
package contracts
import _ "embed"
// EventV01Schema is byte-identical to docs/raw/contracts/event-v0.1.schema.json.
//
//go:embed event-v0.1.schema.json
var EventV01Schema []byte
+283
View File
@@ -0,0 +1,283 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://yovision.local/contracts/event-v0.1.schema.json",
"title": "YoVision Event Instance v0.1",
"description": "推理侧 → 平台侧的唯一契约。冻结于 2026-08-03。所有顶层键必须存在(可为 null),不允许省略——省略与显式 null 无法区分,是这类系统最常见的排查陷阱。",
"type": "object",
"additionalProperties": false,
"required": [
"schema_version",
"id",
"source_event_id",
"tenant_id",
"site_id",
"device_id",
"sensors",
"kind",
"severity",
"confidence",
"occurred_at",
"detected_at",
"latency_seconds",
"config_version",
"rule",
"subject",
"observation",
"evidence",
"dedup_key",
"aggregated_into",
"outcome",
"outcome_source",
"outcome_reason",
"diagnostics",
"ext"
],
"properties": {
"schema_version": {
"description": "契约版本。破坏性变更必须递增主版本。",
"const": "0.1"
},
"id": {
"description": "平台侧生成的全局唯一事件 ID(ULID)。推理侧不得自行生成。",
"type": "string",
"pattern": "^evt_[0-9A-HJKMNP-TV-Z]{26}$"
},
"source_event_id": {
"description": "推理侧原始事件 ID,如 silver_pose 的 FALL-<session>-000001。用于回溯本地截图文件名(截图即按它命名)。会话内唯一,全局不保证唯一——不得用作主键。",
"type": "string",
"pattern": "^[A-Za-z0-9_-]{1,128}$"
},
"tenant_id": { "type": "integer", "minimum": 1 },
"site_id": { "type": "integer", "minimum": 1 },
"device_id": {
"description": "主传感器的平台设备实体主键。由推理侧的 source_id 经平台映射表解析得到。事件中不得冗余 RTSP 地址或任何凭据。多传感器融合事件的完整来源见 sensors。",
"type": "integer",
"minimum": 1
},
"sensors": {
"description": "参与本次判定的全部传感器。单摄像头事件为单元素数组。恰好一个元素的 role 为 primary,且其 device_id 必须等于顶层 device_id。",
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": ["device_id", "modality", "role"],
"properties": {
"device_id": { "type": "integer", "minimum": 1 },
"modality": {
"description": "设备模态。决定隐私区域准入:privacy_flag 为真的区域只允许非成像模态。",
"type": "string",
"enum": ["video", "radar", "contact", "button", "wearable", "other"]
},
"role": {
"description": "primary=判定主依据;corroborating=佐证(如雷达判跌倒、门磁佐证无人离开)。",
"type": "string",
"enum": ["primary", "corroborating"]
}
}
}
},
"kind": {
"description": "事件类型。取值登记在 contracts/README.md 的类型注册表中,新增类型不需要升 schema 版本。v0.1 已登记:fall。",
"type": "string",
"pattern": "^[a-z][a-z0-9_]{2,63}$"
},
"severity": {
"type": "string",
"enum": ["low", "medium", "high", "critical"]
},
"confidence": {
"description": "模型置信度。几何+状态机判定链路没有天然来源,必须填 null——不得用任意常量或阈值余量伪造。",
"type": ["number", "null"],
"minimum": 0,
"maximum": 1
},
"occurred_at": {
"description": "事发时刻(墙钟 UTC)。决定证据回捞窗口。推理侧若只有单调时钟,按 detected_at - latency_seconds 换算。",
"type": "string",
"format": "date-time"
},
"detected_at": {
"description": "判定成立时刻(墙钟 UTC)。决定 SLA 计算。必须 >= occurred_at。",
"type": "string",
"format": "date-time"
},
"latency_seconds": {
"description": "从可疑到确认的耗时。可由两时间戳相减,但显式存储:它是判定质量的直接指标——贴近确认窗口下限说明证据干脆,贴近上限是误报高发区,为误报排查的首选排序键。",
"type": "number",
"minimum": 0
},
"config_version": {
"description": "产出本事件时整套判定配置的版本。粒度高于 rule.version(阈值往往是全局的),用于调参后的回归对比。不得为空串。",
"type": "string",
"minLength": 1,
"maxLength": 128
},
"rule": {
"description": "命中的规则实体。推理侧无规则引擎时为 null,由平台侧按 kind 反查补全。",
"type": ["object", "null"],
"additionalProperties": false,
"required": ["id", "version", "code"],
"properties": {
"id": { "type": "string" },
"version": { "type": "integer", "minimum": 1 },
"code": { "type": "string" }
}
},
"subject": {
"type": "object",
"additionalProperties": false,
"required": ["class", "track_id", "attributes", "anon_id", "identity", "identity_status"],
"properties": {
"class": { "type": "string", "enum": ["person", "vehicle", "object"] },
"track_id": {
"description": "跟踪器内的短期标识,跨会话不保证稳定。",
"type": "string",
"minLength": 1
},
"attributes": {
"description": "A 类属性(年龄段、着装等)。未启用时为空对象,不是 null。",
"type": "object"
},
"anon_id": {
"description": "B+ 类 ReID 匿名标识,站点内会话级有效(≤30min),不做跨日长期关联。未启用为 null。",
"type": ["string", "null"]
},
"identity": {
"description": "C 类人脸身份。仅在租户已授权且比对命中时非 null。",
"type": ["object", "null"],
"additionalProperties": false,
"required": ["person_id", "library_id", "score"],
"properties": {
"person_id": { "type": "string" },
"library_id": { "type": "string" },
"score": { "type": "number", "minimum": 0, "maximum": 1 }
}
},
"identity_status": {
"description": "必须显式。只写 null 无法区分「没开这功能」与「比对失败」,后者是需要排查的故障。",
"type": "string",
"enum": ["not_enabled", "pending", "matched", "below_threshold", "no_candidate", "timeout"]
}
}
},
"observation": {
"description": "结构化观测。bbox/keypoint 序列是数据闭环的唯一原料——只有视频与截图无法用于训练。",
"type": ["object", "null"],
"additionalProperties": false,
"required": ["zone", "dwell_sec", "bbox_seq_uri", "keypoint_seq_uri", "signal_seq_uri"],
"properties": {
"zone": { "type": ["string", "null"] },
"dwell_sec": { "type": ["number", "null"], "minimum": 0 },
"bbox_seq_uri": {
"description": "视觉模态专用。非视觉事件为 null。",
"type": ["string", "null"],
"format": "uri"
},
"keypoint_seq_uri": {
"description": "COCO-17 关键点逐帧序列(JSONL)。视觉模态专用,P1 必补项。",
"type": ["string", "null"],
"format": "uri"
},
"signal_seq_uri": {
"description": "非视觉模态的结构化序列(雷达点云轨迹与多普勒、门磁状态变迁等,JSONL)。与 keypoint_seq_uri 平级——两者是各自模态的数据闭环原料,缺任一模态的序列,该模态就无法参与模型迭代。",
"type": ["string", "null"],
"format": "uri"
}
}
},
"evidence": {
"type": "object",
"additionalProperties": false,
"required": ["snapshot_uris", "clip_uri", "clip_range"],
"properties": {
"snapshot_uris": {
"description": "证据截图。**允许为空数组**:非成像模态(雷达、门磁)产出的事件本就没有画面,隐私区域更是禁止成像。不得据此假设每个事件都有图可看——值班台 UI 必须能渲染无画面事件。文件命名只允许包含事件 ID 与日期目录,绝不得含 RTSP 地址、凭据或客户名称,文件名会出现在日志、URL 与工单中。",
"type": "array",
"minItems": 0,
"items": { "type": "string", "format": "uri" }
},
"clip_uri": {
"description": "含 pre-roll 的证据片段。仅有截图不足以让值班员判断真假,是误报反馈闭环的前置条件。P1 必补项。",
"type": ["string", "null"],
"format": "uri"
},
"clip_range": {
"type": ["array", "null"],
"minItems": 2,
"maxItems": 2,
"items": { "type": "string", "format": "date-time" }
}
}
},
"dedup_key": {
"description": "跨机位/跨时间去重键,由平台侧构造。推理侧进程内按 source_event_id 的去重仍保留——它防的是同帧重复写盘,属不同层次。",
"type": ["string", "null"]
},
"aggregated_into": {
"description": "被合并入的事件 ID。非 null 时本事件不独立触发告警。",
"type": ["string", "null"],
"pattern": "^evt_[0-9A-HJKMNP-TV-Z]{26}$"
},
"outcome": {
"description": "处置结果。事件不可变,误判只能通过本字段标记,不得删改。subject_recovered 由推理侧状态机自动回传(确认后自行起身),无需等人工。",
"type": "string",
"enum": [
"unknown",
"true_positive",
"false_positive",
"subject_recovered",
"duplicate",
"test"
]
},
"outcome_source": {
"type": ["string", "null"],
"enum": ["auto", "manual", null]
},
"outcome_reason": { "type": ["string", "null"] },
"diagnostics": {
"description": "推理侧内部诊断量,仅用于排查,平台不得依赖其语义。单调时钟跨进程无意义,不得用于任何时间计算。",
"type": ["object", "null"],
"additionalProperties": true,
"properties": {
"fsm_state": {
"type": "string",
"enum": ["NORMAL", "SUSPECT", "CONFIRMED", "RECOVERING"]
},
"suspected_at_monotonic": { "type": "number" },
"confirmed_at_monotonic": { "type": "number" },
"horizontal_angle_degrees": { "type": ["number", "null"] },
"visible_joint_count": { "type": ["integer", "null"], "minimum": 0, "maximum": 17 }
}
},
"ext": {
"description": "厂商/场景扩展位。根对象 additionalProperties=false,任何未登记字段一律放这里,避免为实验性字段升版本。",
"type": "object"
}
}
}
+19
View File
@@ -0,0 +1,19 @@
module yovision/bell
go 1.26.0
toolchain go1.26.5
require (
github.com/jackc/pgx/v5 v5.10.0
github.com/oklog/ulid/v2 v2.1.2
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2
)
require (
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
golang.org/x/sync v0.17.0 // indirect
golang.org/x/text v0.29.0 // indirect
)
+33
View File
@@ -0,0 +1,33 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dlclark/regexp2 v1.11.0 h1:G/nrcoOa7ZXlpoa/91N3X7mM3r8eIlMBBJZvsz/mxKI=
github.com/dlclark/regexp2 v1.11.0/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/oklog/ulid/v2 v2.1.2 h1:IEclFb9JNvzYA6MW2SCxbLzcHTVsfqm3PrqGQJH5zec=
github.com/oklog/ulid/v2 v2.1.2/go.mod h1:rcEKHmBBKfef9DhnvX7y1HZBYxjXb0cP5ExxNsTT1QQ=
github.com/pborman/getopt v0.0.0-20170112200414-7148bc3a4c30/go.mod h1:85jBQOZwpVEaDAr341tbn15RS4fCAsIst0qp7i8ex1o=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2 h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=
github.com/santhosh-tekuri/jsonschema/v6 v6.0.2/go.mod h1:JXeL+ps8p7/KNMjDQk3TCwPpBy0wYklyWTfbkIzdIFU=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
golang.org/x/sync v0.17.0 h1:l60nONMj9l5drqw6jlhIELNv9I0A4OFgRsG9k2oT9Ug=
golang.org/x/sync v0.17.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
golang.org/x/text v0.29.0 h1:1neNs90w9YzJ9BocxfsQNHKuAT4pkghyXc4nhZ6sJvk=
golang.org/x/text v0.29.0/go.mod h1:7MhJOA9CD2qZyOKYazxdYMF85OwPdEr9jTtBpO7ydH4=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
+363
View File
@@ -0,0 +1,363 @@
// Package audit authenticates and validates Sense audit relay batches.
package audit
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"io"
"net/http"
"regexp"
"strconv"
"strings"
"time"
"unicode/utf8"
)
const (
RelayPath = "/internal/v1/audit-events:batch"
MaxBatchSize = 100
MaxBodyBytes = 1 << 20
HeaderKeyID = "X-YoVision-Key-Id"
HeaderTimestamp = "X-YoVision-Timestamp"
HeaderNonce = "X-YoVision-Nonce"
HeaderSignature = "X-YoVision-Signature"
)
var ErrReplayConflict = errors.New("audit relay replay conflict")
type Actor struct {
Type string `json:"type"`
ID string `json:"id"`
}
type ProjectionVersions struct {
QuotaSourceVersion *int64 `json:"quota_source_version"`
AreaPolicySourceVersion *int64 `json:"area_policy_source_version"`
}
type Event struct {
EventID string `json:"event_id"`
EventType string `json:"event_type"`
TenantID string `json:"tenant_id"`
SiteID string `json:"site_id"`
DeviceID string `json:"device_id"`
Actor Actor `json:"actor"`
Reason *string `json:"reason"`
TraceID *string `json:"trace_id"`
AggregateGeneration int64 `json:"aggregate_generation"`
ProjectionVersions ProjectionVersions `json:"projection_versions"`
Data json.RawMessage `json:"data"`
OccurredAt time.Time `json:"occurred_at"`
}
type Envelope struct {
SchemaVersion int `json:"schema_version"`
Event Event `json:"event"`
}
type Result struct {
EventID string `json:"event_id"`
Status string `json:"status"`
ErrorCode *string `json:"error_code,omitempty"`
}
type BatchResponse struct {
Results []Result `json:"results"`
}
type Candidate struct {
Envelope Envelope
RecordHash [sha256.Size]byte
ErrorCode string
}
type Repository interface {
ProcessAuditBatch(context.Context, string, string, [sha256.Size]byte, []Candidate) ([]Result, error)
}
type Handler struct {
repository Repository
keys map[string][]byte
now func() time.Time
}
func NewHandler(repository Repository, keys map[string][]byte) (*Handler, error) {
if repository == nil || len(keys) == 0 {
return nil, errors.New("audit handler dependencies are required")
}
copyKeys := make(map[string][]byte, len(keys))
for id, secret := range keys {
if !keyIDPattern.MatchString(id) || len(secret) < 32 {
return nil, errors.New("invalid audit handler key")
}
copyKeys[id] = append([]byte(nil), secret...)
}
return &Handler{repository: repository, keys: copyKeys, now: time.Now}, nil
}
func (h *Handler) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
if request.Method != http.MethodPost || request.URL.Path != RelayPath {
writeError(writer, http.StatusNotFound, "not_found")
return
}
body, err := io.ReadAll(io.LimitReader(request.Body, MaxBodyBytes+1))
if err != nil || len(body) > MaxBodyBytes {
writeError(writer, http.StatusRequestEntityTooLarge, "payload_too_large")
return
}
keyID := request.Header.Get(HeaderKeyID)
timestamp := request.Header.Get(HeaderTimestamp)
nonce := request.Header.Get(HeaderNonce)
provided := request.Header.Get(HeaderSignature)
secret, ok := h.keys[keyID]
seconds, timestampErr := strconv.ParseInt(timestamp, 10, 64)
nonceBytes, nonceErr := base64.RawURLEncoding.DecodeString(nonce)
signatureBytes, signatureErr := base64.RawURLEncoding.DecodeString(provided)
if !ok || timestampErr != nil || len(timestamp) < 10 || nonceErr != nil || len(nonceBytes) < 16 || len(nonceBytes) > 48 ||
signatureErr != nil || len(signatureBytes) != sha256.Size || absDuration(h.now().UTC().Sub(time.Unix(seconds, 0).UTC())) > 300*time.Second {
writeError(writer, http.StatusUnauthorized, "unauthorized")
return
}
expected := signature(secret, canonicalString(request.Method, request.URL.EscapedPath(), timestamp, nonce, body))
if !hmac.Equal(signatureBytes, expected) {
writeError(writer, http.StatusUnauthorized, "unauthorized")
return
}
candidates, err := decodeCandidates(body)
if err != nil {
writeError(writer, http.StatusBadRequest, "invalid_batch")
return
}
requestHash := sha256.Sum256(body)
results, err := h.repository.ProcessAuditBatch(request.Context(), keyID, nonce, requestHash, candidates)
if errors.Is(err, ErrReplayConflict) {
writeError(writer, http.StatusConflict, "replay_conflict")
return
}
if err != nil {
writeError(writer, http.StatusServiceUnavailable, "temporarily_unavailable")
return
}
writeJSON(writer, http.StatusOK, BatchResponse{Results: results})
}
func canonicalString(method, path, timestamp, nonce string, body []byte) string {
digest := sha256.Sum256(body)
return strings.Join([]string{method, path, timestamp, nonce, hex.EncodeToString(digest[:])}, "\n")
}
func signature(secret []byte, canonical string) []byte {
mac := hmac.New(sha256.New, secret)
_, _ = mac.Write([]byte(canonical))
return mac.Sum(nil)
}
func absDuration(value time.Duration) time.Duration {
if value < 0 {
return -value
}
return value
}
type rawBatch struct {
Events []json.RawMessage `json:"events"`
}
func decodeCandidates(body []byte) ([]Candidate, error) {
decoder := json.NewDecoder(bytes.NewReader(body))
decoder.DisallowUnknownFields()
var batch rawBatch
if err := decoder.Decode(&batch); err != nil || len(batch.Events) < 1 || len(batch.Events) > MaxBatchSize {
return nil, errors.New("invalid audit batch")
}
var trailing any
if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) {
return nil, errors.New("invalid audit batch trailing data")
}
values := make([]Candidate, len(batch.Events))
for index, raw := range batch.Events {
values[index].RecordHash = sha256.Sum256(raw)
if !hasExactEnvelopeShape(raw) {
values[index].ErrorCode = "schema_invalid"
continue
}
itemDecoder := json.NewDecoder(bytes.NewReader(raw))
itemDecoder.DisallowUnknownFields()
if err := itemDecoder.Decode(&values[index].Envelope); err != nil {
values[index].ErrorCode = "schema_invalid"
continue
}
values[index].ErrorCode = validateEnvelope(values[index].Envelope)
}
return values, nil
}
var (
eventIDPattern = regexp.MustCompile(`^audit_[0-9a-f]{32}$`)
logicalIDPattern = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`)
keyIDPattern = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`)
)
func validateEnvelope(value Envelope) string {
event := value.Event
if (value.SchemaVersion != 1 && value.SchemaVersion != 2) || !eventIDPattern.MatchString(event.EventID) ||
!logicalIDPattern.MatchString(event.TenantID) || !logicalIDPattern.MatchString(event.SiteID) || !logicalIDPattern.MatchString(event.DeviceID) ||
event.AggregateGeneration < 1 || event.OccurredAt.IsZero() || strings.TrimSpace(event.Actor.ID) == "" || utf8.RuneCountInString(event.Actor.ID) > 200 ||
(event.Actor.Type != "user" && event.Actor.Type != "service" && event.Actor.Type != "system") ||
(event.Reason != nil && utf8.RuneCountInString(*event.Reason) > 500) || (event.TraceID != nil && utf8.RuneCountInString(*event.TraceID) > 128) ||
(event.ProjectionVersions.QuotaSourceVersion != nil && *event.ProjectionVersions.QuotaSourceVersion < 1) ||
(event.ProjectionVersions.AreaPolicySourceVersion != nil && *event.ProjectionVersions.AreaPolicySourceVersion < 1) {
return "schema_invalid"
}
if event.EventType != "device.created" && event.EventType != "device.desired_state.accepted" &&
(event.EventType != "device.configuration.accepted" || value.SchemaVersion != 2) {
return "schema_invalid"
}
var data map[string]any
if err := json.Unmarshal(event.Data, &data); err != nil || data == nil {
return "schema_invalid"
}
expected := map[string]string{
"device.created": "device_created",
"device.desired_state.accepted": "desired_state_accepted",
"device.configuration.accepted": "configuration_accepted",
}[event.EventType]
if data["kind"] != expected || !validateData(event.EventType, data) || containsSensitiveKey(data) {
return "payload_invalid"
}
return ""
}
func hasExactEnvelopeShape(raw []byte) bool {
var envelope map[string]json.RawMessage
if err := json.Unmarshal(raw, &envelope); err != nil || !exactRawKeys(envelope, "schema_version", "event") {
return false
}
var event map[string]json.RawMessage
if err := json.Unmarshal(envelope["event"], &event); err != nil || !exactRawKeys(event,
"event_id", "event_type", "tenant_id", "site_id", "device_id", "actor", "reason", "trace_id",
"aggregate_generation", "projection_versions", "data", "occurred_at",
) {
return false
}
var actor, projections map[string]json.RawMessage
return json.Unmarshal(event["actor"], &actor) == nil && exactRawKeys(actor, "type", "id") &&
json.Unmarshal(event["projection_versions"], &projections) == nil && exactRawKeys(projections, "quota_source_version", "area_policy_source_version")
}
func exactRawKeys(value map[string]json.RawMessage, expected ...string) bool {
if len(value) != len(expected) {
return false
}
for _, key := range expected {
if _, exists := value[key]; !exists {
return false
}
}
return true
}
func validateData(eventType string, data map[string]any) bool {
switch eventType {
case "device.created":
if !exactAnyKeys(data, "kind", "area_id", "modality", "capabilities", "desired_state") || !logicalIDPattern.MatchString(stringValue(data["area_id"])) {
return false
}
if !member(stringValue(data["modality"]), "video", "radar", "contact", "button", "wearable", "other") || !member(stringValue(data["desired_state"]), "disabled", "enabled") {
return false
}
return validStringSet(data["capabilities"], 16, "video_capture", "audio_capture", "spatial_rule", "telemetry")
case "device.desired_state.accepted":
if !exactAnyKeys(data, "kind", "previous_desired_state", "desired_state", "changed") {
return false
}
_, changed := data["changed"].(bool)
return changed && member(stringValue(data["previous_desired_state"]), "disabled", "enabled") && member(stringValue(data["desired_state"]), "disabled", "enabled")
case "device.configuration.accepted":
if !exactAnyKeys(data, "kind", "changed", "changed_fields", "area_id") || !logicalIDPattern.MatchString(stringValue(data["area_id"])) {
return false
}
_, changed := data["changed"].(bool)
return changed && validStringSet(data["changed_fields"], 5, "name", "area_id", "endpoint_ref", "credential_ref", "profile_token")
default:
return false
}
}
func exactAnyKeys(value map[string]any, expected ...string) bool {
if len(value) != len(expected) {
return false
}
for _, key := range expected {
if _, exists := value[key]; !exists {
return false
}
}
return true
}
func stringValue(value any) string {
result, _ := value.(string)
return result
}
func member(value string, allowed ...string) bool {
for _, candidate := range allowed {
if value == candidate {
return true
}
}
return false
}
func validStringSet(value any, maximum int, allowed ...string) bool {
items, ok := value.([]any)
if !ok || len(items) > maximum {
return false
}
seen := make(map[string]bool, len(items))
for _, item := range items {
text, ok := item.(string)
if !ok || !member(text, allowed...) || seen[text] {
return false
}
seen[text] = true
}
return true
}
func containsSensitiveKey(value any) bool {
forbidden := map[string]bool{"password": true, "stream_uri": true, "mediamtx_config": true}
switch typed := value.(type) {
case map[string]any:
for key, child := range typed {
if forbidden[strings.ToLower(key)] || containsSensitiveKey(child) {
return true
}
}
case []any:
for _, child := range typed {
if containsSensitiveKey(child) {
return true
}
}
}
return false
}
func writeError(writer http.ResponseWriter, status int, code string) {
writeJSON(writer, status, map[string]string{"error": code})
}
func writeJSON(writer http.ResponseWriter, status int, value any) {
writer.Header().Set("Content-Type", "application/json")
writer.Header().Set("Cache-Control", "no-store")
writer.WriteHeader(status)
_ = json.NewEncoder(writer).Encode(value)
}
+105
View File
@@ -0,0 +1,105 @@
package audit
import (
"bytes"
"context"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"net/http"
"net/http/httptest"
"strconv"
"testing"
"time"
)
type recordingRepository struct {
candidates []Candidate
results []Result
err error
}
func (r *recordingRepository) ProcessAuditBatch(_ context.Context, _, _ string, _ [sha256.Size]byte, values []Candidate) ([]Result, error) {
r.candidates = values
return r.results, r.err
}
func validBody(t *testing.T) []byte {
t.Helper()
value := map[string]any{"events": []any{map[string]any{
"schema_version": 1,
"event": map[string]any{
"event_id": "audit_00000000000000000000000000000001", "event_type": "device.created",
"tenant_id": "tenant", "site_id": "site", "device_id": "camera-1",
"actor": map[string]any{"type": "system", "id": "sense"}, "reason": nil, "trace_id": nil,
"aggregate_generation": 1,
"projection_versions": map[string]any{"quota_source_version": 1, "area_policy_source_version": 1},
"data": map[string]any{"kind": "device_created", "area_id": "area", "modality": "video", "capabilities": []any{"video_capture"}, "desired_state": "enabled"},
"occurred_at": "2026-08-11T00:00:00Z",
},
}}}
raw, err := json.Marshal(value)
if err != nil {
t.Fatal(err)
}
return raw
}
func signedRequest(t *testing.T, body, secret []byte, timestamp time.Time, nonce string) *http.Request {
t.Helper()
request := httptest.NewRequest(http.MethodPost, RelayPath, bytes.NewReader(body))
stamp := strconv.FormatInt(timestamp.Unix(), 10)
request.Header.Set(HeaderKeyID, "sense-a")
request.Header.Set(HeaderTimestamp, stamp)
request.Header.Set(HeaderNonce, nonce)
request.Header.Set(HeaderSignature, base64.RawURLEncoding.EncodeToString(signature(secret, canonicalString(http.MethodPost, RelayPath, stamp, nonce, body))))
return request
}
func TestHandlerAuthenticatesAndReturnsPerItemResults(t *testing.T) {
secret := bytes.Repeat([]byte{3}, 32)
repository := &recordingRepository{results: []Result{{EventID: "audit_00000000000000000000000000000001", Status: "accepted"}}}
handler, err := NewHandler(repository, map[string][]byte{"sense-a": secret})
if err != nil {
t.Fatal(err)
}
now := time.Date(2026, 8, 11, 0, 1, 0, 0, time.UTC)
handler.now = func() time.Time { return now }
response := httptest.NewRecorder()
handler.ServeHTTP(response, signedRequest(t, validBody(t), secret, now, "AAAAAAAAAAAAAAAAAAAAAA"))
if response.Code != http.StatusOK || len(repository.candidates) != 1 || repository.candidates[0].ErrorCode != "" {
t.Fatalf("valid batch rejected: status=%d candidates=%+v", response.Code, repository.candidates)
}
}
func TestHandlerRejectsStaleOrTamperedRequests(t *testing.T) {
secret := bytes.Repeat([]byte{4}, 32)
repository := &recordingRepository{}
handler, _ := NewHandler(repository, map[string][]byte{"sense-a": secret})
now := time.Date(2026, 8, 11, 0, 10, 0, 0, time.UTC)
handler.now = func() time.Time { return now }
for _, request := range []*http.Request{
signedRequest(t, validBody(t), secret, now.Add(-301*time.Second), "BBBBBBBBBBBBBBBBBBBBBB"),
signedRequest(t, append(validBody(t), ' '), bytes.Repeat([]byte{5}, 32), now, "CCCCCCCCCCCCCCCCCCCCCC"),
} {
response := httptest.NewRecorder()
handler.ServeHTTP(response, request)
if response.Code != http.StatusUnauthorized {
t.Fatalf("unsafe request returned %d", response.Code)
}
}
}
func TestDecodeCandidatesRejectsSensitiveItemWithoutRejectingBatch(t *testing.T) {
body := validBody(t)
var value map[string]any
_ = json.Unmarshal(body, &value)
events := value["events"].([]any)
event := events[0].(map[string]any)["event"].(map[string]any)
event["data"].(map[string]any)["password"] = "must-not-persist"
body, _ = json.Marshal(value)
candidates, err := decodeCandidates(body)
if err != nil || len(candidates) != 1 || candidates[0].ErrorCode != "payload_invalid" {
t.Fatalf("unexpected per-item validation: %+v %v", candidates, err)
}
}
+47
View File
@@ -0,0 +1,47 @@
package audit
import (
"bytes"
"encoding/base64"
"encoding/json"
"errors"
"io"
"os"
)
type keyDocument struct {
Version int `json:"version"`
Keys []struct {
KeyID string `json:"key_id"`
Secret string `json:"secret_base64url"`
} `json:"keys"`
}
func LoadKeys(path string) (map[string][]byte, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, errors.New("read Bell audit key file")
}
var document keyDocument
decoder := json.NewDecoder(bytes.NewReader(raw))
decoder.DisallowUnknownFields()
if err := decoder.Decode(&document); err != nil || document.Version != 1 || len(document.Keys) == 0 {
return nil, errors.New("invalid Bell audit key file")
}
var trailing any
if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) {
return nil, errors.New("invalid Bell audit key file")
}
values := make(map[string][]byte, len(document.Keys))
for _, item := range document.Keys {
secret, err := base64.RawURLEncoding.DecodeString(item.Secret)
if err != nil || !keyIDPattern.MatchString(item.KeyID) || len(secret) < 32 {
return nil, errors.New("invalid Bell audit key")
}
if _, exists := values[item.KeyID]; exists {
return nil, errors.New("duplicate Bell audit key ID")
}
values[item.KeyID] = secret
}
return values, nil
}
+302
View File
@@ -0,0 +1,302 @@
// Package event assembles and validates immutable Bell event facts.
package event
import (
"bytes"
"context"
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"io"
"math"
"net"
"net/url"
"regexp"
"strings"
"time"
"github.com/oklog/ulid/v2"
jsonschema "github.com/santhosh-tekuri/jsonschema/v6"
)
const MaxPayloadBytes = 1 << 20
type ErrorCode string
const (
CodeInvalidJSON ErrorCode = "invalid_json"
CodePayloadTooLarge ErrorCode = "payload_too_large"
CodeUpstreamID ErrorCode = "upstream_id_forbidden"
CodeSchema ErrorCode = "schema_invalid"
CodeTimeOrder ErrorCode = "time_order_invalid"
CodeLatency ErrorCode = "latency_inconsistent"
CodeConfidence ErrorCode = "confidence_forbidden"
CodeEvidence ErrorCode = "evidence_unsafe"
CodePrimarySensor ErrorCode = "primary_sensor_invalid"
CodePrivacyDenied ErrorCode = "privacy_denied"
CodePrivacyUnavailable ErrorCode = "privacy_unavailable"
)
// ValidationError exposes a stable code without returning sensitive payloads.
type ValidationError struct {
Code ErrorCode
Err error
}
func (e *ValidationError) Error() string { return string(e.Code) }
func (e *ValidationError) Unwrap() error { return e.Err }
func validationError(code ErrorCode, err error) error {
return &ValidationError{Code: code, Err: err}
}
// IDGenerator is owned by Bell. Upstream candidates are not allowed to carry id.
type IDGenerator interface {
NewEventID() (string, error)
}
type ULIDGenerator struct{}
func (ULIDGenerator) NewEventID() (string, error) {
return "evt_" + ulid.Make().String(), nil
}
// PrivacyPolicy resolves the authoritative device/Area policy. Implementations
// must fail closed when the mapping is missing or stale.
type PrivacyPolicy interface {
VideoAllowed(ctx context.Context, tenantID, siteID, deviceID int64) (bool, error)
}
// EvidencePolicy checks every evidence/observation URI before persistence.
type EvidencePolicy interface {
ValidateURI(rawURI string) error
}
// EvidenceGuard rejects reusable credentials, network endpoints and configured
// customer/tenant names from persisted evidence URIs.
type EvidenceGuard struct {
forbidden []string
}
func NewEvidenceGuard(forbiddenNames ...string) (*EvidenceGuard, error) {
guard := &EvidenceGuard{}
for _, name := range forbiddenNames {
name = strings.ToLower(strings.TrimSpace(name))
if name == "" {
return nil, errors.New("forbidden evidence name cannot be blank")
}
guard.forbidden = append(guard.forbidden, name)
}
return guard, nil
}
var ipv4Like = regexp.MustCompile(`(?:^|[^0-9])(?:[0-9]{1,3}\.){3}[0-9]{1,3}(?:[^0-9]|$)`)
func (g *EvidenceGuard) ValidateURI(rawURI string) error {
parsed, err := url.Parse(rawURI)
if err != nil || parsed.Scheme == "" {
return errors.New("evidence URI is not absolute")
}
if parsed.User != nil || parsed.Port() != "" || parsed.RawQuery != "" || parsed.Fragment != "" {
return errors.New("evidence URI contains reusable connection material")
}
if host := parsed.Hostname(); host != "" && net.ParseIP(host) != nil {
return errors.New("evidence URI contains an IP address")
}
lower := strings.ToLower(rawURI)
for _, marker := range []string{"password", "passwd", "credential", "secret", "token=", "rtsp://"} {
if strings.Contains(lower, marker) {
return errors.New("evidence URI contains a forbidden marker")
}
}
if ipv4Like.MatchString(lower) {
return errors.New("evidence URI contains an IPv4-like value")
}
for _, name := range g.forbidden {
if strings.Contains(lower, name) {
return errors.New("evidence URI contains a configured sensitive name")
}
}
return nil
}
type Sensor struct {
DeviceID int64 `json:"device_id"`
Modality string `json:"modality"`
Role string `json:"role"`
}
type storedShape struct {
ID string `json:"id"`
TenantID int64 `json:"tenant_id"`
SiteID int64 `json:"site_id"`
DeviceID int64 `json:"device_id"`
SourceEventID string `json:"source_event_id"`
Sensors []Sensor `json:"sensors"`
Kind string `json:"kind"`
Severity string `json:"severity"`
Confidence *float64 `json:"confidence"`
OccurredAt time.Time `json:"occurred_at"`
DetectedAt time.Time `json:"detected_at"`
LatencySeconds float64 `json:"latency_seconds"`
Observation *struct {
BBoxSeqURI *string `json:"bbox_seq_uri"`
KeypointSeqURI *string `json:"keypoint_seq_uri"`
SignalSeqURI *string `json:"signal_seq_uri"`
} `json:"observation"`
Evidence struct {
SnapshotURIs []string `json:"snapshot_uris"`
ClipURI *string `json:"clip_uri"`
} `json:"evidence"`
}
// Event is a final, schema-valid immutable fact. JSON returns a defensive copy.
type Event struct {
shape storedShape
payload []byte
digest [sha256.Size]byte
}
func (e Event) ID() string { return e.shape.ID }
func (e Event) TenantID() int64 { return e.shape.TenantID }
func (e Event) SiteID() int64 { return e.shape.SiteID }
func (e Event) DeviceID() int64 { return e.shape.DeviceID }
func (e Event) SourceEventID() string { return e.shape.SourceEventID }
func (e Event) Kind() string { return e.shape.Kind }
func (e Event) Severity() string { return e.shape.Severity }
func (e Event) OccurredAt() time.Time { return e.shape.OccurredAt }
func (e Event) DetectedAt() time.Time { return e.shape.DetectedAt }
func (e Event) Digest() [sha256.Size]byte { return e.digest }
func (e Event) JSON() []byte { return bytes.Clone(e.payload) }
type Factory struct {
schema *jsonschema.Schema
ids IDGenerator
privacy PrivacyPolicy
evidence EvidencePolicy
}
func NewFactory(schemaJSON []byte, ids IDGenerator, privacy PrivacyPolicy, evidence EvidencePolicy) (*Factory, error) {
if ids == nil || privacy == nil || evidence == nil {
return nil, errors.New("event factory dependencies are required")
}
schemaDoc, err := jsonschema.UnmarshalJSON(bytes.NewReader(schemaJSON))
if err != nil {
return nil, fmt.Errorf("parse event schema: %w", err)
}
compiler := jsonschema.NewCompiler()
compiler.AssertFormat()
if err := compiler.AddResource("event-v0.1.schema.json", schemaDoc); err != nil {
return nil, fmt.Errorf("register event schema: %w", err)
}
compiled, err := compiler.Compile("event-v0.1.schema.json")
if err != nil {
return nil, fmt.Errorf("compile event schema: %w", err)
}
return &Factory{schema: compiled, ids: ids, privacy: privacy, evidence: evidence}, nil
}
// Create turns a producer candidate into the final stored v0.1 event. The
// candidate must contain every v0.1 field except the Bell-owned id.
func (f *Factory) Create(ctx context.Context, candidate []byte) (Event, error) {
if len(candidate) > MaxPayloadBytes {
return Event{}, validationError(CodePayloadTooLarge, nil)
}
decoder := json.NewDecoder(bytes.NewReader(candidate))
decoder.UseNumber()
var object map[string]any
if err := decoder.Decode(&object); err != nil || object == nil {
return Event{}, validationError(CodeInvalidJSON, err)
}
var trailing any
if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) {
return Event{}, validationError(CodeInvalidJSON, errors.New("multiple JSON values"))
}
if _, exists := object["id"]; exists {
return Event{}, validationError(CodeUpstreamID, nil)
}
id, err := f.ids.NewEventID()
if err != nil {
return Event{}, fmt.Errorf("generate Bell event id: %w", err)
}
object["id"] = id
payload, err := json.Marshal(object)
if err != nil {
return Event{}, validationError(CodeInvalidJSON, err)
}
if len(payload) > MaxPayloadBytes {
return Event{}, validationError(CodePayloadTooLarge, nil)
}
instance, err := jsonschema.UnmarshalJSON(bytes.NewReader(payload))
if err != nil {
return Event{}, validationError(CodeInvalidJSON, err)
}
if err := f.schema.Validate(instance); err != nil {
return Event{}, validationError(CodeSchema, nil)
}
var shape storedShape
if err := json.Unmarshal(payload, &shape); err != nil {
return Event{}, validationError(CodeInvalidJSON, err)
}
if err := f.assertSemantics(ctx, shape); err != nil {
return Event{}, err
}
return Event{shape: shape, payload: payload, digest: sha256.Sum256(payload)}, nil
}
func (f *Factory) assertSemantics(ctx context.Context, shape storedShape) error {
if shape.DetectedAt.Before(shape.OccurredAt) {
return validationError(CodeTimeOrder, nil)
}
actual := shape.DetectedAt.Sub(shape.OccurredAt).Seconds()
if math.Abs(actual-shape.LatencySeconds) >= 0.1 {
return validationError(CodeLatency, nil)
}
if shape.Confidence != nil {
return validationError(CodeConfidence, nil)
}
primary := 0
for _, sensor := range shape.Sensors {
if sensor.Role == "primary" {
primary++
if sensor.DeviceID != shape.DeviceID {
return validationError(CodePrimarySensor, nil)
}
}
if sensor.Modality == "video" {
allowed, err := f.privacy.VideoAllowed(ctx, shape.TenantID, shape.SiteID, sensor.DeviceID)
if err != nil {
return validationError(CodePrivacyUnavailable, nil)
}
if !allowed {
return validationError(CodePrivacyDenied, nil)
}
}
}
if primary != 1 {
return validationError(CodePrimarySensor, nil)
}
var uris []string
uris = append(uris, shape.Evidence.SnapshotURIs...)
if shape.Evidence.ClipURI != nil {
uris = append(uris, *shape.Evidence.ClipURI)
}
if shape.Observation != nil {
for _, value := range []*string{
shape.Observation.BBoxSeqURI,
shape.Observation.KeypointSeqURI,
shape.Observation.SignalSeqURI,
} {
if value != nil {
uris = append(uris, *value)
}
}
}
for _, rawURI := range uris {
if err := f.evidence.ValidateURI(rawURI); err != nil {
return validationError(CodeEvidence, nil)
}
}
return nil
}
+184
View File
@@ -0,0 +1,184 @@
package event_test
import (
"context"
"encoding/json"
"errors"
"os"
"path/filepath"
"testing"
"yovision/bell/contracts"
"yovision/bell/internal/event"
)
const fixedEventID = "evt_01J8XQ2K7M3P5R9T0V4W6Y8Z2B"
type fixedIDs struct{ id string }
func (f fixedIDs) NewEventID() (string, error) { return f.id, nil }
type privacy struct {
allowed bool
err error
calls int
}
func (p *privacy) VideoAllowed(context.Context, int64, int64, int64) (bool, error) {
p.calls++
return p.allowed, p.err
}
func contractPath(name string) string {
return filepath.Join("..", "..", "..", "docs", "raw", "contracts", name)
}
func candidate(t *testing.T, name string) []byte {
t.Helper()
raw, err := os.ReadFile(contractPath(name))
if err != nil {
t.Fatal(err)
}
var object map[string]any
if err := json.Unmarshal(raw, &object); err != nil {
t.Fatal(err)
}
delete(object, "id")
encoded, err := json.Marshal(object)
if err != nil {
t.Fatal(err)
}
return encoded
}
func mutate(t *testing.T, raw []byte, fn func(map[string]any)) []byte {
t.Helper()
var object map[string]any
if err := json.Unmarshal(raw, &object); err != nil {
t.Fatal(err)
}
fn(object)
encoded, err := json.Marshal(object)
if err != nil {
t.Fatal(err)
}
return encoded
}
func factory(t *testing.T, policy *privacy) *event.Factory {
t.Helper()
guard, err := event.NewEvidenceGuard("private-customer")
if err != nil {
t.Fatal(err)
}
value, err := event.NewFactory(contracts.EventV01Schema, fixedIDs{id: fixedEventID}, policy, guard)
if err != nil {
t.Fatal(err)
}
return value
}
func assertCode(t *testing.T, err error, code event.ErrorCode) {
t.Helper()
var validation *event.ValidationError
if !errors.As(err, &validation) || validation.Code != code {
t.Fatalf("expected %s, got %v", code, err)
}
}
func TestFrozenContractCopyIsExact(t *testing.T) {
raw, err := os.ReadFile(contractPath("event-v0.1.schema.json"))
if err != nil {
t.Fatal(err)
}
if string(raw) != string(contracts.EventV01Schema) {
t.Fatal("Bell contract copy drifted from the frozen source")
}
}
func TestFactoryAcceptsAllFrozenExamples(t *testing.T) {
for _, name := range []string{
"event-v0.1.example-current.json",
"event-v0.1.example-target.json",
"event-v0.1.example-radar.json",
} {
t.Run(name, func(t *testing.T) {
policy := &privacy{allowed: true}
created, err := factory(t, policy).Create(context.Background(), candidate(t, name))
if err != nil {
t.Fatal(err)
}
if created.ID() != fixedEventID || len(created.JSON()) == 0 {
t.Fatal("Bell did not assemble the final event")
}
if name == "event-v0.1.example-radar.json" && policy.calls != 0 {
t.Fatal("non-video event unexpectedly consulted video policy")
}
})
}
}
func TestFactoryRejectsUpstreamIDAndUnknownField(t *testing.T) {
policy := &privacy{allowed: true}
base := candidate(t, "event-v0.1.example-current.json")
withID := mutate(t, base, func(object map[string]any) { object["id"] = fixedEventID })
_, err := factory(t, policy).Create(context.Background(), withID)
assertCode(t, err, event.CodeUpstreamID)
unknown := mutate(t, base, func(object map[string]any) { object["surprise"] = true })
_, err = factory(t, policy).Create(context.Background(), unknown)
assertCode(t, err, event.CodeSchema)
}
func TestFactoryEnforcesCrossFieldAssertions(t *testing.T) {
base := candidate(t, "event-v0.1.example-current.json")
tests := []struct {
name string
code event.ErrorCode
edit func(map[string]any)
}{
{"time-order", event.CodeTimeOrder, func(v map[string]any) { v["occurred_at"] = "2026-08-03T10:31:23.000Z" }},
{"latency", event.CodeLatency, func(v map[string]any) { v["latency_seconds"] = 9.0 }},
{"confidence", event.CodeConfidence, func(v map[string]any) { v["confidence"] = 0.9 }},
{"primary", event.CodePrimarySensor, func(v map[string]any) {
v["sensors"] = []any{
map[string]any{"device_id": float64(5012), "modality": "video", "role": "primary"},
map[string]any{"device_id": float64(5013), "modality": "radar", "role": "primary"},
}
}},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
_, err := factory(t, &privacy{allowed: true}).Create(context.Background(), mutate(t, base, test.edit))
assertCode(t, err, test.code)
})
}
}
func TestFactoryFailsClosedForPrivacyAndEvidence(t *testing.T) {
base := candidate(t, "event-v0.1.example-current.json")
_, err := factory(t, &privacy{err: errors.New("mapping unavailable")}).Create(context.Background(), base)
assertCode(t, err, event.CodePrivacyUnavailable)
_, err = factory(t, &privacy{allowed: false}).Create(context.Background(), base)
assertCode(t, err, event.CodePrivacyDenied)
unsafe := mutate(t, base, func(v map[string]any) {
evidence := v["evidence"].(map[string]any)
evidence["snapshot_uris"] = []any{"rtsp://user:password@10.0.0.1:554/private-customer.png"}
})
_, err = factory(t, &privacy{allowed: true}).Create(context.Background(), unsafe)
assertCode(t, err, event.CodeEvidence)
}
func TestFactoryRequiresFailClosedPoliciesAndPayloadLimit(t *testing.T) {
guard, err := event.NewEvidenceGuard()
if err != nil {
t.Fatal(err)
}
if _, err := event.NewFactory(contracts.EventV01Schema, fixedIDs{id: fixedEventID}, nil, guard); err == nil {
t.Fatal("nil privacy policy unexpectedly accepted")
}
_, err = factory(t, &privacy{allowed: true}).Create(context.Background(), make([]byte, event.MaxPayloadBytes+1))
assertCode(t, err, event.CodePayloadTooLarge)
}
+138
View File
@@ -0,0 +1,138 @@
package store
import (
"bytes"
"context"
"crypto/sha256"
"database/sql"
"encoding/json"
"errors"
"fmt"
"yovision/bell/internal/audit"
)
func (p *Postgres) AuditRelayReady(ctx context.Context) error {
var version int64
if err := p.db.QueryRowContext(ctx, `SELECT COALESCE(MAX(version), 0) FROM bell.schema_migrations`).Scan(&version); err != nil || version < 4 {
return errors.New("postgres Bell schema migration v4 is required for audit relay")
}
var auditSelect, auditInsert, auditUpdate, auditDelete, auditTruncate bool
var receiptUse bool
if err := p.db.QueryRowContext(ctx, `SELECT
has_table_privilege(current_user, 'bell.audit_events', 'SELECT'),
has_table_privilege(current_user, 'bell.audit_events', 'INSERT'),
has_table_privilege(current_user, 'bell.audit_events', 'UPDATE'),
has_table_privilege(current_user, 'bell.audit_events', 'DELETE'),
has_table_privilege(current_user, 'bell.audit_events', 'TRUNCATE'),
has_table_privilege(current_user, 'bell.audit_relay_receipts', 'SELECT,INSERT,DELETE')`).Scan(
&auditSelect, &auditInsert, &auditUpdate, &auditDelete, &auditTruncate, &receiptUse,
); err != nil {
return errors.New("verify Bell audit relay privileges")
}
if !auditSelect || !auditInsert || auditUpdate || auditDelete || auditTruncate || !receiptUse {
return errors.New("Bell audit relay privileges violate append-only boundary")
}
return nil
}
func (p *Postgres) ProcessAuditBatch(
ctx context.Context,
keyID, nonce string,
requestHash [sha256.Size]byte,
candidates []audit.Candidate,
) ([]audit.Result, error) {
if len(candidates) < 1 || len(candidates) > audit.MaxBatchSize {
return nil, errors.New("invalid audit candidate batch")
}
tx, err := p.db.BeginTx(ctx, nil)
if err != nil {
return nil, errors.New("begin Bell audit batch")
}
defer tx.Rollback()
if _, err := tx.ExecContext(ctx, `SELECT pg_advisory_xact_lock(hashtext($1), hashtext($2))`, keyID, nonce); err != nil {
return nil, errors.New("lock Bell audit receipt")
}
if _, err := tx.ExecContext(ctx, `DELETE FROM bell.audit_relay_receipts WHERE expires_at <= clock_timestamp()`); err != nil {
return nil, errors.New("expire Bell audit receipts")
}
var existingHash, existingBody []byte
err = tx.QueryRowContext(ctx, `SELECT request_hash, response_body::text
FROM bell.audit_relay_receipts WHERE key_id=$1 AND nonce=$2`, keyID, nonce).Scan(&existingHash, &existingBody)
if err == nil {
if !bytes.Equal(existingHash, requestHash[:]) {
return nil, audit.ErrReplayConflict
}
var response audit.BatchResponse
if err := json.Unmarshal(existingBody, &response); err != nil {
return nil, errors.New("decode stored Bell audit receipt")
}
if err := tx.Commit(); err != nil {
return nil, errors.New("commit Bell audit replay")
}
return response.Results, nil
}
if !errors.Is(err, sql.ErrNoRows) {
return nil, errors.New("read Bell audit receipt")
}
results := make([]audit.Result, 0, len(candidates))
for _, candidate := range candidates {
if candidate.ErrorCode != "" {
code := candidate.ErrorCode
results = append(results, audit.Result{EventID: candidate.Envelope.Event.EventID, Status: "rejected", ErrorCode: &code})
continue
}
event := candidate.Envelope.Event
payload, err := json.Marshal(event)
if err != nil {
return nil, errors.New("encode Bell audit fact")
}
result, err := tx.ExecContext(ctx, `INSERT INTO bell.audit_events(
source_system, event_id, schema_version, event_type, tenant_id, site_id,
device_id, actor_type, actor_id, reason, trace_id, aggregate_generation,
quota_source_version, area_policy_source_version, payload, occurred_at, record_hash
) VALUES ('sense',$1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14::jsonb,$15,$16)
ON CONFLICT (source_system,event_id) DO NOTHING`,
event.EventID, candidate.Envelope.SchemaVersion, event.EventType, event.TenantID,
event.SiteID, event.DeviceID, event.Actor.Type, event.Actor.ID, event.Reason,
event.TraceID, event.AggregateGeneration,
event.ProjectionVersions.QuotaSourceVersion,
event.ProjectionVersions.AreaPolicySourceVersion,
payload, event.OccurredAt, candidate.RecordHash[:])
if err != nil {
return nil, fmt.Errorf("insert Bell audit fact: %w", err)
}
affected, err := result.RowsAffected()
if err != nil {
return nil, errors.New("read Bell audit insert result")
}
if affected == 1 {
results = append(results, audit.Result{EventID: event.EventID, Status: "accepted"})
continue
}
var storedHash []byte
if err := tx.QueryRowContext(ctx, `SELECT record_hash FROM bell.audit_events
WHERE source_system='sense' AND event_id=$1`, event.EventID).Scan(&storedHash); err != nil {
return nil, errors.New("read existing Bell audit fact")
}
if bytes.Equal(storedHash, candidate.RecordHash[:]) {
results = append(results, audit.Result{EventID: event.EventID, Status: "duplicate"})
} else {
code := "id_conflict"
results = append(results, audit.Result{EventID: event.EventID, Status: "rejected", ErrorCode: &code})
}
}
encoded, err := json.Marshal(audit.BatchResponse{Results: results})
if err != nil {
return nil, errors.New("encode Bell audit response")
}
if _, err := tx.ExecContext(ctx, `INSERT INTO bell.audit_relay_receipts(
key_id, nonce, request_hash, response_status, response_body, expires_at
) VALUES ($1,$2,$3,200,$4::jsonb,clock_timestamp() + interval '10 minutes')`, keyID, nonce, requestHash[:], encoded); err != nil {
return nil, errors.New("insert Bell audit receipt")
}
if err := tx.Commit(); err != nil {
return nil, errors.New("commit Bell audit batch")
}
return results, nil
}
@@ -0,0 +1,82 @@
package store
import (
"context"
"crypto/sha256"
"database/sql"
"encoding/json"
"errors"
"os"
"testing"
"time"
_ "github.com/jackc/pgx/v5/stdlib"
"yovision/bell/internal/audit"
)
func auditCandidate(t *testing.T, eventID, actorID string) audit.Candidate {
t.Helper()
data := json.RawMessage(`{"kind":"device_created","area_id":"area","modality":"video","capabilities":["video_capture"],"desired_state":"enabled"}`)
value := audit.Envelope{SchemaVersion: 1, Event: audit.Event{
EventID: eventID, EventType: "device.created", TenantID: "tenant", SiteID: "site", DeviceID: "camera-1",
Actor: audit.Actor{Type: "system", ID: actorID}, AggregateGeneration: 1,
ProjectionVersions: audit.ProjectionVersions{}, Data: data, OccurredAt: time.Date(2026, 8, 11, 0, 0, 0, 0, time.UTC),
}}
raw, err := json.Marshal(value)
if err != nil {
t.Fatal(err)
}
return audit.Candidate{Envelope: value, RecordHash: sha256.Sum256(raw)}
}
func TestPostgresAuditBatchReceiptAndImmutableFact(t *testing.T) {
dsn := os.Getenv("YOVISION_TEST_BELL_POSTGRES_DSN")
if dsn == "" {
t.Skip("YOVISION_TEST_BELL_POSTGRES_DSN is not set")
}
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
ctx := context.Background()
repository, err := OpenPostgres(ctx, db)
if err != nil {
t.Fatal(err)
}
if err := repository.AuditRelayReady(ctx); err != nil {
t.Fatal(err)
}
requestHash := sha256.Sum256([]byte("request-one"))
eventID := "audit_10000000000000000000000000000001"
results, err := repository.ProcessAuditBatch(ctx, "sense-a", "AAAAAAAAAAAAAAAAAAAAAA", requestHash, []audit.Candidate{auditCandidate(t, eventID, "sense")})
if err != nil || len(results) != 1 || results[0].Status != "accepted" {
t.Fatalf("first batch: %+v %v", results, err)
}
replayed, err := repository.ProcessAuditBatch(ctx, "sense-a", "AAAAAAAAAAAAAAAAAAAAAA", requestHash, []audit.Candidate{auditCandidate(t, eventID, "ignored-by-receipt")})
if err != nil || replayed[0].Status != "accepted" {
t.Fatalf("receipt replay: %+v %v", replayed, err)
}
different := sha256.Sum256([]byte("request-two"))
if _, err := repository.ProcessAuditBatch(ctx, "sense-a", "AAAAAAAAAAAAAAAAAAAAAA", different, []audit.Candidate{auditCandidate(t, eventID, "sense")}); !errors.Is(err, audit.ErrReplayConflict) {
t.Fatalf("expected replay conflict, got %v", err)
}
duplicate, err := repository.ProcessAuditBatch(ctx, "sense-a", "BBBBBBBBBBBBBBBBBBBBBB", different, []audit.Candidate{auditCandidate(t, eventID, "sense")})
if err != nil || duplicate[0].Status != "duplicate" {
t.Fatalf("event duplicate: %+v %v", duplicate, err)
}
conflictHash := sha256.Sum256([]byte("request-three"))
conflict, err := repository.ProcessAuditBatch(ctx, "sense-a", "CCCCCCCCCCCCCCCCCCCCCC", conflictHash, []audit.Candidate{auditCandidate(t, eventID, "other")})
if err != nil || conflict[0].Status != "rejected" || conflict[0].ErrorCode == nil || *conflict[0].ErrorCode != "id_conflict" {
t.Fatalf("event conflict: %+v %v", conflict, err)
}
if _, err := db.ExecContext(ctx, `UPDATE bell.audit_events SET actor_id='mutated' WHERE event_id=$1`, eventID); err == nil {
t.Fatal("runtime updated immutable audit fact")
}
if _, err := db.ExecContext(ctx, `DELETE FROM bell.audit_events WHERE event_id=$1`, eventID); err == nil {
t.Fatal("runtime deleted immutable audit fact")
}
}
+149
View File
@@ -0,0 +1,149 @@
package store
import (
"bytes"
"context"
"crypto/sha256"
"database/sql"
"encoding/json"
"errors"
"fmt"
"regexp"
"time"
"unicode/utf8"
"yovision/bell/internal/event"
)
var ErrConflict = errors.New("immutable record id conflict")
type Postgres struct {
db *sql.DB
}
func OpenPostgres(ctx context.Context, db *sql.DB) (*Postgres, error) {
if db == nil {
return nil, errors.New("postgres database is required")
}
if err := db.PingContext(ctx); err != nil {
return nil, fmt.Errorf("ping Bell postgres: %w", err)
}
var version int64
if err := db.QueryRowContext(ctx, `SELECT COALESCE(MAX(version), 0) FROM bell.schema_migrations`).Scan(&version); err != nil || version < 3 {
return nil, errors.New("postgres Bell schema migration v3 is required")
}
var canInsert, canSelect, canUpdate, canDelete, canTruncate bool
if err := db.QueryRowContext(ctx, `SELECT
has_table_privilege(current_user, 'bell.events', 'INSERT'),
has_table_privilege(current_user, 'bell.events', 'SELECT'),
has_table_privilege(current_user, 'bell.events', 'UPDATE'),
has_table_privilege(current_user, 'bell.events', 'DELETE'),
has_table_privilege(current_user, 'bell.events', 'TRUNCATE')`).Scan(
&canInsert, &canSelect, &canUpdate, &canDelete, &canTruncate,
); err != nil {
return nil, fmt.Errorf("inspect Bell event privileges: %w", err)
}
if !canInsert || !canSelect || canUpdate || canDelete || canTruncate {
return nil, errors.New("Bell runtime event privileges violate append-only boundary")
}
return &Postgres{db: db}, nil
}
// InsertEvent is idempotent only for the same platform ID and exact payload.
func (p *Postgres) InsertEvent(ctx context.Context, value event.Event) (bool, error) {
digest := value.Digest()
result, err := p.db.ExecContext(ctx, `INSERT INTO bell.events(
id, tenant_id, site_id, device_id, source_event_id, kind, severity,
occurred_at, detected_at, payload_hash, payload
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11::jsonb)
ON CONFLICT (id) DO NOTHING`,
value.ID(), value.TenantID(), value.SiteID(), value.DeviceID(),
value.SourceEventID(), value.Kind(), value.Severity(), value.OccurredAt(),
value.DetectedAt(), digest[:], value.JSON(),
)
if err != nil {
return false, fmt.Errorf("insert immutable Bell event: %w", err)
}
rows, err := result.RowsAffected()
if err != nil {
return false, fmt.Errorf("read Bell event insert result: %w", err)
}
if rows == 1 {
return true, nil
}
var existing []byte
if err := p.db.QueryRowContext(ctx, `SELECT payload_hash FROM bell.events WHERE id=$1`, value.ID()).Scan(&existing); err != nil {
return false, fmt.Errorf("read existing Bell event digest: %w", err)
}
if !bytes.Equal(existing, digest[:]) {
return false, ErrConflict
}
return false, nil
}
type Outcome struct {
ID string `json:"id"`
EventID string `json:"event_id"`
Value string `json:"outcome"`
Source string `json:"source"`
Reason *string `json:"reason"`
ActorType string `json:"actor_type"`
ActorID string `json:"actor_id"`
OccurredAt time.Time `json:"occurred_at"`
}
var outcomeID = regexp.MustCompile(`^out_[0-9A-HJKMNP-TV-Z]{26}$`)
var eventID = regexp.MustCompile(`^evt_[0-9A-HJKMNP-TV-Z]{26}$`)
func (o Outcome) validate() error {
if !outcomeID.MatchString(o.ID) || !eventID.MatchString(o.EventID) || o.ActorID == "" || o.OccurredAt.IsZero() {
return errors.New("invalid outcome identity")
}
validOutcome := map[string]bool{"unknown": true, "true_positive": true, "false_positive": true, "subject_recovered": true, "duplicate": true, "test": true}
if !validOutcome[o.Value] || (o.Source != "auto" && o.Source != "manual") {
return errors.New("invalid outcome value or source")
}
if o.ActorType != "user" && o.ActorType != "service" && o.ActorType != "system" {
return errors.New("invalid outcome actor type")
}
if o.Reason != nil && utf8.RuneCountInString(*o.Reason) > 500 {
return errors.New("outcome reason is too long")
}
return nil
}
// AppendOutcome never mutates the event or an earlier outcome record.
func (p *Postgres) AppendOutcome(ctx context.Context, value Outcome) (bool, error) {
if err := value.validate(); err != nil {
return false, err
}
encoded, err := json.Marshal(value)
if err != nil {
return false, fmt.Errorf("encode outcome: %w", err)
}
digest := sha256.Sum256(encoded)
result, err := p.db.ExecContext(ctx, `INSERT INTO bell.event_outcomes(
id, event_id, outcome, outcome_source, reason, actor_type, actor_id,
occurred_at, record_hash
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9)
ON CONFLICT (id) DO NOTHING`, value.ID, value.EventID, value.Value, value.Source,
value.Reason, value.ActorType, value.ActorID, value.OccurredAt, digest[:])
if err != nil {
return false, fmt.Errorf("append Bell event outcome: %w", err)
}
rows, err := result.RowsAffected()
if err != nil {
return false, fmt.Errorf("read Bell outcome insert result: %w", err)
}
if rows == 1 {
return true, nil
}
var existing []byte
if err := p.db.QueryRowContext(ctx, `SELECT record_hash FROM bell.event_outcomes WHERE id=$1`, value.ID).Scan(&existing); err != nil {
return false, fmt.Errorf("read existing Bell outcome digest: %w", err)
}
if !bytes.Equal(existing, digest[:]) {
return false, ErrConflict
}
return false, nil
}
+120
View File
@@ -0,0 +1,120 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"os"
"path/filepath"
"testing"
"time"
_ "github.com/jackc/pgx/v5/stdlib"
"yovision/bell/contracts"
"yovision/bell/internal/event"
)
type storeIDs struct{ id string }
func (f storeIDs) NewEventID() (string, error) { return f.id, nil }
type allowVideo struct{}
func (allowVideo) VideoAllowed(context.Context, int64, int64, int64) (bool, error) {
return true, nil
}
func testCandidate(t *testing.T, configVersion string) []byte {
t.Helper()
path := filepath.Join("..", "..", "..", "docs", "raw", "contracts", "event-v0.1.example-current.json")
raw, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var object map[string]any
if err := json.Unmarshal(raw, &object); err != nil {
t.Fatal(err)
}
delete(object, "id")
object["config_version"] = configVersion
encoded, err := json.Marshal(object)
if err != nil {
t.Fatal(err)
}
return encoded
}
func newEvent(t *testing.T, configVersion string) event.Event {
t.Helper()
guard, err := event.NewEvidenceGuard()
if err != nil {
t.Fatal(err)
}
factory, err := event.NewFactory(
contracts.EventV01Schema,
storeIDs{id: "evt_01J8XQ2K7M3P5R9T0V4W6Y8Z2B"},
allowVideo{}, guard,
)
if err != nil {
t.Fatal(err)
}
value, err := factory.Create(context.Background(), testCandidate(t, configVersion))
if err != nil {
t.Fatal(err)
}
return value
}
func TestPostgresImmutableEventAndOutcome(t *testing.T) {
dsn := os.Getenv("YOVISION_TEST_BELL_POSTGRES_DSN")
if dsn == "" {
t.Skip("YOVISION_TEST_BELL_POSTGRES_DSN is not set")
}
db, err := sql.Open("pgx", dsn)
if err != nil {
t.Fatal(err)
}
defer db.Close()
ctx := context.Background()
repo, err := OpenPostgres(ctx, db)
if err != nil {
t.Fatal(err)
}
value := newEvent(t, "sp-v1-2026.07.20")
created, err := repo.InsertEvent(ctx, value)
if err != nil || !created {
t.Fatalf("first insert: created=%v err=%v", created, err)
}
created, err = repo.InsertEvent(ctx, value)
if err != nil || created {
t.Fatalf("idempotent replay: created=%v err=%v", created, err)
}
if _, err := repo.InsertEvent(ctx, newEvent(t, "sp-v1-conflict")); !errors.Is(err, ErrConflict) {
t.Fatalf("expected immutable conflict, got %v", err)
}
reason := "confirmed by operator"
outcome := Outcome{
ID: "out_01J8XQ2K7M3P5R9T0V4W6Y8Z2B", EventID: value.ID(),
Value: "true_positive", Source: "manual", Reason: &reason,
ActorType: "user", ActorID: "operator-1", OccurredAt: time.Now().UTC(),
}
created, err = repo.AppendOutcome(ctx, outcome)
if err != nil || !created {
t.Fatalf("append outcome: created=%v err=%v", created, err)
}
created, err = repo.AppendOutcome(ctx, outcome)
if err != nil || created {
t.Fatalf("idempotent outcome replay: created=%v err=%v", created, err)
}
if _, err := db.ExecContext(ctx, `UPDATE bell.events SET kind='changed' WHERE id=$1`, value.ID()); err == nil {
t.Fatal("runtime unexpectedly updated immutable event")
}
if _, err := db.ExecContext(ctx, `DELETE FROM bell.event_outcomes WHERE id=$1`, outcome.ID); err == nil {
t.Fatal("runtime unexpectedly deleted immutable outcome")
}
}
+24 -3
View File
@@ -1,6 +1,6 @@
# Sense M1/M2 接入骨架
本目录是 YoVision Sense 的 M1/M2 接入骨架。数据库保存期望态,ONVIF 和 MediaMTX 通过端口隔离;M1 默认使用 SQLite,T-009~T-012 增加 PostgreSQL 双 schema、Area 准入、本地审计 Outbox、Control API v1、多实例调和 fencing 和孤儿受控处置。默认关闭真实 ONVIF 与公共业务路由;T-006 的真实样机结论仅覆盖已批准的精确海康基线,不能据此宣称多品牌兼容。
本目录是 YoVision Sense 的 M1/M2 接入骨架。数据库保存期望态,ONVIF 和 MediaMTX 通过端口隔离;M1 默认使用 SQLite,T-009~T-016 增加 PostgreSQL 双 schema、Area 准入、本地审计 Outbox、Control API v1、多实例调和 fencing、孤儿受控处置和到 Bell 的审计 relay。默认关闭真实 ONVIF、公共业务路由和 relay;T-006 的真实样机结论仅覆盖已批准的精确海康基线,不能据此宣称多品牌兼容。
## 常用命令
@@ -42,6 +42,11 @@ Unix 将构建产物改为 `bin/sense-api`。服务默认监听 `127.0.0.1:8080`
| `SENSE_CONTROL_AUTH_FILE` | 空 | 仓库外绝对路径;version 1 JSON 只保存 token SHA-256、主体、tenant、Site scope 和权限 |
| `SENSE_CONTROL_CURSOR_KEY_FILE` | 空 | 仓库外绝对路径;内容为至少 32 字节随机值的无填充 base64url |
| `SENSE_CONTROL_ALLOW_INSECURE_HTTP` | `false` | Control API 非回环明文监听的独立风险接受;正常部署应保持回环并在受控代理终止 TLS |
| `SENSE_AUDIT_RELAY_ENABLED` | `false` | 显式开启 PostgreSQL Outbox → Bell relay;SQLite 不支持 |
| `SENSE_AUDIT_RELAY_URL` | 空 | 精确指向 Bell `/internal/v1/audit-events:batch`;非回环必须 HTTPS |
| `SENSE_AUDIT_RELAY_KEY_FILE` | 空 | 仓库外绝对路径 version 1 JSON key 文件,secret 至少 32 字节 |
| `SENSE_AUDIT_RELAY_KEY_ID` | 空 | 本实例用于签名的 key ID |
| `SENSE_AUDIT_RELAY_INTERVAL` | `1s` | 队列轮询间隔,最短 1 秒 |
设备台账只保存 `env://<key>` 凭据引用。真实适配器从进程环境读取以下变量,不把秘密写入 SQLite、日志或 MediaMTX 错误:
@@ -60,7 +65,7 @@ MediaMTX `v1.19.3` 应作为独立二进制启动并只在可信网络开放 API
初始化与增量 SQL 位于 `deploy/postgres/`,由高权限部署步骤按文件名前缀执行;Sense 进程不会自动创建角色、schema 或 Bell 对象。`bell_app` 拥有 Site/Area、配额、`capture_policy` 及两个版本化视图,`sense_app` 只能读取两个视图,不能读取或写入 Bell 源表。T-011 的 v4 schema 增加资源版本、24 小时幂等收据和 batch operation;T-012 的 v5 schema 增加数据库时钟租约、Path 历史归属及脱敏孤儿报告/处置结果。表中不保存 MediaMTX source URI。应用登录角色和密码由部署环境创建,不进入仓库。
PostgreSQL 新建设备必须携带匹配 tenant/Site 的 `area_id`。具有 `video_capture` 能力的设备在创建、移动 Area 和从 disabled 切到 enabled 时执行 Area 准入;`non_imaging_only` 拒绝成像设备但允许非成像设备。投影缺失、非法或版本回退只拒绝新变更,不关闭已有流。创建、配置修改和期望态受理都与对应脱敏 Outbox 事实同事务;停用后调和器只删除该设备的精确 MediaMTX path 并收敛为 offline,不枚举未知 path。Bell relay 尚未实现。
PostgreSQL 新建设备必须携带匹配 tenant/Site 的 `area_id`。具有 `video_capture` 能力的设备在创建、移动 Area 和从 disabled 切到 enabled 时执行 Area 准入;`non_imaging_only` 拒绝成像设备但允许非成像设备。投影缺失、非法或版本回退只拒绝新变更,不关闭已有流。创建、配置修改和期望态受理都与对应脱敏 Outbox 事实同事务;停用后调和器只删除该设备的精确 MediaMTX path 并收敛为 offline,不枚举未知 path。开启 relay 后,Sense 用数据库 lease/fencing 批量投递,成功、重试和 dead letter 均保留本地事实;Sense 不访问 Bell schema。
Windows 本机集成测试从仓库根目录执行:
@@ -78,7 +83,7 @@ $env:SENSE_DB_DSN = '由部署环境私下设置'
go run ./cmd/sense-api
```
PostgreSQL 启动会检查 Sense v5 migration、当前角色对两个 Bell 投影视图和本地控制/对账表的最小权限;权限过宽、视图不可读或 schema 未安装时初始化失败。默认 SQLite 路径和 `cmd/sense-lab` 保持不变,但 SQLite 不实现生产 Area/Outbox、多实例租约或孤儿处置语义,Control API 与孤儿扫描在 SQLite 下不会启动。
PostgreSQL 基础启动会检查 Sense v5 migration、当前角色对两个 Bell 投影视图和本地控制/对账表的最小权限;启用 relay 时额外要求 v6 和本地 Outbox 权限,并确认当前 Sense 登录不能访问 Bell 全局审计表。默认 SQLite 路径和 `cmd/sense-lab` 保持不变,但 SQLite 不实现生产 Area/Outbox、多实例租约、孤儿处置或 relay 语义,Control API、孤儿扫描与 relay 在 SQLite 下不会启动。
### 孤儿报告与受控处置
@@ -138,3 +143,19 @@ Linux amd64 使用同版 `mediamtx_v1.19.3_linux_amd64.tar.gz`,SHA-256 为 `a7
```
`ip_camera.env` 必须保持在 Git 忽略范围内。调试时可把 `-ObservationMinutes` 降为 1;正式 T-006 证据必须使用默认 30 分钟,且最终 `maximum_unconverged`、`final_unconverged` 都为 0。
## T-014 本地 16 路容量基线
T-014 不访问摄像头或客户网络。脚本使用隔离 PostgreSQL v5、真实 Control API、两套 MediaMTX 和 16 个独立的 FFmpeg `-c copy` 合成 publisher,验证 17 路配额拒绝、三轮批量启停、`16 → 0 → 16` Path 收敛、固定四路故障隔离/恢复,以及 30 分钟稳定性和资源观测。
从仓库根目录执行预检、短窗口调试和正式验收:
```powershell
./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -PreflightOnly
./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -ObservationMinutes 1 -OutputPath (Join-Path $env:TEMP 'yovision-t014-smoke.json')
./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -OutputPath (Join-Path $env:TEMP 'yovision-t014-formal.json')
```
只有默认不少于 30 分钟且输出 `formal_eligible=true` 的单次完整运行可作正式证据;调试窗口固定标记为 false。脚本要求 PostgreSQL 17.10、MediaMTX v1.19.3、FFmpeg 8.1.2 和 `Sense/` 模块的 Go 1.26.5,只使用随机回环端口,运行后清理临时媒体、二进制、PGDATA 和秘密文件。结果 JSON 不含 token、DSN、端口、设备 ID 或流地址。
2026-08-10 正式结果为 `1800.1 s / 180` 个 10 秒样本,最大/最终 `unconverged=0`、最终在线 Path 16、帧错误 0;完整版本、资源数据、失败记录和适用边界见 [`../docs/research/sense-16-stream-capacity.md`](../docs/research/sense-16-stream-capacity.md)。该结果只是本机低码率合成负载的软件基线,不代表真实 16 摄像头、客户网络、录像、AI/GPU、64/128 路或生产 SLA。
+28 -1
View File
@@ -12,6 +12,7 @@ import (
"syscall"
"time"
"yovision/sense/internal/auditrelay"
"yovision/sense/internal/auth"
"yovision/sense/internal/config"
"yovision/sense/internal/controlapi"
@@ -55,6 +56,28 @@ func run(logger *slog.Logger) error {
return err
}
defer repository.Close()
var auditWorker *auditrelay.Worker
if cfg.AuditRelayEnabled {
relayStore, ok := repository.(auditrelay.Repository)
if !ok {
return errors.New("selected repository does not support audit relay")
}
if err := relayStore.AuditRelayReady(ctx); err != nil {
return err
}
secret, err := auditrelay.LoadKey(cfg.AuditRelayKeyFile, cfg.AuditRelayKeyID)
if err != nil {
return err
}
client, err := auditrelay.NewClient(cfg.AuditRelayURL, cfg.AuditRelayKeyID, secret, nil)
if err != nil {
return err
}
auditWorker, err = auditrelay.NewWorker(relayStore, client, instanceID)
if err != nil {
return err
}
}
var controlHandler http.Handler
if cfg.ControlAPIEnabled {
controlStore, ok := repository.(store.ControlRepository)
@@ -116,6 +139,9 @@ func run(logger *slog.Logger) error {
if orphanScanner != nil {
startBackground(func() { orphanScanner.Run(ctx, cfg.OrphanScanInterval, report) })
}
if auditWorker != nil {
startBackground(func() { auditWorker.Run(ctx, cfg.AuditRelayInterval, report) })
}
mux := http.NewServeMux()
mux.HandleFunc("GET /healthz", func(writer http.ResponseWriter, _ *http.Request) {
@@ -146,7 +172,8 @@ func run(logger *slog.Logger) error {
go func() {
logger.Info("Sense listening", "address", cfg.HTTPAddress, "version", version,
"instance_id", instanceID,
"control_api_enabled", cfg.ControlAPIEnabled)
"control_api_enabled", cfg.ControlAPIEnabled,
"audit_relay_enabled", cfg.AuditRelayEnabled)
serverErrors <- server.ListenAndServe()
}()
@@ -0,0 +1,132 @@
package auditrelay
import (
"context"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"time"
)
func testEnvelope(id string) Envelope {
return Envelope{SchemaVersion: 1, Event: Event{
EventID: id, EventType: "device.created", TenantID: "tenant", SiteID: "site", DeviceID: "camera-1",
Actor: Actor{Type: "system", ID: "sense"}, AggregateGeneration: 1,
ProjectionVersions: ProjectionVersions{}, Data: json.RawMessage(`{"kind":"device_created"}`), OccurredAt: time.Unix(1, 0).UTC(),
}}
}
func TestClientSignsCanonicalRequest(t *testing.T) {
secret := bytesOf(32, 7)
server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
body := make([]byte, request.ContentLength)
_, _ = request.Body.Read(body)
canonical := CanonicalString(request.Method, request.URL.Path, request.Header.Get(HeaderTimestamp), request.Header.Get(HeaderNonce), body)
if request.Header.Get(HeaderSignature) != Signature(secret, canonical) {
t.Error("request signature did not match canonical vector")
}
_ = json.NewEncoder(writer).Encode(BatchResponse{Results: []Result{{EventID: "audit_00000000000000000000000000000001", Status: "accepted"}}})
}))
defer server.Close()
client, err := NewClient(server.URL+RelayPath, "sense-a", secret, server.Client())
if err != nil {
t.Fatal(err)
}
client.now = func() time.Time { return time.Unix(1_800_000_000, 0) }
client.nonce = func() (string, error) { return "AAAAAAAAAAAAAAAAAAAAAA", nil }
results, err := client.Send(context.Background(), []Envelope{testEnvelope("audit_00000000000000000000000000000001")})
if err != nil || len(results) != 1 || results[0].Status != "accepted" {
t.Fatalf("unexpected relay result: %+v, %v", results, err)
}
}
func TestEndpointAndKeySecurity(t *testing.T) {
if _, err := ValidateEndpoint("http://example.com" + RelayPath); err == nil {
t.Fatal("remote plaintext relay URL was accepted")
}
if _, err := ValidateEndpoint("https://example.com" + RelayPath + "?secret=x"); err == nil {
t.Fatal("relay URL query was accepted")
}
secret := bytesOf(32, 9)
path := filepath.Join(t.TempDir(), "keys.json")
document := map[string]any{"version": 1, "keys": []any{map[string]any{"key_id": "sense-a", "secret_base64url": base64.RawURLEncoding.EncodeToString(secret)}}}
raw, _ := json.Marshal(document)
if err := os.WriteFile(path, raw, 0o600); err != nil {
t.Fatal(err)
}
loaded, err := LoadKey(path, "sense-a")
if err != nil || sha256.Sum256(loaded) != sha256.Sum256(secret) {
t.Fatalf("external key was not loaded: %v", err)
}
}
type fakeRepository struct {
queued []QueuedEvent
completed []Completion
}
func (*fakeRepository) AuditRelayReady(context.Context) error { return nil }
func (f *fakeRepository) ClaimAuditRelayBatch(context.Context, string, int, time.Duration) ([]QueuedEvent, error) {
return f.queued, nil
}
func (f *fakeRepository) CompleteAuditRelayBatch(_ context.Context, _ string, values []Completion) error {
f.completed = append([]Completion(nil), values...)
return nil
}
type fakeSender struct {
results []Result
err error
}
func (f fakeSender) Send(context.Context, []Envelope) ([]Result, error) { return f.results, f.err }
func TestWorkerDispositionAndBackoff(t *testing.T) {
first := "audit_00000000000000000000000000000001"
second := "audit_00000000000000000000000000000002"
repository := &fakeRepository{queued: []QueuedEvent{{Envelope: testEnvelope(first), LeaseToken: 1, AttemptCount: 1}, {Envelope: testEnvelope(second), LeaseToken: 2, AttemptCount: 10}}}
code := "schema_invalid"
worker, _ := NewWorker(repository, fakeSender{results: []Result{{EventID: first, Status: "accepted"}, {EventID: second, Status: "rejected", ErrorCode: &code}}}, "worker")
if err := worker.RelayOnce(context.Background()); err != nil {
t.Fatal(err)
}
if repository.completed[0].Disposition != Delivered || repository.completed[1].Disposition != DeadLetter {
t.Fatalf("unexpected dispositions: %+v", repository.completed)
}
repository.completed = nil
worker.sender = fakeSender{err: errors.New("network")}
if err := worker.RelayOnce(context.Background()); err == nil {
t.Fatal("network failure was hidden")
}
if repository.completed[0].RetryAfter != time.Second || repository.completed[1].RetryAfter != 300*time.Second {
t.Fatalf("retry bounds drifted: %+v", repository.completed)
}
}
func TestWorkerRetriesWholeBatchForIncompleteResponse(t *testing.T) {
first := "audit_00000000000000000000000000000001"
second := "audit_00000000000000000000000000000002"
repository := &fakeRepository{queued: []QueuedEvent{{Envelope: testEnvelope(first), LeaseToken: 1, AttemptCount: 1}, {Envelope: testEnvelope(second), LeaseToken: 2, AttemptCount: 2}}}
worker, _ := NewWorker(repository, fakeSender{results: []Result{{EventID: first, Status: "accepted"}}}, "worker")
if err := worker.RelayOnce(context.Background()); err == nil {
t.Fatal("incomplete response was accepted")
}
if len(repository.completed) != 2 || repository.completed[0].Disposition != Retry || repository.completed[1].Disposition != Retry {
t.Fatalf("incomplete response partially completed the batch: %+v", repository.completed)
}
}
func bytesOf(size int, value byte) []byte {
result := make([]byte, size)
for index := range result {
result[index] = value
}
return result
}
+131
View File
@@ -0,0 +1,131 @@
package auditrelay
import (
"bytes"
"context"
"crypto/hmac"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
const (
HeaderKeyID = "X-YoVision-Key-Id"
HeaderTimestamp = "X-YoVision-Timestamp"
HeaderNonce = "X-YoVision-Nonce"
HeaderSignature = "X-YoVision-Signature"
RelayPath = "/internal/v1/audit-events:batch"
)
type Client struct {
endpoint *url.URL
keyID string
secret []byte
httpClient *http.Client
now func() time.Time
nonce func() (string, error)
}
func NewClient(rawURL, keyID string, secret []byte, client *http.Client) (*Client, error) {
endpoint, err := ValidateEndpoint(rawURL)
if err != nil {
return nil, err
}
if keyID == "" || len(secret) < 32 {
return nil, errors.New("audit relay key ID and 32-byte secret are required")
}
if client == nil {
client = &http.Client{Timeout: 10 * time.Second}
}
return &Client{endpoint: endpoint, keyID: keyID, secret: append([]byte(nil), secret...), httpClient: client, now: time.Now, nonce: randomNonce}, nil
}
func ValidateEndpoint(rawURL string) (*url.URL, error) {
parsed, err := url.Parse(rawURL)
if err != nil || parsed.Host == "" || parsed.Path != RelayPath || parsed.RawQuery != "" || parsed.Fragment != "" || parsed.User != nil {
return nil, errors.New("invalid Bell audit relay URL")
}
host := parsed.Hostname()
ip := net.ParseIP(host)
loopback := strings.EqualFold(host, "localhost") || (ip != nil && ip.IsLoopback())
if parsed.Scheme != "https" && !(parsed.Scheme == "http" && loopback) {
return nil, errors.New("Bell audit relay URL requires HTTPS outside loopback")
}
return parsed, nil
}
func CanonicalString(method, path, timestamp, nonce string, body []byte) string {
digest := sha256.Sum256(body)
return strings.Join([]string{method, path, timestamp, nonce, hex.EncodeToString(digest[:])}, "\n")
}
func Signature(secret []byte, canonical string) string {
mac := hmac.New(sha256.New, secret)
_, _ = mac.Write([]byte(canonical))
return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
}
func randomNonce() (string, error) {
value := make([]byte, 16)
if _, err := rand.Read(value); err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(value), nil
}
func (c *Client) Send(ctx context.Context, events []Envelope) ([]Result, error) {
if len(events) < 1 || len(events) > MaxBatchSize {
return nil, errors.New("audit relay batch must contain 1 to 100 events")
}
body, err := json.Marshal(BatchRequest{Events: events})
if err != nil || len(body) > MaxBodyBytes {
return nil, errors.New("encode audit relay batch")
}
requestContext, cancel := context.WithTimeout(ctx, 10*time.Second)
defer cancel()
timestamp := strconv.FormatInt(c.now().UTC().Unix(), 10)
nonce, err := c.nonce()
if err != nil {
return nil, errors.New("generate audit relay nonce")
}
request, err := http.NewRequestWithContext(requestContext, http.MethodPost, c.endpoint.String(), bytes.NewReader(body))
if err != nil {
return nil, errors.New("create audit relay request")
}
request.Header.Set("Content-Type", "application/json")
request.Header.Set(HeaderKeyID, c.keyID)
request.Header.Set(HeaderTimestamp, timestamp)
request.Header.Set(HeaderNonce, nonce)
request.Header.Set(HeaderSignature, Signature(c.secret, CanonicalString(http.MethodPost, RelayPath, timestamp, nonce, body)))
response, err := c.httpClient.Do(request)
if err != nil {
return nil, errors.New("send audit relay request")
}
defer response.Body.Close()
if response.StatusCode != http.StatusOK {
_, _ = io.Copy(io.Discard, io.LimitReader(response.Body, 4096))
return nil, fmt.Errorf("Bell audit relay returned HTTP %d", response.StatusCode)
}
decoder := json.NewDecoder(io.LimitReader(response.Body, MaxBodyBytes+1))
decoder.DisallowUnknownFields()
var decoded BatchResponse
if err := decoder.Decode(&decoded); err != nil {
return nil, errors.New("decode audit relay response")
}
var trailing any
if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) {
return nil, errors.New("audit relay response contains trailing data")
}
return decoded.Results, nil
}
+53
View File
@@ -0,0 +1,53 @@
package auditrelay
import (
"bytes"
"encoding/base64"
"encoding/json"
"errors"
"io"
"os"
"regexp"
)
var keyIDPattern = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`)
type keyFile struct {
Version int `json:"version"`
Keys []struct {
KeyID string `json:"key_id"`
Secret string `json:"secret_base64url"`
} `json:"keys"`
}
func LoadKey(path, keyID string) ([]byte, error) {
raw, err := os.ReadFile(path)
if err != nil {
return nil, errors.New("read audit relay key file")
}
var document keyFile
decoder := json.NewDecoder(bytes.NewReader(raw))
decoder.DisallowUnknownFields()
if err := decoder.Decode(&document); err != nil || document.Version != 1 || len(document.Keys) == 0 {
return nil, errors.New("invalid audit relay key file")
}
var trailing any
if err := decoder.Decode(&trailing); !errors.Is(err, io.EOF) {
return nil, errors.New("invalid audit relay key file")
}
values := make(map[string][]byte, len(document.Keys))
for _, candidate := range document.Keys {
secret, err := base64.RawURLEncoding.DecodeString(candidate.Secret)
if err != nil || !keyIDPattern.MatchString(candidate.KeyID) || len(secret) < 32 {
return nil, errors.New("invalid audit relay secret")
}
if _, duplicate := values[candidate.KeyID]; duplicate {
return nil, errors.New("duplicate audit relay key ID")
}
values[candidate.KeyID] = secret
}
if secret, exists := values[keyID]; exists {
return secret, nil
}
return nil, errors.New("audit relay key ID not found")
}
+89
View File
@@ -0,0 +1,89 @@
// Package auditrelay delivers Sense-owned audit facts to Bell without sharing databases.
package auditrelay
import (
"context"
"encoding/json"
"errors"
"time"
)
const (
MaxBatchSize = 100
MaxBodyBytes = 1 << 20
LeaseDuration = 30 * time.Second
)
type Actor struct {
Type string `json:"type"`
ID string `json:"id"`
}
type ProjectionVersions struct {
QuotaSourceVersion *int64 `json:"quota_source_version"`
AreaPolicySourceVersion *int64 `json:"area_policy_source_version"`
}
type Event struct {
EventID string `json:"event_id"`
EventType string `json:"event_type"`
TenantID string `json:"tenant_id"`
SiteID string `json:"site_id"`
DeviceID string `json:"device_id"`
Actor Actor `json:"actor"`
Reason *string `json:"reason"`
TraceID *string `json:"trace_id"`
AggregateGeneration int64 `json:"aggregate_generation"`
ProjectionVersions ProjectionVersions `json:"projection_versions"`
Data json.RawMessage `json:"data"`
OccurredAt time.Time `json:"occurred_at"`
}
type Envelope struct {
SchemaVersion int `json:"schema_version"`
Event Event `json:"event"`
}
type BatchRequest struct {
Events []Envelope `json:"events"`
}
type Result struct {
EventID string `json:"event_id"`
Status string `json:"status"`
ErrorCode *string `json:"error_code,omitempty"`
}
type BatchResponse struct {
Results []Result `json:"results"`
}
type QueuedEvent struct {
Envelope
LeaseToken int64
AttemptCount int
}
type Disposition string
const (
Delivered Disposition = "delivered"
DeadLetter Disposition = "dead_letter"
Retry Disposition = "retry"
)
type Completion struct {
EventID string
LeaseToken int64
Disposition Disposition
ErrorCode string
RetryAfter time.Duration
}
var ErrLeaseLost = errors.New("audit relay lease lost")
type Repository interface {
AuditRelayReady(context.Context) error
ClaimAuditRelayBatch(context.Context, string, int, time.Duration) ([]QueuedEvent, error)
CompleteAuditRelayBatch(context.Context, string, []Completion) error
}
+126
View File
@@ -0,0 +1,126 @@
package auditrelay
import (
"context"
"errors"
"fmt"
"regexp"
"time"
)
var stableErrorCode = regexp.MustCompile(`^[a-z][a-z0-9_]{0,63}$`)
type Sender interface {
Send(context.Context, []Envelope) ([]Result, error)
}
type Worker struct {
repository Repository
sender Sender
owner string
}
func NewWorker(repository Repository, sender Sender, owner string) (*Worker, error) {
if repository == nil || sender == nil || owner == "" {
return nil, errors.New("audit relay worker dependencies are required")
}
return &Worker{repository: repository, sender: sender, owner: owner}, nil
}
func RetryDelay(attempt int) time.Duration {
if attempt < 1 {
attempt = 1
}
if attempt > 9 {
return 300 * time.Second
}
delay := time.Second << (attempt - 1)
if delay > 300*time.Second {
return 300 * time.Second
}
return delay
}
func (w *Worker) RelayOnce(ctx context.Context) error {
queued, err := w.repository.ClaimAuditRelayBatch(ctx, w.owner, MaxBatchSize, LeaseDuration)
if err != nil || len(queued) == 0 {
return err
}
events := make([]Envelope, len(queued))
for index := range queued {
events[index] = queued[index].Envelope
}
results, sendErr := w.sender.Send(ctx, events)
if sendErr != nil {
completions := make([]Completion, len(queued))
for index, value := range queued {
completions[index] = Completion{EventID: value.Event.EventID, LeaseToken: value.LeaseToken, Disposition: Retry, ErrorCode: "delivery_failed", RetryAfter: RetryDelay(value.AttemptCount)}
}
if err := w.repository.CompleteAuditRelayBatch(ctx, w.owner, completions); err != nil {
return err
}
return sendErr
}
if len(results) != len(queued) {
return w.retryAll(ctx, queued, "invalid_response")
}
expected := make(map[string]bool, len(queued))
for _, value := range queued {
expected[value.Event.EventID] = true
}
byID := make(map[string]Result, len(results))
for _, result := range results {
validStatus := ((result.Status == "accepted" || result.Status == "duplicate") && result.ErrorCode == nil) ||
(result.Status == "rejected" && result.ErrorCode != nil && stableErrorCode.MatchString(*result.ErrorCode))
if !expected[result.EventID] || !validStatus {
return w.retryAll(ctx, queued, "invalid_response")
}
if _, duplicate := byID[result.EventID]; duplicate {
return w.retryAll(ctx, queued, "invalid_response")
}
byID[result.EventID] = result
}
completions := make([]Completion, 0, len(queued))
for _, value := range queued {
result, ok := byID[value.Event.EventID]
if !ok {
return w.retryAll(ctx, queued, "invalid_response")
}
switch result.Status {
case "accepted", "duplicate":
completions = append(completions, Completion{EventID: value.Event.EventID, LeaseToken: value.LeaseToken, Disposition: Delivered})
case "rejected":
completions = append(completions, Completion{EventID: value.Event.EventID, LeaseToken: value.LeaseToken, Disposition: DeadLetter, ErrorCode: *result.ErrorCode})
}
}
return w.repository.CompleteAuditRelayBatch(ctx, w.owner, completions)
}
func (w *Worker) retryAll(ctx context.Context, queued []QueuedEvent, code string) error {
values := make([]Completion, len(queued))
for index, value := range queued {
values[index] = Completion{EventID: value.Event.EventID, LeaseToken: value.LeaseToken, Disposition: Retry, ErrorCode: code, RetryAfter: RetryDelay(value.AttemptCount)}
}
if err := w.repository.CompleteAuditRelayBatch(ctx, w.owner, values); err != nil {
return err
}
return fmt.Errorf("audit relay %s", code)
}
func (w *Worker) Run(ctx context.Context, interval time.Duration, report func(error)) {
if interval <= 0 {
interval = time.Second
}
for {
if err := w.RelayOnce(ctx); err != nil && ctx.Err() == nil && report != nil {
report(err)
}
timer := time.NewTimer(interval)
select {
case <-ctx.Done():
timer.Stop()
return
case <-timer.C:
}
}
}
+44
View File
@@ -26,6 +26,7 @@ const (
defaultOrphanScanPeriod = time.Minute
defaultONVIFMode = "disabled"
defaultControlAuthMode = "static-sha256"
defaultAuditRelayPeriod = time.Second
)
var instanceIDPattern = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`)
@@ -53,6 +54,11 @@ type Config struct {
ControlAuthFile string
ControlCursorKeyFile string
ControlAllowInsecureHTTP bool
AuditRelayEnabled bool
AuditRelayURL string
AuditRelayKeyFile string
AuditRelayKeyID string
AuditRelayInterval time.Duration
}
func Load() (Config, error) {
@@ -92,6 +98,14 @@ func Load() (Config, error) {
if err != nil {
return Config{}, err
}
auditRelayEnabled, err := boolEnv("SENSE_AUDIT_RELAY_ENABLED", false)
if err != nil {
return Config{}, err
}
auditRelayInterval, err := durationEnv("SENSE_AUDIT_RELAY_INTERVAL", defaultAuditRelayPeriod)
if err != nil {
return Config{}, err
}
metricsEnabled, err := boolEnv("SENSE_METRICS_ENABLED", true)
if err != nil {
return Config{}, err
@@ -130,6 +144,11 @@ func Load() (Config, error) {
ControlAuthFile: stringEnv("SENSE_CONTROL_AUTH_FILE", ""),
ControlCursorKeyFile: stringEnv("SENSE_CONTROL_CURSOR_KEY_FILE", ""),
ControlAllowInsecureHTTP: controlAllowInsecure,
AuditRelayEnabled: auditRelayEnabled,
AuditRelayURL: stringEnv("SENSE_AUDIT_RELAY_URL", ""),
AuditRelayKeyFile: stringEnv("SENSE_AUDIT_RELAY_KEY_FILE", ""),
AuditRelayKeyID: stringEnv("SENSE_AUDIT_RELAY_KEY_ID", ""),
AuditRelayInterval: auditRelayInterval,
}
if err := cfg.Validate(); err != nil {
return Config{}, err
@@ -228,6 +247,31 @@ func (c Config) Validate() error {
return fmt.Errorf("non-loopback Control API requires SENSE_CONTROL_ALLOW_INSECURE_HTTP=true")
}
}
if c.AuditRelayEnabled {
if databaseDriver != postgresDatabaseDriver {
return fmt.Errorf("Sense audit relay requires SENSE_DB_DRIVER=postgres")
}
if c.AuditRelayKeyFile == "" || !filepath.IsAbs(c.AuditRelayKeyFile) {
return fmt.Errorf("SENSE_AUDIT_RELAY_KEY_FILE must be an absolute external path")
}
if !instanceIDPattern.MatchString(c.AuditRelayKeyID) {
return fmt.Errorf("invalid SENSE_AUDIT_RELAY_KEY_ID")
}
if c.AuditRelayInterval < time.Second {
return fmt.Errorf("SENSE_AUDIT_RELAY_INTERVAL must be at least 1s")
}
relayURL, err := url.Parse(c.AuditRelayURL)
if err != nil || relayURL.Host == "" || relayURL.Path != "/internal/v1/audit-events:batch" ||
relayURL.RawQuery != "" || relayURL.Fragment != "" || relayURL.User != nil {
return fmt.Errorf("invalid SENSE_AUDIT_RELAY_URL")
}
relayHost := relayURL.Hostname()
relayIP := net.ParseIP(relayHost)
relayLoopback := relayHost == "localhost" || (relayIP != nil && relayIP.IsLoopback())
if relayURL.Scheme != "https" && !(relayURL.Scheme == "http" && relayLoopback) {
return fmt.Errorf("SENSE_AUDIT_RELAY_URL requires HTTPS outside loopback")
}
}
return nil
}
+28
View File
@@ -166,3 +166,31 @@ func TestValidateControlAPINonLoopbackNeedsSeparateRiskAcceptance(t *testing.T)
t.Fatalf("explicit non-loopback Control API risk acceptance failed: %v", err)
}
}
func TestValidateAuditRelaySecurityBoundary(t *testing.T) {
base := Config{
HTTPAddress: "127.0.0.1:8080", DatabaseDriver: "postgres",
DatabaseDSN: "postgres://sense-runtime@127.0.0.1/yovision?sslmode=disable",
MediaMTXURL: "http://127.0.0.1:9997", ReconcileInterval: time.Second, ProbeInterval: time.Second,
AuditRelayEnabled: true, AuditRelayURL: "http://127.0.0.1:8081/internal/v1/audit-events:batch",
AuditRelayKeyFile: filepath.Join(t.TempDir(), "relay-keys.json"), AuditRelayKeyID: "sense-a", AuditRelayInterval: time.Second,
}
if err := base.Validate(); err != nil {
t.Fatalf("valid loopback relay rejected: %v", err)
}
remoteHTTP := base
remoteHTTP.AuditRelayURL = "http://bell.example/internal/v1/audit-events:batch"
if err := remoteHTTP.Validate(); err == nil {
t.Fatal("remote plaintext relay was accepted")
}
sqlite := base
sqlite.DatabaseDriver, sqlite.DatabaseDSN = "sqlite", "file:test.db"
if err := sqlite.Validate(); err == nil {
t.Fatal("SQLite audit relay was accepted")
}
relativeKey := base
relativeKey.AuditRelayKeyFile = "relay-keys.json"
if err := relativeKey.Validate(); err == nil {
t.Fatal("repository-relative relay key was accepted")
}
}
@@ -0,0 +1,176 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"regexp"
"strings"
"time"
"yovision/sense/internal/auditrelay"
)
var relayErrorCode = regexp.MustCompile(`^[a-z][a-z0-9_]{0,63}$`)
func (s *Postgres) AuditRelayReady(ctx context.Context) error {
var version int64
if err := s.db.QueryRowContext(ctx, `SELECT COALESCE(MAX(version), 0) FROM sense.schema_migrations`).Scan(&version); err != nil || version < 6 {
return errors.New("postgres Sense schema migration v6 is required for audit relay")
}
var canUseOutbox, canReadBell, canWriteBell bool
if err := s.db.QueryRowContext(ctx, `SELECT
has_table_privilege(current_user, 'sense.device_operation_outbox', 'SELECT,INSERT,UPDATE,DELETE'),
has_table_privilege(current_user, 'bell.audit_events', 'SELECT'),
has_table_privilege(current_user, 'bell.audit_events', 'INSERT,UPDATE,DELETE')`).Scan(
&canUseOutbox, &canReadBell, &canWriteBell,
); err != nil {
return errors.New("verify audit relay privileges")
}
if !canUseOutbox || canReadBell || canWriteBell {
return errors.New("Sense audit relay violates schema ownership boundary")
}
return nil
}
func (s *Postgres) ClaimAuditRelayBatch(
ctx context.Context,
owner string,
limit int,
lease time.Duration,
) ([]auditrelay.QueuedEvent, error) {
if strings.TrimSpace(owner) == "" || limit < 1 || limit > auditrelay.MaxBatchSize || lease <= 0 {
return nil, errors.New("invalid audit relay claim")
}
rows, err := s.db.QueryContext(ctx, `WITH due AS (
SELECT event_id FROM sense.device_operation_outbox
WHERE delivered_at IS NULL AND dead_lettered_at IS NULL
AND COALESCE(next_attempt_at, available_at) <= clock_timestamp()
AND (relay_lease_until IS NULL OR relay_lease_until <= clock_timestamp())
ORDER BY COALESCE(next_attempt_at, available_at), event_id
FOR UPDATE SKIP LOCKED LIMIT $1
)
UPDATE sense.device_operation_outbox AS outbox SET
relay_lease_owner = $2,
relay_lease_token = outbox.relay_lease_token + 1,
relay_lease_until = clock_timestamp() + ($3 * interval '1 second'),
attempt_count = outbox.attempt_count + 1,
last_error_code = NULL
FROM due WHERE outbox.event_id = due.event_id
RETURNING outbox.event_id, outbox.event_type, outbox.tenant_id,
outbox.site_id, outbox.device_id, outbox.actor_type, outbox.actor_id,
outbox.reason, outbox.trace_id, outbox.aggregate_generation,
outbox.quota_source_version, outbox.area_policy_source_version,
outbox.payload, outbox.occurred_at, outbox.relay_lease_token,
outbox.attempt_count`, limit, owner, lease.Seconds())
if err != nil {
return nil, errors.New("claim audit relay batch")
}
defer rows.Close()
values := make([]auditrelay.QueuedEvent, 0)
for rows.Next() {
var value auditrelay.QueuedEvent
var reason, trace sql.NullString
var quota, area sql.NullInt64
var payload []byte
if err := rows.Scan(
&value.Event.EventID, &value.Event.EventType, &value.Event.TenantID,
&value.Event.SiteID, &value.Event.DeviceID, &value.Event.Actor.Type,
&value.Event.Actor.ID, &reason, &trace, &value.Event.AggregateGeneration,
&quota, &area, &payload, &value.Event.OccurredAt, &value.LeaseToken,
&value.AttemptCount,
); err != nil {
return nil, errors.New("scan audit relay claim")
}
if reason.Valid {
value.Event.Reason = &reason.String
}
if trace.Valid {
value.Event.TraceID = &trace.String
}
if quota.Valid {
value.Event.ProjectionVersions.QuotaSourceVersion = &quota.Int64
}
if area.Valid {
value.Event.ProjectionVersions.AreaPolicySourceVersion = &area.Int64
}
if !json.Valid(payload) {
return nil, errors.New("invalid audit payload in outbox")
}
value.Event.Data = append(json.RawMessage(nil), payload...)
value.SchemaVersion = 1
if value.Event.EventType == "device.configuration.accepted" {
value.SchemaVersion = 2
}
values = append(values, value)
}
if err := rows.Err(); err != nil {
return nil, errors.New("iterate audit relay claims")
}
return values, nil
}
func (s *Postgres) CompleteAuditRelayBatch(
ctx context.Context,
owner string,
values []auditrelay.Completion,
) error {
if strings.TrimSpace(owner) == "" || len(values) == 0 || len(values) > auditrelay.MaxBatchSize {
return errors.New("invalid audit relay completion")
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return errors.New("begin audit relay completion")
}
defer tx.Rollback()
for _, value := range values {
code := value.ErrorCode
if code != "" && !relayErrorCode.MatchString(code) {
code = "invalid_response"
}
var result sql.Result
switch value.Disposition {
case auditrelay.Delivered:
result, err = tx.ExecContext(ctx, `UPDATE sense.device_operation_outbox SET
delivered_at = clock_timestamp(), last_error_code = NULL,
relay_lease_owner = NULL, relay_lease_until = NULL
WHERE event_id = $1 AND relay_lease_owner = $2 AND relay_lease_token = $3
AND relay_lease_until > clock_timestamp()`, value.EventID, owner, value.LeaseToken)
case auditrelay.DeadLetter:
if code == "" {
return errors.New("dead-letter completion requires an error code")
}
result, err = tx.ExecContext(ctx, `UPDATE sense.device_operation_outbox SET
dead_lettered_at = clock_timestamp(), last_error_code = $4,
relay_lease_owner = NULL, relay_lease_until = NULL
WHERE event_id = $1 AND relay_lease_owner = $2 AND relay_lease_token = $3
AND relay_lease_until > clock_timestamp()`, value.EventID, owner, value.LeaseToken, code)
case auditrelay.Retry:
if code == "" || value.RetryAfter < time.Second || value.RetryAfter > 300*time.Second {
return errors.New("invalid audit relay retry completion")
}
result, err = tx.ExecContext(ctx, `UPDATE sense.device_operation_outbox SET
next_attempt_at = clock_timestamp() + ($4 * interval '1 second'),
last_error_code = $5, relay_lease_owner = NULL, relay_lease_until = NULL
WHERE event_id = $1 AND relay_lease_owner = $2 AND relay_lease_token = $3
AND relay_lease_until > clock_timestamp()`, value.EventID, owner, value.LeaseToken, value.RetryAfter.Seconds(), code)
default:
return errors.New("invalid audit relay disposition")
}
if err != nil {
return errors.New("persist audit relay completion")
}
affected, err := result.RowsAffected()
if err != nil {
return errors.New("read audit relay completion")
}
if affected != 1 {
return auditrelay.ErrLeaseLost
}
}
if err := tx.Commit(); err != nil {
return errors.New("commit audit relay completion")
}
return nil
}
@@ -0,0 +1,45 @@
package store
import (
"context"
"errors"
"testing"
"time"
"yovision/sense/internal/auditrelay"
)
func TestPostgresAuditRelayClaimUsesFencing(t *testing.T) {
repository, admin := openPostgresTestStore(t)
ctx := context.Background()
if err := repository.AuditRelayReady(ctx); err != nil {
t.Fatal(err)
}
insertBellSite(t, admin, "relay-tenant", "relay-site", 1)
device := videoDevice(9001, "relay-tenant", "relay-site")
if err := repository.CreateDevice(ctx, device); err != nil {
t.Fatal(err)
}
first, err := repository.ClaimAuditRelayBatch(ctx, "worker-a", 100, 30*time.Second)
if err != nil || len(first) != 1 || first[0].SchemaVersion != 1 || first[0].AttemptCount != 1 {
t.Fatalf("first claim: %+v %v", first, err)
}
if _, err := admin.ExecContext(ctx, `UPDATE sense.device_operation_outbox SET relay_lease_until=clock_timestamp()-interval '1 second' WHERE event_id=$1`, first[0].Event.EventID); err != nil {
t.Fatal(err)
}
second, err := repository.ClaimAuditRelayBatch(ctx, "worker-b", 100, 30*time.Second)
if err != nil || len(second) != 1 || second[0].LeaseToken <= first[0].LeaseToken {
t.Fatalf("reclaim: %+v %v", second, err)
}
err = repository.CompleteAuditRelayBatch(ctx, "worker-a", []auditrelay.Completion{{EventID: first[0].Event.EventID, LeaseToken: first[0].LeaseToken, Disposition: auditrelay.Delivered}})
if !errors.Is(err, auditrelay.ErrLeaseLost) {
t.Fatalf("stale worker completion returned %v", err)
}
if err := repository.CompleteAuditRelayBatch(ctx, "worker-b", []auditrelay.Completion{{EventID: second[0].Event.EventID, LeaseToken: second[0].LeaseToken, Disposition: auditrelay.Delivered}}); err != nil {
t.Fatal(err)
}
var delivered bool
if err := admin.QueryRowContext(ctx, `SELECT delivered_at IS NOT NULL FROM sense.device_operation_outbox WHERE event_id=$1`, first[0].Event.EventID).Scan(&delivered); err != nil || !delivered {
t.Fatalf("delivered=%v err=%v", delivered, err)
}
}
+973
View File
@@ -0,0 +1,973 @@
[CmdletBinding()]
param(
[string]$PgRoot = 'D:\pgsql17',
[string]$RuntimeRoot = (Join-Path ([IO.Path]::GetTempPath()) 'yovision-t014'),
[ValidateRange(1, 1440)]
[int]$ObservationMinutes = 30,
[string]$OutputPath = '',
[switch]$PreflightOnly
)
$ErrorActionPreference = 'Stop'
$ProgressPreference = 'SilentlyContinue'
Set-StrictMode -Version Latest
$sourceCount = 16
$faultIndexes = @(5, 6, 7, 8)
$sampleSeconds = 10
$mediaMTXVersion = 'v1.19.3'
$mediaMTXSHA256 = '5d82148d1032a6a190d9909a2997d9989457aaadf49af87dd02cd4512d31bebe'
$mediaMTXExecutableSHA256 = '1cda85249312cb9463f9f94c5a712b9f160c9af3fd9490f0d4723911d7880e05'
$migrationNames = @(
'002_bell.sql',
'003_sense.sql',
'004_privileges.sql',
'005_area_policy.sql',
'006_device_operation_outbox.sql',
'007_privileges_area_audit.sql',
'008_control_api.sql',
'009_privileges_control_api.sql',
'010_reconcile_safety.sql',
'011_privileges_reconcile_safety.sql'
)
function ConvertTo-Base64Url([byte[]]$Bytes) {
return [Convert]::ToBase64String($Bytes).TrimEnd('=').Replace('+', '-').Replace('/', '_')
}
function New-RandomBytes([int]$Count) {
$bytes = [byte[]]::new($Count)
[Security.Cryptography.RandomNumberGenerator]::Fill($bytes)
return $bytes
}
function New-SecretToken {
return ConvertTo-Base64Url (New-RandomBytes 32)
}
function Get-SHA256Hex([string]$Value) {
$bytes = [Text.Encoding]::UTF8.GetBytes($Value)
$digest = [Security.Cryptography.SHA256]::HashData($bytes)
return ([Convert]::ToHexString($digest)).ToLowerInvariant()
}
function Get-FreeTcpPort([Collections.Generic.HashSet[int]]$Reserved) {
while ($true) {
$listener = [Net.Sockets.TcpListener]::new([Net.IPAddress]::Loopback, 0)
$listener.Start()
$port = ([Net.IPEndPoint]$listener.LocalEndpoint).Port
$listener.Stop()
if ($Reserved.Add($port)) {
return $port
}
}
}
function Wait-Port([int]$Port, [int]$TimeoutSeconds = 30) {
$watch = [Diagnostics.Stopwatch]::StartNew()
while ($watch.Elapsed.TotalSeconds -lt $TimeoutSeconds) {
$client = [Net.Sockets.TcpClient]::new()
try {
$task = $client.ConnectAsync('127.0.0.1', $Port)
if ($task.Wait(500) -and $client.Connected) {
return
}
}
catch {
}
finally {
$client.Dispose()
}
Start-Sleep -Milliseconds 250
}
throw "local process did not listen before timeout"
}
function Start-ManagedProcess(
[string]$Name,
[string]$FilePath,
[string[]]$Arguments,
[hashtable]$Environment = @{}
) {
$start = [Diagnostics.ProcessStartInfo]::new()
$start.FileName = $FilePath
$start.WorkingDirectory = $script:session
$start.UseShellExecute = $false
$start.CreateNoWindow = $true
$start.RedirectStandardOutput = $true
$start.RedirectStandardError = $true
foreach ($argument in $Arguments) {
$start.ArgumentList.Add($argument)
}
foreach ($entry in $Environment.GetEnumerator()) {
$start.Environment[$entry.Key] = [string]$entry.Value
}
$process = [Diagnostics.Process]::new()
$process.StartInfo = $start
if (-not $process.Start()) {
throw "failed to start required process: $Name"
}
return [pscustomobject]@{
Name = $Name
Process = $process
Stdout = $process.StandardOutput.ReadToEndAsync()
Stderr = $process.StandardError.ReadToEndAsync()
}
}
function Stop-ManagedProcess($Managed) {
if ($null -eq $Managed -or $null -eq $Managed.Process) {
return
}
if (-not $Managed.Process.HasExited) {
$Managed.Process.Kill()
$Managed.Process.WaitForExit(10000) | Out-Null
}
}
function Stop-AllManagedProcesses($Items) {
$processes = @($Items | Where-Object { $null -ne $_ -and $null -ne $_.Process })
foreach ($item in $processes) {
try {
if (-not $item.Process.HasExited) {
$item.Process.Kill()
}
}
catch {
if (-not $item.Process.HasExited) { throw }
}
}
$cleanupWatch = [Diagnostics.Stopwatch]::StartNew()
while ($cleanupWatch.Elapsed.TotalSeconds -lt 10 -and
@($processes | Where-Object { -not $_.Process.HasExited }).Count -gt 0) {
Start-Sleep -Milliseconds 100
}
$remaining = @($processes | Where-Object { -not $_.Process.HasExited })
if ($remaining.Count -gt 0) {
Write-Warning "managed processes did not exit within the shared cleanup deadline: $($remaining.Count)"
}
foreach ($item in $processes) {
if ($item.Process.HasExited) {
$item.Stdout.Wait(2000) | Out-Null
$item.Stderr.Wait(2000) | Out-Null
$item.Process.Dispose()
}
}
}
function Remove-SessionDirectory([string]$Path) {
Get-ChildItem -LiteralPath $Path -Recurse -Force -ErrorAction SilentlyContinue |
ForEach-Object { $_.Attributes = 'Normal' }
(Get-Item -LiteralPath $Path -Force).Attributes = 'Directory'
$deleteWatch = [Diagnostics.Stopwatch]::StartNew()
while (Test-Path -LiteralPath $Path) {
try {
Remove-Item -LiteralPath $Path -Recurse -Force -ErrorAction Stop
}
catch {
if ($deleteWatch.Elapsed.TotalSeconds -ge 10) { throw }
Start-Sleep -Milliseconds 200
}
}
}
function Assert-Alive($Managed) {
if ($null -eq $Managed -or $Managed.Process.HasExited) {
$code = if ($null -eq $Managed) { 'not-started' } else { $Managed.Process.ExitCode }
throw "required process exited: $($Managed.Name), code=$code"
}
}
function Invoke-Checked {
param(
[Parameter(Mandatory)] [string]$FilePath,
[Parameter(ValueFromRemainingArguments)] [string[]]$Arguments
)
& $FilePath @Arguments 2>&1 | Out-Null
if ($LASTEXITCODE -ne 0) {
throw "required command failed with exit code $LASTEXITCODE"
}
}
function Invoke-ControlRequest(
[string]$Method,
[string]$Path,
[int[]]$ExpectedStatus,
$Body = $null,
[hashtable]$ExtraHeaders = @{}
) {
$headers = @{
Authorization = "Bearer $script:controlToken"
Accept = 'application/json'
}
foreach ($entry in $ExtraHeaders.GetEnumerator()) {
$headers[$entry.Key] = $entry.Value
}
$parameters = @{
Method = $Method
Uri = "http://127.0.0.1:$script:senseHTTPPort$Path"
Headers = $headers
TimeoutSec = 15
NoProxy = $true
SkipHttpErrorCheck = $true
}
if ($null -ne $Body) {
$parameters.ContentType = 'application/json'
$parameters.Body = ($Body | ConvertTo-Json -Depth 10 -Compress)
}
$response = Invoke-WebRequest @parameters
$statusCode = [int]$response.StatusCode
$statusMatches = $false
foreach ($expected in $ExpectedStatus) {
if ([int]$expected -eq $statusCode) {
$statusMatches = $true
break
}
}
if (-not $statusMatches) {
$code = 'unknown'
try {
$code = ((Get-ResponseText $response) | ConvertFrom-Json).code
}
catch {
}
throw "Control API returned unexpected status $statusCode, expected=$($ExpectedStatus -join ','), code=$code"
}
return $response
}
function Get-ResponseText($Response) {
if ($Response.Content -is [byte[]]) {
return [Text.Encoding]::UTF8.GetString($Response.Content)
}
return [string]$Response.Content
}
function Convert-ResponseJSON($Response) {
return ((Get-ResponseText $Response) | ConvertFrom-Json)
}
function Get-ResponseETag($Response) {
$value = [string]$Response.Headers.ETag
if ([string]::IsNullOrWhiteSpace($value)) {
throw 'Control API response did not contain ETag'
}
return $value
}
function Get-DevicePage {
$response = Invoke-ControlRequest 'GET' '/api/v1/sites/site-t014/devices?limit=100' @(200)
return Convert-ResponseJSON $response
}
function Get-DeviceEntries([string[]]$DeviceIDs) {
$entries = [Collections.Generic.List[object]]::new()
foreach ($id in $DeviceIDs) {
$response = Invoke-ControlRequest 'GET' "/api/v1/sites/site-t014/devices/$id" @(200)
$entries.Add([pscustomobject]@{
DeviceID = $id
ETag = Get-ResponseETag $response
})
}
return @($entries)
}
function Invoke-BatchDesiredState(
[string]$DesiredState,
[object[]]$Entries,
[string]$KeySuffix
) {
$items = @($Entries | ForEach-Object {
[ordered]@{
device_id = $_.DeviceID
etag = $_.ETag
desired_state = $DesiredState
}
})
$watch = [Diagnostics.Stopwatch]::StartNew()
$response = Invoke-ControlRequest 'POST' '/api/v1/sites/site-t014/devices:batchDesiredState' @(202) ([ordered]@{
items = $items
reason = "T-014 laboratory $DesiredState baseline"
}) @{ 'Idempotency-Key' = "t014-batch-$KeySuffix-$script:runNonce" }
$operation = Convert-ResponseJSON $response
$results = @($operation.results)
if ($operation.status -ne 'succeeded' -or $results.Count -ne $sourceCount -or
@($results | Where-Object { $_.status -ne 'succeeded' }).Count -ne 0) {
$statusCounts = @($results | Group-Object status | Sort-Object Name |
ForEach-Object { "$($_.Name):$($_.Count)" }) -join ','
$errorCounts = @($results | Where-Object { $_.error_code } | Group-Object error_code | Sort-Object Name |
ForEach-Object { "$($_.Name):$($_.Count)" }) -join ','
throw "batch desired-state operation was not fully successful: operation=$($operation.status), items=$($results.Count), statuses=$statusCounts, errors=$errorCounts"
}
return [ordered]@{
status = [string]$operation.status
item_count = $results.Count
request_seconds = [Math]::Round($watch.Elapsed.TotalSeconds, 3)
}
}
function Get-MediaInventory([int]$APIPort) {
$config = Invoke-RestMethod -Method Get -Uri "http://127.0.0.1:$APIPort/v3/config/paths/list?page=0&itemsPerPage=100" -TimeoutSec 5 -NoProxy
$runtime = Invoke-RestMethod -Method Get -Uri "http://127.0.0.1:$APIPort/v3/paths/list?page=0&itemsPerPage=100" -TimeoutSec 5 -NoProxy
$items = @($runtime.items)
[UInt64]$inboundBytes = 0
[UInt64]$frameErrors = 0
foreach ($item in $items) {
if ($null -ne $item.inboundBytes) {
$inboundBytes += [UInt64]$item.inboundBytes
}
if ($null -ne $item.inboundFramesInError) {
$frameErrors += [UInt64]$item.inboundFramesInError
}
}
return [pscustomobject]@{
Configured = [int]$config.itemCount
Runtime = [int]$runtime.itemCount
Online = @($items | Where-Object { $_.online -eq $true -and $_.available -eq $true }).Count
InboundBytes = $inboundBytes
FrameErrors = $frameErrors
}
}
function Get-SenseMetric([string]$Name, [string]$Labels) {
$response = Invoke-WebRequest -Method Get -Uri "http://127.0.0.1:$script:senseHTTPPort/metrics" -TimeoutSec 5 -NoProxy
$prefix = [Regex]::Escape($Name + $Labels)
$match = [Regex]::Match((Get-ResponseText $response), "(?m)^$prefix\s+([-+0-9.eE]+)$")
if (-not $match.Success) {
throw "required Sense metric was not found: $Name"
}
return [double]::Parse($match.Groups[1].Value, [Globalization.CultureInfo]::InvariantCulture)
}
function Wait-SenseHealth($Managed, [int]$TimeoutSeconds = 45) {
$watch = [Diagnostics.Stopwatch]::StartNew()
while ($watch.Elapsed.TotalSeconds -lt $TimeoutSeconds) {
Assert-Alive $Managed
try {
$ready = Invoke-RestMethod -Method Get -Uri "http://127.0.0.1:$script:senseHTTPPort/readyz" -TimeoutSec 2 -NoProxy
if ($ready.status -eq 'ready') {
return
}
}
catch {
}
Start-Sleep -Milliseconds 500
}
throw 'Sense readiness endpoint did not become ready'
}
function Wait-SourcePublishers([int]$TimeoutSeconds = 90) {
$watch = [Diagnostics.Stopwatch]::StartNew()
while ($watch.Elapsed.TotalSeconds -lt $TimeoutSeconds) {
foreach ($publisher in $script:publishers.Values) {
Assert-Alive $publisher
}
try {
$inventory = Get-MediaInventory $script:sourceAPIPort
if ($inventory.Runtime -eq $sourceCount -and $inventory.Online -eq $sourceCount) {
return [Math]::Round($watch.Elapsed.TotalSeconds, 1)
}
}
catch {
}
Start-Sleep -Seconds 1
}
throw 'independent source publishers did not all become online'
}
function Wait-Convergence([ValidateSet('enabled', 'disabled')] [string]$DesiredState, [int]$TimeoutSeconds = 300) {
$watch = [Diagnostics.Stopwatch]::StartNew()
$last = 'no observation'
while ($watch.Elapsed.TotalSeconds -lt $TimeoutSeconds) {
try {
$page = Get-DevicePage
$devices = @($page.items)
$inventory = Get-MediaInventory $script:productionAPIPort
$totalMetric = Get-SenseMetric 'sense_reconcile_devices' '{state="total"}'
$unconvergedMetric = Get-SenseMetric 'sense_reconcile_devices' '{state="unconverged"}'
if ($DesiredState -eq 'enabled') {
$stateOK = $devices.Count -eq $sourceCount -and
@($devices | Where-Object {
$_.desired_state -ne 'enabled' -or $_.actual_state -ne 'online' -or -not $_.converged
}).Count -eq 0
$mediaOK = $inventory.Configured -eq $sourceCount -and
$inventory.Runtime -eq $sourceCount -and $inventory.Online -eq $sourceCount
$metricsOK = $totalMetric -eq $sourceCount -and $unconvergedMetric -eq 0
}
else {
$stateOK = $devices.Count -eq $sourceCount -and
@($devices | Where-Object {
$_.desired_state -ne 'disabled' -or $_.actual_state -ne 'offline' -or -not $_.converged
}).Count -eq 0
$mediaOK = $inventory.Configured -eq 0 -and $inventory.Runtime -eq 0
$metricsOK = $totalMetric -eq 0 -and $unconvergedMetric -eq 0
}
$last = "devices=$($devices.Count), configured=$($inventory.Configured), runtime=$($inventory.Runtime), online=$($inventory.Online), unconverged=$unconvergedMetric"
if ($stateOK -and $mediaOK -and $metricsOK) {
return [Math]::Round($watch.Elapsed.TotalSeconds, 1)
}
}
catch {
$last = $_.Exception.Message
}
Start-Sleep -Seconds 1
}
throw "devices did not reach $DesiredState convergence: $last"
}
function Wait-PartialFault([int]$TimeoutSeconds = 120) {
$watch = [Diagnostics.Stopwatch]::StartNew()
while ($watch.Elapsed.TotalSeconds -lt $TimeoutSeconds) {
try {
$page = Get-DevicePage
$devices = @($page.items)
$online = @($devices | Where-Object { $_.actual_state -eq 'online' }).Count
$notOnline = $devices.Count - $online
if ($devices.Count -eq $sourceCount -and $online -eq ($sourceCount - $faultIndexes.Count) -and
$notOnline -eq $faultIndexes.Count) {
return [ordered]@{
detection_seconds = [Math]::Round($watch.Elapsed.TotalSeconds, 1)
affected = $notOnline
unaffected_online = $online
}
}
}
catch {
}
Start-Sleep -Seconds 1
}
throw 'four-source fault was not isolated to exactly four devices'
}
function Start-Publisher([int]$Index) {
$name = 'synthetic-{0:D2}' -f $Index
return Start-ManagedProcess "publisher-$Index" $script:ffmpeg @(
'-hide_banner', '-loglevel', 'warning', '-re', '-stream_loop', '-1',
'-i', $script:fixturePath, '-map', '0:v:0', '-c', 'copy', '-an',
'-f', 'rtsp', '-rtsp_transport', 'tcp',
"rtsp://127.0.0.1:$script:sourceRTSPPort/$name"
)
}
function Get-ProcessResourceSample($Managed, [double]$PreviousCPU, [double]$ElapsedSeconds) {
Assert-Alive $Managed
$Managed.Process.Refresh()
$totalCPU = $Managed.Process.TotalProcessorTime.TotalSeconds
$normalizedCPU = 0.0
if ($ElapsedSeconds -gt 0) {
$normalizedCPU = (($totalCPU - $PreviousCPU) / $ElapsedSeconds / [Environment]::ProcessorCount) * 100
}
return [pscustomobject]@{
TotalCPU = $totalCPU
CPUPercent = [Math]::Max(0, $normalizedCPU)
WorkingSetMiB = $Managed.Process.WorkingSet64 / 1MB
PrivateMiB = $Managed.Process.PrivateMemorySize64 / 1MB
Handles = $Managed.Process.HandleCount
}
}
function Get-Average([Collections.Generic.List[double]]$Values) {
if ($Values.Count -eq 0) { return 0.0 }
return ($Values | Measure-Object -Average).Average
}
function Get-Maximum([Collections.Generic.List[double]]$Values) {
if ($Values.Count -eq 0) { return 0.0 }
return ($Values | Measure-Object -Maximum).Maximum
}
function Get-PostgresScalar([string]$Query) {
$value = & $script:psql -X -A -t -v ON_ERROR_STOP=1 -d $script:adminDatabaseDSN -c $Query
if ($LASTEXITCODE -ne 0) {
throw 'PostgreSQL observation query failed'
}
return (($value | Out-String).Trim())
}
function Write-Result([string]$JSON) {
if (-not [string]::IsNullOrWhiteSpace($OutputPath)) {
$resolvedOutput = [IO.Path]::GetFullPath($OutputPath)
$repoPrefix = $script:repoRoot.TrimEnd('\') + '\'
if ($resolvedOutput.StartsWith($repoPrefix, [StringComparison]::OrdinalIgnoreCase)) {
throw 'capacity result output must stay outside the repository'
}
$parent = Split-Path -Parent $resolvedOutput
if (-not (Test-Path -LiteralPath $parent -PathType Container)) {
throw 'capacity result output directory does not exist'
}
[IO.File]::WriteAllText($resolvedOutput, $JSON + [Environment]::NewLine, [Text.UTF8Encoding]::new($false))
}
Write-Output $JSON
}
$script:repoRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot '..\..')).Path
$pgRootResolved = (Resolve-Path -LiteralPath $PgRoot).Path
$pgBin = Join-Path $pgRootResolved 'bin'
$script:initdb = Join-Path $pgBin 'initdb.exe'
$script:pgCtl = Join-Path $pgBin 'pg_ctl.exe'
$script:psql = Join-Path $pgBin 'psql.exe'
$script:createdb = Join-Path $pgBin 'createdb.exe'
$postgres = Join-Path $pgBin 'postgres.exe'
foreach ($required in @($script:initdb, $script:pgCtl, $script:psql, $script:createdb, $postgres)) {
if (-not (Test-Path -LiteralPath $required -PathType Leaf)) {
throw 'PostgreSQL executable is missing under the selected PgRoot'
}
}
$postgresVersion = (& $postgres --version 2>&1 | Out-String).Trim()
if ($LASTEXITCODE -ne 0 -or $postgresVersion -notmatch 'PostgreSQL\) 17\.10$') {
throw 'T-014 requires the frozen PostgreSQL 17.10 binaries'
}
$script:ffmpeg = (Get-Command ffmpeg -ErrorAction Stop).Source
$ffmpegVersion = (& $script:ffmpeg -version 2>&1 | Select-Object -First 1)
if ($ffmpegVersion -notmatch '^ffmpeg version 8\.1\.2') {
throw 'T-014 requires the frozen FFmpeg 8.1.2 build'
}
$senseDirectory = Join-Path $script:repoRoot 'Sense'
$goVersion = (& go '-C' $senseDirectory 'version' 2>&1 | Out-String).Trim()
if ($LASTEXITCODE -ne 0 -or $goVersion -notmatch '^go version go1\.26\.5 ') {
throw 'T-014 requires the frozen Sense Go 1.26.5 toolchain'
}
$systemTemp = [IO.Path]::GetFullPath([IO.Path]::GetTempPath()).TrimEnd('\')
$runtimeResolved = [IO.Path]::GetFullPath($RuntimeRoot).TrimEnd('\')
$runtimePrefix = $systemTemp + '\'
if (-not $runtimeResolved.StartsWith($runtimePrefix, [StringComparison]::OrdinalIgnoreCase) -or
[IO.Path]::GetFileName($runtimeResolved) -notlike 'yovision-t014*') {
throw 'RuntimeRoot must be a yovision-t014 directory under the system temporary directory'
}
$mediaDirectory = Join-Path $runtimeResolved 'mediamtx-v1.19.3'
$mediaMTX = Join-Path $mediaDirectory 'mediamtx.exe'
if ($PreflightOnly) {
$memoryGiB = 0.0
try { $memoryGiB = (Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory / 1GB } catch {}
Write-Result (([ordered]@{
success = $true
preflight_only = $true
source_count = $sourceCount
observation_minutes = $ObservationMinutes
formal_eligible = $ObservationMinutes -ge 30
mediamtx_cached = Test-Path -LiteralPath $mediaMTX -PathType Leaf
postgres_version = $postgresVersion
ffmpeg_version = [string]$ffmpegVersion
go_version = $goVersion
logical_processors = [Environment]::ProcessorCount
total_memory_gib = [Math]::Round($memoryGiB, 2)
} | ConvertTo-Json -Depth 5))
exit 0
}
New-Item -ItemType Directory -Path $runtimeResolved -Force | Out-Null
if (-not (Test-Path -LiteralPath $mediaMTX -PathType Leaf)) {
$zip = Join-Path $runtimeResolved 'mediamtx_v1.19.3_windows_amd64.zip'
Invoke-WebRequest 'https://github.com/bluenviron/mediamtx/releases/download/v1.19.3/mediamtx_v1.19.3_windows_amd64.zip' -OutFile $zip
$actualHash = (Get-FileHash -LiteralPath $zip -Algorithm SHA256).Hash.ToLowerInvariant()
if ($actualHash -ne $mediaMTXSHA256) {
throw 'MediaMTX checksum mismatch'
}
New-Item -ItemType Directory -Path $mediaDirectory -Force | Out-Null
Expand-Archive -LiteralPath $zip -DestinationPath $mediaDirectory -Force
}
if ((Get-FileHash -LiteralPath $mediaMTX -Algorithm SHA256).Hash.ToLowerInvariant() -ne $mediaMTXExecutableSHA256) {
throw 'MediaMTX executable checksum mismatch'
}
$script:runNonce = [Guid]::NewGuid().ToString('N')
$script:session = Join-Path $runtimeResolved ("session-$script:runNonce")
New-Item -ItemType Directory -Path $script:session | Out-Null
$reservedPorts = [Collections.Generic.HashSet[int]]::new()
$script:sourceRTSPPort = Get-FreeTcpPort $reservedPorts
$script:sourceAPIPort = Get-FreeTcpPort $reservedPorts
$productionRTSPPort = Get-FreeTcpPort $reservedPorts
$script:productionAPIPort = Get-FreeTcpPort $reservedPorts
$productionMetricsPort = Get-FreeTcpPort $reservedPorts
$script:senseHTTPPort = Get-FreeTcpPort $reservedPorts
$postgresPort = Get-FreeTcpPort $reservedPorts
$sourceConfig = Join-Path $script:session 'mediamtx-source.yml'
$productionConfig = Join-Path $script:session 'mediamtx-production.yml'
$sourceLines = [Collections.Generic.List[string]]::new()
@(
'logLevel: warn',
"rtspAddress: 127.0.0.1:$script:sourceRTSPPort",
'rtspTransports: [tcp]',
'api: true',
"apiAddress: 127.0.0.1:$script:sourceAPIPort",
'metrics: false', 'rtmp: false', 'hls: false', 'webrtc: false', 'srt: false', 'moq: false',
'paths:'
) | ForEach-Object { $sourceLines.Add($_) }
foreach ($index in 1..$sourceCount) {
$sourceLines.Add((' synthetic-{0:D2}:' -f $index))
$sourceLines.Add(' source: publisher')
}
[IO.File]::WriteAllLines($sourceConfig, $sourceLines, [Text.UTF8Encoding]::new($false))
[IO.File]::WriteAllLines($productionConfig, @(
'logLevel: warn',
"rtspAddress: 127.0.0.1:$productionRTSPPort",
'rtspTransports: [tcp]',
'api: true',
"apiAddress: 127.0.0.1:$script:productionAPIPort",
'metrics: true',
"metricsAddress: 127.0.0.1:$productionMetricsPort",
'rtmp: false', 'hls: false', 'webrtc: false', 'srt: false', 'moq: false',
'paths: {}'
), [Text.UTF8Encoding]::new($false))
$script:fixturePath = Join-Path $script:session 'fixture.mp4'
$senseBinary = Join-Path $script:session 'sense-api.exe'
$authPath = Join-Path $script:session 'control-auth.json'
$cursorPath = Join-Path $script:session 'cursor.key'
$pgData = Join-Path $script:session 'pgdata'
$pgLog = Join-Path $script:session 'postgres.log'
$databaseName = 'yovision_t014'
$adminRootDSN = "postgres://postgres@127.0.0.1:$postgresPort/postgres?sslmode=disable"
$script:adminDatabaseDSN = "postgres://postgres@127.0.0.1:$postgresPort/${databaseName}?sslmode=disable"
$senseDSN = "postgres://yovision_t014_sense@127.0.0.1:$postgresPort/${databaseName}?sslmode=disable"
$existing5432 = @(
Get-NetTCPConnection -State Listen -LocalPort 5432 -ErrorAction SilentlyContinue |
Select-Object -ExpandProperty OwningProcess -Unique | Sort-Object
)
$managed = [Collections.Generic.List[object]]::new()
$script:publishers = @{}
$postgresStarted = $false
$postgresStopped = $false
$success = $false
$stage = 'preparing fixtures'
$failure = $null
$resultJSON = $null
try {
Invoke-Checked $script:ffmpeg '-hide_banner' '-loglevel' 'error' '-f' 'lavfi' '-i' 'testsrc2=size=640x360:rate=10' '-t' '30' '-c:v' 'libx264' '-preset' 'ultrafast' '-tune' 'zerolatency' '-pix_fmt' 'yuv420p' '-g' '10' '-an' '-movflags' '+faststart' '-y' $script:fixturePath
Invoke-Checked 'go' '-C' $senseDirectory 'build' '-o' $senseBinary './cmd/sense-api'
$stage = 'starting isolated PostgreSQL'
New-Item -ItemType Directory -Path $pgData | Out-Null
Invoke-Checked $script:initdb '-D' $pgData '-U' 'postgres' '-A' 'trust' '--encoding=UTF8' '--no-locale' '--no-sync'
$serverOptions = "-h 127.0.0.1 -p $postgresPort -c listen_addresses=127.0.0.1"
# Do not pipe pg_ctl start output: postgres can inherit the pipeline handle and
# keep PowerShell waiting until the server exits.
& $script:pgCtl '-D' $pgData '-l' $pgLog '-o' $serverOptions '-w' 'start'
if ($LASTEXITCODE -ne 0) {
throw "required command failed with exit code $LASTEXITCODE"
}
$postgresStarted = $true
Invoke-Checked $script:psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-f' (Join-Path $script:repoRoot 'deploy\postgres\001_roles.sql')
Invoke-Checked $script:createdb '-h' '127.0.0.1' '-p' ([string]$postgresPort) '-U' 'postgres' $databaseName
foreach ($pass in 1..2) {
foreach ($name in $migrationNames) {
Invoke-Checked $script:psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $script:adminDatabaseDSN '-f' (Join-Path $script:repoRoot "deploy\postgres\$name")
}
}
Invoke-Checked $script:psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-c' 'CREATE ROLE yovision_t014_sense LOGIN IN ROLE sense_app'
Invoke-Checked $script:psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $script:adminDatabaseDSN '-c' "INSERT INTO bell.sites(tenant_id,id,name,max_video_channels) VALUES ('tenant-t014','site-t014','T-014 Lab',16); INSERT INTO bell.areas(tenant_id,site_id,id,name,capture_policy) VALUES ('tenant-t014','site-t014','area-t014','T-014 Area','video_allowed');"
$stage = 'starting independent publishers'
$sourceMedia = Start-ManagedProcess 'mediamtx-source' $mediaMTX @($sourceConfig)
$managed.Add($sourceMedia)
Wait-Port $script:sourceAPIPort
foreach ($index in 1..$sourceCount) {
$script:publishers[$index] = Start-Publisher $index
$managed.Add($script:publishers[$index])
}
$publisherReadySeconds = Wait-SourcePublishers
$productionMedia = Start-ManagedProcess 'mediamtx-production' $mediaMTX @($productionConfig)
$managed.Add($productionMedia)
Wait-Port $script:productionAPIPort
$stage = 'starting Sense Control API'
$script:controlToken = New-SecretToken
$tokenDigest = Get-SHA256Hex $script:controlToken
$cursorKey = ConvertTo-Base64Url (New-RandomBytes 32)
$localCredential = New-SecretToken
[IO.File]::WriteAllText($authPath, (([ordered]@{
version = 1
principals = @([ordered]@{
token_sha256 = $tokenDigest
subject_id = 't014-runner'
actor_type = 'service'
tenant_id = 'tenant-t014'
site_ids = @('site-t014')
permissions = @('sense.devices.read', 'sense.devices.write')
})
} | ConvertTo-Json -Depth 6) + [Environment]::NewLine), [Text.UTF8Encoding]::new($false))
[IO.File]::WriteAllText($cursorPath, $cursorKey + [Environment]::NewLine, [Text.UTF8Encoding]::new($false))
$senseEnvironment = @{
SENSE_HTTP_ADDR = "127.0.0.1:$script:senseHTTPPort"
SENSE_DB_DRIVER = 'postgres'
SENSE_DB_DSN = $senseDSN
SENSE_MEDIAMTX_URL = "http://127.0.0.1:$script:productionAPIPort"
SENSE_RECONCILE_INTERVAL = '1s'
SENSE_RECONCILE_LEASE_DURATION = '30s'
SENSE_RECONCILE_OPERATION_TIMEOUT = '20s'
SENSE_PROBE_INTERVAL = '1s'
SENSE_INSTANCE_ID = 't014-lab'
SENSE_METRICS_ENABLED = 'true'
SENSE_ORPHAN_SCAN_ENABLED = 'false'
SENSE_ONVIF_MODE = 'disabled'
SENSE_CONTROL_API_ENABLED = 'true'
SENSE_CONTROL_AUTH_FILE = $authPath
SENSE_CONTROL_CURSOR_KEY_FILE = $cursorPath
SENSE_CREDENTIAL_SYNTHETIC_ONVIF_USERNAME = 't014-local'
SENSE_CREDENTIAL_SYNTHETIC_ONVIF_PASSWORD = $localCredential
SENSE_CREDENTIAL_SYNTHETIC_RTSP_USERNAME = 't014-local'
SENSE_CREDENTIAL_SYNTHETIC_RTSP_PASSWORD = $localCredential
}
$sense = Start-ManagedProcess 'sense-api' $senseBinary @() $senseEnvironment
$managed.Add($sense)
Wait-SenseHealth $sense
$stage = 'creating 16 disabled devices'
$deviceIDs = [Collections.Generic.List[string]]::new()
foreach ($index in 1..$sourceCount) {
$name = 'synthetic-{0:D2}' -f $index
$response = Invoke-ControlRequest 'POST' '/api/v1/sites/site-t014/devices' @(201) ([ordered]@{
serial_number = "t014-$name"
name = "T-014 source $index"
modality = 'video'
capabilities = @('video_capture')
area_id = 'area-t014'
endpoint_ref = "rtsp://127.0.0.1:$script:sourceRTSPPort/$name"
credential_ref = 'env://synthetic'
desired_state = 'disabled'
}) @{ 'Idempotency-Key' = "t014-create-$index-$script:runNonce" }
$created = Convert-ResponseJSON $response
$deviceIDs.Add([string]$created.id)
}
$stage = 'batch enabling 16 devices'
$firstEnableEntries = Get-DeviceEntries @($deviceIDs)
$firstEnableBatch = Invoke-BatchDesiredState 'enabled' $firstEnableEntries 'enable-1'
$firstEnableSeconds = Wait-Convergence 'enabled'
$firstEnabledInventory = Get-MediaInventory $script:productionAPIPort
$stage = 'rejecting seventeenth enabled device'
$overflowResponse = Invoke-ControlRequest 'POST' '/api/v1/sites/site-t014/devices' @(409) ([ordered]@{
serial_number = 't014-overflow-17'
name = 'T-014 overflow source'
modality = 'video'
capabilities = @('video_capture')
area_id = 'area-t014'
endpoint_ref = "rtsp://127.0.0.1:$script:sourceRTSPPort/synthetic-01"
credential_ref = 'env://synthetic'
desired_state = 'enabled'
}) @{ 'Idempotency-Key' = "t014-overflow-17-$script:runNonce" }
$overflowProblem = Convert-ResponseJSON $overflowResponse
$overflowCodeProperty = $overflowProblem.PSObject.Properties['code']
if ($null -eq $overflowCodeProperty) {
$propertyNames = @($overflowProblem.PSObject.Properties.Name) -join ','
throw "quota response did not contain a stable code field; properties=$propertyNames"
}
$overflowCode = [string]$overflowCodeProperty.Value
$pageAfterOverflow = Get-DevicePage
if ($overflowCode -ne 'quota_exceeded' -or @($pageAfterOverflow.items).Count -ne $sourceCount) {
throw 'the seventeenth enabled device did not fail closed at the site quota'
}
$stage = 'batch disabling 16 devices'
$disableEntries = Get-DeviceEntries @($deviceIDs)
$disableBatch = Invoke-BatchDesiredState 'disabled' $disableEntries 'disable'
$disableSeconds = Wait-Convergence 'disabled'
$disabledInventory = Get-MediaInventory $script:productionAPIPort
$stage = 'batch re-enabling 16 devices'
$secondEnableEntries = Get-DeviceEntries @($deviceIDs)
$secondEnableBatch = Invoke-BatchDesiredState 'enabled' $secondEnableEntries 'enable-2'
$secondEnableSeconds = Wait-Convergence 'enabled'
$secondEnabledInventory = Get-MediaInventory $script:productionAPIPort
$stage = 'injecting four-source fault'
foreach ($index in $faultIndexes) {
Stop-ManagedProcess $script:publishers[$index]
}
$fault = Wait-PartialFault
foreach ($index in $faultIndexes) {
$script:publishers[$index] = Start-Publisher $index
$managed.Add($script:publishers[$index])
}
$faultRecoverySeconds = Wait-Convergence 'enabled'
$fault.recovery_seconds = $faultRecoverySeconds
$fault.final_unconverged = 0
$stage = 'observing stable 16-stream capacity'
$observation = [Diagnostics.Stopwatch]::StartNew()
$targetSeconds = $ObservationMinutes * 60
$senseCPU = [Collections.Generic.List[double]]::new()
$senseWorking = [Collections.Generic.List[double]]::new()
$sensePrivate = [Collections.Generic.List[double]]::new()
$senseHandles = [Collections.Generic.List[double]]::new()
$mediaCPU = [Collections.Generic.List[double]]::new()
$mediaWorking = [Collections.Generic.List[double]]::new()
$mediaPrivate = [Collections.Generic.List[double]]::new()
$mediaHandles = [Collections.Generic.List[double]]::new()
$databaseConnections = [Collections.Generic.List[double]]::new()
$previousSenseCPU = $sense.Process.TotalProcessorTime.TotalSeconds
$previousMediaCPU = $productionMedia.Process.TotalProcessorTime.TotalSeconds
$observationSenseCPUStart = $previousSenseCPU
$observationMediaCPUStart = $previousMediaCPU
$previousSampleAt = 0.0
$initialInventory = Get-MediaInventory $script:productionAPIPort
$samples = 0
$requiredSamples = [int][Math]::Ceiling($targetSeconds / $sampleSeconds)
$maximumUnconverged = 0
while ($samples -lt $requiredSamples) {
$nextSampleAt = ($samples + 1) * $sampleSeconds
$delayMilliseconds = [int][Math]::Ceiling(($nextSampleAt - $observation.Elapsed.TotalSeconds) * 1000)
if ($delayMilliseconds -gt 0) {
Start-Sleep -Milliseconds $delayMilliseconds
}
Assert-Alive $sourceMedia
Assert-Alive $productionMedia
Assert-Alive $sense
foreach ($publisher in $script:publishers.Values) { Assert-Alive $publisher }
$elapsed = $observation.Elapsed.TotalSeconds
$delta = $elapsed - $previousSampleAt
$senseSample = Get-ProcessResourceSample $sense $previousSenseCPU $delta
$mediaSample = Get-ProcessResourceSample $productionMedia $previousMediaCPU $delta
$previousSenseCPU = $senseSample.TotalCPU
$previousMediaCPU = $mediaSample.TotalCPU
$previousSampleAt = $elapsed
$senseCPU.Add($senseSample.CPUPercent)
$senseWorking.Add($senseSample.WorkingSetMiB)
$sensePrivate.Add($senseSample.PrivateMiB)
$senseHandles.Add($senseSample.Handles)
$mediaCPU.Add($mediaSample.CPUPercent)
$mediaWorking.Add($mediaSample.WorkingSetMiB)
$mediaPrivate.Add($mediaSample.PrivateMiB)
$mediaHandles.Add($mediaSample.Handles)
$databaseConnections.Add([double](Get-PostgresScalar "SELECT count(*) FROM pg_stat_activity WHERE datname = '$databaseName'"))
$page = Get-DevicePage
$inventory = Get-MediaInventory $script:productionAPIPort
$unconverged = [int](Get-SenseMetric 'sense_reconcile_devices' '{state="unconverged"}')
$maximumUnconverged = [Math]::Max($maximumUnconverged, $unconverged)
if (@($page.items).Count -ne $sourceCount -or
@($page.items | Where-Object { $_.desired_state -ne 'enabled' -or $_.actual_state -ne 'online' -or -not $_.converged }).Count -ne 0 -or
$inventory.Configured -ne $sourceCount -or $inventory.Runtime -ne $sourceCount -or
$inventory.Online -ne $sourceCount -or $unconverged -ne 0) {
throw 'stability observation detected a non-converged stream'
}
$samples++
}
$observationSeconds = [Math]::Round($observation.Elapsed.TotalSeconds, 1)
$finalInventory = Get-MediaInventory $script:productionAPIPort
$sense.Process.Refresh()
$productionMedia.Process.Refresh()
$senseCPUTotal = $sense.Process.TotalProcessorTime.TotalSeconds - $observationSenseCPUStart
$mediaCPUTotal = $productionMedia.Process.TotalProcessorTime.TotalSeconds - $observationMediaCPUStart
$inboundDelta = [UInt64]($finalInventory.InboundBytes - $initialInventory.InboundBytes)
$frameErrorDelta = [UInt64]($finalInventory.FrameErrors - $initialInventory.FrameErrors)
$inboundMbps = if ($observationSeconds -gt 0) { ($inboundDelta * 8.0) / $observationSeconds / 1000000.0 } else { 0.0 }
$databaseSizeBytes = [Int64](Get-PostgresScalar "SELECT pg_database_size('$databaseName')")
$finalUnconverged = [int](Get-SenseMetric 'sense_reconcile_devices' '{state="unconverged"}')
if ($observationSeconds -lt $targetSeconds -or $finalUnconverged -ne 0 -or
$maximumUnconverged -ne 0 -or $samples -lt $requiredSamples) {
throw 'formal observation did not satisfy sampling and convergence requirements'
}
$memoryGiB = 0.0
try { $memoryGiB = (Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory / 1GB } catch {}
$gitCommit = (& git -C $script:repoRoot rev-parse HEAD | Out-String).Trim()
$success = $true
$resultJSON = [ordered]@{
success = $true
formal_eligible = $ObservationMinutes -ge 30
scope = 'laboratory_software_baseline'
source_count = $sourceCount
independent_publishers = $sourceCount
fixture = [ordered]@{ width = 640; height = 360; fps = 10; codec = 'H.264'; audio = $false; publish_mode = 'preencoded_copy' }
versions = [ordered]@{
repository_commit = $gitCommit
sense_sha256 = (Get-FileHash -LiteralPath $senseBinary -Algorithm SHA256).Hash.ToLowerInvariant()
mediamtx = $mediaMTXVersion
mediamtx_package_sha256 = $mediaMTXSHA256
mediamtx_executable_sha256 = $mediaMTXExecutableSHA256
postgres = $postgresVersion
ffmpeg = [string]$ffmpegVersion
go = $goVersion
}
host = [ordered]@{ logical_processors = [Environment]::ProcessorCount; total_memory_gib = [Math]::Round($memoryGiB, 2) }
quota = [ordered]@{ configured = 16; accepted_devices = 16; seventeenth_rejected = $true; error_code = 'quota_exceeded' }
publishers_ready_seconds = $publisherReadySeconds
batch = [ordered]@{
enable_first = $firstEnableBatch
disable = $disableBatch
enable_second = $secondEnableBatch
}
convergence_seconds = [ordered]@{
enable_first = $firstEnableSeconds
disable = $disableSeconds
enable_second = $secondEnableSeconds
}
configured_paths = [ordered]@{
after_enable_first = $firstEnabledInventory.Configured
after_disable = $disabledInventory.Configured
after_enable_second = $secondEnabledInventory.Configured
}
fault_recovery = $fault
observation = [ordered]@{
requested_minutes = $ObservationMinutes
seconds = $observationSeconds
sample_interval_seconds = $sampleSeconds
samples = $samples
maximum_unconverged = $maximumUnconverged
final_unconverged = $finalUnconverged
final_online_paths = $finalInventory.Online
aggregate_inbound_mbps = [Math]::Round($inboundMbps, 3)
inbound_frame_errors = $frameErrorDelta
}
resources = [ordered]@{
sense = [ordered]@{
cpu_average_percent = [Math]::Round((Get-Average $senseCPU), 3)
cpu_peak_percent = [Math]::Round((Get-Maximum $senseCPU), 3)
cpu_total_seconds = [Math]::Round($senseCPUTotal, 3)
working_set_peak_mib = [Math]::Round((Get-Maximum $senseWorking), 2)
private_peak_mib = [Math]::Round((Get-Maximum $sensePrivate), 2)
handles_peak = [int](Get-Maximum $senseHandles)
}
mediamtx = [ordered]@{
cpu_average_percent = [Math]::Round((Get-Average $mediaCPU), 3)
cpu_peak_percent = [Math]::Round((Get-Maximum $mediaCPU), 3)
cpu_total_seconds = [Math]::Round($mediaCPUTotal, 3)
working_set_peak_mib = [Math]::Round((Get-Maximum $mediaWorking), 2)
private_peak_mib = [Math]::Round((Get-Maximum $mediaPrivate), 2)
handles_peak = [int](Get-Maximum $mediaHandles)
}
postgres_peak_connections = [int](Get-Maximum $databaseConnections)
postgres_database_mib = [Math]::Round($databaseSizeBytes / 1MB, 2)
}
limitations = @('synthetic_sources', 'no_customer_network', 'no_recording', 'no_downstream_readers', 'no_ai_or_gpu', 'not_a_production_sla')
} | ConvertTo-Json -Depth 12
}
catch {
$failure = "T-014 stage '$stage' failed: $($_.Exception.Message)"
}
finally {
Stop-AllManagedProcesses @($managed)
if ($postgresStarted) {
& $script:pgCtl '-D' $pgData '-m' 'fast' '-w' 'stop' | Out-Null
$postgresStopped = $LASTEXITCODE -eq 0
}
if (-not $postgresStarted -or $postgresStopped) {
$resolvedSession = [IO.Path]::GetFullPath($script:session)
$expectedPrefix = $runtimeResolved.TrimEnd('\') + '\'
if (-not $resolvedSession.StartsWith($expectedPrefix, [StringComparison]::OrdinalIgnoreCase) -or
[IO.Path]::GetFileName($resolvedSession) -notlike 'session-*') {
throw 'refusing to clean an unexpected T-014 session path'
}
if (Test-Path -LiteralPath $resolvedSession) {
Remove-SessionDirectory $resolvedSession
}
}
else {
Write-Warning 'Temporary PostgreSQL did not stop; the session directory was retained for manual recovery'
}
$after5432 = @(
Get-NetTCPConnection -State Listen -LocalPort 5432 -ErrorAction SilentlyContinue |
Select-Object -ExpandProperty OwningProcess -Unique | Sort-Object
)
if (($existing5432 -join ',') -ne ($after5432 -join ',')) {
throw 'the existing PostgreSQL listener on port 5432 changed during T-014'
}
}
if (-not $success) {
throw $failure
}
Write-Result $resultJSON
+115
View File
@@ -0,0 +1,115 @@
-- Bell v3 immutable event facts and append-only outcomes.
-- bell_app remains the migration/object owner; bell_runtime is a NOLOGIN
-- least-privilege group for the Bell process.
DO $role$
DECLARE
role_record pg_roles%ROWTYPE;
BEGIN
SELECT * INTO role_record FROM pg_roles WHERE rolname = 'bell_runtime';
IF NOT FOUND THEN
CREATE ROLE bell_runtime NOLOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION;
ELSIF role_record.rolcanlogin OR role_record.rolsuper OR role_record.rolcreatedb
OR role_record.rolcreaterole OR role_record.rolreplication THEN
RAISE EXCEPTION 'existing bell_runtime role has unsafe attributes';
END IF;
IF pg_has_role('bell_runtime', 'bell_app', 'MEMBER')
OR pg_has_role('bell_runtime', 'sense_app', 'MEMBER') THEN
RAISE EXCEPTION 'bell_runtime must not inherit migration or Sense roles';
END IF;
END
$role$;
CREATE OR REPLACE FUNCTION bell.reject_immutable_change()
RETURNS trigger
LANGUAGE plpgsql
SECURITY DEFINER
SET search_path = pg_catalog, bell
AS $function$
BEGIN
RAISE EXCEPTION 'Bell immutable facts cannot be updated or deleted';
END
$function$;
ALTER FUNCTION bell.reject_immutable_change() OWNER TO bell_app;
CREATE TABLE IF NOT EXISTS bell.events (
id text PRIMARY KEY,
tenant_id bigint NOT NULL,
site_id bigint NOT NULL,
device_id bigint NOT NULL,
source_event_id text NOT NULL,
kind text NOT NULL,
severity text NOT NULL,
occurred_at timestamptz NOT NULL,
detected_at timestamptz NOT NULL,
payload_hash bytea NOT NULL,
payload jsonb NOT NULL,
created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
CONSTRAINT bell_event_id_format CHECK (id ~ '^evt_[0-9A-HJKMNP-TV-Z]{26}$'),
CONSTRAINT bell_event_ids_positive CHECK (tenant_id >= 1 AND site_id >= 1 AND device_id >= 1),
CONSTRAINT bell_event_source_id CHECK (source_event_id ~ '^[A-Za-z0-9_-]{1,128}$'),
CONSTRAINT bell_event_kind CHECK (kind ~ '^[a-z][a-z0-9_]{2,63}$'),
CONSTRAINT bell_event_severity CHECK (severity IN ('low', 'medium', 'high', 'critical')),
CONSTRAINT bell_event_time_order CHECK (detected_at >= occurred_at),
CONSTRAINT bell_event_hash_length CHECK (octet_length(payload_hash) = 32),
CONSTRAINT bell_event_payload_object CHECK (jsonb_typeof(payload) = 'object'),
CONSTRAINT bell_event_payload_identity CHECK (
payload ->> 'id' = id
AND (payload ->> 'tenant_id')::bigint = tenant_id
AND (payload ->> 'site_id')::bigint = site_id
AND (payload ->> 'device_id')::bigint = device_id
AND payload ->> 'source_event_id' = source_event_id
AND payload ->> 'kind' = kind
AND payload ->> 'severity' = severity
)
);
ALTER TABLE bell.events OWNER TO bell_app;
DROP TRIGGER IF EXISTS bell_events_immutable ON bell.events;
CREATE TRIGGER bell_events_immutable
BEFORE UPDATE OR DELETE ON bell.events
FOR EACH ROW EXECUTE FUNCTION bell.reject_immutable_change();
CREATE INDEX IF NOT EXISTS bell_events_scope_time_idx
ON bell.events(tenant_id, site_id, occurred_at DESC, id DESC);
CREATE INDEX IF NOT EXISTS bell_events_device_time_idx
ON bell.events(tenant_id, site_id, device_id, occurred_at DESC, id DESC);
CREATE INDEX IF NOT EXISTS bell_events_kind_time_idx
ON bell.events(tenant_id, site_id, kind, occurred_at DESC, id DESC);
CREATE INDEX IF NOT EXISTS bell_events_source_lookup_idx
ON bell.events(tenant_id, site_id, device_id, source_event_id);
CREATE TABLE IF NOT EXISTS bell.event_outcomes (
id text PRIMARY KEY,
event_id text NOT NULL REFERENCES bell.events(id),
outcome text NOT NULL,
outcome_source text NOT NULL,
reason text,
actor_type text NOT NULL,
actor_id text NOT NULL,
occurred_at timestamptz NOT NULL,
record_hash bytea NOT NULL,
received_at timestamptz NOT NULL DEFAULT clock_timestamp(),
CONSTRAINT bell_event_outcome_id_format CHECK (id ~ '^out_[0-9A-HJKMNP-TV-Z]{26}$'),
CONSTRAINT bell_event_outcome_value CHECK (
outcome IN ('unknown', 'true_positive', 'false_positive', 'subject_recovered', 'duplicate', 'test')
),
CONSTRAINT bell_event_outcome_source CHECK (outcome_source IN ('auto', 'manual')),
CONSTRAINT bell_event_outcome_actor CHECK (
actor_type IN ('user', 'service', 'system') AND btrim(actor_id) <> ''
),
CONSTRAINT bell_event_outcome_reason CHECK (reason IS NULL OR char_length(reason) <= 500),
CONSTRAINT bell_event_outcome_hash_length CHECK (octet_length(record_hash) = 32)
);
ALTER TABLE bell.event_outcomes OWNER TO bell_app;
DROP TRIGGER IF EXISTS bell_event_outcomes_immutable ON bell.event_outcomes;
CREATE TRIGGER bell_event_outcomes_immutable
BEFORE UPDATE OR DELETE ON bell.event_outcomes
FOR EACH ROW EXECUTE FUNCTION bell.reject_immutable_change();
CREATE INDEX IF NOT EXISTS bell_event_outcomes_event_time_idx
ON bell.event_outcomes(event_id, occurred_at, id);
INSERT INTO bell.schema_migrations(version) VALUES (3)
ON CONFLICT (version) DO NOTHING;
@@ -0,0 +1,15 @@
-- Runtime Bell can append/read event facts but cannot rewrite, delete or
-- truncate them. Object ownership stays with the offline migration role.
REVOKE ALL ON FUNCTION bell.reject_immutable_change() FROM PUBLIC;
REVOKE ALL ON TABLE bell.events, bell.event_outcomes FROM PUBLIC;
REVOKE ALL ON TABLE bell.events, bell.event_outcomes FROM bell_runtime;
GRANT USAGE ON SCHEMA bell TO bell_runtime;
GRANT SELECT ON TABLE bell.schema_migrations TO bell_runtime;
GRANT SELECT, INSERT ON TABLE bell.events, bell.event_outcomes TO bell_runtime;
ALTER DEFAULT PRIVILEGES FOR ROLE bell_app IN SCHEMA bell
REVOKE ALL ON TABLES FROM PUBLIC;
ALTER DEFAULT PRIVILEGES FOR ROLE bell_app IN SCHEMA bell
REVOKE ALL ON FUNCTIONS FROM PUBLIC;
+130
View File
@@ -0,0 +1,130 @@
-- Sense Outbox relay fencing and Bell global audit facts.
ALTER TABLE sense.device_operation_outbox
ADD COLUMN IF NOT EXISTS relay_lease_owner text,
ADD COLUMN IF NOT EXISTS relay_lease_token bigint NOT NULL DEFAULT 0,
ADD COLUMN IF NOT EXISTS relay_lease_until timestamptz,
ADD COLUMN IF NOT EXISTS last_error_code text,
ADD COLUMN IF NOT EXISTS dead_lettered_at timestamptz;
ALTER TABLE sense.device_operation_outbox
DROP CONSTRAINT IF EXISTS sense_outbox_relay_lease_pair;
ALTER TABLE sense.device_operation_outbox
ADD CONSTRAINT sense_outbox_relay_lease_pair CHECK (
(relay_lease_owner IS NULL AND relay_lease_until IS NULL)
OR (relay_lease_owner IS NOT NULL AND btrim(relay_lease_owner) <> ''
AND relay_lease_until IS NOT NULL)
);
ALTER TABLE sense.device_operation_outbox
DROP CONSTRAINT IF EXISTS sense_outbox_relay_token_nonnegative;
ALTER TABLE sense.device_operation_outbox
ADD CONSTRAINT sense_outbox_relay_token_nonnegative CHECK (relay_lease_token >= 0);
ALTER TABLE sense.device_operation_outbox
DROP CONSTRAINT IF EXISTS sense_outbox_relay_error_length;
ALTER TABLE sense.device_operation_outbox
ADD CONSTRAINT sense_outbox_relay_error_length CHECK (
last_error_code IS NULL OR (
char_length(last_error_code) BETWEEN 1 AND 64
AND last_error_code ~ '^[a-z][a-z0-9_]*$'
)
);
ALTER TABLE sense.device_operation_outbox
DROP CONSTRAINT IF EXISTS sense_outbox_relay_terminal_state;
ALTER TABLE sense.device_operation_outbox
ADD CONSTRAINT sense_outbox_relay_terminal_state CHECK (
delivered_at IS NULL OR dead_lettered_at IS NULL
);
CREATE INDEX IF NOT EXISTS sense_outbox_relay_due_idx
ON sense.device_operation_outbox(
COALESCE(next_attempt_at, available_at), event_id
)
WHERE delivered_at IS NULL AND dead_lettered_at IS NULL;
CREATE TABLE IF NOT EXISTS bell.audit_events (
source_system text NOT NULL,
event_id text NOT NULL,
schema_version smallint NOT NULL,
event_type text NOT NULL,
tenant_id text NOT NULL,
site_id text NOT NULL,
device_id text NOT NULL,
actor_type text NOT NULL,
actor_id text NOT NULL,
reason text,
trace_id text,
aggregate_generation bigint NOT NULL,
quota_source_version bigint,
area_policy_source_version bigint,
payload jsonb NOT NULL,
occurred_at timestamptz NOT NULL,
received_at timestamptz NOT NULL DEFAULT clock_timestamp(),
record_hash bytea NOT NULL,
PRIMARY KEY (source_system, event_id),
CONSTRAINT bell_audit_source_system CHECK (source_system = 'sense'),
CONSTRAINT bell_audit_event_id CHECK (event_id ~ '^audit_[0-9a-f]{32}$'),
CONSTRAINT bell_audit_schema_version CHECK (schema_version IN (1, 2)),
CONSTRAINT bell_audit_event_type CHECK (event_type IN (
'device.created',
'device.desired_state.accepted',
'device.configuration.accepted'
)),
CONSTRAINT bell_audit_identity_not_blank CHECK (
btrim(tenant_id) <> '' AND btrim(site_id) <> '' AND btrim(device_id) <> ''
AND btrim(actor_id) <> ''
),
CONSTRAINT bell_audit_actor_type CHECK (actor_type IN ('user', 'service', 'system')),
CONSTRAINT bell_audit_generation_positive CHECK (aggregate_generation >= 1),
CONSTRAINT bell_audit_projection_versions CHECK (
(quota_source_version IS NULL OR quota_source_version >= 1)
AND (area_policy_source_version IS NULL OR area_policy_source_version >= 1)
),
CONSTRAINT bell_audit_reason_length CHECK (reason IS NULL OR char_length(reason) <= 500),
CONSTRAINT bell_audit_trace_length CHECK (trace_id IS NULL OR char_length(trace_id) <= 128),
CONSTRAINT bell_audit_payload_object CHECK (jsonb_typeof(payload) = 'object'),
CONSTRAINT bell_audit_record_hash_length CHECK (octet_length(record_hash) = 32)
);
ALTER TABLE bell.audit_events OWNER TO bell_app;
DROP TRIGGER IF EXISTS bell_audit_events_immutable ON bell.audit_events;
CREATE TRIGGER bell_audit_events_immutable
BEFORE UPDATE OR DELETE ON bell.audit_events
FOR EACH ROW EXECUTE FUNCTION bell.reject_immutable_change();
CREATE INDEX IF NOT EXISTS bell_audit_scope_time_idx
ON bell.audit_events(tenant_id, site_id, occurred_at DESC, event_id DESC);
CREATE INDEX IF NOT EXISTS bell_audit_device_time_idx
ON bell.audit_events(tenant_id, site_id, device_id, occurred_at DESC, event_id DESC);
CREATE TABLE IF NOT EXISTS bell.audit_relay_receipts (
key_id text NOT NULL,
nonce text NOT NULL,
request_hash bytea NOT NULL,
response_status integer NOT NULL,
response_body jsonb NOT NULL,
received_at timestamptz NOT NULL DEFAULT clock_timestamp(),
expires_at timestamptz NOT NULL,
PRIMARY KEY (key_id, nonce),
CONSTRAINT bell_audit_receipt_key_id CHECK (
char_length(key_id) BETWEEN 1 AND 64 AND key_id ~ '^[A-Za-z0-9][A-Za-z0-9._-]*$'
),
CONSTRAINT bell_audit_receipt_nonce CHECK (
char_length(nonce) BETWEEN 22 AND 64 AND nonce ~ '^[A-Za-z0-9_-]+$'
),
CONSTRAINT bell_audit_receipt_hash_length CHECK (octet_length(request_hash) = 32),
CONSTRAINT bell_audit_receipt_status CHECK (response_status = 200),
CONSTRAINT bell_audit_receipt_body CHECK (jsonb_typeof(response_body) = 'object'),
CONSTRAINT bell_audit_receipt_ttl CHECK (
expires_at >= received_at + interval '10 minutes'
AND expires_at <= received_at + interval '11 minutes'
)
);
ALTER TABLE bell.audit_relay_receipts OWNER TO bell_app;
CREATE INDEX IF NOT EXISTS bell_audit_receipt_expiry_idx
ON bell.audit_relay_receipts(expires_at, key_id, nonce);
INSERT INTO bell.schema_migrations(version) VALUES (4)
ON CONFLICT (version) DO NOTHING;
INSERT INTO sense.schema_migrations(version) VALUES (6)
ON CONFLICT (version) DO NOTHING;
@@ -0,0 +1,11 @@
-- Bell runtime may append global audit facts and maintain only short-lived
-- idempotency receipts. Sense keeps ownership of its local Outbox.
REVOKE ALL ON TABLE bell.audit_events, bell.audit_relay_receipts FROM PUBLIC;
REVOKE ALL ON TABLE bell.audit_events, bell.audit_relay_receipts FROM bell_runtime;
GRANT SELECT, INSERT ON TABLE bell.audit_events TO bell_runtime;
GRANT SELECT, INSERT, DELETE ON TABLE bell.audit_relay_receipts TO bell_runtime;
REVOKE ALL ON TABLE bell.audit_events, bell.audit_relay_receipts FROM sense_app;
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE sense.device_operation_outbox TO sense_app;
+4 -2
View File
@@ -1,6 +1,6 @@
# YoVision PostgreSQL 初始化
本目录实现 T-009~T-012 的 PostgreSQL `17.10` schema。SQL 必须按文件名前缀顺序执行:`001`~`004` 创建 NOLOGIN 权限角色、Bell/Sense 初始对象和配额权限;`005`~`007` 增量增加 Area 与审计;`008`~`009` 增加 Control API 状态;`010`~`011` 增加调和 fencing、MediaMTX Path 历史归属、孤儿报告/受控处置结果和对应最小权限。全部 SQL 可重放。应用登录角色及其密码由部署环境或密钥系统创建,并分别加入 `bell_app`、`sense_app`;仓库不保存登录凭据。
本目录实现 T-009~T-012、T-015~T-016 的 PostgreSQL `17.10` schema。SQL 必须按文件名前缀顺序执行:`001`~`004` 创建 NOLOGIN 权限角色、Bell/Sense 初始对象和配额权限;`005`~`007` 增量增加 Area 与本地审计;`008`~`009` 增加 Control API 状态;`010`~`011` 增加调和 fencing、MediaMTX Path 历史归属、孤儿报告/受控处置结果;`012`~`013` 增加 Bell 不可变事件、append-only outcome 和独立 `bell_runtime` 最小权限;`014`~`015` 增加审计 Outbox relay fencing、Bell 全局审计事实、短期防重收据和双方最小权限。全部 SQL 可重放。对象 owner/迁移角色为 `bell_app`/`sense_app`;应用登录角色及密码由部署环境或密钥系统创建,Sense 登录加入 `sense_app`,Bell 运行登录只加入 `bell_runtime`,仓库不保存登录凭据。
生产/共享实例必须由管理员先备份并在 YoVision 专用数据库中执行。Sense 进程不会用高权限自动建库或建角色。示例只使用私有环境变量,不把实际 DSN 写入脚本或日志:
@@ -20,7 +20,9 @@ Get-ChildItem deploy/postgres/[0-9][0-9][0-9]_*.sql |
- `bell_app` 拥有 `bell.sites`/`bell.areas`、版本 trigger、`bell.site_quota_v1` 和 `bell.area_policy_v1`。
- `sense_app` 拥有 `sense` schema,只获得 `bell` schema 的 `USAGE` 和两个投影视图的 `SELECT`。
- `sense_app` 对 Bell 源表、Bell migration 表和 trigger function 没有权限;启动检查发现权限过宽时拒绝运行。
- `sense.device_operation_outbox` 是本地持久化审计事实,不是 Bell 全局审计真相;relay 的传输、签名、确认和留存尚未实现。
- `sense.device_operation_outbox` 是本地持久化队列,Bell 全局审计真相只写入 `bell.audit_events`。Sense 只领取/确认本地行,不获得 Bell 表权限;Bell 只通过签名 HTTP ingress 收取,不读取 Outbox。
- `bell.audit_events` 与事件事实一样不可更新/删除;`bell.audit_relay_receipts` 只为 10 分钟 nonce 幂等窗口保留,Bell runtime 仅可在这张限定表中查询、插入和清理过期记录。
- `bell.events` 与 `bell.event_outcomes` 由 `bell_app` 拥有;`bell_runtime` 只获得 `SELECT/INSERT`,没有 owner、`UPDATE`、`DELETE` 或 `TRUNCATE` 权限,数据库 trigger 再拒绝 owner 路径的意外事实改写。
- `sense.control_idempotency_receipts` 不保存原始 Idempotency-Key,只保存 scope/request SHA-256 和脱敏响应快照;`batch_operations`/items 只保存逻辑 ID、状态和稳定错误,不保存连接秘密。
- 调和与孤儿租约使用 PostgreSQL `clock_timestamp()` 和 fencing token;过期 worker 不能提交完成/失败或扫描报告。`media_path_ownership`、扫描和处置表不保存 endpoint、credential 或 source URI;数据库约束禁止为 `unowned` finding 写删除结果。
- PUBLIC 对两个业务 schema 的表和函数没有权限。
+51 -2
View File
@@ -71,11 +71,33 @@ BEGIN
OR NOT has_table_privilege('sense_app', 'sense.device_operation_outbox', 'DELETE') THEN
RAISE EXCEPTION 'sense_app lacks access to its local audit Outbox';
END IF;
IF (SELECT max(version) FROM bell.schema_migrations) <> 2
OR (SELECT max(version) FROM sense.schema_migrations) <> 5 THEN
IF (SELECT max(version) FROM bell.schema_migrations) <> 4
OR (SELECT max(version) FROM sense.schema_migrations) <> 6 THEN
RAISE EXCEPTION 'schema migration version drift';
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_roles
WHERE rolname = 'bell_runtime' AND NOT rolcanlogin AND NOT rolsuper
AND NOT rolcreatedb AND NOT rolcreaterole AND NOT rolreplication
) THEN
RAISE EXCEPTION 'bell_runtime role is missing or unsafe';
END IF;
IF NOT has_table_privilege('yovision_t015_bell', 'bell.events', 'SELECT,INSERT')
OR NOT has_table_privilege('yovision_t015_bell', 'bell.event_outcomes', 'SELECT,INSERT')
OR has_table_privilege('yovision_t015_bell', 'bell.events', 'UPDATE')
OR has_table_privilege('yovision_t015_bell', 'bell.events', 'DELETE')
OR has_table_privilege('yovision_t015_bell', 'bell.events', 'TRUNCATE')
OR has_table_privilege('yovision_t015_bell', 'bell.event_outcomes', 'UPDATE')
OR has_table_privilege('yovision_t015_bell', 'bell.event_outcomes', 'DELETE')
OR has_table_privilege('yovision_t015_bell', 'bell.event_outcomes', 'TRUNCATE') THEN
RAISE EXCEPTION 'Bell runtime violates immutable event privileges';
END IF;
IF has_table_privilege('public', 'bell.events', 'SELECT,INSERT,UPDATE,DELETE,TRUNCATE')
OR has_table_privilege('public', 'bell.event_outcomes', 'SELECT,INSERT,UPDATE,DELETE,TRUNCATE') THEN
RAISE EXCEPTION 'Bell event facts leaked to PUBLIC';
END IF;
INSERT INTO bell.sites(tenant_id, id, name)
VALUES ('assertion-tenant', 'assertion-site', 'Assertion Site')
RETURNING max_video_channels, version INTO default_quota, initial_version;
@@ -179,3 +201,30 @@ BEGIN
END IF;
END
$reconcile_safety$;
DO $audit_relay$
BEGIN
IF NOT has_table_privilege('yovision_t012_sense', 'sense.device_operation_outbox', 'SELECT,INSERT,UPDATE,DELETE')
OR has_table_privilege('yovision_t012_sense', 'bell.audit_events', 'SELECT,INSERT,UPDATE,DELETE')
OR has_table_privilege('yovision_t012_sense', 'bell.audit_relay_receipts', 'SELECT,INSERT,UPDATE,DELETE') THEN
RAISE EXCEPTION 'Sense runtime violates audit relay schema ownership';
END IF;
IF NOT has_table_privilege('yovision_t015_bell', 'bell.audit_events', 'SELECT,INSERT')
OR has_table_privilege('yovision_t015_bell', 'bell.audit_events', 'UPDATE,DELETE,TRUNCATE')
OR NOT has_table_privilege('yovision_t015_bell', 'bell.audit_relay_receipts', 'SELECT,INSERT,DELETE')
OR has_table_privilege('yovision_t015_bell', 'bell.audit_relay_receipts', 'UPDATE,TRUNCATE') THEN
RAISE EXCEPTION 'Bell runtime violates audit relay privileges';
END IF;
IF has_table_privilege('public', 'bell.audit_events', 'SELECT,INSERT,UPDATE,DELETE,TRUNCATE')
OR has_table_privilege('public', 'bell.audit_relay_receipts', 'SELECT,INSERT,UPDATE,DELETE,TRUNCATE') THEN
RAISE EXCEPTION 'Bell audit relay tables leaked to PUBLIC';
END IF;
IF NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_schema='sense' AND table_name='device_operation_outbox'
AND column_name='relay_lease_token'
) THEN
RAISE EXCEPTION 'Sense audit relay fencing columns are missing';
END IF;
END
$audit_relay$;
+7 -2
View File
@@ -40,13 +40,13 @@ MVP 以默认 16 路跑通一个场景的端到端闭环;架构、数据和 UI
## 当前阶段
当前为 **M0 指定型号实机准入与 M1 Sense 五路混合源集成均已完成,M2 正在补齐生产接入边界**。后续本地开发统一使用已准入的一台海康样机,多路软件闭环使用独立合成 RTSP 源补足;真实多设备证据延后到客户/借用/租赁条件具备时执行。
当前为 **M0 指定型号实机准入、M1 Sense 五路混合源集成和 M2 本地 16 路软件基线均已完成,M3 已建立 Bell 不可变事件存储及 Sense→Bell 全局审计 relay 基础**。后续本地开发统一使用已准入的一台海康样机,多路软件闭环使用独立合成 RTSP 源补足;真实多设备证据延后到客户/借用/租赁条件具备时执行。客户网络尚未提供,T-013 WireGuard 继续后置,不阻塞 Brain/Bell 本地事件链开发。
优先路径:
1. M0:已用一台真实样机完成首期指定“型号 + 硬件版本 + 固件”ONVIF/RTSP 准入;结论不外推为多品牌兼容,真实断网恢复证据按负责人豁免留痕。
2. M1:只在 `Sense/` 建立 MediaMTX 生产接入骨架,以 1 路准入实机 + 至少 4 路独立合成源完成五路自动建 path、探活和断线重建。
3. M2:Control API、多租户投影、调和 fencing、孤儿安全闸已完成;下一步是隧道和至少一个站点的 16 路全流程。
3. M2:Control API、多租户投影、调和 fencing、孤儿安全闸和本地 16 路批量收敛/30 分钟稳定基线已完成;WireGuard 等客户网络条件具备后补验。
4. M3:Brain + Bell 起步,默认 16 路端到端事件、预警、ack 与误报反馈。
5. M4–M5:64/128 路分片、管理端和第二/第三场景包。
@@ -89,6 +89,11 @@ go -C Sense generate ./internal/mtx ./internal/controlapi
go -C Sense test ./...
go -C Sense vet ./...
go -C Sense build ./...
go -C Bell test ./...
go -C Bell vet ./...
go -C Bell build ./...
```
日常优先运行根目录 `./init.ps1` 或 `./init.sh`,它会执行上述治理、生成、测试、静态检查和构建门禁。Sense 本地启动为 `go -C Sense run ./cmd/sense-api`;默认只监听回环地址,具体配置、MediaMTX 版本与校验方法见 [`03-tech-stack.md`](03-tech-stack.md) 和 [`../Sense/README.md`](../Sense/README.md)。
本机 16 路软件容量基线使用 `./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17`;正式证据必须使用默认 30 分钟窗口,且只证明固定低码率合成负载。结果与限制见 [`research/sense-16-stream-capacity.md`](research/sense-16-stream-capacity.md)。
+29 -3
View File
@@ -54,6 +54,23 @@ T-011 复用同一套冻结的 Go、PostgreSQL、pgx 与 `oapi-codegen`,未增
T-012 同样未增加第三方依赖:PostgreSQL v5 使用数据库时钟租约和 fencing token 协调多实例调和/孤儿扫描;MediaMTX Path 枚举继续使用同版生成客户端。`/metrics` 由 Go 标准库输出 Prometheus 文本格式,只包含固定结果枚举、构建/实例信息和汇总 gauge/counter,不使用 tenant、Site、device 或 Path 标签。孤儿处置是 PostgreSQL 专用本地命令,默认周期任务只报告、不自动删除。
T-014 没有增加生产依赖。Windows 容量脚本冻结并核对 Sense 模块 Go 1.26.5、PostgreSQL 17.10、MediaMTX v1.19.3 与 FFmpeg 8.1.2;MediaMTX Windows amd64 ZIP 使用上表官方 SHA-256,解压后 `mediamtx.exe` 另固定为 `1cda85249312cb9463f9f94c5a712b9f160c9af3fd9490f0d4723911d7880e05`。FFmpeg 只生成无人物夹具并以 16 个独立 `-c copy` 进程发布,不进入生产镜像或 Go module。正式本机结果与限制见 [`research/sense-16-stream-capacity.md`](research/sense-16-stream-capacity.md)。
### 1.3 Bell M3 事件存储冻结版本(T-015)
| 组件 | 冻结版本 | 许可证 / 校验 | 使用与退出路线 |
| --- | --- | --- | --- |
| Go | `1.26.5`(与 Sense 相同) | BSD-3-Clause;复用 §1.1 工具链与校验 | Bell 独立 Go module;升级时同时运行两个 module 的 test/vet/build |
| PostgreSQL / pgx | `17.10` / `github.com/jackc/pgx/v5 v5.10.0` | PostgreSQL License / MIT;module sum `h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=` | 复用 T-009 的 `database/sql` 边界;运行身份改用无 owner 权限的 `bell_runtime` 组 |
| JSON Schema | `github.com/santhosh-tekuri/jsonschema/v6 v6.0.2` | Apache-2.0;module sum `h1:KRzFb2m7YtdldCEkzs6KqmJw4nqEVZGK7IN2kJkjTuQ=` | 运行时校验 Draft 2020-12 与 format;退出时可替换 validator,但冻结 schema 和负向契约测试不变 |
| ULID | `github.com/oklog/ulid/v2 v2.1.2` | Apache-2.0;module sum `h1:IEclFb9JNvzYA6MW2SCxbLzcHTVsfqm3PrqGQJH5zec=` | 只封装在 Bell ID generator port 后;平台前缀仍为 `evt_`,替换实现不得改变格式或所有权 |
T-015 不冻结 Brain→Bell transport,也不产生可部署 Bell API 二进制。内部 factory 接收不含 `id` 的候选事实,由 Bell 生成 ULID 后才形成最终 v0.1 事件;不得把该 Go 类型当成公共网络协议。
### 1.4 Sense 审计 relay(T-016)
T-016 不增加第三方依赖:两端使用 Go 标准库 HTTP、HMAC-SHA256、SHA-256、base64url 和 constant-time compare,数据库继续使用已冻结的 PostgreSQL 17.10/pgx。`cmd/bell-api` 只提供回环 health/ready 和 Sense 审计内部端点;非回环监听必须同时提供绝对路径 TLS 证书/私钥。HMAC key 使用仓库外 version 1 JSON 文件,secret 至少 32 字节;该适配器不替代未来 Bell 公共 JWT/OIDC。
## 2. 外部项目边界
- MiBeeNvr:只用于 M0 隔离实验室、ONVIF兼容性和交互参考,不作为生产依赖。
@@ -65,7 +82,7 @@ T-012 同样未增加第三方依赖:PostgreSQL v5 使用数据库时钟租约
- Python、Savant/DeepStream 的精确版本;Go、MediaMTX 与 PostgreSQL 已分别为 Sense M1/M2 冻结,后续阶段可按升级流程调整。
- Bell 前端框架和组件库。
- 事件投递 transport 从 HTTP 起步还是直接采用消息总线。
- Brain→Bell 业务事件投递 transport;Sense→Bell 审计 relay 已独立冻结为内部 HTTP,不能据此默认 Brain transport。
- 目标 GPU/边缘硬件、解码能力和每 worker 的 `max_sources`。
- MinIO/S3 的精确版本、加密实现,以及客户/法务确认后的最终生命周期策略。
- 短信/语音供应商及生产双路径组合;是否开发原生 App 最早在 M4 根据试点反馈决定。
@@ -97,6 +114,15 @@ go -C Sense build ./...
go -C Sense run ./cmd/sense-api
```
Bell 事件域与内部审计 receiver 单独执行:
```powershell
go -C Bell mod download
go -C Bell test ./...
go -C Bell vet ./...
go -C Bell build ./...
```
直接验证:
```powershell
@@ -114,7 +140,7 @@ python scripts/validate_harness_governance.py
| Sense Go | `go -C Sense generate ./internal/mtx ./internal/controlapi`、`go -C Sense test ./...`、`go -C Sense vet ./...`、`go -C Sense build ./...` | ONVIF、存储、MediaMTX、对账或公共 API 变化 | T-006 使用 1 路指定实机 + 4 路独立合成源;T-007 才要求客户/借用/租赁的真实多路矩阵 |
| PostgreSQL schema/repository | `python -m unittest discover -s tests -p "test_postgres_contract.py"`;Windows 本机再运行 `./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` | migration、权限、配额判定或 PostgreSQL driver 变化 | 不需要摄像头;必须核对临时集群未使用现有 data 目录,现有 5432 listener 前后不变 |
| Brain Python | 单元测试、类型/格式检查(命令待项目脚手架冻结) | mapper、判定状态机、模型接口变化 | 命中模型任务时用冻结数据集和目标硬件 |
| Bell Go/Web | 后端测试 + 前端 lint/test/build(命令待脚手架冻结) | schema、RBAC、预警状态机或公共 UI 变化 | P0 流程由产品/值班角色验收 |
| 容量/分片 | 任务内基准脚本 | 16/64/128 路里程碑 | 目标网络、媒体和 GPU 硬件必需 |
| Bell Go/Web | 当前后端:`go -C Bell test ./...`、`go -C Bell vet ./...`、`go -C Bell build ./...`;前端命令待脚手架冻结 | schema、RBAC、预警状态机或公共 UI 变化 | P0 UI 流程由产品/值班角色验收;纯事件存储不需要 UI 人工验收 |
| 容量/分片 | 任务内基准脚本;本地 16 路入口为 `./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17` | 默认 16 路软件基线,以及后续 64/128 路分片里程碑 | 本地 16 路控制面可用独立合成源;真实多路、生产 SLA、64/128 路、AI/GPU、网络与存储必须使用目标环境分别验收 |
代码脚手架落地时必须把真实命令同步到本文、`init.ps1`/`init.sh`、`00-ai-start-here.md` 和 `current-state.md`。
+11 -7
View File
@@ -48,7 +48,7 @@ Sense ── 视频流/触发信号 ──> Brain
1. Bell 持有站点、Area、配额与 `capture_policy`;首期在同一 PostgreSQL 实例内发布 `bell.site_quota_v1` 和 `bell.area_policy_v1` 两个版本化只读视图。T-009/T-010 已实现 Bell 源表/视图、最小权限和 Sense PostgreSQL repository;Sense 按 Area→Site 的固定 advisory-lock 顺序执行策略与配额准入并记录所用版本。未来分库必须发布新版本契约,不能静默改变 v1 语义。
2. Sense 维护设备期望态,通过 MediaMTX API 和对账器收敛实际态;PostgreSQL 多实例以数据库时钟短租约和 fencing token 领取 due row,过期 worker 不得提交结果。
3. Brain 消费视频与触发信号,产生符合 v0.1 的事件。
4. Bell 做 schema 与代码级断言,生成平台 ULID,保存不可变事件。
4. Bell 做 schema 与代码级断言,生成平台 ULID,保存不可变事件。T-015 已实现内部 candidate→final event factory、append-only PostgreSQL repository 和独立 outcome 事实;Brain→Bell transport、认证和公共 API 仍未冻结。
5. 规则命中后创建独立 Alert,先落库再投递,等待 ack 并按策略升级。
6. Bell 发起 pre-roll 证据回捞,Sense 提供切片接口。
7. 用户标记 outcome,反馈进入 Brain 的数据闭环。
@@ -68,7 +68,7 @@ Sense ── 视频流/触发信号 ──> Brain
9. 投递状态机只依赖 Bell provider 接口,不直接依赖某家短信或语音 SDK;生产前至少两条独立路径并能故障切换。
10. 设备领域模型使用 `modality + capabilities`,页面不以摄像头作为唯一根实体;未实现协议适配器明确为 `adapter_not_ready`,不得用模拟遥测伪装交付。
11. Tenant/Site/Area/RBAC、配额、`capture_policy` 与全局审计属于 Bell;Sense Control API v1 只管理 Device 期望态与收敛查询,Sense 只读消费版本化投影并在设备写路径执行,投影不可用时只阻断相关新变更,不静默切断已有链路。
12. Sense 的设备操作审计先写本地持久化 Outbox,再由幂等 relay 异步送入 Bell 全局审计;不得使用“先执行高风险操作、再尽力入队”的顺序。T-010 已冻结脱敏本地事件并实现原子写入;relay 的 transport、签名、确认、重放窗口与留存仍须独立冻结。
12. Sense 的设备操作审计先写本地持久化 Outbox,再由幂等 relay 异步送入 Bell 全局审计;不得使用“先执行高风险操作、再尽力入队”的顺序。T-016 已实现 HMAC/nonce 内部 HTTP relay、数据库时钟 lease/fencing、逐项确认与 Bell 不可变全局事实;Sense 不获得 Bell schema 权限,Bell 不读取 Sense Outbox。
## 6. 容量架构
@@ -79,6 +79,8 @@ Sense ── 视频流/触发信号 ──> Brain
- 单分片故障不能扩散到其他分片。
- 管理端默认查看 16 路,但按 128 路设计分页、虚拟列表、筛选和批量操作。
T-014 已在单台 Windows 主机上用隔离 PostgreSQL、真实 Control API、单个生产 MediaMTX 和 16 个独立低码率合成 publisher 完成 `16 → 0 → 16` 批量收敛、四路发布故障隔离/恢复和 `1800.1 s / 180` 样本稳定观察,最大/最终 `unconverged=0`。这只证明默认 16 路的本地软件控制面与拉流基线,不改变上述分片架构:`media_shard.max_streams=32` 仍是待 64/128 路目标环境压测的初始建议,不能从 T-014 推导单机、真实摄像头、AI/GPU、存储或生产 SLA。完整证据见 [`research/sense-16-stream-capacity.md`](research/sense-16-stream-capacity.md)。
## 7. 一致性与失败处理
- PostgreSQL `sense` schema 是生产期望态真相源;SQLite 只保留为 M1 本地开发/回归路径。MediaMTX、推理 worker 和对象存储是可对账的实际态。
@@ -86,7 +88,9 @@ Sense ── 视频流/触发信号 ──> Brain
- MediaMTX Path 扫描把“Sense 历史拥有但当前失配”和“从未归属 Sense”分开;未知归属永不自动删除。历史拥有项也只允许在 15 分钟二次快照、1~128 项和 `候选 × 100 <= 当前 Path 总数 × 10` 全部通过时由本地运维命令逐项处置,不提供绕过。
- `bell.site_quota_v1` 行缺失、数值越界、版本回退或读取失败只阻止视频设备新增/启用,不中断已有流;降低配额导致超限时不自动停用,后续准入返回稳定错误并产生运维信号。多 Sense 实例使用 PostgreSQL transaction-scoped advisory lock 串行化同 tenant/site 的计数与写入,不能用进程内锁替代。
- `bell.area_policy_v1` 缺失、非法、版本回退或读取失败时,PostgreSQL repository 拒绝相关新增/启用;`non_imaging_only` 允许非成像设备但拒绝具有 `video_capture` 的设备。已有设备保持原状态,策略冲突由 Bell 管理端显式迁移或取消。同库实时视图不以源记录年龄误判 freshness。
- 设备创建和期望态受理在本地事务内同时写脱敏 `sense.device_operation_outbox`;Outbox 失败回滚业务写入,相同期望态不增加 generation 但仍审计。异步 relay 尚未实现。
- 设备创建和期望态受理在本地事务内同时写脱敏 `sense.device_operation_outbox`;Outbox 失败回滚业务写入,相同期望态不增加 generation 但仍审计。relay 最多领取 100 行,以 30 秒数据库 lease 和单调 fencing token 防止过期 worker 确认;成功和 dead letter 都保留本地事实。
- Bell 先验证时间窗、nonce 和 constant-time HMAC,再逐项校验 v1/v2 事件;同 nonce/同摘要重放原结果,同 nonce/不同摘要拒绝。`bell.audit_events` 只追加且不自动清理,只有 10 分钟幂等收据允许 Bell runtime 删除过期行。
- Bell 最终事件写入 `bell.events`;同平台 ID/同摘要仅视为幂等重放,同 ID/不同摘要拒绝。`bell_runtime` 只有 `SELECT/INSERT`,事件与 outcome 的 UPDATE/DELETE 另由数据库 trigger 拒绝;后续人工/自动 outcome 追加到独立表,不改写事件 payload。
- Brain 投递失败落本地队列重试,不阻塞实时推理主链路。
- Alert 先落库再投递,进程重启恢复未完成升级链。
- 值班排班发布前必须按 Site 时区校验班次空档、重叠、联系人停用和通道验证;排班以新版本和未来生效时间发布,不原地改写历史。交接班是进行中 Alert 的显式责任转移事件,不替代排班版本变更。
@@ -95,7 +99,7 @@ Sense ── 视频流/触发信号 ──> Brain
## 8. 数据与契约
- Bell 核心实体:Tenant → Site → Area(含 `capture_policy`)以及 Role/Binding/Quota/Audit;Sense 核心实体:Device(含 `modality + capabilities`)→ StreamBinding/Zone,以及只记录已观察版本的 SiteQuota/AreaPolicyProjection。两个 schema 以稳定逻辑 ID 关联,不跨 schema 写入;配额 v1 为五列,Area v1 固定为 `tenant_id/site_id/area_id/capture_policy/source_version/source_updated_at` 六列。
- Sense Control API v1 使用站点作用域路径、认证上下文 tenant、HMAC cursor 分页、PostgreSQL 幂等收据与资源 ETag;敏感连接引用只写不读。T-011 已实现 7 个 handler,并以 feature flag 限定到 PostgreSQL 路径;首版外部静态 SHA-256 注册表只实现认证 port 的私有部署适配器。正式签名和兼容规则以 [`contracts/`](contracts/) 为准,Bell 管理服务、JWT/OIDC 与 Outbox relay 仍未实现。
- Sense Control API v1 使用站点作用域路径、认证上下文 tenant、HMAC cursor 分页、PostgreSQL 幂等收据与资源 ETag;敏感连接引用只写不读。T-011 已实现 7 个 handler,并以 feature flag 限定到 PostgreSQL 路径;首版外部静态 SHA-256 注册表只实现认证 port 的私有部署适配器。T-016 审计 relay 采用独立外部 HMAC key 文件和内部端点,不等同于 Bell 公共管理认证。正式签名和兼容规则以 [`contracts/`](contracts/) 为准。
- 业务实体:Rule → Event → Alert → DeliveryAttempt/Ack;Event 与 Alert 不合并。
- Bell 通知域分为三个聚合:Contact/Team 保存身份、成员关系和已验证通道;OnCallSchedule/ScheduleVersion/ShiftException 保存时区、轮换与例外;EscalationPolicy/Step 通过 `person / team / on_call_schedule` 类型化 `target_ref` 引用目标。三者共享逻辑 ID,不复制手机号、班次或轮换字段。
- 每个 DeliveryAttempt 创建时解析当时生效的排班版本,并保存实际收件人、通道、`schedule_version` 和解析时间快照;之后联系人或排班修改不得回写既有投递事实。
@@ -111,16 +115,16 @@ Sense/cmd + Sense/internal/{device,onvif,mtx,reconcile,orphan,metrics,probe,trig
Brain/{pipeline,models,judge,emit,trigger,contracts}
Bell/cmd + Bell/internal/{ingest,event,rule,alert,deliver,feedback,tenant,audit,store}
Bell/{web,packs,contracts}
deploy/postgres/{001_roles.sql,...,011_privileges_reconcile_safety.sql,tests}
deploy/postgres/{001_roles.sql,...,015_privileges_audit_relay.sql,tests}
```
Sense 脚手架和 PostgreSQL `001`~`011` 已实现;Brain/Bell 应用目录仍是占位,真实服务必须由对应任务创建。
Sense 脚手架和 PostgreSQL `001`~`015` 已实现;Bell 已有事件校验/不可变存储 Go 基础和只面向 Sense 审计 relay 的最小 `bell-api`,但没有公共管理 API,Brain 仍为目录占位。
## 10. 开发顺序
- M0 不写生产代码。
- M1 只动 Sense,以 1 路 T-001 准入实机 + 至少 4 路独立合成 RTSP 源完成五路接入骨架与 MediaMTX;设备模型从此时起保持模态/能力可扩展,但不提前实现非视频适配器。真实多设备现场门禁移到 T-007,阻塞生产试点但不阻塞本地开发。
- M2 仍以 Sense 为主,完成 16 路开通/停用、对账、多租户投影与隧道。
- M2 仍以 Sense 为主;Control API、对账、多租户投影和本地 16 路开通/停用基线已完成,隧道等待客户网络条件后补验。
- M3 Brain 与 Bell 同时起步,事件契约首次被真实使用。
- M4/M5 再做 64/128 路分片、完整管理端和多个场景包;M6 接入雷达、门磁、按钮和可穿戴等非视频适配器。
+3 -1
View File
@@ -28,7 +28,7 @@
- T-011:按 T-008 契约实现认证 tenant 上下文、7 个设备/operation handler、PostgreSQL 幂等收据、ETag/HMAC cursor、最多 128 项批量操作和停用精确 path 收敛;公共控制 API 默认关闭且只在 PostgreSQL 路径启用。
- T-012:以 PostgreSQL 数据库时钟租约和 fencing token 防止多实例重复调和;增加可证明的 Path 历史归属、默认只读孤儿报告、15 分钟二次确认、10% 不可绕过安全闸和低基数 `/metrics`。
- T-013(后置,尚未创建):WireGuard 边缘隧道与断网恢复;等待客户网络拓扑、地址规划和部署权限,不阻塞本地软件开发。
- T-014:使用隔离 PostgreSQL、真实 Control API 和 16 个独立本地合成 publisher,完成批量开通/停用、局部故障恢复和资源观测;只形成实验室软件基线,不替代 T-007/T-013 或生产 SLA。
- T-014:使用隔离 PostgreSQL、真实 Control API 和 16 个独立本地合成 publisher,完成批量开通/停用、局部故障恢复和资源观测;正式基线为 `1800.1 s / 180` 样本、最大/最终 `unconverged=0`,仍只形成实验室软件证据,不替代 T-007/T-013 或生产 SLA。
## M3:首个 16 路端到端 MVP
@@ -36,6 +36,8 @@
- Brain 模型接口、判定内核和 v0.1 mapper。
- Bell 事件校验、不可变存储和 ULID。
- T-015:建立 Bell Go 事件域基础,复制并校验冻结 v0.1 schema,由 Bell 生成平台 ULID,执行六项代码断言,并以 `bell_runtime` 最小权限保存不可变事件和 append-only outcome;不冻结 Brain transport 或公共 API。
- T-016:冻结并实现 Sense Outbox → Bell 内部审计 relay;使用 HMAC、nonce 收据、数据库时钟 lease/fencing、逐项确认和 dead letter,在不共享 schema 权限的前提下写入 Bell 不可变全局审计事实。
- 规则引擎、场景包加载、预警状态机与双路径投递。
- 最小 Web/App 处置流程、RBAC 与审计。
- 现场误报基线和反馈队列。
+6 -4
View File
@@ -1,6 +1,6 @@
# API 与契约
> Brain → Bell 事件契约 v0.1、Sense Control API v1、Bell 配额/Area 只读投影 v1 与 Sense 本地设备审计事件 v1/v2 已冻结;其他 API 仍在设计阶段。不得把本文的“待定”自行具体化为公共契约。
> Brain → Bell 事件契约 v0.1、Sense Control API v1、Bell 配额/Area 只读投影 v1、Sense 本地设备审计事件 v1/v2 与 Sense→Bell 审计 relay v1 已冻结;其他 API 仍在设计阶段。不得把本文的“待定”自行具体化为公共契约。
## 1. 已冻结:Brain → Bell 事件契约
@@ -17,19 +17,21 @@
- 证据文件名只含事件 ID 与日期目录,不含 IP、端口、凭据或客户名。
- `sensors` 中恰有一个 primary,且其 `device_id` 与顶层一致。
T-015 已实现 Bell 消费端的内部组装与存储边界:可信 ingress 先接收“不含平台 `id`”的候选事实,Bell 生成 `evt_` ULID 后形成最终 v0.1 对象,再执行 schema 与六项代码断言并不可变落库。该候选类型是 Bell 内部 port,不是 Brain 可依赖的 HTTP/消息总线协议;transport、认证和重放语义仍由后续任务冻结。
## 2. 跨系统接口状态
| 调用方 → 提供方 | 用途 | 当前约束 | 状态 |
| --- | --- | --- | --- |
| Sense → Bell | 读取站点视频配额 | 同一 PostgreSQL 实例内只读 `bell.site_quota_v1`;默认 16、最大 128;失败时拒绝新增/启用但不影响已有流 | T-008 冻结,T-009 已实现数据路径 |
| Sense → Bell | 读取 Area 成像准入 | 只读 `bell.area_policy_v1`;`video_allowed | non_imaging_only`;缺失/非法/回退失败关闭但不影响已有设备 | T-010 已冻结并实现数据路径 |
| Sense → Bell | 汇入设备操作审计 | 本地 Outbox 事件已冻结并原子落库;transport、签名、确认与留存未冻结 | T-010 本地基础已实现,relay 待设计 |
| Sense → Bell | 汇入设备操作审计 | `POST /internal/v1/audit-events:batch`;1~100 项、1 MiB、10 秒 deadline、HMAC/nonce、逐项确认;非回环必须 HTTPS | T-016 已冻结并实现 |
| Bell → Sense | 请求事件证据/pre-roll 切片 | 幂等、按租户授权、异步结果、不得暴露原始凭据 | 待 M3 设计 |
| Bell → Brain | outcome/误报反馈 | 原事件不可变;反馈可重试、去重、审计 | 待 M3 设计 |
| Sense → Brain | 流绑定与设备型触发 | 分片可路由,触发入口与流控制解耦 | 待 M2/M3 设计 |
| Worker → 控制面 | 注册、心跳、容量 | `max_sources` 来自 profile/压测,不固定为 16 | 待 M3 设计 |
冻结签名和失败语义见 [`contracts/README.md`](contracts/README.md)、[`contracts/site-quota-v1.sql`](contracts/site-quota-v1.sql)、[`contracts/area-policy-v1.sql`](contracts/area-policy-v1.sql) 与 [`contracts/sense-device-audit-v1.schema.json`](contracts/sense-device-audit-v1.schema.json)。Bell 拥有投影源数据和视图,Sense 数据库角色只有 `SELECT`;未来分库必须发布新版本,不能在 v1 下把本地视图静默替换为网络调用。
冻结签名和失败语义见 [`contracts/README.md`](contracts/README.md)、[`contracts/sense-audit-relay-v1.openapi.json`](contracts/sense-audit-relay-v1.openapi.json)、[`contracts/site-quota-v1.sql`](contracts/site-quota-v1.sql)、[`contracts/area-policy-v1.sql`](contracts/area-policy-v1.sql) 与 [`contracts/sense-device-audit-v1.schema.json`](contracts/sense-device-audit-v1.schema.json)。Bell 拥有投影源数据和视图,Sense 数据库角色只有 `SELECT`;未来分库必须发布新版本,不能在 v1 下把本地视图静默替换为网络调用。
## 3. 已冻结:Sense Control API v1
@@ -45,7 +47,7 @@
- `endpoint_ref`、`credential_ref`、`profile_token` 只写不读;设备 ID 由服务端生成。普通响应和错误不得包含凭据、完整流 URI、token 或 MediaMTX 内部配置。
- v1 不提供删除设备;停用设备保留历史。写入受理只表示期望态已持久化,不能表示实际态已收敛。
T-008 冻结公共控制契约;T-009/T-010 建立 PostgreSQL 投影、准入和本地审计基础;T-011 已实现 7 个 HTTP handler、外部静态摘要认证适配器、tenant/Site scope、幂等收据、ETag/HMAC cursor 和持久化 batch operation。业务路由默认关闭且仅可在 PostgreSQL 上开启;Bell 管理服务、JWT/OIDC 和 Outbox relay 尚未实现。
T-008 冻结公共控制契约;T-009/T-010 建立 PostgreSQL 投影、准入和本地审计基础;T-011 已实现 7 个 HTTP handler、外部静态摘要认证适配器、tenant/Site scope、幂等收据、ETag/HMAC cursor 和持久化 batch operation。业务路由默认关闭且仅可在 PostgreSQL 上开启;T-016 的审计 relay 是独立内部端点,不替代 Bell 管理服务或 JWT/OIDC。
## 4. 待冻结的 Bell 公共 API
+19 -4
View File
@@ -1,6 +1,6 @@
# Sense 控制面、准入投影与本地审计契约 v1
> 冻结日期:2026-08-07。Control API 契约版本:`1.0.0`。Sense 是设备期望态的提供方;Bell 是 Tenant、Site、Area、RBAC、配额与全局审计的所有者。T-011 已实现 Control API handler 与 PostgreSQL 一致性边界;Bell 管理服务、JWT/OIDC 和 Outbox relay 仍未实现。
> 冻结日期:2026-08-11。Control API 与审计 relay 契约版本:`1.0.0`。Sense 是设备期望态的提供方;Bell 是 Tenant、Site、Area、RBAC、配额与全局审计的所有者。T-016 已实现 Sense Outbox 到 Bell 的内部 relay;Bell 公共管理服务与 JWT/OIDC 仍未实现。
## 契约文件
@@ -9,8 +9,9 @@
| [`sense-control-v1.openapi.json`](sense-control-v1.openapi.json) | Sense | Bell 管理面、受控集成方 | 设备查询、创建、修改、启停与批量操作 |
| [`site-quota-v1.sql`](site-quota-v1.sql) | Bell | Sense | 单 PostgreSQL 实例内的站点视频配额只读投影 |
| [`area-policy-v1.sql`](area-policy-v1.sql) | Bell | Sense | Area 归属与 `capture_policy` 只读投影 |
| [`sense-device-audit-v1.schema.json`](sense-device-audit-v1.schema.json) | Sense | 本地 Outbox;未来 Bell relay | 脱敏设备操作审计事实,不包含传输协议 |
| [`sense-device-audit-v2.schema.json`](sense-device-audit-v2.schema.json) | Sense | 本地 Outbox;未来 Bell relay | v1 后继,增加脱敏配置修改受理事实;v1 文件保持不变 |
| [`sense-device-audit-v1.schema.json`](sense-device-audit-v1.schema.json) | Sense | 本地 Outbox;Bell relay | 脱敏设备操作审计事实,不包含传输协议 |
| [`sense-device-audit-v2.schema.json`](sense-device-audit-v2.schema.json) | Sense | 本地 Outbox;Bell relay | v1 后继,增加脱敏配置修改受理事实;v1 文件保持不变 |
| [`sense-audit-relay-v1.openapi.json`](sense-audit-relay-v1.openapi.json) | Bell | Sense | 内部批量端点、HMAC、逐项确认、nonce 防重与重试边界 |
OpenAPI 的 `/api/v1` 路径是公共控制面边界;`/healthz`、`/readyz` 仍是非业务运维探针。v1 不提供设备删除:停用设备使用期望态接口,保留设备、操作和审计历史。Site、Area、配额、RBAC 和审计聚合不由 Sense 提供 CRUD。
@@ -57,7 +58,19 @@ T-010 冻结 `bell.area_policy_v1` 的列顺序为 `tenant_id/site_id/area_id/ca
`sense-device-audit-v1.schema.json` 继续冻结创建与期望态两类事实且不原地扩展严格枚举。T-011 新增 v2 后继,兼容 v1 两类事件并增加 `device.configuration.accepted`;该 payload 只保存是否变化、字段名和 Area 逻辑 ID,不保存字段值。主体类型为 `user | service | system`,投影版本与 generation 随事实保存;endpoint、credential、profile token、path、密码、完整流 URI 或 MediaMTX 配置始终禁止进入审计。
PostgreSQL repository 必须在设备创建/期望态事务内写 `sense.device_operation_outbox`;Outbox 失败回滚业务写入。相同期望态不增加 generation,但仍产生独立审计事实。Schema 不是 Bell relay 协议:传输端点、签名、批量确认、重放窗口和留存由后续任务冻结。
PostgreSQL repository 必须在设备创建/期望态事务内写 `sense.device_operation_outbox`;Outbox 失败回滚业务写入。相同期望态不增加 generation,但仍产生独立审计事实。事件 schema 继续只定义事实;传输由 `sense-audit-relay-v1.openapi.json` 独立冻结。
## Sense → Bell 审计 relay
Sense 向 `/internal/v1/audit-events:batch` 每批发送 1~100 个事件,请求体不超过 1 MiB、deadline 10 秒。请求用外部文件中的至少 32 字节 secret 做 HMAC-SHA256,canonical string 为 method、path、Unix 秒、随机 nonce 与 body SHA-256 的换行拼接;非回环地址必须使用 HTTPS。Bell 允许 300 秒时钟偏差并将 `(key_id, nonce)` 收据保留 600 秒:相同请求摘要返回原结果,不同摘要返回 `409 replay_conflict`。
Outbox 用 30 秒数据库时钟 lease、单调 fencing token 和 `FOR UPDATE SKIP LOCKED` 协调实例。`accepted/duplicate` 才标记 delivered;逐项 `rejected` 进入 dead letter;网络、5xx、认证失败或缺失结果以 1 秒起步、最多 300 秒指数退避。Sense 不直接访问 Bell schema,Bell 不读取 Sense Outbox;全局 `bell.audit_events` 不自动删除,只有短期 relay receipt 自动过期。
两端读取同格式的仓库外 key 文件;Bell 可同时接受多个 key,Sense 用 `SENSE_AUDIT_RELAY_KEY_ID` 选择一个,便于先加新 key、切换发送端、再移除旧 key。占位结构如下,`secret_base64url` 必须替换为至少 32 个随机字节的无填充 base64url,不能提交真实值:
```json
{"version":1,"keys":[{"key_id":"sense-a","secret_base64url":"<external-secret>"}]}
```
## 兼容与废弃
@@ -73,8 +86,10 @@ PostgreSQL repository 必须在设备创建/期望态事务内写 `sense.device_
```powershell
python -m json.tool docs/contracts/sense-control-v1.openapi.json | Out-Null
python -m json.tool docs/contracts/sense-device-audit-v2.schema.json | Out-Null
python -m json.tool docs/contracts/sense-audit-relay-v1.openapi.json | Out-Null
python -m unittest discover -s tests -p "test_sense_control_contract.py"
python -m unittest discover -s tests -p "test_sense_control_implementation.py"
python -m unittest discover -s tests -p "test_sense_audit_relay_contract.py"
```
测试同时校验 OpenAPI 结构、生成 server glue、HTTP handler 与 PostgreSQL migration/事务;它不替代 Bell 消费方联合验收或客户现场容量验证。
@@ -0,0 +1,94 @@
{
"openapi": "3.1.0",
"info": {
"title": "YoVision Sense Audit Relay",
"version": "1.0.0",
"description": "Internal, signed and idempotent delivery of redacted Sense device audit facts to Bell."
},
"paths": {
"/internal/v1/audit-events:batch": {
"post": {
"operationId": "receiveSenseAuditBatch",
"description": "Accepts 1-100 events in a body no larger than 1048576 bytes. The request deadline is 10 seconds. HMAC clock skew is at most 300 seconds and nonce receipts live for 600 seconds.",
"parameters": [
{"name": "X-YoVision-Key-Id", "in": "header", "required": true, "schema": {"type": "string", "pattern": "^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$"}},
{"name": "X-YoVision-Timestamp", "in": "header", "required": true, "description": "Unix seconds", "schema": {"type": "string", "pattern": "^[0-9]{10,}$"}},
{"name": "X-YoVision-Nonce", "in": "header", "required": true, "description": "16-48 random bytes encoded as unpadded base64url", "schema": {"type": "string", "minLength": 22, "maxLength": 64, "pattern": "^[A-Za-z0-9_-]+$"}},
{"name": "X-YoVision-Signature", "in": "header", "required": true, "description": "Unpadded base64url HMAC-SHA256 over POST, path, timestamp, nonce and lowercase SHA-256 body digest joined by newlines", "schema": {"type": "string", "minLength": 43, "maxLength": 43}}
],
"requestBody": {
"required": true,
"content": {"application/json": {"schema": {"$ref": "#/components/schemas/BatchRequest"}}}
},
"responses": {
"200": {"description": "Stored, duplicate or permanently rejected per item. Same key ID, nonce and request digest returns the original response.", "content": {"application/json": {"schema": {"$ref": "#/components/schemas/BatchResponse"}}}},
"400": {"description": "Malformed batch envelope"},
"401": {"description": "Invalid key, timestamp, nonce or signature"},
"409": {"description": "Same key ID and nonce used with a different request digest; error is replay_conflict"},
"413": {"description": "Body exceeds 1048576 bytes"},
"503": {"description": "Bell cannot atomically persist the batch and receipt"}
}
}
}
},
"components": {
"schemas": {
"BatchRequest": {
"type": "object", "additionalProperties": false, "required": ["events"],
"properties": {"events": {"type": "array", "minItems": 1, "maxItems": 100, "items": {"$ref": "#/components/schemas/Envelope"}}}
},
"Envelope": {
"type": "object", "additionalProperties": false, "required": ["schema_version", "event"],
"properties": {
"schema_version": {"type": "integer", "enum": [1, 2]},
"event": {"$ref": "#/components/schemas/AuditEvent"}
}
},
"AuditEvent": {
"type": "object", "additionalProperties": false,
"required": ["event_id", "event_type", "tenant_id", "site_id", "device_id", "actor", "reason", "trace_id", "aggregate_generation", "projection_versions", "data", "occurred_at"],
"properties": {
"event_id": {"type": "string", "pattern": "^audit_[0-9a-f]{32}$"},
"event_type": {"type": "string", "enum": ["device.created", "device.desired_state.accepted", "device.configuration.accepted"]},
"tenant_id": {"$ref": "#/components/schemas/LogicalId"},
"site_id": {"$ref": "#/components/schemas/LogicalId"},
"device_id": {"$ref": "#/components/schemas/LogicalId"},
"actor": {"type": "object", "additionalProperties": false, "required": ["type", "id"], "properties": {"type": {"type": "string", "enum": ["user", "service", "system"]}, "id": {"type": "string", "minLength": 1, "maxLength": 200}}},
"reason": {"type": ["string", "null"], "maxLength": 500},
"trace_id": {"type": ["string", "null"], "maxLength": 128},
"aggregate_generation": {"type": "integer", "minimum": 1},
"projection_versions": {"type": "object", "additionalProperties": false, "required": ["quota_source_version", "area_policy_source_version"], "properties": {"quota_source_version": {"type": ["integer", "null"], "minimum": 1}, "area_policy_source_version": {"type": ["integer", "null"], "minimum": 1}}},
"data": {"type": "object"},
"occurred_at": {"type": "string", "format": "date-time"}
}
},
"LogicalId": {"type": "string", "minLength": 1, "maxLength": 128, "pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"},
"BatchResponse": {
"type": "object", "additionalProperties": false, "required": ["results"],
"properties": {"results": {"type": "array", "minItems": 1, "maxItems": 100, "items": {"$ref": "#/components/schemas/Result"}}}
},
"Result": {
"type": "object", "additionalProperties": false, "required": ["event_id", "status"],
"properties": {
"event_id": {"type": "string"},
"status": {"type": "string", "enum": ["accepted", "duplicate", "rejected"]},
"error_code": {"type": "string", "pattern": "^[a-z][a-z0-9_]{0,63}$"}
}
}
}
},
"x-yovision-signature": {
"canonical": "METHOD\\nPATH\\nTIMESTAMP\\nNONCE\\nLOWERCASE_SHA256_BODY",
"algorithm": "HMAC-SHA256",
"encoding": "base64url-no-padding",
"clock_skew_seconds": 300,
"receipt_ttl_seconds": 600
},
"x-yovision-delivery": {
"lease_seconds": 30,
"initial_retry_seconds": 1,
"maximum_retry_seconds": 300,
"successful_statuses": ["accepted", "duplicate"],
"permanent_status": "rejected"
}
}
+28 -8
View File
@@ -1,26 +1,29 @@
# 当前实现状态
> 快照日期:2026-08-07。只记录仓库现实与 blocker;任务实时状态到 Gitea Issue 查看。
> 快照日期:2026-08-11。只记录仓库现实与 blocker;任务实时状态到 Gitea Issue 查看。
## 当前阶段
- 阶段:M0 指定摄像头型号准入已完成;M1 的“一实机 + 四合成源”实验室软件闭环已通过;M2 已完成 Control API、Site/Area 准入、本地审计事务、多实例调和 fencing 和孤儿受控处置基础,但 WireGuard、16 路容量出口、五条独立真实上游和生产 SLA 尚未验收。
- 生产代码:Sense 已包含可构建进程、SQLite/PostgreSQL repository、Site/Area 准入、设备操作 Outbox、标准 ONVIF SOAP/WS-Security adapter、凭据引用、MediaMTX 生成客户端、Control API v1、对账/探活、数据库租约、孤儿只读扫描/受控命令和低基数指标;Bell 管理服务/JWT、Outbox relay、WireGuard 和完整生产部署仍未实现。
- 阶段:M0 指定摄像头型号准入、M1“一实机 + 四合成源”软件闭环和 M2 本地 16 路批量收敛/稳定基线已通过;M3 已建立 Bell 不可变事件存储及 Sense→Bell 全局审计 relay 基础。客户网络尚未提供,WireGuard T-013 后置,五条独立真实上游和生产 SLA 仍未验收。
- 生产代码:Sense 已包含可构建进程、SQLite/PostgreSQL repository、Site/Area 准入、设备操作 Outbox、可选签名 relay、标准 ONVIF SOAP/WS-Security adapter、凭据引用、MediaMTX 生成客户端、Control API v1、对账/探活、数据库租约、孤儿只读扫描/受控命令、低基数指标和可重复 16 路容量脚本;Bell 已包含事件 v0.1 校验/不可变存储、append-only outcome,以及只服务 Sense 审计的最小 `bell-api` 和全局审计 repository,但仍没有 Brain 事件 ingress、公共管理服务/JWT、规则/Alert 或 Web/H5。
- 默认容量:16 路;单站点本阶段上限 128 路,必须横向分片。
## 仓库现实
- `Sense/` 已有 Go module 与 `cmd/sense-api`;`Brain/`、`Bell/` 仍只有目录占位,Bell 的首个数据库源表/视图 migration 位于根目录 `deploy/postgres/`。
- `Sense/` 已有 Go module 与 `cmd/sense-api`;`Bell/` 已有事件域 Go module 和最小 `cmd/bell-api` 内部审计 receiver;`Brain/` 仍只有目录占位。Bell/Sense migration 统一位于根目录 `deploy/postgres/`。
- Sense 设备模型使用 `modality + capabilities`,SQLite 执行 v1 migration;视频配额默认 16、允许 1~128,17/128/129、新增/启用和“降低配额不关闭已有流”均有测试。
- T-009 冻结 PostgreSQL `17.10` 和 `pgx/v5 v5.10.0`,实现 `bell`/`sense` schema、NOLOGIN 权限角色、Bell Site 版本 trigger、`bell.site_quota_v1` 和 Sense PostgreSQL repository;同站点并发准入用事务级 advisory lock,配额缺失/越界/版本回退时失败关闭且不改变已有流。
- T-010 增量实现 `bell.areas`、`bell.area_policy_v1`、Area 版本观察和 `sense.device_operation_outbox`;`non_imaging_only` 拒绝成像设备创建/启用,失败不改变已有设备。设备创建/期望态受理与脱敏 Outbox 同事务,相同期望态不增加 generation 但仍审计。
- Windows 隔离测试使用 `D:\pgsql17\bin` 启动随机回环端口临时集群,`001`~`011` migration 可重放、权限断言和 29 个真实 PostgreSQL 测试通过后自动清理;现有 `D:\pgsql17\data` 和 5432 服务未被读取、停止或修改。
- Windows 隔离测试使用 `D:\pgsql17\bin` 启动随机回环端口临时集群,`001`~`015` migration 可重放;Sense Outbox fencing、Bell event/global-audit repository、nonce 收据、权限、幂等冲突与不可变性测试通过后自动清理,现有 `D:\pgsql17\data` 和 5432 服务未被读取、停止或修改。
- T-015 冻结 Bell Go 1.26.5、JSON Schema v6.0.2 和 ULID v2.1.2;Bell 拒绝上游自报平台 ID,在内部 candidate 组装后执行冻结 v0.1 schema 与六项语义断言。`bell_runtime` 只允许追加/读取 `bell.events`、`bell.event_outcomes`;Brain transport、整数事件 ID 与现有文本逻辑 ID 的跨系统映射、公共 API 和生产隐私 resolver 仍未冻结,不能把内部 factory 当成已上线入口。
- T-016 冻结 `sense-audit-relay-v1`:每批 1~100 项、1 MiB、10 秒 deadline、300 秒时钟窗、600 秒 nonce 收据、30 秒数据库 lease、1~300 秒退避。Sense 使用 `FOR UPDATE SKIP LOCKED` 和 fencing token;Bell constant-time 校验 HMAC,逐项返回 accepted/duplicate/rejected,并把全局事实追加到不可变 `bell.audit_events`。relay 默认关闭,非回环两端必须 HTTPS/TLS,key 只从仓库外文件读取。
- MediaMTX 固定为独立二进制 `v1.19.3`,官方 OpenAPI 已按 SHA-256 vendoring,并由固定 `oapi-codegen v2.8.0` 生成客户端;手写薄封装有 create/read/delete、幂等 ensure、探活和只返回名称的受限分页枚举测试。
- T-003 对账进度与指数退避持久化,覆盖取消和 SQLite 重启恢复;T-006 增加真实 ONVIF adapter、RTSP router、实验室播种/状态工具、故障代理和五路自动验收。T-012 的普通调和不枚举孤儿;独立 PostgreSQL 扫描默认只报告,未知归属永不删除。
- T-006 正式使用 1 台准入实机和 4 个独立合成 publisher 连续观察 `1806.6 s` / 180 次采样,四类恢复均通过,最大与最终 `unconverged` 均为 0;详细证据见 `docs/research/sense-5-stream-integration.md`。
- T-014 正式使用隔离 PostgreSQL、真实 Control API、两套 MediaMTX 和 16 个独立低码率合成 publisher,完成 17 路配额拒绝、三轮 `16 → 0 → 16` 批量收敛和固定四路故障恢复;稳定观察 `1800.1 s` / 180 次采样,最大与最终 `unconverged` 均为 0、最终在线 Path 16、帧错误 0。证据见 `docs/research/sense-16-stream-capacity.md`;不外推到真实 16 机、网络、录像、AI/GPU、64/128 路或生产 SLA。
- `docs/raw/01`~`08` 已记录需求、分析、方案、客户场景、事件比对和三系统职责。
- `docs/raw/contracts/event-v0.1.schema.json` 已冻结,并有多份示例与语义说明。
- `docs/contracts/sense-control-v1.openapi.json` 的 7 个站点作用域/operation endpoint 已由 T-011 实现:外部静态 SHA-256 主体注册表、tenant/Site scope、HMAC cursor、ETag、PostgreSQL 24 小时幂等收据和最多 128 项 batch operation 均有代码与隔离集成测试。默认 SQLite 只暴露运维探针与低基数 `/metrics`,不注册业务路由;Bell 管理服务、JWT/OIDC 和 Outbox relay 尚未实现。
- `docs/contracts/sense-control-v1.openapi.json` 的 7 个站点作用域/operation endpoint 已由 T-011 实现;`sense-audit-relay-v1.openapi.json` 已由 T-016 实现。默认 SQLite 只暴露运维探针与低基数 `/metrics`,不注册业务路由或 relay;Bell 管理服务与 JWT/OIDC 尚未实现。
- T-012 把 PostgreSQL schema 提升到 v5:due row 用数据库时钟、`FOR UPDATE SKIP LOCKED`、逐项续租和 fencing token 协调;MediaMTX Path 历史归属、15 分钟孤儿快照、最多 10%/128 项安全闸、无 bypass 的本地处置命令及 `/metrics` 已实现。SQLite 明确保留单实例开发语义。
- harness coding 文档、上下文清单、Gitea Issue/PR 模板和治理脚本已接入。
- Gitea 已初始化 12 个协作标签;`status/waiting` 用于依赖或外部条件未满足的未领取任务,实时可领取状态必须从 Gitea 查询,不在本文复制。
@@ -42,6 +45,14 @@ Windows:
go -C Sense run ./cmd/sense-api
```
Bell 事件域和内部审计 receiver 验证:
```powershell
go -C Bell test ./...
go -C Bell vet ./...
go -C Bell build ./...
```
跨平台直接验证:
```powershell
@@ -56,6 +67,15 @@ python scripts/validate_harness_governance.py
./scripts/test_postgres.ps1 -PgRoot D:\pgsql17
```
本机 16 路软件容量预检与正式验证:
```powershell
./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -PreflightOnly
./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -OutputPath (Join-Path $env:TEMP 'yovision-t014-formal.json')
```
只有默认 30 分钟且输出 `formal_eligible=true` 的单次完整运行可作正式证据;`-ObservationMinutes 1` 只用于 smoke。
Sense 默认监听 `127.0.0.1:8080`,提供 `/healthz`、`/readyz` 运维探针和低基数 `/metrics`;探针不代表摄像头或 M1 里程碑健康。MediaMTX 获取、校验和独立启动方法见 `Sense/README.md`。
## 当前 blocker / 待确认
@@ -67,12 +87,12 @@ Sense 默认监听 `127.0.0.1:8080`,提供 `/healthz`、`/readyz` 运维探针
- 人脸方向已延后至 M5 的 S4 成人园区候选试点;必要性/PIP 影响评估、单独同意与替代方式、合法底库来源和删除流程未完成,阻塞人脸能力上线。
- 短信/语音具体供应商未选;生产前必须选定两条独立投递路径并验证故障切换。
- Python/Savant 的精确版本、目标硬件和 Bell 前端栈尚未冻结;Sense M1 的 Go、SQLite driver、MediaMTX、生成器及生成运行时版本已在 T-003 冻结,PostgreSQL/pgx 版本已在 T-009 冻结。
- 本机现有 PostgreSQL 5432 实例使用 SCRAM 且当前开发进程没有管理员密码;T-009~T-012 不绕过认证,自动验收使用隔离临时集群。向共享/生产实例安装 migration 前仍需管理员私下提供专用数据库、登录角色、外部 Control API 安全文件与备份方案。
- 本机现有 PostgreSQL 5432 实例使用 SCRAM 且当前开发进程没有管理员密码;T-009~T-016 不绕过认证,自动验收使用隔离临时集群。向共享/生产实例安装 migration 前仍需管理员私下提供专用数据库、最小权限登录角色、外部 Control API/relay key 文件、TLS 证书与备份方案。
- 代码知识图谱在无业务代码阶段可能为空;工具不可用时使用 `rg` 处理文档与配置。
## 下一步
下一项建议先冻结并实施 WireGuard 边缘隧道与断网恢复,再创建 16 路批量开通/停用和分维度容量基准任务;T-012 完成不等于 M2 容量出口。客户授权、借用或租赁条件具备后再执行 T-007 五条独立真实上游现场门禁。T-006 的合成结果不解除 T-007,也不形成容量或生产 SLA 承诺。
客户网络仍未提供,T-013 WireGuard 继续后置。T-015/T-016 已分别落地 Bell 不可变事件存储和 Sense 全局审计 relay;下一项应独立冻结 Brain→Bell 业务事件 ingress,不把规则/Alert 或公共管理 API 混入。客户授权、借用或租赁条件具备后再执行 T-007 五条独立真实上游现场门禁。T-014~T-016 不解除 T-007/T-013,也不形成真实多路或生产 SLA 承诺。
## 已知风险
+1 -1
View File
@@ -85,7 +85,7 @@ mapper 的完整实现规格。左列出处见《07》§1。
已实测(`REJECT` = schema 能拦,`ACCEPT` = 拦不住):
schema **能**拦:省略必填键 / 非法 ULID / 顶层多余字段 / outcome 越界 / `config_version` 空串 / `snapshot_uris` 空数组 / 负 latency / kind 大写。
schema **能**拦:省略必填键 / 非法 ULID / 顶层多余字段 / outcome 越界 / `config_version` 空串 / 负 latency / kind 大写。`snapshot_uris` 空数组对非成像事件是合法值,见修订记录、雷达示例和 schema 的 `minItems: 0`。
schema **拦不住**,需在 mapper 与平台入口各加一道断言:
+109
View File
@@ -0,0 +1,109 @@
# Sense 本地 16 路容量与批量收敛基线
> T-014 正式证据,运行日期:2026-08-10。本文只证明固定主机、固定版本和低码率合成负载下的本地软件基线,不是生产 SLA。
## 1. 结论
在 24 个逻辑处理器、15.78 GiB 内存的 Windows 主机上,提交 `d029067aa750ad75a2d79ef8e6c51b6bad55096d` 使用隔离 PostgreSQL、真实 Sense Control API、两套独立 MediaMTX 和 16 个独立 FFmpeg publisher 完成正式验收:
- 16 个 disabled 视频设备创建后,三轮 16 项批量操作全部成功,生产 MediaMTX 配置 Path 数按 `16 → 0 → 16` 收敛。
- 16 路 enabled 后,第 17 路 enabled 创建稳定返回 `quota_exceeded`,设备台账仍为 16 项;disabled 项不冒充已占用视频通道。
- 同时停止固定第 5~8 路后,精确 4 路受影响、其余 12 路保持在线;恢复后 5.1 秒回到 16 路在线和 `unconverged=0`。
- 正式稳定窗口为 1800.1 秒,每 10 秒绝对节拍采样,共 180 个样本;最大与最终未收敛数均为 0,最终在线 Path 为 16,入站帧错误增量为 0。
因此,默认 16 路的 Sense/PostgreSQL/MediaMTX 控制面与拉流闭环具备可重复的本地软件基线。该结论不证明 16 台真实摄像头、客户网络、录像、下游观看、AI 解码/推理、GPU、64/128 路分片或生产 SLA,也不解除 T-007/T-013。
## 2. 固定环境与负载
| 项目 | 正式值 |
| --- | --- |
| 仓库提交 | `d029067aa750ad75a2d79ef8e6c51b6bad55096d` |
| Sense 二进制 SHA-256 | `e6fa01991152cf7e9f2a1e420422a12d474dfe7be10530ff2924671d46cc7871` |
| Go | `go1.26.5 windows/amd64`(在 `Sense/` 模块上下文读取) |
| PostgreSQL | `17.10`,单次运行隔离临时集群 |
| MediaMTX | `v1.19.3` |
| MediaMTX Windows amd64 ZIP SHA-256 | `5d82148d1032a6a190d9909a2997d9989457aaadf49af87dd02cd4512d31bebe` |
| MediaMTX EXE SHA-256 | `1cda85249312cb9463f9f94c5a712b9f160c9af3fd9490f0d4723911d7880e05` |
| FFmpeg | `8.1.2-full_build-www.gyan.dev` |
| 主机 | Windows,24 logical processors,15.78 GiB memory |
| 单路夹具 | 640×360、10 fps、H.264、无音频、无人物 |
| 发布方式 | 16 个独立 FFmpeg 进程,各自循环预编码夹具并以 `-c copy` 发布独立 RTSP Path |
预编码 copy 发布是为了不把 16 路软件编码负载混入 Sense/MediaMTX 基线。它仍产生 16 个可独立停止和恢复的发布进程,但不能代表真实摄像头编码器、复杂 GOP、高码率、音频或公网抖动。
## 3. 运行方法与安全边界
从仓库根目录执行:
```powershell
./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -PreflightOnly
./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -OutputPath (Join-Path $env:TEMP 'yovision-t014-formal-final.json')
```
调试可显式传入 `-ObservationMinutes 1`,但输出固定标记 `formal_eligible=false`,不能作为正式验收。
脚本在系统临时目录生成媒体、Sense 二进制、凭据文件和 PGDATA,只绑定随机回环端口;migration 读取 `001`~`011` 并重复回放,不读取或修改 `D:\pgsql17\data`。结果 JSON 不包含 token、DSN、临时端口、设备 ID、Path/source URI 或凭据引用。正式退出后复查 session、FFmpeg、MediaMTX 和 Sense 数量均为 0,本机现有 5432 listener 前后未变化。
## 4. 功能与时序结果
| 门禁 | 正式结果 |
| --- | ---: |
| 16 个独立 publisher 就绪 | 7.6 s |
| 首轮 enable batch | `succeeded`,16 项,请求 0.041 s |
| 首轮 enable 收敛 | 3.1 s |
| 第 17 路 enabled | 拒绝,`quota_exceeded` |
| disable batch | `succeeded`,16 项,请求 0.015 s |
| disable 收敛 | 1.0 s |
| 第二轮 enable batch | `succeeded`,16 项,请求 0.046 s |
| 第二轮 enable 收敛 | 3.1 s |
| 配置 Path | `16 → 0 → 16` |
| 四路故障检测 | 1.0 s,影响 4 路,其余 12 路在线 |
| 四路恢复 | 5.1 s,最终未收敛 0 |
批量启停使用真实 `/api/v1/sites/{site_id}/devices:batchDesiredState`,每轮提交前重新读取各设备最新 ETag;不能用创建响应中的旧 ETag 与后台调和竞速。每轮 operation 必须为 `succeeded` 且 16 个逐项结果全部成功。
## 5. 30 分钟稳定性与资源观测
| 指标 | 正式结果 |
| --- | ---: |
| 观察时长 | 1800.1 s |
| 采样节拍 / 样本 | 10 s / 180 |
| 最大未收敛数 | 0 |
| 最终未收敛数 | 0 |
| 最终在线 Path | 16 |
| 聚合入站码率 | 15.443 Mbps |
| 入站帧错误增量 | 0 |
| Sense CPU 总时间 | 5.594 s |
| Sense working set 峰值 | 69.99 MiB |
| Sense private bytes 峰值 | 57.58 MiB |
| Sense handle 峰值 | 228 |
| 生产 MediaMTX CPU 总时间 | 26.812 s |
| 生产 MediaMTX working set 峰值 | 45.27 MiB |
| 生产 MediaMTX private bytes 峰值 | 79.23 MiB |
| 生产 MediaMTX handle 峰值 | 295 |
| PostgreSQL 业务库连接峰值 | 3 |
| PostgreSQL 数据库大小 | 8.90 MiB |
归一化 CPU 平均值与峰值在 24 逻辑处理器主机上四舍五入到三位小数后为 0.000%,因此报告保留进程 CPU 总秒数作为低负载证据;不能据此推导通用硬件下限。
## 6. 失败记录与工具修正
正式结论只取最后一次完整成功运行。研发过程中保留以下失败,不拼接为成功证据:
1. MediaMTX 发布 ZIP 与解压后 EXE 的 SHA-256 不同,脚本分别冻结并校验两者。
2. disabled 设备不消耗视频通道;第 17 路门禁改为在 16 路 enabled 后创建 enabled 设备,符合实际配额语义。
3. PowerShell 读取 `application/problem+json` 时可能得到 byte array;脚本显式按 UTF-8 解码后校验稳定错误码。
4. 把 `pg_ctl start` 接入 PowerShell 输出管道会让 PostgreSQL 继承管道句柄并阻塞;启动改为直接执行并检查退出码。
5. Windows 退出时 `sense-api.exe` 曾短暂持有文件锁;清理改为共享截止时间内终止全部进程、完成异步输出读取、释放 `Process` 句柄并重试删除。
6. “固定睡 10 秒再查询”会把查询耗时累计进采样间隔,30 分钟不足 180 个样本;改为按绝对时间点调度第 1~180 个样本。
7. 根目录 Go launcher 为 1.23.0,但 `Sense/go.mod` 冻结并实际构建使用 1.26.5;版本证据改为在 `Sense/` 模块上下文读取。
8. 首轮 batch 曾复用创建响应 ETag,与后台调和的资源版本更新竞速;现在三轮 batch 都在提交前读取最新 ETag,失败诊断只输出状态/错误码计数。
每次失败都没有生成正式成功 JSON,且相关临时 PostgreSQL、媒体和 Sense 进程已停止,单次 session 已清理。
## 7. 后续验收
- T-007 仍需至少 5 条独立真实摄像头上游与现场网络,才能形成真实多路故障隔离和生产试点证据。
- T-013 等客户网络拓扑、地址规划和部署权限具备后,再验证 WireGuard 与断网补传。
- 64/128 路必须在后续任务中分别验证媒体分片、带宽、解码、AI/GPU、证据存储和单分片故障域;本报告不外推这些结论。
- 进入 M3 前应优先冻结并实现 Bell 事件不可变存储,以及 Sense Outbox 到 Bell 全局审计的 relay 边界。
+38 -7
View File
@@ -3,12 +3,12 @@ id: T-014
title: 建立 Sense 本地 16 路批量收敛与容量基线
phase: 2
deps: [T-012]
status: TODO
status: DONE
created: 2026-08-08
issue: 47
context_ref: null
claim_branch: null
work_branch: null
context_ref: 8ebb8ced492c614106e766192c59b7f13fb4ad0f
claim_branch: claims/T-014
work_branch: agent/codex/T-014
write_paths:
- docs/tasks/T-014.md
- Sense/README.md
@@ -38,7 +38,7 @@ T-006 已证明一台准入实机与四条独立合成源可以完成五路实
1. 新增 `Sense/scripts/t014-capacity.ps1` 一键基准:校验冻结的 PostgreSQL 17.10、MediaMTX v1.19.3、FFmpeg 与 Go,使用随机回环端口和系统临时目录启动隔离 PostgreSQL、源 MediaMTX、生产 MediaMTX 与 Sense。migration 只读取并重放现有 `001`~`011`,不修改或连接本机已有 `D:\pgsql17\data`/5432。
2. 先用 FFmpeg 生成 640×360、10 fps、H.264、无音频、无人物的短循环媒体夹具,再启动 16 个独立 FFmpeg `-c copy` RTSP publisher,每路有独立进程与 Path,可单独停止/恢复。预编码发布避免把 16 路软件编码开销计入 Sense/MediaMTX 基线;不把同一 publisher fan-out 冒充独立源。
3. 在临时 PostgreSQL 中建立固定实验室 Tenant/Site/Area,Site 配额精确为 16;运行时生成 Control API opaque token、SHA-256 注册表和 cursor key,文件只存在于临时目录。通过真实 HTTP API 创建 16 个 disabled 视频设备,并验证第 17 个同站点设备稳定返回 `quota_exceeded` 且不写入台账。
3. 在临时 PostgreSQL 中建立固定实验室 Tenant/Site/Area,Site 配额精确为 16;运行时生成 Control API opaque token、SHA-256 注册表和 cursor key,文件只存在于临时目录。通过真实 HTTP API 创建 16 个 disabled 视频设备、批量启用并收敛后,验证第 17 个 enabled 同站点设备稳定返回 `quota_exceeded` 且不写入台账;disabled 台账项不冒充已占用通道。
4. 使用真实 `devices:batchDesiredState` 和每台设备最新 ETag 执行 16 路批量启用、批量停用、再次启用;每次要求 operation 的 16 个结果全部成功。分别验证生产 MediaMTX 配置/运行 Path 为 16、0、16,设备最终状态与 generation/observed generation 收敛,`unconverged=0`。
5. 在再次启用后同时停止固定四个 publisher,要求能观察到局部离线/未收敛且其余路径继续在线;恢复四个独立进程后自动回到 16 路在线和 `unconverged=0`。不得通过改数据库、人工编辑 MediaMTX 配置或重建进程集合掩盖失败。
6. 正式稳定观察默认 30 分钟、每 10 秒采样。记录 Sense 与生产 MediaMTX 的归一化 CPU、working set、private bytes、handle 峰值,PostgreSQL连接峰值,以及 MediaMTX aggregate inbound bytes/吞吐、在线 Path、Sense 低基数指标和未收敛数。功能正确性有硬门禁;机器相关资源值只记录,不设置伪通用阈值。
@@ -47,14 +47,14 @@ T-006 已证明一台准入实机与四条独立合成源可以完成五路实
## 不可变约束
- 阈值 / 数值边界:正式源数精确为 16;Site 默认配额 16、允许范围 1~128,17 路在该 Site 必须拒绝,129 仍非法;批量 API 上限保持 128。正式观察不少于 30 分钟、采样周期 10 秒;局部故障固定 4 路且不得影响其余 12 路。短窗口只能 smoke。
- 阈值 / 数值边界:正式源数精确为 16;Site 默认配额 16、允许范围 1~128,已有 16 路 enabled 时第 17 路 enabled 必须拒绝,129 仍非法;批量 API 上限保持 128。正式观察不少于 30 分钟、采样周期 10 秒;局部故障固定 4 路且不得影响其余 12 路。短窗口只能 smoke。
- 判定式 / 状态转换:16 个设备先 disabled 创建,随后 `16 enabled → 16 disabled → 16 enabled`;每轮批量结果必须全部成功。启用完成必须配置 Path=16、运行在线 Path=16、`unconverged=0`;停用完成必须配置 Path=0、设备 offline 且各自 observed generation 追平。四路故障恢复后必须再次满足 16 路在线和 `unconverged=0`。
- 安全边界:只绑定随机回环端口;运行时 token/cursor key/DSN/临时路径不进入 Git、普通日志或结果摘要。脚本不得读取 `ip_camera.env`,不得访问客户网络或现有 PostgreSQL data/service;递归清理前必须验证目标位于系统临时目录且名称含 T-014 随机前缀。
- 既有契约:不修改 Control OpenAPI v1、审计 v1/v2、MediaMTX 生成代码或数据库 migration;不把 16 写入业务数组/协议硬上限。T-001/T-006 的实机结论、T-007 真实五上游门禁与未来 T-013 WireGuard 均保持独立。
## 验收要点
- 任务相关验证:先以短窗口运行脚本 smoke,再执行默认不少于 30 分钟的正式 16 路基准;验证 16 个独立 publisher、17 路配额拒绝、三轮批量状态、16/0/16 Path、四路故障隔离/恢复、最终 `unconverged=0` 和资源摘要。`python -m unittest tests.test_sense_capacity_contract` 锁定脚本安全/语义。
- 任务相关验证:先以短窗口运行脚本 smoke,再执行默认不少于 30 分钟的正式 16 路基准;验证 16 个独立 publisher、第 17 路 enabled 配额拒绝、三轮批量状态、16/0/16 Path、四路故障隔离/恢复、最终 `unconverged=0` 和资源摘要。`python -m unittest discover -s tests -p "test_sense_capacity_contract.py"` 锁定脚本安全/语义。
- 完整门禁:`./init.ps1`、三条 Python 治理命令、`go -C Sense test ./...`、`go -C Sense vet ./...`、`go -C Sense build ./...`、`./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` 和 `git diff --check` 全部通过;本任务不修改 Go 业务代码,但仍回归全部 Sense 包和 PostgreSQL v5。
- 人工 / 设备验收:不需要摄像头、GPU、客户网络或客户数据。任务所有者核对 30 分钟原始 JSON 摘要、进程独立性和报告;真实摄像头/网络验收仍由 T-007/T-013 完成。
- 构建产物:`Sense/scripts/t014-capacity.ps1`、`docs/research/sense-16-stream-capacity.md`、静态契约测试与同步文档;临时二进制、媒体、数据库、凭据和日志运行后删除,不提交。
@@ -74,6 +74,37 @@ T-006 已证明一台准入实机与四条独立合成源可以完成五路实
## 执行记录
### 2026-08-10 正式 16 路基线
- 在提交 `d029067aa750ad75a2d79ef8e6c51b6bad55096d` 执行 `./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -OutputPath (Join-Path $env:TEMP 'yovision-t014-formal-final.json')`,成功输出 `formal_eligible=true`。固定环境为 Go 1.26.5、PostgreSQL 17.10、MediaMTX v1.19.3、FFmpeg 8.1.2;主机为 24 logical processors / 15.78 GiB。
- 16 个独立 publisher 在 `7.6 s` 就绪;三轮 16 项 batch 均 `succeeded`,请求分别为 `0.041 s / 0.015 s / 0.046 s`,收敛分别为 `3.1 s / 1.0 s / 3.1 s`,配置 Path 为 `16 → 0 → 16`。16 路 enabled 后第 17 路 enabled 返回 `quota_exceeded`。
- 固定停止第 5~8 路后 `1.0 s` 检测到精确 4 路受影响,其余 12 路在线;恢复 `5.1 s` 后回到 16 路在线。稳定窗口 `1800.1 s`、10 秒绝对节拍、180 个样本,最大/最终 `unconverged=0`,聚合入站 `15.443 Mbps`,帧错误 0。
- 资源记录:Sense CPU 总计 `5.594 s`、working set 峰值 `69.99 MiB`、private 峰值 `57.58 MiB`、handle 峰值 228;生产 MediaMTX 分别为 `26.812 s / 45.27 MiB / 79.23 MiB / 295`;PostgreSQL 连接峰值 3,数据库 `8.90 MiB`。详细边界与失败记录见 `docs/research/sense-16-stream-capacity.md`。
- 正式退出后复查 T-014 session、FFmpeg、MediaMTX 和 Sense 数量均为 0;结果不含 token、DSN、端口、设备 ID、Path/source URI 或客户信息。本机现有 5432 listener 未被停止或修改。
### 2026-08-10 完整门禁
- PowerShell parser 与 `python -m unittest discover -s tests -p "test_sense_capacity_contract.py" -v` 通过,T-014 静态契约 6/6 通过。
- `./init.ps1` 通过:58 个 Python 测试成功,MediaMTX/Control API 生成漂移检查、全部 Sense Go 包测试、`go vet` 与 `go build` 均成功。
- `./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` 通过:PostgreSQL 17.10 隔离集群重放 `001`~`011`,权限断言与 `yovision/sense/internal/store` 真实 PostgreSQL 测试成功,临时端口和 PGDATA 已清理。
- `python scripts/validate_agent_context.py`、`python -m unittest discover -s tests -p "test_*.py"`(58 项)、`python scripts/validate_harness_governance.py` 与 `git diff --check` 均通过。生成文件只有现有 LF→CRLF 提示,没有内容漂移或任务范围外修改。
### 2026-08-08~2026-08-10 基准工具失败与修正
- 正式结论没有拼接失败样本。依次修复并回归:MediaMTX ZIP/EXE 双指纹、disabled 不占配额、problem+json byte array 解码、`pg_ctl start` 管道句柄继承、Windows EXE 文件锁与进程句柄释放、固定 sleep 导致 30 分钟不足 180 样本、根目录 Go launcher 与 Sense 1.26.5 工具链口径不一致,以及首轮 batch 复用创建 ETag 的调和竞速。
- 中断或失败运行均未输出正式成功 JSON;已核验并停止临时 PostgreSQL,清除精确的单次 session。最终脚本按绝对 10 秒节拍取得 180 个样本,三轮 batch 提交前都读取最新 ETag,清理使用共享截止时间并释放异步输出/进程句柄。
### 2026-08-08 一分钟端到端 smoke
- `./Sense/scripts/t014-capacity.ps1 -PgRoot D:\pgsql17 -ObservationMinutes 1` 通过并输出 `formal_eligible=false`:16 个独立 publisher 全部在线,三轮 16 项 batch 均 `succeeded`,配置 Path 为 `16 → 0 → 16`,第 17 路 enabled 返回 `quota_exceeded` 且台账仍为 16 项。
- 首次启用、停用、再次启用分别在 `3.1 s`、`1.0 s`、`3.1 s` 收敛;同时停止四路后 `1.0 s` 观察到精确四路受影响、其余 12 路在线,恢复 `5.2 s`。稳定观察 `60.8 s`/6 次采样,最大与最终 `unconverged` 均为 0,最终在线 Path 16,aggregate inbound `15.432 Mbps`,帧错误 0。
- 保留三类脚本修正记录:发布 ZIP 与 EXE 需要不同 SHA-256;disabled 台账不消耗视频运行配额,故第 17 路门禁必须在 16 路 enabled 后验证;PowerShell 对 `application/problem+json` 返回 byte array,需显式 UTF-8 解码后校验稳定 `code`。每次失败均停止并清理临时 PostgreSQL/进程,未拼接成功片段。
### 2026-08-08 领取任务
- dispatcher `ila` 将 Issue #47 分配给 `codex`;`context_ref` 为 `8ebb8ced492c614106e766192c59b7f13fb4ad0f`,claim 为 `claims/T-014`,工作分支为 `agent/codex/T-014`。
- 已读回 Issue `status/doing`、assignee、dispatcher 发布的完整 CLAIM 与两个分支 SHA;接受 frontmatter 全部写路径。T-007 继续 waiting,本任务期间不恢复,当前无活跃写路径冲突。
### 2026-08-08 Gitea 映射
- 任务规格通过 PR #46 合入默认分支后创建唯一主 Issue #47;本次只回填双向映射,映射合入并读回前不领取任务。
+92
View File
@@ -0,0 +1,92 @@
---
id: T-015
title: 建立 Bell 事件 v0.1 校验与不可变 PostgreSQL 存储
phase: 3
deps: [T-014]
status: DONE
created: 2026-08-10
issue: 51
context_ref: 5b42f58f23255831067f44206849344909d74ee9
claim_branch: claims/T-015
work_branch: agent/codex/T-015
write_paths:
- docs/tasks/T-015.md
- Bell/
- deploy/postgres/012_bell_events.sql
- deploy/postgres/013_privileges_bell_events.sql
- deploy/postgres/tests/assertions.sql
- deploy/postgres/README.md
- scripts/test_postgres.ps1
- tests/test_bell_event_contract.py
- tests/test_postgres_contract.py
- docs/raw/contracts/README.md
- docs/00-ai-start-here.md
- docs/03-tech-stack.md
- docs/04-architecture.md
- docs/06-tasks.md
- docs/api.md
- docs/current-state.md
- AGENTS.md
- init.ps1
- init.sh
---
## 问题 / 背景
M3 要求 Bell 首次真实消费冻结的事件契约 v0.1,生成平台 ULID,并把事件事实保存为不可变记录。当前 Bell 只有目录占位,PostgreSQL 也只有 Site/Area 投影源表;如果先做规则或 Alert,会把未经校验、可被覆盖的事件数据变成后续状态机的错误基础。
冻结契约中还存在一处文档自相矛盾:`docs/raw/contracts/README.md` 的修订记录、雷达示例和 JSON Schema 均允许非成像事件的 `snapshot_uris=[]`,但 §5 的旧句仍把空数组列为 schema 可拒绝项。本任务先把该句校正为现有冻结事实,不修改 schema 或字段语义。
## 关联需求与交互(如适用)
- 用户故事:无直接 UI;为 US-003、US-004、US-006 的后续事件/处置流程提供数据基础。
- 交互清单:不适用;本任务不实现 Bell Web/H5 页面或公共查询 API。
- 相关页面 / 路由:不适用。
## 方案
1. 在 `Bell/` 建立独立 Go module,复用项目冻结的 Go 1.26.5、PostgreSQL 17.10 和 `pgx/v5 v5.10.0`;冻结 Draft 2020-12 校验器与 ULID 库版本、许可证和退出路线,不引入前端或消息总线。
2. 把冻结 schema 逐字节复制到 `Bell/contracts/event-v0.1.schema.json` 并以测试阻止漂移。Bell 内部 ingest factory 接收不含平台 `id` 的候选事实,拒绝上游自报 `id`,由 Bell 生成 `evt_` ULID 后再执行 schema 与代码级断言。Brain→Bell 的 HTTP/消息总线 transport 继续待定,本任务不借内部 Go 类型冻结公共网络协议。
3. 代码级校验覆盖时间顺序/延迟自洽、`confidence=null`、证据 URI 脱敏、唯一 primary 且与顶层设备一致,以及视频隐私准入。隐私与客户敏感名称通过必需 port 注入;解析/映射不可用时失败关闭,不使用 allow-all 生产默认值。
4. 新增 Bell event repository。`bell.events` 保存最终规范化 JSON、摘要和可查询的最小索引字段;平台 ID 冲突时仅允许“同 ID + 同摘要”的幂等重放,不同摘要稳定冲突。后续 outcome 写入独立的 append-only `bell.event_outcomes`,不得更新原事件事实。
5. migration 增加独立 NOLOGIN `bell_runtime` 最小权限角色。migration owner `bell_app` 持有对象;运行时只获得必需的 `SELECT/INSERT`,明确没有事件表的 `UPDATE/DELETE/TRUNCATE`,并以数据库 trigger 拒绝事实改写。生产/共享实例仍由管理员离线 migration,不由 Bell 高权限自迁移。
6. 扩展隔离 PostgreSQL 17.10 harness,使用 `bell_runtime` 登录运行真实 repository/权限/重放/不可变性集成测试;同步根启动入口、技术栈、架构、API 状态和当前仓库现实。
## 不可变约束
- 阈值 / 数值边界:请求体/单事件规范化 JSON 上限 1 MiB;延迟自洽误差严格小于 100 ms;事件列表/公共 API 不在本任务内;任何默认 16 路值不成为事件数量上限。
- 判定式 / 状态转换:Bell 是 `evt_` ULID 唯一生成者;候选事实不得携带 `id`;最终事实必须同时通过 Draft 2020-12 schema 和 README 六项代码级断言。相同平台 ID/相同摘要为幂等成功,相同 ID/不同摘要为冲突;原事件永不 UPDATE/DELETE,后续 outcome 只能追加。
- 安全边界:未知顶层字段拒绝;视频隐私映射或敏感名称策略不可用时失败关闭;普通错误和测试证据不回显事件全文、证据 URI、客户名、凭据或内部堆栈;`bell_runtime` 不拥有 migration 对象且没有事件事实改写权限。
- 既有契约:`docs/raw/contracts/event-v0.1.schema.json` 字段、枚举与 v0.1 语义不变;只修正文档中与 schema/示例冲突的旧句。Brain transport、规则、Alert、证据回捞、RBAC/JWT/OIDC 和事件公共 API 均另立任务。
## 验收要点
- 任务相关验证:`go -C Bell test ./...`、`go -C Bell vet ./...`、`go -C Bell build ./...`;`python -m unittest discover -s tests -p "test_bell_event_contract.py" -v`;验证三个冻结示例、负向 schema/六断言、ULID 所有权、幂等冲突和 append-only outcome。
- 完整门禁:因命中 Bell 脚手架、PostgreSQL migration、权限、根启动入口和冻结契约消费者,运行 `./scripts/test_postgres.ps1 -PgRoot D:\pgsql17`、`./init.ps1`、三条 Python 治理命令和 `git diff --check`。隔离测试必须证明现有 5432 listener 前后不变并自动清理临时集群。
- 人工 / 设备验收:不适用;本任务不依赖摄像头、客户网络、GPU、MinIO 或产品 UI。生产上线仍受法务/客户留存政策和 M3 现场 dry-run 门禁约束。
- 构建产物:`go -C Bell build ./...` 可构建全部 Bell 包;不生成可部署 Bell API 二进制,因为 Brain transport、认证和公共 API 尚未冻结。
## 边界(不改什么)
- 不实现 Brain mapper/投递、HTTP 或消息总线事件入口、规则引擎、Alert、ack、升级链、对象存储、证据切片、反馈回流、Bell Web/H5 或 JWT/OIDC。
- 不修改冻结事件 schema、示例字段或 Sense 业务代码;不把整数事件契约 ID 与现有文本逻辑 ID 的待定映射悄悄固化为跨系统规则。
- 不把 `bell_app` owner 凭据交给运行进程,不在仓库保存 DSN、密钥、客户数据、真实事件或真实证据 URI。
## 协作约束
- 责任 Agent:codex
- 唯一写入者:codex
- 委派:不启用。
- Gitea:Issue、`context_ref`、claim 与工作分支在双向映射及 dispatcher 分配后回填。
任何新增写路径先检查与其他活跃任务是否重叠;同一时刻只有一个 Agent 修改本任务的 `write_paths`。
## 执行记录
- 2026-08-10:按 M3 建议拆出本任务;完成仓库/远端状态检查和 `./init.ps1` 基线,现有 58 个 Python 测试及 Sense generate/test/vet/build 全部通过。实现尚未开始。
- 2026-08-10:冻结 Bell Go 1.26.5、`jsonschema/v6 v6.0.2`、`ulid/v2 v2.1.2`,复用 PostgreSQL 17.10 / pgx v5.10.0;建立 candidate→Bell ULID→最终 v0.1 schema/语义校验。三个冻结示例均通过,负向测试覆盖上游自报 ID、未知字段、时间/延迟、confidence、primary、证据脱敏、隐私拒绝/不可用和 1 MiB 上限。
- 2026-08-10:新增 `bell.events` 与 `bell.event_outcomes`、不可变 trigger 和独立 NOLOGIN `bell_runtime`。repository 验证同 ID/同摘要幂等、同 ID/不同摘要冲突、outcome 仅追加,以及运行登录无 `UPDATE/DELETE/TRUNCATE`。校正事件 README 中与 schema/雷达示例冲突的“空 snapshot 数组会被拒绝”旧句,未修改冻结 schema。
- 2026-08-10:`go -C Bell test ./...`、`go -C Bell vet ./...`、`go -C Bell build ./...` 通过;`python -m unittest discover -s tests -p "test_bell_event_contract.py" -v` 5/5 通过,`test_postgres_contract.py` 8/8 通过。
- 2026-08-10:`./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` 通过;PostgreSQL 17.10 临时集群将 `001`~`013` migration 重放两次,Sense/Bell 真实 repository 与权限断言通过,随机端口、临时目录均清理,现有 5432 listener 未改变。
- 2026-08-10:更新根 `init.ps1`/`init.sh` 后运行 `./init.ps1` 通过:64 个 Python 测试、Sense generate/test/vet/build 和 Bell test/vet/build 全部成功。人工/设备/UI 验收不适用。
- 边界:本任务没有可部署 Bell API;Brain transport、认证、整数事件 ID 与文本逻辑 ID 映射及生产隐私 resolver 仍待后续任务,内部 Go factory 不能被表述为已上线跨系统入口。
+100
View File
@@ -0,0 +1,100 @@
---
id: T-016
title: 冻结并实现 Sense Outbox 到 Bell 全局审计幂等 relay
phase: 3
deps: [T-015]
status: DONE
created: 2026-08-10
issue: 55
context_ref: aca22f4667f0da77a102eb7ba15a93c0128df656
claim_branch: claims/T-016
work_branch: agent/codex/T-016
write_paths:
- docs/tasks/T-016.md
- docs/contracts/README.md
- docs/contracts/sense-audit-relay-v1.openapi.json
- Sense/cmd/sense-api/main.go
- Sense/internal/config/config.go
- Sense/internal/config/config_test.go
- Sense/internal/auditrelay/
- Sense/internal/store/audit_relay_postgres.go
- Sense/internal/store/audit_relay_postgres_test.go
- Sense/README.md
- Bell/cmd/bell-api/
- Bell/internal/audit/
- Bell/internal/store/audit_postgres.go
- Bell/internal/store/audit_postgres_test.go
- Bell/README.md
- deploy/postgres/014_audit_relay.sql
- deploy/postgres/015_privileges_audit_relay.sql
- deploy/postgres/tests/assertions.sql
- deploy/postgres/README.md
- scripts/test_postgres.ps1
- tests/test_sense_audit_relay_contract.py
- tests/test_postgres_contract.py
- docs/00-ai-start-here.md
- docs/03-tech-stack.md
- docs/04-architecture.md
- docs/06-tasks.md
- docs/api.md
- docs/current-state.md
---
## 问题 / 背景
T-010 已保证 Sense 设备创建、配置和期望态变更与脱敏 `sense.device_operation_outbox` 同事务,但 Outbox 仍不会离开 Sense;Bell 因而没有完整的跨系统全局审计事实。直接让 Sense 写 `bell` schema 会破坏三系统所有权,简单“POST 后标记成功”又无法处理崩溃重试、重复投递、多实例竞争、消息篡改或重放攻击。
T-015 已提供 Bell 的独立运行角色和不可变事实模式。本任务只冻结并实现设备操作审计 relay,不顺带定义 Brain 事件 transport、业务规则、Alert 或公共管理 API。
## 关联需求与交互(如适用)
- 用户故事:为 US-009、US-010 的全局审计追溯提供后端事实基础。
- 交互清单:IX-020 的审计归属后端基础;本任务不实现 `/audit` UI、筛选或深链页面。
- 相关页面 / 路由:公共 UI 不适用;新增仅内部使用的 `/internal/v1/audit-events:batch`。
## 方案
1. 冻结 `sense-audit-relay-v1.openapi.json`:Sense 通过 HTTP POST 批量发送 1~100 个 v1/v2 设备审计事实,Bell 返回与输入逐项对应的 `accepted | duplicate | rejected`;完整批次响应可按签名 nonce 幂等重放。
2. 请求使用 key ID、Unix 秒时间戳、随机 nonce 与 HMAC-SHA256。canonical string 固定为 `METHOD + path + timestamp + nonce + SHA256(body)` 的换行拼接;允许时钟偏差 5 分钟,nonce 收据保留 10 分钟。相同 key/nonce/请求摘要返回原响应,不同摘要返回 `409 replay_conflict`。密钥只从仓库外绝对路径文件读取,至少 32 个随机字节,不进数据库、日志或 Issue。
3. 非回环 transport 必须使用 HTTPS;回环 HTTP 只用于同机私有部署和测试,不提供跳过远端 TLS 的 flag。单次请求体最大 1 MiB、超时 10 秒,Bell 不回显原始 body、签名或敏感字段。
4. PostgreSQL v6 为 Sense Outbox 增加数据库时钟 lease、fencing token、last error 和 dead-letter 状态。最多 100 行用 `FOR UPDATE SKIP LOCKED` 领取;过期 worker 不能确认。`accepted/duplicate` 标记 delivered,逐项永久拒绝进入 dead letter,网络/5xx/整批认证失败按指数退避重试且不删除事实。
5. Bell 新增不可变 `bell.audit_events` 和可过期 `bell.audit_relay_receipts`。全局审计用 `(source_system,event_id)` 唯一;相同摘要为 duplicate,不同摘要逐项 `id_conflict`。审计事实没有 UPDATE/DELETE/TRUNCATE 权限并复用不可变 trigger;收据允许 Bell runtime 在限定表内维护过期记录。
6. 建立最小 `cmd/bell-api`,默认仅回环监听,暴露 health/ready 与内部 relay endpoint;数据库、监听地址、TLS 和 key 文件均来自私有环境。Sense relay 默认关闭,仅在 PostgreSQL v6、合法 Bell URL 和外部 key 文件齐备时启动后台 worker。
7. 单元测试覆盖签名向量、时间窗/nonce、body 限制、逐项确认、重复/冲突、配置失败关闭、退避和崩溃重领;隔离 PostgreSQL 17.10 测试覆盖 migration 重放、两端 repository、fencing、权限和安全清理。
## 不可变约束
- 阈值 / 数值边界:每批 1~100 项;body ≤1 MiB;HTTP deadline 10 秒;lease 30 秒;签名时钟偏差 ≤300 秒;nonce 收据 TTL 600 秒;退避从 1 秒指数增长并封顶 300 秒。16 不是审计批量或队列上限。
- 判定式 / 状态转换:只有 `accepted`/`duplicate` 可设置 `delivered_at`;`rejected` 必须带稳定错误码并 dead-letter;批次级网络/5xx/认证/响应缺项不得误标成功。所有完成/失败写入必须匹配 worker + fencing token + 未过期数据库 lease。
- 安全边界:非回环只允许 HTTPS;HMAC 使用至少 32 字节外部 secret 和 constant-time compare;时间戳、nonce、key ID、签名格式失败统一安全拒绝。Sense 不获得 Bell schema 权限,Bell 不读取 Sense Outbox;日志、响应、测试和指标不包含 secret、Authorization、原始 DSN、完整 payload 或租户/设备标签。
- 既有契约:本地事实继续符合 `sense-device-audit-v1/v2`,不原地修改两份 schema;relay v1 只增加传输 envelope/确认语义。Bell 全局审计是追加事实,不能改写 Sense 原事件或设备状态。
## 验收要点
- 任务相关验证:`go -C Sense test ./internal/auditrelay ./internal/store ./internal/config ./cmd/sense-api`、`go -C Bell test ./internal/audit ./internal/store ./cmd/bell-api`;`python -m unittest discover -s tests -p "test_sense_audit_relay_contract.py" -v`;OpenAPI JSON 可解析且签名/状态/阈值与实现一致。
- 完整门禁:因命中 Sense/Bell Go、PostgreSQL schema/权限和内部 HTTP 安全边界,运行 `./scripts/test_postgres.ps1 -PgRoot D:\pgsql17`、`./init.ps1`、三条 Python 治理命令及 `git diff --check`。临时 PostgreSQL 必须重放 `001`~`015` 两次,现有 5432 listener 不变。
- 人工 / 设备验收:不适用;回环 `httptest` 和隔离 PostgreSQL 足以验收协议与持久化。跨主机证书、客户网络、防火墙和密钥轮换演练留给部署任务,不能据此宣称生产网络已验收。
- 构建产物:`go -C Sense build ./...` 与 `go -C Bell build ./...`;Bell 最小 receiver 可由 `go -C Bell build ./cmd/bell-api` 构建,未配置 DSN/key 时必须拒绝启动。
## 边界(不改什么)
- 不实现 Brain→Bell 事件 transport、规则引擎、Alert/ack/升级、Bell 公共审计查询 API、Web/H5、JWT/OIDC、消息总线或客户网络部署。
- 不删除本地 delivered/dead-letter Outbox 行,不擅自冻结法务审计留存期;本版本只自动清理 10 分钟后 relay nonce 收据,全局审计事实不自动删除。
- 不提交 key 文件、DSN、客户标识、真实 payload 或私有实例配置;不为测试修改现有 PostgreSQL data 目录或 5432 服务。
## 协作约束
- 责任 Agent:codex
- 唯一写入者:codex
- 委派:不启用。
- Gitea:Issue、`context_ref`、claim 与工作分支在双向映射及 dispatcher 分配后回填。
任何新增写路径先检查与其他活跃任务是否重叠;同一时刻只有一个 Agent 修改本任务的 `write_paths`。
## 执行记录
- 2026-08-10:在 T-015 合并并关闭后拆出本任务;实现尚未开始。
- 2026-08-11:冻结 `sense-audit-relay-v1.openapi.json`,实现 Sense 外部 key/HMAC client、30 秒数据库 lease/fencing worker、逐项结果/dead letter/1~300 秒退避,以及 PostgreSQL v6 Outbox relay repository;默认关闭且远端 URL 强制 HTTPS。
- 2026-08-11:实现 Bell 最小 `cmd/bell-api`、300 秒时间窗与 constant-time HMAC 校验、600 秒 nonce 幂等收据、逐项 v1/v2 校验,以及 PostgreSQL v4 不可变 `bell.audit_events`;Sense/Bell 登录权限保持单向隔离,只有 Bell 可清理过期收据。
- 2026-08-11:`./init.ps1`、三条 Python 治理命令、Sense/Bell `test/vet/build` 与 `git diff --check` 全部通过;Python 共 68 项测试通过。
- 2026-08-11:`./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` 通过;PostgreSQL 17.10 临时集群将 `001`~`015` 重放两次,Sense fencing、Bell receipt/重复/冲突/不可变性和双方权限断言通过,随机端口与临时目录已清理,现有 5432 listener 未改变。
+77
View File
@@ -0,0 +1,77 @@
---
id: T-017
title: 建立 Brain 单路匿名区域事件可视化工程原型
phase: 3
deps: [T-016]
status: TODO
created: 2026-08-11
issue: 59
context_ref: null
claim_branch: null
work_branch: null
write_paths:
- docs/tasks/T-017.md
- Brain/
- docs/design/brain/
- docs/00-ai-start-here.md
- docs/03-tech-stack.md
- docs/04-architecture.md
- docs/06-tasks.md
- docs/07-user-stories.md
- docs/08-interaction-checklist.md
- docs/routes.md
- docs/current-state.md
- init.ps1
- init.sh
---
## 问题 / 背景
Sense 已能稳定提供 MediaMTX 路径,Bell 已有不可变事件存储,但 `Brain/` 仍是目录占位,客户无法看到“画面 → 匿名检测 → 区域判定 → 事件候选”的直接效果。直接把 `silver_pose` 合并进来会破坏独立交付边界;未经授权引入 Ultralytics 生产依赖又有商用许可风险。首步需要一个可重复、可替换检测器、不会伪装算法效果的单路工程原型,为后续 Brain→Bell ingress 提供真实生产者。
## 关联需求与交互(如适用)
- 用户故事:新增 US-014;同时关联 US-005 的区域规则配置目标。
- 交互清单:新增 IX-024;几何编辑遵守 IX-017 的绘制、撤销、清空、键盘等效和待校准语义。
- 相关页面 / 路由:`/brain-demo` 仅为回环工程演示入口,不属于 Bell 公共业务路由;原型为 `docs/design/brain/index.html`。
## 方案
1. 冻结 T-017 工程原型栈:本机已验证的 Python 3.10.11、NumPy 1.26.4、OpenCV 4.9.0.80。OpenCV HOG 人员检测只作为可替换 `Detector` port 的演示适配器;不冻结 Savant/DeepStream、生产模型或 GPU 容量。
2. 建立单路 frame source、detector、轻量 track、归一化多边形 zone、进入/停留判定和内存事件候选流水线。默认合成回放携带明确的 fixture 标识,保证无摄像头/模型也能重复验收;真实流模式只从仓库外绝对路径文件读取 URL,并优先指向 Sense 管理的 MediaMTX path。
3. 提供仅监听回环地址的本地演示服务:画面、检测框、track、区域、命中状态、推理延迟和最近事件可观察;区域支持鼠标绘制、撤销、清空、保存,以及无需精确点击的归一化坐标表单。页面使用自包含 HTML/CSS/JS,不加载 CDN、字体或第三方脚本。
4. 本地演示 API 不是跨系统契约,不接受平台事件 `id`,不写数据库、不发送 Alert、不管理录像。T-018 再冻结 Brain→Bell transport、逻辑身份、Bell ULID、持久重试与幂等确认。
5. 单元测试覆盖几何边界、进入/停留只触发一次、track 过期、事件候选不含平台 ID、URL 文件校验、回环绑定、API 输入限制和敏感信息不回显;HTML 契约测试覆盖响应式、键盘、焦点、44px 操作目标、reduced-motion、状态文本和无外链资源。
## 不可变约束
- 阈值 / 数值边界:单路原型默认处理 2 FPS;区域点使用 `[0,1]` 归一化坐标,3~32 个顶点;最近事件仅保留 100 项内存环;HTTP body 不超过 64 KiB。任何值不得成为 16/128 路或生产 worker 的硬上限。
- 判定式 / 状态转换:仅 track 中心点从区域外进入区域内时产生一次 `zone_entry` 候选;持续位于区域内不重复产生;离开后再次进入才可重新产生。fixture 与真实检测结果必须显式区分,HOG 无检测不得伪造框。
- 安全边界:服务只允许回环监听;真实流 URL 只从仓库外绝对路径、单行、受限大小文件读取,响应、日志、页面和异常不得回显 URL/凭据。禁止提交真实视频、截图、模型权重、凭据或客户标识。
- 既有契约:Brain 只产生 `source_event_id` 和候选事实,平台 `evt_` ULID 仍由 Bell 生成;本任务不修改冻结 event v0.1,不把本地 demo API 宣称为 Brain→Bell transport。`silver_pose` 保持独立仓库且不进入依赖/构建上下文。
## 验收要点
- 任务相关验证:`python -m unittest discover -s Brain/tests -p "test_*.py" -v`、`python -m compileall -q Brain`;启动合成模式后用回环 HTTP smoke 验证状态、画面、区域保存和事件候选;HTML 静态契约测试通过。
- 完整门禁:因首次建立 Brain 脚手架并更新标准入口,运行 `./init.ps1`、三条 Python 治理命令以及 `git diff --check`;Sense/Bell 既有 test/vet/build 仍须由标准入口通过。
- 人工 / 设备验收:真实摄像头和目标 GPU 不作为本任务 DONE 门禁;合成回放验证工程闭环。真实流只做可选本机 smoke,结果不能外推为算法效果、GPU 容量或生产 SLA。UI 产品确认留给 Bell 业务闭环任务,工程演示页必须明确“非生产模型验收”。
- 构建产物:源码运行原型,无独立安装包;入口为 `python -m Brain.yovision_brain --source synthetic`,依赖清单和启动说明写入 `Brain/README.md`。
## 边界(不改什么)
- 不实现 Brain→Bell 网络投递、持久 Outbox、Bell ingress、规则/Alert/ack、证据切片、对象存储、反馈回流或公共认证。
- 不承诺检测召回率/误报率,不购买或引入 Ultralytics 生产许可,不冻结 Savant/DeepStream、CUDA、跟踪/ReID 或每 worker 路数。
- 不修改 Sense、Bell、`_reference/` 或 `D:\OPC\silver_pose`;不直接连接摄像头管理端或绕过 MediaMTX 系统边界。
## 协作约束
- 责任 Agent:codex
- 唯一写入者:codex
- 委派:不启用。
- Gitea:Issue、`context_ref`、claim 与工作分支在双向映射及 dispatcher 分配后回填。
任何新增写路径先检查与其他活跃任务是否重叠;同一时刻只有一个 Agent 修改本任务的 `write_paths`。
## 执行记录
- 2026-08-11:按用户指定顺序建立 T-017;实现尚未开始。主分支 `./init.ps1` 基线通过(68 项根测试,Sense/Bell test/vet/build 通过)。
+3 -3
View File
@@ -10,9 +10,9 @@
$ErrorActionPreference = "Stop"
Set-Location -Path $PSScriptRoot
# Sense 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。
$InstallCmd = "go -C Sense mod download"
$VerifyCmd = "python scripts/validate_agent_context.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python -m unittest discover -s tests -p 'test_*.py'; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python scripts/validate_harness_governance.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense generate ./internal/mtx ./internal/controlapi; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go Sense/internal/controlapi/generated.gen.go; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense test ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense vet ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense build ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }"
# Sense/Bell 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。
$InstallCmd = "go -C Sense mod download; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Bell mod download"
$VerifyCmd = "python scripts/validate_agent_context.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python -m unittest discover -s tests -p 'test_*.py'; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python scripts/validate_harness_governance.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense generate ./internal/mtx ./internal/controlapi; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go Sense/internal/controlapi/generated.gen.go; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense test ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense vet ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense build ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Bell test ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Bell vet ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Bell build ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }"
$StartCmd = "go -C Sense run ./cmd/sense-api"
function Assert-Configured {
+3 -3
View File
@@ -12,9 +12,9 @@ set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
cd "$ROOT_DIR"
# Sense 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。
INSTALL_CMD=(go -C Sense mod download)
VERIFY_CMD=(bash -lc "python3 scripts/validate_agent_context.py && python3 -m unittest discover -s tests -p 'test_*.py' && python3 scripts/validate_harness_governance.py && go -C Sense generate ./internal/mtx ./internal/controlapi && git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go Sense/internal/controlapi/generated.gen.go && go -C Sense test ./... && go -C Sense vet ./... && go -C Sense build ./...")
# Sense/Bell 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。
INSTALL_CMD=(bash -lc "go -C Sense mod download && go -C Bell mod download")
VERIFY_CMD=(bash -lc "python3 scripts/validate_agent_context.py && python3 -m unittest discover -s tests -p 'test_*.py' && python3 scripts/validate_harness_governance.py && go -C Sense generate ./internal/mtx ./internal/controlapi && git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go Sense/internal/controlapi/generated.gen.go && go -C Sense test ./... && go -C Sense vet ./... && go -C Sense build ./... && go -C Bell test ./... && go -C Bell vet ./... && go -C Bell build ./...")
START_CMD=(go -C Sense run ./cmd/sense-api)
ensure_configured() {
+17 -1
View File
@@ -49,6 +49,8 @@ $hadSenseDSN = Test-Path Env:YOVISION_TEST_POSTGRES_DSN
$previousSenseDSN = if ($hadSenseDSN) { (Get-Item Env:YOVISION_TEST_POSTGRES_DSN).Value } else { $null }
$hadAdminDSN = Test-Path Env:YOVISION_TEST_POSTGRES_ADMIN_DSN
$previousAdminDSN = if ($hadAdminDSN) { (Get-Item Env:YOVISION_TEST_POSTGRES_ADMIN_DSN).Value } else { $null }
$hadBellDSN = Test-Path Env:YOVISION_TEST_BELL_POSTGRES_DSN
$previousBellDSN = if ($hadBellDSN) { (Get-Item Env:YOVISION_TEST_BELL_POSTGRES_DSN).Value } else { $null }
function Invoke-Checked {
param(
@@ -87,17 +89,25 @@ try {
'008_control_api.sql',
'009_privileges_control_api.sql',
'010_reconcile_safety.sql',
'011_privileges_reconcile_safety.sql'
'011_privileges_reconcile_safety.sql',
'012_bell_events.sql',
'013_privileges_bell_events.sql',
'014_audit_relay.sql',
'015_privileges_audit_relay.sql'
)) {
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminDatabaseDSN '-f' (Join-Path $repoRoot "deploy\postgres\$name")
}
}
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-c' 'CREATE ROLE yovision_t012_sense LOGIN IN ROLE sense_app'
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-c' 'CREATE ROLE yovision_t015_bell LOGIN IN ROLE bell_runtime'
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminDatabaseDSN '-f' (Join-Path $repoRoot 'deploy\postgres\tests\assertions.sql')
$env:YOVISION_TEST_POSTGRES_DSN = $senseDSN
$env:YOVISION_TEST_POSTGRES_ADMIN_DSN = $adminDatabaseDSN
$bellDSN = "postgres://yovision_t015_bell@127.0.0.1:$port/${databaseName}?sslmode=disable"
$env:YOVISION_TEST_BELL_POSTGRES_DSN = $bellDSN
Invoke-Checked 'go' '-C' (Join-Path $repoRoot 'Sense') 'test' './internal/store' '-run' '^TestPostgres' '-count=1'
Invoke-Checked 'go' '-C' (Join-Path $repoRoot 'Bell') 'test' './internal/store' '-run' '^TestPostgres' '-count=1'
}
finally {
if ($started) {
@@ -132,6 +142,12 @@ finally {
else {
Remove-Item Env:YOVISION_TEST_POSTGRES_ADMIN_DSN -ErrorAction SilentlyContinue
}
if ($hadBellDSN) {
$env:YOVISION_TEST_BELL_POSTGRES_DSN = $previousBellDSN
}
else {
Remove-Item Env:YOVISION_TEST_BELL_POSTGRES_DSN -ErrorAction SilentlyContinue
}
$after5432 = @(
Get-NetTCPConnection -State Listen -LocalPort 5432 -ErrorAction SilentlyContinue |
Select-Object -ExpandProperty OwningProcess -Unique |
+63
View File
@@ -0,0 +1,63 @@
"""Static contract checks for T-015 Bell immutable event storage."""
from pathlib import Path
import unittest
ROOT = Path(__file__).resolve().parents[1]
def text(path: str) -> str:
return (ROOT / path).read_text(encoding="utf-8")
class BellEventContractTests(unittest.TestCase):
def test_runtime_schema_copy_matches_frozen_contract(self) -> None:
self.assertEqual(
(ROOT / "docs/raw/contracts/event-v0.1.schema.json").read_bytes(),
(ROOT / "Bell/contracts/event-v0.1.schema.json").read_bytes(),
)
def test_bell_module_freezes_reviewed_dependencies(self) -> None:
module = text("Bell/go.mod")
self.assertIn("github.com/jackc/pgx/v5 v5.10.0", module)
self.assertIn("github.com/oklog/ulid/v2 v2.1.2", module)
self.assertIn("github.com/santhosh-tekuri/jsonschema/v6 v6.0.2", module)
def test_event_migration_is_append_only_and_not_limited_to_sixteen(self) -> None:
migration = text("deploy/postgres/012_bell_events.sql").lower()
privileges = text("deploy/postgres/013_privileges_bell_events.sql").lower()
for marker in (
"create table if not exists bell.events",
"create table if not exists bell.event_outcomes",
"bell.reject_immutable_change",
"insert into bell.schema_migrations(version) values (3)",
):
self.assertIn(marker, migration)
self.assertNotIn("limit 16", migration)
self.assertIn("grant select, insert on table bell.events, bell.event_outcomes to bell_runtime", privileges)
self.assertNotIn("grant update", privileges)
self.assertNotIn("grant delete", privileges)
self.assertNotIn("grant truncate", privileges)
def test_factory_owns_id_and_checks_all_semantic_boundaries(self) -> None:
source = text("Bell/internal/event/event.go")
for marker in (
"upstream_id_forbidden",
"latency_inconsistent",
"confidence_forbidden",
"evidence_unsafe",
"primary_sensor_invalid",
"privacy_unavailable",
"privacy_denied",
):
self.assertIn(marker, source)
def test_contract_document_no_longer_rejects_non_imaging_empty_snapshot(self) -> None:
readme = text("docs/raw/contracts/README.md")
self.assertNotIn("`snapshot_uris` 空数组 / 负 latency", readme)
self.assertIn("`snapshot_uris` 空数组对非成像事件是合法值", readme)
if __name__ == "__main__":
unittest.main()
+28
View File
@@ -58,6 +58,10 @@ class PostgresContractTests(unittest.TestCase):
"009_privileges_control_api.sql",
"010_reconcile_safety.sql",
"011_privileges_reconcile_safety.sql",
"012_bell_events.sql",
"013_privileges_bell_events.sql",
"014_audit_relay.sql",
"015_privileges_audit_relay.sql",
],
names,
)
@@ -71,6 +75,15 @@ class PostgresContractTests(unittest.TestCase):
)
self.assertNotRegex(text, r"\bpassword\b")
def test_bell_runtime_role_is_separate_from_migration_owner(self) -> None:
text = normalized(migration_text("012_bell_events.sql"))
self.assertIn(
"create role bell_runtime nologin nosuperuser nocreatedb nocreaterole noreplication",
text,
)
self.assertNotIn("grant bell_app to bell_runtime", text)
self.assertIn("pg_has_role('bell_runtime', 'bell_app', 'member')", text)
def test_bell_quota_contract_and_version_are_enforced(self) -> None:
text = normalized(migration_text("002_bell.sql"))
expected_signature = (
@@ -118,6 +131,21 @@ class PostgresContractTests(unittest.TestCase):
self.assertIn(marker, text)
self.assertNotIn("D:\\pgsql17\\data", text)
def test_audit_relay_uses_fencing_and_separate_schema_ownership(self) -> None:
migration = normalized(migration_text("014_audit_relay.sql"))
privileges = normalized(migration_text("015_privileges_audit_relay.sql"))
for marker in (
"relay_lease_owner",
"relay_lease_token",
"relay_lease_until",
"create table if not exists bell.audit_events",
"create table if not exists bell.audit_relay_receipts",
"expires_at >= received_at + interval '10 minutes'",
):
self.assertIn(marker, migration)
self.assertIn("grant select, insert on table bell.audit_events to bell_runtime", privileges)
self.assertIn("revoke all on table bell.audit_events, bell.audit_relay_receipts from sense_app", privileges)
if __name__ == "__main__":
unittest.main()
+54
View File
@@ -0,0 +1,54 @@
"""Cross-language invariants for the Sense-to-Bell audit relay contract."""
from __future__ import annotations
import json
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
CONTRACT = ROOT / "docs" / "contracts" / "sense-audit-relay-v1.openapi.json"
class SenseAuditRelayContractTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.document = json.loads(CONTRACT.read_text(encoding="utf-8"))
def test_transport_and_response_are_frozen(self) -> None:
operation = self.document["paths"]["/internal/v1/audit-events:batch"]["post"]
self.assertEqual({"200", "400", "401", "409", "413", "503"}, set(operation["responses"]))
headers = {item["name"] for item in operation["parameters"]}
self.assertEqual(
{"X-YoVision-Key-Id", "X-YoVision-Timestamp", "X-YoVision-Nonce", "X-YoVision-Signature"},
headers,
)
def test_numeric_safety_bounds_match_task(self) -> None:
events = self.document["components"]["schemas"]["BatchRequest"]["properties"]["events"]
self.assertEqual((1, 100), (events["minItems"], events["maxItems"]))
signature = self.document["x-yovision-signature"]
delivery = self.document["x-yovision-delivery"]
self.assertEqual(300, signature["clock_skew_seconds"])
self.assertEqual(600, signature["receipt_ttl_seconds"])
self.assertEqual((30, 1, 300), (delivery["lease_seconds"], delivery["initial_retry_seconds"], delivery["maximum_retry_seconds"]))
def test_contract_and_implementations_share_literals(self) -> None:
sense = (ROOT / "Sense" / "internal" / "auditrelay" / "client.go").read_text(encoding="utf-8")
bell = (ROOT / "Bell" / "internal" / "audit" / "audit.go").read_text(encoding="utf-8")
for marker in (
"/internal/v1/audit-events:batch",
"X-YoVision-Key-Id",
"X-YoVision-Timestamp",
"X-YoVision-Nonce",
"X-YoVision-Signature",
):
self.assertIn(marker, sense)
self.assertIn(marker, bell)
self.assertIn("10 * time.Second", sense)
self.assertIn("300*time.Second", (ROOT / "Sense" / "internal" / "auditrelay" / "worker.go").read_text(encoding="utf-8"))
if __name__ == "__main__":
unittest.main()
+76
View File
@@ -0,0 +1,76 @@
import re
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
SCRIPT = ROOT / "Sense" / "scripts" / "t014-capacity.ps1"
TASK = ROOT / "docs" / "tasks" / "T-014.md"
class SenseCapacityContractTests(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.script = SCRIPT.read_text(encoding="utf-8")
cls.task = TASK.read_text(encoding="utf-8")
def test_uses_sixteen_independent_copy_publishers(self):
self.assertIn("$sourceCount = 16", self.script)
self.assertIn("foreach ($index in 1..$sourceCount)", self.script)
self.assertIn("'-c', 'copy'", self.script)
self.assertIn("$faultIndexes = @(5, 6, 7, 8)", self.script)
self.assertNotIn("ip_camera.env", self.script)
def test_exercises_real_control_api_and_quota_boundary(self):
self.assertIn("/api/v1/sites/site-t014/devices:batchDesiredState", self.script)
self.assertIn("desired_state = 'disabled'", self.script)
self.assertIn("$firstEnableEntries = Get-DeviceEntries @($deviceIDs)", self.script)
self.assertIn("t014-overflow-17", self.script)
self.assertIn("quota_exceeded", self.script)
self.assertIn("'enabled'", self.script)
self.assertIn("'disabled'", self.script)
def test_formal_window_and_functional_gates_are_fixed(self):
self.assertRegex(self.script, r"\[int\]\$ObservationMinutes = 30")
self.assertIn("$sampleSeconds = 10", self.script)
self.assertIn("$requiredSamples = [int][Math]::Ceiling($targetSeconds / $sampleSeconds)", self.script)
self.assertIn("$nextSampleAt = ($samples + 1) * $sampleSeconds", self.script)
self.assertIn("formal_eligible = $ObservationMinutes -ge 30", self.script)
self.assertIn("Wait-Convergence 'enabled'", self.script)
self.assertIn("Wait-Convergence 'disabled'", self.script)
self.assertIn("maximum_unconverged", self.script)
self.assertIn("go1\\.26\\.5", self.script)
self.assertIn("& go '-C' $senseDirectory 'version'", self.script)
def test_isolated_postgres_and_cleanup_are_guarded(self):
for migration in range(2, 12):
self.assertRegex(self.script, rf"'{migration:03d}_[^']+\.sql'")
self.assertIn("PostgreSQL\\) 17\\.10$", self.script)
self.assertIn("yovision-t014", self.script)
self.assertIn("[IO.Path]::GetFileName($resolvedSession) -notlike 'session-*'", self.script)
self.assertIn("the existing PostgreSQL listener on port 5432 changed", self.script)
self.assertNotRegex(self.script, r"Invoke-Checked \$script:pgCtl .* 'start'")
self.assertIn("$cleanupWatch.Elapsed.TotalSeconds -lt 10", self.script)
self.assertIn("$item.Process.Dispose()", self.script)
self.assertIn("$deleteWatch.Elapsed.TotalSeconds -ge 10", self.script)
def test_runtime_secrets_and_sensitive_inventory_are_not_reported(self):
self.assertIn("New-SecretToken", self.script)
self.assertIn("token_sha256", self.script)
self.assertIn("capacity result output must stay outside the repository", self.script)
result_block = self.script.split("$resultJSON = [ordered]@{", 1)[1]
for forbidden in ("controlToken", "senseDSN", "adminDatabaseDSN", "sourceRTSPPort", "endpoint_ref", "credential_ref"):
self.assertNotIn(forbidden, result_block)
self.assertNotRegex(self.script, r"(?i)rtsp://[^\s\"']+:[^\s\"']+@")
def test_task_preserves_product_boundaries(self):
self.assertIn("deps: [T-012]", self.task)
self.assertIn("默认配额 16、允许范围 1~128", self.task)
self.assertIn("129 仍非法", self.task)
self.assertIn("不代表 16 台真实摄像头", self.task)
self.assertIn("不解除 T-007", self.task)
self.assertIn("WireGuard T-013 暂缓", self.task)
if __name__ == "__main__":
unittest.main()