[T-011] 实现 Sense Control API v1 与 PostgreSQL 一致性边界 #41

Merged
ila merged 3 commits from agent/codex/T-011 into main 2026-08-07 20:57:18 +08:00
45 changed files with 4854 additions and 86 deletions
+38 -7
View File
@@ -1,20 +1,20 @@
# Sense M1/M2 接入骨架 # Sense M1/M2 接入骨架
本目录是 YoVision Sense 的 M1/M2 接入骨架。数据库保存期望态,ONVIF 和 MediaMTX 通过端口隔离;M1 默认使用 SQLite,T-009/T-010 增加 PostgreSQL 双 schema、Area 准入和本地审计 Outbox 生产基础。默认关闭真实 ONVIF,显式设置 `SENSE_ONVIF_MODE=standard` 后才启用标准 SOAP/WS-Security 适配器。T-006 的真实样机结论仅覆盖已批准的精确海康基线,不能据此宣称多品牌兼容。 本目录是 YoVision Sense 的 M1/M2 接入骨架。数据库保存期望态,ONVIF 和 MediaMTX 通过端口隔离;M1 默认使用 SQLite,T-009~T-011 增加 PostgreSQL 双 schema、Area 准入、本地审计 Outbox 和 Control API v1。默认关闭真实 ONVIF 与公共业务路由;T-006 的真实样机结论仅覆盖已批准的精确海康基线,不能据此宣称多品牌兼容。
## 常用命令 ## 常用命令
```powershell ```powershell
cd Sense cd Sense
go mod download go mod download
go generate ./internal/mtx go generate ./internal/mtx ./internal/controlapi
go test ./... go test ./...
go vet ./... go vet ./...
go build -o bin/sense-api.exe ./cmd/sense-api go build -o bin/sense-api.exe ./cmd/sense-api
go run ./cmd/sense-api go run ./cmd/sense-api
``` ```
Unix 将构建产物改为 `bin/sense-api`。服务默认监听 `127.0.0.1:8080`,SQLite 默认写入 `Sense/data/sense.db`,MediaMTX 控制 API 默认是 `http://127.0.0.1:9997`。当前运行代码只有 `/healthz` 与 `/readyz`;Sense Control API v1 虽已冻结,但 HTTP handler 和认证尚未实现。 Unix 将构建产物改为 `bin/sense-api`。服务默认监听 `127.0.0.1:8080`,SQLite 默认写入 `Sense/data/sense.db`,MediaMTX 控制 API 默认是 `http://127.0.0.1:9997`。默认运行只暴露 `/healthz` 与 `/readyz`;只有显式选择 PostgreSQL 并完成安全配置后才注册 7 个 `/api/v1` Control API 路由。
常用环境变量: 常用环境变量:
@@ -31,6 +31,11 @@ Unix 将构建产物改为 `bin/sense-api`。服务默认监听 `127.0.0.1:8080`
| `SENSE_ONVIF_RTSP_REWRITE_HOST` | 空 | NAT 或故障代理场景下重写 ONVIF 返回的 RTSP 主机 | | `SENSE_ONVIF_RTSP_REWRITE_HOST` | 空 | NAT 或故障代理场景下重写 ONVIF 返回的 RTSP 主机 |
| `SENSE_ONVIF_RTSP_REWRITE_PORT` | `0` | 非零时重写 ONVIF 返回的 RTSP 端口 | | `SENSE_ONVIF_RTSP_REWRITE_PORT` | `0` | 非零时重写 ONVIF 返回的 RTSP 端口 |
| `SENSE_ONVIF_RTSP_STRIP_QUERY` | `false` | 仅在已验证设备返回不可用查询串时显式移除;默认保留标准 URI 语义 | | `SENSE_ONVIF_RTSP_STRIP_QUERY` | `false` | 仅在已验证设备返回不可用查询串时显式移除;默认保留标准 URI 语义 |
| `SENSE_CONTROL_API_ENABLED` | `false` | 显式开启 Control API v1;只允许与 PostgreSQL 一起使用 |
| `SENSE_CONTROL_AUTH_MODE` | `static-sha256` | 首版外部摘要注册表适配器;token 格式不属于公共 API 契约 |
| `SENSE_CONTROL_AUTH_FILE` | 空 | 仓库外绝对路径;version 1 JSON 只保存 token SHA-256、主体、tenant、Site scope 和权限 |
| `SENSE_CONTROL_CURSOR_KEY_FILE` | 空 | 仓库外绝对路径;内容为至少 32 字节随机值的无填充 base64url |
| `SENSE_CONTROL_ALLOW_INSECURE_HTTP` | `false` | Control API 非回环明文监听的独立风险接受;正常部署应保持回环并在受控代理终止 TLS |
设备台账只保存 `env://<key>` 凭据引用。真实适配器从进程环境读取以下变量,不把秘密写入 SQLite、日志或 MediaMTX 错误: 设备台账只保存 `env://<key>` 凭据引用。真实适配器从进程环境读取以下变量,不把秘密写入 SQLite、日志或 MediaMTX 错误:
@@ -45,11 +50,11 @@ SENSE_CREDENTIAL_<KEY>_RTSP_PASSWORD
MediaMTX `v1.19.3` 应作为独立二进制启动并只在可信网络开放 API。获取与 SHA-256 校验值见 `docs/03-tech-stack.md`。生成客户端使用固定版本工具和 vendored 官方 OpenAPI;`internal/mtx/generated/client.gen.go` 不可手改。 MediaMTX `v1.19.3` 应作为独立二进制启动并只在可信网络开放 API。获取与 SHA-256 校验值见 `docs/03-tech-stack.md`。生成客户端使用固定版本工具和 vendored 官方 OpenAPI;`internal/mtx/generated/client.gen.go` 不可手改。
## T-009/T-010 PostgreSQL 17.10 ## T-009~T-011 PostgreSQL 17.10
初始化与增量 SQL 位于 `deploy/postgres/`,由高权限部署步骤按文件名前缀执行;Sense 进程不会自动创建角色、schema 或 Bell 对象。`bell_app` 拥有 Site/Area、配额、`capture_policy` 及两个版本化视图,`sense_app` 只能读取 `bell.site_quota_v1` 和 `bell.area_policy_v1`,不能读取或写入 Bell 源表。应用登录角色和密码由部署环境创建,不进入仓库。 初始化与增量 SQL 位于 `deploy/postgres/`,由高权限部署步骤按文件名前缀执行;Sense 进程不会自动创建角色、schema 或 Bell 对象。`bell_app` 拥有 Site/Area、配额、`capture_policy` 及两个版本化视图,`sense_app` 只能读取两个视图,不能读取或写入 Bell 源表。T-011 的 v4 schema 另有资源版本、24 小时幂等收据和 batch operation 表;收据只保存 scope/body 摘要与脱敏响应快照。应用登录角色和密码由部署环境创建,不进入仓库。
PostgreSQL 新建设备必须携带匹配 tenant/Site 的 `area_id`。具有 `video_capture` 能力的设备在创建和从 disabled 切到 enabled 时执行 Area 准入;`non_imaging_only` 拒绝成像设备但允许非成像设备。投影缺失、非法或版本回退只拒绝新变更,不关闭已有流。设备创建和期望态受理与 `sense.device_operation_outbox` 在同一事务写入;Outbox 不保存 endpoint、credential、完整 URI 或 MediaMTX 配置。Bell relay 尚未实现。 PostgreSQL 新建设备必须携带匹配 tenant/Site 的 `area_id`。具有 `video_capture` 能力的设备在创建、移动 Area 和从 disabled 切到 enabled 时执行 Area 准入;`non_imaging_only` 拒绝成像设备但允许非成像设备。投影缺失、非法或版本回退只拒绝新变更,不关闭已有流。创建、配置修改和期望态受理都与对应脱敏 Outbox 事实同事务;停用后调和器只删除该设备的精确 MediaMTX path 并收敛为 offline,不枚举未知 path。Bell relay 尚未实现。
Windows 本机集成测试从仓库根目录执行: Windows 本机集成测试从仓库根目录执行:
@@ -67,7 +72,33 @@ $env:SENSE_DB_DSN = '由部署环境私下设置'
go run ./cmd/sense-api go run ./cmd/sense-api
``` ```
PostgreSQL 启动会检查 Sense migration 版本及当前角色对两个 Bell 投影视图的最小权限;权限过宽、视图不可读或 schema 未安装时 readiness 初始化失败。默认 SQLite 路径和 `cmd/sense-lab` 保持不变,但 SQLite 不实现生产 Area/Outbox 一致性,后续公共控制 API 不得在 SQLite 上启用。 PostgreSQL 启动会检查 Sense migration 版本及当前角色对两个 Bell 投影视图的最小权限;权限过宽、视图不可读或 schema 未安装时 readiness 初始化失败。默认 SQLite 路径和 `cmd/sense-lab` 保持不变,但 SQLite 不实现生产 Area/Outbox 一致性,Control API feature flag 在 SQLite 下会直接拒绝启动。
### 开启 Control API
复制 [`api/control-auth.example.json`](api/control-auth.example.json) 到仓库外受限目录并替换占位项。`token_sha256` 是至少 128 bit 随机 Bearer token 的 64 位小写 SHA-256,不是 token 明文;`site_ids` 支持精确 ID 或 `"*"`,权限只接受 `sense.devices.read`、`sense.devices.write`。注册表在启动时读取,轮换后需要受控重启。
另在仓库外生成 cursor key。PowerShell 示例只把 key 写入指定秘密文件,不把值打印到日志:
```powershell
$cursorBytes = [byte[]]::new(32)
[Security.Cryptography.RandomNumberGenerator]::Fill($cursorBytes)
$cursorKey = [Convert]::ToBase64String($cursorBytes).TrimEnd('=').Replace('+', '-').Replace('/', '_')
Set-Content -LiteralPath 'D:\private\sense-cursor.key' -Value $cursorKey -NoNewline
```
完成 PostgreSQL migration、专用登录角色和外部文件权限后,以私有环境开启:
```powershell
$env:SENSE_DB_DRIVER = 'postgres'
$env:SENSE_DB_DSN = '由部署环境私下设置'
$env:SENSE_CONTROL_API_ENABLED = 'true'
$env:SENSE_CONTROL_AUTH_FILE = 'D:\private\sense-auth.json'
$env:SENSE_CONTROL_CURSOR_KEY_FILE = 'D:\private\sense-cursor.key'
go run ./cmd/sense-api
```
业务响应使用 `Cache-Control: no-store`;ETag 是写并发令牌,cursor 与认证 tenant/Site/筛选绑定。静态摘要文件只是首版私有部署适配器;公网/TLS、Bell 会话、JWT/OIDC 与热加载需后续任务,不能靠设置 `SENSE_CONTROL_ALLOW_INSECURE_HTTP=true` 冒充完成。
Windows 本地准备 MediaMTX(从仓库根目录执行): Windows 本地准备 MediaMTX(从仓库根目录执行):
+13
View File
@@ -0,0 +1,13 @@
{
"version": 1,
"principals": [
{
"token_sha256": "0000000000000000000000000000000000000000000000000000000000000000",
"subject_id": "replace-with-service-id",
"actor_type": "service",
"tenant_id": "tenant-demo",
"site_ids": ["site-demo"],
"permissions": ["sense.devices.read", "sense.devices.write"]
}
]
}
+23 -1
View File
@@ -12,7 +12,9 @@ import (
"syscall" "syscall"
"time" "time"
"yovision/sense/internal/auth"
"yovision/sense/internal/config" "yovision/sense/internal/config"
"yovision/sense/internal/controlapi"
"yovision/sense/internal/mtx" "yovision/sense/internal/mtx"
"yovision/sense/internal/onvif" "yovision/sense/internal/onvif"
"yovision/sense/internal/probe" "yovision/sense/internal/probe"
@@ -43,6 +45,22 @@ func run(logger *slog.Logger) error {
return err return err
} }
defer repository.Close() defer repository.Close()
var controlHandler http.Handler
if cfg.ControlAPIEnabled {
controlStore, ok := repository.(store.ControlRepository)
if !ok {
return errors.New("selected repository does not support Sense Control API")
}
authenticator, err := auth.LoadStaticSHA256(cfg.ControlAuthFile)
if err != nil {
return err
}
cursors, err := controlapi.LoadCursorCodec(cfg.ControlCursorKeyFile)
if err != nil {
return err
}
controlHandler = controlapi.NewHTTPHandler(controlStore, authenticator, cursors)
}
mediaClient, err := mtx.NewClient(cfg.MediaMTXURL, nil) mediaClient, err := mtx.NewClient(cfg.MediaMTXURL, nil)
if err != nil { if err != nil {
return err return err
@@ -86,6 +104,9 @@ func run(logger *slog.Logger) error {
writer.WriteHeader(http.StatusOK) writer.WriteHeader(http.StatusOK)
_, _ = writer.Write([]byte(`{"status":"ready"}`)) _, _ = writer.Write([]byte(`{"status":"ready"}`))
}) })
if cfg.ControlAPIEnabled {
mux.Handle("/api/v1/", controlHandler)
}
server := &http.Server{ server := &http.Server{
Addr: cfg.HTTPAddress, Handler: mux, Addr: cfg.HTTPAddress, Handler: mux,
@@ -96,7 +117,8 @@ func run(logger *slog.Logger) error {
} }
serverErrors := make(chan error, 1) serverErrors := make(chan error, 1)
go func() { go func() {
logger.Info("Sense listening", "address", cfg.HTTPAddress, "version", version) logger.Info("Sense listening", "address", cfg.HTTPAddress, "version", version,
"control_api_enabled", cfg.ControlAPIEnabled)
serverErrors <- server.ListenAndServe() serverErrors <- server.ListenAndServe()
}() }()
+181
View File
@@ -0,0 +1,181 @@
// Package auth defines the verified principal boundary for Sense HTTP APIs.
package auth
import (
"context"
"crypto/sha256"
"crypto/subtle"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"regexp"
"strings"
)
const (
PermissionDevicesRead = "sense.devices.read"
PermissionDevicesWrite = "sense.devices.write"
)
var (
ErrUnauthenticated = errors.New("unauthenticated")
logicalIDPattern = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._:-]{0,63}$`)
)
type Principal struct {
SubjectID string
ActorType string
TenantID string
SiteIDs []string
Permissions map[string]struct{}
}
func (p Principal) AllowsSite(siteID string) bool {
for _, allowed := range p.SiteIDs {
if allowed == "*" || allowed == siteID {
return true
}
}
return false
}
func (p Principal) Has(permission string) bool {
_, ok := p.Permissions[permission]
return ok
}
type Authenticator interface {
Authenticate(context.Context, string) (Principal, error)
}
type staticFile struct {
Version int `json:"version"`
Principals []staticPrincipal `json:"principals"`
}
type staticPrincipal struct {
TokenSHA256 string `json:"token_sha256"`
SubjectID string `json:"subject_id"`
ActorType string `json:"actor_type"`
TenantID string `json:"tenant_id"`
SiteIDs []string `json:"site_ids"`
Permissions []string `json:"permissions"`
}
type staticEntry struct {
digest [sha256.Size]byte
principal Principal
}
// StaticSHA256 authenticates opaque tokens against externally provisioned
// SHA-256 digests. The source file is read only during process startup.
type StaticSHA256 struct {
entries []staticEntry
}
func LoadStaticSHA256(path string) (*StaticSHA256, error) {
contents, err := os.ReadFile(path)
if err != nil {
return nil, errors.New("read static authentication registry")
}
decoder := json.NewDecoder(strings.NewReader(string(contents)))
decoder.DisallowUnknownFields()
var document staticFile
if err := decoder.Decode(&document); err != nil {
return nil, errors.New("decode static authentication registry")
}
if err := decoder.Decode(&struct{}{}); err != io.EOF {
return nil, errors.New("static authentication registry has trailing JSON")
}
if document.Version != 1 || len(document.Principals) == 0 {
return nil, errors.New("static authentication registry must contain version 1 principals")
}
entries := make([]staticEntry, 0, len(document.Principals))
seenDigests := make(map[string]struct{}, len(document.Principals))
for index, value := range document.Principals {
entry, validationErr := parseStaticPrincipal(value)
if validationErr != nil {
return nil, fmt.Errorf("invalid static authentication principal %d: %w", index, validationErr)
}
if _, exists := seenDigests[value.TokenSHA256]; exists {
return nil, errors.New("duplicate static authentication token digest")
}
seenDigests[value.TokenSHA256] = struct{}{}
entries = append(entries, entry)
}
return &StaticSHA256{entries: entries}, nil
}
func parseStaticPrincipal(value staticPrincipal) (staticEntry, error) {
var entry staticEntry
if len(value.TokenSHA256) != sha256.Size*2 || value.TokenSHA256 != strings.ToLower(value.TokenSHA256) {
return entry, errors.New("token_sha256 must be 64 lowercase hexadecimal characters")
}
digest, err := hex.DecodeString(value.TokenSHA256)
if err != nil {
return entry, errors.New("token_sha256 must be hexadecimal")
}
copy(entry.digest[:], digest)
if strings.TrimSpace(value.SubjectID) == "" || len(value.SubjectID) > 200 {
return entry, errors.New("subject_id must contain 1 to 200 characters")
}
if value.ActorType != "user" && value.ActorType != "service" {
return entry, errors.New("actor_type must be user or service")
}
if !logicalIDPattern.MatchString(value.TenantID) {
return entry, errors.New("tenant_id is invalid")
}
if len(value.SiteIDs) == 0 {
return entry, errors.New("site_ids must not be empty")
}
sites := make([]string, 0, len(value.SiteIDs))
seenSites := make(map[string]struct{}, len(value.SiteIDs))
for _, siteID := range value.SiteIDs {
if siteID != "*" && !logicalIDPattern.MatchString(siteID) {
return entry, errors.New("site_ids contains an invalid site")
}
if _, exists := seenSites[siteID]; exists {
return entry, errors.New("site_ids contains a duplicate")
}
seenSites[siteID] = struct{}{}
sites = append(sites, siteID)
}
permissions := make(map[string]struct{}, len(value.Permissions))
for _, permission := range value.Permissions {
if permission != PermissionDevicesRead && permission != PermissionDevicesWrite {
return entry, errors.New("permissions contains an unsupported value")
}
if _, exists := permissions[permission]; exists {
return entry, errors.New("permissions contains a duplicate")
}
permissions[permission] = struct{}{}
}
if len(permissions) == 0 {
return entry, errors.New("permissions must not be empty")
}
entry.principal = Principal{
SubjectID: value.SubjectID, ActorType: value.ActorType, TenantID: value.TenantID,
SiteIDs: sites, Permissions: permissions,
}
return entry, nil
}
func (a *StaticSHA256) Authenticate(_ context.Context, token string) (Principal, error) {
if len(token) < 22 || len(token) > 4096 {
return Principal{}, ErrUnauthenticated
}
digest := sha256.Sum256([]byte(token))
match := -1
for index := range a.entries {
if subtle.ConstantTimeCompare(digest[:], a.entries[index].digest[:]) == 1 {
match = index
}
}
if match < 0 {
return Principal{}, ErrUnauthenticated
}
return a.entries[match].principal, nil
}
+57
View File
@@ -0,0 +1,57 @@
package auth
import (
"context"
"crypto/sha256"
"encoding/hex"
"os"
"path/filepath"
"strings"
"testing"
)
func TestStaticSHA256AuthenticatesScopedPrincipal(t *testing.T) {
token := "0123456789abcdef0123456789abcdef"
digest := sha256.Sum256([]byte(token))
path := filepath.Join(t.TempDir(), "auth.json")
document := `{"version":1,"principals":[{` +
`"token_sha256":"` + hex.EncodeToString(digest[:]) + `",` +
`"subject_id":"operator-1","actor_type":"user","tenant_id":"tenant-a",` +
`"site_ids":["site-a"],"permissions":["sense.devices.read"]}]}`
if err := os.WriteFile(path, []byte(document), 0o600); err != nil {
t.Fatal(err)
}
authenticator, err := LoadStaticSHA256(path)
if err != nil {
t.Fatal(err)
}
principal, err := authenticator.Authenticate(context.Background(), token)
if err != nil {
t.Fatal(err)
}
if principal.TenantID != "tenant-a" || !principal.AllowsSite("site-a") ||
principal.AllowsSite("site-b") || !principal.Has(PermissionDevicesRead) {
t.Fatalf("unexpected principal: %+v", principal)
}
if _, err := authenticator.Authenticate(context.Background(), strings.Repeat("x", 32)); err == nil {
t.Fatal("unknown token authenticated")
}
}
func TestStaticSHA256RejectsUnsafeRegistry(t *testing.T) {
tests := []string{
`{"version":1,"principals":[]}`,
`{"version":1,"principals":[{"token_sha256":"ABC","subject_id":"x","actor_type":"service","tenant_id":"tenant","site_ids":["*"],"permissions":["sense.devices.read"]}]}`,
`{"version":1,"principals":[{"token_sha256":"` + strings.Repeat("0", 64) + `","subject_id":"x","actor_type":"system","tenant_id":"tenant","site_ids":["*"],"permissions":["sense.devices.read"]}]}`,
`{"version":1,"principals":[{"token_sha256":"` + strings.Repeat("0", 64) + `","subject_id":"x","actor_type":"service","tenant_id":"tenant","site_ids":["*"],"permissions":["admin"]}]}`,
}
for index, document := range tests {
path := filepath.Join(t.TempDir(), "auth.json")
if err := os.WriteFile(path, []byte(document), 0o600); err != nil {
t.Fatal(err)
}
if _, err := LoadStaticSHA256(path); err == nil {
t.Fatalf("unsafe registry %d was accepted", index)
}
}
}
+37
View File
@@ -6,6 +6,7 @@ import (
"net" "net"
"net/url" "net/url"
"os" "os"
"path/filepath"
"strconv" "strconv"
"strings" "strings"
"time" "time"
@@ -19,6 +20,7 @@ const (
defaultReconcilePeriod = 5 * time.Second defaultReconcilePeriod = 5 * time.Second
defaultProbePeriod = 10 * time.Second defaultProbePeriod = 10 * time.Second
defaultONVIFMode = "disabled" defaultONVIFMode = "disabled"
defaultControlAuthMode = "static-sha256"
) )
type Config struct { type Config struct {
@@ -33,6 +35,11 @@ type Config struct {
RTSPRewriteHost string RTSPRewriteHost string
RTSPRewritePort int RTSPRewritePort int
RTSPStripQuery bool RTSPStripQuery bool
ControlAPIEnabled bool
ControlAuthMode string
ControlAuthFile string
ControlCursorKeyFile string
ControlAllowInsecureHTTP bool
} }
func Load() (Config, error) { func Load() (Config, error) {
@@ -56,6 +63,14 @@ func Load() (Config, error) {
if err != nil { if err != nil {
return Config{}, err return Config{}, err
} }
controlEnabled, err := boolEnv("SENSE_CONTROL_API_ENABLED", false)
if err != nil {
return Config{}, err
}
controlAllowInsecure, err := boolEnv("SENSE_CONTROL_ALLOW_INSECURE_HTTP", false)
if err != nil {
return Config{}, err
}
cfg := Config{ cfg := Config{
HTTPAddress: stringEnv("SENSE_HTTP_ADDR", defaultHTTPAddress), HTTPAddress: stringEnv("SENSE_HTTP_ADDR", defaultHTTPAddress),
@@ -69,6 +84,11 @@ func Load() (Config, error) {
RTSPRewriteHost: stringEnv("SENSE_ONVIF_RTSP_REWRITE_HOST", ""), RTSPRewriteHost: stringEnv("SENSE_ONVIF_RTSP_REWRITE_HOST", ""),
RTSPRewritePort: rewritePort, RTSPRewritePort: rewritePort,
RTSPStripQuery: stripQuery, RTSPStripQuery: stripQuery,
ControlAPIEnabled: controlEnabled,
ControlAuthMode: stringEnv("SENSE_CONTROL_AUTH_MODE", defaultControlAuthMode),
ControlAuthFile: stringEnv("SENSE_CONTROL_AUTH_FILE", ""),
ControlCursorKeyFile: stringEnv("SENSE_CONTROL_CURSOR_KEY_FILE", ""),
ControlAllowInsecureHTTP: controlAllowInsecure,
} }
if err := cfg.Validate(); err != nil { if err := cfg.Validate(); err != nil {
return Config{}, err return Config{}, err
@@ -121,6 +141,23 @@ func (c Config) Validate() error {
return fmt.Errorf("invalid SENSE_ONVIF_RTSP_REWRITE_HOST") return fmt.Errorf("invalid SENSE_ONVIF_RTSP_REWRITE_HOST")
} }
} }
if c.ControlAPIEnabled {
if databaseDriver != "postgres" {
return fmt.Errorf("Sense Control API requires SENSE_DB_DRIVER=postgres")
}
if c.ControlAuthMode != defaultControlAuthMode {
return fmt.Errorf("SENSE_CONTROL_AUTH_MODE must be static-sha256")
}
if c.ControlAuthFile == "" || !filepath.IsAbs(c.ControlAuthFile) {
return fmt.Errorf("SENSE_CONTROL_AUTH_FILE must be an absolute external path")
}
if c.ControlCursorKeyFile == "" || !filepath.IsAbs(c.ControlCursorKeyFile) {
return fmt.Errorf("SENSE_CONTROL_CURSOR_KEY_FILE must be an absolute external path")
}
if !isLoopback && !c.ControlAllowInsecureHTTP {
return fmt.Errorf("non-loopback Control API requires SENSE_CONTROL_ALLOW_INSECURE_HTTP=true")
}
}
return nil return nil
} }
+44 -1
View File
@@ -1,6 +1,9 @@
package config package config
import "testing" import (
"path/filepath"
"testing"
)
func TestValidateRejectsNonLoopbackByDefault(t *testing.T) { func TestValidateRejectsNonLoopbackByDefault(t *testing.T) {
t.Parallel() t.Parallel()
@@ -93,3 +96,43 @@ func TestValidateDatabaseDriver(t *testing.T) {
t.Fatal("unknown database driver must be rejected") t.Fatal("unknown database driver must be rejected")
} }
} }
func TestValidateControlAPIRequiresPostgresAndExternalSecurityFiles(t *testing.T) {
cfg := Config{
HTTPAddress: "127.0.0.1:8080", DatabaseDriver: "sqlite", DatabaseDSN: "file:test.db",
MediaMTXURL: "http://127.0.0.1:9997", ReconcileInterval: 1, ProbeInterval: 1,
ControlAPIEnabled: true, ControlAuthMode: "static-sha256",
ControlAuthFile: filepath.Join(t.TempDir(), "sense-auth.json"),
ControlCursorKeyFile: filepath.Join(t.TempDir(), "sense-cursor.key"),
}
if err := cfg.Validate(); err == nil {
t.Fatal("Control API was accepted on SQLite")
}
cfg.DatabaseDriver = "postgres"
cfg.DatabaseDSN = "postgres://sense-runtime@127.0.0.1/yovision?sslmode=disable"
if err := cfg.Validate(); err != nil {
t.Fatalf("valid Control API configuration failed: %v", err)
}
cfg.ControlAuthFile = "relative-auth.json"
if err := cfg.Validate(); err == nil {
t.Fatal("repository-relative authentication file was accepted")
}
}
func TestValidateControlAPINonLoopbackNeedsSeparateRiskAcceptance(t *testing.T) {
cfg := Config{
HTTPAddress: "0.0.0.0:8080", AllowNonLoopback: true,
DatabaseDriver: "postgres", DatabaseDSN: "postgres://sense-runtime@127.0.0.1/yovision?sslmode=disable",
MediaMTXURL: "http://127.0.0.1:9997", ReconcileInterval: 1, ProbeInterval: 1,
ControlAPIEnabled: true, ControlAuthMode: "static-sha256",
ControlAuthFile: filepath.Join(t.TempDir(), "sense-auth.json"),
ControlCursorKeyFile: filepath.Join(t.TempDir(), "sense-cursor.key"),
}
if err := cfg.Validate(); err == nil {
t.Fatal("non-loopback plaintext Control API was accepted without explicit risk acceptance")
}
cfg.ControlAllowInsecureHTTP = true
if err := cfg.Validate(); err != nil {
t.Fatalf("explicit non-loopback Control API risk acceptance failed: %v", err)
}
}
+5
View File
@@ -0,0 +1,5 @@
// Package controlapi implements the frozen Sense Control API v1.
package controlapi
// The input is the repository-owned public contract frozen by T-008.
//go:generate go tool oapi-codegen -config oapi-codegen.yaml ../../../docs/contracts/sense-control-v1.openapi.json
File diff suppressed because it is too large Load Diff
+42
View File
@@ -0,0 +1,42 @@
package controlapi
import (
"crypto/rand"
"encoding/hex"
"errors"
"math/big"
"time"
)
const crockford = "0123456789ABCDEFGHJKMNPQRSTVWXYZ"
func newULID(prefix string, now time.Time) (string, error) {
value := make([]byte, 16)
milliseconds := uint64(now.UTC().UnixMilli())
value[0] = byte(milliseconds >> 40)
value[1] = byte(milliseconds >> 32)
value[2] = byte(milliseconds >> 24)
value[3] = byte(milliseconds >> 16)
value[4] = byte(milliseconds >> 8)
value[5] = byte(milliseconds)
if _, err := rand.Read(value[6:]); err != nil {
return "", errors.New("generate identifier randomness")
}
number := new(big.Int).SetBytes(value)
base := big.NewInt(32)
remainder := new(big.Int)
encoded := make([]byte, 26)
for index := len(encoded) - 1; index >= 0; index-- {
number.QuoRem(number, base, remainder)
encoded[index] = crockford[remainder.Int64()]
}
return prefix + string(encoded), nil
}
func newTraceID() (string, error) {
value := make([]byte, 16)
if _, err := rand.Read(value); err != nil {
return "", errors.New("generate trace identifier")
}
return "trace_" + hex.EncodeToString(value), nil
}
+66
View File
@@ -0,0 +1,66 @@
package controlapi
import (
"yovision/sense/internal/store"
)
func publicDevice(value store.ControlDevice) Device {
capabilities := make([]Capability, len(value.Capabilities))
for index := range value.Capabilities {
capabilities[index] = Capability(value.Capabilities[index])
}
tenantID := LogicalID(value.TenantID)
generation, observed := value.Generation, value.ObservedGeneration
converged, failureCount := value.Converged, value.FailureCount
endpointConfigured, credentialConfigured := value.EndpointConfigured, value.CredentialConfigured
createdAt, updatedAt := value.CreatedAt, value.UpdatedAt
result := Device{
Id: value.ID, TenantId: &tenantID, SiteId: value.SiteID,
SerialNumber: value.SerialNumber, Name: value.Name, Modality: Modality(value.Modality),
Capabilities: &capabilities, AreaId: value.AreaID,
DesiredState: DesiredState(value.DesiredState), ActualState: ActualState(value.ActualState),
AdapterStatus: AdapterStatus(value.AdapterStatus),
EndpointConfigured: &endpointConfigured, CredentialConfigured: &credentialConfigured,
Generation: &generation, ObservedGeneration: &observed, Converged: &converged,
FailureCount: &failureCount, NextAttemptAt: value.NextAttemptAt,
LastErrorCode: value.LastErrorCode, CreatedAt: &createdAt, UpdatedAt: &updatedAt,
}
if value.ProjectionVersions.QuotaSourceVersion != nil {
version := *value.ProjectionVersions.QuotaSourceVersion
result.ProjectionVersions.QuotaSourceVersion = &version
}
if value.ProjectionVersions.AreaPolicySourceVersion != nil {
version := *value.ProjectionVersions.AreaPolicySourceVersion
result.ProjectionVersions.AreaPolicySourceVersion = &version
}
result.ProjectionVersions.SyncedAt = value.ProjectionVersions.SyncedAt
return result
}
func publicQuota(value store.ControlSiteQuota) SiteQuotaStatus {
return SiteQuotaStatus{
Status: SiteQuotaStatusStatus(value.Status), UsedVideoChannels: value.UsedVideoChannels,
MaxVideoChannels: value.MaxVideoChannels,
AvailableVideoChannels: value.AvailableVideoChannels,
OverLimit: value.OverLimit, SourceVersion: value.SourceVersion, SyncedAt: value.SyncedAt,
}
}
func publicOperation(value store.ControlBatchOperation) BatchOperation {
results := make([]BatchItemResult, 0, len(value.Results))
for _, item := range value.Results {
var code *ErrorCode
if item.ErrorCode != nil {
converted := ErrorCode(*item.ErrorCode)
code = &converted
}
results = append(results, BatchItemResult{
DeviceId: item.DeviceID, Status: BatchItemResultStatus(item.Status),
ErrorCode: code, Message: item.Message, Generation: item.Generation,
})
}
return BatchOperation{
Id: value.ID, Status: BatchOperationStatus(value.Status), SubmittedAt: value.SubmittedAt,
CompletedAt: value.CompletedAt, Results: results, TraceId: value.TraceID,
}
}
@@ -0,0 +1,7 @@
package: controlapi
output: generated.gen.go
generate:
models: true
std-http-server: true
output-options:
skip-prune: false
+108
View File
@@ -0,0 +1,108 @@
package controlapi
import (
"crypto/hmac"
"crypto/sha256"
"crypto/subtle"
"encoding/base64"
"encoding/json"
"errors"
"os"
"strings"
"time"
)
var ErrInvalidCursor = errors.New("invalid cursor")
type CursorPosition struct {
CreatedAt time.Time
DeviceID string
}
type cursorPayload struct {
Version int `json:"v"`
TenantID string `json:"t"`
SiteID string `json:"s"`
FilterHash string `json:"f"`
CreatedAt string `json:"c"`
DeviceID string `json:"d"`
}
type CursorCodec struct {
key []byte
}
func LoadCursorCodec(path string) (*CursorCodec, error) {
contents, err := os.ReadFile(path)
if err != nil {
return nil, errors.New("read Control API cursor key")
}
key, err := base64.RawURLEncoding.DecodeString(strings.TrimSpace(string(contents)))
if err != nil || len(key) < 32 {
return nil, errors.New("Control API cursor key must be base64url for at least 32 bytes")
}
return NewCursorCodec(key), nil
}
func NewCursorCodec(key []byte) *CursorCodec {
copyOfKey := append([]byte(nil), key...)
return &CursorCodec{key: copyOfKey}
}
func (c *CursorCodec) Encode(tenantID, siteID, filterHash string, position CursorPosition) (string, error) {
payload, err := json.Marshal(cursorPayload{
Version: 1, TenantID: tenantID, SiteID: siteID, FilterHash: filterHash,
CreatedAt: position.CreatedAt.UTC().Format(time.RFC3339Nano), DeviceID: position.DeviceID,
})
if err != nil {
return "", errors.New("encode cursor payload")
}
encoded := base64.RawURLEncoding.EncodeToString(payload)
mac := hmac.New(sha256.New, c.key)
_, _ = mac.Write([]byte(encoded))
return encoded + "." + base64.RawURLEncoding.EncodeToString(mac.Sum(nil)), nil
}
func (c *CursorCodec) Decode(value, tenantID, siteID, filterHash string) (CursorPosition, error) {
var position CursorPosition
parts := strings.Split(value, ".")
if len(parts) != 2 || len(value) > 512 {
return position, ErrInvalidCursor
}
provided, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil || len(provided) != sha256.Size {
return position, ErrInvalidCursor
}
mac := hmac.New(sha256.New, c.key)
_, _ = mac.Write([]byte(parts[0]))
if subtle.ConstantTimeCompare(provided, mac.Sum(nil)) != 1 {
return position, ErrInvalidCursor
}
payloadBytes, err := base64.RawURLEncoding.DecodeString(parts[0])
if err != nil {
return position, ErrInvalidCursor
}
var payload cursorPayload
decoder := json.NewDecoder(strings.NewReader(string(payloadBytes)))
decoder.DisallowUnknownFields()
if err := decoder.Decode(&payload); err != nil || payload.Version != 1 ||
payload.TenantID != tenantID || payload.SiteID != siteID || payload.FilterHash != filterHash {
return position, ErrInvalidCursor
}
createdAt, err := time.Parse(time.RFC3339Nano, payload.CreatedAt)
if err != nil || payload.DeviceID == "" {
return position, ErrInvalidCursor
}
position.CreatedAt = createdAt.UTC()
position.DeviceID = payload.DeviceID
return position, nil
}
func filterFingerprint(values ...string) string {
hash := sha256.New()
for _, value := range values {
_, _ = hash.Write([]byte{byte(len(value) >> 8), byte(len(value))})
_, _ = hash.Write([]byte(value))
}
return base64.RawURLEncoding.EncodeToString(hash.Sum(nil))
}
@@ -0,0 +1,58 @@
package controlapi
import (
"bytes"
"strings"
"testing"
"time"
)
func TestCursorIsScopedAndTamperEvident(t *testing.T) {
codec := NewCursorCodec(bytes.Repeat([]byte{7}, 32))
position := CursorPosition{CreatedAt: time.Date(2026, 8, 7, 1, 2, 3, 4, time.UTC), DeviceID: "dev_1"}
value, err := codec.Encode("tenant-a", "site-a", "filters", position)
if err != nil {
t.Fatal(err)
}
decoded, err := codec.Decode(value, "tenant-a", "site-a", "filters")
if err != nil || decoded.DeviceID != position.DeviceID || !decoded.CreatedAt.Equal(position.CreatedAt) {
t.Fatalf("cursor did not round trip: %+v %v", decoded, err)
}
for name, candidate := range map[string]string{
"tenant": "tenant-b", "site": "site-b", "filter": "other",
} {
tenant, site, filter := "tenant-a", "site-a", "filters"
switch name {
case "tenant":
tenant = candidate
case "site":
site = candidate
case "filter":
filter = candidate
}
if _, err := codec.Decode(value, tenant, site, filter); err == nil {
t.Fatalf("cursor was accepted across %s scope", name)
}
}
tampered := value[:len(value)-1] + strings.ToUpper(value[len(value)-1:])
if tampered == value {
tampered = value[:len(value)-1] + "A"
}
if _, err := codec.Decode(tampered, "tenant-a", "site-a", "filters"); err == nil {
t.Fatal("tampered cursor was accepted")
}
}
func TestGeneratedIdentifiersMatchContractShape(t *testing.T) {
value, err := newULID("op_", time.Now())
if err != nil {
t.Fatal(err)
}
if len(value) != 29 || !strings.HasPrefix(value, "op_") {
t.Fatalf("invalid operation ID %q", value)
}
trace, err := newTraceID()
if err != nil || len(trace) != 38 {
t.Fatalf("invalid trace ID %q: %v", trace, err)
}
}
+601
View File
@@ -0,0 +1,601 @@
package controlapi
import (
"context"
"crypto/sha256"
"encoding/json"
"errors"
"net/http"
"strings"
"time"
"yovision/sense/internal/auth"
"yovision/sense/internal/device"
"yovision/sense/internal/store"
)
type principalContextKey struct{}
type traceContextKey struct{}
type Server struct {
store store.ControlRepository
authenticator auth.Authenticator
cursors *CursorCodec
now func() time.Time
}
func NewHTTPHandler(
repository store.ControlRepository, authenticator auth.Authenticator, cursors *CursorCodec,
) http.Handler {
server := &Server{
store: repository, authenticator: authenticator, cursors: cursors, now: time.Now,
}
generated := HandlerWithOptions(server, StdHTTPServerOptions{ErrorHandlerFunc: server.bindError})
return server.authenticate(generated)
}
func (s *Server) authenticate(next http.Handler) http.Handler {
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
traceID, err := newTraceID()
if err != nil {
http.Error(writer, "service unavailable", http.StatusServiceUnavailable)
return
}
ctx := context.WithValue(request.Context(), traceContextKey{}, traceID)
request = request.WithContext(ctx)
values := request.Header.Values("Authorization")
if len(values) != 1 {
writer.Header().Set("WWW-Authenticate", `Bearer realm="sense-control"`)
s.writeProblem(writer, request, http.StatusUnauthorized, ErrorCodeUnauthenticated, "authentication is required")
return
}
parts := strings.Fields(values[0])
if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") {
writer.Header().Set("WWW-Authenticate", `Bearer realm="sense-control"`)
s.writeProblem(writer, request, http.StatusUnauthorized, ErrorCodeUnauthenticated, "authentication is required")
return
}
principal, err := s.authenticator.Authenticate(request.Context(), parts[1])
if err != nil {
writer.Header().Set("WWW-Authenticate", `Bearer realm="sense-control"`)
s.writeProblem(writer, request, http.StatusUnauthorized, ErrorCodeUnauthenticated, "authentication is required")
return
}
ctx = context.WithValue(request.Context(), principalContextKey{}, principal)
next.ServeHTTP(writer, request.WithContext(ctx))
})
}
func principalFromContext(ctx context.Context) auth.Principal {
value, _ := ctx.Value(principalContextKey{}).(auth.Principal)
return value
}
func traceFromContext(ctx context.Context) string {
value, _ := ctx.Value(traceContextKey{}).(string)
return value
}
func (s *Server) bindError(writer http.ResponseWriter, request *http.Request, err error) {
var required *RequiredHeaderError
if errors.As(err, &required) && required.ParamName == "If-Match" {
s.writeProblem(writer, request, http.StatusPreconditionRequired,
ErrorCodePreconditionRequired, "If-Match is required")
return
}
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "request parameters are invalid")
}
func (s *Server) requireSite(
writer http.ResponseWriter, request *http.Request, siteID, permission string,
) (auth.Principal, bool) {
principal := principalFromContext(request.Context())
if !validLogicalID(siteID) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "site_id is invalid")
return auth.Principal{}, false
}
if !principal.AllowsSite(siteID) {
s.writeProblem(writer, request, http.StatusNotFound, ErrorCodeNotFound, "resource was not found")
return auth.Principal{}, false
}
if !principal.Has(permission) {
s.writeProblem(writer, request, http.StatusForbidden, ErrorCodeForbidden, "permission is required")
return auth.Principal{}, false
}
return principal, true
}
func (s *Server) writeProblem(
writer http.ResponseWriter, request *http.Request, status int, code ErrorCode, message string,
) {
traceID := traceFromContext(request.Context())
writer.Header().Set("Content-Type", "application/problem+json")
writer.Header().Set("Cache-Control", "no-store")
writer.Header().Set("X-Trace-ID", traceID)
writer.WriteHeader(status)
_ = json.NewEncoder(writer).Encode(Problem{
Type: "/problems/" + string(code), Title: http.StatusText(status), Status: status,
Code: code, Message: message, TraceId: traceID, FieldErrors: []FieldError{},
})
}
func writeJSON(writer http.ResponseWriter, status int, traceID string, value any) {
writer.Header().Set("Content-Type", "application/json")
writer.Header().Set("Cache-Control", "no-store")
writer.Header().Set("X-Trace-ID", traceID)
writer.WriteHeader(status)
_ = json.NewEncoder(writer).Encode(value)
}
func (s *Server) ListDevices(
writer http.ResponseWriter, request *http.Request, siteID SiteID, params ListDevicesParams,
) {
principal, ok := s.requireSite(writer, request, siteID, auth.PermissionDevicesRead)
if !ok {
return
}
limit := 50
if params.Limit != nil {
limit = *params.Limit
}
if limit < 1 || limit > 100 ||
(params.Modality != nil && !Modality(*params.Modality).Valid()) ||
(params.Capability != nil && !Capability(*params.Capability).Valid()) ||
(params.DesiredState != nil && !DesiredState(*params.DesiredState).Valid()) ||
(params.ActualState != nil && !ActualState(*params.ActualState).Valid()) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "list filters are invalid")
return
}
filterHash := filterFingerprint(
optionalString(params.Modality), optionalString(params.Capability),
optionalString(params.DesiredState), optionalString(params.ActualState),
)
filter := store.ControlListFilter{Limit: limit}
if params.Modality != nil {
value := device.Modality(*params.Modality)
filter.Modality = &value
}
if params.Capability != nil {
value := device.Capability(*params.Capability)
filter.Capability = &value
}
if params.DesiredState != nil {
value := device.DesiredState(*params.DesiredState)
filter.DesiredState = &value
}
if params.ActualState != nil {
value := device.ActualState(*params.ActualState)
filter.ActualState = &value
}
if params.Cursor != nil {
position, err := s.cursors.Decode(*params.Cursor, principal.TenantID, siteID, filterHash)
if err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "cursor is invalid")
return
}
filter.AfterCreated = &position.CreatedAt
filter.AfterDeviceID = position.DeviceID
}
page, err := s.store.ListControlDevices(request.Context(), principal.TenantID, siteID, filter)
if err != nil {
s.writeStoreError(writer, request, err, false)
return
}
items := make([]Device, 0, len(page.Items))
for _, value := range page.Items {
items = append(items, publicDevice(value))
}
var nextCursor *string
if page.HasMore && len(page.Items) > 0 {
last := page.Items[len(page.Items)-1]
value, err := s.cursors.Encode(principal.TenantID, siteID, filterHash,
CursorPosition{CreatedAt: last.CreatedAt, DeviceID: last.ID})
if err != nil {
s.writeProblem(writer, request, http.StatusInternalServerError, ErrorCodeInternalError, "response could not be created")
return
}
nextCursor = &value
}
writeJSON(writer, http.StatusOK, traceFromContext(request.Context()), DevicePage{
Items: items, Page: PageInfo{Limit: limit, HasMore: page.HasMore, NextCursor: nextCursor},
Quota: publicQuota(page.Quota),
})
}
func optionalString[T ~string](value *T) string {
if value == nil {
return ""
}
return string(*value)
}
func (s *Server) CreateDevice(
writer http.ResponseWriter, request *http.Request, siteID SiteID, params CreateDeviceParams,
) {
principal, ok := s.requireSite(writer, request, siteID, auth.PermissionDevicesWrite)
if !ok {
return
}
if !idempotencyKeyRegex.MatchString(params.IdempotencyKey) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "Idempotency-Key is invalid")
return
}
contents, err := readRequestBody(request, "application/json")
if err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
if err := rejectTopLevelNulls(contents); err != nil {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCodeInvalidRequest, err.Error())
return
}
var body DeviceCreate
if err := decodeStrictJSON(contents, &body); err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
if code, message := validateDeviceCreate(body); code != "" {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCode(code), message)
return
}
desired := Disabled
if body.DesiredState != nil {
desired = *body.DesiredState
}
body.DesiredState = &desired
canonical, _ := json.Marshal(body)
requestHash := sha256.Sum256(canonical)
now := s.now().UTC()
deviceID, err := newULID("dev_", now)
if err != nil {
s.writeProblem(writer, request, http.StatusServiceUnavailable, ErrorCodeServiceUnavailable, "identifier could not be generated")
return
}
capabilities := make([]device.Capability, len(body.Capabilities))
for index := range body.Capabilities {
capabilities[index] = device.Capability(body.Capabilities[index])
}
value := device.Device{
ID: deviceID, TenantID: principal.TenantID, SiteID: siteID, AreaID: body.AreaId,
SerialNumber: body.SerialNumber, Name: body.Name, Modality: device.Modality(body.Modality),
Capabilities: capabilities, DesiredState: device.DesiredState(desired),
ActualState: device.ActualPending, PathName: "devices/" + deviceID,
Generation: 1, ResourceVersion: 1,
}
if body.EndpointRef != nil {
value.EndpointRef = *body.EndpointRef
}
if body.CredentialRef != nil {
value.CredentialRef = *body.CredentialRef
}
if body.ProfileToken != nil {
value.ProfileToken = *body.ProfileToken
}
traceID := traceFromContext(request.Context())
ctx := store.WithAuditContext(request.Context(), auditContext(principal, "", traceID))
result, err := s.store.CreateControlDevice(ctx, store.ControlCreateRequest{
Scope: store.IdempotencyScope{
PrincipalID: principal.SubjectID, TenantID: principal.TenantID, SiteID: siteID,
Operation: "createDevice", Key: params.IdempotencyKey,
RequestHash: requestHash, TraceID: traceID,
},
Device: value,
})
if err != nil {
s.writeStoreError(writer, request, err, true)
return
}
writer.Header().Set("ETag", result.ETag)
writer.Header().Set("Location", result.Location)
writeJSON(writer, http.StatusCreated, result.TraceID, publicDevice(result.Device))
}
func validateDeviceCreate(body DeviceCreate) (string, string) {
if !validLogicalID(body.AreaId) || !validLength(body.SerialNumber, 1, 128) ||
!validLength(body.Name, 1, 200) || strings.TrimSpace(body.SerialNumber) == "" ||
strings.TrimSpace(body.Name) == "" || !body.Modality.Valid() ||
len(body.Capabilities) < 1 || len(body.Capabilities) > 16 {
return "invalid_request", "device fields are invalid"
}
seen := make(map[Capability]struct{}, len(body.Capabilities))
hasVideo := false
for _, capability := range body.Capabilities {
if !capability.Valid() {
return "invalid_request", "device capability is invalid"
}
if _, exists := seen[capability]; exists {
return "invalid_request", "device capabilities contain a duplicate"
}
seen[capability] = struct{}{}
hasVideo = hasVideo || capability == VideoCapture
}
if body.Modality == Video && !hasVideo {
return "invalid_request", "video modality requires video_capture"
}
if hasVideo && (body.EndpointRef == nil || body.CredentialRef == nil) {
return "adapter_not_ready", "video capture requires endpoint and credential references"
}
if body.EndpointRef != nil && !validateEndpoint(*body.EndpointRef) {
return "endpoint_credentials_forbidden", "endpoint reference is invalid"
}
if body.CredentialRef != nil && (!validLength(*body.CredentialRef, 1, 512) || strings.TrimSpace(*body.CredentialRef) == "") {
return "endpoint_credentials_forbidden", "credential reference is invalid"
}
if body.ProfileToken != nil && !validLength(*body.ProfileToken, 1, 256) {
return "invalid_request", "profile token is invalid"
}
if body.DesiredState != nil && !body.DesiredState.Valid() {
return "invalid_request", "desired_state is invalid"
}
return "", ""
}
func auditContext(principal auth.Principal, reason, traceID string) store.AuditContext {
actorType := store.AuditActorService
if principal.ActorType == "user" {
actorType = store.AuditActorUser
}
return store.AuditContext{
ActorType: actorType, ActorID: principal.SubjectID, Reason: reason, TraceID: traceID,
}
}
func (s *Server) GetDevice(
writer http.ResponseWriter, request *http.Request, siteID SiteID, deviceID DeviceID,
) {
principal, ok := s.requireSite(writer, request, siteID, auth.PermissionDevicesRead)
if !ok {
return
}
if !validLogicalID(deviceID) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "device_id is invalid")
return
}
value, err := s.store.GetControlDevice(request.Context(), principal.TenantID, siteID, deviceID)
if err != nil {
s.writeStoreError(writer, request, err, false)
return
}
writer.Header().Set("ETag", store.DeviceETag(value.ID, value.ResourceVersion))
writeJSON(writer, http.StatusOK, traceFromContext(request.Context()), publicDevice(value))
}
func (s *Server) UpdateDevice(
writer http.ResponseWriter, request *http.Request, siteID SiteID, deviceID DeviceID,
params UpdateDeviceParams,
) {
principal, ok := s.requireSite(writer, request, siteID, auth.PermissionDevicesWrite)
if !ok {
return
}
if !validLogicalID(deviceID) || !validStrongETag(params.IfMatch) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "device_id or If-Match is invalid")
return
}
contents, err := readRequestBody(request, "application/merge-patch+json")
if err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
var fields map[string]json.RawMessage
if err := json.Unmarshal(contents, &fields); err != nil || len(fields) == 0 {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCodeInvalidRequest, "patch must contain a field")
return
}
for _, value := range fields {
if string(value) == "null" {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCodeInvalidRequest, "patch fields cannot be null")
return
}
}
var body DevicePatch
if err := decodeStrictJSON(contents, &body); err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
if code, message := validateDevicePatch(body); code != "" {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCode(code), message)
return
}
patch := store.ControlPatch{
Name: body.Name, EndpointRef: body.EndpointRef,
CredentialRef: body.CredentialRef, ProfileToken: body.ProfileToken,
}
if body.AreaId != nil {
value := string(*body.AreaId)
patch.AreaID = &value
}
traceID := traceFromContext(request.Context())
ctx := store.WithAuditContext(request.Context(), auditContext(principal, "", traceID))
result, err := s.store.PatchControlDevice(
ctx, principal.TenantID, siteID, deviceID, params.IfMatch, patch,
)
if err != nil {
s.writeStoreError(writer, request, err, true)
return
}
writer.Header().Set("ETag", result.ETag)
writeJSON(writer, http.StatusAccepted, result.TraceID, MutationReceipt{
Device: publicDevice(result.Device), AcceptedAt: result.AcceptedAt, TraceId: result.TraceID,
})
}
func validateDevicePatch(body DevicePatch) (string, string) {
if body.Name != nil && (!validLength(*body.Name, 1, 200) || strings.TrimSpace(*body.Name) == "") {
return "invalid_request", "name is invalid"
}
if body.AreaId != nil && !validLogicalID(*body.AreaId) {
return "invalid_request", "area_id is invalid"
}
if body.EndpointRef != nil && !validateEndpoint(*body.EndpointRef) {
return "endpoint_credentials_forbidden", "endpoint reference is invalid"
}
if body.CredentialRef != nil && (!validLength(*body.CredentialRef, 1, 512) || strings.TrimSpace(*body.CredentialRef) == "") {
return "endpoint_credentials_forbidden", "credential reference is invalid"
}
if body.ProfileToken != nil && !validLength(*body.ProfileToken, 1, 256) {
return "invalid_request", "profile token is invalid"
}
return "", ""
}
func (s *Server) SetDeviceDesiredState(
writer http.ResponseWriter, request *http.Request, siteID SiteID, deviceID DeviceID,
params SetDeviceDesiredStateParams,
) {
principal, ok := s.requireSite(writer, request, siteID, auth.PermissionDevicesWrite)
if !ok {
return
}
if !validLogicalID(deviceID) || !validStrongETag(params.IfMatch) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "device_id or If-Match is invalid")
return
}
contents, err := readRequestBody(request, "application/json")
if err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
var body DesiredStateChange
if err := decodeStrictJSON(contents, &body); err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
if !body.DesiredState.Valid() || !validLength(body.Reason, 1, 500) || strings.TrimSpace(body.Reason) == "" {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCodeInvalidRequest, "desired_state or reason is invalid")
return
}
traceID := traceFromContext(request.Context())
ctx := store.WithAuditContext(request.Context(), auditContext(principal, body.Reason, traceID))
result, err := s.store.SetControlDesiredState(
ctx, principal.TenantID, siteID, deviceID, params.IfMatch,
device.DesiredState(body.DesiredState),
)
if err != nil {
s.writeStoreError(writer, request, err, true)
return
}
writer.Header().Set("ETag", result.ETag)
writeJSON(writer, http.StatusAccepted, result.TraceID, MutationReceipt{
Device: publicDevice(result.Device), AcceptedAt: result.AcceptedAt, TraceId: result.TraceID,
})
}
func (s *Server) BatchSetDeviceDesiredState(
writer http.ResponseWriter, request *http.Request, siteID SiteID,
params BatchSetDeviceDesiredStateParams,
) {
principal, ok := s.requireSite(writer, request, siteID, auth.PermissionDevicesWrite)
if !ok {
return
}
if !idempotencyKeyRegex.MatchString(params.IdempotencyKey) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "Idempotency-Key is invalid")
return
}
contents, err := readRequestBody(request, "application/json")
if err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
var body BatchDesiredStateRequest
if err := decodeStrictJSON(contents, &body); err != nil {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, err.Error())
return
}
if len(body.Items) > 128 {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCodeBatchTooLarge, "batch contains more than 128 items")
return
}
if len(body.Items) == 0 || !validLength(body.Reason, 1, 500) || strings.TrimSpace(body.Reason) == "" {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCodeInvalidRequest, "batch items or reason is invalid")
return
}
items := make([]store.ControlBatchItem, 0, len(body.Items))
for _, item := range body.Items {
if !validLogicalID(item.DeviceId) || !validStrongETag(item.Etag) || !item.DesiredState.Valid() {
s.writeProblem(writer, request, http.StatusUnprocessableEntity, ErrorCodeInvalidRequest, "batch item is invalid")
return
}
items = append(items, store.ControlBatchItem{
DeviceID: item.DeviceId, ETag: item.Etag,
DesiredState: device.DesiredState(item.DesiredState),
})
}
canonical, _ := json.Marshal(body)
requestHash := sha256.Sum256(canonical)
traceID := traceFromContext(request.Context())
ctx := store.WithAuditContext(request.Context(), auditContext(principal, body.Reason, traceID))
operation, err := s.store.BatchSetControlDesiredState(ctx, store.ControlBatchRequest{
Scope: store.IdempotencyScope{
PrincipalID: principal.SubjectID, TenantID: principal.TenantID, SiteID: siteID,
Operation: "batchSetDeviceDesiredState", Key: params.IdempotencyKey,
RequestHash: requestHash, TraceID: traceID,
},
Reason: body.Reason, Items: items,
})
if err != nil {
s.writeStoreError(writer, request, err, true)
return
}
writer.Header().Set("Location", "/api/v1/operations/"+operation.ID)
writeJSON(writer, http.StatusAccepted, operation.TraceID, publicOperation(operation))
}
func (s *Server) GetOperation(
writer http.ResponseWriter, request *http.Request, operationID OperationID,
) {
principal := principalFromContext(request.Context())
if !principal.Has(auth.PermissionDevicesRead) {
s.writeProblem(writer, request, http.StatusForbidden, ErrorCodeForbidden, "permission is required")
return
}
if !operationIDRegex.MatchString(operationID) {
s.writeProblem(writer, request, http.StatusBadRequest, ErrorCodeInvalidRequest, "operation_id is invalid")
return
}
operation, err := s.store.GetControlOperation(request.Context(), principal.TenantID, operationID)
if err != nil {
s.writeStoreError(writer, request, err, false)
return
}
if !principal.AllowsSite(operation.SiteID) {
s.writeProblem(writer, request, http.StatusNotFound, ErrorCodeNotFound, "resource was not found")
return
}
writeJSON(writer, http.StatusOK, traceFromContext(request.Context()), publicOperation(operation))
}
func (s *Server) writeStoreError(
writer http.ResponseWriter, request *http.Request, err error, mutation bool,
) {
status, code, message := http.StatusInternalServerError, ErrorCodeInternalError, "request could not be completed"
switch {
case errors.Is(err, store.ErrNotFound):
status, code, message = http.StatusNotFound, ErrorCodeNotFound, "resource was not found"
case errors.Is(err, store.ErrETagMismatch):
status, code, message = http.StatusPreconditionFailed, ErrorCodeEtagMismatch, "device ETag does not match"
case errors.Is(err, store.ErrIdempotencyConflict):
status, code, message = http.StatusConflict, ErrorCodeIdempotencyConflict, "Idempotency-Key was used with another request"
case errors.Is(err, store.ErrDuplicateSerialNumber):
status, code, message = http.StatusConflict, ErrorCodeDuplicateSerialNumber, "serial_number already exists in this site"
case errors.Is(err, store.ErrAreaPolicyDenied):
status, code, message = http.StatusUnprocessableEntity, ErrorCodeAreaPolicyDenied, "Area policy denies this change"
case errors.Is(err, store.ErrAreaPolicyUnavailable), errors.Is(err, store.ErrAreaPolicyInvalid):
status, code, message = http.StatusServiceUnavailable, ErrorCodeAreaPolicyUnavailable, "Area policy is unavailable"
case errors.Is(err, store.ErrQuotaProjectionUnavailable):
status, code, message = http.StatusServiceUnavailable, ErrorCodeQuotaProjectionUnavailable, "Site quota is unavailable"
case errors.Is(err, store.ErrQuotaProjectionInvalid):
status, code, message = http.StatusServiceUnavailable, ErrorCodeQuotaProjectionInvalid, "Site quota is invalid"
default:
var quotaError *device.QuotaExceededError
if errors.As(err, &quotaError) {
status, code, message = http.StatusConflict, ErrorCodeQuotaExceeded, "Site video channel quota is exceeded"
} else if mutation {
status, code, message = http.StatusServiceUnavailable, ErrorCodeServiceUnavailable, "device change could not be accepted"
}
}
s.writeProblem(writer, request, status, code, message)
}
var _ ServerInterface = (*Server)(nil)
+272
View File
@@ -0,0 +1,272 @@
package controlapi
import (
"bytes"
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"yovision/sense/internal/auth"
"yovision/sense/internal/device"
"yovision/sense/internal/store"
)
const testOperationID = "op_01K20Y8Q3E0000000000000000"
type fixedAuthenticator struct {
principal auth.Principal
err error
}
func (a fixedAuthenticator) Authenticate(context.Context, string) (auth.Principal, error) {
return a.principal, a.err
}
type fakeControlStore struct {
device store.ControlDevice
err error
}
func (f *fakeControlStore) ListControlDevices(context.Context, string, string, store.ControlListFilter) (store.ControlDevicePage, error) {
return store.ControlDevicePage{
Items: []store.ControlDevice{f.device}, Quota: store.ControlSiteQuota{Status: "current"},
}, f.err
}
func (f *fakeControlStore) CreateControlDevice(_ context.Context, request store.ControlCreateRequest) (store.ControlCreateResult, error) {
return store.ControlCreateResult{
Device: f.device, AcceptedAt: time.Now(), TraceID: request.Scope.TraceID,
ETag: store.DeviceETag(f.device.ID, f.device.ResourceVersion),
Location: "/api/v1/sites/site-a/devices/" + f.device.ID,
}, f.err
}
func (f *fakeControlStore) GetControlDevice(context.Context, string, string, string) (store.ControlDevice, error) {
return f.device, f.err
}
func (f *fakeControlStore) PatchControlDevice(_ context.Context, _, _, _, _ string, _ store.ControlPatch) (store.ControlMutationResult, error) {
return f.mutation(), f.err
}
func (f *fakeControlStore) SetControlDesiredState(_ context.Context, _, _, _, _ string, _ device.DesiredState) (store.ControlMutationResult, error) {
return f.mutation(), f.err
}
func (f *fakeControlStore) mutation() store.ControlMutationResult {
return store.ControlMutationResult{
Device: f.device, AcceptedAt: time.Now(), TraceID: "trace_0123456789abcdef0123456789abcdef",
ETag: store.DeviceETag(f.device.ID, f.device.ResourceVersion),
}
}
func (f *fakeControlStore) BatchSetControlDesiredState(_ context.Context, request store.ControlBatchRequest) (store.ControlBatchOperation, error) {
completed := time.Now().UTC()
generation := f.device.Generation
return store.ControlBatchOperation{
ID: testOperationID, TenantID: request.Scope.TenantID, SiteID: request.Scope.SiteID,
Status: "succeeded", SubmittedAt: completed, CompletedAt: &completed,
TraceID: request.Scope.TraceID,
Results: []store.ControlBatchItemResult{{
DeviceID: f.device.ID, Status: "succeeded", Generation: &generation,
}},
}, f.err
}
func (f *fakeControlStore) GetControlOperation(context.Context, string, string) (store.ControlBatchOperation, error) {
completed := time.Now().UTC()
return store.ControlBatchOperation{
ID: testOperationID, TenantID: "tenant-a", SiteID: "site-a", Status: "succeeded",
SubmittedAt: completed, CompletedAt: &completed, Results: []store.ControlBatchItemResult{},
TraceID: "trace_0123456789abcdef0123456789abcdef",
}, f.err
}
func testHTTPHandler(repository *fakeControlStore, permissions ...string) http.Handler {
grants := make(map[string]struct{}, len(permissions))
for _, permission := range permissions {
grants[permission] = struct{}{}
}
principal := auth.Principal{
SubjectID: "operator-1", ActorType: "user", TenantID: "tenant-a",
SiteIDs: []string{"site-a"}, Permissions: grants,
}
return NewHTTPHandler(repository, fixedAuthenticator{principal: principal}, NewCursorCodec(bytes.Repeat([]byte{4}, 32)))
}
func testControlDevice() store.ControlDevice {
quotaVersion, areaVersion := int64(1), int64(2)
now := time.Date(2026, 8, 7, 0, 0, 0, 0, time.UTC)
return store.ControlDevice{
ID: "dev_01K20Y8Q3E0000000000000000", TenantID: "tenant-a", SiteID: "site-a",
SerialNumber: "serial-1", Name: "Camera", Modality: device.ModalityVideo,
Capabilities: []device.Capability{device.CapabilityVideoCapture}, AreaID: "area-a",
DesiredState: device.DesiredDisabled, ActualState: device.ActualPending,
AdapterStatus: "pending", EndpointConfigured: true, CredentialConfigured: true,
Generation: 1, ResourceVersion: 1, FailureCount: 0,
ProjectionVersions: store.ControlProjectionVersions{
QuotaSourceVersion: &quotaVersion, AreaPolicySourceVersion: &areaVersion, SyncedAt: &now,
},
CreatedAt: now, UpdatedAt: now,
}
}
func performRequest(handler http.Handler, method, path, contentType, body string) *httptest.ResponseRecorder {
request := httptest.NewRequest(method, path, strings.NewReader(body))
request.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
if contentType != "" {
request.Header.Set("Content-Type", contentType)
}
recorder := httptest.NewRecorder()
handler.ServeHTTP(recorder, request)
return recorder
}
func TestAllFrozenControlRoutesReturnContractShapes(t *testing.T) {
value := testControlDevice()
repository := &fakeControlStore{device: value}
handler := testHTTPHandler(repository, auth.PermissionDevicesRead, auth.PermissionDevicesWrite)
etag := store.DeviceETag(value.ID, value.ResourceVersion)
list := performRequest(handler, http.MethodGet, "/api/v1/sites/site-a/devices", "", "")
if list.Code != http.StatusOK || !strings.Contains(list.Body.String(), `"quota"`) {
t.Fatalf("list route failed: %d %s", list.Code, list.Body.String())
}
createBody := `{"serial_number":"serial-1","name":"Camera","modality":"video",` +
`"capabilities":["video_capture"],"area_id":"area-a",` +
`"endpoint_ref":"onvif://camera","credential_ref":"env://CAMERA"}`
create := httptest.NewRequest(http.MethodPost, "/api/v1/sites/site-a/devices", strings.NewReader(createBody))
create.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
create.Header.Set("Content-Type", "application/json")
create.Header.Set("Idempotency-Key", "create-request-0001")
createResult := httptest.NewRecorder()
handler.ServeHTTP(createResult, create)
if createResult.Code != http.StatusCreated || createResult.Header().Get("ETag") == "" ||
strings.Contains(createResult.Body.String(), "endpoint_ref") || strings.Contains(createResult.Body.String(), "credential_ref") {
t.Fatalf("create route failed or leaked write-only data: %d %s", createResult.Code, createResult.Body.String())
}
get := performRequest(handler, http.MethodGet, "/api/v1/sites/site-a/devices/"+value.ID, "", "")
if get.Code != http.StatusOK || get.Header().Get("ETag") != etag {
t.Fatalf("get route failed: %d %s", get.Code, get.Body.String())
}
patch := httptest.NewRequest(http.MethodPatch, "/api/v1/sites/site-a/devices/"+value.ID, strings.NewReader(`{"name":"Updated"}`))
patch.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
patch.Header.Set("Content-Type", "application/merge-patch+json")
patch.Header.Set("If-Match", etag)
patchResult := httptest.NewRecorder()
handler.ServeHTTP(patchResult, patch)
if patchResult.Code != http.StatusAccepted || !strings.Contains(patchResult.Body.String(), `"accepted_at"`) {
t.Fatalf("patch route failed: %d %s", patchResult.Code, patchResult.Body.String())
}
desired := httptest.NewRequest(http.MethodPut, "/api/v1/sites/site-a/devices/"+value.ID+"/desired-state", strings.NewReader(`{"desired_state":"enabled","reason":"test"}`))
desired.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
desired.Header.Set("Content-Type", "application/json")
desired.Header.Set("If-Match", etag)
desiredResult := httptest.NewRecorder()
handler.ServeHTTP(desiredResult, desired)
if desiredResult.Code != http.StatusAccepted {
t.Fatalf("desired-state route failed: %d %s", desiredResult.Code, desiredResult.Body.String())
}
batchBody := `{"items":[{"device_id":"` + value.ID + `","etag":"` + strings.ReplaceAll(etag, `"`, `\"`) +
`","desired_state":"enabled"}],"reason":"test"}`
batch := httptest.NewRequest(http.MethodPost, "/api/v1/sites/site-a/devices:batchDesiredState", strings.NewReader(batchBody))
batch.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
batch.Header.Set("Content-Type", "application/json")
batch.Header.Set("Idempotency-Key", "batch-request-0001")
batchResult := httptest.NewRecorder()
handler.ServeHTTP(batchResult, batch)
if batchResult.Code != http.StatusAccepted || batchResult.Header().Get("Location") == "" {
t.Fatalf("batch route failed: %d %s", batchResult.Code, batchResult.Body.String())
}
operation := performRequest(handler, http.MethodGet, "/api/v1/operations/"+testOperationID, "", "")
if operation.Code != http.StatusOK || !strings.Contains(operation.Body.String(), `"results"`) {
t.Fatalf("operation route failed: %d %s", operation.Code, operation.Body.String())
}
}
func TestAuthenticationScopeAndPreconditionsUseStableProblems(t *testing.T) {
value := testControlDevice()
handler := testHTTPHandler(&fakeControlStore{device: value}, auth.PermissionDevicesRead)
unauthenticated := httptest.NewRecorder()
handler.ServeHTTP(unauthenticated, httptest.NewRequest(http.MethodGet, "/api/v1/sites/site-a/devices", nil))
if unauthenticated.Code != http.StatusUnauthorized || unauthenticated.Header().Get("WWW-Authenticate") == "" {
t.Fatalf("missing auth did not return 401: %d", unauthenticated.Code)
}
hidden := performRequest(handler, http.MethodGet, "/api/v1/sites/site-b/devices", "", "")
if hidden.Code != http.StatusNotFound {
t.Fatalf("out-of-scope site was not hidden: %d", hidden.Code)
}
missingPrecondition := performRequest(handler, http.MethodPatch, "/api/v1/sites/site-a/devices/"+value.ID,
"application/merge-patch+json", `{"name":"Updated"}`)
if missingPrecondition.Code != http.StatusPreconditionRequired {
t.Fatalf("missing If-Match did not return 428: %d %s", missingPrecondition.Code, missingPrecondition.Body.String())
}
var problem Problem
if err := json.Unmarshal(missingPrecondition.Body.Bytes(), &problem); err != nil || problem.Code != ErrorCodePreconditionRequired || problem.TraceId == "" {
t.Fatalf("invalid Problem response: %+v %v", problem, err)
}
}
func TestStrictJSONAndStoreConflictsAreMapped(t *testing.T) {
value := testControlDevice()
handler := testHTTPHandler(&fakeControlStore{device: value}, auth.PermissionDevicesWrite)
etag := store.DeviceETag(value.ID, value.ResourceVersion)
duplicate := httptest.NewRequest(http.MethodPatch, "/api/v1/sites/site-a/devices/"+value.ID,
strings.NewReader(`{"name":"one","name":"two"}`))
duplicate.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
duplicate.Header.Set("Content-Type", "application/merge-patch+json")
duplicate.Header.Set("If-Match", etag)
result := httptest.NewRecorder()
handler.ServeHTTP(result, duplicate)
if result.Code != http.StatusBadRequest {
t.Fatalf("duplicate JSON property was accepted: %d %s", result.Code, result.Body.String())
}
nullCreate := httptest.NewRequest(http.MethodPost, "/api/v1/sites/site-a/devices", strings.NewReader(
`{"serial_number":"serial","name":"Camera","modality":"radar",`+
`"capabilities":["telemetry"],"area_id":"area-a","profile_token":null}`))
nullCreate.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
nullCreate.Header.Set("Content-Type", "application/json")
nullCreate.Header.Set("Idempotency-Key", "null-create-0001")
nullResult := httptest.NewRecorder()
handler.ServeHTTP(nullResult, nullCreate)
if nullResult.Code != http.StatusUnprocessableEntity {
t.Fatalf("explicit null was accepted: %d %s", nullResult.Code, nullResult.Body.String())
}
conflictHandler := testHTTPHandler(&fakeControlStore{device: value, err: store.ErrETagMismatch}, auth.PermissionDevicesWrite)
request := httptest.NewRequest(http.MethodPatch, "/api/v1/sites/site-a/devices/"+value.ID, strings.NewReader(`{"name":"two"}`))
request.Header.Set("Authorization", "Bearer 0123456789abcdef0123456789abcdef")
request.Header.Set("Content-Type", "application/merge-patch+json")
request.Header.Set("If-Match", etag)
recorder := httptest.NewRecorder()
conflictHandler.ServeHTTP(recorder, request)
if recorder.Code != http.StatusPreconditionFailed || !strings.Contains(recorder.Body.String(), "etag_mismatch") {
t.Fatalf("ETag mismatch was not mapped: %d %s", recorder.Code, recorder.Body.String())
}
}
func TestProjectionVersionsAlwaysEmitRequiredNullableKeys(t *testing.T) {
encoded, err := json.Marshal(ProjectionVersions{})
if err != nil {
t.Fatal(err)
}
for _, key := range []string{"quota_source_version", "area_policy_source_version", "synced_at"} {
if !strings.Contains(string(encoded), `"`+key+`":null`) {
t.Fatalf("required nullable key %s was omitted: %s", key, encoded)
}
}
}
+153
View File
@@ -0,0 +1,153 @@
package controlapi
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"mime"
"net/http"
"net/url"
"regexp"
"strings"
"unicode/utf8"
)
const maximumRequestBody = 1 << 20
var (
logicalIDRegex = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9._:-]{0,63}$`)
idempotencyKeyRegex = regexp.MustCompile(`^[A-Za-z0-9._:-]{16,128}$`)
operationIDRegex = regexp.MustCompile(`^op_[0-9A-HJKMNP-TV-Z]{26}$`)
strongETagRegex = regexp.MustCompile(`^"[A-Za-z0-9_-]{24}"$`)
)
func readRequestBody(request *http.Request, expectedMediaType string) ([]byte, error) {
mediaType, _, err := mime.ParseMediaType(request.Header.Get("Content-Type"))
if err != nil || mediaType != expectedMediaType {
return nil, fmt.Errorf("Content-Type must be %s", expectedMediaType)
}
contents, err := io.ReadAll(io.LimitReader(request.Body, maximumRequestBody+1))
if err != nil {
return nil, errors.New("read request body")
}
if len(contents) == 0 || len(contents) > maximumRequestBody {
return nil, errors.New("request body is empty or too large")
}
if err := rejectDuplicateJSONKeys(contents); err != nil {
return nil, err
}
return contents, nil
}
func decodeStrictJSON(contents []byte, destination any) error {
decoder := json.NewDecoder(bytes.NewReader(contents))
decoder.DisallowUnknownFields()
if err := decoder.Decode(destination); err != nil {
return errors.New("request body does not match the API schema")
}
if decoder.Decode(&struct{}{}) != io.EOF {
return errors.New("request body contains trailing JSON")
}
return nil
}
func rejectTopLevelNulls(contents []byte) error {
var fields map[string]json.RawMessage
if err := json.Unmarshal(contents, &fields); err != nil || fields == nil {
return errors.New("request body must be a JSON object")
}
for _, value := range fields {
if bytes.Equal(bytes.TrimSpace(value), []byte("null")) {
return errors.New("request body properties cannot be null")
}
}
return nil
}
func rejectDuplicateJSONKeys(contents []byte) error {
decoder := json.NewDecoder(bytes.NewReader(contents))
decoder.UseNumber()
var visit func(int) error
visit = func(depth int) error {
if depth > 64 {
return errors.New("request body nesting is too deep")
}
token, err := decoder.Token()
if err != nil {
return errors.New("request body is not valid JSON")
}
delimiter, ok := token.(json.Delim)
if !ok {
return nil
}
switch delimiter {
case '{':
seen := make(map[string]struct{})
for decoder.More() {
keyToken, err := decoder.Token()
if err != nil {
return errors.New("request body is not valid JSON")
}
key, ok := keyToken.(string)
if !ok {
return errors.New("request body is not a JSON object")
}
if _, exists := seen[key]; exists {
return errors.New("request body contains a duplicate property")
}
seen[key] = struct{}{}
if err := visit(depth + 1); err != nil {
return err
}
}
end, err := decoder.Token()
if err != nil || end != json.Delim('}') {
return errors.New("request body is not valid JSON")
}
case '[':
for decoder.More() {
if err := visit(depth + 1); err != nil {
return err
}
}
end, err := decoder.Token()
if err != nil || end != json.Delim(']') {
return errors.New("request body is not valid JSON")
}
default:
return errors.New("request body is not valid JSON")
}
return nil
}
if err := visit(0); err != nil {
return err
}
if _, err := decoder.Token(); err != io.EOF {
return errors.New("request body contains trailing JSON")
}
return nil
}
func validLogicalID(value string) bool {
return logicalIDRegex.MatchString(value)
}
func validLength(value string, minimum, maximum int) bool {
length := utf8.RuneCountInString(value)
return utf8.ValidString(value) && length >= minimum && length <= maximum
}
func validateEndpoint(value string) bool {
if !validLength(value, 1, 2048) {
return false
}
parsed, err := url.Parse(value)
return err == nil && parsed.Scheme != "" && parsed.User == nil &&
!strings.ContainsAny(value, "\r\n")
}
func validStrongETag(value string) bool {
return strongETagRegex.MatchString(value) && value != "*" && !strings.Contains(value, ",")
}
+19
View File
@@ -0,0 +1,19 @@
package controlapi
import "encoding/json"
// MarshalJSON preserves the OpenAPI-required nullable projection keys. The
// generator represents JSON null as nil pointers but marks read-only pointers
// omitempty, so the default encoder would otherwise violate the v1 wire shape.
func (value ProjectionVersions) MarshalJSON() ([]byte, error) {
type projectionWire struct {
QuotaSourceVersion *int64 `json:"quota_source_version"`
AreaPolicySourceVersion *int64 `json:"area_policy_source_version"`
SyncedAt any `json:"synced_at"`
}
return json.Marshal(projectionWire{
QuotaSourceVersion: value.QuotaSourceVersion,
AreaPolicySourceVersion: value.AreaPolicySourceVersion,
SyncedAt: value.SyncedAt,
})
}
+2
View File
@@ -87,8 +87,10 @@ type Device struct {
ActualState ActualState ActualState ActualState
EndpointRef string EndpointRef string
CredentialRef string CredentialRef string
ProfileToken string
PathName string PathName string
Generation int64 Generation int64
ResourceVersion int64
QuotaSourceVersion int64 QuotaSourceVersion int64
AreaPolicySourceVersion int64 AreaPolicySourceVersion int64
CreatedAt time.Time CreatedAt time.Time
+3 -1
View File
@@ -92,7 +92,9 @@ func (c *Client) DeletePath(ctx context.Context, name string) error {
return fmt.Errorf("MediaMTX delete path transport: %w", err) return fmt.Errorf("MediaMTX delete path transport: %w", err)
} }
if response.StatusCode() == http.StatusNotFound { if response.StatusCode() == http.StatusNotFound {
return ErrPathNotFound // Deletion is an idempotent convergence operation. A missing exact path
// already satisfies the disabled desired state.
return nil
} }
if response.StatusCode() != http.StatusOK { if response.StatusCode() != http.StatusOK {
return &APIError{Operation: "delete path", StatusCode: response.StatusCode()} return &APIError{Operation: "delete path", StatusCode: response.StatusCode()}
+17
View File
@@ -8,6 +8,7 @@ import (
"net/url" "net/url"
"time" "time"
"yovision/sense/internal/device"
"yovision/sense/internal/onvif" "yovision/sense/internal/onvif"
"yovision/sense/internal/store" "yovision/sense/internal/store"
) )
@@ -22,6 +23,7 @@ type Repository interface {
type MediaPaths interface { type MediaPaths interface {
EnsurePath(ctx context.Context, name, source string) (bool, error) EnsurePath(ctx context.Context, name, source string) (bool, error)
DeletePath(ctx context.Context, name string) error
} }
type Reconciler struct { type Reconciler struct {
@@ -60,6 +62,15 @@ func (r *Reconciler) RunOnce(ctx context.Context) error {
} }
func (r *Reconciler) reconcileOne(ctx context.Context, candidate store.ReconcileCandidate, now time.Time) error { func (r *Reconciler) reconcileOne(ctx context.Context, candidate store.ReconcileCandidate, now time.Time) error {
if candidate.Device.DesiredState == device.DesiredDisabled {
if err := r.media.DeletePath(ctx, candidate.Device.PathName); err == nil {
return r.repository.MarkReconciled(ctx, candidate.Device.ID, candidate.Device.Generation, now)
} else if ctx.Err() != nil {
return ctx.Err()
} else {
return r.persistFailure(ctx, candidate, now, err)
}
}
result, err := r.discovery.Probe(ctx, onvif.Target{ result, err := r.discovery.Probe(ctx, onvif.Target{
EndpointRef: candidate.Device.EndpointRef, CredentialRef: candidate.Device.CredentialRef, EndpointRef: candidate.Device.EndpointRef, CredentialRef: candidate.Device.CredentialRef,
}) })
@@ -75,6 +86,12 @@ func (r *Reconciler) reconcileOne(ctx context.Context, candidate store.Reconcile
if ctx.Err() != nil { if ctx.Err() != nil {
return ctx.Err() return ctx.Err()
} }
return r.persistFailure(ctx, candidate, now, err)
}
func (r *Reconciler) persistFailure(
ctx context.Context, candidate store.ReconcileCandidate, now time.Time, err error,
) error {
failureCount := candidate.FailureCount + 1 failureCount := candidate.FailureCount + 1
nextAttempt := now.Add(r.backoff(failureCount)) nextAttempt := now.Add(r.backoff(failureCount))
errorCode := string(onvif.CodeOf(err)) errorCode := string(onvif.CodeOf(err))
@@ -15,6 +15,7 @@ import (
type recordingMedia struct { type recordingMedia struct {
calls int calls int
changed int changed int
deleted int
paths map[string]string paths map[string]string
} }
@@ -31,6 +32,15 @@ func (m *recordingMedia) EnsurePath(_ context.Context, name, source string) (boo
return true, nil return true, nil
} }
func (m *recordingMedia) DeletePath(_ context.Context, name string) error {
m.calls++
if m.paths != nil {
delete(m.paths, name)
}
m.deleted++
return nil
}
func TestReconcileConvergesOnceAndPersistsGeneration(t *testing.T) { func TestReconcileConvergesOnceAndPersistsGeneration(t *testing.T) {
t.Parallel() t.Parallel()
repository := openRepository(t, filepath.Join(t.TempDir(), "sense.db")) repository := openRepository(t, filepath.Join(t.TempDir(), "sense.db"))
@@ -132,6 +142,26 @@ func TestCancellationDoesNotPersistFailure(t *testing.T) {
} }
} }
func TestDisabledDeviceDeletesOnlyItsExactPath(t *testing.T) {
repository := openRepository(t, filepath.Join(t.TempDir(), "sense.db"))
createReconcileDevice(t, repository)
media := &recordingMedia{paths: map[string]string{"camera-1": "source", "other": "keep"}}
if err := repository.SetDesiredState(context.Background(), "camera-1", device.DesiredDisabled); err != nil {
t.Fatal(err)
}
reconciler := New(repository, onvif.NewFake(nil), media)
if err := reconciler.RunOnce(context.Background()); err != nil {
t.Fatal(err)
}
if media.deleted != 1 || media.paths["other"] != "keep" {
t.Fatalf("disabled convergence touched the wrong paths: %+v", media.paths)
}
value, err := repository.GetDevice(context.Background(), "camera-1")
if err != nil || value.ActualState != device.ActualOffline {
t.Fatalf("disabled device did not converge offline: %+v %v", value, err)
}
}
func openRepository(t *testing.T, path string) *store.SQLite { func openRepository(t *testing.T, path string) *store.SQLite {
t.Helper() t.Helper()
repository, err := store.OpenSQLite(context.Background(), "file:"+filepath.ToSlash(path)) repository, err := store.OpenSQLite(context.Background(), "file:"+filepath.ToSlash(path))
+164
View File
@@ -0,0 +1,164 @@
package store
import (
"context"
"crypto/sha256"
"encoding/base64"
"errors"
"strconv"
"time"
"yovision/sense/internal/device"
)
var (
ErrETagMismatch = errors.New("device ETag mismatch")
ErrIdempotencyConflict = errors.New("idempotency key body conflict")
ErrDuplicateSerialNumber = errors.New("duplicate device serial number")
)
type ControlProjectionVersions struct {
QuotaSourceVersion *int64 `json:"quota_source_version"`
AreaPolicySourceVersion *int64 `json:"area_policy_source_version"`
SyncedAt *time.Time `json:"synced_at"`
}
// ControlDevice is deliberately safe to serialize. It contains configured
// booleans, never endpoint, credential or profile-token values.
type ControlDevice struct {
ID string `json:"id"`
TenantID string `json:"tenant_id"`
SiteID string `json:"site_id"`
SerialNumber string `json:"serial_number"`
Name string `json:"name"`
Modality device.Modality `json:"modality"`
Capabilities []device.Capability `json:"capabilities"`
AreaID string `json:"area_id"`
DesiredState device.DesiredState `json:"desired_state"`
ActualState device.ActualState `json:"actual_state"`
AdapterStatus string `json:"adapter_status"`
EndpointConfigured bool `json:"endpoint_configured"`
CredentialConfigured bool `json:"credential_configured"`
Generation int64 `json:"generation"`
ObservedGeneration int64 `json:"observed_generation"`
Converged bool `json:"converged"`
FailureCount int `json:"failure_count"`
NextAttemptAt *time.Time `json:"next_attempt_at,omitempty"`
LastErrorCode *string `json:"last_error_code,omitempty"`
ProjectionVersions ControlProjectionVersions `json:"projection_versions"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
ResourceVersion int64 `json:"-"`
}
type ControlSiteQuota struct {
Status string
UsedVideoChannels int
MaxVideoChannels *int
AvailableVideoChannels *int
OverLimit bool
SourceVersion *int64
SyncedAt *time.Time
}
type ControlListFilter struct {
Limit int
AfterCreated *time.Time
AfterDeviceID string
Modality *device.Modality
Capability *device.Capability
DesiredState *device.DesiredState
ActualState *device.ActualState
}
type ControlDevicePage struct {
Items []ControlDevice
HasMore bool
Quota ControlSiteQuota
}
type IdempotencyScope struct {
PrincipalID string
TenantID string
SiteID string
Operation string
Key string
RequestHash [sha256.Size]byte
TraceID string
}
type ControlCreateRequest struct {
Scope IdempotencyScope
Device device.Device
}
type ControlCreateResult struct {
Device ControlDevice
AcceptedAt time.Time
TraceID string
ETag string
Location string
Replay bool
}
type ControlPatch struct {
Name *string
AreaID *string
EndpointRef *string
CredentialRef *string
ProfileToken *string
}
type ControlMutationResult struct {
Device ControlDevice
AcceptedAt time.Time
TraceID string
ETag string
}
type ControlBatchItem struct {
DeviceID string
ETag string
DesiredState device.DesiredState
}
type ControlBatchRequest struct {
Scope IdempotencyScope
Reason string
Items []ControlBatchItem
}
type ControlBatchItemResult struct {
DeviceID string `json:"device_id"`
Status string `json:"status"`
ErrorCode *string `json:"error_code"`
Message *string `json:"message"`
Generation *int64 `json:"generation"`
}
type ControlBatchOperation struct {
ID string `json:"id"`
TenantID string `json:"-"`
SiteID string `json:"-"`
Status string `json:"status"`
SubmittedAt time.Time `json:"submitted_at"`
CompletedAt *time.Time `json:"completed_at"`
Results []ControlBatchItemResult `json:"results"`
TraceID string `json:"trace_id"`
Replay bool `json:"-"`
}
type ControlRepository interface {
ListControlDevices(context.Context, string, string, ControlListFilter) (ControlDevicePage, error)
CreateControlDevice(context.Context, ControlCreateRequest) (ControlCreateResult, error)
GetControlDevice(context.Context, string, string, string) (ControlDevice, error)
PatchControlDevice(context.Context, string, string, string, string, ControlPatch) (ControlMutationResult, error)
SetControlDesiredState(context.Context, string, string, string, string, device.DesiredState) (ControlMutationResult, error)
BatchSetControlDesiredState(context.Context, ControlBatchRequest) (ControlBatchOperation, error)
GetControlOperation(context.Context, string, string) (ControlBatchOperation, error)
}
func DeviceETag(deviceID string, resourceVersion int64) string {
digest := sha256.Sum256([]byte(deviceID + "\x00" + strconv.FormatInt(resourceVersion, 10)))
return `"` + base64.RawURLEncoding.EncodeToString(digest[:18]) + `"`
}
+889
View File
@@ -0,0 +1,889 @@
package store
import (
"context"
"crypto/rand"
"crypto/sha256"
"crypto/subtle"
"database/sql"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"math/big"
"net/url"
"sort"
"strings"
"time"
"github.com/jackc/pgx/v5/pgconn"
"yovision/sense/internal/device"
)
const controlReceiptTTL = 24 * time.Hour
const controlDeviceColumns = `d.id, d.tenant_id, d.site_id, d.serial_number, d.name, d.modality,
d.area_id, d.desired_state, d.actual_state,
(d.endpoint_ref <> ''), (d.credential_ref <> ''),
d.generation, d.resource_version,
r.observed_generation, r.failure_count, r.next_attempt_at, r.last_error_code,
d.quota_source_version, d.area_policy_source_version,
GREATEST(
(SELECT q.synced_at FROM sense.site_quota_projection_state q
WHERE q.tenant_id = d.tenant_id AND q.site_id = d.site_id
AND q.source_version = d.quota_source_version),
(SELECT a.synced_at FROM sense.area_policy_projection_state a
WHERE a.tenant_id = d.tenant_id AND a.site_id = d.site_id
AND a.area_id = d.area_id AND a.source_version = d.area_policy_source_version)
),
d.created_at, d.updated_at,
COALESCE((SELECT jsonb_agg(c.capability ORDER BY c.capability)
FROM sense.device_capabilities c WHERE c.device_id = d.id), '[]'::jsonb)::text`
const controlDeviceSelect = `SELECT ` + controlDeviceColumns + `
FROM sense.devices d JOIN sense.reconcile_state r ON r.device_id = d.id`
type controlScanner interface {
Scan(...any) error
}
func scanControlDevice(row controlScanner) (ControlDevice, error) {
var value ControlDevice
var nextAttempt, syncedAt sql.NullTime
var lastError sql.NullString
var quotaVersion, areaVersion sql.NullInt64
var capabilitiesJSON string
err := row.Scan(
&value.ID, &value.TenantID, &value.SiteID, &value.SerialNumber, &value.Name,
&value.Modality, &value.AreaID, &value.DesiredState, &value.ActualState,
&value.EndpointConfigured, &value.CredentialConfigured,
&value.Generation, &value.ResourceVersion, &value.ObservedGeneration,
&value.FailureCount, &nextAttempt, &lastError, &quotaVersion, &areaVersion,
&syncedAt, &value.CreatedAt, &value.UpdatedAt, &capabilitiesJSON,
)
if err != nil {
return ControlDevice{}, err
}
if err := json.Unmarshal([]byte(capabilitiesJSON), &value.Capabilities); err != nil {
return ControlDevice{}, errors.New("decode postgres control device capabilities")
}
if value.Capabilities == nil {
value.Capabilities = make([]device.Capability, 0)
}
if nextAttempt.Valid {
point := nextAttempt.Time.UTC()
value.NextAttemptAt = &point
}
if lastError.Valid {
code := lastError.String
value.LastErrorCode = &code
}
if quotaVersion.Valid {
version := quotaVersion.Int64
value.ProjectionVersions.QuotaSourceVersion = &version
}
if areaVersion.Valid {
version := areaVersion.Int64
value.ProjectionVersions.AreaPolicySourceVersion = &version
}
if syncedAt.Valid {
point := syncedAt.Time.UTC()
value.ProjectionVersions.SyncedAt = &point
}
value.Converged = value.ObservedGeneration >= value.Generation && value.FailureCount == 0
value.AdapterStatus = controlAdapterStatus(value)
return value, nil
}
func controlAdapterStatus(value ControlDevice) string {
if value.LastErrorCode != nil {
switch *value.LastErrorCode {
case "authentication_failed":
return "authentication_failed"
case "adapter_not_ready":
return "adapter_not_ready"
default:
return "unavailable"
}
}
if value.Converged {
return "ready"
}
if value.ActualState == device.ActualFailed || value.ActualState == device.ActualOffline {
return "unavailable"
}
return "pending"
}
func (s *Postgres) GetControlDevice(
ctx context.Context, tenantID, siteID, deviceID string,
) (ControlDevice, error) {
value, err := scanControlDevice(s.db.QueryRowContext(ctx, controlDeviceSelect+`
WHERE d.tenant_id = $1 AND d.site_id = $2 AND d.id = $3`, tenantID, siteID, deviceID))
if errors.Is(err, sql.ErrNoRows) {
return ControlDevice{}, ErrNotFound
}
if err != nil {
return ControlDevice{}, errors.New("get postgres control device")
}
return value, nil
}
func (s *Postgres) ListControlDevices(
ctx context.Context, tenantID, siteID string, filter ControlListFilter,
) (ControlDevicePage, error) {
quota, err := s.controlSiteQuota(ctx, tenantID, siteID)
if err != nil {
return ControlDevicePage{}, err
}
query := controlDeviceSelect + ` WHERE d.tenant_id = $1 AND d.site_id = $2`
arguments := []any{tenantID, siteID}
appendCondition := func(clause string, value any) {
arguments = append(arguments, value)
query += fmt.Sprintf(clause, len(arguments))
}
if filter.Modality != nil {
appendCondition(` AND d.modality = $%d`, *filter.Modality)
}
if filter.Capability != nil {
appendCondition(` AND EXISTS (SELECT 1 FROM sense.device_capabilities fc
WHERE fc.device_id = d.id AND fc.capability = $%d)`, *filter.Capability)
}
if filter.DesiredState != nil {
appendCondition(` AND d.desired_state = $%d`, *filter.DesiredState)
}
if filter.ActualState != nil {
appendCondition(` AND d.actual_state = $%d`, *filter.ActualState)
}
if filter.AfterCreated != nil {
arguments = append(arguments, filter.AfterCreated.UTC(), filter.AfterDeviceID)
query += fmt.Sprintf(` AND (d.created_at, d.id) > ($%d, $%d)`, len(arguments)-1, len(arguments))
}
arguments = append(arguments, filter.Limit+1)
query += fmt.Sprintf(` ORDER BY d.created_at ASC, d.id ASC LIMIT $%d`, len(arguments))
rows, err := s.db.QueryContext(ctx, query, arguments...)
if err != nil {
return ControlDevicePage{}, errors.New("list postgres control devices")
}
defer rows.Close()
values := make([]ControlDevice, 0, filter.Limit+1)
for rows.Next() {
value, scanErr := scanControlDevice(rows)
if scanErr != nil {
return ControlDevicePage{}, errors.New("scan postgres control device page")
}
values = append(values, value)
}
if err := rows.Err(); err != nil {
return ControlDevicePage{}, errors.New("iterate postgres control device page")
}
hasMore := len(values) > filter.Limit
if hasMore {
values = values[:filter.Limit]
}
return ControlDevicePage{Items: values, HasMore: hasMore, Quota: quota}, nil
}
func (s *Postgres) controlSiteQuota(ctx context.Context, tenantID, siteID string) (ControlSiteQuota, error) {
var maximum int
var sourceVersion int64
var syncedAt time.Time
err := s.db.QueryRowContext(ctx, `SELECT max_video_channels, source_version, source_updated_at
FROM bell.site_quota_v1 WHERE tenant_id = $1 AND site_id = $2`, tenantID, siteID).
Scan(&maximum, &sourceVersion, &syncedAt)
if errors.Is(err, sql.ErrNoRows) {
return ControlSiteQuota{}, ErrNotFound
}
if err != nil {
return ControlSiteQuota{}, errors.New("read postgres control site quota")
}
var used int
if err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM sense.devices d
WHERE d.tenant_id = $1 AND d.site_id = $2 AND d.desired_state = 'enabled'
AND EXISTS (SELECT 1 FROM sense.device_capabilities c
WHERE c.device_id = d.id AND c.capability = 'video_capture')`, tenantID, siteID).Scan(&used); err != nil {
return ControlSiteQuota{}, errors.New("count postgres control site video channels")
}
status := "current"
if maximum < 1 || maximum > device.MaximumVideoChannels || sourceVersion < 1 || syncedAt.IsZero() {
status = "invalid"
return ControlSiteQuota{Status: status, UsedVideoChannels: used, OverLimit: false}, nil
}
available := maximum - used
if available < 0 {
available = 0
}
point := syncedAt.UTC()
return ControlSiteQuota{
Status: status, UsedVideoChannels: used, MaxVideoChannels: &maximum,
AvailableVideoChannels: &available, OverLimit: used > maximum,
SourceVersion: &sourceVersion, SyncedAt: &point,
}, nil
}
type controlReceipt struct {
Status int
Body []byte
ETag string
Location string
TraceID string
CreatedAt time.Time
}
func controlScopeHash(scope IdempotencyScope) [sha256.Size]byte {
encoded, _ := json.Marshal([]string{
scope.PrincipalID, scope.TenantID, scope.SiteID, scope.Operation, scope.Key,
})
return sha256.Sum256(encoded)
}
func readControlReceipt(
ctx context.Context, tx *sql.Tx, scope IdempotencyScope, now time.Time,
) (controlReceipt, bool, error) {
var receipt controlReceipt
scopeHash := controlScopeHash(scope)
if _, err := tx.ExecContext(ctx, `SELECT pg_advisory_xact_lock(hashtext($1))`, hex.EncodeToString(scopeHash[:])); err != nil {
return receipt, false, errors.New("lock postgres Control API idempotency scope")
}
var storedRequestHash []byte
var etag, location sql.NullString
var expiresAt time.Time
err := tx.QueryRowContext(ctx, `SELECT request_hash, response_status, response_body::text,
response_etag, response_location, trace_id, created_at, expires_at
FROM sense.control_idempotency_receipts WHERE scope_hash = $1`, scopeHash[:]).
Scan(&storedRequestHash, &receipt.Status, &receipt.Body, &etag, &location,
&receipt.TraceID, &receipt.CreatedAt, &expiresAt)
if errors.Is(err, sql.ErrNoRows) {
return receipt, false, nil
}
if err != nil {
return receipt, false, errors.New("read postgres Control API idempotency receipt")
}
if !expiresAt.After(now) {
if _, err := tx.ExecContext(ctx, `DELETE FROM sense.control_idempotency_receipts
WHERE scope_hash = $1`, scopeHash[:]); err != nil {
return receipt, false, errors.New("expire postgres Control API idempotency receipt")
}
return controlReceipt{}, false, nil
}
if subtle.ConstantTimeCompare(storedRequestHash, scope.RequestHash[:]) != 1 {
return receipt, false, ErrIdempotencyConflict
}
receipt.ETag = etag.String
receipt.Location = location.String
return receipt, true, nil
}
func writeControlReceipt(
ctx context.Context, tx *sql.Tx, scope IdempotencyScope, receipt controlReceipt,
) error {
scopeHash := controlScopeHash(scope)
_, err := tx.ExecContext(ctx, `INSERT INTO sense.control_idempotency_receipts(
scope_hash, request_hash, operation_name, principal_id, tenant_id, site_id,
response_status, response_body, response_etag, response_location, trace_id,
created_at, expires_at
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, NULLIF($9, ''), NULLIF($10, ''), $11, $12, $13)`,
scopeHash[:], scope.RequestHash[:], scope.Operation, scope.PrincipalID,
scope.TenantID, scope.SiteID, receipt.Status, string(receipt.Body),
receipt.ETag, receipt.Location, receipt.TraceID, receipt.CreatedAt,
receipt.CreatedAt.Add(controlReceiptTTL))
if err != nil {
return errors.New("write postgres Control API idempotency receipt")
}
// Bound opportunistic cleanup; never scans or deletes unexpired receipts.
_, _ = tx.ExecContext(ctx, `DELETE FROM sense.control_idempotency_receipts
WHERE scope_hash IN (SELECT scope_hash FROM sense.control_idempotency_receipts
WHERE expires_at <= $1 ORDER BY expires_at LIMIT 32)`, receipt.CreatedAt)
return nil
}
func (s *Postgres) CreateControlDevice(
ctx context.Context, request ControlCreateRequest,
) (ControlCreateResult, error) {
now := time.Now().UTC()
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return ControlCreateResult{}, errors.New("begin postgres Control API device create")
}
defer tx.Rollback()
receipt, found, err := readControlReceipt(ctx, tx, request.Scope, now)
if err != nil {
return ControlCreateResult{}, err
}
if found {
var value ControlDevice
if err := json.Unmarshal(receipt.Body, &value); err != nil {
return ControlCreateResult{}, errors.New("decode postgres device creation receipt")
}
if err := tx.Commit(); err != nil {
return ControlCreateResult{}, errors.New("commit postgres device creation replay")
}
return ControlCreateResult{
Device: value, AcceptedAt: receipt.CreatedAt, TraceID: receipt.TraceID,
ETag: receipt.ETag, Location: receipt.Location, Replay: true,
}, nil
}
value := request.Device
if value.Generation == 0 {
value.Generation = 1
}
if value.ResourceVersion == 0 {
value.ResourceVersion = 1
}
if value.ActualState == "" {
value.ActualState = device.ActualPending
}
value.CreatedAt = now
value.UpdatedAt = now
if err := createControlDeviceTx(ctx, tx, value, now); err != nil {
return ControlCreateResult{}, err
}
created, err := scanControlDevice(tx.QueryRowContext(ctx, controlDeviceSelect+`
WHERE d.tenant_id = $1 AND d.site_id = $2 AND d.id = $3`, value.TenantID, value.SiteID, value.ID))
if err != nil {
return ControlCreateResult{}, errors.New("read created postgres control device")
}
responseBody, err := json.Marshal(created)
if err != nil {
return ControlCreateResult{}, errors.New("encode created postgres control device")
}
etag := DeviceETag(created.ID, created.ResourceVersion)
location := "/api/v1/sites/" + url.PathEscape(created.SiteID) + "/devices/" + url.PathEscape(created.ID)
receipt = controlReceipt{
Status: 201, Body: responseBody, ETag: etag, Location: location,
TraceID: request.Scope.TraceID, CreatedAt: now,
}
if err := writeControlReceipt(ctx, tx, request.Scope, receipt); err != nil {
return ControlCreateResult{}, err
}
if err := tx.Commit(); err != nil {
return ControlCreateResult{}, errors.New("commit postgres Control API device create")
}
return ControlCreateResult{
Device: created, AcceptedAt: now, TraceID: receipt.TraceID,
ETag: etag, Location: location,
}, nil
}
func createControlDeviceTx(ctx context.Context, tx *sql.Tx, value device.Device, now time.Time) error {
if err := value.Validate(); err != nil {
return fmt.Errorf("validate Control API device: %w", err)
}
areaVersion, err := checkPostgresAreaPolicy(
ctx, tx, value.TenantID, value.SiteID, value.AreaID,
value.HasCapability(device.CapabilityVideoCapture), now,
)
if err != nil {
return err
}
var quotaVersion int64
if value.ConsumesVideoChannel() {
quotaVersion, err = checkPostgresVideoQuota(ctx, tx, value.TenantID, value.SiteID, now)
if err != nil {
return err
}
}
_, err = tx.ExecContext(ctx, `INSERT INTO sense.devices(
id, tenant_id, site_id, area_id, serial_number, name, modality,
desired_state, actual_state, endpoint_ref, credential_ref, profile_token,
path_name, generation, resource_version, quota_source_version,
area_policy_source_version, created_at, updated_at
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19)`,
value.ID, value.TenantID, value.SiteID, value.AreaID, value.SerialNumber, value.Name,
value.Modality, value.DesiredState, value.ActualState, value.EndpointRef,
value.CredentialRef, value.ProfileToken, value.PathName, value.Generation,
value.ResourceVersion, nullableVersion(quotaVersion), areaVersion,
value.CreatedAt, value.UpdatedAt)
if err != nil {
var postgresError *pgconn.PgError
if errors.As(err, &postgresError) && postgresError.Code == "23505" &&
strings.Contains(postgresError.ConstraintName, "serial_number") {
return ErrDuplicateSerialNumber
}
return errors.New("insert postgres Control API device")
}
for _, capability := range sortedCapabilities(value.Capabilities) {
if _, err := tx.ExecContext(ctx, `INSERT INTO sense.device_capabilities(device_id, capability)
VALUES ($1, $2)`, value.ID, capability); err != nil {
return errors.New("insert postgres Control API device capability")
}
}
if _, err := tx.ExecContext(ctx, `INSERT INTO sense.reconcile_state(device_id, updated_at)
VALUES ($1, $2)`, value.ID, now); err != nil {
return errors.New("insert postgres Control API reconcile state")
}
return insertPostgresAudit(ctx, tx, postgresAuditEvent{
EventType: "device.created", TenantID: value.TenantID, SiteID: value.SiteID,
DeviceID: value.ID, Generation: value.Generation,
QuotaSourceVersion: quotaVersion, AreaPolicySourceVersion: areaVersion,
OccurredAt: now,
Payload: map[string]any{
"kind": "device_created", "area_id": value.AreaID, "modality": value.Modality,
"capabilities": sortedCapabilities(value.Capabilities), "desired_state": value.DesiredState,
},
})
}
var _ ControlRepository = (*Postgres)(nil)
func (s *Postgres) PatchControlDevice(
ctx context.Context, tenantID, siteID, deviceID, expectedETag string, patch ControlPatch,
) (ControlMutationResult, error) {
now := time.Now().UTC()
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return ControlMutationResult{}, errors.New("begin postgres Control API device patch")
}
defer tx.Rollback()
var name, areaID, endpointRef, credentialRef, profileToken string
var generation, resourceVersion int64
var desired device.DesiredState
var quotaVersion, areaVersion sql.NullInt64
var hasVideo bool
err = tx.QueryRowContext(ctx, `SELECT d.name, d.area_id, d.endpoint_ref,
d.credential_ref, d.profile_token, d.generation, d.resource_version,
d.desired_state, d.quota_source_version, d.area_policy_source_version,
EXISTS (SELECT 1 FROM sense.device_capabilities c
WHERE c.device_id = d.id AND c.capability = 'video_capture')
FROM sense.devices d
WHERE d.tenant_id = $1 AND d.site_id = $2 AND d.id = $3 FOR UPDATE`,
tenantID, siteID, deviceID).Scan(
&name, &areaID, &endpointRef, &credentialRef, &profileToken,
&generation, &resourceVersion, &desired, &quotaVersion, &areaVersion, &hasVideo,
)
if errors.Is(err, sql.ErrNoRows) {
return ControlMutationResult{}, ErrNotFound
}
if err != nil {
return ControlMutationResult{}, errors.New("read postgres Control API device patch state")
}
if DeviceETag(deviceID, resourceVersion) != expectedETag {
return ControlMutationResult{}, ErrETagMismatch
}
changedFields := make([]string, 0, 5)
reconcileChanged := false
if patch.Name != nil && *patch.Name != name {
name = *patch.Name
changedFields = append(changedFields, "name")
}
if patch.AreaID != nil && *patch.AreaID != areaID {
version, policyErr := checkPostgresAreaPolicy(
ctx, tx, tenantID, siteID, *patch.AreaID, hasVideo, now,
)
if policyErr != nil {
return ControlMutationResult{}, policyErr
}
areaID = *patch.AreaID
areaVersion = sql.NullInt64{Int64: version, Valid: true}
changedFields = append(changedFields, "area_id")
}
if patch.EndpointRef != nil && *patch.EndpointRef != endpointRef {
endpointRef = *patch.EndpointRef
changedFields = append(changedFields, "endpoint_ref")
reconcileChanged = true
}
if patch.CredentialRef != nil && *patch.CredentialRef != credentialRef {
credentialRef = *patch.CredentialRef
changedFields = append(changedFields, "credential_ref")
reconcileChanged = true
}
if patch.ProfileToken != nil && *patch.ProfileToken != profileToken {
profileToken = *patch.ProfileToken
changedFields = append(changedFields, "profile_token")
reconcileChanged = true
}
if len(changedFields) > 0 {
resourceVersion++
if reconcileChanged {
generation++
}
_, err = tx.ExecContext(ctx, `UPDATE sense.devices SET
name = $1, area_id = $2, endpoint_ref = $3, credential_ref = $4,
profile_token = $5, generation = $6, resource_version = $7,
area_policy_source_version = $8,
actual_state = CASE WHEN $9 THEN 'pending' ELSE actual_state END,
updated_at = $10
WHERE tenant_id = $11 AND site_id = $12 AND id = $13`,
name, areaID, endpointRef, credentialRef, profileToken, generation,
resourceVersion, nullableVersion(areaVersion.Int64), reconcileChanged, now,
tenantID, siteID, deviceID)
if err != nil {
return ControlMutationResult{}, errors.New("update postgres Control API device configuration")
}
if reconcileChanged {
if _, err := tx.ExecContext(ctx, `UPDATE sense.reconcile_state SET
failure_count = 0, next_attempt_at = NULL, last_error_code = NULL,
updated_at = $1 WHERE device_id = $2`, now, deviceID); err != nil {
return ControlMutationResult{}, errors.New("reset postgres Control API reconcile state")
}
}
}
if err := insertPostgresAudit(ctx, tx, postgresAuditEvent{
EventType: "device.configuration.accepted", TenantID: tenantID, SiteID: siteID,
DeviceID: deviceID, Generation: generation,
QuotaSourceVersion: quotaVersion.Int64, AreaPolicySourceVersion: areaVersion.Int64,
OccurredAt: now,
Payload: map[string]any{
"kind": "configuration_accepted", "changed": len(changedFields) > 0,
"changed_fields": changedFields, "area_id": areaID,
},
}); err != nil {
return ControlMutationResult{}, err
}
updated, err := scanControlDevice(tx.QueryRowContext(ctx, controlDeviceSelect+`
WHERE d.tenant_id = $1 AND d.site_id = $2 AND d.id = $3`, tenantID, siteID, deviceID))
if err != nil {
return ControlMutationResult{}, errors.New("read patched postgres control device")
}
if err := tx.Commit(); err != nil {
return ControlMutationResult{}, errors.New("commit postgres Control API device patch")
}
return ControlMutationResult{
Device: updated, AcceptedAt: now, TraceID: auditFromContext(ctx).TraceID,
ETag: DeviceETag(updated.ID, updated.ResourceVersion),
}, nil
}
func (s *Postgres) SetControlDesiredState(
ctx context.Context, tenantID, siteID, deviceID, expectedETag string, desired device.DesiredState,
) (ControlMutationResult, error) {
now := time.Now().UTC()
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return ControlMutationResult{}, errors.New("begin postgres Control API desired-state update")
}
defer tx.Rollback()
updated, err := setControlDesiredStateTx(
ctx, tx, tenantID, siteID, deviceID, expectedETag, desired, now,
)
if err != nil {
return ControlMutationResult{}, err
}
if err := tx.Commit(); err != nil {
return ControlMutationResult{}, errors.New("commit postgres Control API desired-state update")
}
return ControlMutationResult{
Device: updated, AcceptedAt: now, TraceID: auditFromContext(ctx).TraceID,
ETag: DeviceETag(updated.ID, updated.ResourceVersion),
}, nil
}
func setControlDesiredStateTx(
ctx context.Context, tx *sql.Tx, tenantID, siteID, deviceID, expectedETag string,
desired device.DesiredState, now time.Time,
) (ControlDevice, error) {
var areaID, endpointRef, pathName string
var current device.DesiredState
var generation, resourceVersion int64
var quotaVersion, areaVersion sql.NullInt64
var hasVideo bool
err := tx.QueryRowContext(ctx, `SELECT d.area_id, d.desired_state, d.endpoint_ref,
d.path_name, d.generation, d.resource_version, d.quota_source_version,
d.area_policy_source_version,
EXISTS (SELECT 1 FROM sense.device_capabilities c
WHERE c.device_id = d.id AND c.capability = 'video_capture')
FROM sense.devices d
WHERE d.tenant_id = $1 AND d.site_id = $2 AND d.id = $3 FOR UPDATE`,
tenantID, siteID, deviceID).Scan(
&areaID, &current, &endpointRef, &pathName, &generation, &resourceVersion,
&quotaVersion, &areaVersion, &hasVideo,
)
if errors.Is(err, sql.ErrNoRows) {
return ControlDevice{}, ErrNotFound
}
if err != nil {
return ControlDevice{}, errors.New("read postgres Control API desired state")
}
if DeviceETag(deviceID, resourceVersion) != expectedETag {
return ControlDevice{}, ErrETagMismatch
}
if current != desired {
var admittedQuota, admittedArea int64
if desired == device.DesiredEnabled && hasVideo {
if strings.TrimSpace(endpointRef) == "" || strings.TrimSpace(pathName) == "" {
return ControlDevice{}, errors.New("video adapter configuration is incomplete")
}
admittedArea, err = checkPostgresAreaPolicy(ctx, tx, tenantID, siteID, areaID, true, now)
if err != nil {
return ControlDevice{}, err
}
admittedQuota, err = checkPostgresVideoQuota(ctx, tx, tenantID, siteID, now)
if err != nil {
return ControlDevice{}, err
}
}
generation++
resourceVersion++
err = tx.QueryRowContext(ctx, `UPDATE sense.devices SET
desired_state = $1, actual_state = 'pending', generation = $2,
resource_version = $3,
quota_source_version = COALESCE($4, quota_source_version),
area_policy_source_version = COALESCE($5, area_policy_source_version),
updated_at = $6
WHERE tenant_id = $7 AND site_id = $8 AND id = $9
RETURNING quota_source_version, area_policy_source_version`,
desired, generation, resourceVersion, nullableVersion(admittedQuota),
nullableVersion(admittedArea), now, tenantID, siteID, deviceID).
Scan(&quotaVersion, &areaVersion)
if err != nil {
return ControlDevice{}, errors.New("update postgres Control API desired state")
}
if _, err := tx.ExecContext(ctx, `UPDATE sense.reconcile_state SET
failure_count = 0, next_attempt_at = NULL, last_error_code = NULL,
updated_at = $1 WHERE device_id = $2`, now, deviceID); err != nil {
return ControlDevice{}, errors.New("reset postgres Control API desired-state reconciliation")
}
}
if err := insertPostgresAudit(ctx, tx, postgresAuditEvent{
EventType: "device.desired_state.accepted", TenantID: tenantID, SiteID: siteID,
DeviceID: deviceID, Generation: generation,
QuotaSourceVersion: quotaVersion.Int64, AreaPolicySourceVersion: areaVersion.Int64,
OccurredAt: now,
Payload: map[string]any{
"kind": "desired_state_accepted", "previous_desired_state": current,
"desired_state": desired, "changed": current != desired,
},
}); err != nil {
return ControlDevice{}, err
}
value, err := scanControlDevice(tx.QueryRowContext(ctx, controlDeviceSelect+`
WHERE d.tenant_id = $1 AND d.site_id = $2 AND d.id = $3`, tenantID, siteID, deviceID))
if err != nil {
return ControlDevice{}, errors.New("read updated postgres control desired state")
}
return value, nil
}
func (s *Postgres) BatchSetControlDesiredState(
ctx context.Context, request ControlBatchRequest,
) (ControlBatchOperation, error) {
now := time.Now().UTC()
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return ControlBatchOperation{}, errors.New("begin postgres Control API batch")
}
defer tx.Rollback()
receipt, found, err := readControlReceipt(ctx, tx, request.Scope, now)
if err != nil {
return ControlBatchOperation{}, err
}
if found {
var operation ControlBatchOperation
if err := json.Unmarshal(receipt.Body, &operation); err != nil {
return ControlBatchOperation{}, errors.New("decode postgres Control API batch receipt")
}
operation.TenantID = request.Scope.TenantID
operation.SiteID = request.Scope.SiteID
operation.Replay = true
if err := tx.Commit(); err != nil {
return ControlBatchOperation{}, errors.New("commit postgres Control API batch replay")
}
return operation, nil
}
operationID, err := newControlOperationID(now)
if err != nil {
return ControlBatchOperation{}, err
}
counts := make(map[string]int, len(request.Items))
for _, item := range request.Items {
counts[item.DeviceID]++
}
if err := lockControlBatchDevices(
ctx, tx, request.Scope.TenantID, request.Scope.SiteID, counts,
); err != nil {
return ControlBatchOperation{}, err
}
results := make([]ControlBatchItemResult, 0, len(request.Items))
succeeded := 0
for index, item := range request.Items {
if counts[item.DeviceID] > 1 {
code, message := "invalid_request", "device_id is duplicated in this request"
results = append(results, ControlBatchItemResult{
DeviceID: item.DeviceID, Status: "rejected", ErrorCode: &code, Message: &message,
})
continue
}
savepoint := fmt.Sprintf("control_batch_%d", index)
if _, err := tx.ExecContext(ctx, "SAVEPOINT "+savepoint); err != nil {
return ControlBatchOperation{}, errors.New("create postgres Control API batch savepoint")
}
updated, itemErr := setControlDesiredStateTx(
ctx, tx, request.Scope.TenantID, request.Scope.SiteID,
item.DeviceID, item.ETag, item.DesiredState, now,
)
if itemErr != nil {
if _, rollbackErr := tx.ExecContext(ctx, "ROLLBACK TO SAVEPOINT "+savepoint); rollbackErr != nil {
return ControlBatchOperation{}, errors.New("rollback postgres Control API batch item")
}
code, status, message := controlBatchError(itemErr)
results = append(results, ControlBatchItemResult{
DeviceID: item.DeviceID, Status: status, ErrorCode: &code, Message: &message,
})
} else {
generation := updated.Generation
results = append(results, ControlBatchItemResult{
DeviceID: item.DeviceID, Status: "succeeded", Generation: &generation,
})
succeeded++
}
if _, err := tx.ExecContext(ctx, "RELEASE SAVEPOINT "+savepoint); err != nil {
return ControlBatchOperation{}, errors.New("release postgres Control API batch savepoint")
}
}
status := "partially_succeeded"
if succeeded == len(results) {
status = "succeeded"
} else if succeeded == 0 {
status = "failed"
}
completedAt := now
operation := ControlBatchOperation{
ID: operationID, TenantID: request.Scope.TenantID, SiteID: request.Scope.SiteID,
Status: status, SubmittedAt: now, CompletedAt: &completedAt,
Results: results, TraceID: request.Scope.TraceID,
}
if _, err := tx.ExecContext(ctx, `INSERT INTO sense.batch_operations(
id, tenant_id, site_id, principal_id, status, trace_id, submitted_at, completed_at
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8)`, operation.ID, operation.TenantID,
operation.SiteID, request.Scope.PrincipalID, operation.Status, operation.TraceID,
operation.SubmittedAt, operation.CompletedAt); err != nil {
return ControlBatchOperation{}, errors.New("insert postgres Control API batch operation")
}
for index, result := range results {
if _, err := tx.ExecContext(ctx, `INSERT INTO sense.batch_operation_items(
operation_id, ordinal, device_id, status, error_code, message, generation
) VALUES ($1,$2,$3,$4,$5,$6,$7)`, operation.ID, index, result.DeviceID,
result.Status, result.ErrorCode, result.Message, result.Generation); err != nil {
return ControlBatchOperation{}, errors.New("insert postgres Control API batch item")
}
}
body, err := json.Marshal(operation)
if err != nil {
return ControlBatchOperation{}, errors.New("encode postgres Control API batch operation")
}
receipt = controlReceipt{
Status: 202, Body: body,
Location: "/api/v1/operations/" + url.PathEscape(operation.ID),
TraceID: operation.TraceID, CreatedAt: now,
}
if err := writeControlReceipt(ctx, tx, request.Scope, receipt); err != nil {
return ControlBatchOperation{}, err
}
if err := tx.Commit(); err != nil {
return ControlBatchOperation{}, errors.New("commit postgres Control API batch")
}
return operation, nil
}
func lockControlBatchDevices(
ctx context.Context, tx *sql.Tx, tenantID, siteID string, deviceIDs map[string]int,
) error {
ordered := make([]string, 0, len(deviceIDs))
for deviceID := range deviceIDs {
ordered = append(ordered, deviceID)
}
sort.Strings(ordered)
for _, deviceID := range ordered {
var locked string
err := tx.QueryRowContext(ctx, `SELECT id FROM sense.devices
WHERE tenant_id = $1 AND site_id = $2 AND id = $3 FOR UPDATE`,
tenantID, siteID, deviceID).Scan(&locked)
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return errors.New("lock postgres Control API batch devices")
}
}
return nil
}
func controlBatchError(err error) (code, status, message string) {
switch {
case errors.Is(err, ErrNotFound):
return "not_found", "rejected", "device was not found"
case errors.Is(err, ErrETagMismatch):
return "etag_mismatch", "rejected", "device ETag does not match"
case errors.Is(err, ErrAreaPolicyDenied):
return "area_policy_denied", "rejected", "Area policy denies this change"
case errors.Is(err, ErrAreaPolicyUnavailable), errors.Is(err, ErrAreaPolicyInvalid):
return "area_policy_unavailable", "failed", "Area policy is unavailable"
case errors.Is(err, ErrQuotaProjectionUnavailable):
return "quota_projection_unavailable", "failed", "Site quota is unavailable"
case errors.Is(err, ErrQuotaProjectionInvalid):
return "quota_projection_invalid", "failed", "Site quota is invalid"
}
var quotaError *device.QuotaExceededError
if errors.As(err, &quotaError) {
return "quota_exceeded", "rejected", "Site video channel quota is exceeded"
}
return "service_unavailable", "failed", "device change could not be accepted"
}
func (s *Postgres) GetControlOperation(
ctx context.Context, tenantID, operationID string,
) (ControlBatchOperation, error) {
var value ControlBatchOperation
var completedAt sql.NullTime
err := s.db.QueryRowContext(ctx, `SELECT id, tenant_id, site_id, status,
submitted_at, completed_at, trace_id
FROM sense.batch_operations WHERE tenant_id = $1 AND id = $2`, tenantID, operationID).
Scan(&value.ID, &value.TenantID, &value.SiteID, &value.Status,
&value.SubmittedAt, &completedAt, &value.TraceID)
if errors.Is(err, sql.ErrNoRows) {
return ControlBatchOperation{}, ErrNotFound
}
if err != nil {
return ControlBatchOperation{}, errors.New("read postgres Control API batch operation")
}
if completedAt.Valid {
point := completedAt.Time.UTC()
value.CompletedAt = &point
}
rows, err := s.db.QueryContext(ctx, `SELECT device_id, status, error_code, message, generation
FROM sense.batch_operation_items WHERE operation_id = $1 ORDER BY ordinal`, operationID)
if err != nil {
return ControlBatchOperation{}, errors.New("list postgres Control API batch results")
}
defer rows.Close()
value.Results = make([]ControlBatchItemResult, 0)
for rows.Next() {
var item ControlBatchItemResult
var code, message sql.NullString
var generation sql.NullInt64
if err := rows.Scan(&item.DeviceID, &item.Status, &code, &message, &generation); err != nil {
return ControlBatchOperation{}, errors.New("scan postgres Control API batch result")
}
if code.Valid {
item.ErrorCode = &code.String
}
if message.Valid {
item.Message = &message.String
}
if generation.Valid {
item.Generation = &generation.Int64
}
value.Results = append(value.Results, item)
}
if err := rows.Err(); err != nil {
return ControlBatchOperation{}, errors.New("iterate postgres Control API batch results")
}
return value, nil
}
func newControlOperationID(now time.Time) (string, error) {
value := make([]byte, 16)
milliseconds := uint64(now.UTC().UnixMilli())
value[0], value[1], value[2] = byte(milliseconds>>40), byte(milliseconds>>32), byte(milliseconds>>24)
value[3], value[4], value[5] = byte(milliseconds>>16), byte(milliseconds>>8), byte(milliseconds)
if _, err := rand.Read(value[6:]); err != nil {
return "", errors.New("generate Control API operation ID")
}
number := new(big.Int).SetBytes(value)
base, remainder := big.NewInt(32), new(big.Int)
const alphabet = "0123456789ABCDEFGHJKMNPQRSTVWXYZ"
encoded := make([]byte, 26)
for index := len(encoded) - 1; index >= 0; index-- {
number.QuoRem(number, base, remainder)
encoded[index] = alphabet[remainder.Int64()]
}
return "op_" + string(encoded), nil
}
+42 -16
View File
@@ -53,8 +53,8 @@ func (s *Postgres) Close() error {
func (s *Postgres) verifySchemaAndPrivileges(ctx context.Context) error { func (s *Postgres) verifySchemaAndPrivileges(ctx context.Context) error {
var version sql.NullInt64 var version sql.NullInt64
if err := s.db.QueryRowContext(ctx, if err := s.db.QueryRowContext(ctx,
`SELECT MAX(version) FROM sense.schema_migrations`).Scan(&version); err != nil || !version.Valid || version.Int64 < 3 { `SELECT MAX(version) FROM sense.schema_migrations`).Scan(&version); err != nil || !version.Valid || version.Int64 < 4 {
return errors.New("postgres sense schema migration v3 is required") return errors.New("postgres sense schema migration v4 is required")
} }
var canReadQuotaView, canWriteQuotaView, canReadSiteSource, canWriteSiteSource bool var canReadQuotaView, canWriteQuotaView, canReadSiteSource, canWriteSiteSource bool
var canReadAreaView, canWriteAreaView, canReadAreaSource, canWriteAreaSource bool var canReadAreaView, canWriteAreaView, canReadAreaSource, canWriteAreaSource bool
@@ -77,6 +77,24 @@ func (s *Postgres) verifySchemaAndPrivileges(ctx context.Context) error {
!canReadAreaView || canWriteAreaView || canReadAreaSource || canWriteAreaSource { !canReadAreaView || canWriteAreaView || canReadAreaSource || canWriteAreaSource {
return errors.New("postgres role violates Bell projection privilege boundary") return errors.New("postgres role violates Bell projection privilege boundary")
} }
var canUseReceipts, canUseOperations, canUseOperationItems bool
var publicReceipts, publicOperations, publicOperationItems bool
if err := s.db.QueryRowContext(ctx, `SELECT
has_table_privilege(current_user, 'sense.control_idempotency_receipts', 'SELECT,INSERT,UPDATE,DELETE'),
has_table_privilege(current_user, 'sense.batch_operations', 'SELECT,INSERT,UPDATE,DELETE'),
has_table_privilege(current_user, 'sense.batch_operation_items', 'SELECT,INSERT,UPDATE,DELETE'),
has_table_privilege('public', 'sense.control_idempotency_receipts', 'SELECT,INSERT,UPDATE,DELETE'),
has_table_privilege('public', 'sense.batch_operations', 'SELECT,INSERT,UPDATE,DELETE'),
has_table_privilege('public', 'sense.batch_operation_items', 'SELECT,INSERT,UPDATE,DELETE')`).Scan(
&canUseReceipts, &canUseOperations, &canUseOperationItems,
&publicReceipts, &publicOperations, &publicOperationItems,
); err != nil {
return errors.New("verify postgres Control API state privileges")
}
if !canUseReceipts || !canUseOperations || !canUseOperationItems ||
publicReceipts || publicOperations || publicOperationItems {
return errors.New("postgres role violates Control API state privilege boundary")
}
return nil return nil
} }
@@ -87,6 +105,9 @@ func (s *Postgres) CreateDevice(ctx context.Context, value device.Device) error
if value.ActualState == "" { if value.ActualState == "" {
value.ActualState = device.ActualPending value.ActualState = device.ActualPending
} }
if value.ResourceVersion == 0 {
value.ResourceVersion = 1
}
if err := value.Validate(); err != nil { if err := value.Validate(); err != nil {
return fmt.Errorf("validate device: %w", err) return fmt.Errorf("validate device: %w", err)
} }
@@ -118,14 +139,15 @@ func (s *Postgres) CreateDevice(ctx context.Context, value device.Device) error
} }
_, err = tx.ExecContext(ctx, `INSERT INTO sense.devices( _, err = tx.ExecContext(ctx, `INSERT INTO sense.devices(
id, tenant_id, site_id, area_id, serial_number, name, modality, id, tenant_id, site_id, area_id, serial_number, name, modality,
desired_state, actual_state, endpoint_ref, credential_ref, desired_state, actual_state, endpoint_ref, credential_ref, profile_token,
path_name, generation, quota_source_version, area_policy_source_version, path_name, generation, resource_version, quota_source_version, area_policy_source_version,
created_at, updated_at created_at, updated_at
) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17)`, ) VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19)`,
value.ID, value.TenantID, value.SiteID, value.AreaID, value.SerialNumber, value.Name, value.ID, value.TenantID, value.SiteID, value.AreaID, value.SerialNumber, value.Name,
value.Modality, value.DesiredState, value.ActualState, value.EndpointRef, value.Modality, value.DesiredState, value.ActualState, value.EndpointRef,
value.CredentialRef, value.PathName, value.Generation, nullableVersion(quotaVersion), value.CredentialRef, value.ProfileToken, value.PathName, value.Generation,
areaVersion, value.CreatedAt, value.UpdatedAt) value.ResourceVersion, nullableVersion(quotaVersion), areaVersion,
value.CreatedAt, value.UpdatedAt)
if err != nil { if err != nil {
return errors.New("insert postgres device") return errors.New("insert postgres device")
} }
@@ -354,6 +376,7 @@ func (s *Postgres) SetDesiredState(ctx context.Context, id string, desired devic
var updatedQuotaVersion, updatedAreaVersion sql.NullInt64 var updatedQuotaVersion, updatedAreaVersion sql.NullInt64
err = tx.QueryRowContext(ctx, `UPDATE sense.devices SET err = tx.QueryRowContext(ctx, `UPDATE sense.devices SET
desired_state = $1, actual_state = 'pending', generation = generation + 1, desired_state = $1, actual_state = 'pending', generation = generation + 1,
resource_version = resource_version + 1,
quota_source_version = COALESCE($2, quota_source_version), quota_source_version = COALESCE($2, quota_source_version),
area_policy_source_version = COALESCE($3, area_policy_source_version), area_policy_source_version = COALESCE($3, area_policy_source_version),
updated_at = $4 updated_at = $4
@@ -413,10 +436,10 @@ func (s *Postgres) ListDueReconcile(ctx context.Context, now time.Time, limit in
rows, err := s.db.QueryContext(ctx, `SELECT `+postgresDeviceColumns+`, r.failure_count, r.next_attempt_at rows, err := s.db.QueryContext(ctx, `SELECT `+postgresDeviceColumns+`, r.failure_count, r.next_attempt_at
FROM sense.devices d FROM sense.devices d
JOIN sense.reconcile_state r ON r.device_id = d.id JOIN sense.reconcile_state r ON r.device_id = d.id
WHERE d.desired_state = 'enabled' WHERE EXISTS (SELECT 1 FROM sense.device_capabilities c
AND EXISTS (SELECT 1 FROM sense.device_capabilities c
WHERE c.device_id = d.id AND c.capability = 'video_capture') WHERE c.device_id = d.id AND c.capability = 'video_capture')
AND (r.observed_generation < d.generation OR r.failure_count > 0) AND (r.observed_generation < d.generation
OR (d.desired_state = 'enabled' AND r.failure_count > 0))
AND (r.next_attempt_at IS NULL OR r.next_attempt_at <= $1) AND (r.next_attempt_at IS NULL OR r.next_attempt_at <= $1)
ORDER BY d.updated_at, d.id LIMIT $2`, now, limit) ORDER BY d.updated_at, d.id LIMIT $2`, now, limit)
if err != nil { if err != nil {
@@ -434,7 +457,8 @@ func (s *Postgres) ListDueReconcile(ctx context.Context, now time.Time, limit in
&areaID, &candidate.Device.SerialNumber, &candidate.Device.Name, &candidate.Device.Modality, &areaID, &candidate.Device.SerialNumber, &candidate.Device.Name, &candidate.Device.Modality,
&candidate.Device.DesiredState, &candidate.Device.ActualState, &candidate.Device.DesiredState, &candidate.Device.ActualState,
&candidate.Device.EndpointRef, &candidate.Device.CredentialRef, &candidate.Device.EndpointRef, &candidate.Device.CredentialRef,
&candidate.Device.PathName, &candidate.Device.Generation, &candidate.Device.ProfileToken, &candidate.Device.PathName,
&candidate.Device.Generation, &candidate.Device.ResourceVersion,
&quotaVersion, &areaVersion, &quotaVersion, &areaVersion,
&candidate.Device.CreatedAt, &candidate.Device.UpdatedAt, &candidate.Device.CreatedAt, &candidate.Device.UpdatedAt,
&candidate.FailureCount, &nextAttempt, &candidate.FailureCount, &nextAttempt,
@@ -511,7 +535,8 @@ func (s *Postgres) MarkReconciled(ctx context.Context, id string, generation int
return ErrNotFound return ErrNotFound
} }
if _, err := tx.ExecContext(ctx, `UPDATE sense.devices if _, err := tx.ExecContext(ctx, `UPDATE sense.devices
SET actual_state = 'pending', updated_at = $1 WHERE id = $2`, now, id); err != nil { SET actual_state = CASE WHEN desired_state = 'disabled' THEN 'offline' ELSE 'pending' END,
updated_at = $1 WHERE id = $2`, now, id); err != nil {
return errors.New("mark postgres reconciled device pending") return errors.New("mark postgres reconciled device pending")
} }
if err := tx.Commit(); err != nil { if err := tx.Commit(); err != nil {
@@ -628,8 +653,8 @@ func (s *Postgres) ConvergenceSnapshot(ctx context.Context) (ConvergenceSnapshot
} }
const postgresDeviceColumns = `d.id, d.tenant_id, d.site_id, d.area_id, d.serial_number, d.name, d.modality, const postgresDeviceColumns = `d.id, d.tenant_id, d.site_id, d.area_id, d.serial_number, d.name, d.modality,
d.desired_state, d.actual_state, d.endpoint_ref, d.credential_ref, d.desired_state, d.actual_state, d.endpoint_ref, d.credential_ref, d.profile_token,
d.path_name, d.generation, d.quota_source_version, d.area_policy_source_version, d.path_name, d.generation, d.resource_version, d.quota_source_version, d.area_policy_source_version,
d.created_at, d.updated_at` d.created_at, d.updated_at`
const postgresDeviceSelect = `SELECT ` + postgresDeviceColumns + ` FROM sense.devices d` const postgresDeviceSelect = `SELECT ` + postgresDeviceColumns + ` FROM sense.devices d`
@@ -641,8 +666,9 @@ func scanPostgresDevice(row scanner) (device.Device, error) {
err := row.Scan( err := row.Scan(
&value.ID, &value.TenantID, &value.SiteID, &areaID, &value.SerialNumber, &value.ID, &value.TenantID, &value.SiteID, &areaID, &value.SerialNumber,
&value.Name, &value.Modality, &value.DesiredState, &value.ActualState, &value.Name, &value.Modality, &value.DesiredState, &value.ActualState,
&value.EndpointRef, &value.CredentialRef, &value.PathName, &value.EndpointRef, &value.CredentialRef, &value.ProfileToken, &value.PathName,
&value.Generation, &quotaVersion, &areaVersion, &value.CreatedAt, &value.UpdatedAt, &value.Generation, &value.ResourceVersion, &quotaVersion, &areaVersion,
&value.CreatedAt, &value.UpdatedAt,
) )
value.AreaID = areaID.String value.AreaID = areaID.String
value.QuotaSourceVersion = quotaVersion.Int64 value.QuotaSourceVersion = quotaVersion.Int64
+363 -4
View File
@@ -2,6 +2,7 @@ package store
import ( import (
"context" "context"
"crypto/sha256"
"database/sql" "database/sql"
"errors" "errors"
"fmt" "fmt"
@@ -286,12 +287,12 @@ func TestPostgresOpenRejectsOverprivilegedRuntimeRole(t *testing.T) {
_, admin := openPostgresTestStore(t) _, admin := openPostgresTestStore(t)
ctx := context.Background() ctx := context.Background()
if _, err := admin.ExecContext(ctx, if _, err := admin.ExecContext(ctx,
`GRANT UPDATE ON bell.site_quota_v1 TO yovision_t010_sense`); err != nil { `GRANT UPDATE ON bell.site_quota_v1 TO yovision_t011_sense`); err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer func() { defer func() {
_, _ = admin.ExecContext(context.Background(), _, _ = admin.ExecContext(context.Background(),
`REVOKE UPDATE ON bell.site_quota_v1 FROM yovision_t010_sense`) `REVOKE UPDATE ON bell.site_quota_v1 FROM yovision_t011_sense`)
}() }()
value, err := OpenPostgres(ctx, os.Getenv(postgresTestDSNEnv)) value, err := OpenPostgres(ctx, os.Getenv(postgresTestDSNEnv))
if value != nil { if value != nil {
@@ -596,12 +597,12 @@ func TestPostgresOpenRejectsAreaSourcePrivilege(t *testing.T) {
_, admin := openPostgresTestStore(t) _, admin := openPostgresTestStore(t)
ctx := context.Background() ctx := context.Background()
if _, err := admin.ExecContext(ctx, if _, err := admin.ExecContext(ctx,
`GRANT SELECT ON bell.areas TO yovision_t010_sense`); err != nil { `GRANT SELECT ON bell.areas TO yovision_t011_sense`); err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer func() { defer func() {
_, _ = admin.ExecContext(context.Background(), _, _ = admin.ExecContext(context.Background(),
`REVOKE SELECT ON bell.areas FROM yovision_t010_sense`) `REVOKE SELECT ON bell.areas FROM yovision_t011_sense`)
}() }()
value, err := OpenPostgres(ctx, os.Getenv(postgresTestDSNEnv)) value, err := OpenPostgres(ctx, os.Getenv(postgresTestDSNEnv))
if value != nil { if value != nil {
@@ -613,6 +614,361 @@ func TestPostgresOpenRejectsAreaSourcePrivilege(t *testing.T) {
} }
} }
func TestPostgresOpenRejectsPublicControlStatePrivilege(t *testing.T) {
_, admin := openPostgresTestStore(t)
ctx := context.Background()
if _, err := admin.ExecContext(ctx, `GRANT SELECT ON sense.batch_operations TO PUBLIC`); err != nil {
t.Fatal(err)
}
defer func() {
_, _ = admin.ExecContext(context.Background(), `REVOKE SELECT ON sense.batch_operations FROM PUBLIC`)
}()
value, err := OpenPostgres(ctx, os.Getenv(postgresTestDSNEnv))
if value != nil {
_ = value.Close()
t.Fatal("PUBLIC Control API table privilege was accepted")
}
if err == nil || !strings.Contains(err.Error(), "Control API state privilege boundary") {
t.Fatalf("expected Control API privilege-boundary error, got %v", err)
}
}
func TestPostgresControlCreateIdempotencyAndRedactedSnapshot(t *testing.T) {
postgres, admin := openPostgresTestStore(t)
insertBellSite(t, admin, "tenant", "site", 2)
value := videoDevice(1, "tenant", "site")
value.ID = "dev_control_create_1"
value.DesiredState = device.DesiredDisabled
hash := sha256.Sum256([]byte("canonical-create"))
ctx := WithAuditContext(context.Background(), AuditContext{
ActorType: AuditActorService, ActorID: "bell-control", TraceID: "trace-control-create",
})
request := ControlCreateRequest{
Scope: IdempotencyScope{
PrincipalID: "bell-control", TenantID: "tenant", SiteID: "site",
Operation: "createDevice", Key: "create-control-0001",
RequestHash: hash, TraceID: "trace-control-create",
},
Device: value,
}
first, err := postgres.CreateControlDevice(ctx, request)
if err != nil {
t.Fatal(err)
}
request.Device.ID = "dev_control_create_retry"
replayed, err := postgres.CreateControlDevice(ctx, request)
if err != nil {
t.Fatal(err)
}
if !replayed.Replay || replayed.Device.ID != first.Device.ID || replayed.TraceID != first.TraceID {
t.Fatalf("create replay drifted: first=%+v replay=%+v", first, replayed)
}
conflictHash := sha256.Sum256([]byte("different-create"))
request.Scope.RequestHash = conflictHash
if _, err := postgres.CreateControlDevice(ctx, request); !errors.Is(err, ErrIdempotencyConflict) {
t.Fatalf("same key with different body was not rejected: %v", err)
}
var devices, receipts int
var responseBody string
if err := admin.QueryRow(`SELECT count(*) FROM sense.devices`).Scan(&devices); err != nil {
t.Fatal(err)
}
if err := admin.QueryRow(`SELECT count(*), min(response_body::text)
FROM sense.control_idempotency_receipts`).Scan(&receipts, &responseBody); err != nil {
t.Fatal(err)
}
if devices != 1 || receipts != 1 {
t.Fatalf("idempotent create counts drifted: devices=%d receipts=%d", devices, receipts)
}
for _, forbidden := range []string{value.EndpointRef, value.CredentialRef, "profile_token", "path_name"} {
if strings.Contains(responseBody, forbidden) {
t.Fatalf("idempotency response snapshot leaked %q", forbidden)
}
}
}
func TestPostgresControlListIsStableFilteredAndTenantScoped(t *testing.T) {
postgres, admin := openPostgresTestStore(t)
insertBellSite(t, admin, "tenant", "site", 4)
insertBellSite(t, admin, "other", "site", 4)
for index := 1; index <= 3; index++ {
value := videoDevice(index, "tenant", "site")
value.ID = fmt.Sprintf("dev_list_%d", index)
value.DesiredState = device.DesiredDisabled
if index == 3 {
value.DesiredState = device.DesiredEnabled
}
if err := postgres.CreateDevice(context.Background(), value); err != nil {
t.Fatal(err)
}
}
other := videoDevice(9, "other", "site")
other.ID, other.DesiredState = "dev_list_other", device.DesiredDisabled
if err := postgres.CreateDevice(context.Background(), other); err != nil {
t.Fatal(err)
}
first, err := postgres.ListControlDevices(context.Background(), "tenant", "site", ControlListFilter{Limit: 1})
if err != nil || len(first.Items) != 1 || !first.HasMore || first.Quota.UsedVideoChannels != 1 {
t.Fatalf("unexpected first page: %+v %v", first, err)
}
second, err := postgres.ListControlDevices(context.Background(), "tenant", "site", ControlListFilter{
Limit: 1, AfterCreated: &first.Items[0].CreatedAt, AfterDeviceID: first.Items[0].ID,
})
if err != nil || len(second.Items) != 1 || second.Items[0].ID == first.Items[0].ID {
t.Fatalf("stable cursor position failed: %+v %v", second, err)
}
desired := device.DesiredEnabled
filtered, err := postgres.ListControlDevices(context.Background(), "tenant", "site", ControlListFilter{
Limit: 100, DesiredState: &desired,
})
if err != nil || len(filtered.Items) != 1 || filtered.Items[0].ID != "dev_list_3" {
t.Fatalf("desired-state filter or tenant scope failed: %+v %v", filtered, err)
}
if _, err := postgres.ListControlDevices(context.Background(), "tenant", "missing", ControlListFilter{Limit: 50}); !errors.Is(err, ErrNotFound) {
t.Fatalf("missing Site did not return not found: %v", err)
}
}
func TestPostgresControlPatchUsesETagAndAuditV2(t *testing.T) {
postgres, admin := openPostgresTestStore(t)
insertBellSite(t, admin, "tenant", "site", 2)
value := videoDevice(1, "tenant", "site")
value.ID = "dev_control_patch_1"
value.DesiredState = device.DesiredDisabled
if err := postgres.CreateDevice(context.Background(), value); err != nil {
t.Fatal(err)
}
current, err := postgres.GetControlDevice(context.Background(), "tenant", "site", value.ID)
if err != nil {
t.Fatal(err)
}
etag := DeviceETag(current.ID, current.ResourceVersion)
newName, newProfile := "Updated camera", "profile-main"
ctx := WithAuditContext(context.Background(), AuditContext{
ActorType: AuditActorUser, ActorID: "operator-1", TraceID: "trace-control-patch",
})
updated, err := postgres.PatchControlDevice(ctx, "tenant", "site", value.ID, etag, ControlPatch{
Name: &newName, ProfileToken: &newProfile,
})
if err != nil {
t.Fatal(err)
}
if updated.Device.ResourceVersion != current.ResourceVersion+1 ||
updated.Device.Generation != current.Generation+1 || updated.ETag == etag {
t.Fatalf("patch did not advance versions: before=%+v after=%+v", current, updated)
}
if _, err := postgres.PatchControlDevice(ctx, "tenant", "site", value.ID, etag, ControlPatch{Name: &newName}); !errors.Is(err, ErrETagMismatch) {
t.Fatalf("stale ETag was accepted: %v", err)
}
var profileToken, eventType, payload string
if err := admin.QueryRow(`SELECT profile_token FROM sense.devices WHERE id = $1`, value.ID).Scan(&profileToken); err != nil {
t.Fatal(err)
}
if err := admin.QueryRow(`SELECT event_type, payload::text
FROM sense.device_operation_outbox WHERE event_type = 'device.configuration.accepted'`).
Scan(&eventType, &payload); err != nil {
t.Fatal(err)
}
if profileToken != newProfile || eventType != "device.configuration.accepted" ||
strings.Contains(payload, newProfile) || !strings.Contains(payload, "profile_token") {
t.Fatalf("configuration persistence/audit mismatch: profile=%q event=%q payload=%s", profileToken, eventType, payload)
}
}
func TestPostgresControlBatchIsPerItemDurableAndReplayable(t *testing.T) {
postgres, admin := openPostgresTestStore(t)
insertBellSite(t, admin, "tenant", "site", 4)
first := videoDevice(1, "tenant", "site")
first.ID, first.DesiredState = "dev_batch_1", device.DesiredDisabled
second := videoDevice(2, "tenant", "site")
second.ID, second.DesiredState = "dev_batch_2", device.DesiredDisabled
for _, value := range []device.Device{first, second} {
if err := postgres.CreateDevice(context.Background(), value); err != nil {
t.Fatal(err)
}
}
hash := sha256.Sum256([]byte("canonical-batch"))
ctx := WithAuditContext(context.Background(), AuditContext{
ActorType: AuditActorUser, ActorID: "operator-1", Reason: "approved", TraceID: "trace-batch",
})
request := ControlBatchRequest{
Scope: IdempotencyScope{
PrincipalID: "operator-1", TenantID: "tenant", SiteID: "site",
Operation: "batchSetDeviceDesiredState", Key: "batch-control-0001",
RequestHash: hash, TraceID: "trace-batch",
},
Reason: "approved",
Items: []ControlBatchItem{
{DeviceID: first.ID, ETag: DeviceETag(first.ID, 1), DesiredState: device.DesiredEnabled},
{DeviceID: first.ID, ETag: DeviceETag(first.ID, 1), DesiredState: device.DesiredEnabled},
{DeviceID: second.ID, ETag: DeviceETag(second.ID, 1), DesiredState: device.DesiredEnabled},
},
}
operation, err := postgres.BatchSetControlDesiredState(ctx, request)
if err != nil {
t.Fatal(err)
}
if operation.Status != "partially_succeeded" || len(operation.Results) != 3 ||
operation.Results[0].Status != "rejected" || operation.Results[1].Status != "rejected" ||
operation.Results[2].Status != "succeeded" {
t.Fatalf("unexpected batch result: %+v", operation)
}
replayed, err := postgres.BatchSetControlDesiredState(ctx, request)
if err != nil || !replayed.Replay || replayed.ID != operation.ID {
t.Fatalf("batch replay drifted: %+v %v", replayed, err)
}
read, err := postgres.GetControlOperation(context.Background(), "tenant", operation.ID)
if err != nil || read.SiteID != "site" || len(read.Results) != 3 {
t.Fatalf("stored operation could not be read: %+v %v", read, err)
}
if _, err := postgres.GetControlOperation(context.Background(), "other-tenant", operation.ID); !errors.Is(err, ErrNotFound) {
t.Fatalf("cross-tenant operation was visible: %v", err)
}
var enabled, operations, receipts int
if err := admin.QueryRow(`SELECT count(*) FROM sense.devices WHERE desired_state = 'enabled'`).Scan(&enabled); err != nil {
t.Fatal(err)
}
if err := admin.QueryRow(`SELECT count(*) FROM sense.batch_operations`).Scan(&operations); err != nil {
t.Fatal(err)
}
if err := admin.QueryRow(`SELECT count(*) FROM sense.control_idempotency_receipts`).Scan(&receipts); err != nil {
t.Fatal(err)
}
if enabled != 1 || operations != 1 || receipts != 1 {
t.Fatalf("batch durability counts drifted: enabled=%d operations=%d receipts=%d", enabled, operations, receipts)
}
}
func TestPostgresConcurrentControlCreateExecutesOnce(t *testing.T) {
postgres, admin := openPostgresTestStore(t)
insertBellSite(t, admin, "tenant", "site", 2)
hash := sha256.Sum256([]byte("concurrent-create"))
start := make(chan struct{})
results := make(chan ControlCreateResult, 2)
errorsFound := make(chan error, 2)
var wait sync.WaitGroup
for index := 1; index <= 2; index++ {
wait.Add(1)
go func(index int) {
defer wait.Done()
value := videoDevice(index, "tenant", "site")
value.ID = fmt.Sprintf("dev_concurrent_%d", index)
value.SerialNumber = "same-semantic-serial"
value.DesiredState = device.DesiredDisabled
request := ControlCreateRequest{
Scope: IdempotencyScope{
PrincipalID: "service", TenantID: "tenant", SiteID: "site",
Operation: "createDevice", Key: "concurrent-create-0001",
RequestHash: hash, TraceID: fmt.Sprintf("trace-concurrent-%d", index),
}, Device: value,
}
<-start
result, err := postgres.CreateControlDevice(context.Background(), request)
if err != nil {
errorsFound <- err
return
}
results <- result
}(index)
}
close(start)
wait.Wait()
close(results)
close(errorsFound)
for err := range errorsFound {
t.Fatalf("concurrent idempotent create failed: %v", err)
}
ids := make(map[string]struct{})
for result := range results {
ids[result.Device.ID] = struct{}{}
}
if len(ids) != 1 {
t.Fatalf("concurrent create returned multiple resources: %+v", ids)
}
var devices, audits, receipts int
if err := admin.QueryRow(`SELECT count(*) FROM sense.devices`).Scan(&devices); err != nil {
t.Fatal(err)
}
if err := admin.QueryRow(`SELECT count(*) FROM sense.device_operation_outbox`).Scan(&audits); err != nil {
t.Fatal(err)
}
if err := admin.QueryRow(`SELECT count(*) FROM sense.control_idempotency_receipts`).Scan(&receipts); err != nil {
t.Fatal(err)
}
if devices != 1 || audits != 1 || receipts != 1 {
t.Fatalf("concurrent create executed more than once: devices=%d audits=%d receipts=%d", devices, audits, receipts)
}
}
func TestPostgresConcurrentControlBatchesUseStableDeviceLockOrder(t *testing.T) {
postgres, admin := openPostgresTestStore(t)
insertBellSite(t, admin, "tenant", "site", 4)
ids := []string{"dev_lock_a", "dev_lock_b"}
for index, id := range ids {
value := videoDevice(index+1, "tenant", "site")
value.ID, value.DesiredState = id, device.DesiredDisabled
if err := postgres.CreateDevice(context.Background(), value); err != nil {
t.Fatal(err)
}
}
start := make(chan struct{})
operations := make(chan ControlBatchOperation, 2)
errorsFound := make(chan error, 2)
var wait sync.WaitGroup
for index := 0; index < 2; index++ {
wait.Add(1)
go func(index int) {
defer wait.Done()
order := ids
if index == 1 {
order = []string{ids[1], ids[0]}
}
hash := sha256.Sum256([]byte(fmt.Sprintf("batch-order-%d", index)))
request := ControlBatchRequest{
Scope: IdempotencyScope{
PrincipalID: "operator", TenantID: "tenant", SiteID: "site",
Operation: "batchSetDeviceDesiredState",
Key: fmt.Sprintf("batch-lock-order-%04d", index), RequestHash: hash,
TraceID: fmt.Sprintf("trace-lock-order-%d", index),
},
Reason: "concurrency test",
Items: []ControlBatchItem{
{DeviceID: order[0], ETag: DeviceETag(order[0], 1), DesiredState: device.DesiredEnabled},
{DeviceID: order[1], ETag: DeviceETag(order[1], 1), DesiredState: device.DesiredEnabled},
},
}
<-start
operation, err := postgres.BatchSetControlDesiredState(context.Background(), request)
if err != nil {
errorsFound <- err
return
}
operations <- operation
}(index)
}
close(start)
wait.Wait()
close(operations)
close(errorsFound)
for err := range errorsFound {
t.Fatalf("opposite-order batch failed or deadlocked: %v", err)
}
var succeeded, failed int
for operation := range operations {
switch operation.Status {
case "succeeded":
succeeded++
case "failed":
failed++
default:
t.Fatalf("unexpected concurrent batch status: %+v", operation)
}
}
if succeeded != 1 || failed != 1 {
t.Fatalf("expected one winner and one stale loser, got succeeded=%d failed=%d", succeeded, failed)
}
}
func openPostgresTestStore(t *testing.T) (*Postgres, *sql.DB) { func openPostgresTestStore(t *testing.T) (*Postgres, *sql.DB) {
t.Helper() t.Helper()
dsn := os.Getenv(postgresTestDSNEnv) dsn := os.Getenv(postgresTestDSNEnv)
@@ -629,6 +985,9 @@ func openPostgresTestStore(t *testing.T) (*Postgres, *sql.DB) {
t.Fatal("connect PostgreSQL test administrator") t.Fatal("connect PostgreSQL test administrator")
} }
if _, err := admin.ExecContext(context.Background(), `TRUNCATE if _, err := admin.ExecContext(context.Background(), `TRUNCATE
sense.control_idempotency_receipts,
sense.batch_operation_items,
sense.batch_operations,
sense.device_operation_outbox, sense.device_operation_outbox,
sense.device_capabilities, sense.device_capabilities,
sense.reconcile_state, sense.reconcile_state,
+6 -4
View File
@@ -366,12 +366,12 @@ func (s *SQLite) ListDueReconcile(ctx context.Context, now time.Time, limit int)
rows, err := s.db.QueryContext(ctx, `SELECT `+deviceColumns+`, r.failure_count, r.next_attempt_at rows, err := s.db.QueryContext(ctx, `SELECT `+deviceColumns+`, r.failure_count, r.next_attempt_at
FROM sense_devices d FROM sense_devices d
JOIN sense_reconcile_state r ON r.device_id = d.id JOIN sense_reconcile_state r ON r.device_id = d.id
WHERE d.desired_state = 'enabled' WHERE EXISTS (
AND EXISTS (
SELECT 1 FROM sense_device_capabilities c SELECT 1 FROM sense_device_capabilities c
WHERE c.device_id = d.id AND c.capability = 'video_capture' WHERE c.device_id = d.id AND c.capability = 'video_capture'
) )
AND (r.observed_generation < d.generation OR r.failure_count > 0) AND (r.observed_generation < d.generation
OR (d.desired_state = 'enabled' AND r.failure_count > 0))
AND (r.next_attempt_at IS NULL OR r.next_attempt_at <= ?) AND (r.next_attempt_at IS NULL OR r.next_attempt_at <= ?)
ORDER BY d.updated_at, d.id ORDER BY d.updated_at, d.id
LIMIT ?`, formatTime(now), limit) LIMIT ?`, formatTime(now), limit)
@@ -487,7 +487,9 @@ func (s *SQLite) MarkReconciled(ctx context.Context, id string, generation int64
return ErrNotFound return ErrNotFound
} }
if _, err := tx.ExecContext(ctx, ` if _, err := tx.ExecContext(ctx, `
UPDATE sense_devices SET actual_state = 'pending', updated_at = ? WHERE id = ?`, UPDATE sense_devices SET
actual_state = CASE WHEN desired_state = 'disabled' THEN 'offline' ELSE 'pending' END,
updated_at = ? WHERE id = ?`,
formatTime(now), id); err != nil { formatTime(now), id); err != nil {
return fmt.Errorf("mark reconciled device pending: %w", err) return fmt.Errorf("mark reconciled device pending: %w", err)
} }
+149
View File
@@ -0,0 +1,149 @@
-- Sense Control API v1 durable concurrency, idempotency and batch-operation state.
ALTER TABLE sense.devices ADD COLUMN IF NOT EXISTS profile_token text NOT NULL DEFAULT '';
ALTER TABLE sense.devices ADD COLUMN IF NOT EXISTS resource_version bigint NOT NULL DEFAULT 1;
DO $area_preflight$
BEGIN
IF EXISTS (SELECT 1 FROM sense.devices WHERE area_id IS NULL OR btrim(area_id) = '') THEN
RAISE EXCEPTION 'assign every legacy Sense device to a valid Area before installing Control API v4';
END IF;
END
$area_preflight$;
ALTER TABLE sense.devices ALTER COLUMN area_id SET NOT NULL;
DO $constraints$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_constraint
WHERE conrelid = 'sense.devices'::regclass
AND conname = 'sense_devices_resource_version_positive'
) THEN
ALTER TABLE sense.devices ADD CONSTRAINT sense_devices_resource_version_positive
CHECK (resource_version >= 1);
END IF;
END
$constraints$;
CREATE TABLE IF NOT EXISTS sense.control_idempotency_receipts (
scope_hash bytea PRIMARY KEY,
request_hash bytea NOT NULL,
operation_name text NOT NULL,
principal_id text NOT NULL,
tenant_id text NOT NULL,
site_id text NOT NULL,
response_status integer NOT NULL,
response_body jsonb NOT NULL,
response_etag text,
response_location text,
trace_id text NOT NULL,
created_at timestamptz NOT NULL,
expires_at timestamptz NOT NULL,
CONSTRAINT sense_control_receipt_hash_lengths CHECK (
octet_length(scope_hash) = 32 AND octet_length(request_hash) = 32
),
CONSTRAINT sense_control_receipt_identity_not_blank CHECK (
btrim(operation_name) <> '' AND btrim(principal_id) <> ''
AND btrim(tenant_id) <> '' AND btrim(site_id) <> '' AND btrim(trace_id) <> ''
),
CONSTRAINT sense_control_receipt_status CHECK (response_status BETWEEN 200 AND 299),
CONSTRAINT sense_control_receipt_operation CHECK (
operation_name IN ('createDevice', 'batchSetDeviceDesiredState')
),
CONSTRAINT sense_control_receipt_lengths CHECK (
char_length(principal_id) <= 200 AND char_length(trace_id) <= 128
AND (response_etag IS NULL OR char_length(response_etag) <= 128)
AND (response_location IS NULL OR char_length(response_location) <= 512)
),
CONSTRAINT sense_control_receipt_body_object CHECK (jsonb_typeof(response_body) = 'object'),
CONSTRAINT sense_control_receipt_ttl CHECK (expires_at >= created_at + interval '24 hours')
);
ALTER TABLE sense.control_idempotency_receipts OWNER TO sense_app;
CREATE INDEX IF NOT EXISTS sense_control_receipts_expiry_idx
ON sense.control_idempotency_receipts(expires_at, scope_hash);
CREATE TABLE IF NOT EXISTS sense.batch_operations (
id text PRIMARY KEY,
tenant_id text NOT NULL,
site_id text NOT NULL,
principal_id text NOT NULL,
status text NOT NULL,
trace_id text NOT NULL,
submitted_at timestamptz NOT NULL,
completed_at timestamptz,
CONSTRAINT sense_batch_operation_id_format CHECK (
id ~ '^op_[0-9A-HJKMNP-TV-Z]{26}$'
),
CONSTRAINT sense_batch_operation_identity_not_blank CHECK (
btrim(tenant_id) <> '' AND btrim(site_id) <> ''
AND btrim(principal_id) <> '' AND btrim(trace_id) <> ''
),
CONSTRAINT sense_batch_operation_status CHECK (
status IN ('queued', 'running', 'succeeded', 'partially_succeeded', 'failed')
),
CONSTRAINT sense_batch_operation_completion CHECK (
(status IN ('queued', 'running') AND completed_at IS NULL)
OR (status IN ('succeeded', 'partially_succeeded', 'failed') AND completed_at IS NOT NULL)
)
);
ALTER TABLE sense.batch_operations OWNER TO sense_app;
CREATE TABLE IF NOT EXISTS sense.batch_operation_items (
operation_id text NOT NULL REFERENCES sense.batch_operations(id) ON DELETE CASCADE,
ordinal integer NOT NULL,
device_id text NOT NULL,
status text NOT NULL,
error_code text,
message text,
generation bigint,
PRIMARY KEY (operation_id, ordinal),
CONSTRAINT sense_batch_item_ordinal CHECK (ordinal BETWEEN 0 AND 127),
CONSTRAINT sense_batch_item_device_not_blank CHECK (btrim(device_id) <> ''),
CONSTRAINT sense_batch_item_status CHECK (
status IN ('accepted', 'rejected', 'succeeded', 'failed')
),
CONSTRAINT sense_batch_item_error_pair CHECK (
(status IN ('accepted', 'succeeded') AND error_code IS NULL AND message IS NULL)
OR (status IN ('rejected', 'failed') AND error_code IS NOT NULL AND message IS NOT NULL)
),
CONSTRAINT sense_batch_item_error_code CHECK (
error_code IS NULL OR error_code IN (
'invalid_request', 'unauthenticated', 'forbidden', 'not_found',
'conflict', 'precondition_required', 'etag_mismatch',
'idempotency_conflict', 'duplicate_serial_number', 'quota_exceeded',
'quota_projection_unavailable', 'quota_projection_invalid',
'area_policy_denied', 'area_policy_unavailable', 'adapter_not_ready',
'authentication_failed', 'endpoint_credentials_forbidden',
'batch_too_large', 'service_unavailable', 'internal_error'
)
),
CONSTRAINT sense_batch_item_message_length CHECK (
message IS NULL OR char_length(message) <= 500
),
CONSTRAINT sense_batch_item_generation CHECK (generation IS NULL OR generation >= 1)
);
ALTER TABLE sense.batch_operation_items OWNER TO sense_app;
CREATE INDEX IF NOT EXISTS sense_batch_operations_scope_idx
ON sense.batch_operations(tenant_id, site_id, submitted_at DESC, id);
ALTER TABLE sense.device_operation_outbox DROP CONSTRAINT IF EXISTS sense_outbox_event_type;
ALTER TABLE sense.device_operation_outbox ADD CONSTRAINT sense_outbox_event_type CHECK (
event_type IN (
'device.created',
'device.desired_state.accepted',
'device.configuration.accepted'
)
);
ALTER TABLE sense.device_operation_outbox DROP CONSTRAINT IF EXISTS sense_outbox_payload_kind;
ALTER TABLE sense.device_operation_outbox ADD CONSTRAINT sense_outbox_payload_kind CHECK (
(event_type = 'device.created' AND payload ->> 'kind' = 'device_created')
OR (event_type = 'device.desired_state.accepted'
AND payload ->> 'kind' = 'desired_state_accepted')
OR (event_type = 'device.configuration.accepted'
AND payload ->> 'kind' = 'configuration_accepted')
);
INSERT INTO sense.schema_migrations(version) VALUES (4)
ON CONFLICT (version) DO NOTHING;
@@ -0,0 +1,11 @@
-- Extend the least-privilege boundary to Control API durable state.
REVOKE ALL ON TABLE sense.control_idempotency_receipts FROM PUBLIC;
REVOKE ALL ON TABLE sense.batch_operations FROM PUBLIC;
REVOKE ALL ON TABLE sense.batch_operation_items FROM PUBLIC;
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLE
sense.control_idempotency_receipts,
sense.batch_operations,
sense.batch_operation_items
TO sense_app;
+5 -2
View File
@@ -1,9 +1,11 @@
# YoVision PostgreSQL 初始化 # YoVision PostgreSQL 初始化
本目录实现 T-009/T-010 的 PostgreSQL `17.10` schema。SQL 必须按文件名前缀顺序执行:`001`~`004` 创建 NOLOGIN 权限角色、Bell/Sense 初始对象和配额权限;`005`~`007` 增量增加 Area 策略投影、设备操作审计 Outbox 和对应最小权限。全部 SQL 可重放。应用登录角色及其密码由部署环境或密钥系统创建,并分别加入 `bell_app`、`sense_app`;仓库不保存登录凭据。 本目录实现 T-009~T-011 的 PostgreSQL `17.10` schema。SQL 必须按文件名前缀顺序执行:`001`~`004` 创建 NOLOGIN 权限角色、Bell/Sense 初始对象和配额权限;`005`~`007` 增量增加 Area 与审计;`008`~`009` 增加 Control API 的资源版本、write-only profile token、24 小时幂等收据、batch operation 和对应最小权限。全部 SQL 可重放。应用登录角色及其密码由部署环境或密钥系统创建,并分别加入 `bell_app`、`sense_app`;仓库不保存登录凭据。
生产/共享实例必须由管理员先备份并在 YoVision 专用数据库中执行。Sense 进程不会用高权限自动建库或建角色。示例只使用私有环境变量,不把实际 DSN 写入脚本或日志: 生产/共享实例必须由管理员先备份并在 YoVision 专用数据库中执行。Sense 进程不会用高权限自动建库或建角色。示例只使用私有环境变量,不把实际 DSN 写入脚本或日志:
安装 `008` 前必须先查询并为全部历史 `sense.devices.area_id IS NULL/blank` 记录完成经批准的 Area 归属;migration 会失败关闭而不会猜测或自动搬迁这些设备。
```powershell ```powershell
$env:YOVISION_PG_ADMIN_DSN = '由管理员私下设置' $env:YOVISION_PG_ADMIN_DSN = '由管理员私下设置'
Get-ChildItem deploy/postgres/[0-9][0-9][0-9]_*.sql | Get-ChildItem deploy/postgres/[0-9][0-9][0-9]_*.sql |
@@ -19,8 +21,9 @@ Get-ChildItem deploy/postgres/[0-9][0-9][0-9]_*.sql |
- `sense_app` 拥有 `sense` schema,只获得 `bell` schema 的 `USAGE` 和两个投影视图的 `SELECT`。 - `sense_app` 拥有 `sense` schema,只获得 `bell` schema 的 `USAGE` 和两个投影视图的 `SELECT`。
- `sense_app` 对 Bell 源表、Bell migration 表和 trigger function 没有权限;启动检查发现权限过宽时拒绝运行。 - `sense_app` 对 Bell 源表、Bell migration 表和 trigger function 没有权限;启动检查发现权限过宽时拒绝运行。
- `sense.device_operation_outbox` 是本地持久化审计事实,不是 Bell 全局审计真相;relay 的传输、签名、确认和留存尚未实现。 - `sense.device_operation_outbox` 是本地持久化审计事实,不是 Bell 全局审计真相;relay 的传输、签名、确认和留存尚未实现。
- `sense.control_idempotency_receipts` 不保存原始 Idempotency-Key,只保存 scope/request SHA-256 和脱敏响应快照;`batch_operations`/items 只保存逻辑 ID、状态和稳定错误,不保存连接秘密。
- PUBLIC 对两个业务 schema 的表和函数没有权限。 - PUBLIC 对两个业务 schema 的表和函数没有权限。
## 恢复 ## 恢复
T-009/T-010 不迁移或删除现有 SQLite 数据,默认运行配置也仍为 SQLite。SQLite 不具备生产 Area/Outbox 一致性,不能作为后续公共控制 API 的生产后端。若 PostgreSQL 切换验证失败,停止 PostgreSQL 模式的 Sense 进程并恢复原实验室 SQLite DSN;不要把此回退误报为生产语义等价。schema 不提供自动 down migration;需要移除时由管理员先备份,再只删除明确确认的 YoVision 专用数据库,禁止对共享实例做模糊或递归清理。 T-009~T-011 不迁移或删除现有 SQLite 数据,默认运行配置也仍为 SQLite。SQLite 不具备生产 Area/Outbox/Control API 一致性,业务路由不会在 SQLite 上注册。若 PostgreSQL 切换验证失败,先关闭 Control API feature flag;需要实验室回退时停止 PostgreSQL 模式进程并恢复 SQLite DSN,不得误报为生产语义等价。schema 不提供自动 down migration;需要移除时由管理员先备份,再只删除明确确认的 YoVision 专用数据库,禁止对共享实例做模糊或递归清理。
+26 -1
View File
@@ -72,7 +72,7 @@ BEGIN
RAISE EXCEPTION 'sense_app lacks access to its local audit Outbox'; RAISE EXCEPTION 'sense_app lacks access to its local audit Outbox';
END IF; END IF;
IF (SELECT max(version) FROM bell.schema_migrations) <> 2 IF (SELECT max(version) FROM bell.schema_migrations) <> 2
OR (SELECT max(version) FROM sense.schema_migrations) <> 3 THEN OR (SELECT max(version) FROM sense.schema_migrations) <> 4 THEN
RAISE EXCEPTION 'schema migration version drift'; RAISE EXCEPTION 'schema migration version drift';
END IF; END IF;
@@ -126,3 +126,28 @@ BEGIN
WHERE tenant_id = 'assertion-tenant' AND id = 'assertion-site'; WHERE tenant_id = 'assertion-tenant' AND id = 'assertion-site';
END END
$assertions$; $assertions$;
DO $control_api$
BEGIN
IF NOT has_table_privilege('yovision_t011_sense', 'sense.control_idempotency_receipts', 'SELECT,INSERT,UPDATE,DELETE')
OR NOT has_table_privilege('yovision_t011_sense', 'sense.batch_operations', 'SELECT,INSERT,UPDATE,DELETE')
OR NOT has_table_privilege('yovision_t011_sense', 'sense.batch_operation_items', 'SELECT,INSERT,UPDATE,DELETE') THEN
RAISE EXCEPTION 'Sense runtime lacks Control API table privileges';
END IF;
IF has_table_privilege('public', 'sense.control_idempotency_receipts', 'SELECT')
OR has_table_privilege('public', 'sense.batch_operations', 'SELECT') THEN
RAISE EXCEPTION 'Control API durable state leaked to PUBLIC';
END IF;
IF NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_schema = 'sense' AND table_name = 'devices'
AND column_name = 'resource_version'
) OR NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_schema = 'sense' AND table_name = 'devices'
AND column_name = 'profile_token'
) THEN
RAISE EXCEPTION 'Control API device columns are missing';
END IF;
END
$control_api$;
+1 -1
View File
@@ -85,7 +85,7 @@ MVP 以默认 16 路跑通一个场景的端到端闭环;架构、数据和 UI
python scripts/validate_agent_context.py python scripts/validate_agent_context.py
python -m unittest discover -s tests -p "test_*.py" python -m unittest discover -s tests -p "test_*.py"
python scripts/validate_harness_governance.py python scripts/validate_harness_governance.py
go -C Sense generate ./internal/mtx go -C Sense generate ./internal/mtx ./internal/controlapi
go -C Sense test ./... go -C Sense test ./...
go -C Sense vet ./... go -C Sense vet ./...
go -C Sense build ./... go -C Sense build ./...
+4 -2
View File
@@ -50,6 +50,8 @@ MediaMTX 官方 `v1.19.3` OpenAPI 固定保存于 `Sense/api/vendor/mediamtx-v1.
M2 仍保留 SQLite 作为单机开发与 T-006 回归入口,但生产真相源切换目标是 PostgreSQL。运行时必须显式设置 `SENSE_DB_DRIVER=postgres` 和私有 `SENSE_DB_DSN` 才使用 PostgreSQL;仓库不提供带密码 DSN。初始 schema 由 `deploy/postgres/` 的顺序 migration 创建,不在 Sense 进程启动时使用高权限自动迁移。 M2 仍保留 SQLite 作为单机开发与 T-006 回归入口,但生产真相源切换目标是 PostgreSQL。运行时必须显式设置 `SENSE_DB_DRIVER=postgres` 和私有 `SENSE_DB_DSN` 才使用 PostgreSQL;仓库不提供带密码 DSN。初始 schema 由 `deploy/postgres/` 的顺序 migration 创建,不在 Sense 进程启动时使用高权限自动迁移。
T-011 复用同一套冻结的 Go、PostgreSQL、pgx 与 `oapi-codegen`,未增加第三方依赖。Sense Control API v1 从仓库 OpenAPI 生成 `net/http` server glue;首版认证是可替换 port 下的外部静态 SHA-256 注册表,公共路由默认关闭且只在 PostgreSQL v4 schema 上启用。静态适配器是私有部署启动方案,不替代后续 Bell/JWT/OIDC 身份服务。
## 2. 外部项目边界 ## 2. 外部项目边界
- MiBeeNvr:只用于 M0 隔离实验室、ONVIF兼容性和交互参考,不作为生产依赖。 - MiBeeNvr:只用于 M0 隔离实验室、ONVIF兼容性和交互参考,不作为生产依赖。
@@ -86,7 +88,7 @@ Sense 单独执行:
```powershell ```powershell
go -C Sense mod download go -C Sense mod download
go -C Sense generate ./internal/mtx go -C Sense generate ./internal/mtx ./internal/controlapi
go -C Sense test ./... go -C Sense test ./...
go -C Sense vet ./... go -C Sense vet ./...
go -C Sense build ./... go -C Sense build ./...
@@ -107,7 +109,7 @@ python scripts/validate_harness_governance.py
| --- | --- | --- | --- | | --- | --- | --- | --- |
| Harness 文档/任务/Gitea 模板 | 上述三条 Python 命令 | 任一治理协议、清单或任务 schema 变化 | 不适用 | | Harness 文档/任务/Gitea 模板 | 上述三条 Python 命令 | 任一治理协议、清单或任务 schema 变化 | 不适用 |
| `docs/raw/contracts/` | JSON Schema 校验 + 契约代码断言(实现后补命令) | schema/示例/mapper 任一变化 | 生产者与消费者联合评审 | | `docs/raw/contracts/` | JSON Schema 校验 + 契约代码断言(实现后补命令) | schema/示例/mapper 任一变化 | 生产者与消费者联合评审 |
| Sense Go | `go -C Sense generate ./internal/mtx`、`go -C Sense test ./...`、`go -C Sense vet ./...`、`go -C Sense build ./...` | ONVIF、存储、MediaMTX、对账或公共 API 变化 | T-006 使用 1 路指定实机 + 4 路独立合成源;T-007 才要求客户/借用/租赁的真实多路矩阵 | | Sense Go | `go -C Sense generate ./internal/mtx ./internal/controlapi`、`go -C Sense test ./...`、`go -C Sense vet ./...`、`go -C Sense build ./...` | ONVIF、存储、MediaMTX、对账或公共 API 变化 | T-006 使用 1 路指定实机 + 4 路独立合成源;T-007 才要求客户/借用/租赁的真实多路矩阵 |
| PostgreSQL schema/repository | `python -m unittest discover -s tests -p "test_postgres_contract.py"`;Windows 本机再运行 `./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` | migration、权限、配额判定或 PostgreSQL driver 变化 | 不需要摄像头;必须核对临时集群未使用现有 data 目录,现有 5432 listener 前后不变 | | PostgreSQL schema/repository | `python -m unittest discover -s tests -p "test_postgres_contract.py"`;Windows 本机再运行 `./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` | migration、权限、配额判定或 PostgreSQL driver 变化 | 不需要摄像头;必须核对临时集群未使用现有 data 目录,现有 5432 listener 前后不变 |
| Brain Python | 单元测试、类型/格式检查(命令待项目脚手架冻结) | mapper、判定状态机、模型接口变化 | 命中模型任务时用冻结数据集和目标硬件 | | Brain Python | 单元测试、类型/格式检查(命令待项目脚手架冻结) | mapper、判定状态机、模型接口变化 | 命中模型任务时用冻结数据集和目标硬件 |
| Bell Go/Web | 后端测试 + 前端 lint/test/build(命令待脚手架冻结) | schema、RBAC、预警状态机或公共 UI 变化 | P0 流程由产品/值班角色验收 | | Bell Go/Web | 后端测试 + 前端 lint/test/build(命令待脚手架冻结) | schema、RBAC、预警状态机或公共 UI 变化 | P0 流程由产品/值班角色验收 |
+1 -1
View File
@@ -95,7 +95,7 @@ Sense ── 视频流/触发信号 ──> Brain
## 8. 数据与契约 ## 8. 数据与契约
- Bell 核心实体:Tenant → Site → Area(含 `capture_policy`)以及 Role/Binding/Quota/Audit;Sense 核心实体:Device(含 `modality + capabilities`)→ StreamBinding/Zone,以及只记录已观察版本的 SiteQuota/AreaPolicyProjection。两个 schema 以稳定逻辑 ID 关联,不跨 schema 写入;配额 v1 为五列,Area v1 固定为 `tenant_id/site_id/area_id/capture_policy/source_version/source_updated_at` 六列。 - Bell 核心实体:Tenant → Site → Area(含 `capture_policy`)以及 Role/Binding/Quota/Audit;Sense 核心实体:Device(含 `modality + capabilities`)→ StreamBinding/Zone,以及只记录已观察版本的 SiteQuota/AreaPolicyProjection。两个 schema 以稳定逻辑 ID 关联,不跨 schema 写入;配额 v1 为五列,Area v1 固定为 `tenant_id/site_id/area_id/capture_policy/source_version/source_updated_at` 六列。
- Sense Control API v1 使用站点作用域路径、认证上下文 tenant、cursor 分页、幂等键与 ETag;敏感连接引用只写不读。正式签名和兼容规则以 [`contracts/`](contracts/) 为准。T-009/T-010 已实现 PostgreSQL 准入与审计事务基础,不等于 handler、认证、Bell 管理服务或 Outbox relay 已实现。 - Sense Control API v1 使用站点作用域路径、认证上下文 tenant、HMAC cursor 分页、PostgreSQL 幂等收据与资源 ETag;敏感连接引用只写不读。T-011 已实现 7 个 handler,并以 feature flag 限定到 PostgreSQL 路径;首版外部静态 SHA-256 注册表只实现认证 port 的私有部署适配器。正式签名和兼容规则以 [`contracts/`](contracts/) 为准,Bell 管理服务、JWT/OIDC 与 Outbox relay 仍未实现。
- 业务实体:Rule → Event → Alert → DeliveryAttempt/Ack;Event 与 Alert 不合并。 - 业务实体:Rule → Event → Alert → DeliveryAttempt/Ack;Event 与 Alert 不合并。
- Bell 通知域分为三个聚合:Contact/Team 保存身份、成员关系和已验证通道;OnCallSchedule/ScheduleVersion/ShiftException 保存时区、轮换与例外;EscalationPolicy/Step 通过 `person / team / on_call_schedule` 类型化 `target_ref` 引用目标。三者共享逻辑 ID,不复制手机号、班次或轮换字段。 - Bell 通知域分为三个聚合:Contact/Team 保存身份、成员关系和已验证通道;OnCallSchedule/ScheduleVersion/ShiftException 保存时区、轮换与例外;EscalationPolicy/Step 通过 `person / team / on_call_schedule` 类型化 `target_ref` 引用目标。三者共享逻辑 ID,不复制手机号、班次或轮换字段。
- 每个 DeliveryAttempt 创建时解析当时生效的排班版本,并保存实际收件人、通道、`schedule_version` 和解析时间快照;之后联系人或排班修改不得回写既有投递事实。 - 每个 DeliveryAttempt 创建时解析当时生效的排班版本,并保存实际收件人、通道、`schedule_version` 和解析时间快照;之后联系人或排班修改不得回写既有投递事实。
+1 -1
View File
@@ -25,7 +25,7 @@
- T-008:冻结 Sense Control API v1 与 Bell `site_quota_v1` 只读投影契约;只建立接口和测试门禁,不等于 handler、Bell 表或 migration 已实现。 - T-008:冻结 Sense Control API v1 与 Bell `site_quota_v1` 只读投影契约;只建立接口和测试门禁,不等于 handler、Bell 表或 migration 已实现。
- T-009:使用 PostgreSQL 17.10 建立 `sense`/`bell` schema、Bell 配额源表/只读视图、最小权限和 Sense PostgreSQL repository;SQLite 保留为 M1 本地路径。 - T-009:使用 PostgreSQL 17.10 建立 `sense`/`bell` schema、Bell 配额源表/只读视图、最小权限和 Sense PostgreSQL repository;SQLite 保留为 M1 本地路径。
- T-010:建立 Bell Area/`capture_policy` 只读投影、Sense 成像准入与设备操作审计 Outbox 原子写入;relay 传输协议留给后续任务。 - T-010:建立 Bell Area/`capture_policy` 只读投影、Sense 成像准入与设备操作审计 Outbox 原子写入;relay 传输协议留给后续任务。
- 后续 T-011 按 T-008 契约实现认证 tenant 上下文、设备管理 handler、幂等/ETag/cursor 与批量操作;公共控制 API 只在 PostgreSQL 路径启用,不得把 T-009/T-010 数据基础误报为 API 已交付。 - T-011:按 T-008 契约实现认证 tenant 上下文、7 个设备/operation handler、PostgreSQL 幂等收据、ETag/HMAC cursor、最多 128 项批量操作和停用精确 path 收敛;公共控制 API 默认关闭且只在 PostgreSQL 路径启用。
- 对账器幂等/退避/并发/10% 安全闸。 - 对账器幂等/退避/并发/10% 安全闸。
- WireGuard 边缘隧道与断网恢复。 - WireGuard 边缘隧道与断网恢复。
- 16 路批量开通、停用和容量基准。 - 16 路批量开通、停用和容量基准。
+3 -3
View File
@@ -1,6 +1,6 @@
# API 与契约 # API 与契约
> Brain → Bell 事件契约 v0.1、Sense Control API v1、Bell 配额/Area 只读投影 v1 与 Sense 本地设备审计事件 v1 已冻结;其他 API 仍在设计阶段。不得把本文的“待定”自行具体化为公共契约。 > Brain → Bell 事件契约 v0.1、Sense Control API v1、Bell 配额/Area 只读投影 v1 与 Sense 本地设备审计事件 v1/v2 已冻结;其他 API 仍在设计阶段。不得把本文的“待定”自行具体化为公共契约。
## 1. 已冻结:Brain → Bell 事件契约 ## 1. 已冻结:Brain → Bell 事件契约
@@ -45,7 +45,7 @@
- `endpoint_ref`、`credential_ref`、`profile_token` 只写不读;设备 ID 由服务端生成。普通响应和错误不得包含凭据、完整流 URI、token 或 MediaMTX 内部配置。 - `endpoint_ref`、`credential_ref`、`profile_token` 只写不读;设备 ID 由服务端生成。普通响应和错误不得包含凭据、完整流 URI、token 或 MediaMTX 内部配置。
- v1 不提供删除设备;停用设备保留历史。写入受理只表示期望态已持久化,不能表示实际态已收敛。 - v1 不提供删除设备;停用设备保留历史。写入受理只表示期望态已持久化,不能表示实际态已收敛。
T-008 冻结公共控制契约;T-009/T-010 已实现 PostgreSQL Site/Area 源表、两个只读视图、最小权限、Sense 准入 repository 和本地审计 Outbox,但尚未实现 Sense HTTP handler、认证中间件、Bell 管理服务或 Outbox relay。 T-008 冻结公共控制契约;T-009/T-010 建立 PostgreSQL 投影、准入和本地审计基础;T-011 已实现 7 个 HTTP handler、外部静态摘要认证适配器、tenant/Site scope、幂等收据、ETag/HMAC cursor 和持久化 batch operation。业务路由默认关闭且仅可在 PostgreSQL 上开启;Bell 管理服务、JWT/OIDC 和 Outbox relay 尚未实现。
## 4. 待冻结的 Bell 公共 API ## 4. 待冻结的 Bell 公共 API
@@ -83,7 +83,7 @@ MediaMTX 薄封装调用同版官方 OpenAPI 的 `/v3/config/paths/get|add|patch
### 5.3 Sense 进程 HTTP 面 ### 5.3 Sense 进程 HTTP 面
T-003 只实现了运维探针:`GET /healthz` 表示进程存活,`GET /readyz` 表示所选数据库已打开且 schema/权限前置检查完成。两者返回 JSON,均不等价于摄像头、MediaMTX path 或里程碑健康。T-008 已冻结站点作用域的 `/api/v1/sites/{site_id}/devices` 等设备管理契约,但尚未实现对应 handler;当前可运行进程仍只暴露探针,后续实现不得发布 `/api/v1/devices` 等无站点边界的临时接口。 `GET /healthz` 表示进程存活,`GET /readyz` 表示所选数据库已打开且 schema/权限前置检查完成;两者不要求认证,也不等价于摄像头、MediaMTX path 或里程碑健康。T-011 在 `SENSE_CONTROL_API_ENABLED=true`、PostgreSQL v4 schema 和外部安全文件全部有效时注册冻结的 `/api/v1` 路由;默认 SQLite 运行仍只暴露探针。业务路由不提供无 Site 边界的 `/api/v1/devices` 临时接口。
## 6. 变更流程 ## 6. 变更流程
+6 -3
View File
@@ -1,6 +1,6 @@
# Sense 控制面、准入投影与本地审计契约 v1 # Sense 控制面、准入投影与本地审计契约 v1
> 冻结日期:2026-08-07。契约版本:`1.0.0`。Sense 是设备期望态的提供方;Bell 是 Tenant、Site、Area、RBAC、配额与全局审计的所有者。T-009/T-010 已实现两个只读投影和本地审计事务基础;HTTP handler、Bell 管理服务和 Outbox relay 仍未实现。 > 冻结日期:2026-08-07。Control API 契约版本:`1.0.0`。Sense 是设备期望态的提供方;Bell 是 Tenant、Site、Area、RBAC、配额与全局审计的所有者。T-011 已实现 Control API handler 与 PostgreSQL 一致性边界;Bell 管理服务、JWT/OIDC 和 Outbox relay 仍未实现。
## 契约文件 ## 契约文件
@@ -10,6 +10,7 @@
| [`site-quota-v1.sql`](site-quota-v1.sql) | Bell | Sense | 单 PostgreSQL 实例内的站点视频配额只读投影 | | [`site-quota-v1.sql`](site-quota-v1.sql) | Bell | Sense | 单 PostgreSQL 实例内的站点视频配额只读投影 |
| [`area-policy-v1.sql`](area-policy-v1.sql) | Bell | Sense | Area 归属与 `capture_policy` 只读投影 | | [`area-policy-v1.sql`](area-policy-v1.sql) | Bell | Sense | Area 归属与 `capture_policy` 只读投影 |
| [`sense-device-audit-v1.schema.json`](sense-device-audit-v1.schema.json) | Sense | 本地 Outbox;未来 Bell relay | 脱敏设备操作审计事实,不包含传输协议 | | [`sense-device-audit-v1.schema.json`](sense-device-audit-v1.schema.json) | Sense | 本地 Outbox;未来 Bell relay | 脱敏设备操作审计事实,不包含传输协议 |
| [`sense-device-audit-v2.schema.json`](sense-device-audit-v2.schema.json) | Sense | 本地 Outbox;未来 Bell relay | v1 后继,增加脱敏配置修改受理事实;v1 文件保持不变 |
OpenAPI 的 `/api/v1` 路径是公共控制面边界;`/healthz`、`/readyz` 仍是非业务运维探针。v1 不提供设备删除:停用设备使用期望态接口,保留设备、操作和审计历史。Site、Area、配额、RBAC 和审计聚合不由 Sense 提供 CRUD。 OpenAPI 的 `/api/v1` 路径是公共控制面边界;`/healthz`、`/readyz` 仍是非业务运维探针。v1 不提供设备删除:停用设备使用期望态接口,保留设备、操作和审计历史。Site、Area、配额、RBAC 和审计聚合不由 Sense 提供 CRUD。
@@ -54,7 +55,7 @@ T-010 冻结 `bell.area_policy_v1` 的列顺序为 `tenant_id/site_id/area_id/ca
## 本地设备操作审计 ## 本地设备操作审计
`sense-device-audit-v1.schema.json` 冻结本地审计事实的逻辑 envelope。当前事件只有 `device.created` 和 `device.desired_state.accepted`;主体类型为 `user | service | system`,投影版本与 generation 随事实保存。`data` 只包含 Area、模态、能力和状态变化等脱敏字段,禁止 endpoint、credential、profile token、path、密码、完整流 URI 或 MediaMTX 配置。 `sense-device-audit-v1.schema.json` 继续冻结创建与期望态两类事实且不原地扩展严格枚举。T-011 新增 v2 后继,兼容 v1 两类事件并增加 `device.configuration.accepted`;该 payload 只保存是否变化、字段名和 Area 逻辑 ID,不保存字段值。主体类型为 `user | service | system`,投影版本与 generation 随事实保存;endpoint、credential、profile token、path、密码、完整流 URI 或 MediaMTX 配置始终禁止进入审计。
PostgreSQL repository 必须在设备创建/期望态事务内写 `sense.device_operation_outbox`;Outbox 失败回滚业务写入。相同期望态不增加 generation,但仍产生独立审计事实。Schema 不是 Bell relay 协议:传输端点、签名、批量确认、重放窗口和留存由后续任务冻结。 PostgreSQL repository 必须在设备创建/期望态事务内写 `sense.device_operation_outbox`;Outbox 失败回滚业务写入。相同期望态不增加 generation,但仍产生独立审计事实。Schema 不是 Bell relay 协议:传输端点、签名、批量确认、重放窗口和留存由后续任务冻结。
@@ -71,7 +72,9 @@ PostgreSQL repository 必须在设备创建/期望态事务内写 `sense.device_
```powershell ```powershell
python -m json.tool docs/contracts/sense-control-v1.openapi.json | Out-Null python -m json.tool docs/contracts/sense-control-v1.openapi.json | Out-Null
python -m json.tool docs/contracts/sense-device-audit-v2.schema.json | Out-Null
python -m unittest discover -s tests -p "test_sense_control_contract.py" python -m unittest discover -s tests -p "test_sense_control_contract.py"
python -m unittest discover -s tests -p "test_sense_control_implementation.py"
``` ```
测试校验本仓库依赖的 OpenAPI 结构与安全不变量,并不替代后续实现任务对完整 OpenAPI 标准验证器、HTTP handler 和 PostgreSQL migration 的验证。 测试同时校验 OpenAPI 结构、生成 server glue、HTTP handler 与 PostgreSQL migration/事务;它不替代 Bell 消费方联合验收或客户现场容量验证。
@@ -0,0 +1,124 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://yovision.local/contracts/sense-device-audit-v2.schema.json",
"title": "Sense Device Audit Event v2",
"description": "v1 的向后兼容后继:增加脱敏设备配置受理事实,仍不定义 Bell relay。",
"type": "object",
"additionalProperties": false,
"required": [
"event_id", "event_type", "tenant_id", "site_id", "device_id",
"actor", "reason", "trace_id", "aggregate_generation",
"projection_versions", "data", "occurred_at"
],
"properties": {
"event_id": {"type": "string", "pattern": "^audit_[0-9a-f]{32}$"},
"event_type": {
"enum": [
"device.created",
"device.desired_state.accepted",
"device.configuration.accepted"
]
},
"tenant_id": {"$ref": "#/$defs/logicalId"},
"site_id": {"$ref": "#/$defs/logicalId"},
"device_id": {"$ref": "#/$defs/logicalId"},
"actor": {
"type": "object",
"additionalProperties": false,
"required": ["type", "id"],
"properties": {
"type": {"enum": ["user", "service", "system"]},
"id": {"type": "string", "minLength": 1, "maxLength": 200}
}
},
"reason": {"type": ["string", "null"], "maxLength": 500},
"trace_id": {"type": ["string", "null"], "maxLength": 128},
"aggregate_generation": {"type": "integer", "minimum": 1},
"projection_versions": {
"type": "object",
"additionalProperties": false,
"required": ["quota_source_version", "area_policy_source_version"],
"properties": {
"quota_source_version": {"type": ["integer", "null"], "minimum": 1},
"area_policy_source_version": {"type": ["integer", "null"], "minimum": 1}
}
},
"data": {
"oneOf": [
{"$ref": "#/$defs/deviceCreatedData"},
{"$ref": "#/$defs/desiredStateData"},
{"$ref": "#/$defs/configurationData"}
]
},
"occurred_at": {"type": "string", "format": "date-time"}
},
"allOf": [
{
"if": {"properties": {"event_type": {"const": "device.created"}}},
"then": {"properties": {"data": {"$ref": "#/$defs/deviceCreatedData"}}}
},
{
"if": {"properties": {"event_type": {"const": "device.desired_state.accepted"}}},
"then": {"properties": {"data": {"$ref": "#/$defs/desiredStateData"}}}
},
{
"if": {"properties": {"event_type": {"const": "device.configuration.accepted"}}},
"then": {"properties": {"data": {"$ref": "#/$defs/configurationData"}}}
}
],
"$defs": {
"logicalId": {
"type": "string",
"minLength": 1,
"maxLength": 128,
"pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$"
},
"desiredState": {"enum": ["disabled", "enabled"]},
"deviceCreatedData": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "area_id", "modality", "capabilities", "desired_state"],
"properties": {
"kind": {"const": "device_created"},
"area_id": {"$ref": "#/$defs/logicalId"},
"modality": {"enum": ["video", "radar", "contact", "button", "wearable", "other"]},
"capabilities": {
"type": "array",
"maxItems": 16,
"uniqueItems": true,
"items": {"enum": ["video_capture", "audio_capture", "spatial_rule", "telemetry"]}
},
"desired_state": {"$ref": "#/$defs/desiredState"}
}
},
"desiredStateData": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "previous_desired_state", "desired_state", "changed"],
"properties": {
"kind": {"const": "desired_state_accepted"},
"previous_desired_state": {"$ref": "#/$defs/desiredState"},
"desired_state": {"$ref": "#/$defs/desiredState"},
"changed": {"type": "boolean"}
}
},
"configurationData": {
"type": "object",
"additionalProperties": false,
"required": ["kind", "changed", "changed_fields", "area_id"],
"properties": {
"kind": {"const": "configuration_accepted"},
"changed": {"type": "boolean"},
"changed_fields": {
"type": "array",
"maxItems": 5,
"uniqueItems": true,
"items": {
"enum": ["name", "area_id", "endpoint_ref", "credential_ref", "profile_token"]
}
},
"area_id": {"$ref": "#/$defs/logicalId"}
}
}
}
}
+3 -3
View File
@@ -20,7 +20,7 @@
- T-006 正式使用 1 台准入实机和 4 个独立合成 publisher 连续观察 `1806.6 s` / 180 次采样,四类恢复均通过,最大与最终 `unconverged` 均为 0;详细证据见 `docs/research/sense-5-stream-integration.md`。 - T-006 正式使用 1 台准入实机和 4 个独立合成 publisher 连续观察 `1806.6 s` / 180 次采样,四类恢复均通过,最大与最终 `unconverged` 均为 0;详细证据见 `docs/research/sense-5-stream-integration.md`。
- `docs/raw/01`~`08` 已记录需求、分析、方案、客户场景、事件比对和三系统职责。 - `docs/raw/01`~`08` 已记录需求、分析、方案、客户场景、事件比对和三系统职责。
- `docs/raw/contracts/event-v0.1.schema.json` 已冻结,并有多份示例与语义说明。 - `docs/raw/contracts/event-v0.1.schema.json` 已冻结,并有多份示例与语义说明。
- `docs/contracts/sense-control-v1.openapi.json` 已冻结站点作用域的设备查询、创建、修改、启停与最多 128 项批量操作;配额/Area 只读视图和本地设备审计事件均已冻结并实现数据库基础。HTTP handler、认证、Bell 管理服务和 Outbox relay 尚未实现。 - `docs/contracts/sense-control-v1.openapi.json` 的 7 个站点作用域/operation endpoint 已由 T-011 实现:外部静态 SHA-256 主体注册表、tenant/Site scope、HMAC cursor、ETag、PostgreSQL 24 小时幂等收据和最多 128 项 batch operation 均有代码与隔离集成测试。默认 SQLite 仍只暴露探针;Bell 管理服务、JWT/OIDC 和 Outbox relay 尚未实现。
- harness coding 文档、上下文清单、Gitea Issue/PR 模板和治理脚本已接入。 - harness coding 文档、上下文清单、Gitea Issue/PR 模板和治理脚本已接入。
- Gitea 已初始化 12 个协作标签;`status/waiting` 用于依赖或外部条件未满足的未领取任务,实时可领取状态必须从 Gitea 查询,不在本文复制。 - Gitea 已初始化 12 个协作标签;`status/waiting` 用于依赖或外部条件未满足的未领取任务,实时可领取状态必须从 Gitea 查询,不在本文复制。
- T-002 已关闭架构影响型问题:首期 S2 民办寄宿学校、客户侧私有部署、ONVIF/RTSP、NVIDIA 主路径、自研 Bell、Web/H5 与客户侧证据存储等边界已批准。 - T-002 已关闭架构影响型问题:首期 S2 民办寄宿学校、客户侧私有部署、ONVIF/RTSP、NVIDIA 主路径、自研 Bell、Web/H5 与客户侧证据存储等边界已批准。
@@ -66,12 +66,12 @@ Sense 默认监听 `127.0.0.1:8080`,提供 `/healthz` 与 `/readyz` 运维探
- 人脸方向已延后至 M5 的 S4 成人园区候选试点;必要性/PIP 影响评估、单独同意与替代方式、合法底库来源和删除流程未完成,阻塞人脸能力上线。 - 人脸方向已延后至 M5 的 S4 成人园区候选试点;必要性/PIP 影响评估、单独同意与替代方式、合法底库来源和删除流程未完成,阻塞人脸能力上线。
- 短信/语音具体供应商未选;生产前必须选定两条独立投递路径并验证故障切换。 - 短信/语音具体供应商未选;生产前必须选定两条独立投递路径并验证故障切换。
- Python/Savant 的精确版本、目标硬件和 Bell 前端栈尚未冻结;Sense M1 的 Go、SQLite driver、MediaMTX、生成器及生成运行时版本已在 T-003 冻结,PostgreSQL/pgx 版本已在 T-009 冻结。 - Python/Savant 的精确版本、目标硬件和 Bell 前端栈尚未冻结;Sense M1 的 Go、SQLite driver、MediaMTX、生成器及生成运行时版本已在 T-003 冻结,PostgreSQL/pgx 版本已在 T-009 冻结。
- 本机现有 PostgreSQL 5432 实例使用 SCRAM 且当前开发进程没有管理员密码;T-009/T-010 不绕过认证,自动验收使用隔离临时集群。向共享/生产实例安装 migration 前仍需管理员私下提供专用数据库、登录角色与备份方案。 - 本机现有 PostgreSQL 5432 实例使用 SCRAM 且当前开发进程没有管理员密码;T-009~T-011 不绕过认证,自动验收使用隔离临时集群。向共享/生产实例安装 migration 前仍需管理员私下提供专用数据库、登录角色、外部 Control API 安全文件与备份方案。
- 代码知识图谱在无业务代码阶段可能为空;工具不可用时使用 `rg` 处理文档与配置。 - 代码知识图谱在无业务代码阶段可能为空;工具不可用时使用 `rg` 处理文档与配置。
## 下一步 ## 下一步
下一步按 Gitea 流程建立 T-011 Sense Control API v1 实现任务:基于 T-009/T-010 PostgreSQL repository 实现认证 tenant 上下文、7 个设备/operation handler、幂等收据、ETag/cursor 和最多 128 项批量操作;公共 API 不在 SQLite 路径启用。客户授权、借用或租赁条件具备后再执行 T-007 五条独立真实上游现场门禁。T-006 的合成结果不解除 T-007,也不形成容量或生产 SLA 承诺。 下一项建议创建 T-012,补齐 M2 对账安全闸、孤儿检测(只报告/受控处置)与多实例可观测性,再进入 WireGuard 和 16 路批量开通基准;不要把 Control API 完成误报为 M2 容量出口。客户授权、借用或租赁条件具备后再执行 T-007 五条独立真实上游现场门禁。T-006 的合成结果不解除 T-007,也不形成容量或生产 SLA 承诺。
## 已知风险 ## 已知风险
+20 -4
View File
@@ -3,12 +3,12 @@ id: T-011
title: 实现 Sense Control API v1 与 PostgreSQL 一致性边界 title: 实现 Sense Control API v1 与 PostgreSQL 一致性边界
phase: 2 phase: 2
deps: [T-010] deps: [T-010]
status: TODO status: DONE
created: 2026-08-07 created: 2026-08-07
issue: 39 issue: 39
context_ref: null context_ref: 6402d4384d5c2b1c0f3852858c3cf0d63d948262
claim_branch: null claim_branch: claims/T-011
work_branch: null work_branch: agent/codex/T-011
write_paths: write_paths:
- docs/tasks/T-011.md - docs/tasks/T-011.md
- docs/contracts/ - docs/contracts/
@@ -26,6 +26,7 @@ write_paths:
- Sense/internal/reconcile/ - Sense/internal/reconcile/
- Sense/internal/store/ - Sense/internal/store/
- Sense/README.md - Sense/README.md
- docs/00-ai-start-here.md
- docs/03-tech-stack.md - docs/03-tech-stack.md
- docs/04-architecture.md - docs/04-architecture.md
- docs/06-tasks.md - docs/06-tasks.md
@@ -95,6 +96,21 @@ T-008 已冻结 Sense Control API v1 的 7 个 endpoint,T-009/T-010 已提供
## 执行记录 ## 执行记录
### 2026-08-07 完成 Sense Control API v1
- 使用冻结的 `oapi-codegen v2.8.0` 从 T-008 OpenAPI 生成 Go 1.22+ `net/http` server glue(生成 SHA-256 `39a52e4f54bb1a742f58b64dcf15ce98359385236da5ccc0dae77e8ded2b218b`),实现全部 7 个 endpoint、严格 JSON/1 MiB body、稳定 Problem、write-only 字段脱敏和 `Cache-Control: no-store`;生成漂移已进入 `init.ps1`/`init.sh`。
- 新增可替换认证 port 与 `static-sha256` 私有部署适配器:外部注册表只保存 token 摘要、主体、tenant、Site scope 与两项权限,摘要 constant-time 比较;业务路由默认关闭,只能在 PostgreSQL v4 schema 与外部 32 字节 HMAC cursor key 有效时开启。SQLite 默认路径继续只暴露探针。
- 新增 `008`/`009` migration:设备 `resource_version`/write-only profile token、24 小时摘要幂等收据、持久化 batch operation/逐项结果与最小权限。创建收据与设备/审计同事务;batch 以 savepoint 隔离逐项失败,并先按设备 ID 排序锁定目标,避免相反请求顺序死锁。ETag 只在配置/期望态实际变化时前进;相同期望态仍审计但不增加 generation/resource version。
- 保留严格审计 v1 文件不变,新增 v2 后继和 `device.configuration.accepted` 脱敏事实;Area 修改重新准入。停用调和只对台账中的精确 path 做幂等删除,成功后 observed generation 收敛且 actual state 为 offline,不枚举或清理未知 path。
- `./scripts/test_postgres.ps1 -PgRoot D:\pgsql17` 通过:`001`~`009` 连续重放、SQL 权限断言和 25 个 `TestPostgres*` 全绿,覆盖列表稳定分页/过滤、创建并发幂等、收据脱敏/冲突、ETag/PATCH、审计 v2、逐项 batch、operation tenant 隐藏、相反顺序批量锁与 PUBLIC 权限负例;随机回环临时集群已停止并清理,现有 `D:\pgsql17\data`/5432 listener 未读取、停止或修改。
- `./init.ps1` 通过;Python 契约/治理测试共 47 项通过;`go -C Sense test ./...`、`go -C Sense vet ./...`、`go -C Sense build ./...`、生成漂移和 `git diff --check` 均通过;`go test -race ./internal/auth ./internal/controlapi ./internal/reconcile ./internal/store` 通过。不需要摄像头或 UI,本结果不解除 T-007,也不形成 16/128 路容量承诺。
### 2026-08-07 领取任务
- dispatcher `ila` 将 Issue #39 分配给 `codex`;`context_ref` 为 `6402d4384d5c2b1c0f3852858c3cf0d63d948262`,claim 为 `claims/T-011`,工作分支为 `agent/codex/T-011`。
- 已读回 Issue `status/doing`、assignee、dispatcher 发布的结构化 CLAIM 与两个分支 SHA;接受 frontmatter 全部写路径。T-007 仍为 waiting,当前没有活跃写路径冲突。
- 实现中发现标准生成入口还需同步 `docs/00-ai-start-here.md`;dispatcher 复查无活跃冲突后发布完整 CLAIM RENEWAL,本任务据此增加该精确路径。
### 2026-08-07 Gitea 映射 ### 2026-08-07 Gitea 映射
- 任务规格先合入默认分支,再创建唯一主 Issue #39;本提交只回填双向映射,映射合入并读回前不领取任务。 - 任务规格先合入默认分支,再创建唯一主 Issue #39;本提交只回填双向映射,映射合入并读回前不领取任务。
+1 -1
View File
@@ -12,7 +12,7 @@ Set-Location -Path $PSScriptRoot
# Sense 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。 # Sense 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。
$InstallCmd = "go -C Sense mod download" $InstallCmd = "go -C Sense mod download"
$VerifyCmd = "python scripts/validate_agent_context.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python -m unittest discover -s tests -p 'test_*.py'; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python scripts/validate_harness_governance.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense generate ./internal/mtx; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense test ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense vet ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense build ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }" $VerifyCmd = "python scripts/validate_agent_context.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python -m unittest discover -s tests -p 'test_*.py'; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; python scripts/validate_harness_governance.py; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense generate ./internal/mtx ./internal/controlapi; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go Sense/internal/controlapi/generated.gen.go; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense test ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense vet ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }; go -C Sense build ./...; if (`$LASTEXITCODE -ne 0) { exit `$LASTEXITCODE }"
$StartCmd = "go -C Sense run ./cmd/sense-api" $StartCmd = "go -C Sense run ./cmd/sense-api"
function Assert-Configured { function Assert-Configured {
+1 -1
View File
@@ -14,7 +14,7 @@ cd "$ROOT_DIR"
# Sense 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。 # Sense 使用锁定 Go toolchain/module;生成漂移、测试、vet 与构建均进入标准门禁。
INSTALL_CMD=(go -C Sense mod download) INSTALL_CMD=(go -C Sense mod download)
VERIFY_CMD=(bash -lc "python3 scripts/validate_agent_context.py && python3 -m unittest discover -s tests -p 'test_*.py' && python3 scripts/validate_harness_governance.py && go -C Sense generate ./internal/mtx && git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go && go -C Sense test ./... && go -C Sense vet ./... && go -C Sense build ./...") VERIFY_CMD=(bash -lc "python3 scripts/validate_agent_context.py && python3 -m unittest discover -s tests -p 'test_*.py' && python3 scripts/validate_harness_governance.py && go -C Sense generate ./internal/mtx ./internal/controlapi && git diff --exit-code -- Sense/internal/mtx/generated/client.gen.go Sense/internal/controlapi/generated.gen.go && go -C Sense test ./... && go -C Sense vet ./... && go -C Sense build ./...")
START_CMD=(go -C Sense run ./cmd/sense-api) START_CMD=(go -C Sense run ./cmd/sense-api)
ensure_configured() { ensure_configured() {
+8 -6
View File
@@ -40,7 +40,7 @@ $port = ([System.Net.IPEndPoint]$listener.LocalEndpoint).Port
$listener.Stop() $listener.Stop()
$systemTemp = [IO.Path]::GetFullPath([IO.Path]::GetTempPath()) $systemTemp = [IO.Path]::GetFullPath([IO.Path]::GetTempPath())
$clusterName = 'yovision-t010-pg-' + [guid]::NewGuid().ToString('N') $clusterName = 'yovision-t011-pg-' + [guid]::NewGuid().ToString('N')
$dataDir = Join-Path $systemTemp $clusterName $dataDir = Join-Path $systemTemp $clusterName
$logPath = Join-Path $systemTemp ($clusterName + '.log') $logPath = Join-Path $systemTemp ($clusterName + '.log')
$started = $false $started = $false
@@ -69,9 +69,9 @@ try {
$started = $true $started = $true
$adminRootDSN = "postgres://postgres@127.0.0.1:$port/postgres?sslmode=disable" $adminRootDSN = "postgres://postgres@127.0.0.1:$port/postgres?sslmode=disable"
$databaseName = 'yovision_t010' $databaseName = 'yovision_t011'
$adminDatabaseDSN = "postgres://postgres@127.0.0.1:$port/${databaseName}?sslmode=disable" $adminDatabaseDSN = "postgres://postgres@127.0.0.1:$port/${databaseName}?sslmode=disable"
$senseDSN = "postgres://yovision_t010_sense@127.0.0.1:$port/${databaseName}?sslmode=disable" $senseDSN = "postgres://yovision_t011_sense@127.0.0.1:$port/${databaseName}?sslmode=disable"
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-f' (Join-Path $repoRoot 'deploy\postgres\001_roles.sql') Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-f' (Join-Path $repoRoot 'deploy\postgres\001_roles.sql')
Invoke-Checked $createdb '-h' '127.0.0.1' '-p' ([string]$port) '-U' 'postgres' $databaseName Invoke-Checked $createdb '-h' '127.0.0.1' '-p' ([string]$port) '-U' 'postgres' $databaseName
@@ -83,12 +83,14 @@ try {
'004_privileges.sql', '004_privileges.sql',
'005_area_policy.sql', '005_area_policy.sql',
'006_device_operation_outbox.sql', '006_device_operation_outbox.sql',
'007_privileges_area_audit.sql' '007_privileges_area_audit.sql',
'008_control_api.sql',
'009_privileges_control_api.sql'
)) { )) {
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminDatabaseDSN '-f' (Join-Path $repoRoot "deploy\postgres\$name") Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminDatabaseDSN '-f' (Join-Path $repoRoot "deploy\postgres\$name")
} }
} }
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-c' 'CREATE ROLE yovision_t010_sense LOGIN IN ROLE sense_app' Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminRootDSN '-c' 'CREATE ROLE yovision_t011_sense LOGIN IN ROLE sense_app'
Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminDatabaseDSN '-f' (Join-Path $repoRoot 'deploy\postgres\tests\assertions.sql') Invoke-Checked $psql '-X' '-v' 'ON_ERROR_STOP=1' '-d' $adminDatabaseDSN '-f' (Join-Path $repoRoot 'deploy\postgres\tests\assertions.sql')
$env:YOVISION_TEST_POSTGRES_DSN = $senseDSN $env:YOVISION_TEST_POSTGRES_DSN = $senseDSN
@@ -103,7 +105,7 @@ finally {
if (-not $started -or $stopped) { if (-not $started -or $stopped) {
$resolvedData = [IO.Path]::GetFullPath($dataDir) $resolvedData = [IO.Path]::GetFullPath($dataDir)
if (-not $resolvedData.StartsWith($systemTemp, [StringComparison]::OrdinalIgnoreCase) -or if (-not $resolvedData.StartsWith($systemTemp, [StringComparison]::OrdinalIgnoreCase) -or
[IO.Path]::GetFileName($resolvedData) -notlike 'yovision-t010-pg-*') { [IO.Path]::GetFileName($resolvedData) -notlike 'yovision-t011-pg-*') {
throw "Refusing to clean unexpected temporary path." throw "Refusing to clean unexpected temporary path."
} }
if (Test-Path -LiteralPath $resolvedData) { if (Test-Path -LiteralPath $resolvedData) {
+3 -1
View File
@@ -54,6 +54,8 @@ class PostgresContractTests(unittest.TestCase):
"005_area_policy.sql", "005_area_policy.sql",
"006_device_operation_outbox.sql", "006_device_operation_outbox.sql",
"007_privileges_area_audit.sql", "007_privileges_area_audit.sql",
"008_control_api.sql",
"009_privileges_control_api.sql",
], ],
names, names,
) )
@@ -107,7 +109,7 @@ class PostgresContractTests(unittest.TestCase):
"initdb.exe", "initdb.exe",
"pg_ctl.exe", "pg_ctl.exe",
"127.0.0.1", "127.0.0.1",
"yovision-t010-pg-", "yovision-t011-pg-",
"YOVISION_TEST_POSTGRES_DSN", "YOVISION_TEST_POSTGRES_DSN",
"Get-NetTCPConnection", "Get-NetTCPConnection",
): ):
@@ -0,0 +1,99 @@
"""Static safety checks for the T-011 Sense Control API implementation."""
from __future__ import annotations
import json
import pathlib
import re
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[1]
def text(relative: str) -> str:
return (ROOT / relative).read_text(encoding="utf-8")
def normalized(value: str) -> str:
return re.sub(r"\s+", " ", value.lower())
class SenseControlImplementationTests(unittest.TestCase):
def test_generated_server_covers_all_frozen_operations(self) -> None:
generated = text("Sense/internal/controlapi/generated.gen.go")
self.assertIn("Code generated by github.com/oapi-codegen/oapi-codegen/v2", generated)
for operation in (
"ListDevices",
"CreateDevice",
"GetDevice",
"UpdateDevice",
"SetDeviceDesiredState",
"BatchSetDeviceDesiredState",
"GetOperation",
):
self.assertIn(operation, generated)
config = text("Sense/internal/controlapi/oapi-codegen.yaml")
self.assertIn("std-http-server: true", config)
self.assertIn("./internal/controlapi", text("init.ps1"))
def test_control_api_is_feature_gated_to_postgres(self) -> None:
config = text("Sense/internal/config/config.go")
main = text("Sense/cmd/sense-api/main.go")
self.assertIn('SENSE_CONTROL_API_ENABLED', config)
self.assertIn('Control API requires SENSE_DB_DRIVER=postgres', config)
self.assertIn('mux.Handle("/api/v1/", controlHandler)', main)
self.assertNotIn('mux.Handle("/api/v1/"', text("Sense/internal/store/sqlite.go"))
def test_static_authentication_stores_only_digests(self) -> None:
source = text("Sense/internal/auth/auth.go")
example = json.loads(text("Sense/api/control-auth.example.json"))
self.assertIn("subtle.ConstantTimeCompare", source)
self.assertIn("sha256.Sum256([]byte(token))", source)
self.assertNotIn('json:"token"', source)
for principal in example["principals"]:
self.assertRegex(principal["token_sha256"], r"^[0-9a-f]{64}$")
self.assertNotIn("token", principal.keys() - {"token_sha256"})
def test_postgres_receipts_are_hashed_durable_and_at_least_24_hours(self) -> None:
migration = normalized(text("deploy/postgres/008_control_api.sql"))
implementation = text("Sense/internal/store/control_postgres.go")
self.assertIn("create table if not exists sense.control_idempotency_receipts", migration)
self.assertIn("scope_hash bytea primary key", migration)
self.assertIn("expires_at >= created_at + interval '24 hours'", migration)
self.assertNotIn("idempotency_key", migration)
self.assertIn("pg_advisory_xact_lock", implementation)
self.assertIn("ConstantTimeCompare", implementation)
def test_concurrency_batch_and_limits_are_not_hardcoded_to_16(self) -> None:
migration = normalized(text("deploy/postgres/008_control_api.sql"))
server = text("Sense/internal/controlapi/server.go")
store = text("Sense/internal/store/control_postgres.go")
self.assertIn("resource_version bigint not null default 1", migration)
self.assertIn("len(body.Items) > 128", server)
self.assertIn("limit > 100", server)
self.assertIn("SAVEPOINT ", store)
self.assertNotIn("len(body.Items) > 16", server)
def test_audit_v1_is_unchanged_and_v2_is_a_strict_superset(self) -> None:
first = json.loads(text("docs/contracts/sense-device-audit-v1.schema.json"))
second = json.loads(text("docs/contracts/sense-device-audit-v2.schema.json"))
v1_events = set(first["properties"]["event_type"]["enum"])
v2_events = set(second["properties"]["event_type"]["enum"])
self.assertEqual(
{"device.created", "device.desired_state.accepted"}, v1_events
)
self.assertEqual(v1_events | {"device.configuration.accepted"}, v2_events)
self.assertIn("configurationData", second["$defs"])
def test_disabled_reconciliation_deletes_only_exact_path(self) -> None:
reconciler = text("Sense/internal/reconcile/reconciler.go")
media = text("Sense/internal/mtx/client.go")
self.assertIn("candidate.Device.DesiredState == device.DesiredDisabled", reconciler)
self.assertIn("r.media.DeletePath(ctx, candidate.Device.PathName)", reconciler)
self.assertIn("Deletion is an idempotent convergence operation", media)
self.assertNotIn("ListPaths", reconciler)
if __name__ == "__main__":
unittest.main()