Server analytics and on-site Jetson appliances for camera alerting

The problem

Operators need immediate notice when a person or vehicle crosses a zone or line, across many cameras, around the clock. Some sites forbid sending video off site; others run analytics on a central server.

Many RTSP streams per site; one slow or dead stream must not block the rest.

A stale detection has no operational value; freshness beats processing every frame.

Several customers on one platform; each tenant sees only its own cameras and events.

Edge appliances must update on restricted networks with a path to roll back a bad package.

System architecture

Option A — Central server analytics

Use this when video may travel to a central server. The cameras stream to the server and detection runs there. This is one complete deployment — it is not a stage before the appliance in Option B.

flowchart LR
  subgraph cameras ["At the site"]
    cameras_0(["Cameras at the site"])
  end
  subgraph server ["Server analytics"]
    server_0["RTSP ingest"]
    server_1[["Bounded queue · drop-oldest"]]
    server_2["YOLO detect and track"]
    server_3["Zone and line rules"]
  end
  subgraph messaging ["Event paths"]
    messaging_0[["Alert queue"]]
    messaging_1[["Metric queue"]]
    messaging_2[["Archival queue"]]
    messaging_3["Redis alert dedup"]
  end
  subgraph control ["Control plane"]
    control_0["NestJS camera admin and APIs"]
    control_1["OIDC identity service"]
  end
  subgraph persist ["Stores"]
    persist_0[("PostgreSQL alerts")]
    persist_1[("TimescaleDB metrics")]
    persist_2[("Object storage clips")]
  end
  cameras_0 -->|"video streams"| server_0
  server_0 --> server_1
  server_1 --> server_2
  server_2 --> server_3
  server_3 --> messaging_0
  server_3 --> messaging_1
  server_3 --> messaging_2
  messaging_0 --> messaging_3
  messaging_3 --> persist_0
  messaging_1 --> persist_1
  messaging_2 --> persist_2
  control_0 -->|"config"| server_0
  control_1 --> control_0

Option B — On-site Jetson appliance (instead of A, not after it)

Use this instead when video must not leave the site. Detection runs on the appliance at the site; only alert metadata leaves. A site runs A or B — never both in sequence.

flowchart LR
  subgraph site ["At the site"]
    site_0(["Cameras at the site"])
    site_1(["Orin NX appliance"])
  end
  subgraph appliance ["On the appliance"]
    appliance_0["DeepStream and TensorRT engine"]
    appliance_1["Python appliance control software"]
    appliance_2["Signed USB or OTA update"]
  end
  subgraph leaving ["What leaves the site"]
    leaving_0[["Alert queue"]]
    leaving_1["Redis alert dedup"]
    leaving_2[("PostgreSQL alerts")]
  end
  site_0 -->|"local video only"| site_1
  site_1 --> appliance_0
  appliance_1 -->|"manages the appliance"| appliance_0
  appliance_2 -->|"verified update"| appliance_1
  appliance_0 -->|"alert metadata only — no video"| leaving_0
  leaving_0 --> leaving_1
  leaving_1 --> leaving_2

Interaction schematic

Schematic of the interaction described above. Generic interface, no real data, no measured figures. Not a product screenshot.

Key architecture decisions

Option A — Drop oldest when the bounded queue fills

Between decode and inference, drop the oldest frame when the queue is full, not the newest.

Queued frames push zone and line events later and later; a late alert is operationally useless.

Rejected Unbounded queue — rejected because alert latency grows with incident duration.

Trade-off Some frames are never scored; alerts that fire remain timely.

Option A — Three RabbitMQ paths with Redis dedup on alerts

Alert, metric, and archival workloads use separate queues; repeated alerts for the same event are suppressed for a short cooldown window.

A slow archival job must not sit in front of safety-style alerts, and high-volume metrics must not take capacity away from alerts.

Rejected Single queue with priority — rejected because one consumer can still block on a large archival message.

Trade-off More queues and consumers to operate. If de-duplication is unavailable, alerts are still delivered — a duplicate is preferred to a missed alert.

Option B — Lab-built TensorRT engine per device class

A model build is approved only after it has been checked, using footage from the site, on the same class of device that will run it in production.

TensorRT engines bind to GPU architecture and TensorRT version; a server GPU build does not run on Orin NX.

Rejected Copying an engine built on a lab server GPU — rejected after it failed to run correctly on the target device hardware.

Trade-off Worker, queue, and wait time before a new model version is viewable on site.

Option B — Signed updates, checked before installation

An update package is verified before it is installed; if the new version does not start healthily, the appliance returns to the previous working version. The same package works over USB and over the network.

Restricted sites still need updates without putting the version currently running at risk.

Trade-off Key management and rollback testing on every release train.

What went wrong

One failing camera used to stall processing for unrelated cameras.

Only the affected camera is restarted, with increasing delays between attempts; the other streams keep moving.

Heavy extra work placed inside the video pipeline stalled the stream itself.

The pipeline only observes and passes frames on; heavier work happens elsewhere, and frames stay on the GPU.

An engine built on the wrong GPU profile looked healthy in FPS but missed small objects.

A model is now built and checked on the same class of device it will run on, using footage from the site, before it is approved for release.

What I own, wrote, and delegate

Own

Control plane and analytics data plane split
How a new model is checked on footage from the site and approved before release
Shared NestJS APIs and OIDC identity reused across enterprise products
Event schema, drop-frame policy, update signing, tenant filter — not delegated

Wrote

Server-side inference path and camera or event APIs
C++ DeepStream inference loop and Python appliance control plane
RabbitMQ routing and Redis dedup on the alert path

Delegate

UI and mobile surfaces after contracts were set
Repetitive asset conversion once patterns were fixed

Scale

Technical lead for a team of up to 10 engineers; hands-on in inference and APIs
Control-plane core adopted by 3 additional enterprise products
Two deployment shapes: central server analytics and on-prem Orin NX appliances

Boundaries

CUDA in this path is the TensorRT and NVDEC runtime. There are no custom CUDA kernels.
The detector is a YOLO-family model via ONNX. Version and tracker follow the site.
Lab builds TensorRT engines on the target GPU class; engines are not copied from a datacenter GPU to Orin.

What we watch in operation

Alert latency after frame drop policyPer-stream frames dropped and ingest FPSQueue depth on alert versus archival consumers

Rules the design never breaks

One overloaded stream does not block the others.
Alerts stay within seconds under load, not minutes behind the scene.
No tenant reads another tenant's cameras or events.

Broader application

Large fixed-camera estates on imperfect networks.
Sites that require on-prem video with central administration.
Operations centres that need low-noise, high-confidence zone and line alerting.