Edge vs Core: Reference Architectures for Real-Time AI

What this article delivers

• Three reference architectures (Retail AR, Industrial Vision, Fleet Telematics) with ASCII diagrams
• Latency budgets, SLOs, and CAM pillar mapping (I-PWR, I-COOL, I-NWK, I-DATA, I-CTRL)
• Bill of materials, deployment steps, and failure-mode notes
• Copy-paste snippets (Kubernetes/Helm, policy) you can adapt today

1) Shared assumptions and quick math

Latency budget model

End-to-end latency (RTT) ≈ Access(last-mile) + Metro + Backbone + DC-fabric + Inference + App logic

Speed of light in fiber ≈ 5 µs/km one way.
Every router/LB hop adds ~0.1–0.5 ms; TLS adds 1–2 RTT for handshake unless resumed.
P99 matters. Budget to P99, not P50.

SLO taxonomy (use or adapt)

• Availability — % of successful requests within SLO latency window
• Latency — p50/p95/p99 target for “time to first token” (LLM) or “decision time” (vision)
• Freshness — Max model staleness (hours) allowed before redeploy
• Durability — RPO/RTO for authoritative data & artifacts

CAM in one minute

• A-Level: business tolerance for downtime/RPO (A0–A5).
• I-Score: mean of 5 pillars (0–5).
• CAM Tier: intersection of A-Level and I-Score (0–5).

Architectures below show target pillar scores and the resulting CAM Tier you can certify.

2) Reference Architecture A — Retail AR “Pick-and-See” (sub-20 ms UX)

Use case: A shopper lifts a product; a phone/tablet overlay shows price, reviews, and in-stock alternatives within 20 ms.

[Phone/Tablet] --Wi-Fi 6E [In-Store Edge Pod] ==(Anycast)==> [Metro Edge PoP A/B]
       |                           |                               |
    Camera + AR SDK             Triton/vLLM                   Model Registry (quorum)
       |                           |                               |
       +-- tiny preproc         + Feature store read         + Object storage (core)

Component                     Budget
Wi-Fi + local switch          2 ms
In-store edge pod (LLM/vision)8–10 ms
Optional metro round-trip     8–12 ms
Total                         18–24 ms

Failure modes and behavior

• Store power loss: Fail to metro PoP; degrade to “lite overlay” without personalization.
• WAN flap: Keep serving from store; queue writes, eventually sync features.
• Model rollout gone bad: Blue-green with store→metro canary; auto-rollback via GitOps.

Deployment quick start (Helm)

# values-retail.yaml
ingress:
  anycast: true
  protocols: [h3, h2]
triton:
  image: nvcr.io/nvidia/tritonserver:24.05
  args: ["--exit-on-error=false","--pinned-memory-pool-byte-size=268435456"]
  resources:
    limits: { nvidia.com/gpu: 1 }
featureStore:
  endpoint: http://feature-cache.metro.local:8080
mesh:
  spiffe: true
  mTLS: required

helm repo add retail-ai https://charts.retail.example
helm install store-edge retail-ai/retail-stack -f values-retail.yaml

3) Reference Architecture B — Industrial Vision “Stop the Arm” (≤ 20 ms deterministic)

Use case: High-speed line rejects defective parts; robotic arm must halt before contact.

[GigE Camera] -> [PoE Switch] -> [Edge GPU Node(s)] -> [PLC/SCADA]
                                         |               ^
                                         | mTLS+PTP      | EtherCAT/Profinet
                                         v               |
                                   [Local Feature DB]    |
                             (bounded write-behind to core)

Failure modes and behavior

• GPU node fails: Hot spare takes camera stream within 1 s via VIP; PLC alarm but no stop if spare ok.
• WAN outage: No impact on line; metrics buffered; alerts via LTE.
• Cooling loop blockage: Thermal kill-switch halts inference; PLC puts line into safe state; maintenance SLA < 5 min.

Policy gate (OPA Rego) to block unsigned models

package admission

default allow = false

allow {
  input.request.kind.kind == "ModelDeployment"
  sig := input.request.object.spec.signature
  verify(sig, input.request.object.spec.digest)
  input.request.object.spec.tags[_] == "safety-approved"
}

verify(sig, digest) {
  # abstracted; call out to cosign/rekor verifier
}

4) Reference Architecture C — Fleet Telematics “Sense → Decide → Sync” (≤ 70 ms assist; sub-second aggregation)

Use case: 100k vehicles run local safety inference; roadway MEC clusters aggregate events and serve personalized models back to the fleet.

[Vehicle Edge (ARM or x86)] --multi-IMSI/5G [MEC PoP A/B] ==Anycast==> [Regional Agg]
       |                                 |                        |
  Camera/LiDAR + tiny model             vLLM/Triton + KV cache   Model registry (quorum)
       |                                 |                        |
       +-- local decision (10–30 ms)  Driver UI               Cloud core analytics

Failure modes and behavior

• Carrier down: Multi-IMSI modems prefer alt carrier; MEC Anycast shifts; degraded but safe.
• MEC loss: Vehicles operate locally; miss regional enrichments until next PoP reached.
• Model rollback: Signed model fails post-deploy checks; vehicle uses previous local model.

MEC bootstrap (Kubernetes)

# Control plane across two MEC sites
kubeadm init --control-plane-endpoint mec.anycast.local:6443 --upload-certs
kubeadm join mec-a ... --control-plane
kubeadm join mec-b ... --control-plane

# SPIRE/SPIRE agents
helm install spire spiffe/spire-server -n spire
helm install spire-agent spiffe/spire-agent -n spire --set nodeAttestor=tpm

5) Decision matrix: where should your model run?

Pick the lowest-cost tier that meets your SLO. Prove it with CAM; invest only in the pillars that cap your Tier.

6) Implementation checklist (copy/paste into your runbook)

• Define SLOs (p95 latency, availability, freshness, durability) per user journey.
• Choose A-Level from the Availability Standard (A2 for “important,” A3/A4 for revenue-critical).
• Place compute: device → store → metro → core, stopping at the first tier that meets SLO.
• Score pillars with CAM rubric; list the cheapest improvements to hit target Tier.
• Ship identity: SPIFFE/SPIRE + mTLS; sign every artifact (images, models, policies).
• Test RTO/RPO: cut power, flap WAN, corrupt a replica; verify SLOs don’t break.
• Automate rollouts: GitOps, progressive delivery, canaries; fail closed on unsigned deploys.
• Attest continuously: export pillar KPIs to Grafana; publish green CAM badge.

7) Final take

Real-time AI isn’t a monolith. Some answers must come from the phone, some from the back room, some from the metro, and a few from the core. The right architecture is the one that meets your SLO at the lowest total cost, and the right proof is a CAM Tier you can show to customers, auditors, and your CFO.