Facility Monitoring & Management Network (FMMN) – Reference Architecture (v1.0)

2. Terms & roles

3. Design objectives (normative)

4. Physical topology & placement

4.1 Rooms & pathways

• Provide two CNRs: CNR-A and CNR-B, ideally on opposite sides of the facility; if multi-story, place on different floors and corners.
• MMR diversity: either (a) one MMR inside each CNR, or (b) a central MMR with two physically diverse conduit banks to CNR-A/CNR-B.
• Entrances: At least two provider entrances from diverse streets/duct banks. Provider A terminates in/near CNR-A; Provider B in/near CNR-B.

4.2 Powering

• All network devices MUST have dual PSUs on separate PDUs fed from different UPS/generator sources where available.
• OOB equipment SHOULD be on an independent UPS string or DC plant to allow management during partial facility outages.

4.3 Media & cabling

• Inter-CNR backbone SHOULD be dual diverse (e.g., two tray paths) and dual-fiber bundles (OM4/OS2).
• Labeling MUST be consistent: PATH(A|B)-ROOM-RACK-U-PORT. Keep path and room in labels to verify diversity at a glance.

5. External access layer (L1/L2/L3)

5.1 External provider demarcation

Each ENP terminates on an ENP handoff (e.g., optical NID or media converter) in its closest CNR. From each handoff, connect to a dedicated ENP switch (L2) in that same CNR:

• ENP-A-SW in CNR-A
• ENP-B-SW in CNR-B

Rationale: isolates provider optics and lets you fan out to multiple routers/firewalls without re-terminating the provider.

5.2 Edge routing/firewall layer

• Deploy two edge router/firewall appliances (RFW-A in CNR-A, RFW-B in CNR-B).
• Each ENP switch MUST uplink to both RFW-A and RFW-B (diverse optics/paths).
• RFW-A and RFW-B MUST run an HA pair using FHRP and a state-sync link (encrypted sync; dedicated cable or secure inter-CNR link).
• External routing MAY be static + SLA-monitor failover (Small) or eBGP with each ENP (Medium/Large).
• NAT policy SHOULD be centralized on the RFWs. For dual-stack, prefer native IPv6 with stateful firewalling; avoid NAT66.

5.3 Core switching layer

• Deploy two core switches (CORE-A in CNR-A, CORE-B in CNR-B).
• Each RFW uplinks to both core switches (diverse optics/paths).
• Enable MLAG (or equivalent) between CORE-A and CORE-B for active/active L2. If MLAG is not supported, use FHRP gateways per VLAN and no L2 loops (single active path using LACP).

5.4 Small-site minimal stack

ENP-A -> ENP-A-SW -> RFW-A/B -> CORE-A/B
ENP-B -> ENP-B-SW -> RFW-A/B -> CORE-A/B

Where CORE-A/B can be PoE capable to power access switches and endpoints.

5.5 Device uplinks & single-homed endpoints

• All distributions/access switches SHOULD dual-home via LACP to CORE-A and CORE-B (MLAG).
• Endpoints that cannot dual-home (e.g., PoE cameras, sensors) MUST be evenly distributed across access switches and diverse paths, with per-switch failure impact documented.

6. Layer-2 design

6.1 Spanning & aggregation

• Prefer loop-free L2 using LACP and MLAG at the core.
• If STP is required, use a single STP domain with CORE-A as root primary and CORE-B as root secondary; block only well-understood redundant links.
• Disable BPDU Guard only on known trunk ports; enable BPDU Guard on all access ports.

6.2 VLAN catalog (reference set)

Rules: OOB-MGMT (VLAN 10) MUST be physically/logically isolated from in-band (VLAN 20). OT-CONTROL (VLAN 70) MUST be L3-firewalled; no L2 adjacency with IT networks. Guest (VLAN 100) MUST have internet-only egress; no east-west or north-south into facility networks.

7. Layer-3 addressing & summarization

7.1 Site supernetting (IPv4)

• Small (S): /22 (1024 addresses → ~6–8 VLANs @ /26–/27)
• Medium (M): /21 (2048 addresses → ~10–16 VLANs @ /24–/26)
• Large (L): /20 (4096 addresses → ~16–32 VLANs @ /24)

Allocate from a centrally managed RFC1918 block. Two common schemes:

• Scheme A (10/8 with site index): 10.<SITE_ID>.0.0/20 (L) or /21 (M) or /22 (S). VLAN → third octet, Subnet → fourth. Example (Site 37, Medium /21): VLAN 10 OOB-MGMT: 10.37.10.0/26; VLAN 40 CCTV: 10.37.40.0/24; VLAN 70 OT-CONTROL: 10.37.70.0/25.
• Scheme B (172.16/12 with region + site): 172.<REGION_ID>.<SITE_ID>.0/20; REGION_ID 16–31 for Americas, 32–47 EMEA, 48–63 APAC (example).

Rules: Do not assign more than a /20 to a single site unless objectively required. Every VLAN MUST have a documented mask sized to endpoints (+30% growth). Summarize per-site supernets in WAN advertisements (eBGP or static).

7.2 IPv6 plan

Use /48 per site for internal addressing; assign one /64 per VLAN. Canonical scheme: fdAA:REG:SITE:VLAN::/64. Example: Region 1, Site 37, VLAN 40 CCTV → fdaa:1:37:40::/64.

7.3 VRFs & inter-site routing

Create VRFs at minimum for: OOB, MGMT/IT, OT, GUEST, and TENANT (if applicable). Inter-site reachability via route-based VPNs or WAN with eBGP. Sites SHOULD advertise only their per-site summary and any tenant summaries. Avoid RFC1918 overlap via a central registry.

8. High availability behaviors

• Edge RFW pair: stateful failover; maintain symmetric routing where required; health-check ENP paths.
• Core MLAG peer-link MUST be physically diverse; configure orphan port handling for single-homed endpoints.
• Access switches: dual-home with LACP where possible; otherwise distribute endpoints; conduct quarterly failover tests.

9. Security controls (baseline)

9.1 Edge policy (RFWs)

• Default-deny inbound from internet; allow only VPN and required monitoring.
• Outbound restricted to necessary destinations; reputation filtering MAY apply to guest.

9.2 East-west policy

Inter-VRF via L3 firewalls with least-privilege ACLs; discovery protocols blocked across VLANs.

9.3 Administrative access

SSH/API via VPN with MFA; no public management planes; Git-backed config approvals recommended.

9.4 Remote access VPN

Route-based IPsec IKEv2 with cert auth; dual head-ends; split-tunnel by role; full-tunnel for break-glass only.

10. Time, logging & telemetry

• NTP: two servers on different CNRs; PTP only if OT requires.
• Syslog: dual collectors (VLAN 110), TLS transport; 90 days online + 1 year archive.
• Telemetry: SNMPv3/streaming; export NetFlow/IPFIX; critical link-down alerts MUST page.

11. Bandwidth & sizing guidance

Plan for CCTV, OT bursts, logging, and headroom; see small/medium/large capacity examples.

12. Baseline firewall policy (template)

Ingress: allow VPN/SOC; deny all else. Egress: restrict per-VRF. Inter-VRF: least-privilege; jump-host for OT.

13. Example addressing (IPv4/IPv6) by size

13.0 Summary (S / M / L)

13.1 Small site (S) VLAN examples

13.2 Medium (M) and 13.3 Large (L)

Medium sites use 10.37.0.0/21 with /24s for major VLANs (CCTV, WLAN) and /25–/26 for others. Large sites use 10.37.0.0/20 with dedicated /23 or /22 for CCTV and multiple tenant /24s; IPv6 remains a /48 with one /64 per VLAN.

14. VPN patterns

Site-to-site: dual tunnels from each head-end, prefer eBGP across the tunnels with per-site summary routes only. Remote admin: IKEv2 with MFA; split-tunnel to NET-MGMT; jump-only to OT.

15. Operations & lifecycle

Backups daily; configs in Git with PRs; monitoring SLAs for ENP/OOB; quarterly drills pulling ENP and core links.

16. Security hardening checklist (abridged)

• Unique local admin creds vaulted; TACACS+/RADIUS for AAA.
• SSHv2 only; rotate host keys; disable unused services.
• Storm-control; DHCP snooping; ARP inspection; port-security for CCTV/OT.

17. Deliverables

• Rack elevations for CNR-A/B; cable schedules showing A/B paths.
• VLAN/VRF matrix; addressing templates for S/M/L; firewall CSV/YAML.
• Monitoring profile: SNMPv3 users, syslog targets, IPFIX exporters.

18. Reference ASCII diagram (S/M baseline)

          ENP A (diverse entry)                    ENP B (diverse entry)
                |                                         |
         [ENP-A Handoff]                          [ENP-B Handoff]
                |                                         |
            [ENP-A-SW]-----+                       +-----[ENP-B-SW]
                |          |                       |          |
                |        (A-path fiber)         (B-path fiber) |
                |          |                       |          |
             [RFW-A]======[RFW-B]  <== HA pair (state sync; encrypted; diverse)
                |  \      /   |                 |   \    /    |
              (A)   \  /    (B)                 |    \/      |
                |    \/      |              [CORE-A]=====[CORE-B]  <== MLAG/ICC (diverse paths)
                ||            ||         A-Path ||            || B-Path
           +----++            ++----+
           |  Access / Distribution (dual-homed LACP where possible)
           |     \               /
          OOB --- Console/Modem/LTE (separate UPS) -----------+