← All posts

Multicast failure-mode lab series · 2 of 3 · Companion post: Multicast Scoping — Solution Analysis & EVE-NG Lab Guide · Based on: NetworkLessons Multicast lessons “RPF (Reverse Path Forwarding)” & “Source Specific Multicast” (member content) · Target platform: EVE-NG (Cisco IOL), Cisco IOS commands · Prepared: 2026-10-03

Contents

1. The Exercise
2. Solution Analysis — RPF on the data plane and the control plane
3. EVE-NG Lab Design — topology, images, cabling, addressing
4. Complete Node Configurations
5. Lab Procedure (three acts) and Verification
6. Troubleshooting Notes
7. One-Page Summary

Knowledge points covered

• RPF check: multicast is accepted only on the interface that leads back to the source (RFC 4601 §4.5.2 concept)
• RPF failures on the control plane (PIM joins follow the unicast RIB toward the source — wrong unicast = joins go the wrong way, state never gets built) and on the data plane (packets arriving on a non-RPF interface are dropped — debug ip mpacket shows not RPF interface)
• SSM (RFC 4607): ip pim ssm default, IGMPv3 INCLUDE joins, (S,G)-only trees, no RP — the model exchanges use for market data (the market-data practice notes)
• The A/B feed model: same channel from two sources; receivers arbitrate duplicates at the application layer (sequence numbers); one dead feed = silent quality degradation
• The fix and why production uses it: ip mroute pins the multicast RPF path independently of unicast convergence
• Why the ping reply path is a separate unicast problem — a diagnostic lesson in itself

---

1 The Exercise

A firm receives one market-data channel as two identical feeds — feed A and feed B — published from two different source networks, entering your core through two different uplinks. After a routing change, feed B stops arriving. Unicast to everything is fine. The receiver never sees B again; nobody touched B’s path “on purpose”.

Reproduce it: make the core’s unicast route toward source B point out the A uplink (in real life this happens whenever an IGP metric change, a failed link, or a summarization boundary makes the two paths asymmetric). Watch feed B die, diagnose it with show ip rpf and show ip mroute count, then fix it with a static multicast route — and understand why production trading networks configure those on purpose.

2 Solution Analysis — RPF on the data plane and the control plane

PIM forwards multicast by asking one question for every packet — and a second question for every join:

• Data plane: accept the packet only if it arrives on the interface the router would use to reach the source. Otherwise drop it (debug ip mpacket → not RPF interface — the RPF lesson shows this signature).
• Control plane (this lab’s main event): to build (S,G) state, the last-hop router sends PIM joins toward the source, following the unicast RIB. If unicast says “source B is via the A uplink”, the join travels to the wrong edge router — which has no route to B, drops the join, and feed B’s path never gets built. B is silent even though its data is flowing happily right up to your doorstep.

The RPF lesson also shows the control-plane RPF failure against an RP ((*,G) with Incoming interface: Null, RPF nbr 0.0.0.0) — same disease, different victim. This lab uses SSM so there is no RP at all and the failure is purely per-source.

Why SSM for this lab: with SSM the receiver subscribes (S,G) for each source separately (ip igmp join-group <G> source <S>), which is exactly the A/B feed subscription model exchanges publish (SSM lesson: no shared trees, no RP, no Auto-RP/BSR). Flags to know: (S,G) shows sT on transit routers.

The fix — a static multicast route in the multicast RIB:

ip mroute 10.0.200.0 255.255.255.0 10.0.13.3

Static mroutes are not used for unicast forwarding — they exist only for multicast RPF lookups (and they take precedence over unicast routes for RPF). That is the industry pattern: pin the feed’s RPF path so unicast reconvergence can never break the market-data path (the multicast failure-modes notes).

Takeaway: unicast asymmetry that is harmless for unicast traffic is fatal for multicast, because multicast routing state is derived from unicast reachability — on both planes.

3 EVE-NG Lab Design

3.1 Design rationale

• All five nodes are IOL L3 routers. Sources R4/R5 are host-style (plain IP, ping the group) — like the send-only feed publishers in the market-data practice notes, they never join anything.
• The receiver is R1’s Loopback0 with two IGMPv3 source-specific joins (feed A and feed B of the same channel). ip igmp join-group makes R1 both the group member (answers pings) and the last-hop router (sends (S,G) joins) — the same trick as the scoping lab.
• The fault is injected with a static route on R1 pointing 10.0.200.0/24 at the A-side edge (10.0.12.2). Deterministic, visible, and trivially reversible — no IGP metric games needed.
• Group 232.1.1.1 sits in the default SSM range (232.0.0.0/8), so ip pim ssm default enforces (S,G)-only semantics — a (*,G) join in this range is refused, which is itself a good thing to see once.

3.2 Images

Role	EVE-NG image
R1 (core / receiver), R2 (edge-A), R3 (edge-B), R4 (source A), R5 (source B)	Cisco IOL (L3)

3.3 Cabling

From	Port	To	Port	Subnet
R1	E0/0	R2	E0/0	10.0.12.0/24
R1	E0/1	R3	E0/0	10.0.13.0/24
R2	E0/1	R4	E0/0	10.0.100.0/24
R3	E0/1	R5	E0/0	10.0.200.0/24

3.4 Addressing

Node	Interface	IP	Role
R1	Lo0	1.1.1.1/32	Receiver — joins (10.0.100.100, 232.1.1.1) and (10.0.200.200, 232.1.1.1)
R1	E0/0 / E0/1	10.0.12.1 / 10.0.13.1	Core uplinks (A / B)
R2	E0/0 / E0/1	10.0.12.2 / 10.0.100.1	Edge toward feed A
R3	E0/0 / E0/1	10.0.13.3 / 10.0.200.1	Edge toward feed B
R4	E0/0	10.0.100.100/24	Feed A source (host-style)
R5	E0/0	10.0.200.200/24	Feed B source (host-style)

The injected fault (configure only in Act 1): ip route 10.0.200.0 255.255.255.0 10.0.12.2 on R1 — unicast toward source B deliberately leaves via the A uplink.

4 Complete Node Configurations

! === R1 (core + receiver) ===
hostname R1
!
ip multicast-routing
ip pim ssm default
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
 ip pim sparse-mode
 ip igmp version 3
 ip igmp join-group 232.1.1.1 source 10.0.100.100
 ip igmp join-group 232.1.1.1 source 10.0.200.200
!
interface Ethernet0/0
 ip address 10.0.12.1 255.255.255.0
 ip pim sparse-mode
!
interface Ethernet0/1
 ip address 10.0.13.1 255.255.255.0
 ip pim sparse-mode
!
! Act 1 fault — unicast toward feed B points at the A uplink:
ip route 10.0.100.0 255.255.255.0 10.0.12.2
ip route 10.0.200.0 255.255.255.0 10.0.12.2
!
end

! === R2 (edge, feed A) ===
hostname R2
!
ip multicast-routing
ip pim ssm default
!
interface Ethernet0/0
 ip address 10.0.12.2 255.255.255.0
 ip pim sparse-mode
!
interface Ethernet0/1
 ip address 10.0.100.1 255.255.255.0
 ip pim sparse-mode
!
end

! === R3 (edge, feed B) ===
hostname R3
!
ip multicast-routing
ip pim ssm default
!
interface Ethernet0/0
 ip address 10.0.13.3 255.255.255.0
 ip pim sparse-mode
!
interface Ethernet0/1
 ip address 10.0.200.1 255.255.255.0
 ip pim sparse-mode
!
end

! === R4 (feed A source, host-style) ===
hostname R4
!
interface Ethernet0/0
 ip address 10.0.100.100 255.255.255.0
 no shutdown
!
end

! === R5 (feed B source, host-style) ===
hostname R5
!
interface Ethernet0/0
 ip address 10.0.200.200 255.255.255.0
 no shutdown
!
end

Act 2 adds exactly one line to R1: ip mroute 10.0.200.0 255.255.255.0 10.0.13.3.

5 Lab Procedure and Verification

Act 1 — Reproduce: feed B is dead, feed A is fine

1. Start all nodes. From R4: ping 232.1.1.1 repeat 9999 (feed A). From R5: ping 232.1.1.1 repeat 9999 (feed B).
2. On R1, diagnose B:

Check on R1	Expected (broken)
show ip rpf 10.0.100.100	E0/0 … via 10.0.12.2 — correct
show ip rpf 10.0.200.200	E0/0 … via 10.0.12.2 — B’s RPF interface is the A uplink. This is the whole bug in one line
show ip mroute 232.1.1.1	(10.0.100.100, 232.1.1.1) … flags: sT exists; no (10.0.200.200, …) entry
show ip mroute 232.1.1.1 count	Only A’s (S,G) counters increment; B never appears
Ping replies	A’s pings answered by 1.1.1.1; B’s pings get no replies

3. Watch the join go the wrong way — on R2: debug ip pim shows the (S,G) join for 10.0.200.200 arriving on E0/0 and being dropped (R2 has no route toward 10.0.200.0/24); on R3, show ip mroute 232.1.1.1 is empty. Control-plane RPF failure: state is built toward the wrong uplink.

Act 2 — Fix: pin B’s RPF path with a static mroute

1. On R1:

R1(config)# ip mroute 10.0.200.0 255.255.255.0 10.0.13.3

2. Re-check within seconds (PIM re-joins periodically):

Check on R1	Expected (fixed)
show ip rpf 10.0.200.200	RPF succeeded … E0/1, via 10.0.13.3 — look for the route origin marked Mroute
show ip mroute 232.1.1.1	(10.0.200.200, 232.1.1.1) … flags: sT now exists, Incoming interface: Ethernet0/1, RPF nbr 10.0.13.3, Mroute
R3 show ip mroute 232.1.1.1	B’s (S,G) built through R3: Incoming interface: Ethernet0/1
show ip mroute count	Both (S,G) counters increment — both feeds alive
Ping replies	A answered; B still shows no replies — see below, this is expected

3. Why B’s pings still get no replies: the reply is unicast from R1 to 10.0.200.200 — and R1’s unicast route to 10.0.200.0/24 still points at the A uplink, where R2 has no route back. Diagnose it: show ip route 10.0.200.0. Fix the return path too (Act 3).

Act 3 — Industry framing: unicast fix vs multicast pin

1. Repair unicast on R1: no ip route 10.0.200.0 255.255.255.0 10.0.12.2, then ip route 10.0.200.0 255.255.255.0 10.0.13.3. B’s pings are now answered by 1.1.1.1.
2. Discuss with the config in front of you: once unicast is symmetric, the ip mroute is no longer necessary — but production keeps it, because the static mroute pins the multicast RPF path. When the IGP reconverges (link flap, metric change, summarization), unicast reroutes freely; the feed’s RPF path does not move. That is precisely the A/B feed asymmetry protection in the multicast failure-modes notes.
3. Optional bridge to the receiver side: both (S,G) counters on R1 incrementing at the same time is what a market-data arbitrator consumes — two duplicate streams, deduplicated by sequence number at the application layer, with gap-recovery (TCP replay) hiding the loss windows (the market-data practice notes).

6 Troubleshooting Notes

Symptom	Check	Likely cause
One feed silent, unicast fine	show ip rpf <source> on the last-hop router	RPF points at the wrong interface — unicast asymmetry (this lab)
(*,G) has Incoming interface: Null, RPF nbr 0.0.0.0	show ip rpf <RP-address>	Control-plane RPF failure toward the RP (ASM variant — same fix with ip mroute <RP> …)
(S,G) exists on the edge but not the core	debug ip pim on the transit router	Join dropped in transit — transit has no route toward the source
Packets flow but counters stay at 0	debug ip mpacket on the last-hop router	Data-plane RPF drop: not RPF interface (the lesson’s dual-link variant)
Fix applied but state doesn’t appear	show ip pim neighbor on every hop	PIM not enabled on some interface of the path — joins need PIM adjacencies end to end
ip pim ssm default set, join refuses	Try a (*,G) join in 232/8	Expected: SSM range rejects (*,G); joins must be source-specific

Key mechanism: multicast state is derived from unicast reachability. show ip rpf <source> is always the first command after “feed is dead” — before touching the switches, before blaming the publisher.

7 One-Page Summary

                 unicast RIB: "10.0.200.0/24 via 10.0.12.2"  (WRONG - via A uplink)
                        │
   R1 wants feed B ──► (S,G) JOIN toward 10.0.200.200 ──► travels via E0/0 to R2 ──► dropped
                        │                                                    (R2: no route to B)
   R5 sends feed B ──► R3 has no (S,G) state ──► dropped at R3
                        │
   Result: feed B dead end-to-end, no error anywhere, unicast 100% fine

   FIX:  ip mroute 10.0.200.0 255.255.255.0 10.0.13.3     (multicast RIB overrides RPF)
         → join goes via E0/1 → R3 builds state → B flows; pin survives unicast reconvergence

• Interview one-liner: “RPF failure doesn’t always drop packets — with PIM joins following the unicast RIB, asymmetric unicast quietly builds the tree toward the wrong uplink and the feed just never arrives. show ip rpf <source> is the first diagnostic, ip mroute is the pin that keeps feeds stable across unicast convergence.”
• Command set to memorize: show ip rpf <S> · show ip mroute <G> (+ count) · debug ip pim · debug ip mpacket (not RPF interface) · ip mroute <prefix> <mask> <interface|nh>.
• Design rules: sources subscribe per-(S,G) (SSM/IGMPv3); pin feed RPF paths with static mroutes; remember replies travel unicast — fix both planes.
• Cross-references: the multicast failure-modes notes trap #2 · the market-data practice notes (A/B arbitration, gap recovery) · RFC 4607 (SSM) · RFC 4601 (PIM-SM, RPF).