The call came in the way these always do: “the whole network is slow.”
Not down. Slow. Which is worse, because slow means it’s still trying.
I pulled up the switch stats from the site and the picture was ugly. Broadcast counters spinning like a gas pump. Port LEDs across the access switches strobing in unison — that synchronized, hypnotic blink that should make the hair on your neck stand up. CPU on the core switch pinned. Every wired device crawling, half the VLANs timing out.
That’s not congestion. That’s a storm.
The investigation
A broadcast storm has a signature. One frame gets flooded out every port, comes back in another port, gets flooded again, forever, at line rate. The switches aren’t broken — they’re doing exactly what they’re told, infinitely.
So the question is never “what’s slow.” It’s “where’s the loop.”
I started where loops live: the edge. I pulled the MAC address table and watched a handful of addresses flapping between ports — the same MAC showing up on port 14, then port 9, then port 14, dozens of times a second. A stable MAC doesn’t teleport. A looped one does.
show mac address-table | the same MAC, two ports, flapping
─────────────────────────────────────────────────────────
02:00:00:aa:bb:cc Gi0/14 ◄─┐
02:00:00:aa:bb:cc Gi0/9 ◄─┤ same address,
02:00:00:aa:bb:cc Gi0/14 ◄─┤ two ports,
02:00:00:aa:bb:cc Gi0/9 ◄─┘ ~40x/sec ✗
I traced both ports. Port 14 and port 9 went to two units of a consumer multi-room speaker system — the kind that builds its own little wireless mesh between speakers so they stay in sync.
And somebody had wired both of them into the LAN.
The “aha”
There it was. Each speaker had an Ethernet drop. Each speaker was also bridging to its sibling over the proprietary wireless mesh. So a frame could leave the switch on port 14, ride into speaker A, hop the wireless mesh to speaker B, and come right back into the switch on port 9.
Copper out, radio across, copper back in. A loop with one leg made of Wi-Fi.
The reason it had run fine for months and then exploded? Doesn’t matter much. Someone re-cabled a closet, both speakers ended up plugged in, the mesh did its job, and nothing on those access ports was watching for a loop.
The fix
The speaker was the trigger. The real bug was a switch fabric with no loop protection. So we fixed both.
First, the immediate bleeding — pull one of the two speaker uplinks. One leg of copper gone, loop broken, network breathes.
Then, the actual fix. Turn on spanning tree, and pick the root on purpose instead of letting MAC addresses elect a random one:
# Core switch — make it the root, deliberately. Lowest priority wins.
spanning-tree mode rstp
spanning-tree vlan 1-4094 priority 4096
# Distribution — second in line
spanning-tree vlan 1-4094 priority 8192
Then lock down the edge so an end-device port can never become a transit path:
# Access port to the speaker / any end device
interface GigabitEthernet0/14
spanning-tree portfast
spanning-tree bpduguard enable # any BPDU here = err-disable the port
spanning-tree guard root # this port may never be a path to root
storm-control broadcast level 1.00 # cap broadcast at 1% of line rate
storm-control action shutdown
Mesh-style managed gear is the same idea in different clothes — flip on STP, set tiered priorities so the core is root, and enable Loop Protection plus BPDU Guard on access ports.
edge port discipline
─────────────────────────────────
PortFast ✓ fast end-device link
BPDU Guard ✓ sees a switch → kills port
Loop Guard ✓ detects the loop
Storm-control ✓ broadcast ceiling
─────────────────────────────────
result: ▓▓▓▓▓ blast radius = one port
We re-cabled the speakers to a single drop each, verified one stayed on copper and the rest synced over mesh, and watched the broadcast counter flatline.
Why it happened
Consumer mesh gear is a loop machine wearing a friendly face. It will happily bridge two of its own units, and it has no idea it’s also sitting on your switched LAN. Plug two of them into copper and you’ve handed your network a second path it never agreed to.
The switch didn’t fail. It had no spanning tree configured, so it had no mechanism to notice the loop and break it. The default config trusted every port to behave. Edge ports never behave.
Takeaways
- Always run STP/RSTP. A modern loop-free topology still needs spanning tree on standby for the day someone hands you a loop.
- Set the root bridge on purpose. Lowest priority value wins; don’t let a random MAC address decide where your traffic converges.
- Guard the edge. BPDU Guard, Loop Protection, and storm-control on access ports turn a network-wide meltdown into one dead port.
- Treat consumer mesh gear as hostile to your switch fabric. Never wire two mesh-bridging devices into the same LAN.
- A flapping MAC is a loop until proven otherwise. The mac-address-table is your fastest path from “it’s slow” to “it’s port 9 and 14.”