<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Nvr on ErrorZap</title><link>https://errorzap.com/tags/nvr/</link><description>Recent content in Nvr on ErrorZap</description><image><title>ErrorZap</title><url>https://errorzap.com/og.png</url><link>https://errorzap.com/og.png</link></image><generator>Hugo</generator><language>en-US</language><copyright>ErrorZap</copyright><lastBuildDate>Fri, 13 Nov 2026 17:30:00 -0700</lastBuildDate><atom:link href="https://errorzap.com/tags/nvr/index.xml" rel="self" type="application/rss+xml"/><item><title>My UniFi NVR Had 8 CPU Cores. One of Them Was Doing All the Work.</title><link>https://errorzap.com/posts/unifi-nvr-one-cpu-core-doing-all-the-work/</link><pubDate>Fri, 13 Nov 2026 17:30:00 -0700</pubDate><guid>https://errorzap.com/posts/unifi-nvr-one-cpu-core-doing-all-the-work/</guid><description>Troubleshooting laggy, freezing UniFi Protect live view on a 10G G2-class NVR: the network card sends every packet to a single CPU core and Linux RPS is off by default. Turning on Receive Packet Steering cut dropped packets about 99.7% with no reboot.</description><content:encoded><![CDATA[<p><em>Eight cores in the box. Seven drinking coffee. One drowning. And me, losing my mind.</em></p>
<p>It started the way it always starts: &ldquo;The cameras are lagging again.&rdquo;</p>
<p>Again. <em>Again.</em> I have heard that sentence so many times on this job that I hear it in my sleep.</p>
<p><img alt="Comic: the host face-down on his desk at night, glasses crooked, as his phone lights up with a text reading THE CAMERAS ARE LAGGING AGAIN. A wall calendar is covered in red tally marks and his mug says AGAIN." loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/lagging-again.webp"></p>
<p>UniFi Protect live view was freezing, stuttering and loading slowly. On the cloud portal and by direct IP. On a brand-new PC. On a port right next to the NVR. The person watching the cameras had already ruled out everything you&rsquo;d normally blame, so it was my turn.</p>
<p>Spoiler: it wasn&rsquo;t the PC, the cable, the switch or the internet. It was one CPU core inside the NVR doing 100% of the network work while the other seven sat around like it was a Sunday.</p>
<p><img alt="Comic: a police lineup of a desktop PC, a network cable and a network switch, all looking offended, while the host in a trench coat glares at them with a magnifying glass. Behind him a small sweating box labeled NVR tiptoes away. Caption: WE BLAMED EVERYONE ELSE FIRST." loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/lineup.webp"></p>
<h2 id="the-setup">The setup</h2>
<ul>
<li>UniFi G2-class NVR (Qualcomm QCS8550 inside, UniFi OS 6.0.x)</li>
<li>Uplinked on the 10G SFP+ port with a DAC cable</li>
<li>About 50 cameras, roughly 200 Mbps coming in</li>
<li>Blue Iris pulling about 90 streams off the NVR around the clock, plus people watching live</li>
</ul>
<p>That&rsquo;s a busy NVR, but 200 Mbps on a 10G link is nothing. The link wasn&rsquo;t the problem.</p>
<h2 id="step-1-stop-guessing-start-logging">Step 1: Stop guessing, start logging</h2>
<p>Before touching anything I put monitoring on both ends:</p>
<ul>
<li><strong>On the NVR:</strong> a small script logging every 30 seconds. CPU per core, the network card&rsquo;s dropped-packet counter, traffic, temperatures, fans, and ping times to the viewer&rsquo;s PC.</li>
<li><strong>On the viewer&rsquo;s PC:</strong> a logger plus a rolling packet capture.</li>
<li><strong>On the viewer&rsquo;s desktop:</strong> a big &ldquo;Cameras Lagging, Click Here&rdquo; button. When it freezes, they click it, and it saves the last few minutes of capture with a timestamp.</li>
</ul>
<p>That button is my favorite trick for &ldquo;it happens sometimes&rdquo; problems. Nobody has to remember what time it was.</p>
<h2 id="step-2-the-data-pointed-straight-at-the-nvr">Step 2: The data pointed straight at the NVR</h2>
<p>The PC side was clean. Fast responses, no packet loss to the NVR.</p>
<p>The NVR side was not:</p>
<ul>
<li><strong>CPU4 was 90 to 98% busy</strong>, almost all of it handling network interrupts.</li>
<li>The other cores had room to spare.</li>
<li>The network card was <strong>dropping incoming packets every single sample</strong>. About 164 per second.</li>
</ul>
<p>Dropped camera packets mean missing video. Missing video means frozen and stuttering live view, for every viewer, and gaps in Blue Iris too.</p>
<p>So the box had eight cores, and one of them was doing the whole job while the rest stood around the water cooler. I have worked places like that.</p>
<p><img alt="Comic: inside a computer drawn as a break room, seven little robots lounge with coffee and cards. In the doorway a robot labeled CORE 4, gauge pinned at 95%, is buried in a flood of envelopes and yells LITTLE HELP?! One lounging robot answers NOT MY JOB." loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/break-room.webp"></p>
<div style="margin:1.5rem 0">
<svg viewBox="0 0 640 300" xmlns="http://www.w3.org/2000/svg" role="img" aria-label="Diagram: before, every packet goes to CPU4 which overflows. After, CPU4 takes the interrupt and hands the work to CPUs 0-3." style="width:100%;height:auto;background:#141a1e;border-radius:10px">
  <style>
    .t{fill:#cfc9bb;font:600 15px system-ui,sans-serif}
    .s{fill:#9a958a;font:12px system-ui,sans-serif}
    .c{fill:#1f282d;stroke:#3a464d;stroke-width:1.5}
    .hot{fill:#3a2420;stroke:#b85648;stroke-width:2}
    .ok{fill:#1d2e2b;stroke:#4d8f85;stroke-width:2}
    .a{stroke:#c89445;stroke-width:2.5;fill:none;marker-end:url(#ar)}
  </style>
  <defs><marker id="ar" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="6" markerHeight="6" orient="auto"><path d="M0,0L10,5L0,10z" fill="#c89445"/></marker></defs>
  <text x="160" y="26" text-anchor="middle" class="t">Before: RPS off</text>
  <text x="480" y="26" text-anchor="middle" class="t">After: RPS on</text>
  <line x1="320" y1="40" x2="320" y2="285" stroke="#2c363c" stroke-width="1.5"/>
  <rect x="110" y="44" width="100" height="34" rx="6" class="c"/><text x="160" y="66" text-anchor="middle" class="s">10G NIC</text>
  <path d="M160 78 L160 118" class="a"/>
  <rect x="105" y="122" width="110" height="46" rx="8" class="hot"/><text x="160" y="143" text-anchor="middle" class="t">CPU4</text><text x="160" y="160" text-anchor="middle" class="s">95%, dropping</text>
  <g>
    <rect x="25" y="200" width="56" height="34" rx="6" class="c"/><text x="53" y="221" text-anchor="middle" class="s">CPU0</text>
    <rect x="91" y="200" width="56" height="34" rx="6" class="c"/><text x="119" y="221" text-anchor="middle" class="s">CPU1</text>
    <rect x="173" y="200" width="56" height="34" rx="6" class="c"/><text x="201" y="221" text-anchor="middle" class="s">CPU2</text>
    <rect x="239" y="200" width="56" height="34" rx="6" class="c"/><text x="267" y="221" text-anchor="middle" class="s">CPU3</text>
  </g>
  <text x="160" y="262" text-anchor="middle" class="s">idle while packets spill on the floor</text>
  <rect x="430" y="44" width="100" height="34" rx="6" class="c"/><text x="480" y="66" text-anchor="middle" class="s">10G NIC</text>
  <path d="M480 78 L480 118" class="a"/>
  <rect x="425" y="122" width="110" height="46" rx="8" class="ok"/><text x="480" y="143" text-anchor="middle" class="t">CPU4</text><text x="480" y="160" text-anchor="middle" class="s">takes the interrupt</text>
  <path d="M455 168 L373 198" class="a"/><path d="M470 168 L439 198" class="a"/><path d="M490 168 L521 198" class="a"/><path d="M505 168 L587 198" class="a"/>
  <rect x="345" y="200" width="56" height="34" rx="6" class="ok"/><text x="373" y="221" text-anchor="middle" class="s">CPU0</text>
  <rect x="411" y="200" width="56" height="34" rx="6" class="ok"/><text x="439" y="221" text-anchor="middle" class="s">CPU1</text>
  <rect x="493" y="200" width="56" height="34" rx="6" class="ok"/><text x="521" y="221" text-anchor="middle" class="s">CPU2</text>
  <rect x="559" y="200" width="56" height="34" rx="6" class="ok"/><text x="587" y="221" text-anchor="middle" class="s">CPU3</text>
  <text x="480" y="262" text-anchor="middle" class="s">packet processing shared, drops ~gone</text>
</svg>
</div>
<h2 id="step-3-why-one-core">Step 3: Why one core?</h2>
<p>This is the part that&rsquo;s actually interesting.</p>
<p>Inside the NVR is a phone-class Qualcomm chip driving a 10G network card through a Toshiba PCIe Ethernet bridge. That card delivers incoming packets through <strong>one interrupt</strong>, and Ubiquiti pins that interrupt to <strong>CPU4</strong> on purpose. They even fence CPU4 off from normal apps at boot so recording and AI never compete with networking. (It shows up as a &ldquo;thermal pause&rdquo; flag on CPU4. It is not a heat problem. I chased that for way longer than I want to admit.)</p>
<p><img alt="Comic: the host charges in with a fire extinguisher at a computer chip lounging in a beach chair with sunglasses and a THERMAL PAUSE sign. The chip says RELAX. I&rsquo;M NOT HOT. I&rsquo;M RESERVED. Caption: WASTED AN HOUR ON THIS." loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/thermal-pause.webp"></p>
<p>On a lightly loaded NVR, that&rsquo;s a smart design. Networking gets its own core.</p>
<p>The catch is a Linux feature called <strong>RPS, Receive Packet Steering</strong>. With RPS on, the core that takes the interrupt can hand the heavier packet processing to other cores. With RPS off, that one core does everything.</p>
<p>Linux ships with RPS off. UniFi doesn&rsquo;t turn it on. So one mid-size core is a hard ceiling for how much camera traffic the box can take in. Fifty cameras plus Blue Iris plus live viewers blew right through it.</p>
<p>An 8-core box, sold to record dozens of cameras, with the setting that spreads the network load turned off out of the box. Cool. Love that for us.</p>
<p><img alt="Comic: a bored factory robot stamps boxes on a conveyor belt with RPS: OFF under a FACTORY DEFAULT sign, while the host holds up a stamped box with steam coming out of his ears yelling WHO SHIPS IT LIKE THIS?!" loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/factory-default.webp"></p>
<p>You can check yours over SSH:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#282a36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>cat /sys/class/net/eth1/queues/rx-*/rps_cpus
</span></span></code></pre></div><p>All <code>00</code> means RPS is off. Mine were all <code>00</code>.</p>
<h2 id="step-4-the-fix">Step 4: The fix</h2>
<p>One short loop, live, no reboot:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#282a36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#8be9fd;font-style:italic">cd</span> /sys/class/net/eth1/queues
</span></span><span style="display:flex;"><span><span style="color:#ff79c6">for</span> q in rx-*; <span style="color:#ff79c6">do</span>
</span></span><span style="display:flex;"><span>  <span style="color:#8be9fd;font-style:italic">echo</span> 0f &gt; <span style="color:#8be9fd;font-style:italic">$q</span>/rps_cpus
</span></span><span style="display:flex;"><span><span style="color:#ff79c6">done</span>
</span></span></code></pre></div><p><code>0f</code> is a bitmask for CPUs 0 through 3. CPU4 still takes the interrupt, but now it passes the heavy lifting to four other cores.</p>
<p>That&rsquo;s it. That&rsquo;s the fix. Weeks of &ldquo;it&rsquo;s lagging again&rdquo; and the answer was one line of shell.</p>
<p><img alt="Comic: the host types one short line on a laptop with one finger, deadpan, in front of a mountain of crumpled paper, coffee cups and printouts labeled THEORIES, and a calendar with weeks crossed out. He says WEEKS. FOR ONE LINE." loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/one-line.webp"></p>
<p>I armed a rollback before touching it, applied it during business hours with cameras recording, and nothing went offline.</p>
<p>Undo is just as short:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#282a36;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-text-size-adjust:none;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#8be9fd;font-style:italic">cd</span> /sys/class/net/eth1/queues
</span></span><span style="display:flex;"><span><span style="color:#ff79c6">for</span> q in rx-*; <span style="color:#ff79c6">do</span>
</span></span><span style="display:flex;"><span>  <span style="color:#8be9fd;font-style:italic">echo</span> <span style="color:#bd93f9">00</span> &gt; <span style="color:#8be9fd;font-style:italic">$q</span>/rps_cpus
</span></span><span style="display:flex;"><span><span style="color:#ff79c6">done</span>
</span></span></code></pre></div><h2 id="step-5-prove-it-dont-just-feel-it">Step 5: Prove it, don&rsquo;t just feel it</h2>
<p>&ldquo;Seems better&rdquo; isn&rsquo;t proof. So I ran an A/B test: pulled 9 high-quality camera streams from the viewer&rsquo;s PC and flipped RPS on, off, and on again, two minutes each.</p>
<table>
	<thead>
			<tr>
					<th>RPS</th>
					<th>Drops</th>
					<th>Stalls</th>
					<th>Worst</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>on</td>
					<td>60</td>
					<td>0</td>
					<td>0.24 s</td>
			</tr>
			<tr>
					<td><strong>off</strong></td>
					<td><strong>66,145</strong></td>
					<td><strong>126</strong></td>
					<td><strong>1.83 s</strong></td>
			</tr>
			<tr>
					<td>on</td>
					<td>37</td>
					<td>0</td>
					<td>0.23 s</td>
			</tr>
	</tbody>
</table>
<p><em>Stalls = video gaps over 300 ms. With RPS off, 10 of them lasted over a full second.</em></p>
<p>Turn RPS off and I could reproduce their freezes on demand. Turn it on and they vanished. I felt like a mad scientist. A very tired, very annoyed mad scientist.</p>
<p><img alt="Comic: the host in a lab coat and goggles yanks a giant RPS lever. One monitor labeled OFF shows a frozen camera picture and 66,145 DROPS. The other labeled ON shows a smooth picture and 60 DROPS. He yells IT&rsquo;S ALIVE!" loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/flip-the-switch.webp"></p>
<p>The longer before and after:</p>
<table>
	<thead>
			<tr>
					<th></th>
					<th>Before</th>
					<th>After</th>
			</tr>
	</thead>
	<tbody>
			<tr>
					<td>Dropped packets</td>
					<td>~164 per second</td>
					<td>~0.4 per second</td>
			</tr>
			<tr>
					<td>Ping from viewer to NVR (95th percentile)</td>
					<td>12.7 ms</td>
					<td>1.0 ms</td>
			</tr>
			<tr>
					<td>Traffic</td>
					<td>same</td>
					<td>same</td>
			</tr>
	</tbody>
</table>
<p>That&rsquo;s about a <strong>99.7% drop in lost packets</strong>. Then we loaded it for real: Blue Iris plus UniFi showing every camera at once. Both &ldquo;working really well,&rdquo; in the client&rsquo;s words.</p>
<h2 id="making-it-stick">Making it stick</h2>
<p>A setting in <code>/sys</code> resets on reboot. So I added a tiny systemd service that sets the mask at boot. On these boxes the root filesystem is an overlay, so the service survives a reboot.</p>
<p><strong>But a firmware update may wipe it.</strong> After any UniFi OS update, run the <code>cat</code> command above again. If it&rsquo;s back to <code>00</code>, put it back. I fully expect to be writing &ldquo;the cameras are lagging again, part 2&rdquo; someday because of this.</p>
<p><img alt="Comic: a sneaky villain robot in a cape and top hat labeled FIRMWARE UPDATE creeps toward a small NVR box holding a giant eraser. The host guards the box with a baseball bat and a sticky note on his forehead reading RE-CHECK RPS, saying DON&rsquo;T. YOU. DARE." loading="lazy" src="/posts/unifi-nvr-one-cpu-core-doing-all-the-work/firmware-villain.webp"></p>
<h2 id="things-that-looked-scary-but-werent">Things that looked scary but weren&rsquo;t</h2>
<ul>
<li><strong>&ldquo;Thermal pause&rdquo; on CPU4:</strong> that&rsquo;s the network driver reserving the core on purpose, not overheating.</li>
<li><strong>CPU temps in the 80s and 90s C:</strong> normal for this chip. The real trip point is 108 C.</li>
<li><strong>Fans running slow:</strong> UniFi&rsquo;s fan daemon follows the board sensors, not the CPU. Working as designed. Don&rsquo;t fight it.</li>
</ul>
<h2 id="i-checked-my-own-nvr-too">I checked my own NVR too</h2>
<p>My home NVR, same model family, had the exact same setup: interrupt pinned to CPU4, RPS off. With 21 cameras it was using about 78% of that core. Not hurting yet, but heading there. I turned RPS on there too (mask <code>6f</code>, which also uses two of the faster cores), and CPU4&rsquo;s softirq load dropped from 24% to 17% with zero drops.</p>
<p>One honest limit: RPS only moves the <em>processing</em> half of the work. The interrupt itself still lands on CPU4, and that part can&rsquo;t be spread on this hardware. So RPS buys real headroom, it doesn&rsquo;t make the ceiling go away. If you&rsquo;re adding a lot of cameras, it also helps to have Blue Iris pull sub-streams or pull from the cameras directly instead of through the NVR.</p>
<h2 id="the-short-version">The short version</h2>
<p>If your UniFi Protect live view stutters on a busy G2-class NVR and you&rsquo;ve already blamed the PC, the cable and the switch:</p>
<ol>
<li>SSH in and check <code>rps_cpus</code> on eth1.</li>
<li>Look at per-core CPU. If one core is pegged and the rest are bored, that&rsquo;s it.</li>
<li>Turn on RPS, test it, and keep the undo command handy.</li>
<li>Re-check after every firmware update.</li>
</ol>
<p>This is an unsupported tweak to a box Ubiquiti manages, so do it with a rollback ready and at your own risk. But it took a &ldquo;cameras are lagging&rdquo; ticket from weeks of frustration to a one-line fix. And now the seven lazy cores finally have to work for a living.</p>
<p><em>Got a stubborn tech problem like this? Text or email me: <a href="sms:+17195304040">719-530-4040</a> or <a href="mailto:service@kccsonline.com">service@kccsonline.com</a>. More at <a href="https://kccsonline.com">kccsonline.com</a>.</em></p>
]]></content:encoded></item></channel></rss>