Running Rabbits: More About My Cloud NUCs

As I suspected, the Intel NUCs included in my “Rabbit” cloud hardware are proving much more useful than the Nvidia Jetson TK1s. I received both “Silvermont” NUC5 and “Goldmont” NUC6 machines in my surplus eBay cloud hardware, and both are good little machines, with some notable limitations.

Read my entire Rabbit Cloud series

• Introducing Rabbit: I Bought a Cloud!
• Running Rabbits: More About My Cloud NUCs
• Tortoise or Hare? Nvidia Jetson TK1
• Powering Rabbits: The Mean Well LRS-350-12 Power Supply

My NUC Census

I purchased three “trays” of ex-rabb.it encoding hardware from eBay. Each included 10x Intel NUC single-board computers, 5x NVIDIA Jetson TK1 single-board computers, a Gigabit Ethernet switch and cabling, and a power supply. Although most purchasers received only NUC5 machines with “Silvermont” Pentium N3700 processors, one of my trays included NUC6 machines with “Goldmont” Celeron J3455 processors. The latter also support dual-channel memory (and have two SO-DIMM slots) up to 16 GB.

This last point is a bit of surprise since Intel’s documentation for the NUC6CAYH specifically says it’s limited to 8 GB of RAM. But when I loaded it with two 8 GB DIMMs, the BIOS recognized the RAM and it was available to the Linux kernel.

Another notable difference between these machines is the BIOS. The NUC6 has many more options, especially when it comes to power. One interesting one is “Power Sense”, which will apparently throttle the CPU if the board is going to draw more than the rated power supply level. But I imagine that the Rabbit fuse block will do that too! And frankly it’s not likely I would run into this, with a 10 Watt TDP CPU and no hard disk or peripherals.

Speaking of power, I’m actually using a simple solution to run them in testing. If you’re read my blog much, you know that I’ve often “shucked” external hard disk drives for the SATA drive contained within. I’ve got a pile of these extra “wall warts” in a box, and was pleased to find that they’re just fine to bring up a NUC or Jetson for testing. They supply 12 Volts and the barrel connector is configured correctly with a positive tip. Although they’re rated at just 1.5 Amps, this is plenty of power in practice.

The highest power draw I saw when running Geekbench was just 0.20 Amps (12.8 Watts). Trying to generate more power draw, I fired up the xmrig Monero miner with four threads. Although the CPU is running at maximum speed (2.3 GHz) the fan is barely pushing any air and gets only warm to the touch even after hours of running. These little “bunnies” are amazingly power efficient!

And while we’re on that topic, you can put any thoughts of a massive Monero miner out of your head. Neither Atom CPU has enough L2 cache to run the RandomX algorithm efficiently, so it has to go to main memory constantly. The Pentium N3700 can complete 280 hashes per second, while the Celeron J3455 hits 382 hashes per second. Dual-channel RAM in the Celeron allows it to hit 420 hashes per second with two RAM sticks. This is small potatoes: The Core i3-5010U in my NUC5 can do 640, my Core i7-6700K hits 1800, and my new Ryzen 9 3900X is steady at 9975. Or to put it another way, all 30 NUCs can’t match the hash power of a single Ryzen 9 CPU, and they would use twice as much power doing it!

How to Boot? That is the Question!

The NUC5 and NUC6 can boot from a variety of media:

• PXE Network Boot (this is how they come from rabb.it)
• SATA (requires a powered drive or custom cable)
• USB disk drive (draws more power, expensive)
• USB flash drive (iffy reliability, cheap)
• SD card (not the fastest, cheap)

I’m sticking to the latter for now. I definitely don’t want to mess around with PXE network booting, so that’s right out. Then there’s the issue of sourcing SATA cables and SDDs at a reasonable price. And external USB disk drives don’t sound that attractive. So I’m sticking with SD cards for now.

After browsing Jeff Geerling’s SD card tests, I’m using SanDisk Extreme and Samsung Evo Plus cards. My own experience has shown me just how unreliable and slow generic SD cards can be, and how many counterfeit and defective cards there are in the e-commerce supply chain, so I’m paying a little more for name-brand cards from major retailers.

Best Buy had the best price on the Samsung cards so I bought a bunch of them at retail, and boy are they fast! Plus, SD cards support TRIM out of the box, unlike most other media, and have gotten much more reliable. Short of a real SSD, booting from SD seems like a winner.

Installation of Ubuntu went pretty smoothly once I got started with the SD cards. This is no surprise, since these are basically little PCs with well-supported hardware. This is one reason the NUCs are so useful: Even though they use an oddball Atom CPU, they’re not really unusual at all. I expect that they will be useful and supported for years, if not decades to come. The same can not be said of the Jetson TK1, as I will discuss in my next article!

I created a generic install of Ubuntu 18 on one of the Rabbits and copied the card image to my Mac using dd. Now I can create more copies whenever needed and only have to do a little tuning to get the next Rabbit running: Change the BIOS to disable PXE, enable boot from USB, and enable USB port 4 (which is mysteriously disabled in BIOS on every NUC) and set the hostname to a generic value based on its position in the Rabbit tray.

Note: I experienced some “weirdness” getting the Ubuntu Server install process to recognize an SD card or a USB flash drive as a valid install target. I am not sure if I did something wrong or if Ubuntu’s latest installer really doesn’t want to work with them. But the older images and installers seem to work fine. I solved this by putting the SD card in an external USB reader, and the installer happily wrote to it.

Stephen’s Stance: Good Little Rabbits

My experiments so far have cemented my opinion on the value of the NUCs in a cloud lab environment: They’re plenty snappy with 8 or 16 GB of RAM, even using an SD card for storage. The tray power supply is efficient and capable, and the whole tray doesn’t make much noise or heat. I’ll write more about that soon! Overall, with just about $1k invested, this “cloud of rabbits” is working out fine.