Orange Pi 5: Proxmox, Ubuntu Jammy, LXC, and Snap

/
127
proxmoxdebianubuntulxcfusesnaparm64orange-pi-5

Lately I've been paying a lot more attention to my energy usage, and I've been trying to find ways to reduce it where I can without sacrificing too much. I've already migrated almost all personal and client cloud servers to ARM instances, and I've been experimenting with running my home lab running Proxmox. My file server is ARM based, but it's a bit underpowered for use cases that require any real processing power. I've been waiting for an affordable ARM based SBC to come out that would be powerful enough to run Proxmox, and I finally found it: the Orange Pi 5 (OPi5)

The OPi5 comes with a 64-bit 8-core ARM processor with up to 32GB of RAM. It's a bit more expensive than the Raspberry Pi 4, but it's also a lot more powerful. The OPi5 also has a built-in NVMe SSD slot, which is perfect for running Proxmox. It even outperformed my AMD Ryzen 3960x Threadripper by a whopping 20% on the sysbench CPU benchmark.

Benchmarks

The left is the Threadripper machine, and the right is the OPi5. Bottoms are VMs on their respective machines... VMs! Not containers! Though they are both set to "host" CPU types:)

Anyways, I'm really excited to share this guide with you, and I hope you enjoy it!

Thanks to TheWingsOfWar on Reddit for the this post which got me started on the install.

Allocating Hardware

  1. Procure Orange Pi 5 (Amazon)
  2. Procure SD Card and NVMe SSD (1 TB NVMe SSD on Amazon)

Install Debian

Orange Pi has a Debian image ready for use, and it's pretty easy to get up and running. As a note, the login credentials are orangepi for both the username and password. Let's get started!

  1. Download the Official Orange Pi Debian Image
  2. Flash both the SD Card and the NVMe SSD with the image, I use balenaEtcher
  3. Install both the SD Card and the NVMe SSD into the OPi5.
  4. Make sure your keyboard and monitor are plugged in, and power on the OPi5.
  5. Once you're booted, you should be automatically logged in.
  6. Now we need to update the onboard SPI flash with the bootloader for the NVMe SSD. We'll do this by running the following command:sudo nand-sata-install
  7. Select "Install/Update the bootloader on SPI Flash" and press enter. The system may stop responding for a few minutes, but it will eventually come back, and spit out "Done."
  8. Now we need to shutdown the system, and remove the SD Card. We'll do this by running the following command:sudo shutdown now
  9. Once the device powers off, remove the SD Card, and power on the OPi5.
  10. Your OPi5 should now boot into the Orange Pi flavor of Debian off of your NVMe SSD, and you should be automatically logged in.

If that's all you want to do, you're done! You can now use your OPi5 as a Debian server.

If you want to use Proxmox, you'll want to continue on to the next section :)

Install Proxmox, using Pimox7

Pimox7 is a version of Proxmox that is designed to run on ARM based SBCs. It's a bit more difficult to install, but it's not too bad. I'm going to assume you've already installed Debian on your OPi5, and you're ready to install Proxmox.

  1. The original sequence of commands to download and run the Pimox7 install process had a lengthy URL, this method has fewer keystrokes, since you might be unable to copy/paste to your console. (I'm installing on bare metal)
  2. it a gist you can review, and because gist raw files also have long URLs, I wrapped it with a shortened bitly link.

You could type it all out, if you want, though :)

The install MUST be done as root, so you'll need to run sudo -s to become root. Please review any/all shell scripts before running.

Once you're in a root shell, you can either:

  1. Use this one-liner this, which can be dangerous, but it's a lot less to type:curl -L http://bit.ly/3K4x8Tr | bash
  2. Download the script, make it executable, and run it (this runs the script from the original post):curl -L http://bit.ly/3K4x8Tr -o pimox7.sh chmod +x pimox7.sh ./pimox7.sh

Follow the steps to configure the Proxmox installation. Note: the password you set here will be the password for the Proxmox web interface.

Once the installation is complete, you'll want to do a hardware reboot after the soft reboot. You can do this by running the following command: sudo shutdown -r now

Once the device powers off, power it back on, and you should be able to access the Proxmox interface at https://<your-ip>:8006/

You'll need to login with the username root and the password you set during the installation.

Using the boot device for storing VM and or container data

The OPi5 only has a single NVMe slot, and it's used for the boot device. So, if you want to use the NVMe SSD for storing VM and or container data, you'll need to allow it explicitly.

  1. Navigate to Datacenter > Storage and edit the local storage.
  2. Under Content Types, make sure Container is added.

Once you've done that, you should be able to create LXC containers and use the NVMe SSD for storage, as we do below.

Adding a LXC Container template for Ubuntu Jammy

Ubuntu Jammy is currently the latest LTS version of Ubuntu. This is currently in my preferred flavor of Linux, and I wanted to be able to use it as a LXC container template, as VMs are a bit overkill when I'm just running Linux anyways and the host OS is already running Linux as well.

This means we can get nearly all the benefits of a VM, without the overhead or performance hits of running a VM — and we can run it on a low power, single board computer. This is where the OPi5 really starts to shine.

  1. We'll need to grab the Ubuntu Jammy LXC template from the official Canonical repository. We'll do this by running the going to the following URL: https://uk.lxd.images.canonical.com/images/ubuntu/jammy/arm64/default/
  2. There are daily builds so you're on your own picking the latest one, but I'm going to use the one from 2023-04-03.
  3. Once you've found the latest build, you'll want to download the rootfs.tar.xz file.
  4. Rename the file to ubuntu-jammy-lxd-image.tar.xz (or something similar).
  5. Open the Proxmox web interface, and navigate to Datacenter > (Proxmox Instance) > local and select CT Templates
  6. Click the Upload button, and select the ubuntu-jammy-lxd-image.tar.xz file you downloaded. On macOS I had to click "Show Options" and set the format to "All Files" to be able to select the file.
  7. Once the upload is complete, you should see a message that says "finished file import successfully"
  8. Close the upload window, and you should see the ubuntu-jammy-lxd-image.tar.xz file in the list of templates.

Initialize an LXC Container from Proxmox

Now that we have a template, we can create a LXC container. This is pretty easy, and we'll be able to use the container just like any other VM... almost :)

  1. Open the Proxmox web interface, and navigate to Datacenter > (Proxmox Instance)
  2. In the top right corner, click the Create CT button (not VM!)
  3. Give the container a hostname, and set a password. SSH key option hasn't worked for me, so I just use a password.
  4. Click the Next button.
  5. Select the ubuntu-jammy-lxd-image.tar.xz template you uploaded earlier. Click the Next button.
  6. Select the local storage option, and configure the disk size. Click the Next button.
  7. Configure the number of CPUs, our OPi5 has 8 cores, so we'll use 8. Click the Next button.
  8. Configure the amount of RAM, our OPi5 has 16GB of RAM, we'll use 8GB (8192). I set swap to 1024. Click the Next button.
  9. You may want to configure a static IP here, or select DHCP. I'm going to use DHCP. Click the Next button.
  10. I want to use the DNS settings from the host OS, so I leave the inputs blank. Click the Next button.
  11. Review your settings, tick "Start after created", and click the Finish button.

You should see some command output in the popup window, and hopefully TASK OK - once you see that, you can close the popup.

You should now see your new LXC container in the list of containers on the left hand side. You can double click to launch the console.

Login with the username root and the password you set in step 3.

Getting the LXC container working with snapd (and installing NodeJS!)

The LXC container is almost ready to go, but we need to get snapd working. Because we're using a LXC container, we need to adjust some of the default settings for snapd to work properly.

  1. Run the following commands to install snapd:apt update apt install snapd
  2. If we try to install a snap (I'm attempting to install NodeJS), we get an error:root@test:~# sudo snap install node --classic --channel=18 error: system does not fully support snapd: cannot mount squashfs image using "squashfs": mount: /tmp/syscheck-mountpoint-145357369: mount failed: Operation not permitted.
  3. Open the Proxmox web interface, find your container, and open the Options tab.
  4. Double-click the Features row, and tick/enable keyctl AND FUSE.
  5. Reboot your container, I use reboot from the command line.
  6. Once the container is back up, try installing the snap again:root@test:~# sudo snap install node --classic --channel=18 2023-04-04T05:09:32Z INFO Waiting for automatic snapd restart... node (18/stable) 18.15.0 from OpenJS Foundation (iojs✓) installed

Awesome. Now we're cooking! We can now install whatever we want using snapd, or apt.

Installing SSH (bonus)

I like to have SSH installed on my containers, so I can SSH into them from my host OS. This is especially useful if you're using a headless host machine (like the Orange Pi 5) that is tucked away somewhere without a monitor or keyboard.

  1. Install SSH:apt install openssh-server systemctl enable ssh systemctl start ssh
  2. You can add your public key from your host OS to the ~/.ssh/authorized_keys file, or...
  3. Enable password authentication by editing the /etc/ssh/sshd_config file and uncommenting the line that says:#PasswordAuthentication yes
  4. I suggest you create a new user and add that user to the sudo group, but if you plan on logging in as root, you'll need to adjust the config to allow that.
  5. Restart SSH:systemctl restart ssh

In Conclusion

The Orange Pi 5 has proven to be an excellent choice for running Proxmox and LXC containers, offering a powerful ARM-based alternative to the Raspberry Pi 4. By following the steps outlined in this article, you should now have a functioning OPi5 setup with Proxmox, the ability to create LXC containers, and even install and use snapd within those containers.

The versatility and performance of the Orange Pi 5, combined with the energy efficiency of the ARM architecture, make it a fantastic choice for those looking to reduce their energy consumption without sacrificing too much in terms of capability. As more ARM-based SBCs become available, we can expect to see further improvements in performance and efficiency, which will only serve to enhance the options available to home lab enthusiasts and professionals alike.

As always, keep experimenting, learning, and pushing the boundaries of what can be achieved with Linux and ARM-based systems. The ever-evolving landscape of technology offers countless opportunities to discover new solutions and optimize existing ones. So, whether you're a seasoned Linux veteran or just getting started, continue to explore, innovate, and build upon the incredible work of the open-source community.

I hope this article has been helpful, and I look forward to seeing what you build next!

Until next time, happy hacking :)

TOY MODE
π