I noticed something a little counter intuitive for me…

My home computer has the following specs:

• CPU: Ryzen 9 7900X : 12 Cores / 24 threads (2 CCDs) / Special NUMA architecture

• So There is basically 6 cores per silicon die.

The computer is running Windows 11.

I created an Kubuntu 24.04 with initially 4 CPUs assigned in VBox.

As I have overall 24 threads/vcpus available I bumped CPU count to 8 for the VM.

All of a sudden when logged in the VM, the UI was feeling sluggish/laggy as if there was not enough RAM…

Turns out because all the Cores are spread accross 2 dies/chips/sockets-ish, there is some cost when a program is running threads accross the 2 dies. So the OS needs to know not to do that…

Turns out VBox is not NUMA aware so when assigning 8 vpus, the 8 vcpus might spill to the other die which again is very costly when a single program has to make threads talk to each other accross the dies.

VMWare of KVM/Quemu(on linux) are NUMA aware so they know that iniherent achitecture of the CPUs so they know to prefrer scheduling the vcpus on a single die to prevent the overhead of cross-die communicating.

Also because there is only actually 12 real cores and there is also a hyperthreading cost so if one of the vcpus is actually a hyperthread, its actually going to also be a little slower (if a vcpus-hypertrhread is actually on another cpu die, it’s even worse)

So its best to only assign at most half of the real cores one has.

So for my processor, I only have 6 real cores, I shouldnt do anything that has some chance of spilling to the other cpu die.

But I still have 12 threads however, again VBox is not ccd/numa aware, and vbox all vcpus are equal which is not true at all. So even though VBox shows 24 vcpus,

Because 12 vcpus of those are on another ccd we shouldnt get above 12

Because 6 of those 12 vcpus per ccd are hyperthreaded we really shouldnt go above 6 as we are certainly be using vcpus which are hyperthreads or on another die which can also cause latency

But because we only really have 6 real cores usable per ccd, using 6 vcpus will likely result in some of those being hyperthreads which can also cause latency / co-scheduling issues

I though i should just go with 3 vcpus but nope Linux guests work best when number of cpus is a power of 2. so the best is 4 vcpus which leave in some room to keep all those vcpus on the same ccd, and not cause any hyperthread vcpus related issue…

However all this BS is just because VBox is not NUMA/CCD aware. If i were on linux using QUEMU/KVM or VMWare I think I wouldnt have these issues.

ChatGPT rewrite

Title: A Counter-Intuitive VirtualBox Performance Lesson on Ryzen

Why My Linux VM Got Slower When I Gave It More CPUs

I recently ran into a counter-intuitive performance issue while running a Linux VM.

My system uses a Ryzen 9 7900X:

• 12 physical cores / 24 threads

• 2 CCDs (chiplets)

• NUMA-like architecture (6 cores per CCD)

The host OS is Windows 11.

The Setup

I created a Kubuntu 24.04 VM in VirtualBox, initially assigning 4 vCPUs.
The VM felt smooth and responsive.

Since the CPU exposes 24 logical threads, I increased the VM to 8 vCPUs, expecting better performance.

Instead, the desktop became noticeably laggy — window movement stuttered and the UI felt unresponsive, despite normal RAM and CPU usage.

The Cause

The Ryzen 7900X is not a single monolithic chip. Each CCD has its own L3 cache, and communication between CCDs has a real latency cost.

The problem is that VirtualBox is not NUMA or CCD aware. When assigning 8 vCPUs, VirtualBox may schedule them across both CCDs, even for a single workload like a desktop environment. This forces threads to communicate across CCDs, hurting latency-sensitive tasks such as GUI rendering.

Hypervisors like KVM/QEMU or VMware are topology-aware and usually try to keep vCPUs within the same CCD, avoiding this issue.

SMT Makes It Worse

Out of the 24 logical CPUs:

• Only 12 are real physical cores

• The rest are SMT (hyper-threaded) siblings

VirtualBox treats all vCPUs as equal, even though SMT threads are slower and more prone to contention — especially when combined with cross-CCD scheduling.

The Practical Limit

To avoid cross-CCD traffic, vCPUs should stay well below the 6 physical cores per CCD.
However, assigning all 6 risks SMT contention and scheduling latency.

Although 3 vCPUs seemed like the safest option, Linux guests tend to behave better with power-of-two CPU counts.

The Sweet Spot

4 vCPUs turned out to be ideal:

• Fits comfortably within one CCD

• Leaves scheduling headroom

• Minimizes SMT contention

• Eliminates cross-CCD latency

• Results in a smooth, responsive desktop

Switching back to 4 vCPUs immediately fixed the lag.

Takeaway

This issue exists because VirtualBox is not NUMA/CCD aware.

On the same hardware, KVM/QEMU or VMware would likely handle 8 vCPUs without issue.

The lesson is simple:

More vCPUs does not always mean better performance — especially on modern multi-CCD CPUs when using VirtualBox.

Sometimes, fewer CPUs really are faster.

Resources:

• https://forums.virtualbox.org/viewtopic.php?t=110399&start=15

• https://automatestack.dev/the-ultimate-guide-to-right-sizing-cpu-and-memory-for-virtual-machines

• https://forums.virtualbox.org/viewtopic.php?t=94076

• https://www.xda-developers.com/how-many-cpus-vm/

Make sure container doesn’t start right after creation

Append extra config for to lxc container conf file

On the proxmox host edit the file /etc/pve/lxc/<container id>.conf and add the following content

lxc.apparmor.profile: unconfined
lxc.cgroup.devices.allow: a
lxc.cap.drop:
lxc.mount.auto: "proc:rw sys:rw"

Create missing /dev/kmsg file

Start the container and inside the container create the file /etc/rc.local if it doesnt exist. Add the following content to the file and update the file permissions like so:

cat <<EOF > /etc/rc.local
#!/bin/sh -e
if [ ! -e /dev/kmsg ]; then
    ln -s /dev/console /dev/kmsg
fi
mount --make-rshared /
EOF

chmod +x /etc/rc.local
/etc/rc.local

Install K3s

curl -sfL https://get.k3s.io | sh -s - --disable=traefik --disable=servicelb --node-name control.k8s

# Setup kubectl for non-root user access
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc
echo 'source <(kubectl completion bash)' >>~/.bashrc
echo 'alias k=kubectl' >>~/.bashrc
echo 'complete -o default -F __start_kubectl k' >>~/.bashrc
source ~/.bashrc
mkdir ~/.kube 2> /dev/null
sudo k3s kubectl config view --raw > "$KUBECONFIG"
chmod 600 "$KUBECONFIG"
# Test to make sure non-root kubectl is working
kubectl get nodes

Install Helm

# Install Helm
curl https://baltocdn.com/helm/signing.asc | gpg --dearmor | sudo tee /usr/share/keyrings/helm.gpg > /dev/null
sudo apt-get install apt-transport-https --yes
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/helm.gpg] https://baltocdn.com/helm/stable/debian/ all main" | sudo tee /etc/apt/sources.list.d/helm-stable-debian.list
sudo apt-get update
sudo apt-get install helm
helm version

Install MetalLB

# Install MetalLB
helm repo add metallb https://metallb.github.io/metallb
helm install metallb metallb/metallb --namespace metallb-system --create-namespace

cat <<EOF > metallb_config.yaml
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: nginx-ingress-pool
  namespace: metallb-system
spec:
  addresses:
  - 10.10.10.100/32
  autoAssign: false
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: regular-lb-pool
  namespace: metallb-system
spec:
  addresses:
  - 10.10.10.110-10.10.10.120
  autoAssign: false
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: default-l2advertisement
  namespace: metallb-system
spec:
  ipAddressPools:
  - nginx-ingress-pool
  - regular-lb-pool
EOF

kubectl apply -f metallb_config.yaml
# Ensure the address pools are well created
kubectl describe ipaddresspools.metallb.io -n metallb-system

Install Nginx Ingress Controller

# Install NGINX ingress controller
cat <<EOF > nginx-ingress_values.yaml
controller:
    service:
        annotations:
            metallb.universe.tf/address-pool: nginx-ingress-pool
EOF

helm install nginx-ingress-release --values app1/nginx-ingress_values.yaml --create-namespace --namespace nginx-ingress oci://ghcr.io/nginxinc/charts/nginx-ingress --version 1.4.0 --values app1/nginx-ingress_values.yaml # --set-json 'controller.service.annotations={"test1": "de"}'
helm upgrade nginx-ingress-release --values app1/nginx-ingress_values.yaml -n nginx-ingress oci://ghcr.io/nginxinc/charts/nginx-ingress

# Ensure that the supplied values are taken into account
helm get values -n nginx-ingress nginx-ingress-release
# Uninstall to reinstall again if needed
helm -n nginx-ingress uninstall nginx-ingress-release
# Check the nginx-ingress controller service is provided an IP address by MetalLB
kubectl describe svc -n nginx-ingress nginx-ingress-release-controller

Install cert-manager

helm repo add jetstack https://charts.jetstack.io --force-update

helm install cert-manager jetstack/cert-manager --create-namespace --namespace cert-manager --version v1.16.0 --set crds.enabled=true --set 'extraArgs={--dns01-recursive-nameservers-only,--dns01-recursive-nameservers=8.8.8.8:53\,1.1.1.1:53}'

Sources:

• https://kevingoos.medium.com/kubernetes-inside-proxmox-lxc-cce5c9927942

• https://bobcares.com/blog/rancher-lxc-proxmox/

• https://docs.k3s.io/quick-start#install-script

• https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#bash

• https://devops.stackexchange.com/questions/16043/error-error-loading-config-file-etc-rancher-k3s-k3s-yaml-open-etc-rancher

• https://helm.sh/docs/intro/install/

• https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/

• https://blog.mossroy.fr/2024/04/16/passage-sur-metallb-au-lieu-de-servicelb-sur-un-cluster-k3s/

• https://metallb.universe.tf/installation/

• https://github.com/metallb/metallb/issues/1964

• https://blog.mossroy.fr/2024/04/16/passage-sur-metallb-au-lieu-de-servicelb-sur-un-cluster-k3s/

• https://cert-manager.io/docs/installation/helm/

Resources:

• https://forum.proxmox.com/threads/e1000e-reset-adapter-unexpectedly.87769/

• https://serverfault.com/questions/616485/e1000e-reset-adapter-unexpectedly-detected-hardware-unit-hang

• https://kasmweb.com/docs/latest/install/single_server_install.html

apt update && apt upgrade -y && apt install curl -y
cd /tmp
curl -O https://kasm-static-content.s3.amazonaws.com/kasm_release_1.13.1.421524.tar.gz
tar -xf kasm_release_1.13.1.421524.tar.gz
bash kasm_release/install.sh --accept-eula --swap-size 8192 --admin-password admin --user-password user

Start/Stop

# restart all services on a server
cd /opt/kasm/bin
./stop
./start

# restart individual components
sudo docker restart kasm_agent
sudo docker restart kasm_api
sudo docker restart kasm_manager
sudo docker restart kasm_db
sudo docker restart kasm_proxy

Uninstallation

• https://kasmweb.com/docs/latest/install/uninstall.html

sudo /opt/kasm/current/bin/stop
sudo docker rm -f $(sudo docker container ls -qa --filter="label=kasm.kasmid")
export KASM_UID=$(id kasm -u)
export KASM_GID=$(id kasm -g)
sudo -E docker compose -f /opt/kasm/current/docker/docker-compose.yaml rm
sudo docker network rm kasm_default_network
sudo docker volume rm kasm_db_1.13.1

sudo rm -rf /opt/kasm/
sudo deluser kasm_db
sudo deluser kasm

Tuning

• https://kasmweb.com/docs/latest/guide/kasm_performance.html

Resources:

• https://kasmweb.com/docs/latest/guide/kasm_performance.html

• https://kasmweb.com/docs/latest/guide/settings.html

chmod 666 /dev/dri/renderD128

# For a persistent way to give the permissions (this worked for me the first time but later on stopped working)
cat > /etc/udev/rules.d/99-intel-chmod666.rules << 'EOF'
KERNEL=="renderD128", MODE="0666"
KERNEL=="card0", MODE="0666"
EOF
# Reboot proxmox host

cat >> /etc/pve/lxc/xyz.conf <<EOF
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file 0, 0
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir
EOF

apt install vainfo
vainfo

To check the usage of the iGPU one can run on the proxmox host the command:

sudo apt install intel-gpu-tools
sudo intel_gpu_top

Resources:

• https://ketanvijayvargiya.com/302-hardware-transcoding-inside-an-unprivileged-lxc-container-on-proxmox/

• https://github.com/blakeblackshear/frigate/discussions/5773

• https://bookstack.swigg.net/books/linux/page/lxc-gpu-access

• https://forum.proxmox.com/threads/mediated-device-passthrough-to-lxc-container.128636/

• https://forum.proxmox.com/threads/lxc-emby-dont-detect-hardware-acceleration.117173/

• https://forum.proxmox.com/threads/lxc-i9-12900t-gpu-plex-passthrough.109439/

`~/.ssh/config`

Host github_ssh_connection
   HostName github.com
   IdentityFile ~/.ssh/id_rsa_custom

git clone git@github_ssh_connection:user/repo.git your-folder-name

Resources:

• https://ralphjsmit.com/git-custom-ssh-key

Resources:

• https://blogmotion.fr/internet/pfsense-clavier-azerty-16564

$ sudo apt update
$ sudo apt install lightdm
$ sudo reboot
$ sudo apt install x11vnc

$ sudo nano /lib/systemd/system/x11vnc.service

!Copy and paste these commands - change the password
[Unit]
Description=x11vnc service
After=display-manager.service network.target syslog.target

[Service]
Type=simple
ExecStart=/usr/bin/x11vnc -forever -display :0 -auth guess -passwd password
ExecStop=/usr/bin/killall x11vnc
Restart=on-failure

[Install]
WantedBy=multi-user.target

!Save file and run these commands:

$ systemctl daemon-reload
$ systemctl enable x11vnc.service
$ systemctl start x11vnc.service
$ systemctl status x11vnc.service

Resources:

• https://www.crazy-logic.co.uk/projects/computing/how-to-install-x11vnc-vnc-server-as-a-service-on-ubuntu-20-04-for-remote-access-or-screen-sharing
• https://youtu.be/3K1hUwxxYek - [Ubuntu VNC Server - David Bombal]
• https://youtu.be/633OWaW3cyo - [Linux Desktop in the Cloud Tutorial | Create and Access From Anywhere - LearnLinuxTV - Linode]

lxc.idmap = u 1000000 1000000 200000
lxc.idmap = g 1000000 1000000 200000
lxc.idmap = u 0 100000 65536
lxc.idmap = g 0 100000 65536

Make sure to add these in /etc/{subuid,subgid}

# FreeIPA ids
root:1000000:2000000
# FreeIPA ids

And make sure to install the ipa server with the options: ipa-server-install --setup-dns --no-ntp --mkhomedir --idstart=1000000 --idmax=1999999

Resources:

• https://kiwix.ounapuu.ee/serverfault.com_en_all_2019-02/A/question/848620.html
• https://kiwix.ounapuu.ee/serverfault.com_en_all_2019-02/A/question/777095.html
• https://forum.proxmox.com/threads/can-i-ask-an-uid-range-not-to-be-mapped-in-an-unprivileged-container.49544/
• https://forum.proxmox.com/threads/problems-using-a-mount-point-and-lxc-idmap.77370/
• https://superuser.com/questions/1518783/how-do-i-take-advantage-of-freeipa-centralized-authentication-in-an-lxc-containe
• https://ubuntu.com/blog/nested-containers-in-lxd

$ scp source destination [--sparse=always] @destination $ fallocate -d filename_on_destination_server

can also use dd to convert into or create sparse files

Resources:

• https://serverfault.com/questions/665335/what-is-fastest-way-to-copy-a-sparse-file-what-method-results-in-the-smallest-f

$ nmcli connection modify <connection-name> ipv4.dhcp.hostname=<new_name>

I needed to do exactly the same some days ago, but I was facing some issues, so I am basically writing this article to help out those who may be having similar issues.

Basically the workflow to go from a domU to a template is the following:

• Power off the domU
• Make the domU disk available to the host system as a folder (mount the domu disk into the dom0)
• Create a tar of the folder
• Then, use the tar file as the installation source of other domUs

1. Power off the domU
2. Make the domU disk available to the host system as a folder (mount the domu disk into the dom0)
3. Create a tar of the folder
4. Then, use the tar file as the installation source of other domUs

Resources:

• https://wiki.xenproject.org/wiki/Xen_Project_Beginners_Guide

• https://wiki.debian.org/NetworkConfiguration

davidassigbi@hp-elitebook:~$ ip route 
default via 10.188.0.1 dev wlo1 proto dhcp metric 600 
10.10.10.0/24 dev enp0s25 proto kernel scope link src 10.10.10.2 metric 100 
10.188.0.0/16 dev wlo1 proto kernel scope link src 10.188.197.57 metric 600 
169.254.0.0/16 dev mybridge scope link metric 1000 
192.168.1.0/24 via 192.168.1.5 dev mybridge 
192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 

davidassigbi@hp-elitebook:~$ sudo ip route add 100.96.1.0/24 via 192.168.1.1 dev mybridge 
Error: Nexthop has invalid gateway.

davidassigbi@hp-elitebook:~$ ip a show mybridge
4: mybridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether ba:5a:99:6e:19:87 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.5/24 brd 192.168.1.255 scope global mybridge
       valid_lft forever preferred_lft forever
    inet6 fe80::b85a:99ff:fe6e:1987/64 scope link 
       valid_lft forever preferred_lft forever

davidassigbi@hp-elitebook:~$ sudo brctl show mybridge
bridge name    bridge id        STP enabled    interfaces
mybridge        8000.ba5a996e1987    no        vnet1
                            vnet4
                            vnet6

And the final solution was to add the onlink argument, which according to the ip route man page says it forces the kernel to assume the interface is directly connected to the system while it actually is not.

davidassigbi@hp-elitebook:~$ sudo ip route add 100.96.1.0/24 via 192.168.1.1 dev mybridge  onlink

Resources:

• https://man7.org/linux/man-pages/man8/ip-route.8.html : look up the onlink on the page
• https://unix.stackexchange.com/a/644486 : a question on stackexchange discussing the same issue

$ sudo vi /etc/systemd/logind.conf # and the change the parameter to ignore

[Login]
HandleLidSwitch=ignore
HandleLidSwitchDocked=ignore # This option specifically is the default on Debian so if on your system did not modify that option you can just not write it 

$ sudo systemctl restart systemd-logind.service # Or reboot for the change to take effect

There are also a ton of other options in that file, so you can explore the content of the file just to have an idea of what you can do.

Resources:

• https://wiki.debian.org/Suspend
• https://www.alphr.com/keep-laptop-when-closed/

• https://www.linuxuprising.com/2019/01/fix-windows-10-or-8-partition-mounted.html

PS > Set-VMProcessor -VMName <VMName> -ExposeVirtualizationExtensions $true

Virtual Box:

/home/<username> $ VBoxManage modifyvm <name> --nested-hw-virt on
PS C:\Program Files\Oracle\VirtualBox> .\VBoxManage.exe modifyvm <name> --nested-hw-virt on

for kvm Resources :

• https://docs.fedoraproject.org/en-US/quick-docs/using-nested-virtualization-in-kvm/

• https://ostechnix.com/how-to-enable-nested-virtualization-in-kvm-in-linux/

• https://techviewleo.com/how-to-enable-nested-virtualization-on-kvm-qemu/

Virtual Box resources:

• https://docs.oracle.com/en/virtualization/virtualbox/6.0/admin/nested-virt.html

• https://forums.virtualbox.org/viewtopic.php?t=109174

Remove old snap versions that might be taking space

#!/bin/bash
# Removes old revisions of snaps
# CLOSE ALL SNAPS BEFORE RUNNING THIS
set -eu
snap list --all | awk '/disabled/{print $1, $3}' |
    while read snapname revision; do
        snap remove "$snapname" --revision="$revision"
    done

Resources:

• https://ubuntuhandbook.org/index.php/2020/12/clear-systemd-journal-logs-ubuntu/
• https://itsfoss.com/free-up-space-ubuntu-linux/