Proxmox cluster

Four nodes meshed via corosync over the LAN, plus a Raspberry Pi off to the side that doubles as the backup server and a corosync QDevice. Hover any node:

The whole rack is on a UPS, but the laptop’s internal battery buys the cluster a much longer tail in an extended outage. The HAOS virtual machine (Home Assistant’s control plane for the rest of the house) lives on that node so the orchestration brain is the last thing to lose power. The laptop also owns the UPS: the unit reports to it over USB, and the other three nodes follow it over the network as NUT secondaries.

A 5-minute systemd timer pushes the battery’s charge, voltage, cycle count, AC state and charge cap into InfluxDB. The charge is capped at 80 % to spare the pack, which still leaves 1.6 hours of measured runtime. The obvious health metric, full capacity against design, is a stored constant rather than a measurement: the fuel gauge only re-learns it on a complete discharge, and the node halts itself at 20 % before one can happen. So Grafana watches the cap for drift, and once a year a timed unplug measures real watts and runtime.

# /usr/local/bin/push-battery-metrics.sh (excerpt)
BAT=/sys/class/power_supply/BAT1
while IFS='=' read -r k v; do
  [[ $k == POWER_SUPPLY_* ]] && B[$k]=$v
done < "$BAT/uevent"
capacity_pct=${B[POWER_SUPPLY_CAPACITY]:-0}
cycle_count=${B[POWER_SUPPLY_CYCLE_COUNT]:-0}
# eco-charge cap: 80 or 100 only, so anything but 80 is drift
charge_limit=$(cat "$BAT/charge_control_end_threshold")
curl -fsS -u "$user:$pass" --data-binary \
  "battery,host=pvelaptop capacity_pct=${capacity_pct}i,cycle_count=${cycle_count}i,charge_limit=${charge_limit}i" \
  "$INFLUX_URL"

The desktop host used to ride two onboard I226-V 2.5 Gbps interfaces. A known firmware bug on that NIC produced a hard link flap every few minutes, which broke long-running TCP streams, most painfully PBS chunk uploads, which the kernel held open well past the HTTP/2 idle timeout.

A Mellanox ConnectX-3 EN now collapses everything onto a single 10 G SFP+ uplink to the aggregation switch. iperf3 sustains line rate with zero retransmits:

$ iperf3 -c 192.168.1.223 -t 30 -P 4
[SUM]   0.00-30.00  sec  3.29 GBytes  942 Mbits/sec  0    sender
[SUM]   0.00-30.04  sec  3.29 GBytes  942 Mbits/sec       receiver
# (capped at ~942 Mb/s by the Pi side, not the link)

The trade-off is a single point of failure, which is the right one to take for a home lab: the two onboard ports remain cabled in, just admin-down, ready to bond back in if the Mellanox ever dies.

Four cluster nodes (1 vote each) plus a Pi-hosted QDevice (3 votes under the LMS algorithm) gives 7 total votes, quorum = 4. One surviving node plus the Pi clears that bar, so three of the four nodes can drop and the cluster keeps running.

$ sudo pvecm status
Quorum information
  Date:             Fri May 22 21:47:50 2026
  Nodes:            4
  Quorate:          Yes

Votequorum information
  Expected votes:   7
  Highest expected: 7
  Total votes:      7
  Quorum:           4
  Flags:            Quorate Qdevice

Membership information
  Nodeid      Votes    Qdevice Name
  0x00000001       1   A,V,NMW pvedesktopmsi
  0x00000002       1   A,V,NMW pvelaptop
  0x00000003       1   A,V,NMW pvewyse5070
  0x00000004       1   A,V,NMW pvehp
  0x00000000       3            Qdevice

Proxmox Backup Server runs on the same Pi. The schedule is staggered per-node (21:00, 22:00, 23:00, 00:00) because the Pi cannot drain all four nodes’ dirty bitmaps simultaneously without stalling, and a stalled qemu I/O path corrupted ext4 inside HAOS twice. That VM is now out of PBS entirely; Home Assistant’s own nightly backup to the NAS is its recovery path.

The NAS container gets its own weekly slot at 2 a.m. on Sunday. A container backup reads every file whether it changed or not, so its 2–4 hour run stays clear of everything else. The photo library used to ride that slot too; in August it moved to a single SSD on the EliteDesk and now goes out with that node’s nightly job.