OVH UK Bare-Metal Proxmox Additional IP’s and Enhance Control Panel

Jordan

I wrote this after having issues deploying Enhance in a Proxmox Guest on a OVH UK bare-metal instance with additional IP's. Figure it might help people out, but this setup will change in December 2024 allegedly.

https://managingwp.io/live-blog/ovh-uk-bare-metal-proxmox-additional-ips-and-enhance-control-panel/

Zoinkies

Correct me, but isn't Proxmox not really suitable for Enhance as a whole, how can you perform failover etc?

Jordan

Zoinkies Correct me, but isn't Proxmox not really suitable for Enhance as a whole, how can you perform failover etc?

It all depends on what you use for storage, how many nodes you have (Qorum needs 3) and how your application can help with failover. There are lots of threads on the proxmox forum.

https://forum.proxmox.com/threads/high-availability-with-local-zfs-storage.122922/

For instance, WordPress's biggest issue is the database, so typically, running an active/active failover is ideal. Doing most of the data syncing at the guest level (virtual machine) versus Proxmox seems the easiest. Especially with MySQL replication or a Galera cluster.

As for files, pick your poison, you can sync or provide clustered storage. Syncing is easier due to the fact that you're keeping the storage local to each server. Clustered storage can be expensive and can result in latency, which isn't a huge issue in some applications. For file syncing, there's a number of options, rsync, syncthing, unison, or reslio. For clustered file systems drdb comes to mind, just haven't had to setup much up in this regard.

There's so many options to be honest, and sometimes we just offload it all to S3 if it's public images and use git for code management. For storage

Zoinkies

Have you had luck setting proxmox up with shared storage like ceph? I really struggled there with enhance.

Jordan

Zoinkies Have you had luck setting proxmox up with shared storage like ceph? I really struggled there with enhance.

Not yet, that will be another project either when it's require or when I have free time of which isn't abundant right now.

Personally, I can jump into anything and learn it pretty quickly, I've had to pick things up pretty quickly in the HPC world especially working with researchers.

8Dweb

If you have the time and resources, consider setting up an XCP-ng pool with shared storage. We’ve transitioned storage nodes from 10G to 25G to 100G seamlessly over the years, initially using XenServer before switching to XCP-ng.

XOA management can be built from source, offering nearly all enterprise features except XO-SAN, which I never used. For storage, you can choose iSCSI (thick provision) or NFS (preferred for thin provision).

XOA supports full failover backups between SANs, and everything just works reliably. With years of experience, we’ve even restored entire Enhance servers from XOA backups without issues. It’s worth exploring.

One thing, we had preferred to utilize the older xenserver management interface, which was compiled from source by the community. It was a window application, and frankly we simply used it for so many years, it was second nature. That has now been deprecated in the latest XCP-ng 8.3, though it works fine on 8.2.x and earlier.

XOA is a small vm that runs a web interface and manages XCP-ng. It has some quirky views of things, but the XOA v. 6 will clean up and modernize the UI

We have a LOT of clusters, both public facing and for private customers running on this platform for many years and it has been bullet proof, even in smaller 2 node Pools (cluster).

wenani

8Dweb Please allow me to ask this: I have a number of dedicated servers running in different regions, but I have been thinking of going the collocation way as that would reduce my operations cost compared to what I am spending currently, and also add more to my offerings.

I have been looking at getting our own IPs so that even vps provisioning would not brand a different ISP name with the IP. But the big question has been which program we will use for managing all the nodes and provisioning the nodes and vps as well. We have tested proxmox before, then recently came across XCP-ng. I know a company like contabo is using proxmox, not sure what others are using, like hetzner and netcup, but we wanted to have a solid program that we will settle with for the rest of our days in business.

I am still at the early stages of researching, so maybe this is something I will implement a year from now, but then that means a lot of learning along the way, although currently learning everything.

Any pointers, I will appreciate!!!

Jordan

8Dweb If you have the time and resources, consider setting up an XCP-ng pool with shared storage. We’ve transitioned storage nodes from 10G to 25G to 100G seamlessly over the years, initially using XenServer before switching to XCP-ng.

Yea, I use to use xenserver then xm and just moved to Proxmox because it just seemed to fit the bill. The only issue is the failover being a minute and requiring a quorum. But then I think about glueing together a zfs cluster with 10G networking or 50/100G if needd.

8Dweb XOA supports full failover backups between SANs, and everything just works reliably. With years of experience, we’ve even restored entire Enhance servers from XOA backups without issues. It’s worth exploring.

What is your RPO/RTO on this? Just curious.

8Dweb One thing, we had preferred to utilize the older xenserver management interface, which was compiled from source by the community. It was a window application, and frankly we simply used it for so many years, it was second nature. That has now been deprecated in the latest XCP-ng 8.3, though it works fine on 8.2.x and earlier.

I remember this app, was fun to use. But yea I only used it for a couple of months before moving to VMWare which was for a contract position. I didn't make the decisions.

Right now it looks like the only thing actively developed that is free is XO Lite. Then there is of course XOA if you compile from source as per https://xen-orchestra.com/docs/installation.html#from-the-sources

Why hasn't anyone automated this build via github and published it? Are there license restrictions?

8Dweb We have a LOT of clusters, both public facing and for private customers running on this platform for many years and it has been bullet proof, even in smaller 2 node Pools (cluster).

Good to know, will indeed check it out.

Jordan

wenani Any pointers, I will appreciate!!!

There are so many options, and although owning your own hardware is cost-effective versus renting, you still need something to manage it all and ensure failover or high availability at some point.

Not everything needs high availability; let's just get that straight. A small percentage of applications truly need it, can afford it and for someone to host it without losing money.

So if you can live with 99.99% downtime with a data center, you don't need high-availability geographically. So, really, you just need to build out your own cloud that has computing and storage that is redundant and fault-tolerant. Most of which you can achieve within hardware.

3 x Compute Nodes
2 x Storage Nodes
2 x Network Switches 10G

Connect all of these together pyhsically and ensure you can lose one of each and still operate.

As for the software layer, for storage I would look at providing storage via iSCSI and NFS. Just so you have the options. For compute, you can look at Proxmox or XCP-ng. If failover is 1 minute, then that's easy to live with, and Proxmox will work. If you need seconds, then look at an alternative. But a minute failover is not bad.

Anyways, there's lots more to talk about. I've got limited time, but this kinda breaks it down easily. Some one can pick it apart and make it better, the best idea is made up of multiple other ideas.