2-Server High Availability Setup using DNS?

lcs

Hello,

I'm new to Enhance, having just started working with it today, although I've been observing its progress over the last few months. With the recent release of the Ubuntu package management in v12 and significant bug fixes over the past 45 days, I decided now was the right time to give it a try.

We're a small IT company managing around 45 low-traffic client websites, and we're exploring solutions for redundancy and high availability. Currently, we use Cyberpanel, but it has grown increasingly unstable in the past year. We recently experienced an outage, and the difficulty we encountered in restoring from backups highlighted the need for a more reliable and easily recoverable platform.

Our goal is to achieve redundancy by setting up two web servers—one primary and one secondary—that replicate site data, files, and databases in near real-time or through scheduled synchronization. Ideally, we'd like automatic failover handled through DNS changes or round-robin DNS. However, we don't directly control DNS records for some of our client sites, although we do manually manage DNS for others through Cloudflare. Enhance's Cloudflare integration for DNS management is particularly appealing to us.

Currently, our server setup is as follows:

Primary server: Located in Chicago (roles: CP, App, DB, DNS)
Secondary server: Located in Ashburn (roles: App, DB, DNS)
Backup server: Located in Chicago (same colo as primary), with plans to add an additional backup at another location

After exploring Enhance, it's unclear to me whether automatic DNS-based failover or automated DNS updates in Cloudflare can be achieved seamlessly. Is automated DNS failover or rapid synchronization between primary and secondary servers supported natively by Enhance?

Any guidance or suggestions you can provide would be greatly appreciated.

xyzulu

lcs Is automated DNS failover or rapid synchronization between primary and secondary servers supported natively by Enhance?

No it is not. There are not many panels that are going to support this either.
It's entirely doable, but worth it? (You'll need at least a load balancer, 2 web servers and a separate database server).. that is up to you to decide. Instead we prefer the Enhance backups route with the ability to restore very easily an entire server (within mins) in the event of a total server failure. DNS will be auto updated, even if you are using Cloudflare (provided you use the Cloudflare Enhance integration).

Edit: beaten to it by @twest 😉

cosmoshosting

Best to do it at an infrastructure level and stay away from DNS for failover.

Plenty of low cost layer-2 load balancers that assign a single floating IP.
They will automatically load balance and failover between the servers you have behind it.

Then you have 2 or more identical WEB/DB servers, usually a master and read-only slaves.
Put mail and DNS on it's own server,

lcs difficulty we encountered in restoring from backups highlighted the need for a more reliable and easily recoverable platform.

HA all boils down to failure mode. You get hacked or muck up an update, you replicate your mistake. HA environments can be significantly more complex to restore.

lcs Currently, we use Cyberpanel

If that is your benchmark. A good solid server through a tier 1 hosting providing (Amazon or Azure) and a good reliable panel (cPanel, DirectAdmin, Enhance) as-well as snapshotting will be magnitudes more reliable and easy to manage. No need to complicate it. This will get you 99.95 - 99.99 at an affordable budget and a easy gui to restore.

Thomas_W

lcs if you need a real HA solution, you will need to go for something like three Proxmox HA Nodes with keepalived in one datacenter. They will need min. 10 GB NICs between the Nodes an the datacenter must allow you to switch the IP between the Servers. There is no RAID needed, because with CEPH the data are in all servers distributed (that's why you need the 10 GB NICS min.). But a low latency between the Nodes is needed, why a different data center is not possible. With keepalived you can setup a virtual IP that moves to the server that run the website.

Enhance as CP on top of this should work, but needs to be tested. Enhance have a great Backup solution and it's easy to spin up a new server and Restore a Website with all changes of IP addresses if it goes worst.

But for 45 low traffic websites it's the question if this makes sense. Also it does not prevent from Viruses or another software issues. Take care to have proper hardware and a good monitoring. And you can spin up a new server in a short time if it's needed. Also you can run backups all few hours without to waste a lot of Disk space. And the Cloudflare integration will help you a lot with customers that like to control the nameserver. Also a good solution against attacks helps a lot with stability.

lcs

xyzulu Who cares about the IP? If an IP changes your infrstructure should have a way to handle this. Enhance allows you to change the server IP or restore from backups to a new server. The DNS will be automatically updated (even if you do use the Cloudflare integration)

I do because if you read the thread above you'll see that we don't have control over DNS for some of our client domains. I'm looking for a way when I restore a backup and failover websites to the secondary server, to not have to work with a client to change their DNS, and then change it a few days later.

You mention DNS automatically being updated- that's if I have a client domain onboarded into enhance, which I'm not planning to do. I'd like to leave DNS in Cloudflare which seems like it will work with the integration.

xyzulu Your control panel server can also be backed up (so that can be easily restored) see: https://community.enhance.com/d/2548-backup-your-control-panel-server-to-a-remote-location-for-easy-disaster-recovery

I haven't gotten back to that thread yet- I did ask some questions in there the other day.

xyzulu Why not install Enhance yourself ( I recommend at least 2 or 3 servers to test things). It might help you get more of an idea where things are at.

If you read the thread, you'll see that I already have:

lcs I'm new to Enhance, having just started working with it today

lcs Primary server: Located in Chicago (roles: CP, App, DB, DNS)
Secondary server: Located in Ashburn (roles: App, DB, DNS)
Backup server: Located in Chicago (same colo as primary), with plans to add an additional backup at another location

So, how does having set it up help here- please provide more detail if you can.

xyzulu Who cares

Lastly, I'm just trying to understand- you seem pretty active on this forum, and seem like you have a lot of experience working with this control panel. As this is a user to user forum, I think you'll get a wide variety of users and experience levels. It strikes me as odd that you would respond to anyone opening with 'Who cares', as opposed to something more constructive. Clearly you believe that this shouldn't be something an admin should care about, but you fail in your reply to explain that. You say that the infrastructure should have a way to handle IP changes, but don't really illustrate how that would occur. I personally, and I'm sure others would appreciate more constructive and less flippant responses when we're trying to engage with the community. I appreciate you engaging with me.

twest

Is automated DNS failover or rapid synchronization between primary and secondary servers supported natively by Enhance?
No.

Where Enhance shines for disaster recovery for a server is in its decommissioning and migrating sites from a dead server to a new server and automatically updating the DNS to match the new server. It can pull the latest backups off the backup server for redeployment to the new server. This isn't an automated process though, and most people probably shouldn't want it to be, because a downed server doesn't always require a total replacement - sometimes you just need to restart a service or reboot or fix a config. That would be part of your DR plan = assess the situation, find out if recovery of the down server is possible, assess whether it makes more sense to troubleshoot or abandon the server and redeploy from backups.

We attempt to remove as many bottlenecks to the redeployment process as possible = make sure the backup server has plenty of CPU/RAM, make sure it has speedy NVMe drives, make sure it has a fast network to send files when needed. We also keep a extra server on standby - it's completely unused, but is fully setup and configured to host sites, so in a DR situation it's ready to receive backups and start running the show for the downed server.

With the Cloudflare sync setup, Enhance will update the IPs for the redeployed sites, saving even more time (don't have to go update 100 dns records on 100 sites, etc)... In the past we would mitigate this issue by using floating IPs that could be shifted from one server to another - but in Enhance it will handle all that, good time saver during DR.

If you want automated hardware recovery I think your best bet would be to use AWS, they have ranges of servers that will automatically migrate during hardware fault and handle the networking too. It's expensive though.

lcs

Thank you all for the feedback. I guess I'm wondering then, with our current setup.. well, let me ask:

Does Enhance support replica servers at this time? It sounds like this would have to be done at the hypervisor level to create some redundancy.
What's a solid US provider that provides floating IP service that you've used? (or has US data centers?) We're looking at Chicago since that's where all of these clients are based. It would be great to have Chicago and Dallas/Ashburn as alternates. Note: We were using Vultr for our previous hosting, and thought we were in a good place but they pulled the IP we were using because their upstream provider 'reallocated' it. That, and their customer service response to the issue has made me want to move away.
Is there a floating IP service that is external from any specific VPS provider? I'm thinking like SD-WAN I guess, but maybe something I can purchase from Cloudflare to allow me to use different VPS providers in different geographic locations hosting my Enhance servers with a Cloudflare-purchased IP, to attain some level of auto-failover redundancy?
What happens when say for example, my provider deletes my VPS. Poof, gone. Can't login to anything. What's the path forward? Assuming I have a backup server running (with no websites on it), how can I promote that to the Master server, and restore all websites from backup? I'm hearing folks say Enhance makes this easy, but it would seem challenging since the cp. domain would be attached to the first server that's gone, right? I'm just trying to understand recovery process.

xyzulu

lcs

No
Who cares about the IP? If an IP changes your infrstructure should have a way to handle this. Enhance allows you to change the server IP or restore from backups to a new server. The DNS will be automatically updated (even if you do use the Cloudflare integration)
See: https://enhance.com/docs/server-management/disaster-recovery.html

Your control panel server can also be backed up (so that can be easily restored) see: https://community.enhance.com/d/2548-backup-your-control-panel-server-to-a-remote-location-for-easy-disaster-recovery

Why not install Enhance yourself ( I recommend at least 2 or 3 servers to test things). It might help you get more of an idea where things are at.

xyzulu

Sorry.. I was short in my replies.. I figured it was better than not replying at all 😉 I might have to show more restraint next time since these kinds of threads often end up heading in the same direction.

In my opinion you should not ever be tied to an ip address. You also should have your customer delegate their dns to you or have access to their dns. I believe it’s “old school” to get hung up on this, especially since IPv6 is here too.

lcs So, how does having set it up help here- please provide more detail if you can.

Test out DR then and see how it works on Enhance.

PS perhaps you should be contacting Enhance with these questions. At the least I’ll bow out now, maybe other users want to jump in.. and even disagree with me, I’m fine with that.

8Dweb

xyzulu Mostly agree. Further, when a customer refuses to point DNS to our servers or to use Cloudflare and the API it makes things a problem to make changes, migrate sites to another IP. Even though we own our blocks, sometimes things need to change.

We have a clear clause in our hosting agreements that if a customer maintains their own name-servers, and does not respond in a timely fashion to our IP/Server change notices, we will not be held responsible for their hosting/email "going offline" Now, we are generous at 30 days prior notice in all but true emergency situations.

As to the original poster, lcs
If you have the time to make sure two servers have the same exact content, do not need dynamic database activity (e-commerce, many updates per day/week) then look at load-balancing through Cloudflare (in Beta in some parts of the world) or look into Kemp, Akaimi, or other providers who have this ability. Or learn HA-Proxy and set it up with a good VPS provider who has global anycast ability.

Load Balancing can be an expensive system to maintain if you set it up yourself, and it will cost significant money if you utilize the big providers.

if you need to LB two enhance servers, instead of using DR, then you are probably in an under 1% use case category.