S3 Backup sizes

xyzulu

I currently do something similar, see: https://community.enhance.com/d/1185-enhance-backups-replication
kopia instead of restic, but the same result (I think it's better actually)

DracoBlue

In my opinion, the S3 backup should have significantly more features, similar to what I had with the Plesk panel. Deleting a backup in the Plesk panel would also remove it from the S3 service, meaning it worked both ways. Currently, the S3 backup doesn't even work well one way, as it doesn't handle deleting old backups efficiently, leading to increased disk usage and higher costs.

xyzulu

Enhance have a much better option than S3, their own backup system. You should use it.. it's so much better. Kopia is a way I take a backup of the Enhance backups (at least in my case)

Edit: in your case @DracoBlue a second VPS with about 100Gb would be enough to use for Enhance backups.. you'd find the actual disk usage for the backups to be less than 50Gb. That being said, if you are planing to grow bigger soon, you should probably look at either an in-house server (running Enhance) for backups, or a VPS/dedicated with block storage (cheaper).

I don't quite get why this is so hard for you, you only have one server.. backups are easy for a single server.. so many options... but anyway.. your choice.

DracoBlue

xyzulu Unfortunately, the S3 backup is still cheaper than the backup disk provided by the company where I have my server. If it were otherwise, I would have already made the change.

xyzulu

DracoBlue Use another company for the backup server??

Isaia-Arknet_PTY_LTD

DracoBlue It is actually suggested that the backup server should be in a different DC (hosting a backup server with OVH or Hetzner) on the same continent (I guess you are Europe, and OVH and Hetzner are both European)
should be such an easy choice.

Another option is Storage VPS
https://hosthatch.com/products
https://www.time4vps.com/storage-vps/

Considering the costs of storage available, it is unreasonable to use anything else instead of a built-in Enhance backup server.

MarkD

Isaia-Arknet_PTY_LTD https://hosthatch.com/products

Those Storage VPS prices look good at hosthatch - 4TB for $16.00 a month, whats the catch?

cristianuibar

Wow. I just checked mine as well. I never really had a look at the S3 backups. I use it to complement the normal Enhance backups for which I use a small Hetzner VPS CPX11 (around 4€/m) with added 200GB block storage (another 8.8€/m).

Of the block storage 185GB is used right now.

As for S3, Mine is at 1TB usage already. Currently hosting around 20 sites of which just 3 of them are bigger at 20, 30 and 40GB each.

I configured it using R2 - Cloudflare, pricing is at 15$/TB right now.

I configured the retention policy via Cloudflare R2 at 30 days.

Is 30 days enough? Or too much? Should I change it to 7 days maybe to save on space and costs?

Current stats:
TOTAL NUMBER OF SNAPSHOTS
749

BACKUPS TAKEN IN THE LAST 24 HOURS
41

EDIT:
Another problem I noticed is that the PHP error log file can get big, fast. On website had 20GB of PHP error logs in just 2 days or so and it got backed up and used the backups storage for no reason.

Can we do something about this? Can we please NOT backup the error logs?

EDIT 2:
I am currently looking into ways to mount a Hetzner storage box to a VPS so I can use that as a backup drive, rather then the normal block storage that is using SSDs and gets expensive fast.

For context, for the price of 250GB block storage, you can get a 5TB storage box.

xyzulu

I don't get why no one reads links I post 😉
cristianuibar please see: https://community.enhance.com/d/1185-enhance-backups-replication

(and read the entire thread please)

Everyone.. if you use Enhance backups, a better way than then adding S3 (via Enhance) is to backup your backups (industry standard) to S3 etc.. you can use Kopia, Restic or a host of other options to do this. Doing it this way doesn't waste any space since you have BtrFS and incremental uploads to S3. Your Enhance servers will also use far less space (for the backup creation), less CPU and far less data transfer.

cristianuibar

xyzulu Thank you. I did read your posts and was going to look more into these backup options. I just did the S3-backup since it was quick to implement and available straight from the Enhance dashboard.

I will definitely look into Kopia and Restic as I am not familiar with them.

xyzulu

S3 in Enhance is easy, and if you only host a few sites on one server.. probably a good choice. For any serious hosting though, it's never going to be the best since:

It's not incremental and likely never to be
Uses much more CPU, Data and storage (even if it's for temporary files)

rdbf

ivansalloum

Similar to xyzulu, I haven't used Kopia but I suspect it might actually be an easier/better choice than the Restic version in the Ubuntu 22.04 repository. It's why I use the latest binary version. Either way, I think the approaches are similar.

Get any server with a large enough disk (mine is a VM on an old server in a basement)
50GB vdisk for the Enhance install with only Backup role, 500GB (or whatever you need) vdisk for the 'block device' on /backups. For reference, with mainly simple wordpress sites I barely use 1.5x the total website sizes added together for 14d retention, the btrfs snapshots approach by the Enhance role is awesome.
Set up Restic, in my case following the Backblaze guide (https://www.backblaze.com/docs/cloud-storage-integrate-restic-with-backblaze-b2). I'm sure other major players have similar guides.
Write a bash script to do the backup of the block device and keep the last 60 (or whatever) days, something like:

#!/bin/bash

source /etc/restic-env
restic -v --tag daily backup /backups
restic -v forget --keep-within 60d

make a systemd service to run the script, and a systemd timer to run it daily.

Your requirements may vary (like, not use a basement server)

xyzulu

rdbf Yes! Slightly different tools to us, but same result. Great work!

ivansalloum

rdbf Thanks 🙏

But let‘s say, your server with the backup role is down and lost. What would you do to restore these backups?

Isaia-Arknet_PTY_LTD

ivansalloum

If the backup server goes down, I would bring up another server, move all accounts ROLE to the new backup server and continue backing up.

It is backup, not history 🙂

I speak for myself, as some other businesses might look at it a bit differently, as they sell this as a service with X amount of restore points in the past; we offer backup as a complementary service, and the Client should have as much care for their data as we do, or more.

There are others who use the Enhance backup server, and they back up the backup server, Just in case 🙂
There is no limit on how much you can backup a backup!

rdbf

ivansalloum

If anyone would need a backup restored that was on the server that died, I'd do it manually. Just mount the backup. Whatever happens, I'd retain independent access to the 14+60d of backups. Would be ideal if I could spin up another backup role and restore the backup to the block device and everything would continue uninterrupted, but I'm not sure a fresh backup role + restored block device would work smoothly. I assume some things are handled locally on the backup role that are kept track of locally rather than on the control panel server. I'll let the v11 + v12 dust settle before I stir that up. So in that way it could be seen like "what do you do if your control panel server dies", similar challenge.

Another option would be if you use a VPS, have the provider take snapshots. Then you restore the snapshot if the VPS is lost. Possibly only the OS disk, and restore the block device from your S3. Depends on the size of your business and what you want to spend on it. But if there's an account issue (like payment issue, copyright claim, illegal content claim, abuse report, spam allegation or whatever else bad you can imagine) it's possible they kill/rapture the VPS and you can't have the snapshots anymore because your account is gone.

slimx

mendozal hostslick is one of them, i just used lowend box to find big vps-es for cheap 🙂

ivansalloum

Isaia-Arknet_PTY_LTD If you would spin up a new server and move websites to the new backup role, then for what would I need these tools they use? That’s what I don’t understand.

Isaia-Arknet_PTY_LTD

ivansalloum

There are a few things people speak about when talking about backups regarding Enhance:

1 Enhance controller backup (this is the VM/Server control panel runs on).
Currently, there is no way to replicate it somewhere else, and if the server fails, you will have a big problem if you do not have a backup.

What I do is I run Enhance in a VM on ESXI and back it up with Veeam. (simple and efficient), if something fails, I restore the VM, power up, and keep going also Veeam allows me to restore it on a fresh ESXI if, let's say, the server has a hardware fault. I install ESXI (5 minutes job) and restore the VM (ready to power up.)

2 Backup Client data, some providers use the backup server as the name suggests as a backup, a secondary place where data is placed in case of main HOSTING server failure.

If an account fails (data loss, user's fault, etc.), you restore data from the backup server; if a backup server fails, bring another server, move all sites ROLE to a new server and keep backing up.

Other providers use the backup server for versioning and history. Let's say you have a server with very active sites, like eCommerce, and you want at any given time to have 30 points of backup going back; in this case, you need to backup the backup server, as in case the backup server fails, you lost your 30 days backups. Some providers in this case, can backup the backup server 🙂 And this is where all the additional tools come into play,

I do not need it, as in T&C, I clearly specify that it is the client's responsibility to keep a backup of the data from the server. We offer backups as a complementary service. (others offer eMail service as a complementary service, included in the hosting package, this is why the eMail service was neglected for so long as Enhance was never considering eMail service as a Core product that was supposed to be Professionally packaged and displayed until now, even now some of the guys here will tell you, that eMail should not be offered, but the same goes for backup, you can let client sort out Backups 🙂 )

So, in reality, Everything you need Enhance has it. Do not stress too much about the custom work others do for very particular scenarios 🙂

ivansalloum

rdbf I had similar approaches in my mind and the reason I asked you about it is because @Adam told me once that even if I backup the backups, the new backup role won’t recognize them after you copy them to the new server.

pratik_asabe

Use enhance native backup its production ready and use less resources in terms of storage and somewhat cpu ram, given how much concurrent backups you allow in backup settings, now s3 backup is also a great solution but only if it has incremental backups feature, which enhance will not be implementing for a long time i guess or never will who knows?!

So just deploy vps (from any cheap/affordable remote provider) which supports block storage and enable enhance's native backup, you'll not regret it, its worth the investment.

ivansalloum

Isaia-Arknet_PTY_LTD I‘m not stressing myself and I agree with you. I‘m just trying to come to the point that even if you backup the backups (I‘m talking about the backup role), it won’t work.