Heads Up! S3 will destroy BW on cloud hosting.
twest I'd note that for people wanting Enhance to improve the S3 backups with something like incremental backups, well, JetBackups has incremental backups on S3 and you know what it does? Runs up insane costs from all the operations being run on S3's side as part of the incremental/checking mechanism... If you think bandwidth costs are high, you've not seen a thing yet lol. I'd hope Enhance could figure something out for S3 incremental backups that uses compute on our side to avoid those costs at S3.
As far as I know, to achieve incremental backups with S3, we would need to store all of the backup data locally and then send only the incremental changes to the S3 bucket. It's definitely possible but most customers using S3 are doing so because they are running a small or single server cluster and they don't want to expend server resources to store backups.
There has to be some data to compare the changes with so either we have to store some kind of index in the S3 bucket or maintain a local copy of the files. It's very complex but not impossible.
If you have large data volumes then I recommend using the built-in Enhance backups. They are incremental by design, consume no space on the app server and are very fast to back up and restore.
It is impractical to use local backups as an intermediary step. as it would require the server to keep a safe free amount always available. It is a waste of storage, in my opinion. Storage that could be and should be sold to make an income.
I know some people chase S3 or similar cloud storage providers as a backup option, but in my opinion, they do not have a complete picture of what they were requesting and do not compare apples with apples.
There is no better way, at the moment, to backup (considering savings on bandwidth + storage + restore options) than the backup server provided by Enhance.
To buy a storage VM is cheaper compared to using the same amount of storage on any cloud provider. Also, most of them come with unlimited bandwidth or if not, with more than enough.
Looking for S3 or similar options for backup using Enhance it is like looking for trouble 
What a waste of time.
Isaia-Arknet_PTY_LTD I can only second this, we have a cheap auction server from Hetzner with 2 10TB disks in it we Backup daily, keep 30 days of backups and it uses less then 2TB for 585 >ebsite ruinning on enhance, i would never use S3 when the enhance backup solution is rocksolid.
What I'd really like to see is a native way to replicate the backup servers.
The backup server sending incrementals to S3 would be ideal.
You guys just have 1 backup of your client sites?
Oh and SFTP support would be great too.
- Edited
hwcltjn I actually really liked Isaia's idea. Using S3 seems like a huge waste of bandwidth and $ when we can just get a second backup server running in another data center, then you have 2 Enhance backups running and both space/bandwidth saving incremental backups... Perfect solution! No more huge S3 costs, and I bet backup runs will be super fast! My cPanel backups usually take about 3-4 hours for 700GB, the equivalent backups using JetBackup as the Enhance S3 backup. I haven't gotten to the point of testing Enhance backups yet, but this is an exciting development
It's odd that I don't think I've heard anyone talk about backups to Google Drive (maybe I've missed it) That's the best backup-related feature with Plesk (and is incremental of course). I backup to both my Plesk server and Google Drive seamlessly. Zero issues.
Restic does it well with deduplication and compression, Hope enhance might have explored it already.
why not rustic?
- Edited
Isaia-Arknet_PTY_LTD there is no data redundancy for backups on disk. Having it on s3 gives you a better data resiliency
don't get the hate to s3, there are many s3 providers very cheap, and if backups have been implemented correctly (incremental) there would be no complaints.
s3 with no incremental is not what the real feature request was, and i bet nobody will use it on the long run. I mean, you shouldn't.
standtech Backups are not redundancy for your content, instead, it is a copy of data stored in a different place (preferably a different DC)
You can also create the backup of the backup, and so on, for additional redundancy of the backups, but I think it is a waste. (this is definitely on own opinion only)
The "backup" word (in itself) is a way of restoring original content. It implies that the data is already sitting on the Original server and the backup server (if one of the 2 fails, you fix it and continue backing up)
s3 or other cloud providers offer some type of raid and data cloned on probably multiple places, but do not think it is bulletproof. They fail, too, and sometimes data sitting there can also get corrupted.
I am not promoting anything or going against S3 or other providers, but I do consider it a waste of time to develop tools for backing up data when we all know enhance for s3 backups are very far away compared to jet-backup backups on the same cloud providers.
You should look and learn each how they work, and then you would probably be able to assess and figure out how far one from the other is
read this: https://docs.jetbackup.com/manual/whm/Destinations/destinationsOverview.html
Enhance to s3 is not the same as jetbackup to s3
i do not want to make this post too long. Still, in simple words, a backup that is compressed and archived is a waste of bandwidth and storage, as it requires Local storage in the production server for the archive to be created and compressed, then sent to S3 or another cloud if your file created is 100 GB then every night you will be moving 100 GB with enhance backup to S3 (by now you should have figured it out how bad this is!)
The missing term here is Incremental. That would be possible and implemented, and then S3 or any other destination would make perfect sense in achieving your desired backup redundancy.
- Edited
- Edited
Isaia-Arknet_PTY_LTD let me rephrase .. if you take multiple backup copies on local disks it can't provide you backup redundancy. if you take a single backup copy on disk then data is at risk. what product you use to take backup is irrelevant when you are talking about its reliability which depends upon underlying storage. S3 and incremental backup is a new norm. if the Enhance panel doesn't have it you can still achieve it with other tools. S3 for backup is the best bargain at a cheaper price just don't go "hybrid", transfer will cost you more than storage.
Isaia-Arknet_PTY_LTD I never use JetBackup Incremental because of the insane cost. JetBackups runs queries on S3's side, each query you have to pay for. If you host a lot of sites, that can be extremely expensive. For example, I run backups on S3 that are compressed archives and the cost is around $100/mo. When I switch to Incremental, those same backups cost $75/day. Yes that's per day. Of course as soon as I saw the cost I switched back to archive/compressed...
The pricing for GET, POST, LIST, COPY requests is only $0.005 per 1000 requests. Unfortunately with hundreds of sites to backup with TBs of data it's sifting through (highly inefficiently) it's making MILLIONS of requests to S3... Insane cost overrun.
JetBackups is a terrible product to compare to for incremental backups - it does a terrible job.
The norm in the hosting industry is not to S3. Instead, the norm is to have a backup on the cheapest possible storage and free restore if possible.
You might do backups in a better way regardless of cost, and that is a good thing, but in reality, and the norm is in T&C, most of the companies and especially the bigger ones who create and drive the norm, is to ask clients nicely to keep and maintain their copy of data for the website.
I am not pro or against it, but considering enhance already has a rock-solid backup solution that can be deployed on raid1 or other raid options, it is already there. And once you have a better-than-great solution, the rest of the development is good but not necessary
I will stop here regarding this topic, but i guess i could see some good ideas here that lots of us can learn from.
Isaia-Arknet_PTY_LTD Yes everyone has different RPO and RTO, and use cases. just because it works in one case shouldn't stop development.
twest I like Isaia's idea too, but as others have suggested one of the benefit's of using something like S3 is to have the data replicated.
If I'm not mistaken (as I have not yet tested Enhance's backup) it is my understanding that a website can only be backed up to 1x Enhance Backup server - this means, unless you manually take a backup of the Enhance Backup server, you only have 1x copy of the data.
My idea was to create a mechanism that would allow you to incrementally backup the data on the Enhance backup server to S3 - that way if for whatever reason my Enhance Backup server were to go offline, I can still get to my backups and have more than 1x copy.
Personally, in Enhance's case, S3 would be my "second line of defence" - as others have suggested, no point using it for full backups on a daily basis, it would be too costly and slow I imagine.
Yes exactly, this is v1 of the S3 backup integration, nothing is ever finished in software development so there will be improvements to come.
One other thing to bear in mind, is that nothing stops you from taking your own backups outside of Enhance. If you're using a virtualization platform that could be snapshots, or it could just be as simple as rsyncing your data to another box on a daily basis or any other method for disaster recovery purposes.