Heads Up! S3 will destroy BW on cloud hosting.: Page 2 - Enhance Control Panel

Heads Up! S3 will destroy BW on cloud hosting.

twest

Isaia-Arknet_PTY_LTD I never use JetBackup Incremental because of the insane cost. JetBackups runs queries on S3's side, each query you have to pay for. If you host a lot of sites, that can be extremely expensive. For example, I run backups on S3 that are compressed archives and the cost is around $100/mo. When I switch to Incremental, those same backups cost $75/day. Yes that's per day. Of course as soon as I saw the cost I switched back to archive/compressed...

The pricing for GET, POST, LIST, COPY requests is only $0.005 per 1000 requests. Unfortunately with hundreds of sites to backup with TBs of data it's sifting through (highly inefficiently) it's making MILLIONS of requests to S3... Insane cost overrun.

JetBackups is a terrible product to compare to for incremental backups - it does a terrible job.

Isaia-Arknet_PTY_LTD

The norm in the hosting industry is not to S3. Instead, the norm is to have a backup on the cheapest possible storage and free restore if possible.

You might do backups in a better way regardless of cost, and that is a good thing, but in reality, and the norm is in T&C, most of the companies and especially the bigger ones who create and drive the norm, is to ask clients nicely to keep and maintain their copy of data for the website.

I am not pro or against it, but considering enhance already has a rock-solid backup solution that can be deployed on raid1 or other raid options, it is already there. And once you have a better-than-great solution, the rest of the development is good but not necessary 🙂

I will stop here regarding this topic, but i guess i could see some good ideas here that lots of us can learn from.

standtech

Isaia-Arknet_PTY_LTD Yes everyone has different RPO and RTO, and use cases. just because it works in one case shouldn't stop development.

Darren

standtech

Yes exactly, this is v1 of the S3 backup integration, nothing is ever finished in software development so there will be improvements to come.

One other thing to bear in mind, is that nothing stops you from taking your own backups outside of Enhance. If you're using a virtualization platform that could be snapshots, or it could just be as simple as rsyncing your data to another box on a daily basis or any other method for disaster recovery purposes.

hwcltjn

twest I like Isaia's idea too, but as others have suggested one of the benefit's of using something like S3 is to have the data replicated.

If I'm not mistaken (as I have not yet tested Enhance's backup) it is my understanding that a website can only be backed up to 1x Enhance Backup server - this means, unless you manually take a backup of the Enhance Backup server, you only have 1x copy of the data.
My idea was to create a mechanism that would allow you to incrementally backup the data on the Enhance backup server to S3 - that way if for whatever reason my Enhance Backup server were to go offline, I can still get to my backups and have more than 1x copy.

Personally, in Enhance's case, S3 would be my "second line of defence" - as others have suggested, no point using it for full backups on a daily basis, it would be too costly and slow I imagine.

twest

hwcltjn ah dang it, yeah I haven't gotten to the point of testing a backups server yet either - I just assumed it would be configurable since isaia suggested it lol. I'll be testing the backups system soon, yesterday I finished my testing phase for the staging sites system.

digitalexpanse

kyzoeadmin
this is exactly what we do, we have 24 TB storage server
which is then synced to S3

hwcltjn

digitalexpanse are you just syncing the whole /backups/*/backup-subvolume dir to S3 at a time when you know no backup jobs are running?

Isaia-Arknet_PTY_LTD

While reading this thread, I learned a few things like everyone else 🙂

Can anyone provide a cost analysis for a basic scenario and explain the underlying S3 functionality (how it should be implemented in the future). I am very interested in what costs someone should expect using the best version of S3 backup and S3 costs to date.

Let's consider 500 GB of data and about 20GB of new data daily, also restoring 1GB daily (thinking clients need up their sites and want yesterday's copy).

Real numbers, if possible, not guesses 🙂

Thank you

hwcltjn

I personally don't use AWS S3 for this type of backup, it's just too expensive (~$23/TB) when I can get the same quality of service from other companies at more than half the price. Yes, there is Glacier which is cheaper, but I wouldn't use if for shared hosting.

So I use both Backblaze B2 and Minio (self-hosted) extensively. They're both rock-solid. Wasabi is good too.

Current B2 pricing is:

$6 per TB (1000GB), per month
Egress is free for up to 3x the amount of data you have stored, so 1GB daily (30GB/month) will be free

Starting with 500GB of data, and assuming an average increase of 600GB per month (20GB * 30 days).

By the end of year 1, you'll have 7.1 TB of data – which will cost you a total of $273.60.
By the end of year 2, you'll have 14.3 TB of data – which will cost you a total of $792.

In your/this example, no data is ever being deleted so your costs will just continue to rise.

Quick and dirty excel sheet:

Zoinkies

hwcltjn need to factor in transfer from vm to s3 bandwidth costs

hwcltjn

Zoinkies I haven't really ever hit any data caps on my nodes (I don't use the likes of DO and Vultr), and ingress to B2 is free.

jeff

We also use Backblaze and forecast similar costs to hwcltjn. In our use case we also back up to Wasabi.

JohnB

hwcltjn

You guys just have 1 backup of your client sites?

You would be surprised what "quality" providers think is acceptable and a single backup in the same datacenter is often what they do. As a customer I never trust provider backups (which is why you can't rely on VPS snapshot type backups).

« Previous Page