Import Backups larger than 10GB [RELEASED]

prasad0889

Hello Guys,

We will be moving some customers to Enhance this month. What should be done if the backups are larger than 10GB in size? We have some accounts with around 30-40GB data.

paul-bob

For now, contact the support using the helpdesk or use rsync, @prasad0889 .

We have this issue here..

But soon, the migration tool will receive a update to solve this and others problems.

MediaServe

A workaround suggested to me by @Adam was if you skip the homedir but then add the mail directory, you should be able to import the cPanel account and then rsync the files from /home/[user] (cPanel server) to /var/www/[container] (Enhance server):

cd /home
/scripts/pkgacct --skiphomedir [user]
gunzip cpmove-[user].tar.gz
tar -rvf cpmove-[user].tar /home/[user]/mail
gzip cpmove-[user].tar

Replace [user] with the cPanel username of the account. This should result in an importable backup, and if that's under 10GB size it should be fine after you rsync the files and set the proper ownership and permissions.

Since the email would presumably end up in another container, even a different server, you might need to skip the mail directory in the rsync to save that extra storage or remove it after import.

I have yet to try this, but I'm supposing it should work. If I try it (not sure when) I can confirm.

prasad0889

Thanks MediaServe for your detailed explanation. I will definitely give it a try. Also, I am looking forward to the SCP backup import via cPanel which will really help import larger backups.

prasad0889

One more thing guys, I have seen that the backup upload happens very slowly. I have a good internet connection and the control panel server also has a dedicated 1 gbps connection, but when I upload it uploads only at around 10-15mbps. If I directly upload to the control panel via FTP the file transfer is very fast.

MediaServe

prasad0889 I agree! The slowest part of the cPanel importer is the web UI uploading the file, and I think the problem is that we have the Enhance controller outside of our data center, so the upload must be transferring outside and then inside the data center, which is a huge waste. We have one Windows server within the data center which makes the process of the cPanel backup lightning fast, but then when we import a 5GB backup file it takes at least 30 minutes or more (didn't time it). I'm regretting the setup now! I should have put everything at our data center. except our primary website outside on a separate Enhance server. My mistake was that I didn't realize how centrally controlled Enhance.

prasad0889

@MediaServe we are using cloud servers from all big cloud providers, yet the backup uploading feels very slow. @Adam @Darren do you have any knowledge about this or how to get it sorted?

The best way was to directly pull from the server, but it comes in July/August!

mendozal

prasad0889

V2 cPanel importer
Ability to restore from backups SCP'd to the control panel server.

I believe this is coming in April

Adam

prasad0889 @MediaServe we are using cloud servers from all big cloud providers, yet the backup uploading feels very slow. @Adam @Darren do you have any knowledge about this or how to get it sorted?

The best way was to directly pull from the server, but it comes in July/August!

I haven't seen this behaviour before. What is the latency between your PC and the control panel server? Which browser are you using?

MediaServe

Adam Within our data center, FileZilla retrieves a 5GB+ cPanel backup in mere seconds, but the upload to Enhance (a VM at Vultr) takes over 30 minutes. I didn't time it but it was at least that length of time.

I'm guessing because the controller does the initial work, we're essentially downloading from our data center to vultr and then back again, which is not optimal. If I had better thought through the Enhance design I would have done it much differently, but hindsight plus the current difficulty of migrating a controller means I either need to do it manually or wait for the new feature.

prasad0889

@Adam, I have tried using Chrome as well as Firefox, both of where uploads are extremely slow. An 8GB backup took almost over 3 hours to upload. Latency should be around 100ms at the most. If I upload using FileZilla, it takes less than 30 minutes.

prasad0889

@Adam, I would suggest if you are based in UK, deploy a vm in the US, and then try doing it, and let us know.

PradeepC

Yes, we are facing an issue too, where the uploads are very slow when either importing a cPanel backup or even uploading a file via the control panel for a user. Doesn't have an issue with cPanel. @Adam, please suggest a solution.

Adam

We were trying to recreate this issue yesterday and have so far been unable. The limiting factor will be bandwidth/latency between you and the control panel server but the browser-based upload itself should be close to wire speed and comparable with SCP (to the same server) and only slightly slower than plain FTP.

@PradeepC and @prasad0889 could you send a support ticket so we can ask for some details specific to your installation?

prasad0889

I will send a ticket shortly @Adam!

Aliysa_Enhance

A Master organisation can now SCP a backup file to the cPanel importer:
https://enhance.com/support/release-notes

Dan

Aliysa_Enhance Is there any documentation on this? i.e where the file should be SCP'd to?

We've not yet updated to 9.3.0 so apologies if this is obvious in the UI.

Aliysa_Enhance

Dan No problem. When you update this info can be found in the features blurb.

mendozal

Dan the migration process tells you where to copy the file when the time comes before continuing.

MediaServe

mendozal So the master organization cannot impersonate the customer so the website is added to his subscription. Seems the importer restores the website somewhere and then you have to move it to the customer's dedicated server (for Managed VPS clients.) More work and time required and it doesn't make things any easier. It does at least overcome the 10GB limit, so a bit of improvement there.