Addressing High Server Load Issues On Enhance Servers

cPFence

Hi everyone,

Lately, we've noticed an increase in support tickets related to high server load. I'm sure the Enhance team encounters similar tickets from time to time. While high load troubleshooting isn’t part of cPFence’s support scope, we wanted to share some tips to help the community manage these issues better.

One of the most common mistakes we see is skipping resource limits in hosting packages or setting them too high (or even unlimited). For shared hosting, this can quickly lead to server load problems. Properly configuring limits can save you and your customers a lot of headaches.

To help, we’ve created a guide that explains how to manage high server load effectively. While it’s written for cPFence users, the principles can be adapted for other setups and software as well. Check it out here:
How to Address High Server Load Issues on your Enhance Server

Please note that the limits depend on the power of your CPU cores, so you may need to adjust them based on your setup.

We hope this helps reduce these issues, lighten the load on your servers, and ease the support burden for both us and the Enhance team! 😊

Feel free to share your thoughts or additional tips you are using.

gmakhs

cPFence There is an issue with allowing the websites to do unlimited swap, which causes high load, its something enhance should solve or a user can disable swap .

Even with that random peaks happen, the fact that many have those issues means that there is another underlying issue that needs to be resolved or enhance needs to improve how the limits work that being said thanks for sharing your experience

And of-course there is mysql which enhance doesn't limit

All that being said setting nproc to 100 is also too high and causes load issues , something around 35-50 works without too many 503 but again it aint perfect

Have a happy new year!

*** Editing :
Also many dont understand that the limits are per website and not per package which means that if you set 1 core cpu and allow 10 websites, this user can consume 10 cores and is not working well in most use case scenarios, at least not with the current UI, naming etc which is confusing to clients

rdbf

cPFence

Good write-up.

I'm seeing two different approaches though regarding resource limits, and they seem opposing.

On the one hand there's Adam, saying a couple things here:
https://community.enhance.com/d/558-nproc-limit-on-every-account/4
https://community.enhance.com/d/558-nproc-limit-on-every-account/19
https://community.enhance.com/d/558-nproc-limit-on-every-account/37

Basically it's saying, only limit RAM and set NPROC to 10-20, let the rest unlimited. Any other limits may cause issues working against each other under some circumstances.

But on the other hand, your guide is using a much higher NPROC, while clamping down all other limits. Could you elaborate on that? What are some real-world examples where one set might cause issues and the other would not?

cPFence

gmakhs

We've always kept nproc to 100 and swap enabled on all our servers and our clients' servers, and we've never encountered any issues or random peaks. Everything runs smoothly , like clockwork. (True story)

cPFence

Here's another tricky tip: One client had set the nproc limit to just 10, which turned out to be a bottleneck and contributed to high load issues. The nproc limit controls the number of processes a user can run simultaneously, and setting it too low can cause processes to queue up, leading to server overload.

The tricky part is that tools like htop or MySQL process lists won’t show anything unusual , just a very high load average with no clear signs of abusive processes. Based on our experience, it’s best to avoid setting this limit too low or to unlimited; find a balanced value that suits your setup.

Kosta

cPFence hi, can I know how much is yours:
Swapiness and vfs cache pressure set to for the swap on yours servers. Thanks

Ops and how many websites runs on the server

cPFence

Kosta
All left to the defaults: Swappiness at 60 and VFS Cache Pressure at 100.

Although cgroups and CL are two different products, these are the CloudLinux recommended default values for Nproc, IO, and IOPS for reference.

Kosta

cPFence for reference this is a link to setting up vfs cache pressure and swapiness
https://www.digitalocean.com/community/tutorials/how-to-add-swap-space-on-ubuntu-22-04

Says they need to be adjusted.?
Swapiness 10 vfs 50?
Thanks

Zoinkies

Hint hint someone make a calculator

cPFence

rdbf

Kosta

There's a lot of debate around these settings in general. For example, CloudLinux recommends 0 swap, cPanel suggests 2–4 GB swap size, and others swear by using half the memory or even increasing swap to 12 GB or more.

The same goes for Nproc, Swapiness, vfs and other settings.

This guide is specifically tailored for shared hosting servers already experiencing load issues with no clear solution. It provides general guidance to help get things under control.

If you're not dealing with load issues and are just looking for performance tweaks, this guide isn’t for you. As a rule of thumb: if it ain't broken, don't fix it.

twest

I kept my swap intact, but set swapiness to 0. That fixed like 75% of load spikes. Swapiness at 0 keeps swap usage available, but it will only be used in dire situations to avoid OOM. On my dedi servers that have 250gb of ram this is ideal, as ram usage never surpasses 60gb and the rest is cache - if I ever needed to use swap for any reason I would have MUCH bigger issues to deal with.

On small vps swap might be better to allow usage in special cases where it's preferable to OOM situations that could more easily creep up due to limitation of resources, but I would still keep swapiness real low, because (especially on vps) you're gonna have IO limits that will bottleneck and cause load spikes, so I might only run a 5 swapiness.

Removing limitations for IO, and limiting backups to 1 concurrent also greatly improved random load spikes. Still keep limitations for cpu/ram/nproc, though they are pretty generous. I think when the backup system runs it takes up 1 cpu at 100% against the website/account, so if you have a package with only 1 cpu limit then it's gonna have problems during the cpu run, so I would recommend no package lower than 2 cpu to avoid backups from causing issues/load spikes/503 errors on sites. Of course, that's not too bad a problem if it's only 1 backup per day and done at a time of night where most of your customers don't have traffic or work to do - but we run backups every 4 hours all day so it could be a big issue for customers if it didn't have the resources available to run.

gmakhs

twest it's impossible to have backups only at night at the moment since you can't specify different times, for different servers so if the servers are in different continents one will be during high traffic hours doesn't matter what you do , also I didn't know backups are using user's resources - limits are you certain for that ?
I personally use similar limits asecpfence, and had big load spikes due to user swapping (server had available ram ) , also in some of the tests the user could use 30 GB ram, when the limit was 1024 .

And I can share more examples , unfortunately the answer most of the times is either that"my servers works like clockwork can't be something wrong " , or in case of enhance even after many tests we came to no conclusion as for why the high load occurs .

Same tests performed on DA +CL setup on the same hardware, without load spikes .

On none of the tests mysql performance was tested due to the fact that enhance doesn't limit anything there .

cPFence

gmakhs And I can share more examples , unfortunately the answer most of the times is either that"my servers works like clockwork can't be something wrong "

Giving complicated replication instructions (like the one you mentioned in the other thread) can take hours to replicate. Whenever you’re sharing issues, it’s always a good idea to include a simple, clear step-by-step guide on how to reproduce the problem (using both your current limits and the exact limits we recommended above). This will hopefully encourage someone from the community to step in and help. Otherwise, you’ll keep getting responses like 'works good for us, not sure about yours,' which I agree isn’t really helpful for anyone.

Zoinkies

Can you even change:

vm.dirty_ratio = 8
vm.dirty_background_ratio = 4

on Ubuntu 24? The file /etc/systctl.conf doesn't exist on this version.

cPFence

Zoinkies

Yes, you can edit this file /etc/sysctl.d/99-sysctl.conf

Zoinkies

cPFence Man, you come in clutch every day I swear.

pratik_asabe

We always use this configuration on our small vps i.e. 4cpu, 16gb ram,

recently we started caping iops & io to 2048, 8mb/s respectively (we're yet to see how it goes), we were not caping those before but the following config we are using throughout the year i guess, of course also we constantly tune database thanks to releem.. P.S. nproc we set to 150 on every package because we found unsual queuing chocking the server with 1024, 8mb/s iops and io while nproc was 50 first and even after set to 100...

cPFence any suggestions?

vm.overcommit_memory = 1
vm.swappiness=10
vm.vfs_cache_pressure=100
vm.dirty_background_ratio = 5
vm.dirty_ratio = 20

Kosta

pratik_asabe change vfs to 50 let me know is it better

cPFence

pratik_asabe

We prefer to keep it at the defaults:

vm.overcommit_memory = 0 vm.swappiness = 60 vm.vfs_cache_pressure = 100 vm.dirty_background_ratio = 10 vm.dirty_ratio = 20

Swap size is set to 4 GB (never larger than this, no matter how much RAM is installed).

We also apply the cgroups limits mentioned above and set max_user_connections=25 in MariaDB. If a user causes trouble, we simply add them to the Owl blacklist and move on. This setup works well for us.

pratik_asabe

Kosta we have done a extensive testing in past and these are the optimal stable* settings we ever found out, so when you use 50 in cache pressure the kernal the becomes more aggressive in term of memory usage (correct me if im saying something wrong) in my experience of being a decent sys admin and setting up so many servers i think these are the sweet spot values, and yes these may not work for everyone!

pratik_asabe

cPFence Interesting! i'll definitely test your suggested config and share my experiences.. i agree sometimes default works like a charm than over-tweaking like we did, we spend almost3-4 months in R&D setting these values to find out stable* config.. lol..

pratik_asabe

cPFence set max_user_connections=25 in MariaDB

btw, do you use max_connections paired with max_user_connections? like,

max_connections = 151
max_user_connections = 25