SSH service was down, issue after recent update

Kosta

Hello,
I started having issues with ssh after recent apt upgrade on servers. Maybe newer version ssh causing this maybe some issue with enhance.
Happy to have cpfence as it restarting the service as it’s gets detected.

Please advice if any had this issues recently, thanks

SSH service was down, but the attempt to start it was SUCCESSFUL.
This might indicate a service issue

I have this issue for few days since 12.3

cPFence

Kosta

Yes, same here, it happened last night after the automatic apt upgrade across all servers. We didn’t have time to dig into it yet, but since SSH has been stable since then, I’m not worried about it for now. cPFence will alert you if it happens again.

twest

This is one of the critical services I set a systemd override on and set to restart=always and 5 second delay. Never had an issue with ssh, yet, but it sure would be a pain in the arse if it crashed and there wasn't something monitoring it, hence systemd override is essential.

Kosta

Yeah after the upgrade it happened one one server straight away and now this morning appears again on 5 servers etc

cPFence

Kosta

Check the timezone on those 5 servers, you’ll probably find they’re different, and the apt upgrade ran later this morning because of that.

kelsallp

I'm getting it too. Cpfence picks the service back up.

gmakhs

Haven't had the issue yet, are you guys on latest enhance version ?
Do those servers have free ram ?

kelsallp

gmakhs On V12. Servers have lots of memory and not much going on. One server was a dedicated server.

Kosta

I don’t think timezone have anything to do with the issue currently. However I’ve run apt upgrade now so we see how it goes…

cPFence

Kosta

Timezone has nothing to do with the issue, it only affects the timing of your daily update process if you have one. Running apt upgrade now won't change anything because the SSH package doesn't have a pending update in the Ubuntu repos like the one that was probably released yesterday.

Kosta

cPFence thanks, after running apt upgrade and rebooting servers seems SSH keeps up so far…

gmakhs

kelsallp it's strange, did you check if OOM killed it ?

I never had SSH killed, but since the update monit is getting killed randomly, haven't investigated yet, but I know this server pushed the memory limits due to some tests .

Running cpfence I believe can cause OOM errors also due to clamAV if the server is over loaded, I don't say this is a problem in cpfence but admins should consider the consumption of everything runs on the servers, you can run 20 websites and all bells and wisles on a 8 GB VPS .
The other culprit oh high memory is mysql and php processes .

It's a good thing to setup monit to warn you when the server reaches specific CPU or memory limits usage, so you are aware ahead of time .

kelsallp

gmakhs I have had the issue on four servers in my V12 cluster.

I have just checked the logs for all four - no OOM issues. What I have noticed is that SSH goes down in response to a Signal 15 when apt-daily-upgrade.service starts.

pratik_asabe

I noticed same for few servers for three days, thanks to cpfence everytime it went down cpfence bring ssh back up again, but im not seeing it going down now for past 24 hours, i hope its not a random thing..

p.s. it was before v12.3.0, last night i upgraded, let's see now...

cPFence

pratik_asabe

kelsallp

There’s no real issue here, it’s just a simple apt package upgrade that caused the service to stop. This is very common on Linux, and it’s exactly why service monitoring exists. On v11, Docker was handling it silently in the background for you. Even without cPFence, you can easily restore this behavior by editing the service override.

gmakhs

cPFence I don't disagree with your point, but is the first time since 2012 I see ssh going down from update

cPFence

gmakhs

Never got a service down notification from cPanel’s watchdog after an update since 2012? And you claim you’ve been in this business a long time? Any real admin knows it’s normal to get those, especially after package upgrades.

gmakhs

cPFence is this post about notification or the service actually being terminated ??