Need help thinking through an issue with a web site

Derek

I had the machine this Enhance VM is virtualized on go down when the power failed the day before yesterday, because of a UPS issue. Not sure if I'll need to send the new UPS back or not yet, but since then I've run into two issues on that server that caused pages to stop serving:

The web site went down because it couldn't connect to redis. A reboot fixed this quickly.
Maybe 18 hours later it went down saying it a mysqli php adapter was missing, but again a quick reboot brought everything up cleanly.

Litespeed installed, the only thing running that matters as far as load is a forum running Xenforo, I'm not seeing any errors in any logs (other than the forum software, which throws a redis error as the web site was connecting after reboot before redis was ready). 48G memory assigned, only 16G was in use during the failure.

Any thoughts on where I should be looking next?

cPFence

Derek

Probably best to wait and see if the issue comes back. When it does, grab the logs right away and share them here so we can help dig into it.

gmakhs

Derek There is a bug with litespeed which kills services, it can also kill mysql, i suggest you upgrade litespeed to the latest version and not to use the one provided with enhance which is pretty outdated.

That should resolve the issue with redis.

Derek

Well, I upgraded to the stable tree of Litespeed and things have been stable for a while, but I just had redis drop on that site again.

Fatal error: Uncaught CredisException: Connection to Redis tcp://127.0.0.1:6379 failed after 2 failures.Last Error : (1)

(Plus a lot of garbledygook).

Any more thoughts on what the source of the problem might be?

Derek

And it looks like I needed to capture the redis log before rebooting the server. Will try again next time.

xyzulu

Derek why not just restart redis?

gmakhs

Derek enhance needs to add watchdog functionality for such things to avoid such issues .

What you experience still sound like the litespeed bug , but try to capture the error, I ended up not using redis in enhance , until that is fixed.

Derek

xyzulu Restarting the PHP container didn't fix it.

And I might just disable redis. This web server has been getting hammered by a DDoS attack (it's a forum and was seeing 50,000 extra "users" that would all show up at once every few minutes) so Redis was required, but since I got that fixed my load average is under 1, and disabling Redis would probably be fine. Was not fine with all those users hitting it.

I moved to Apache until this DDoS - load spikes with Apache were insane (over 100) and I couldn't optimize it down below 80, and turning Litespeed back on got things to a load average of 5ish.

And it's kind of cool to see the number of lsphp processes ramp from 40 to 120 in about a second. Other than this weird issue I'm really liking Litespeed.

xyzulu

Derek Restarting the PHP container didn't fix it.

I don’t think restarting php affects redis. You would need to restart redis.

In any case, good to hear you made some progress.